Treatment plan generation method, device, equipment and medium

Abstract

The embodiment of the present application provides a treatment plan generation method, device and storage medium, which relates to the field of medical information technology. The method includes: obtaining a target contour of a target area; searching for a target target point set corresponding to the target contour in a preset target point mapping relationship; the target target point set includes the number of targets and the size of each target point; according to the size of each target point, determining the position of each target point in the target area; according to the position of each target point and the preset prescription dose, determining the dose of each target point, and generating a treatment plan. The present application can improve the efficiency of formulating treatment plans.

Description

Technical Field

[0001] The present application relates to the field of medical information technology, and in particular to a treatment plan generation method, device, equipment and medium. Background Art

[0002] Radiation therapy, or radiotherapy for short, is a common cancer treatment. Before using radiotherapy equipment to treat a patient, a treatment plan for the patient must be designed first.

[0003] At present, treatment plans are mostly designed by physicists based on clinical experience, using a treatment plan system (TPS) to design the number, size, position, and dose of targets in the target area to obtain a treatment plan.

[0004] However, manually designing treatment plans requires a high level of clinical experience from physiotherapists, and in order to ensure that the designed treatment plan meets the prescribed dosage, continuous trial and error adjustments are required, and the entire process takes a long time. Summary of the invention

[0005] The embodiments of the present application provide a treatment plan generation method, apparatus, device and medium to improve the efficiency of treatment plan formulation.

[0006] In a first aspect, an embodiment of the present application provides a method for generating a treatment plan, comprising:

[0007] Obtaining the target contour of the target area;

[0008] Searching for a target point set corresponding to the target contour in a preset target point mapping relationship; the target point set includes the number of target points and the size of each target point;

[0009] Determining the position of each target point within the target area according to the size of each target point;

[0010] Based on the location of each target and the preset prescription dose, the dose of each target is determined and a treatment plan is generated.

[0011] In another implementation, determining the position of each target point in the target area according to the size of each target point includes:

[0012] Determining a target mask for each target according to the size of the target;

[0013] The target contour is convolutionally matched according to the target point mask to determine the position of each target point in the target area.

[0014] In yet another implementation, determining the dose of each target point according to the position of each target point and the preset prescription dose includes:

[0015] Calculate the dose according to the size, position and weight of each target point to obtain the dose line distribution of the target area;

[0016] The dose of each target point is determined according to the dose line distribution and the preset prescription dose.

[0017] In yet another implementation, before searching for a target point set corresponding to the target contour in a preset target point mapping relationship, the method further includes:

[0018] Acquire multiple target areas;

[0019] Delineating the plurality of target areas to obtain target area contours of the plurality of target areas;

[0020] Perform deep reinforcement learning training according to each target area contour to obtain a target point set corresponding to the target area contour;

[0021] The preset target point mapping relationship is established according to the target area contour and the target point set corresponding to the target area contour.

[0022] In another possible implementation, performing deep reinforcement learning training according to each target area contour to obtain a target point set corresponding to the target area contour includes:

[0023] According to the target area contour, a target area mask corresponding to the target area is produced;

[0024] Constructing a state matrix corresponding to the target area according to the target area mask, wherein the state matrix includes: the target area mask;

[0025] According to the state matrix, a target point set of the target area contour is obtained.

[0026] In yet another possible implementation, obtaining the target point set corresponding to the target area contour according to the state matrix includes:

[0027] Extracting features from the state matrix based on a convolutional neural network to determine the size of the first target;

[0028] Determine the position of the first target and the dose of the first target according to the size of the first target, and update the state matrix of the corresponding target area;

[0029] According to the updated state matrix, the size, position and dose of subsequent targets are determined in sequence until the preset prescription dose is met, and the state matrix of the corresponding target area is updated;

[0030] The number of the above target points is counted, and the number of target points and the size of each target point are used as a target point set corresponding to the target area contour.

[0031] In yet another possible implementation, extracting features from the state matrix based on a convolutional neural network to obtain the size of the first target point includes:

[0032] Using the convolutional neural network to extract features from the state matrix to obtain initial state features of the corresponding target area;

[0033] The initial state feature is processed using a preset action selection network to obtain the size of the first target point.

[0034] In yet another achievable manner, determining the position of the first target and the dose of the first target according to the size of the first target includes:

[0035] Determining a mask for the first target point according to a size of the first target point;

[0036] Performing convolution shape matching on the target area contour according to the mask of the first target point to determine the position of the first target point;

[0037] The dosage of the first target is determined according to the position of the first target.

[0038] In another possible implementation, the state matrix further includes: a dose state of a corresponding target area, and updating the state matrix of the corresponding target area includes:

[0039] According to the dose of the t-th target point and the volume of the corresponding target area, the dose status information of t targets in the corresponding target area is calculated; wherein t is an integer greater than or equal to 1;

[0040] The dosage status is updated according to the dosage status information of the t targets.

[0041] In another possible implementation, the dose state includes: dose coverage distribution, dose conformal distribution and dose overflow distribution; the dose state information includes: dose coverage information, dose conformal information and dose overflow information;

[0042] The updating of the dosage status according to the dosage status information of the t targets includes:

[0043] According to the dose coverage information, dose conformity information and dose overflow information of the t targets, the dose coverage distribution, the dose conformity distribution and the dose overflow distribution are updated respectively.

[0044] In yet another possible implementation, the method further includes:

[0045] The reward information of the t-th target is calculated according to the dose coverage information, the dose conformity information and the dose overflow information of the t targets.

[0046] In yet another possible implementation, the method further includes:

[0047] Calculate the current cumulative reward information of t targets in the corresponding target area according to the reward information of the t-th target;

[0048] The current accumulated reward information is used to characterize the reliability of the current target point set of the target area contour.

[0049] In yet another possible implementation, the method further includes:

[0050] The updated state matrix, the current target point set of the target area contour, and the current accumulated reward information are stored.

[0051] In yet another possible implementation, the method further includes:

[0052] Calculate the relative advantage parameter between the current target point set of the target area contour and the previous target point set of the target area contour according to the current cumulative reward information and the historical cumulative reward information of the corresponding target area; the historical cumulative reward information is used to characterize the reliability of the previous target point set;

[0053] According to the relative advantage parameter, a target target point set of the target area contour is determined and updated from the current target point set and the previous target point set.

[0054] In a second aspect, the embodiment of the present application may also provide a treatment plan generating device including:

[0055] An acquisition module, used for acquiring a target contour of a target area;

[0056] A search module, used to search for a target point set corresponding to the target contour in a preset target point mapping relationship; the target point set includes the number of target points and the size of each target point;

[0057] A first determination module is used to determine the position of each target point in the target area according to the size of each target point;

[0058] The second determination module is used to determine the dose of each target point according to the position of each target point and the preset prescription dose, and generate a treatment plan.

[0059] In a third aspect, an embodiment of the present application further provides a computer device, characterized in that it comprises: a memory and a processor, wherein the memory stores a computer program executable by the processor, and when the processor executes the computer program, the treatment plan generation method described in any one of the first aspects above is implemented.

[0060] In a fourth aspect, an embodiment of the present application further provides a non-volatile storage medium, on which a computer program is stored. When the computer program is read and executed, the treatment plan generation method described in any one of the first aspects above is implemented.

[0061] The treatment plan generation method, device, equipment and medium provided in the embodiments of the present application can determine the target target set of the target contour based on the target contour of the target area by shape matching with the pre-obtained target mapping relationship, thereby realizing the determination of the optimal target combination in the target area. When the optimal target combination, that is, the target target set, is determined, the position of each target in the target area is determined according to the size of each target in the determined target target set, and the optimal position for placing targets of each size in the target area is determined. Secondly, the dose of each target is determined according to the position of each target and the preset prescription dose, thereby realizing the dose calculation of each target at the corresponding position in the target area, and then realizing the automatic programmed calculation of the optimal target combination of the number of targets and target sizes, the target position and the dose in the treatment plan generation process, thereby avoiding the repetition of each step in the process of manually formulating the treatment plan, simplifying the process of formulating and generating the treatment plan, reducing the dependence of the treatment plan on clinical experience, and improving the accuracy and efficiency of the treatment plan, so that the application of the treatment plan is effectively guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0063] Figure 1 A flowchart of a treatment plan generation method provided in an embodiment of the present application;

[0064] Figure 2 A flow chart of a method for determining a target dose in a treatment plan generation method provided in an embodiment of the present application;

[0065] Figure 3 A flow chart of a method for training a preset target mapping relationship in a treatment plan generation method provided in an embodiment of the present application;

[0066] Figure 4 A flowchart of a method for deep reinforcement learning training of target area contours in a treatment plan generation method provided in an embodiment of the present application;

[0067] Figure 5A flow chart of a method for obtaining a target point set of a target area contour in a treatment plan generation method provided in an embodiment of the present application;

[0068] Figure 6 A schematic diagram of a framework of a training module in a treatment plan generation method provided in an embodiment of the present application;

[0069] Figure 7 A flowchart of another method for deep reinforcement learning training of target area contours in a method for generating a treatment method provided in an embodiment of the present application;

[0070] Figure 8 A schematic diagram of a treatment plan generating device provided in an embodiment of the present application;

[0071] Fig. 9 A schematic diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0073] Compared with the traditional technology that manually involves treatment planning, the embodiment of the present application aims to provide a solution that can automatically generate a treatment plan for the target area through an algorithm program, without the need for human intervention, and avoids repeated trial and error adjustments in the treatment plan design process, thereby improving the efficiency of treatment plan formulation.

[0074] The device for executing the treatment plan generation method provided in the embodiment of the present application may be a computer device installed with a treatment plan generation algorithm, and the computer device may execute the corresponding treatment plan generation method by running the treatment plan generation algorithm. The radiotherapy plan generation algorithm may be a sub-functional module of TPS, which may also be referred to as a TPS calculation module. It is necessary to refer to that the treatment plan generated by the treatment plan generation method provided in the embodiment of the present application can be applied to any radiotherapy equipment. Radiotherapy equipment may include: focusing therapy equipment or conformal therapy equipment. In the following embodiments, the application in a multi-source focusing therapy head is taken as an example, and a gamma knife therapy head is specifically taken as an example for illustration.

[0075] Gamma Knife is a common cancer treatment device. Currently, Gamma Knife treatment plans are mostly designed by physicists using a treatment planning system (TPS) to design the number and location of targets of different sizes based on clinical experience, so that the target area meets the prescribed dose while minimizing the exposure of organs at risk (OAR). This is a process of continuous trial and error and adjustment, which is complicated and time-consuming.

[0076] The existing Gamma Knife treatment plan optimization method is mainly based on geometric features, that is, it is necessary to estimate the number of targets of different sizes according to the shape and area of ​​the target area, and the plan optimization process is the process of optimizing the target position. Manual treatment plans can ensure that the prescribed dose is met but it takes a long time. Automatic treatment plans based on target position optimization are fast and require less manpower, but the initialization of the target size combination requires rich experience, the optimization process is long, and the results are often unsatisfactory. Finally, physicists need to make adjustments.

[0077] In view of the above problems, the embodiment of the present application proposes a treatment plan algorithm based on deep reinforcement learning and morphological matching. Deep reinforcement learning is an artificial intelligence algorithm that combines the perception ability of deep learning on environmental features and the decision-making ability of reinforcement learning on events with images as input. Deep reinforcement learning is to obtain the image state (State) at time t from the environment in the interaction between the agent (Agent) and the environment (Env), perform a certain behavior (Action) according to the current state, and obtain the corresponding reward and punishment (Reward). The purpose of the reinforcement learning algorithm is to learn a set of action strategies (Policy) through training to maximize the future cumulative reward (CumulativeReward). Since artificial experience data is not required as a training template during the reinforcement learning training process, the labor cost is greatly reduced. Therefore, the use of a deep reinforcement learning algorithm meets the characteristics of the gamma knife treatment plan design, allowing the computer to implement a repeated trial and error process to obtain the best treatment plan, which can reduce the dependence on clinical experience, without the need to manually set the target combination and target position, and without the need for artificial training samples, to improve the effect of the treatment plan and improve the efficiency of the physicist in formulating the treatment plan.

[0078] The following is a combination of multiple examples to first explain the treatment plan generation method provided in the embodiment of the present application. Figure 1 A flowchart of a treatment plan generation method provided in an embodiment of the present application. Figure 1 As shown, the treatment plan generation method may include:

[0079] S101, obtaining a target contour of a target area.

[0080] In a possible implementation example, an image of the object to be irradiated is acquired in advance, and the image is outlined to obtain the outline of the target target area of ​​the object to be irradiated in the image, that is, the target outline. The target target area includes the area to be irradiated of the object to be irradiated. By way of example, the object to be irradiated may be a phantom, a human body, an animal, etc. The target target area may be a tumor area of ​​a human body or a tumor area of ​​an animal, etc., and the target outline is the outline of the tumor area. The outline may be drawn manually by a physician or may be drawn automatically. The embodiment of the present application may obtain the target outline of the target target area that is pre-outlined from a preset storage location, or may obtain the target outline of the target target area that is outlined in real time from a target area outline device.

[0081] It should be noted that the target volume may also be referred to as a planning target volume (PTV), and the target contour may also be referred to as a PTV shape or a PTV contour.

[0082] S102: Searching for a target point set corresponding to the target contour in a preset target point mapping relationship.

[0083] The target point set includes the number of targets and the size of each target point. The preset target point mapping relationship may store at least a plurality of target area contour target point sets. In a possible implementation process, the target contour may be matched with each target area contour in the preset target point mapping relationship for shape matching. According to the shape matching result, the target point set of the target area contour with the highest shape matching degree with the target area in the preset target point mapping relationship is determined as the target target point set corresponding to the target contour. In other words, corresponding to the target contour means that the shape matches the shape of the target contour. The target point set of each target area contour in the preset target point mapping relationship includes: the number of targets in the corresponding target area contour, and the size of each target point, which may refer to the size of each target point that is best placed in the corresponding target area contour, and the number of targets under each size. In other words, in the preset target point mapping relationship, the target point set of each target area contour is actually a combination of the number of targets under each target size under the corresponding target area contour, that is, the best combination of target size and number under the corresponding target area contour. The preset target point mapping relationship may be pre-stored in a memory unit, for example. The target point set found through the preset target point mapping relationship stored in the memory unit is the optimal target point strategy for the target target area.

[0084] Unless otherwise specified below, for ease of description, the number of targets and the size of targets involved below refer to the number of targets in the target set and the size of each target.

[0085] S103, determining the position of each target point in the target area according to the size of each target point.

[0086] In the specific implementation process, the optimal position of the target point of the corresponding size in the target area can be determined by matching the size of each target point with the target contour of the target area, that is, the position of each target point in the target area.

[0087] In a possible implementation example, determining the position of each target point in the target area according to the size of each target point in S103 may include:

[0088] Determine the target mask of each target according to the size of the target;

[0089] The target contour is convolutionally matched according to the target mask to determine the position of each target point in the target area.

[0090] For example, a target mask of a target can be generated according to the size of the target, based on a preset target shape, and using a preset target mask making method. The target mask is a target mask corresponding to the size of the target. One size of the target corresponds to one target mask, and different sizes correspond to different target masks. After obtaining the target mask, the target mask can be used to perform convolution shape matching on the target contour to determine the best position for placing a target of this size in the target target area, that is, the position of the target in the target target area. In a specific implementation example, the target mask and the target contour can be input into a preset convolution shape matching network, so that the convolution shape matching network can be used to perform shape matching on the target mask and the target contour to obtain the position of the target in the target target area.

[0091] S104: Determine the dose of each target point according to the position of each target point and the preset prescription dose, and generate a treatment plan.

[0092] In a possible implementation, each target point may be placed at a corresponding position in the target area, and the dose of each target point placed at the corresponding position may be calculated in combination with a preset prescription dose, i.e., the dose of each target point, wherein the preset prescription dose refers to a prescription dose preset for the target area. For example, in the process of calculating the dose of each target point, each target point may be placed at a corresponding position in the target area, and the dose calculation may be performed on each target point to obtain the dose distribution of each target point at the corresponding position, and then the dose of each target point may be calculated according to the dose distribution and the preset prescription dose.

[0093] The formulation of a treatment plan is actually to determine the number of targets in the target area, the size of each target, and the dose of each target when performing radiotherapy in the target area. In this way, after obtaining the dose of each target, a treatment plan for the target area can be generated based on the number of targets in the target area, the size of each target, the position of each target, and the dose of each target.

[0094] The treatment plan generation method provided in the embodiment of the present application can determine the target target set of the target contour based on the target contour of the target area by shape matching with the pre-obtained target mapping relationship, thereby realizing the determination of the optimal target combination in the target area. When the optimal target combination, that is, the target target set, is determined, the position of each target in the target area is determined according to the size of each target in the determined target target set, and the optimal position for placing targets of each size in the target area is determined. Secondly, the dose of each target is determined according to the position of each target and the preset prescription dose, thereby realizing the dose calculation of each target at the corresponding position in the target area. Then, in the process of generating the treatment plan, the optimal target combination of the number of targets and target sizes, the target position and the dose are automatically and programmedly calculated, thereby avoiding the repetition of each step in the process of manually formulating the treatment plan, simplifying the process of formulating and generating the treatment plan, reducing the dependence of the treatment plan on clinical experience, and improving the accuracy and efficiency of the treatment plan, thereby effectively ensuring the application of the treatment plan.

[0095] Secondly, in the method provided in this embodiment, the target mask of each target is determined by the size of the target, and the target contour is convolutionally matched using the target mask to determine the position of each target in the target area, thereby achieving automatic determination of the optimal position of targets of various sizes, avoiding repeated manual determination and adjustment of the target position, and improving the accuracy and efficiency of the treatment plan.

[0096] Based on the treatment plan generation method provided in the above embodiment, the embodiment of the present application provides a possible implementation method for the specific implementation of determining the dose of each target point mentioned in the above embodiment. Figure 2 This is a flow chart of a method for determining target dose in a treatment plan generation method provided in an embodiment of the present application. Figure 2 As shown, in the above method, determining the dose of each target point according to the position of each target point and the preset prescription dose in S104 may include:

[0097] S201. Calculate the dose according to the size, position and weight of each target point to obtain the dose line distribution of the target area.

[0098] For example, a preset dose calculation method can be used to calculate the dose line distribution of the target area by placing each target at a corresponding position in the target area according to the size, position and weight of each target point in the target area. Among them, the preset dose calculation method can be, for example, a Monte Carlo simulation calculation method, or other dose calculation methods. The dose line distribution can be a 50% dose line distribution, that is, the area in the target area covered by 50% dose lines; it can also be other numerical percentage metering lines, which are set according to actual needs and are not specifically limited in the embodiments of the present application.

[0099] S202: Determine the dose of each target point according to the dose line distribution and the preset prescription dose.

[0100] The dose line distribution of the target area is obtained when each target point in the target area is at the corresponding position through dose calculation. In this way, the dose of each target point can be obtained by multiplying the dose line distribution by the preset prescription dose. Of course, after multiplying the dose line distribution by the preset prescription dose, other target point dose correction methods can also be used to correct or adjust the calculated dose, which will not be elaborated in the embodiments of the present application.

[0101] In order to better understand this scheme, the dose line distribution and dose are now explained. The dose line distribution obtained by dose calculation can be used to characterize the dose distribution in the target area, but it is not equal to the actual dose. The dose of each target point determined based on the preset prescription dose is the actual dose of each target point, which is used to guide the radiation dose at the target point based on the treatment plan.

[0102] In the method provided in this embodiment, the dose can be calculated first according to the size, position and weight of each target point to obtain the dose line distribution of the target target area, and then the dose of each target point can be determined according to the dose line distribution and the preset prescription dose, thereby realizing the calculation of the dose of each target point in the target target area, ensuring the prescription dose requirements in the target target area, avoiding repeated confirmation of the target dose in the manual treatment plan, and reducing the time spent on ensuring the prescription dose in the process of generating the treatment plan.

[0103] The preset target mapping relationship shown above can be obtained by deep reinforcement learning training based on multiple target area contours in advance. The preset target mapping relationship obtained by deep reinforcement learning training can be executed by a processing unit of a computer device, for example, and the processing unit can be a central processing unit or a graphics processing unit (Graphics Processing Unit, GPU). Using a GPU to perform deep reinforcement learning training can effectively ensure the training efficiency of the preset target mapping relationship. The computer device that performs deep reinforcement learning training to obtain the preset target mapping relationship can be the same computer device as the computer device that performs the treatment plan generation method, or it can be a different computer device.

[0104] The following is an explanation with specific examples: Figure 3 A flowchart of a method for training preset target mapping relationships in a treatment plan generation method provided in an embodiment of the present application. Figure 3 As shown, before searching for a target point set corresponding to the target contour in the preset target point mapping relationship in the above method S102, the method may further include:

[0105] S301, acquiring multiple target areas.

[0106] The multiple target areas may be multiple target areas of random shapes. The multiple target areas may be generated simultaneously or sequentially, and the embodiments of the present application are not limited thereto. The multiple target areas may be target areas obtained by acquiring multiple medical images of different target areas and then extracting target area images therefrom; or they may be multiple target areas of different shapes generated by simulation.

[0107] S302, outlining multiple target areas to obtain target area contours of the multiple target areas.

[0108] The delineation of each target area is similar to the specific implementation of obtaining the target area contour of the target area in the above embodiment. Please refer to the above for details and will not be repeated here.

[0109] S303: Perform deep reinforcement learning training according to each target area contour to obtain a target point set corresponding to the target area contour.

[0110] In a possible implementation, a preset training module can be used to perform deep reinforcement learning training according to each target area contour, and a planning model that meets the prescription dose requirements for each target area contour can be trained, that is, a target point set corresponding to the target area contour. In the process of using deep reinforcement learning to train each target area contour, each target area contour is actually trained continuously by machine trial and error until the number of targets allowed to be placed in the corresponding target area contour and the size of each target point are determined to meet the prescription dose requirements. In this way, a target point set corresponding to the target area contour is obtained.

[0111] S304: Establish a preset target point mapping relationship according to the target area contour and the target point set corresponding to the target area contour.

[0112] After performing deep reinforcement learning training on the target contours of multiple target areas to obtain the target point sets of each target contour, a mapping relationship between the target contour and the target point set can be established based on the target contour and the target point set corresponding to the target contour, thereby achieving the establishment of a preset target point mapping relationship. When the preset target point mapping relationship is established, the preset target mapping relationship can also be used as an experience parameter obtained by deep reinforcement learning training and stored in a memory unit. A memory unit can also be called an experience memory unit.

[0113] In the method provided in the embodiment of the present application, in the process of deep reinforcement learning training for each target area contour, the target point set corresponding to the target area contour is obtained through repeated trial and error by the machine, and the optimal target point combination is obtained, thereby realizing the automatic generation of the target point contour corresponding to the target area contour. There is no need to manually set the target point combination, and no need for artificial empirical data as training samples, which greatly reduces labor costs and reduces dependence on clinical experience, thereby effectively ensuring the effectiveness of the treatment plan generated based on the target point set and improving the efficiency of physiotherapists in formulating treatment plans.

[0114] The following is an explanation of the deep reinforcement learning training process for each target area contour through specific examples in conjunction with the accompanying drawings. Figure 4 A flowchart of a method for deep reinforcement learning training of target area contours in a treatment plan generation method provided in an embodiment of the present application, such as Figure 4 As shown, in S303 of the method shown above, deep reinforcement learning training is performed according to each target area contour, and the target point set corresponding to the target area contour may include:

[0115] S401, creating a target area mask corresponding to the target area according to the target area contour.

[0116] In the specific implementation process, the preset three-dimensional image can be processed according to the target contour and the organ at risk (OAR) image to produce a target mask. The organ at risk image refers to the image of the normal organs, that is, the organs without lesions, around the area corresponding to the target contour. The preset three-dimensional image can be a three-dimensional image with a preset size, and can also be called a three-dimensional matrix of uniform size. In the process of making the target mask, for example, based on the target contour, different operations can be performed on the area corresponding to the target contour in the preset three-dimensional image and the OAR area, respectively, to generate a target mask, so that the area corresponding to the target contour in the target mask is distinguished from the OAR area. At the same time, the subsequent target position search space is limited to the area corresponding to the target contour in the target mask to avoid the target position falling into the OAR area, thereby avoiding treatment damage to the OAR area.

[0117] For example, in order to unify the search space of the target position, in the process of making the target mask, the pixel value of the target contour corresponding area in the three-dimensional pixel matrix of preset size can be set to 1, the pixel value of the OAR area can be set to -1, and the pixel value of other tissue areas can be set to 0. In other words, the generated target contour is actually a three-dimensional pixel matrix, that is, a three-dimensional target mask image.

[0118] Although the search space for the target position is in the three-dimensional pixel matrix of the target mask, due to the different pixel values ​​in different areas of the target mask, in the process of obtaining the target point set based on the state matrix subsequently generated by the target mask, the target position of the target area corresponding to the target contour can only select the area with a pixel value of 1 as the feasible domain, that is, the target position is selected based on the area corresponding to the target contour in the target mask.

[0119] S402: construct a state matrix corresponding to the target area according to the target area mask.

[0120] In a possible implementation process, a preset state matrix model can be used to construct a state matrix corresponding to the target area according to the target area mask. As an environmental state matrix of a deep reinforcement learning system, the state matrix model can be used to characterize environmental state characteristics, for example, it may include: a multi-layer feature matrix, with the target area mask as one of the layers, to characterize the state characteristics of the target area mask. In other words, the state matrix can at least be used to represent the state matrix of the target area mask. For example, the target area mask can be used as the first layer feature matrix in the state matrix, that is, the basic feature matrix corresponding to the target area.

[0121] S403: Obtain a target point set of the target area contour according to the state matrix.

[0122] In a possible implementation process, the target area contour can be subjected to in-depth reinforcement learning training according to the state matrix until a target point set of the target area contour that meets the preset prescription dose requirements is obtained.

[0123] In the method provided in this embodiment, on the basis of generating a target mask corresponding to the target area by using the target area contour, the target point is restricted to the area corresponding to the target area contour for a specific target area shape, i.e., the target area contour, to avoid searching for the target point position in other areas, thereby reducing the search space for the target point position and improving the training speed based on the target area contour, thereby effectively improving the speed of formulating the treatment plan.

[0124] In the above Figure 4 Based on the deep reinforcement learning training shown, the embodiment of the present application also provides a possible implementation example of obtaining a target set based on a state matrix. Figure 5 A flow chart of a method for obtaining a target point set of a target area contour in a treatment plan generation method provided in an embodiment of the present application, such as Figure 5 As shown, in the method shown above, obtaining the target point set of the target area contour according to the state matrix in S403 may include:

[0125] S501. Perform feature extraction on the state matrix based on Convolutional Neural Networks (CNN) to obtain the size of the first target.

[0126] In one possible implementation, a convolutional neural network can be used to extract features from the state matrix to obtain initial state features of the corresponding target area; and a preset action selection network is used to process the initial state features to obtain the size of the first target point.

[0127] In the process of learning based on the state matrix, the state matrix can be input into a preset intelligent agent, also known as an intelligent network model, to obtain the size of the first target. The preset intelligent agent may include: CNN, which can extract features from the state matrix based on CNN to obtain the initial state features of the corresponding target area. After obtaining the initial state features, the process of placing the first target in the corresponding target area can be simulated based on the initial state features to obtain the size of the first target.

[0128] In a possible implementation example, the preset agent also includes: an action selection network, also known as an action network (ActorNet), the input layer of the action selection network is connected to the output layer of the CNN, so that the initial state features of the corresponding target area obtained by the CNN are input into the action selection network. In this way, after using CNN to extract features from the state matrix and obtain the initial state features of the corresponding target area, the action selection network can also be used to process the initial state features to obtain the size of the first target point.

[0129] Among them, CNN is a three-layer three-dimensional convolutional neural network, and the action selection network can be an intelligent network model of a neural network architecture to determine the target size based on the state features extracted by CNN. The action selection network can be, for example, a neural network model based on a proximal policy optimization algorithm (PPO), which can use the PPO algorithm according to the extracted state features to obtain the target size, such as using the PPO algorithm according to the initial state features to obtain the size of the first target.

[0130] S502: Determine the position of the first target and the dose of the first target according to the size of the first target, and update the state matrix of the corresponding target area.

[0131] When the size of the first target is determined, the size of the first target can be matched with the shape of the corresponding target area contour to determine the best position for placing the first target with the corresponding size in the target area, which is the position of the first target. Then, the dose of the first target is determined based on the position of the first target and the preset prescription dose.

[0132] In a possible implementation, determining the position of the first target and the dose of the first target according to the size of the first target in S103 may include:

[0133] Determine a mask for the first target according to the size of the first target;

[0134] Perform convolution shape matching on the target area contour according to the mask of the first target point to determine the position of the first target point;

[0135] Based on the location of the first target, the dose for the first target is determined.

[0136] In the present embodiment, the specific implementation process of determining the mask of the first target is similar to the example of the specific implementation process of determining the target mask of each target in the target target set, and the present embodiment will not repeat it here; accordingly, the specific implementation process of determining the position of the first target in the present embodiment is similar to the specific implementation process of determining the position of each target in the target target set within the target target area, and the specific description is referred to above, and the present embodiment will not repeat it here; accordingly, the specific implementation process of determining the dose of the first target is similar to the specific implementation of determining the dose of each target in the target target set in S104, and the specific description is referred to above, and the present embodiment will not repeat it here.

[0137] After the position of the first target and the dose of the first target are determined, the state matrix of the corresponding target area may be updated based on the dose of the first target.

[0138] S503: According to the updated state matrix, the size, position and dose of subsequent targets are determined in sequence until the preset prescription dose is met.

[0139] According to the updated state matrix, the process of determining the size, position and dose of the target is re-executed, and the state matrix is ​​updated once each time the size, position and dose of a target is determined, until the dose distribution of each target determined for the target area corresponding to the target area contour meets the requirements of the preset prescription dose, and the number of targets does not exceed the preset threshold value of the number of targets in a single target area. Among them, the dose distribution of each target determined for the target area corresponding to the target area contour meets the requirements of the preset prescription dose, for example, the sum of the total doses of all targets determined for the target area corresponding to the target area contour reaches the preset prescription dose.

[0140] Among them, the specific implementation of determining the size, position and dose of each subsequent target based on the updated state matrix can be similar to the implementation process of determining the size, position and dose of the first target based on the state matrix. For specific description, see the above.

[0141] Based on the method provided in the above embodiment, the state matrix may include not only the target area mask but also the dose state of the corresponding target area. Accordingly, in the above method, updating the state matrix of the corresponding target area may include:

[0142] According to the dose of the t-th target point and the volume of the corresponding target area, the dose status information of t targets in the corresponding target area is calculated; wherein t is an integer greater than or equal to 1;

[0143] According to the dose status information of t targets, the dose status in the status matrix is ​​updated.

[0144] The dose state information of t targets in the target area refers to the dose state information of the corresponding target area when t targets are placed in the corresponding target area. That is to say, when the dose of each target is determined by the above method, the dose state information of all current targets in the corresponding target area can be calculated based on the dose of the currently determined target and the volume of the corresponding target area, and then the dose state is updated. Among them, the volume of the target area refers to the volume of the three-dimensional area corresponding to the target area. For example, if t is 1, that is, the first target, when the dose of the first target is determined, the dose state information of 1 target in the corresponding target area can be calculated according to the dose of the first target and the volume of the corresponding target, and then the dose state in the state matrix is ​​updated according to the dose state information of 1 target. If t is an integer greater than 1, when the dose of the t-th target is determined, the dose state information of t targets in the corresponding target area can be calculated according to the dose of the t-th target, the volume of the corresponding target and the dose of the t-1 target before the t-th target, and then the dose state in the state matrix is ​​updated according to the dose state information of t targets.

[0145] In a possible implementation, the dose state shown above may include: dose coverage distribution, dose conformal distribution and dose overflow distribution. Among them, the dose coverage distribution, dose conformal distribution and dose overflow distribution are respectively used as three layers of feature matrices in the state matrix. That is to say, the state matrix may include: four layers of feature matrices, and the four layers of feature matrices may be target mask, dose coverage distribution, dose conformal distribution and dose overflow distribution, respectively. Among them, the target mask can be used as the first layer of feature matrix, the dose coverage distribution can be used as the second layer of feature matrix, the dose conformal distribution can be used as the third layer of feature matrix, and the dose overflow distribution is used as the fourth layer of feature matrix. The dose coverage distribution can be used to characterize the area where the dose distribution exists in the corresponding target area, the dose conformal distribution can be used to characterize the area where the dose is insufficient in the corresponding target area, and the dose overflow distribution can be used to characterize the area where the dose overflow exists in the corresponding target area.

[0146] Correspondingly, the dose status information includes: dose coverage information, dose conformity information and dose overflow information.

[0147] The dose coverage information may be, for example, based on the dose of the t-th target point and the volume of the corresponding target area, the dose coverage information, dose conformity information and dose overflow information of t targets in the corresponding target area are calculated using the following formulas (1) to (3) respectively.

[0148]

[0149]

[0150]

[0151] Among them, Cov t is the dose coverage information of t targets, also known as dose coverage, is the 50% dose of the t-th target and the t-1-th target, that is, the 50% dose after the two targets are executed. PTV is the volume of the corresponding target area. Con t The dose conformity information of t targets is also called dose conformity. t The dose overflow information of t targets is also called dose overflow degree.

[0152] In the method shown above, updating the dose status according to the dose status information of t targets may include:

[0153] According to the dose coverage information, dose conformal information and dose overflow information of t targets, the dose coverage distribution, dose conformal distribution and dose overflow distribution are updated respectively.

[0154] That is, according to the dose coverage information of the t targets, the dose coverage distribution in the state matrix is ​​updated. Accordingly, according to the dose conformity information of the t targets, the dose conformity distribution in the state matrix is ​​updated. Accordingly, according to the dose overflow information of the t targets, the dose overflow distribution in the state matrix is ​​updated.

[0155] S504, counting the number of the above target points, and using the number of target points and the size of each target point as a target point set of the target area contour.

[0156] For example, when the doses of all target points determined for the target area corresponding to the target area contour meet the preset prescription dose, the number of targets of different sizes among all the target points determined for the target area corresponding to the target area contour can be counted to obtain the number of target points of each size corresponding to the target area contour, and the number of target points and the size of each target point are stored as the target point set of the target area contour. In other words, the number of target points in the target point set of the target area contour can be the number for each target point size. For example, the target point set of the target area contour is: X targets of size A, Y targets of size B, and so on.

[0157] Of course, in some other possible implementations, the number of targets may also be the total number of targets in the target area corresponding to the target area contour, and the size of each target may be listed one by one, for example, the size of the first target is A size, the size of the second target is B size... the size of the tth target is U size... and so on.

[0158] Based on the method shown in the above embodiment, the method may further include:

[0159] According to the dose coverage information, dose conformity information and dose overflow information of t targets, the reward information of the tth target is calculated.

[0160] For example, the reward information of the tth target can be calculated using the following formula (4) based on the dose coverage information, dose conformity information, and dose overflow information of t targets, and the dose coverage information, dose conformity information, and dose overflow information of t-1 targets.

[0161]

[0162] Among them, R t is the reward information of the t-th target, Cov t is the dose coverage information of t targets, is the dose coverage information of t-1 targets, Con t is the dose conformal information of t targets, Con t-1 is the dose conformity information of t-1 targets, Out t is the dose overflow information of t targets, Out t-1 is the dose overflow information of t-1 targets.

[0163] In a possible implementation, the method may further include:

[0164] According to the reward information of the t-th target point, the current cumulative reward information of the t targets in the corresponding target area is calculated; the current cumulative reward information is used to characterize the reliability of the current target point set of the target area contour.

[0165] For example, according to the reward discount rate corresponding to each target among the t targets, the reward information of each target can be accumulated using the following formula (5) to obtain the current cumulative reward information of the t targets in the corresponding target area.

[0166]

[0167] Among them, R i is the i-th planning strategy π for the corresponding target area i , the current cumulative reward information, where i is the current planning strategy, indicating that each calculation of the size, position, and dose of a target is equivalent to obtaining a planning strategy for the corresponding target area. γ t-1 is the reward discount rate corresponding to the t-1th target, where γ can be approximately calculated and set to 0.1 based on the preset maximum number of targets, that is, the target number threshold. 1 / 20 ≈0.9, which can be used to characterize the impact of the 20 actions after t targets on the current state. Due to the existence of the discount rate, the impact of the targets determined by the later actions on the targets determined by the current action is smaller. R t is the reward information of the t-th target. t is an integer greater than or equal to 1.

[0168] The cumulative reward information is the cumulative return information of t targets. The higher its value, the higher the reliability of the target point set representing the target area contour. Conversely, the lower the value of the cumulative reward information, the lower the reliability of the current target point set representing the target area contour.

[0169] In a possible implementation, the method may further include:

[0170] Store the updated state matrix of the corresponding target area, the current target point set of the target area contour, and the current accumulated reward information of the corresponding target area.

[0171] The updated state matrix, the current target point set of the target area contour, and the current accumulated reward information of the corresponding target area are stored in the memory unit.

[0172] In another possible implementation, the method may further include:

[0173] According to the current cumulative reward information and the historical cumulative reward information of the corresponding target area, the relative advantage parameter between the current target point set of the target area contour and the historical target point set of the target area contour is calculated; the historical cumulative reward information is used to characterize the reliability of the historical target point set;

[0174] According to the relative advantage parameter, a target point set of the target area contour is determined and updated from the current target point set and the historical target point set.

[0175] For example, based on the current cumulative reward information and the historical cumulative reward information of the corresponding target area, the following formula (6) can be used to calculate the current target point set of the target area contour and the historical target point set of the target area contour, that is, the relative advantage parameter between the new and old strategies of the target area contour.

[0176]

[0177] In the above formula, Indicates t In state a, take action t The ratio of the new and old strategies, in this embodiment, action a t is the currently determined target size, P θ (a t |s t ) is action a t The probability of selecting the historical target set is For action a t The probability of selecting the current target set is s, so the ratio of the new and old strategies is actually s t The ratio of the selection probability of the current target set to the selection probability of the historical target set in the state. Among them, the selection probability of the current target set is determined based on the current cumulative reward information, and the selection probability of the historical target set is determined based on the historical cumulative reward information. represents the expected reward under the current strategy, which can be calculated based on the current cumulative reward information and the selection probability of the current target set. ε is the preset clipping coefficient of the advantage function. is the relative advantage parameter.

[0178] After the relative advantage parameter is calculated, the optimal target point set can be determined from the current target point set and the historical target point set based on the relative advantage parameter, and the target point set of the target area contour in the memory unit is updated to the optimal target target set. For example, if the relative advantage parameter satisfies the first condition, it can be determined that the current target point set is better than the historical target point set, so the current target set can be determined and updated as the target point set of the target area contour; if the relative advantage parameter satisfies the second condition, it can be determined that the historical target set is better than the current target set, and the historical target set can be determined as the optimal target set of the target area contour.

[0179] The method provided in this embodiment realizes machine trial and error during deep reinforcement learning training of target area contours, thereby finding the target point set corresponding to the target area contour, avoiding manual setting of target point combinations, and eliminating the need for artificial experience data as training samples, which greatly reduces labor costs and reduces dependence on clinical experience. It also ensures the accuracy of the target point set of the target area contour, thereby effectively ensuring the accuracy and efficiency of the treatment plan.

[0180] In order to ensure a better understanding of the solution provided in this application, the following will continue to explain the deep reinforcement learning training process of the target area contour through a complete and detailed implementation example in combination with the accompanying drawings. Figure 6 The framework diagram of a training module in a treatment plan generation method provided in an embodiment of the present application is a training module that is an algorithm model for deep reinforcement learning training of the target area contour. The training module can be used to train a plan model that meets the requirements of the preset prescription dose according to the input target area contour, also known as the target area shape, that is, the target point set of the target area contour. The process of training the target area contour using the training module is actually a process of accumulating experience through machine trial and error, which is equivalent to the trial and error process of a physicist in the treatment plan.

[0181] Input of the training module: based on the target area contour, a target area mask corresponding to the target area contour is produced.

[0182] The state characteristics of the environment are divided into four parts. Correspondingly, the state matrix is ​​a four-layer feature matrix. The first layer of the feature matrix, i.e., State[0], is the target mask. The second layer of the feature matrix, i.e., State[1], is the dose coverage distribution. The third layer of the feature matrix, i.e., State[2], is the dose conformal distribution. The fourth layer of the feature matrix, i.e., State[3], is the dose overflow distribution.

[0183] The intelligent agent trained by deep reinforcement learning is composed of a CNN and two different networks connected by the CNN, namely the action network (ActorNet) and the evaluation network (CriticNet). Among them, CNN is used to extract features from the state matrix to obtain state features, the action network is used to select actions based on the state features, that is, to determine the size of the target in the corresponding target area according to the state features, and the evaluation network is used to calculate the state value parameters of the corresponding target area based on the state features, that is, to obtain the evaluation parameter value of the state, and then calculate the loss function value of the intelligent agent according to the state evaluation parameters, and then determine whether it meets the conditions for stopping iteration.

[0184] L=αL actor +βL critic

[0185] L actor =mean(min(ratio*A t , clip(ratio, 1-ε, 1+ε)*A t ))

[0186] L critic =(A t +value-V critic ) 2

[0187] Where α=1,β=0.5,L actor is the loss function value of ActorNet, L critic is the loss function value of CriticNet. Radio is the relative advantage of the new and old strategies, that is, the relative advantage parameter, value is the evaluation parameter value of the state, A t The action selected at time t is the target size determined at time t.

[0188] The trained intelligent agent can be obtained by deep reinforcement learning training on the target area contour, and the target point size of the target area contour is determined by combining the convolution shape matching method to determine the position of each target point.

[0189] Reward mechanism of deep reinforcement learning training process: The reward is divided into two parts, real-time reward and delayed reward. The real-time reward is the growth value of 50% dose coverage and target conformity after calculating the size of the two targets before and after execution as a positive reward. At the same time, as few targets as possible are used, and a negative reward is given after each target is placed. The delayed reward is a large positive reward after the dose coverage when the preset prescription dose is met. Finally, the cumulative reward information of each planning strategy, that is, each target, is calculated, that is, the cumulative return.

[0190] The experience memory unit is used to store the combination of state matrix St, action At, reward Rt, value Vt and other information. The specific description of the state matrix St can be found above, the action At is the size of the t-th target determined by the action network, the reward Rt is the reward information of the t-th target, and the value Vt is the evaluation parameter value obtained by evaluating the state matrix St.

[0191] Strategy update module: The proximal policy optimization (PPO) algorithm can be used to optimize and update the target point set of the target area contour stored in the memory unit, that is, the strategy. For example, based on the current cumulative reward information of the corresponding target area and the historical cumulative reward information of the corresponding target area, the PPO algorithm can be used to calculate the current target point set of the target area contour and the historical target point set of the target area contour, that is, the relative advantage parameters between the new and old strategies of the target area contour. Among them, the current target point set can be represented as the new strategy (New_n) of the corresponding target area, and the historical target set can be represented as the old strategy (Old_n) of the corresponding target area.

[0192] Figure 7 Another method flow chart of deep reinforcement learning training of target area contour in a treatment method generation method provided in an embodiment of the present application, such as Figure 7 As shown, the method includes:

[0193] S701 : Generate a target mask corresponding to a target contour according to a randomly generated target contour and an OAR image.

[0194] S702: Determine the area corresponding to the target area outline in the target area mask as the target search area.

[0195] By determining the area corresponding to the target contour in the target mask as the target search area, the target position can be restricted to the area corresponding to the target contour, while preventing the target position from falling into other areas, thereby causing unnecessary damage to other areas such as endangered organ areas during treatment. This also reduces search controls and improves the efficiency of training and planning.

[0196] S703: construct a state matrix corresponding to the target area according to the target area mask.

[0197] S704, using CNN to extract features from the state matrix to obtain state features, and using an action network to obtain the first target point size of the corresponding target area based on the state features.

[0198] S705 . Perform convolution shape matching on the target area contour according to the size of the first target point to determine the position of the first target point.

[0199] S706: Determine the dose of the first target point according to the position of the first target point and the preset prescription dose.

[0200] S707. Calculate the dose status information of a target in the corresponding target area according to the dose of the first target and the volume of the corresponding target area, update the status matrix of the corresponding target area, and calculate the cumulative reward information of the target.

[0201] S708. Based on the updated state matrix, the process is repeated within the preset target number threshold to determine the size, position and dose of subsequent targets, and the state matrix is ​​updated to calculate the cumulative reward information of the current target until the dose distribution of the targets in the corresponding target area meets the preset prescription dose, and the target set of the target area contour is stored.

[0202] S709, re-randomly generate another target area contour, re-perform deep reinforcement learning training based on the re-generated target area contour, and obtain a target point set of another target area contour until a preset stop iteration condition is reached.

[0203] S710: Establish a preset target point mapping relationship according to each target area contour and a target point set corresponding to the target area contour.

[0204] The method provided in this embodiment performs deep reinforcement learning training on randomly generated target area contours, thereby realizing machine trial and error in the deep reinforcement learning training process, thereby finding the target point set corresponding to the target area contour, greatly reducing labor costs, while also reducing dependence on clinical experience, and ensuring the accuracy of the target point set of the target area contour, thereby effectively ensuring the accuracy and efficiency of the treatment plan.

[0205] The following is a description of the treatment plan generation device, equipment, storage medium, etc. provided in the present application for execution. The specific implementation process and technical effects are as mentioned above and will not be repeated below.

[0206] Figure 8 A schematic diagram of a treatment plan generating device provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the treatment plan generating device 800 may include:

[0207] An acquisition module 801 is used to acquire a target contour of a target area;

[0208] The search module 802 is used to search for a target point set corresponding to the target contour in the preset target point mapping relationship; the target point set includes the number of target points and the size of each target point.

[0209] A determination module 803 is used to determine the position of each target point in the target area according to the size of each target point;

[0210] The generation module 804 is used to determine the dose of each target point according to the position of each target point and the preset prescription dose, and generate a treatment plan.

[0211] Optionally, the determination module 803 is specifically used to determine the target mask of each target point according to the size of the target point; perform convolution shape matching on the target contour according to the target mask to determine the position of each target point in the target area.

[0212] In one embodiment, the generation module 804 is specifically used to calculate the dose according to the size, position and weight of each target point to obtain the dose line distribution of the target area; determine the dose of each target point according to the dose line distribution and the preset prescription dose.

[0213] In one embodiment, the treatment plan generating device 800 further includes:

[0214] The training module is used to obtain multiple target areas before searching for a target point set corresponding to a target contour in a preset target point mapping relationship; delineate multiple target areas to obtain target area contours of multiple target areas; perform deep reinforcement learning training based on each target area contour to obtain a target point set corresponding to the target area contour; and establish a preset target point mapping relationship based on the target area contour and the target point set corresponding to the target area contour.

[0215] In one of the embodiments, the training module is specifically used to create a target area mask corresponding to the target area according to the target area contour; construct a state matrix corresponding to the target area according to the target area mask, wherein the state matrix includes: the target area mask; and obtain a target point set of the target area contour according to the state matrix.

[0216] In one of the embodiments, the training module is specifically used to extract features from the state matrix based on a convolutional neural network to obtain the size of the first target; determine the position of the first target and the dose of the first target according to the size of the first target, and update the state matrix of the corresponding target area; determine the size, position and dose of subsequent targets in sequence according to the updated state matrix until the preset prescription dose is met; count the number of the above-mentioned targets, and use the number of targets and the size of each target as the target set of the target area contour.

[0217] In one of the embodiments, the training module is specifically used to use a convolutional neural network to extract features from the state matrix to obtain initial state features of the corresponding target area; and use a preset action selection network to process the initial state features to obtain the size of the first target point.

[0218] In one of the embodiments, the training module is specifically used to determine the mask of the first target according to the size of the first target; perform convolution shape matching on the target area contour according to the mask of the first target to determine the position of the first target; and determine the dose of the first target according to the position of the first target.

[0219] In one embodiment, the state matrix also includes: a dose state of the corresponding target area, a training module, which is specifically used to calculate the dose state information of t target points in the corresponding target area based on the dose of the tth target point and the volume of the corresponding target area; wherein t is an integer greater than or equal to 1; and update the dose state based on the dose state information of the t target points.

[0220] In one embodiment, the dose status includes: dose coverage distribution, dose conformal distribution and dose overflow distribution; the dose status information includes: dose coverage information, dose conformal information and dose overflow information.

[0221] The training module is specifically used to update the dose coverage distribution, dose conformal distribution and dose overflow distribution according to the dose coverage information, dose conformal information and dose overflow information of t targets.

[0222] In one embodiment, the training module is further used to calculate the reward information of the t-th target based on the dose coverage information, dose conformity information and dose overflow information of the t targets.

[0223] In one embodiment, the training module is further used to calculate the current cumulative reward information of t targets in the corresponding target area based on the reward information of the tth target point; the current cumulative reward information is used to characterize the reliability of the current target point set of the target area contour.

[0224] In one of the embodiments, the training module is also used to store the updated state matrix, the current target point set of the target area contour, and the current accumulated reward information.

[0225] In one of the embodiments, the training module is also used to calculate the relative advantage parameter between the current target point set of the target area contour and the previous target point set of the target area contour based on the current cumulative reward information and the historical cumulative reward information of the corresponding target area; the historical cumulative reward information is used to characterize the reliability of the previous target point set; based on the relative advantage parameter, the target target point set of the target area contour is determined and updated from the current target point set and the previous target set.

[0226] The above-mentioned device is used to execute the treatment plan generation method provided in the above-mentioned embodiment. Its implementation principle and technical effect are similar and will not be described in detail here.

[0227] The above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital singnal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0228] Fig. 9 A schematic diagram of a computer device provided in an embodiment of the present application, the specific product form of the computer device may be a computing device or server with computing and processing functions.

[0229] The computer device 900 includes a memory 901 and a processor 902. The memory 901 and the processor 902 are connected via a bus.

[0230] The memory 901 stores a computer program executable by the processor 902. When the processor 902 executes the computer program, it can implement the above method embodiment. The specific implementation method and technical effect are similar and will not be repeated here.

[0231] Based on the above-mentioned radiotherapy dose determination method, the embodiment of the present application can also provide a computer-readable storage medium for executing the above-mentioned radiotherapy dose determination method, which can be a non-volatile storage medium on which a computer program is stored. The computer program can execute the above-mentioned method embodiment when it is read and executed.

[0232] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0233] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0234] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0235] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English: Read-Only Memory, abbreviated: ROM), random access memory (English: Random Access Memory, abbreviated: RAM), disk or optical disk and other media that can store program codes.

[0236] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A computer device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program executable by the processor, and when the processor executes the computer program, a treatment plan generation method is implemented, wherein the treatment plan generation method includes: Obtaining the target contour of the target area; Searching for a target point set corresponding to the target contour in a preset target point mapping relationship; the target point set includes the number of targets and the size of each target point, and the preset target point mapping relationship is obtained by deep reinforcement learning training based on a plurality of target area contours with different shapes in advance; Determining the position of each target point within the target area according to the size of each target point; Determine the dose of each target and generate a treatment plan based on the location of each target and the preset prescription dose; The pre-set target mapping relationship is obtained by performing deep reinforcement learning training according to a plurality of target area contours with different shapes, including: According to the target area contour, a target area mask corresponding to the target area is produced; Constructing a state matrix corresponding to the target area according to the target area mask, wherein the state matrix includes: the target area mask; According to the state matrix, a target point set corresponding to the target area contour is obtained; The preset target point mapping relationship is established according to the target area contour and the target point set corresponding to the target area contour.

2. The device according to claim 1, characterized in that Determining the position of each target point in the target area according to the size of each target point includes: Determining a target mask for each target according to the size of the target; The target contour is convolutionally matched according to the target point mask to determine the position of each target point in the target area.

3. The device according to claim 1, characterized in that Determining the dose of each target point according to the position of each target point and the preset prescription dose includes: Calculate the dose according to the size, position and weight of each target point to obtain the dose line distribution of the target area; The dose of each target point is determined according to the dose line distribution and the preset prescription dose.

4. The device according to claim 1, characterized in that The step of obtaining a target point set of the target area contour according to the state matrix comprises: Performing feature extraction on the state matrix based on a convolutional neural network to obtain the size of the first target; Determine the position of the first target and the dose of the first target according to the size of the first target, and update the state matrix of the corresponding target area; According to the updated state matrix, the size, position and dose of subsequent targets are determined in sequence until the preset prescription dose is met; The number of the above target points is counted, and the number of target points and the size of each target point are used as the target point set of the target area contour.

5. The device according to claim 4, characterized in that The step of extracting features from the state matrix based on a convolutional neural network to obtain the size of the first target point includes: Using the convolutional neural network to extract features from the state matrix to obtain initial state features of the corresponding target area; The initial state feature is processed using a preset action selection network to obtain the size of the first target point.

6. The device according to claim 4, characterized in that Determining the position of the first target and the dose of the first target according to the size of the first target includes: Determining a mask for the first target point according to a size of the first target point; Performing convolution shape matching on the target area contour according to the mask of the first target point to determine the position of the first target point; The dosage of the first target is determined according to the position of the first target.

7. The device according to claim 4, characterized in that The state matrix also includes: a dose state of the corresponding target area, and updating the state matrix of the corresponding target area includes: According to the dose of the t-th target point and the volume of the corresponding target area, the dose status information of t targets in the corresponding target area is calculated; wherein t is an integer greater than or equal to 1; The dosage status is updated according to the dosage status information of the t targets.

8. The device according to claim 7, characterized in that The dose state includes: dose coverage distribution, dose conformal distribution and dose overflow distribution; the dose state information includes: dose coverage information, dose conformal information and dose overflow information; The updating of the dosage status according to the dosage status information of the t targets includes: According to the dose coverage information, dose conformity information and dose overflow information of the t targets, the dose coverage distribution, the dose conformity distribution and the dose overflow distribution are updated respectively.

9. The device according to claim 8, characterized in that The method further comprises: The reward information of the t-th target is calculated according to the dose coverage information, the dose conformity information and the dose overflow information of the t targets.

10. The device according to claim 9, characterized in that The method further comprises: Calculate the current cumulative reward information of t targets in the corresponding target area according to the reward information of the t-th target; The current accumulated reward information is used to characterize the reliability of the current target point set of the target area contour.

11. The device according to claim 10, characterized in that The method further comprises: The updated state matrix, the current target point set of the target area contour, and the current accumulated reward information are stored.

12. The device according to claim 11, characterized in that The method further comprises: Calculate the relative advantage parameter between the current target point set of the target area contour and the previous target point set of the target area contour according to the current cumulative reward information and the historical cumulative reward information of the corresponding target area; the historical cumulative reward information is used to characterize the reliability of the previous target point set; According to the relative advantage parameter, a target target point set of the target area contour is determined and updated from the current target point set and the previous target point set.

13. A non-volatile storage medium, characterized in that: The storage medium stores a computer program. When the computer program is read and executed, a treatment plan generation method is implemented. The treatment plan generation method includes: Obtaining the target contour of the target area; Searching for a target point set corresponding to the target contour in a preset target point mapping relationship; the target point set includes the number of targets and the size of each target point, and the preset target point mapping relationship is obtained by deep reinforcement learning training based on a plurality of target area contours with different shapes in advance; Determining the position of each target point within the target area according to the size of each target point; Determine the dose of each target and generate a treatment plan based on the location of each target and the preset prescription dose; The pre-set target mapping relationship is obtained by performing deep reinforcement learning training according to a plurality of target area contours with different shapes, including: According to the target area contour, a target area mask corresponding to the target area is produced; Constructing a state matrix corresponding to the target area according to the target area mask, wherein the state matrix includes: the target area mask; According to the state matrix, a target point set corresponding to the target area contour is obtained; The preset target point mapping relationship is established according to the target area contour and the target point set corresponding to the target area contour.

Images