Intensity modulated radiotherapy flux prediction method and system based on cascaded three-dimensional U-Net network

Abstract

The invention relates to the technical field of radiotherapy, and discloses an intensity modulated radiotherapy flux prediction method and system based on a cascaded three-dimensional U-Net network. The method comprises the following steps: inputting a three-dimensional CT image and a three-dimensional contour into a first three-dimensional U-net network, and extracting an initial three-dimensional flux feature map; and inputting the three-dimensional CT image, the three-dimensional contour and the initial three-dimensional flux feature map into a second three-dimensional U-net network, extracting a final three-dimensional flux feature map, and converting the final three-dimensional flux feature map into a final two-dimensional flux map through embedded matrix geometric projection operation to output the final two-dimensional flux map. The method does not need to carry out two-dimensional projection on the organ contour or indirectly predict the flux through the prediction and projection of the radiation field dose, can directly input the 3D organ contour to predict the 2D flux, and adopts an end-to-end input and output mode, so that the data generation is faster, the feature extraction is more comprehensive, and the prediction robustness and reliability are stronger.

Description

Technical Field

[0001] The present application relates to the field of radiotherapy technology, and in particular to an intensity modulated radiotherapy flux prediction method and system based on a cascaded three-dimensional U-Net network. Background Art

[0002] About 70% of cancer patients need radiotherapy during the treatment process. Its working principle is to use high-energy rays to kill or hinder the growth of cancer cells. Intensity-modulated radiation therapy (IMRT) adjusts the intensity of high-energy X-ray beams through multi-leaf collimators (MLCs) and other devices. It has the unique advantage of delivering conformal doses to the target area and protecting adjacent normal tissues and organs.

[0003] Radiotherapy plans are designed through the Treatment Planning Treatment (TPS). Plan optimization is the most critical and time-consuming step. It adopts a reverse optimization strategy, adjusts the objective function and constraints according to the prescribed dose and the limit of organs at risk, and uses the optimization algorithm to solve the optimal flux distribution or MLC sequence. It takes a lot of time, and the quality of the plan depends on the knowledge level of the designer.

[0004] Research on improving the efficiency of plan design and the consistency of plan quality can be mainly divided into knowledge-based plan design, multi-criteria optimization methods, dose prediction methods, and flux prediction methods. Knowledge-based planning (KBP) can provide guidance for reverse planning by using the radiotherapy plan database to estimate the achievable dose-volume histogram (DVH) curve, improve the planning efficiency to a certain extent, and reduce the variability of plan quality, but DVH cannot provide information on the location distribution of dose. The multi-criteria optimization method (MCO) simultaneously generates an "anchor" plan that maximizes each dosimetric criterion. Multiple anchor point schemes form a Pareto surface, and interpolation is used to find the optimal plan. It has clinical benefits, but there are difficulties in establishing mathematical models and time-consuming calculations. Through deep learning methods, predicting three-dimensional dose distribution from convolutional neural networks can help guide plan optimization, reduce the number of iterations, and improve efficiency and plan quality. However, in order to transform dose distribution into a deliverable plan, reverse optimization is still required. Recent studies directly predict the flux itself, bypassing the inverse optimization, and then convert the predicted flux into the MLC blade sequence to obtain the final solution.

[0005] The method of directly predicting the flux map by bypassing inverse optimization can effectively improve the efficiency of plan design and the consistency of plan quality. The existing flux prediction methods need to convert the three-dimensional anatomical structure into a two-dimensional projection first, or need to predict the three-dimensional dose first and then project the three-dimensional dose into two dimensions, or use another convolutional network to assist in the conversion of the three-dimensional anatomical structure to the two-dimensional flux, ensuring that the input dimension of the flux prediction network is two-dimensional before predicting the flux. However, the two-dimensional projection is to compress the information carried by the three-dimensional contour. If the patient's anatomical structure is more complex clinically, or there are more target areas and organs at risk that need to be considered, the reliability and robustness of the information captured after projection are questionable. Once a clinical patient with a more complex situation is encountered, the flux map predicted after projection is risky. Furthermore, the input must first be converted from a three-dimensional volume to a two-dimensional projection before being sent to the network, which requires additional data processing operations, increasing the complexity and time overhead of the prediction process. Summary of the invention

[0006] The present application provides an intensity modulated radiotherapy flux prediction method based on a cascaded three-dimensional U-Net network to solve the problems in the prior art that the existing flux prediction method converts the three-dimensional anatomical structure into a two-dimensional projection, the flux map predicted has low reliability, and the prediction process is complicated due to additional data processing steps.

[0007] Correspondingly, the present application also provides an intensity modulated radiotherapy flux prediction system based on a cascaded three-dimensional U-Net network, an electronic device, and a computer-readable storage medium to ensure the implementation and application of the above method.

[0008] In order to solve the above technical problems, the present application discloses an intensity modulated radiotherapy flux prediction method based on a cascaded three-dimensional U-Net network, wherein the cascaded 3D U-Net network includes a first three-dimensional U-net network and a second three-dimensional U-net network. The method based on the cascaded 3D U-Net network includes:

[0009] Inputting the 3D CT image and the 3D contour into the first 3D U-net network to extract the initial 3D flux feature map;

[0010] Inputting the three-dimensional CT image, the three-dimensional contour and the initial three-dimensional flux feature map into the second three-dimensional U-net network, extracting the final three-dimensional flux feature map, and predicting and outputting the final two-dimensional flux map based on the final three-dimensional flux feature map;

[0011] Among them, the second three-dimensional U-net network converts the final three-dimensional flux feature map into the final two-dimensional flux map output through the embedded matrix geometric projection operation.

[0012] The present application also discloses an intensity modulated radiotherapy flux prediction system based on a cascaded three-dimensional U-Net network, wherein the cascaded 3D U-Net network includes a first three-dimensional U-net network and a second three-dimensional U-net network, and the system based on the cascaded 3D U-Net network includes:

[0013] A first prediction module is used to input the three-dimensional CT image and the three-dimensional contour into a first three-dimensional U-net network to extract an initial three-dimensional flux feature map;

[0014] A second prediction module is used to input the three-dimensional CT image, the three-dimensional contour and the initial three-dimensional flux feature map into a second three-dimensional U-net network, extract the final three-dimensional flux feature map, and predict and output the final two-dimensional flux map based on the final three-dimensional flux feature map;

[0015] Among them, the second three-dimensional U-net network converts the final three-dimensional flux feature map into the final two-dimensional flux map output through the embedded matrix geometric projection operation.

[0016] The present application also discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, one or more methods described in the present application are implemented.

[0017] The present application also discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, one or more methods described in the present application are implemented.

[0018] In this application, a three-dimensional CT image and a three-dimensional contour are input into a first three-dimensional U-net network to extract an initial three-dimensional flux feature map; a three-dimensional CT image, a three-dimensional contour and an initial three-dimensional flux feature map are input into a second three-dimensional U-net network to extract a final three-dimensional flux feature map, and a final two-dimensional flux map is predicted and output based on the final three-dimensional flux feature map. Among them, the second three-dimensional U-net network converts the final three-dimensional flux feature map into a final two-dimensional flux map output through an embedded matrix geometric projection operation. The matrix geometric projection operation can be easily inserted into the forward and reverse propagation process of network training, without first projecting the organ contour in two dimensions or indirectly predicting the flux through the prediction and projection of the radiation field dose. The 3D organ contour can be directly input to predict the 2D flux. By adopting an end-to-end input and output method, data generation is faster, feature extraction is more comprehensive, and the robustness and reliability of the prediction are also stronger.

[0019] Additional aspects and advantages of the present application will be given in the following description, which will become apparent from the following description, or will be understood through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0021] Figure 1 A flowchart of an intensity modulated radiotherapy flux prediction method based on a cascaded three-dimensional U-Net network provided in an embodiment of the present application;

[0022] Figure 2 A cascaded three-dimensional U-Net network structure diagram provided in an embodiment of the present application;

[0023] Figure 3 A comparison diagram of two-dimensional flux diagrams of test cases provided in embodiments of the present application;

[0024] Figure 4 The MAE, SSIM and gamma pass rate analysis result diagram between the predicted value and the true value of the two-dimensional flux map provided in the embodiment of the present application;

[0025] Figure 5 A schematic diagram of the structure of an intensity modulated radiotherapy flux prediction system based on a cascaded three-dimensional U-Net network provided in an embodiment of the present application;

[0026] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0028] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0029] Those skilled in the art will appreciate that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the field to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0030] The solution provided in the embodiments of the present application can be executed by any electronic device, such as a terminal device or a server, wherein the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this. With regard to the technical problems existing in the prior art, the intensity modulated radiotherapy flux prediction method and system based on the cascaded three-dimensional U-Net network provided in this application is intended to solve at least one of the technical problems of the prior art.

[0031] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0032] The present application embodiment provides a possible implementation method, such as Figure 1 As shown, a flowchart of an intensity modulated radiotherapy flux prediction method based on a cascaded three-dimensional U-Net network is provided. The scheme can be executed by any electronic device, and optionally, can be executed on a server or terminal device.

[0033] Among them, flux: In the radiation field, the flux at a certain point is the ratio of the number of particles entering the small sphere with the point as the center to its cross-section, that is, the number of incident particles per unit area.

[0034] Relationship between flux and three-dimensional flux volume: The three-dimensional flux volume d can be related to the flux f through the geometric projection matrix D, which describes the mapping relationship from f to d: d = D × f, where the element value of the geometric projection matrix D ij represents the flux contribution of voxel i received from the unit intensity photon pencil beam j.

[0035] In the embodiment of the present application, an improvement is made on the basis of the three-dimensional U-Net network, and an extended network is proposed: a cascaded three-dimensional U-Net network, including a first three-dimensional U-net network and a second three-dimensional U-net network, such as Figure 1 As shown in , based on the cascaded three-dimensional U-Net network, the method may include the following steps:

[0036] Step 101: input the three-dimensional CT image and the three-dimensional contour into a first three-dimensional U-net network to extract an initial three-dimensional flux feature map.

[0037] Step 102, input the three-dimensional CT image, the three-dimensional contour and the initial three-dimensional flux feature map into a second three-dimensional U-net network, extract the final three-dimensional flux feature map, and predict and output the final two-dimensional flux map based on the final three-dimensional flux feature map.

[0038] Among them, the second three-dimensional U-net network converts the final three-dimensional flux feature map into the final two-dimensional flux map output through the embedded matrix geometric projection operation.

[0039] In an embodiment of the present application, the three-dimensional contour may include a PTV mask and an OAR mask. The input of the cascaded 3D U-Net network is the PTV mask filled with the prescription dose, the OAR binary mask and the CT image. The input data is normalized and input into the cascaded 3DU-Net network. After deep convolutional network inference calculation, the predicted two-dimensional flux is output, and finally denormalized to obtain the final two-dimensional flux map to achieve fast and accurate flux prediction.

[0040] The first three-dimensional U-net network is used to predict the rough flux distribution, and the second three-dimensional U-net network is used to further fine-tune the prediction results of the first three-dimensional U-net network. The cascaded three-dimensional U-Net network structure is as follows: Figure 2As shown in the figure, the basic structures of the first 3D U-net network (model A) and the second 3D U-net network (model B) are the same. The cascade mechanism is as follows: the input of the first 3D U-net network (input A) has 19 channels, including 1 PTV (Plan Target Volume) mask, 17 OARs (Organs At Risk) and masks, and 1 CT image. The input of the first 3D U-net network (19 channels) and the 3D flux feature map before the matrix projection operation (1 channel) are used as the input of the second 3D U-net network (input B), so that the input of the second 3D U-net network has 19+1 channels, that is, 20 channels. The first 3D U-net network and the second 3D U-net network predict the 2D flux map respectively, and only the output of the second 3D U-net network is used as the final output of the cascaded 3D U-Net network. The output of the first 3D U-net network is used to assist the optimization of the second 3D U-net network and reduce a certain burden of network training. During the model training process, the initial prediction flux of the first three-dimensional U-net network and the final prediction flux of the second three-dimensional U-net network are synchronously supervised. This end-to-end input and output method reduces certain data preprocessing steps and generates data faster. At the same time, the cascaded three-dimensional U-Net network can fully extract three-dimensional features to improve the robustness and reliability of prediction.

[0041] In an embodiment of the present application, the backbone network of the first three-dimensional U-net network and the second three-dimensional U-net network can adopt any three-dimensional U-net network or its variants. When the two-dimensional flux map is output, it includes the matrix geometric projection transformation operation of the corresponding three-dimensional flux feature map to the two-dimensional flux map. The embodiment of the present application can improve the planning design efficiency and the consistency of planning design quality.

[0042] In an embodiment of the present application, a three-dimensional CT image and a three-dimensional contour are input into a first three-dimensional U-net network to extract an initial three-dimensional flux feature map; a three-dimensional CT image, a three-dimensional contour and an initial three-dimensional flux feature map are input into a second three-dimensional U-net network to extract a final three-dimensional flux feature map, and a final two-dimensional flux map is predicted and output based on the final three-dimensional flux feature map. Among them, the second three-dimensional U-net network converts the final three-dimensional flux feature map into a final two-dimensional flux map output through an embedded matrix geometric projection operation. The matrix geometric projection operation can be easily inserted into the forward and reverse propagation process of network training, without first projecting the organ contour in two dimensions or indirectly predicting the flux through the prediction and projection of the radiation field dose. The 3D organ contour can be directly input to predict the 2D flux. By adopting an end-to-end input and output method, data generation is faster, feature extraction is more comprehensive, and the robustness and reliability of the prediction are also stronger.

[0043] In an optional embodiment, the first three-dimensional U-net network and the second three-dimensional U-net network both include an encoding path and a decoding path;

[0044] The encoding path uses convolution and downsampling operations to extract key features and reduce the resolution of the input data;

[0045] The decoding path uses upsampling operations, convolution operations, and skip connections to recover the information lost during downsampling of the input data.

[0046] like Figure 2 As shown in Figure 1, the basic model of the cascaded 3D U-Net includes an encoding path with five resolution levels and a decoding path with four resolution levels. In the encoding path, traditional convolution and downsampling operations are used to extract key features and reduce the resolution of the image. The first level of the encoding path consists of two convolution operations with a convolution kernel size of 3×3×3 and a stride of 1. In the following four levels of the encoding path, each level contains a 3×3×3 convolution with a stride of 2 for downsampling and a 3×3×3 convolution with a stride of 1 for learning features. After each downsampling, the number of channels of the feature map doubles and the size of the feature map is halved. After the encoding path, the number of feature maps increases from 32 to 512. In the decoding path, upsampling, convolution and skip connections are used to restore image details and size, where the upsampling layer consists of trilinear interpolation and convolution. The information lost during downsampling is restored by skipping the corresponding feature maps from the encoding path to the decoding path. After the decoding path, the number of feature maps is reduced from 512 to 32, and finally a 1×1×1 convolution block with a step size of 1 is passed to obtain the final output result with a channel of 1.

[0047] In an optional embodiment, instance normalization and rectified linear unit activation are used after each convolution operation, downsampling operation, and upsampling operation.

[0048] Optionally, each 3×3×3 convolution is followed by Instance Normalization (IN) and Rectified Linear Unit (ReLU) activation to prevent overfitting and gradient explosion.

[0049] In an optional embodiment, the matrix geometry projection operation is to multiply the projection matrix and the final three-dimensional flux feature map:

[0050] f=P T ·v

[0051] Where f is the final two-dimensional flux map, v is the final three-dimensional flux feature map predicted by the network, T is the transposition operation, and P is the number of elements P ijThe geometric projection matrix P refers to the flux contribution that the flux voxel i receives from the photon beam j of unit intensity. T The calculation of takes into account only the inverse square effect and is stored as a sparse matrix.

[0052] The data dimension of the flux map is two-dimensional, but the transmission data dimension in the network model is three-dimensional. Therefore, in the decoder path for generating the flux map, in order to solve the dimension conversion problem between the three-dimensional volume feature space and the two-dimensional flux map, the embodiment of the present application introduces a three-dimensional to two-dimensional matrix geometric projection operation to obtain the flux map.

[0053] The above geometric projection operation can be easily inserted into the forward and back-propagation processes of network training.

[0054] In an optional embodiment, after inputting the three-dimensional CT image and the three-dimensional contour into the first three-dimensional U-net network and extracting the initial three-dimensional flux feature map, the method further includes:

[0055] An initial two-dimensional flux map is predicted and output based on the initial three-dimensional flux characteristic map;

[0056] Among them, the first three-dimensional U-net network converts the initial three-dimensional flux feature map into an initial two-dimensional flux map output through the embedded matrix geometric projection operation.

[0057] In the embodiment of the present application, the first three-dimensional U-net network and the second three-dimensional U-net network respectively predict the corresponding two-dimensional flux map, but only the two-dimensional flux map output by the second three-dimensional U-net network (i.e., the final two-dimensional flux map) is used as the final output of the cascaded three-dimensional U-net network. Therefore, in the embodiment of the present application, after the three-dimensional CT image and three-dimensional contour of the patient are input into the first three-dimensional U-net network, an initial three-dimensional flux feature map is obtained through an encoding path and a decoding path, and then the initial three-dimensional flux feature map is converted into an initial two-dimensional flux map output through an embedded matrix geometric projection operation, but the initial three-dimensional flux feature map obtained before the matrix geometric projection operation is used as the input of the second three-dimensional U-net network to assist in the optimization of the second three-dimensional U-net network.

[0058] In an optional embodiment, before inputting the three-dimensional CT image and the three-dimensional contour into the first three-dimensional U-net network and extracting the initial three-dimensional flux feature map, the method further includes:

[0059] Multiple consecutive slices are randomly selected as input data, and the two-dimensional flux map of the corresponding position is extracted as the label value;

[0060] Normalize the input data and generate a training data set based on the normalized input data and label values;

[0061] Perform data augmentation on the data in the training data set; the data augmentation includes randomly flipping in the left-right direction and randomly rotating the specified rack angle along the central axis of the cross section;

[0062] The first 3D U-net network and the second 3D U-net network are trained based on the enhanced data.

[0063] Optionally, in the embodiment of the present application, 32 consecutive slices are randomly selected as input data, a PTV mask, seventeen OAR masks and a CT image are stacked into a 4D matrix of size 19×32×224×224, and a two-dimensional flux map patch (9×32×160) at the corresponding position is extracted as a label value. Before training, the PTV mask is normalized by 7000cGy, the flux map is normalized by 2000 monitoring units (MU), and the value of the CT image is first trimmed to the range of -1024–2000HU and then normalized by 2000HU.

[0064] During training, two data augmentations were used: random flipping in the left-right direction with a probability of 0.6, and random rotation around the cross-sectional center axis at an angle of {40°, 80°, 120°, 160°, 200°, 240°, 280°, and 320°} with a probability of 0.4. The input data of the cascaded 3D U-Net network was augmented to expand the dataset and avoid overfitting. No data augmentation was performed on the validation set.

[0065] In an optional embodiment, the loss function of the cascaded 3D U-Net network is:

[0066] loss=0.5*MAE(A output ,truth)+MAE(B output ,truth)

[0067] Among them, truth represents the label value of the real flux, A output ,、B output They represent the prediction flux of the first three-dimensional U-net network and the second three-dimensional U-net network respectively. MAE represents the mean absolute error between the calculated output value and the label value. Finally, the sum value is used for back propagation to update the network parameters. The coefficient of 0.5 indicates that the prediction result of model A has a certain influence on the network prediction, but the prediction result of the second three-dimensional U-net network is still the main weight.

[0068] In an optional embodiment, the optimizer uses Adam to minimize the loss function, the batch size (BatchSize) is set to 2, the initial learning rate is set to 0.0003, and when the validation loss does not decrease after 4 cycles of training, the learning rate is reduced by 30% using the ReduceLROnPlateau scheduler. Network training is done on an NVIDIA RTX Titan GPU with 24GB of memory, using Python 3.8 and Pytorch 1.10.1 algorithm framework, and a total of 100 epochs are trained, with 400 iterations in each epoch.

[0069] The model with the smallest loss in the validation set was used to test the 40-case test set. The test process used a sliding window method to sample the input data of each patient into a patch with a stride size of 24 (8 overlapping patches), which was input into the trained network to generate the corresponding flux patch, and the overlapping area was smoothed using a logarithmic function for splicing, and finally the final two-dimensional flux map of the original size was generated. Figure 3 As shown, the left side is the actual flux map of three of the fields, the middle is the final predicted flux map, and the right side is the difference between the final predicted flux map and the actual value.

[0070] like Figure 4 As shown, the predicted final two-dimensional flux map was evaluated by using the final two-dimensional flux map difference, MAE (%), structural similarity index (SSIM) and gamma pass rate (standard is 3% / 3mm) under two flux thresholds (0%, 10%). Among them, the MAE, SSIM and gamma pass rate (mean ± variance) between the predicted value and the true value of the final two-dimensional flux map of 40 test patients are shown in Table 1:

[0071] Table 1

[0072]

[0073] Based on the same principle as the method provided in the embodiment of the present application, the embodiment of the present application also provides an intensity modulated radiotherapy flux prediction system based on a cascaded three-dimensional U-Net network, wherein the cascaded 3D U-Net network includes a first three-dimensional U-net network and a second three-dimensional U-net network. Figure 5 As shown, the system based on the cascaded 3D U-Net network includes:

[0074] The first prediction module 501 is used to input the three-dimensional CT image and the three-dimensional contour into a first three-dimensional U-net network to extract an initial three-dimensional flux feature map;

[0075] A second prediction module 502 is used to input the 3D CT image, the 3D contour and the initial 3D flux feature map into a second 3D U-net network, extract the final 3D flux feature map, and predict and output the final 2D flux map based on the final 3D flux feature map;

[0076] Among them, the second three-dimensional U-net network converts the final three-dimensional flux feature map into the final two-dimensional flux map output through the embedded matrix geometric projection operation.

[0077] In an embodiment of the present application, a three-dimensional CT image and a three-dimensional contour are input into a first three-dimensional U-net network to extract an initial three-dimensional flux feature map; a three-dimensional CT image, a three-dimensional contour and an initial three-dimensional flux feature map are input into a second three-dimensional U-net network to extract a final three-dimensional flux feature map, and a final two-dimensional flux map is predicted and output based on the final three-dimensional flux feature map. Among them, the second three-dimensional U-net network converts the final three-dimensional flux feature map into a final two-dimensional flux map output through an embedded matrix geometric projection operation. The matrix geometric projection operation can be easily inserted into the forward and reverse propagation process of network training, without first projecting the organ contour in two dimensions or indirectly predicting the flux through the prediction and projection of the radiation field dose. The 3D organ contour can be directly input to predict the 2D flux. By adopting an end-to-end input and output method, data generation is faster, feature extraction is more comprehensive, and the robustness and reliability of the prediction are also stronger.

[0078] The intensity modulated radiotherapy flux prediction system based on the cascaded three-dimensional U-Net network provided in the embodiment of the present application can achieve Figures 1 to 4 To avoid repetition, the various processes implemented in the method embodiment are not described here.

[0079] The intensity modulated radiotherapy flux prediction system based on a cascaded three-dimensional U-Net network in the embodiment of the present application can execute the intensity modulated radiotherapy flux prediction method based on a cascaded three-dimensional U-Net network provided in the embodiment of the present application. The implementation principle is similar. The actions performed by each module and unit in the intensity modulated radiotherapy flux prediction system based on a cascaded three-dimensional U-Net network in each embodiment of the present application correspond to the steps in the intensity modulated radiotherapy flux prediction method based on a cascaded three-dimensional U-Net network in each embodiment of the present application. For the detailed functional description of each module of the intensity modulated radiotherapy flux prediction system based on a cascaded three-dimensional U-Net network, please refer to the description of the corresponding intensity modulated radiotherapy flux prediction method based on a cascaded three-dimensional U-Net network shown in the previous text, which will not be repeated here.

[0080] Based on the same principle as the method shown in the embodiment of the present application, the embodiment of the present application also provides an electronic device, which may include but is not limited to: a processor and a memory; a memory for storing a computer program; a processor for executing the intensity modulated radiotherapy flux prediction method based on a cascaded three-dimensional U-Net network shown in any optional embodiment of the present application by calling a computer program. Compared with the prior art, the intensity modulated radiotherapy flux prediction method based on a cascaded three-dimensional U-Net network provided by the present application does not need to first perform a two-dimensional projection of the organ contour or indirectly predict the flux through the prediction and projection of the radiation field dose. The 3D organ contour can be directly input to predict the 2D flux. By adopting an end-to-end input and output method, data generation is faster, feature extraction is more comprehensive, and the robustness and reliability of the prediction are also stronger.

[0081] In an optional embodiment, an electronic device is also provided, such as Figure 6 As shown, Figure 6 The electronic device 600 shown may be a server, including: a processor 601 and a memory 603. The processor 601 and the memory 603 are connected, such as through a bus 602. Optionally, the electronic device 600 may further include a transceiver 604. It should be noted that in actual applications, the transceiver 604 is not limited to one, and the structure of the electronic device 600 does not constitute a limitation on the embodiments of the present application.

[0082] Processor 601 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 601 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0083] The bus 602 may include a path to transmit information between the above components. The bus 602 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 602 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0084] The memory 603 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0085] The memory 603 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 601. The processor 601 is used to execute the application code stored in the memory 603 to implement the contents shown in the above method embodiment.

[0086] Among them, electronic devices include but are not limited to: mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0087] The server provided in this application can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected by wired or wireless communication, and this application does not limit this.

[0088] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding content in the aforementioned method embodiment.

[0089] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0090] It should be noted that the computer-readable storage medium mentioned above in the present application can also be a computer-readable signal medium or a combination of a computer-readable storage medium and a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0091] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0092] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0093] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the intensity modulated radiotherapy flux prediction method and system based on a cascaded three-dimensional U-Net network provided in the above-mentioned various optional implementations.

[0094] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0095] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0096] The modules involved in the embodiments described in the present application can be implemented by software or hardware. The name of the module does not limit the module itself in some cases. For example, the first prediction module can also be described as "a first prediction module for inputting a three-dimensional CT image and a three-dimensional contour into a first three-dimensional U-net network to extract an initial three-dimensional flux feature map".

[0097] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A method for predicting intensity modulated radiotherapy flux based on a cascaded three-dimensional U-Net network, characterized in that: The cascaded 3DU-Net network includes a first three-dimensional U-net network and a second three-dimensional U-net network, and the method includes: Inputting the 3D CT image and the 3D contour into the first 3D U-net network to extract the initial 3D flux feature map; Inputting the three-dimensional CT image, the three-dimensional contour and the initial three-dimensional flux feature map into a second three-dimensional U-net network, extracting a final three-dimensional flux feature map, and predicting and outputting a final two-dimensional flux map based on the final three-dimensional flux feature map; The second three-dimensional U-net network converts the final three-dimensional flux feature map into a final two-dimensional flux map output through an embedded matrix geometric projection operation.

2. The method for predicting intensity modulated radiotherapy flux based on a cascaded three-dimensional U-Net network according to claim 1, characterized in that: The first three-dimensional U-net network and the second three-dimensional U-net network both include an encoding path and a decoding path; The encoding path uses convolution operations and downsampling operations to extract key features and reduce the resolution of input data; The decoding path uses upsampling operations, convolution operations, and skip connections to recover information lost during downsampling of the input data.

3. The method for predicting intensity modulated radiotherapy flux based on a cascaded three-dimensional U-Net network according to claim 2, characterized in that: Instance normalization and rectified linear unit activations are used after each convolution, downsampling, and upsampling operation.

4. The method for predicting intensity modulated radiotherapy flux based on a cascaded three-dimensional U-Net network according to claim 1, characterized in that: The matrix geometry projection operation is to multiply the projection matrix and the final three-dimensional flux feature map: f=P T ·v Where f is the final two-dimensional flux map, v is the final three-dimensional flux feature map predicted by the network, T is the transposition operation, and P is the number of elements P ij The geometric projection matrix indicates that the flux voxel i receives the flux contribution from the unit intensity photon beam j.

5. The method for predicting intensity modulated radiotherapy flux based on a cascaded three-dimensional U-Net network according to claim 1, characterized in that: After inputting the three-dimensional CT image and the three-dimensional contour into the first three-dimensional U-net network and extracting the initial three-dimensional flux feature map, the method further includes: An initial two-dimensional flux map is predicted and output based on the initial three-dimensional flux characteristic map; The first three-dimensional U-net network converts the initial three-dimensional flux feature map into an initial two-dimensional flux map output through an embedded matrix geometric projection operation.

6. The method for predicting intensity modulated radiotherapy flux based on a cascaded three-dimensional U-Net network according to claim 1, characterized in that: Before inputting the three-dimensional CT image and the three-dimensional contour into the first three-dimensional U-net network and extracting the initial three-dimensional flux feature map, the method further includes: Multiple consecutive slices are randomly selected as input data, and the two-dimensional flux map of the corresponding position is extracted as the label value; Normalizing the input data, and generating a training data set based on the normalized input data and the label value; Performing data enhancement processing on the data in the training data set; the data enhancement processing includes randomly flipping in the left-right direction and randomly rotating a specified rack angle along the central axis of the cross section; The first three-dimensional U-net network and the second three-dimensional U-net network are trained according to the enhanced data.

7. The method for predicting intensity modulated radiotherapy flux based on a cascaded three-dimensional U-Net network according to claim 1, characterized in that: The loss function of the cascaded 3D U-Net network is: loss=0.5*MAE(A output ,truth)+MAE(B output ,truth) Among them, truth represents the label value of the real flux, A output ,、B output They respectively represent the prediction flux of the first three-dimensional U-net network and the second three-dimensional U-net network, and MAE represents the mean absolute error between the calculated output value and the label value.

8. An intensity modulated radiotherapy flux prediction system based on a cascaded three-dimensional U-Net network, characterized in that: The cascaded 3DU-Net network includes a first three-dimensional U-net network and a second three-dimensional U-net network, and the system includes: A first prediction module is used to input the three-dimensional CT image and the three-dimensional contour into a first three-dimensional U-net network to extract an initial three-dimensional flux feature map; A second prediction module is used to input the three-dimensional CT image, the three-dimensional contour and the initial three-dimensional flux feature map into a second three-dimensional U-net network, extract the final three-dimensional flux feature map, and predict and output a final two-dimensional flux map based on the final three-dimensional flux feature map; The second three-dimensional U-net network converts the final three-dimensional flux feature map into a final two-dimensional flux map output through an embedded matrix geometric projection operation.

9. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images