Method and system for performing privacy-based radiation therapy treatment planning

Abstract

Techniques for generating privacy-preserving segmentations are provided. These techniques include receiving a medical image; processing the medical image using a student machine learning model to estimate radiation therapy planning parameters, the student machine learning model being trained to establish relationships between a plurality of public training medical images and corresponding radiation therapy planning parameters, the radiation therapy planning parameters for the plurality of public training medical images being generated by aggregating a plurality of radiation therapy planning parameter estimates, the radiation therapy planning parameter estimates being generated by processing the plurality of public training medical images using a plurality of teacher machine learning models to generate a set of radiation therapy planning parameter estimates, and reducing the corresponding dimensionality of the set of radiation therapy planning parameter estimates or the medical images, perturbing the radiation therapy planning parameters for the plurality of public training medical images according to a privacy criterion; and generating a radiation therapy treatment plan based on the estimated radiation therapy planning parameters.

Description

[0001] Priority claim

[0002] This application claims the benefit of priority to U.S. Provisional Application No. 62 / 857,052, filed June 4, 2019, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates generally to radiation therapy or radiotherapy and to radiation therapy planning parameters with privacy assurances. Background Art

[0004] Radiation therapy is used to treat cancer and other diseases in the tissues of mammals (e.g., humans and animals). The direction and shape of the radiation beam should be accurately controlled to ensure that the tumor receives the prescribed radiation, and the beam should be arranged so as to minimize damage to surrounding healthy tissue (commonly known as organs at risk (OAR)). A treatment plan can be used to control the radiation beam parameters, and the radiation therapy device achieves treatment by delivering a spatially varying dose distribution to the patient.

[0005] Conventionally, for each patient, a radiation therapy treatment plan ("treatment plan") can be created using optimization techniques based on clinical and dosimetric goals and constraints (e.g., maximum dose, minimum dose, and mean dose to the tumor and critical organs). The treatment planning process can include using three-dimensional (3D) images of the patient to identify the target volume (e.g., tumor) and critical organs near the tumor. Creating a treatment plan can be a time-consuming process in which the planner attempts to comply with each treatment goal or constraint (e.g., dose volume histogram (DVH) goal) while considering their respective importance (e.g., weight) in order to produce a clinically acceptable treatment plan. This task can be a time-consuming trial-and-error process that is complicated by the various OARs, as the complexity of the process increases with the number of OARs (e.g., typically segmented into 21 in head and neck treatment). OARs far from the tumor can be easily shielded from radiation, while OARs close to or overlapping the target tumor can be difficult to shield from radiation.

[0006] Segmentation can be performed to identify the OARs and the area to be treated (e.g., a planning target volume (PTV)). After segmentation, a dose plan can be created for the patient that indicates the desired amount of radiation to be received by the PTV (e.g., target) and / or OARs. The PTV may have an irregular volume, and its size, shape, and position may be unique. The treatment plan can be calculated after optimizing a large number of planning parameters to ensure that an adequate dose is provided to the PTV while providing the lowest possible dose to surrounding healthy tissue. Thus, a radiation therapy treatment plan can be determined by balancing effective control of the dose to treat the tumor with sparing any OARs. In general, the quality of the radiation treatment plan may depend on the experience level of the planner. Further complications may arise from anatomical variations between patients.

[0007] Machine learning can play an important role in assisting with the creation of radiation therapy treatment plans. Most machine learning models that can be used to create radiation therapy treatment plans are trained on sensitive datasets (e.g., medical images) from other patients and hospitals. Such models fail to protect the privacy of the patients associated with the medical images used to train such machine learning models. In particular, machine learning models cannot guarantee that they will not infer whether a specific individual was part of the training set. Summary of the Invention

[0008] In some embodiments, a computer-implemented method, a non-transitory computer-readable medium, and a system including a memory and a processor are provided for performing privacy-based radiotherapy treatment planning by: receiving a medical image of a patient by a processor circuit system; processing the medical image by the processor circuit system using a student machine learning model to estimate one or more radiotherapy planning parameters, wherein the student machine learning model is trained to establish a relationship between multiple public training medical images and corresponding radiotherapy planning parameters of the public training medical images, wherein the radiotherapy planning parameters of the multiple public training medical images are generated by aggregating multiple radiotherapy planning parameter estimates, and the above-mentioned multiple radiotherapy planning parameter estimates are generated in the following manner: processing the multiple public training medical images using multiple teacher machine learning models to generate a set of radiotherapy planning parameter estimates; and reducing the corresponding dimensions of the set of radiotherapy planning parameter estimates or the multiple public training medical images, wherein the radiotherapy planning parameters of the multiple public training medical images are perturbed according to a privacy criterion; and generating a radiotherapy treatment plan for the patient by the processor circuit system based on the estimated one or more radiotherapy planning parameters of the patient's medical image.

[0009] In some implementations, the student machine learning model and multiple teacher machine learning models are implemented by corresponding neural networks or deep learning networks.

[0010] In some implementations, a computer-implemented method, a non-transitory computer-readable medium, and a system including a memory and a processor perform the following operations: training a teacher machine learning model based on private medical information including private medical images and private radiation therapy plan parameters for the private medical images, wherein a student machine learning model is trained on data that does not include the private medical information.

[0011] In some implementations, training the teacher machine learning model includes: generating disjoint datasets based on the private medical information; and training the teacher machine learning model based on corresponding datasets in the disjoint datasets.

[0012] In some implementations, a computer-implemented method, a non-transitory computer-readable medium, and a system including a memory and a processor perform the following operations: generating a first radiotherapy plan parameter estimate from a plurality of radiotherapy plan parameter estimates by processing a plurality of common training medical images using a first trained teacher machine learning model from a plurality of trained teacher machine learning models; generating a second radiotherapy plan parameter estimate from a plurality of radiotherapy plan parameter estimates by processing a plurality of common training medical images using a second trained teacher machine learning model from a plurality of trained teacher machine learning models; reducing the dimensionality of each of the first radiotherapy plan parameter estimate and the second radiotherapy plan parameter estimate from the plurality of radiotherapy plan parameter estimates to produce a first radiotherapy plan parameter estimate and a second radiotherapy plan parameter estimate of reduced dimensionality; and aggregating the first radiotherapy plan parameter estimate and the second radiotherapy plan parameter estimate of reduced dimensionality.

[0013] In some implementations, a first radiation therapy planning parameter estimate of the plurality of radiation therapy planning parameter estimates includes a first number of entries, and wherein the first reduced-dimensional radiation therapy planning parameter estimate includes a second number of entries that is less than the first number of entries.

[0014] In some implementations, aggregating the first and second radiation therapy plan parameter estimates of reduced dimensionality includes calculating a mean, a trimmed mean, a median, or a generalized f-mean of the first and second radiation therapy plan parameter estimates of reduced dimensionality.

[0015] In some implementations, aggregating the reduced-dimensionality first radiotherapy plan parameter estimates and the second radiotherapy plan parameter estimates includes estimating an aggregation of the reduced-dimensional first radiotherapy plan parameter estimates and the second radiotherapy plan parameter estimates by processing the reduced-dimensional first radiotherapy plan parameter estimates and the second radiotherapy plan parameter estimates using an aggregated machine learning model, wherein the aggregated machine learning model is trained to establish a relationship between a plurality of training individual radiotherapy plan parameter estimates and an aggregated result of the plurality of training individual radiotherapy plan parameter estimates, and to minimize an amount of perturbation required to meet a privacy standard.

[0016] In some implementations, a computer-implemented method, a non-transitory computer-readable medium, and a system including a memory and a processor perform the following operations: perturbing at least one of the first reduced dimensionality radiotherapy plan parameter estimate and the second radiotherapy plan parameter estimate or the aggregated first reduced dimensionality radiotherapy plan parameter estimate and the second radiotherapy plan parameter estimate by adding noise based on a selected privacy level to at least one of the first reduced dimensionality radiotherapy plan parameter estimate and the second radiotherapy plan parameter estimate or the aggregated first reduced dimensionality radiotherapy plan parameter estimate and the second radiotherapy plan parameter estimate.

[0017] In some implementations, adding noise includes adding samples according to a Gaussian distribution, a Beta distribution, a Dirichlet distribution, or a Laplace distribution to at least one of the first and second radiotherapy plan parameter estimates of reduced dimensionality or the aggregated first and second radiotherapy plan parameter estimates of reduced dimensionality.

[0018] In some implementations, a computer-implemented method, a non-transitory computer-readable medium, and a system including a memory and a processor perform the following operations: increasing the dimensionality of the perturbed aggregated reduced-dimensionality first and second radiation therapy planning parameter estimates to output a plurality of radiation therapy planning parameters.

[0019] In some implementations, increasing the dimensionality includes processing the perturbed aggregated reduced dimensionality first and second radiation therapy plan parameter estimates using a variational autoencoder, an autoencoder, principal component analysis, or homomorphic compression.

[0020] In some implementations, reducing the dimensionality of each of a first radiotherapy plan parameter estimate and a second radiotherapy plan parameter estimate in a plurality of radiotherapy plan parameter estimates includes processing each of the first radiotherapy plan parameter estimate and the second radiotherapy plan parameter estimate in the plurality of radiotherapy plan parameter estimates using a variational autoencoder, an autoencoder, principal component analysis, or homomorphic compression.

[0021] In some implementations, a variational autoencoder, autoencoder, or principal component analysis is trained on a common set of segmentation atlases.

[0022] In some implementations, the privacy criterion includes at least one of differential privacy, Renyi differential privacy, pooled differential privacy, mutual information, conditional entropy, Fisher information, generative adversarial privacy, or k-anonymity.

[0023] In some implementations, the medical image comprises a magnetic resonance (MR) image or a computed tomography (CT) scan, and wherein the one or more radiation therapy planning parameters comprise at least one of: a labeling of the medical image, a three-dimensional volume corresponding to the medical image, a dose distribution, a synthetic CT image, the medical image, a processed version of the medical image, or a radiation therapy device parameter.

[0024] In some embodiments, a computer-implemented method, a non-transitory computer-readable medium, and a system including a memory and a processor are provided for training a machine learning model to perform privacy-based radiotherapy treatment planning by: receiving a public training medical image of a patient by a processor circuit system; and training a student machine learning model by the processor circuit system to estimate one or more radiotherapy planning parameters of the public training medical images by establishing a relationship between multiple public training medical images and corresponding radiotherapy planning parameters of the above multiple public training medical images, wherein the radiotherapy planning parameters of the multiple public training medical images are generated by aggregating multiple radiotherapy planning parameter estimates, and the above multiple radiotherapy planning parameter estimates are generated by: processing the public training medical images using multiple teacher machine learning models to generate a set of radiotherapy planning parameter estimates; and reducing the corresponding dimensions of the set of radiotherapy planning parameter estimates or the multiple public training medical images, wherein the radiotherapy planning parameters of the multiple public training medical images are perturbed according to a privacy criterion.

[0025] In some implementations, a computer-implemented method, a non-transitory computer-readable medium, and a system including a memory and a processor perform the following operations: training a teacher machine learning model based on private medical information including private medical images and private radiation therapy plan parameters for the private medical images, wherein a student machine learning model is trained on data that does not include the private medical information.

[0026] In some implementations, training the teacher machine learning model includes: generating disjoint datasets based on the private medical information; and training the teacher machine learning model based on corresponding datasets in the disjoint datasets.

[0027] In some implementations, a computer-implemented method, a non-transitory computer-readable medium, and a system including a memory and a processor perform the following operations: generating a first radiotherapy plan parameter estimate from a plurality of radiotherapy plan parameter estimates by processing a plurality of common training medical images using a first trained teacher machine learning model from a plurality of trained teacher machine learning models; generating a second radiotherapy plan parameter estimate from a plurality of radiotherapy plan parameter estimates by processing a plurality of common training medical images using a second trained teacher machine learning model from a plurality of trained teacher machine learning models; reducing the dimensionality of each of the first radiotherapy plan parameter estimate and the second radiotherapy plan parameter estimate from the plurality of radiotherapy plan parameter estimates to produce a first radiotherapy plan parameter estimate and a second radiotherapy plan parameter estimate of reduced dimensionality; and aggregating the first radiotherapy plan parameter estimate and the second radiotherapy plan parameter estimate of reduced dimensionality.

[0028] In some implementations, training the student machine learning model includes performing the following operations for each of a plurality of public training medical images: obtaining a pair of a given public training medical image among the plurality of public training medical images and a given corresponding radiotherapy planning parameter among the corresponding radiotherapy planning parameters; applying the student machine learning model to the obtained public training medical image to generate an estimate of the radiotherapy planning parameter for the obtained public training medical image; calculating a deviation between the estimate of the radiotherapy planning parameter for the obtained public training medical image and the obtained radiotherapy planning parameter; and updating one or more parameters of the student machine learning model based on the calculated deviation. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In the accompanying drawings, which are not necessarily drawn to scale, like reference numerals describe substantially similar components throughout the several views. Like reference numerals with different letter suffixes represent different instances of substantially similar components. The accompanying drawings generally illustrate various embodiments discussed in this document by way of example and not by way of limitation.

[0030] Figure 1 An exemplary radiation therapy system suitable for performing a treatment plan generation process according to some examples is shown.

[0031] Figure 2 An exemplary image-guided radiation therapy apparatus according to some examples of the present disclosure is shown.

[0032] Figure 3 An exemplary data flow for training and using (one or more) machine learning models according to some examples of the present disclosure is shown.

[0033] Figure 4A and Figure 4B An exemplary data flow illustrating training and using (one or more) machine learning models to provide privacy according to some examples of the present disclosure is shown.

[0034] Figures 5 to 7 A flowchart illustrating exemplary operations for training and using a machine learning model in a manner that maintains data privacy, according to some examples of the present disclosure. DETAILED DESCRIPTION

[0035] The present disclosure includes various techniques for generating radiation therapy treatment plans using a student machine learning (ML) model trained on a public dataset that has been labeled by the teacher machine learning model in a manner that maintains data privacy (e.g., in a manner that meets privacy standards). Technical benefits include reduced computational processing time for generating radiation therapy treatment plans and associated improvements in processing, memory, and network resources used to generate the plans. Technical benefits also include enhanced data privacy guarantees for medical images and information, which increases the size of the training dataset used to train the ML model and, thereby, improves the accuracy and reliability of the ML model used to generate the treatment plan. In particular, due to the enhanced data privacy guarantees, hospitals and patients are more likely to participate in sharing sensitive medical information to increase the size of the training dataset. These radiation therapy treatment plans can be applied to various medical treatment and diagnostic devices or radiation therapy treatment equipment and devices. Therefore, in addition to these technical benefits, the present technology can also provide many significant medical treatment benefits (including improved accuracy of radiation therapy treatments, reduced exposure to undesirable radiation, etc.).

[0036] Existing approaches have discussed ways to provide data privacy for ML models trained to predict a single value. In particular, such existing approaches have discussed ways to train a teacher ML model on sensitive data and apply such trained teacher model to estimate the value of public data. These existing approaches have also discussed using the estimated public data to train a student model to be applied to a new dataset. Such approaches also consider increasing privacy in the process of training the teacher model, but do so on very low dimensional datasets. This makes such approaches unsuitable for radiation therapy applications, such as those involving medical image analysis. This is because radiation therapy applications operate on very highly correlated, very high dimensional datasets (e.g., a computed tomography (CT) scan volume may have 512 3 or about 10 8 Applying existing techniques in such scenarios would be computationally infeasible and inefficient. In addition to the computational resource challenges of applying existing methods to radiotherapy applications, unreasonable levels of noise would be required to provide adequate privacy guarantees due to the high correlation between high-dimensional datasets. For example, if the labels of nearby highly correlated voxels are treated as a set of independent classification tasks, then according to existing methods, an unreasonable amount of noise would need to be added to each medical image voxel, making it computationally infeasible; if it is possible at all, unreasonably large storage resources for the training data would be required to train a useful segmentation ML model.

[0037] The disclosed technology addresses these challenges by encoding (or reducing the dimensionality of) radiotherapy application data using a dimensionality adjustment ML model (e.g., a variational autoencoder (VAE)), where the data has been labeled by a teacher model before being perturbed according to privacy criteria. This allows for the introduction of low noise levels in a computationally efficient manner. Subsequently, the dimensionality adjustment ML model is applied to the perturbed data to decode (increase) the dimensionality, thereby restoring the size of the radiotherapy application data for training a student model. Specifically, the disclosed technology includes receiving a medical image and processing the medical image using a student machine learning model to estimate radiotherapy planning parameters. The student machine learning model is trained to establish a relationship between a plurality of public training medical images and corresponding radiotherapy planning parameters. The radiotherapy planning parameters for the plurality of public training medical images are generated by aggregating a plurality of radiotherapy planning parameter estimates generated by: processing the plurality of public training medical images using a plurality of teacher machine learning models to generate a set of radiotherapy planning parameter estimates; and reducing the corresponding dimensionality of the set of radiotherapy planning parameter estimates, wherein the radiotherapy planning parameters for the plurality of public training medical images are perturbed according to privacy criteria. The disclosed technique generates a radiation therapy treatment plan based on estimated radiation therapy planning parameters provided by a student model.

[0038] According to some embodiments, a VAE is used to compress segmentation maps predicted by teacher models (e.g., teacher ML models that each estimate segmentation labels for CT images, magnetic resonance (MR) images, and PET images) to obtain a low-dimensional feature vector for each teacher model. Specifically, the VAE can learn highly dense nonlinear compression, and the decompression of the VAE is robust to noise perturbations of the feature vector, which allows such an approach to data privacy to be adopted.

[0039] In a specific example, during training, sensitive (private) data is divided into disjoint data subsets, and a separate teacher segmentation ML model is trained on each subset. After training the teacher ML model on the sensitive data for each teacher segmentation ML model, the teacher segmentation ML model is applied to an unlabeled public dataset. The output or predictions for the unlabeled public dataset are collected. A VAE is trained on a separate non-sensitive segmentation atlas, and the teacher segmentation ML predictions for the unlabeled public dataset are compressed using the trained VAE. For each data point, the compressed feature vectors are aggregated across the teacher segmentation ML models using an average and / or a learned function (e.g., an aggregation ML model). The aggregated results are perturbed according to a privacy standard (e.g., using differential privacy techniques) to add noise from a normal (Gaussian) distribution that is appropriately scaled to the desired privacy level. The perturbed predictions are decompressed using the VAE, and a student segmentation ML model is trained on the public dataset that has been labeled with the decompressed predictions. In an embodiment, the student ML model and the teacher ML model are implemented using separate implementations of the same ML architecture and process.

[0040] The teacher ML model and the student ML model can be trained to estimate any one or more radiotherapy planning parameters. Specifically, in some embodiments, the teacher ML model and the student ML model are trained to estimate segmentation as a radiotherapy planning parameter, and are specifically trained to segment radiotherapy medical images such as CT images, MR images, and / or sCT images. As another example, in some embodiments, the teacher ML model and the student ML model are trained to estimate a three-dimensional (3D) model as a radiotherapy planning parameter, and are specifically trained to estimate a 3D model of radiotherapy medical images such as CT images, MR images, and / or sCT images. As another example, in some embodiments, the teacher ML model and the student ML model are trained to estimate a dose distribution as a radiotherapy planning parameter, and are specifically trained to estimate the dose distribution based on one or more radiotherapy images. As another example, in some embodiments, the teacher ML model and the student ML model are trained to generate or estimate an sCT image as a radiotherapy planning parameter, and are specifically trained to estimate an sCT image based on a CT or MR image. As another example, in some embodiments, the teacher ML model and the student ML model are trained to estimate radiotherapy device parameters (e.g., control points) as radiotherapy planning parameters, and are specifically trained to estimate control points based on one or more radiotherapy images and / or a distance map specifying (possibly signed) distances to a region of interest.

[0041] Figure 1 An exemplary radiation therapy system 100 is shown that is suitable for performing radiation therapy planning processing operations using one or more of the methods discussed herein. These radiation therapy planning processing operations are performed to enable the radiation therapy system to provide radiation therapy to a patient based on captured medical imaging data and treatment dose calculations or specific aspects of radiation therapy machine configuration parameters. Specifically, the following processing operations can be implemented as part of the treatment processing logic 120. However, it should be understood that many variations and use cases of the following trained models and treatment processing logic 120 can be provided, including in data validation, visualization, and other medical evaluation and diagnostic settings.

[0042] The radiation therapy system 100 includes a radiation therapy processing computing system 110 that hosts treatment processing logic 120. The radiation therapy processing computing system 110 can be connected to a network (not shown), and such a network can be connected to the Internet. For example, the network can connect the radiation therapy processing computing system 110 with one or more private and / or public medical information sources (e.g., a radiation information system (RIS), a medical record system (e.g., an electronic medical record (EMR) / electronic health record (EHR) system), an oncology information system (OIS)), one or more image data sources 150, an image acquisition device 170 (e.g., an imaging modality), a treatment device 180 (e.g., a radiation therapy device), and a treatment data source 160. As an example, the radiation therapy processing computing system 110 can be configured to perform the following operations as part of generating a treatment plan to be used by the treatment device 180 and / or output on the device 146: receive a treatment target for a subject (e.g., from one or more MR images), and generate a radiation therapy treatment plan by executing instructions or data from the treatment processing logic 120. In an embodiment, the treatment processing logic includes a student ML model that has been trained on public medical information to estimate one or more radiation therapy parameters. The public information used to train the student ML model is generated by multiple teacher ML models trained on sensitive or private medical information. The teacher ML model generates public information (e.g., segmentation or labeling of CT images), which is then compressed, aggregated, perturbed according to privacy criteria, and decompressed before being used for training the student ML model. In this way, the student ML model can be trained on data that meets privacy criteria (e.g., segmentation or labeling of CT images), which enhances and ensures data privacy and does not reveal the identity of any individual patient or hospital.

[0043] The radiation therapy treatment computing system 110 may include a processing circuit system 112, a memory 114, a storage device 116, and other hardware and software operable features such as a user interface 142, a communication interface (not shown), etc. The storage device 116 may store transient computer-executable instructions or non-transient computer-executable instructions, such as an operating system, radiation therapy treatment plans, training data, software programs (e.g., image processing software, image or anatomical visualization software, artificial intelligence (AI) or ML implementations and algorithms such as those provided by deep learning models, ML models, and neural networks (NNs), etc.), and any other computer-executable instructions to be executed by the processing circuit system 112.

[0044] In an example, processing circuit system 112 may include a processing device, for example, one or more general-purpose processing devices such as a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), etc. More specifically, processing circuit system 112 may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor that implements other instruction sets, or a processor that implements a combination of instruction sets. Processing circuit system 112 may also be implemented by one or more special-purpose processing devices such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), a system on a chip (SoC), etc.

[0045] As will be appreciated by those skilled in the art, in some examples, processing circuitry 112 may be a dedicated processor rather than a general purpose processor. Processing circuitry 112 may include one or more known processing devices, such as those from Intel. TM Pentium manufactured TM 、Core TM 、Xeon TM or series of microprocessors and those from AMD TM Turion TM , Athlon TM 、Sempron TM , Opteron TM FX TM 、Phenom TM series of microprocessors or any of the various processors manufactured by Sun Microsystems. Processing circuit system 112 may also include processors such as those from Nvidia TM manufactured series and by Intel TM GMA, Iris TM series or by AMD TM Radeon TM The processing circuit system 112 may also include a graphics processing unit such as a graphics processing unit of the Intel series GPU. TM Xeon Phi manufactured TMseries of accelerated processing units. The disclosed embodiments are not limited to any type of processor that is otherwise configured to meet the computational needs of identifying, analyzing, maintaining, generating and / or providing large amounts of data or manipulating such data to perform the methods disclosed herein. In addition, the term "processor" may include more than one physical (circuitry-based) processor or software-based processor (e.g., a multi-core design or multiple processors each having a multi-core design). The processing circuit system 112 can execute a sequence of transient or non-transient computer program instructions stored in the memory 114 and accessed from the storage device 116 to perform various operations, processes and methods that will be described in more detail below. It should be understood that any component in the system 100 can be implemented separately and operate as a stand-alone device, and can be coupled to any other component in the system 100 to perform the techniques described in the present disclosure.

[0046] Memory 114 may include read-only memory (ROM), phase-change random access memory (PRAM), static random access memory (SRAM), flash memory, random access memory (RAM), dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM), electrically erasable programmable read-only memory (EEPROM), static memory (e.g., flash memory, flash disk, static random access memory), and other types of random access memory, cache memory, registers, compact disc read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage device, magnetic cassette, other magnetic storage device, or any other non-transitory medium that can be used to store information that can be accessed by processing circuitry 112 or any other type of computer device, including images, training data, one or more ML model or technique parameters, data, or transient or non-transitory computer-executable instructions (e.g., stored in any format). For example, computer program instructions can be accessed by processing circuitry 112, can be read from ROM or any other suitable memory location, and can be loaded into RAM for execution by processing circuitry 112.

[0047] The storage device 116 may constitute a drive unit including a transitory or non-transitory machine-readable medium having stored thereon one or more transitory or non-transitory instruction sets and data structures (e.g., software) that implement or are utilized by any one or more of the methodologies or functionalities described herein (including, in various examples, the treatment processing logic 120 and the user interface 142). During execution of the instructions by the radiation therapy treatment computing system 110, the instructions may also reside, in whole or in part, within the memory 114 and / or the processing circuitry 112, with the memory 114 and the processing circuitry 112 also constituting transitory or non-transitory machine-readable media.

[0048] The memory 114 and the storage device 116 may constitute non-transitory computer-readable media. For example, the memory 114 and the storage device 116 may store or load transient or non-transitory instructions for one or more software applications on a computer-readable medium. The software applications stored or loaded using the memory 114 and the storage device 116 may include, for example, operating systems for general-purpose computer systems and devices controlled by software. The radiation therapy treatment computing system 110 may also operate various software programs including software code for implementing the treatment treatment logic 120 and the user interface 142. In addition, the memory 114 and the storage device 116 may store or load an entire software application, a portion of a software application, or code or data associated with the software application that can be executed by the processing circuit system 112. In another example, the memory 114 and the storage device 116 can store, load, and manipulate one or more radiation therapy treatment plans, imaging data, segmentation data, treatment visualizations, histograms or measurements, one or more AI model data (e.g., weights and parameters of a teacher ML model, a student ML model, an aggregated ML model, and / or a dimensionality adjustment model), training data, labeling and mapping data, etc. It is contemplated that software programs can be stored not only on the storage device 116 and the memory 114, but also on removable computer media such as a hard drive, a computer disk, a CD-ROM, a DVD, a Blu-ray DVD, a USB flash drive, an SD card, a memory stick, or any other suitable media; such software programs can also be transmitted or received over a network.

[0049] Although not shown, the radiation therapy treatment computing system 110 may include a communication interface, a network interface card, and communication circuitry. Example communication interfaces may include, for example, a network adapter, a cable connector, a serial connector, a USB connector, a parallel connector, a high-speed data transfer adapter (e.g., such as fiber optic, USB 3.0, Thunderbolt, etc.), a wireless network adapter (e.g., such as an IEEE 802.11 / Wi-Fi adapter), a telecommunications adapter (e.g., for communicating with 3G, 4G / LTE, and 5G networks, etc.), etc. Such communication interfaces may include one or more digital and / or analog communication devices that allow the machine to communicate with other machines and devices, such as remotely located components, via a network. The network may provide the functionality of a local area network (LAN), a wireless network, a cloud computing environment (e.g., software as a service, platform as a service, infrastructure as a service, etc.), a client-server, a wide area network (WAN), etc. For example, the network may be a LAN or WAN that may include other systems, including additional image processing computing systems or image-based components associated with medical imaging or radiation therapy operations.

[0050] In an example, the radiation therapy processing computing system 110 may obtain image data 152 from an image data source 150 (e.g., an MR image) for hosting on the storage device 116 and the memory 114. In yet another example, the software program may substitute a function of the patient image, such as a signed distance function or an image processed version that emphasizes certain aspects of the image information.

[0051] In an example, the radiation therapy treatment computing system 110 may obtain image data 152 from or transmit image data 152 to the image data source 150. In another example, the treatment data source 160 receives or updates planning data as a result of a treatment plan generated by the treatment processing logic 120. The image data source 150 may also provide or host imaging data for use in the treatment processing logic 120.

[0052] As referred to herein, "public" or "non-sensitive" data includes data sets that are publicly available via one or more public databases and do not contain any data subject to data privacy regulations (e.g., the data does not identify a specific individual or individuals who have given explicit or implicit consent). "Private" or "sensitive" data includes data sets that contain private hospital and / or patient medical information and are subject to data privacy regulations. In some cases, private data is not publicly available but is available to certain organizations and entities on a limited-access basis without privacy concerns. Ideally, such data should not be shared outside the organization without the patient's or hospital's explicit permission and should be maintained in a secure, non-publicly accessible database. In embodiments, the parameters of the teacher ML model are maintained as private and sensitive data because such parameters, if accessed by an adversary, could reveal the identity of the individual. Data can be received from external sources via queries and applied to the private teacher ML model to generate a result output. According to disclosed embodiments, the output or results of the teacher ML model, subject to privacy standards, can be publicly accessed by, for example, a student ML model in response to a query. The parameters of the student ML model can be publicly accessible.

[0053] In an example, the computing system 110 can communicate with the treatment data source 160 and the input device 148 to generate private pairs of prior patient radiation therapy treatment information, such as pairs of labeled or segmented radiation therapy medical images (e.g., CT, MR, and / or sCT images); pairs of 3D models and one or more corresponding radiation therapy medical images; pairs of dose distributions and one or more radiation therapy images; pairs of sCT images and CT images; pairs of control points and one or more radiation therapy images and / or distance maps; and pairs of individual radiation therapy plan parameter estimates and aggregated results of multiple training individual radiation therapy plan parameter estimates. In an example, the computing system 110 can communicate with the treatment data source 160 and the input device 148 to generate pairs of public prior patient radiation therapy treatment information, such as pairs of a segmentation map and a compressed dimensional segmentation map; and pairs of a compressed dimensional segmentation map and an uncompressed segmentation map. The computing system 110 can continue to generate such pairs of training data until a threshold number of pairs is obtained.

[0054] The processing circuitry 112 can be communicatively coupled to the memory 114 and the storage device 116, and the processing circuitry 112 can be configured to execute computer-executable instructions stored thereon from the memory 114 or the storage device 116. The processing circuitry 112 can execute instructions for causing a medical image from the image data 152 received or obtained in the memory 114 to be processed using the treatment processing logic 120. In particular, the treatment processing logic 120 implements a trained student ML model that is applied to the medical image to generate one or more radiation therapy parameters for a treatment plan. In an example, the student ML model segments a radiation therapy medical image, such as a CT image, an MR image, and / or an sCT image. As another example, in some embodiments, the student ML model estimates a 3D model of a radiation therapy medical image, such as a CT image, an MR image, and / or an sCT image. As another example, in some embodiments, the student ML model estimates a dose distribution based on one or more radiation therapy images. As another example, in some embodiments, the student ML model estimates an sCT image based on a CT image. As another example, in some embodiments, the student ML model estimates control points of a radiation therapy treatment device based on one or more radiation therapy images and / or MR scan range maps.

[0055] In addition, the processing circuit system 112 can utilize a software program to generate intermediate data, such as, for example, updated parameters to be used by the NN model, the machine learning model, the treatment processing logic 120, or other aspects involved in generating a treatment plan as discussed herein. In addition, such a software program can utilize the treatment processing logic 120, using the techniques further discussed herein, to generate new or updated treatment plan parameters for deployment to the treatment data source 160 and / or presentation on the output device 146. The processing circuit system 112 can then transmit the new or updated achievable treatment plan parameters to the treatment device 180 via the communication interface and network, wherein the radiation treatment plan will be used via the treatment device 180 to utilize the results of the trained student ML model implemented by the treatment processing logic 120 (e.g., according to the following in conjunction with Figure 3 The patient is treated with radiation consistent with the process discussed in Figure 4).

[0056] In the examples herein, the processing circuitry 112 may execute a software program that calls the treatment processing logic 120 to implement ML, deep learning, NN, and other aspects of artificial intelligence functionality for generating a treatment plan based on input radiation therapy medical information (e.g., CT images, MR images, and / or sCT images and / or dose information). For example, the processing circuitry 112 may execute a software program that trains, analyzes, predicts, evaluates, and generates treatment plan parameters based on the received radiation therapy medical information as discussed herein.

[0057] In an example, the image data 152 may include one or more MRI images (e.g., 2D MRI, 3D MRI, 2D flow MRI, 4D MRI, 4D volume MRI, 4D imaging MRI, etc.), functional MRI images (e.g., fMRI, DCE-MRI, diffusion MRI), computed tomography (CT) images (e.g., 2D CT, 2D cone-beam CT, 3D CT, 3D CBCT, 4DCT, 4DCBCT), ultrasound images (e.g., 2D ultrasound, 3D ultrasound, 4D ultrasound), positron emission tomography (PET) images, X-ray images, fluoroscopic images, radiotherapy portal images, single photon emission computed tomography (SPECT) images, computer-generated synthetic images (e.g., pseudo-CT images), etc. In addition, the image data 152 may also include or be associated with medical image processing data (e.g., training images, ground truth images, contour images, and dose images). In other examples, equivalent representations of the anatomical region may be represented in a non-image format (eg, coordinates, mapping, etc.).

[0058] In an example, image data 152 may be received from an image acquisition device 170 and stored in one or more of an image data source 150 (e.g., a picture archiving and communication system (PACS), a vendor neutral archive (VNA), a medical record or information system, a data warehouse, etc.). Thus, the image acquisition device 170 may include an MRI imaging device, a CT imaging device, a PET imaging device, an ultrasound imaging device, a fluoroscope device, a SPECT imaging device, an integrated linear accelerator and MRI imaging device, a CBCT imaging device, or other medical imaging device for acquiring medical images of a patient. The image data 152 may be received and stored in any data type or format type (e.g., in Digital Imaging and Communications in Medicine (DICOM) format) that the image acquisition device 170 and the radiation therapy processing computing system 110 can use to perform operations consistent with the disclosed embodiments. Furthermore, in some examples, the models discussed herein may be trained to process the original image data format or a derivative thereof.

[0059] In an example, the image acquisition device 170 can be integrated with the treatment device 180 into a single device (e.g., an MRI device combined with a linear accelerator, also referred to as an "MRI-Linac"). Such an MRI-Linac can be used, for example, to determine the location of a target organ or target tumor within a patient's body, thereby accurately directing radiation therapy to a predetermined target according to a radiation therapy treatment plan. For example, a radiation therapy treatment plan can provide information about a specific radiation dose to be applied to each patient. The radiation therapy treatment plan can also include other radiation therapy information, including control points of the radiation therapy treatment device, such as bed position, beam intensity, beam angle, dose-histogram-volume information, the number of radiation beams to be used during treatment, the dose of each beam, etc.

[0060] The radiation therapy processing computing system 110 can communicate with an external database over a network to send / receive a plurality of various types of data related to image processing and radiation therapy operations. For example, the external database may include machine data (including device constraints) that provides information associated with the treatment device 180, the image acquisition device 170, or other machines related to radiation therapy or medical procedures. Machine data information (e.g., control points) may include radiation beam size, arc placement, beam on and off durations, machine parameters, segments, multi-leaf collimator (MLC) configurations, gantry speeds, MRI pulse sequences, etc. The external database may be a storage device and may be equipped with an appropriate database management software program. In addition, such a database or data source may include multiple devices or systems located in a central or distributed manner.

[0061] The radiation therapy treatment computing system 110 can collect and acquire data via a network and communicate with other systems using one or more communication interfaces that are communicatively coupled to the processing circuitry 112 and the memory 114. For example, the communication interface can provide a communication connection between the radiation therapy treatment computing system 110 and components of the radiation therapy system (e.g., allowing data to be exchanged with external devices). For example, in some examples, the communication interface can have appropriate interface circuitry based on the output device 146 or the input device 148 to connect to the user interface 142, which can be a hardware keyboard, keypad, or touch screen through which a user can enter information into the radiation therapy system.

[0062] As an example, the output device 146 may include a display device that outputs: a representation of the user interface 142; and one or more aspects, visualizations, or representations of medical images, treatment plans, and the status of training, generation, validation, or implementation of such plans. The output device 146 may include one or more display screens that display medical images, interface information, treatment plan parameters (e.g., contours, doses, beam angles, markers, maps, etc.), treatment plans, targets, target positioning and / or target tracking, or any related information to a user. The input device 148 connected to the user interface 142 may be a keyboard, keypad, touch screen, or any type of device that a user can use to input information to the radiation therapy system 100. Alternatively, the output device 146, input device 148, and features of the user interface 142 may be integrated into a device such as a smart phone or tablet computer (e.g., Apple Lenovo Samsung etc.) in a single device.

[0063] Furthermore, any and all components of the radiation therapy system 100 may be implemented as virtual machines (e.g., via a virtualization platform such as VMWare, Hyper-V, or the like) or as standalone devices. For example, a virtual machine may be software that acts as hardware. Thus, a virtual machine may include at least one or more virtual processors, one or more virtual memories, and one or more virtual communication interfaces that together act as hardware. For example, the radiation therapy processing computing system 110, the image data source 150, or similar components may be implemented as virtual machines or in a cloud-based virtualization environment.

[0064] The image acquisition device 170 can be configured to acquire one or more images of the patient's anatomical structure for a region of interest (e.g., a target organ, a target tumor, or both). Each image (typically a 2D image or slice) can include one or more parameters (e.g., 2D slice thickness, orientation, and position, etc.). In an example, the image acquisition device 170 can acquire 2D slices of any orientation. For example, the orientation of the 2D slice can include a sagittal orientation, a coronal orientation, or an axial orientation. The processing circuit system 112 can adjust one or more parameters, such as the thickness and / or orientation of the 2D slice, to include the target organ and / or target tumor. In an example, the 2D slice can be determined based on information such as a 3D CBCT or CT or MRI volume. Such 2D slices can be acquired by the image acquisition device 170 in "near real time" while the patient is undergoing radiation therapy treatment (e.g., when using the treatment device 180 (where "near real time" means acquiring data in at least a few milliseconds or less)).

[0065] The treatment processing logic 120 in the radiation therapy treatment computing system 110 implements a student ML model, which involves using a trained (learned) student ML model. The ML model can be provided by a NN that is trained as part of the NN model. One or more teacher ML models can be provided by different entities or at an off-site facility associated with the treatment processing logic 120 and can be accessed by issuing one or more queries to the off-site facility. The teacher ML model can include architecture and processes that complement the architecture and processes of the student ML model. The teacher ML models are each implemented using the same or a common set of private ML parameters, while the student ML model is implemented using a public set of ML parameters. The discussion about ML models applies equally to any ML model discussed herein.

[0066] Machine learning (ML) algorithms or ML models or techniques can be summarized as function approximations. Training data containing some type of input-output pairs (e.g., CT images with segmentation) is obtained from, for example, expert clinicians, and a function is "trained" to approximate this mapping. Some methods involve NNs. In these methods, a parameterized function A is chosen. θ where θ is a set of parameters (e.g., convolution kernels and biases) chosen by minimizing the average error on the training data. If the input-output pair is given by (x m ,y m ), then the function can be formalized by solving a minimization problem such as Equation 1:

[0067]

[0068] Once the network has been trained (e.g., θ has been chosen), the function A θ can be applied to any new input. For example, in the above CT image segmentation setting, a CT image that has never been seen before can be fed to A θ , and estimate a segmentation that matches what an expert clinician would find. In some cases, an autoencoder can be trained as an unsupervised model by attempting to reconstruct the input (e.g., by setting y = x in Eq. 1).

[0069] A simple NN comprises an input layer, an intermediate layer or hidden layer, and an output layer, each of which contains a computational unit or node. The hidden layer nodes have inputs from all input layer nodes and are connected to all nodes in the output layer. Such a network is called "fully connected." Each node transmits a signal to the output node based on a nonlinear function of the sum of its inputs. For a classifier, the number of input layer nodes is generally equal to the number of features of each object in the set of objects classified into each class, and the number of output layer nodes is equal to the number of classes. The network is trained by presenting the features of objects of known classes to the network and adjusting the node weights to reduce the training error through an algorithm called backpropagation. Therefore, the trained network can classify new objects whose classes are unknown.

[0070] Neural networks have the ability to discover the relationship between data and classes or regression values, and under certain conditions, can simulate any function y = f (x), including nonlinear functions. In ML, it is assumed that both training data and test data are generated by the same data generation process p. data Generate, where each {x i ,y i The samples are identically and independently distributed (iid). In ML, the goal is to minimize the training error and to minimize the difference between the training error and the test error. If the training error is too large, underfitting occurs; when the training-test error gap is too large, overfitting occurs. Both types of performance flaws are related to the model capacity: a large capacity can fit the training data well but lead to overfitting, while a small capacity can lead to underfitting.

[0071] Figure 2 An exemplary image-guided radiation therapy apparatus 202 is shown, comprising a radiation source, such as an x-ray source or a linear accelerator, a couch 216, an imaging detector 214, and a radiation therapy output 204. The radiation therapy apparatus 202 can be configured to emit a radiation therapy beam 208 to provide treatment to a patient. The radiation therapy output 204 can include one or more attenuators or collimators, such as an MLC.

[0072] As an example, a patient can be positioned in area 212, supported by a treatment couch 216, to receive a radiation therapy dose according to a radiation therapy treatment plan. The radiation therapy output 204 can be mounted or attached to a gantry 206 or other mechanical support. When the couch 216 is inserted into the treatment area, one or more chassis motors (not shown) can rotate the gantry 206 and the radiation therapy output 204 about the couch 216. In an example, the gantry 206 can continuously rotate about the couch 216 when the couch 216 is inserted into the treatment area. In another example, the gantry 206 can rotate to a predetermined position when the couch 216 is inserted into the treatment area. For example, the gantry 206 can be configured to rotate the therapy output 204 about an axis ("A"). Both the couch 216 and the radiation therapy output 204 can be independently moved to other positions around the patient, for example, in a transverse direction ("T"), in a lateral direction ("L"), or rotated about one or more other axes, such as a transverse axis (denoted as "R"). A controller communicatively coupled to one or more actuators (not shown) can control the movement or rotation of the couch 216 in accordance with the radiation therapy treatment plan to appropriately position the patient in or out of the radiation therapy beam 208. Both the couch 216 and the gantry 204 can be moved independently of each other in multiple degrees of freedom, which allows the patient to be positioned so that the radiation therapy beam 208 can be precisely targeted at the tumor.

[0073] A coordinate system (including axes A, T, and L) can have an origin located at an isocenter 210. Isocenter 210 can be defined as the location where the central axis of the radiation therapy beam 208 intersects the origin of the coordinate axes, e.g., to deliver a prescribed radiation dose to or within a patient. Alternatively, isocenter 210 can be defined as the location where the central axis of the radiation therapy beam 208 intersects the patient for various rotational positions of the radiation therapy output 204 about axis A, as positioned by the gantry 206.

[0074] The gantry 206 may also have an attached imaging detector 214. The imaging detector 214 is preferably located opposite the radiation source (output 204), and in an example, the imaging detector 214 may be located within the field of the therapy radiation beam 208. The imaging detector 214 may preferably be mounted on the gantry 206 opposite the radiation therapy output 204 so as to remain aligned with the radiation therapy beam 208. As the gantry 206 rotates, the imaging detector 214 rotates about the axis of rotation. In an example, the imaging detector 214 may be a flat panel detector (e.g., a direct detector or a scintillator detector). In this way, the imaging detector 214 may be used to monitor the radiation therapy beam 208, or the imaging detector 214 may be used to image the patient's anatomy, such as portal imaging. The control circuitry of the radiation therapy device 202 may be integrated within the radiation therapy system 100 or remote from the radiation therapy system 100.

[0075] In the illustrative example, one or more of the couch 216, therapy output 204, or gantry 206 can be automatically positioned, and the therapy output 204 can create a therapy radiation beam 208 according to a specified dose for a particular therapy delivery instance. A sequence of therapy deliveries can be specified according to a radiation therapy treatment plan, for example, using one or more different orientations or positions of the gantry 206, couch 216, or therapy output 204. The therapy deliveries can occur sequentially but can intersect at a desired therapy site on or within the patient, for example, at the isocenter 210. Thus, a prescribed cumulative dose of radiation therapy can be delivered to the therapy site while reducing or avoiding damage to tissue near the therapy site.

[0076] therefore, Figure 2 Specifically shown is an example of a radiotherapy device 202 that is operable to provide radiotherapy treatment to a patient that is consistent with or in accordance with a radiotherapy treatment plan, the radiotherapy device 202 having a configuration in which a radiotherapy output can rotate about a central axis (e.g., axis "A"). Other radiotherapy output configurations can be used. For example, the radiotherapy output can be mounted to a robotic arm or manipulator having multiple degrees of freedom. In yet another example, the therapy output can be fixed, such as positioned in an area laterally separated from the patient, and a platform supporting the patient can be used to align the radiotherapy isocenter with a designated target site in the patient's body. In another example, the radiotherapy device can be a combination of a linear accelerator and an image acquisition device. As will be appreciated by one of ordinary skill in the art, in some examples, the image acquisition device can be an MRI, X-ray, CT, CBCT, spiral CT, PET, SPECT, optical tomography, fluorescence imaging, ultrasound imaging, or a radiotherapy portal imaging device, among others.

[0077] As described above, the training data used by the treatment processing logic 120 may include a plurality of previously estimated private or public treatment plan parameters paired with previously private or public patient images stored in the memory 114. For example, the stored training data may include diagnostic images, treatment images (dose maps), segmentation information, etc. associated with one or more previously estimated treatment plans. The training data may include a plurality of training samples. Each training sample may include a feature vector and a corresponding output vector.

[0078] The feature vector may include one or more feature elements. Each feature element may indicate an observation of a medical image used in a past radiotherapy session (e.g., provided by the image acquisition device 140). The observation may be the distance between a volume (e.g., a voxel) and an anatomical region (such as a target or surface of a body part in a medical image). In another example, the observation may include the spatial coordinates of the anatomical region or the probability that the anatomical region includes a specific tissue type. In another example, the feature elements may include patient-specific information, the attending physician, organ or volume segmentation data of interest, functional organ modeling data (e.g., serial versus parallel organs, and appropriate dose response models), radiation dose (e.g., also including DVH information), laboratory data (e.g., hemoglobin, platelets, cholesterol, triglycerides, creatinine, sodium, glucose, calcium, weight), vital signs (blood pressure, temperature, respiratory rate, etc.), genomic data (e.g., genetic map), demographic characteristics (age, sex), other diseases affecting the patient (e.g., cardiovascular or respiratory diseases, diabetes, radiation hypersensitivity syndrome, etc.), medications and drug reactions, diet and lifestyle (e.g., smoking or non-smoking), environmental hazards Factors, tumor characteristics (histological type, tumor grade, hormone and other receptor status, tumor size, vascular cell type, cancer stage, Gleason score), previous treatment (e.g., surgery, radiation, chemotherapy, hormone therapy), lymph node and distant metastasis status, genetic / protein biomarkers (e.g., such as MYC, GADD45A, PPM1D, BBC3, CDKN1A, PLK3, XPC, AKT1, RELA, BCL2L1, PTEN, CDK1, XIAP, etc.), single nucleotide polymorphism (SNP) analysis (e.g., XRCC1, XRCC3, APEX1, MDM2, TNFR, MTHFR, MTRR, VEGF, TGFβ, TNFα), etc. A feature vector may include one or more such feature elements, regardless of whether these feature elements are related to each other.

[0079] The output vector may include one or more output elements. Each output element may indicate a corresponding estimated plan outcome or parameter in a past radiotherapy session based on observations included in the feature vector. For example, an output element may include an estimated dose applied or received at a particular spatial location (e.g., a voxel). In another example, an output element may include a patient survival time based on observations such as treatment type, treatment parameters, patient history, and / or patient anatomy. Additional examples of output elements include, but are not limited to, a normal tissue complication probability (NTCP), a probability of zone displacement during treatment, or a probability that a set of coordinates in a reference image is mapped to another set of coordinates in a target image. The output vector may include one or more such output elements, regardless of whether these output elements are related to each other.

[0080] As an example of an embodiment, an output element may include a dose to be applied to a voxel of a particular OAR. In addition, a feature element may be used to determine the output element. The feature element may include the distance between a voxel in the OAR and the nearest boundary voxel in the target tumor. Thus, the feature element may include a signed distance x that indicates the distance between a voxel in the OAR and the nearest boundary voxel in the target for radiation therapy. The output element may include the dose D in the voxel of the OAR from which x was measured. In some other embodiments, each training sample may correspond to a specific voxel in the target or OAR, such that multiple training samples within the training data correspond to the entire volume of the target or OAR and other anatomical parts undergoing radiation therapy treatment.

[0081] Figure 3 An exemplary data flow for training and using one or more private and public machine learning models to generate radiation therapy treatment plans according to some examples of the present disclosure is shown. The data flow includes training input 310, ML model (technique) training 330, and model use 350.

[0082] The training input 310 includes model parameters 312 and training data 320, which may include paired training data sets 322 (e.g., input-output training pairs) and constraints 326. The model parameters 312 store or provide the machine learning model The model parameters 312 may include the private parameters of the teacher ML model and the aggregate model and the public parameters of the student model and the dimension adjustment model. The model parameters 312 may be shared and Even if the teacher ML model and the student ML model have the same architecture and process, the student ML model The model parameters 312 of the teacher ML model may also be different from the model parameters of the teacher ML model. During training, these parameters 312 are adapted based on the input-output training pairs of the training data set 322. After the parameters 312 are adapted (after training), the parameters are used by the trained model 360 to implement the trained machine learning model on the new data set 370. A corresponding machine learning model in Trained teacher model Trained aggregation model and / or trained dimensionality resizing models ).

[0083] The training data 320 includes constraints 326 that can define the physical constraints of a given radiotherapy device. The training data set 322 can include a collection of private input-output pairs and public input-output pairs, such as private pairs of previous patient radiotherapy treatment information, such as pairs of labeled or segmented radiotherapy medical images (e.g., CT, MR, and / or sCT images); pairs of 3D models and one or more corresponding radiotherapy medical images; pairs of dose distributions and one or more radiotherapy images; pairs of sCT images and CT images; pairs of control points and one or more radiotherapy images and / or distance maps; and pairs of individual radiotherapy plan parameter estimates and aggregated results of multiple training individual radiotherapy plan parameter estimates; pairs of public previous patient radiotherapy treatment information, such as pairs of segmentation maps and compressed dimensional segmentation maps; and pairs of compressed dimensional segmentation maps and uncompressed segmentation maps. Some components of the training input 310 can be stored at one or more off-site facilities that are different from other components. For example, a pair of private parameters and private training data may contain sensitive or private information and should be restricted to be accessed or queried only by authorized parties.

[0084] Machine learning model training 330 is based on a set of private and public input-output pairs of the training dataset 322 to train one or more machine learning techniques. Training is performed. For example, model training 330 can train student ML model parameters 312 by minimizing a first loss function based on public training patient input data and corresponding radiation therapy plan parameters generated using the teacher ML model and meeting privacy criteria. For example, treatment model training 330 can train teacher ML model parameters 312 by minimizing a second loss function based on private training patient input data and corresponding private radiation therapy plan parameters. For example, treatment model training 330 can train aggregate ML model parameters 312 by minimizing a third loss function based on individual radiation therapy plan parameter estimates and an aggregate result of multiple training individual radiation therapy plan parameter estimates and an amount of perturbation required to meet privacy criteria. For example, treatment model training 330 can train dimensioning ML model parameters 312 (e.g., VAE encoder and decoder, autoencoder, and / or principal component analysis) by minimizing a fourth loss function based on the public segmentation map. In some embodiments, the teacher ML model, aggregate ML model, and dimensioning ML model are trained in parallel or sequentially before training the student ML model.

[0085] The results of minimizing these loss functions for multiple training data sets are used to train, adapt or optimize the model parameters 312 of the corresponding ML model. Figure 4A and Figure 4B In this way, the student ML model is trained to establish the relationship between the unlabeled public radiotherapy medical information and the estimated radiotherapy plan parameters provided and personalized by the teacher ML model.

[0086] In some embodiments, when a machine learning model After each machine learning model in is trained, new data 370 including one or more patient input parameters (e.g., MR images, medical images, segmentation information of an object of interest associated with the patient, or dose prescription information) may be received. The generated results 380 may be applied to new data 370 to generate one or more parameters of a radiotherapy treatment plan. For example, after being trained on sensitive private medical information, the trained teacher ML model 360 may be applied to public radiotherapy information (e.g., public medical images) to generate corresponding one or more radiotherapy parameters of a treatment plan (e.g., labeled CT images, sCT images corresponding to CT images, medical images, processed versions of medical images, dose distributions for images, and / or control points of a radiotherapy treatment device corresponding to the radiotherapy images and / or MR scan range maps). The generated one or more radiotherapy parameters of the treatment plan are processed by the trained dimensionality adjustment ML model 360 to reduce the dimensionality of the one or more radiotherapy parameters of the treatment plan (e.g., by processing multiple radiotherapy plan parameters using variational autoencoders, autoencoders, principal component analysis, or homomorphic compression). The reduced-dimensional one or more radiotherapy parameters of the treatment plan are then processed by the trained aggregate ML model 360 (e.g., the trained aggregate ML model 360 calculates the mean, trimmed mean, median, or generalized f-mean of the reduced-dimensional radiotherapy plan parameters) and perturbed according to a privacy criterion (e.g., differential privacy, Renyi differential privacy, pooled differential privacy, mutual information, conditional entropy, Fisher information, generative adversarial privacy, or k-anonymity).

[0087] In some embodiments, when a machine learning model After each machine learning model in is trained, new data 370 including one or more patient input parameters (e.g., MR images, medical images, segmentation information of an object of interest associated with the patient, or dose prescription information) may be received. The teacher ML model 360 may be applied to new data 370 to generate a generated result 380 including one or more parameters for a radiotherapy treatment plan. For example, the new data 370 may include public medical images that are processed by the trained dimensionality adjustment ML model 360 to reduce the dimensionality of the public medical images in the new data 370 plan (e.g., by processing the medical images using a variational autoencoder, an autoencoder, principal component analysis, or homomorphic compression). After training on sensitive private medical information, the trained teacher ML model 360 may be applied to the reduced dimensionality public medical images to generate corresponding one or more radiotherapy parameters for the treatment plan (e.g., labels for a CT image, an sCT image corresponding to the CT image, a medical image, a processed version of the medical image, a dose distribution for the image, and / or control points for a radiotherapy treatment device corresponding to the radiotherapy image and / or an MR scan range map). One or more radiotherapy parameters of the generated treatment plan are then processed by the trained aggregate ML model 360 (e.g., the trained aggregate ML model 360 calculates the mean, trimmed mean, median, or generalized f-mean of the reduced-dimensional radiotherapy plan parameters) and perturbed according to a privacy criterion (e.g., differential privacy, Renyi differential privacy, centralized differential privacy, mutual information, conditional entropy, Fisher information, generative adversarial privacy, or k-anonymity) for training the student ML model. Alternatively, one or more radiotherapy parameters of the generated treatment plan are first perturbed according to a privacy criterion (e.g., differential privacy, Renyi differential privacy, pooled differential privacy, mutual information, conditional entropy, Fisher information, generative adversarial privacy, or k-anonymity), processed by a trained aggregate ML model 360 (e.g., the trained aggregate ML model 360 calculates the mean, trimmed mean, median, or generalized f-mean of the reduced-dimensional radiotherapy plan parameters), and then a student ML model is trained based on the output of the aggregate ML model 360.

[0088] In an embodiment, the perturbation is performed by adding noise comprising samples according to a Gaussian distribution, a Beta distribution, a Dirichlet distribution, or a Laplace distribution. The perturbed aggregated one or more radiotherapy parameters of the treatment plan are then processed by a trained dimensionality adjustment ML model 360 to restore the dimensionality of the above-mentioned radiotherapy parameters (increase their dimensionality) (e.g., by processing the perturbed aggregated multiple radiotherapy plan parameters using a variational autoencoder, an autoencoder, principal component analysis, or homomorphic compression), and then provided to train the student ML model 360. After the student ML model 360 is trained, the trained student ML model 360 is applied to new radiotherapy information (e.g., a medical image) to generate one or more radiotherapy parameters (e.g., a label of a CT image, an sCT image corresponding to the CT image, a dose distribution for the image, a medical image, a processed version of the medical image, and / or control points of a radiotherapy treatment device corresponding to the radiotherapy image and / or an MR scan range map).

[0089] Figure 4A An exemplary data flow 400A is shown for training and using one or more private and public machine learning models to generate a radiation therapy treatment plan according to some examples of the present disclosure. The data flow 400A includes a teacher ML model training portion 401, dimension adjustment ML models 430A and 430B, an aggregator ML model 440, a privacy standard noise addition module 450, a student ML model 460, and public training data 412. Figure 4A and Figure 4B The ML models discussed above can be combined with Figure 3 The discussed approaches are similar to the approaches to training and application.

[0090] Initially, the teacher ML model training portion 401 operates on a collection of private and sensitive training data 410 to train a plurality of teacher ML models 420. Each of the plurality of teacher ML models 420 can be identical in implementation and utilize a common set of ML parameters 312. In some implementations, the sensitive training data 410 is provided by one or more hospitals and / or patients. The sensitive training data 410 can include one or more pairs of labeled or segmented radiotherapy medical images (e.g., CT, MR, and / or sCT images); pairs of 3D models and one or more corresponding radiotherapy medical images; pairs of dose distributions and one or more radiotherapy images; pairs of sCT images and CT images; pairs of control points and one or more radiotherapy images and / or MR scan range maps; and pairs of individual radiotherapy plan parameter estimates and aggregated results of multiple training individual radiotherapy plan parameter estimates. The pairs included in the sensitive training data 410 can depend on the type of teacher ML model 420 used. For example, when the teacher ML model 420 and its supplementary student ML model 460 are configured to generate labels or segmentations of CT images, the pairs of sensitive training data 410 may include pairs of CT images and corresponding labels. As another example, when the teacher ML model 420 and its supplementary student ML model 460 are configured to generate sCT images of CT images, the pairs of sensitive training data 410 may include pairs of sCT images and corresponding CT images.

[0091] Sensitive training data 410 can be divided into disjoint datasets 411. Each dataset 411 is provided to a corresponding instance of teacher ML model 420. In one example, datasets 411 in one set can include a set of sensitive training data 410 from one hospital and / or one set of patients, and datasets 411 in another set can include a set of sensitive training data 410 from another hospital and / or another set of patients. Unauthorized parties outside of portion 401 may not be able to access sensitive training data 410. That is, sensitive training data 410 can be stored in one or more secure databases external to public training data 412 and / or student model 460.

[0092] As an example, when the teacher ML model 420 is configured to generate labels or segmentations of CT images, the teacher ML model 420 operates on various sets of sensitive training data 410 to be trained to estimate labels or segmentations of the CT images given the CT images. For example, the teacher ML model 420 can receive corresponding training CT images and generate corresponding labels or segmentations for the training CT images. Each generated labeling or segmentation is compared with the true labeling or segmentation in the paired training data. Based on the deviation between the generated labeling or segmentation and the true labeling or segmentation in the paired training data and a loss function associated with the teacher ML model, the teacher ML model parameters are updated until the training data pair is iterated a threshold number of times and / or until the deviation reaches a threshold amount.

[0093] After being trained, the teacher model 420 processes the public training data 412 to generate radiation therapy treatment plan parameters for the public training data 412. For example, the public training data 412 may include unlabeled, unsegmented CT images. The teacher model 420 processes the public training data 412 to generate labeled or segmented CT images in the public training data 412. The results are provided to the trained dimensionality adjustment ML model 430A. The trained dimensionality adjustment ML model 430A may include a variational autoencoder, an autoencoder, principal component analysis, or homomorphic compression trained based on the public segmentation map. The trained dimensionality adjustment ML model 430A can reduce the dimensionality of the labeled CT images generated by the teacher ML model 420.

[0094] After reducing the dimensionality of the labeled CT images generated by the teacher ML model 420, the output of the trained dimensionality adjustment ML model 430A is processed by the aggregator ML model 440. In an embodiment, the aggregator ML model 440 calculates the mean, trimmed mean, median, or generalized f-mean of the individual reduced dimensionality labeled CT images generated by the teacher ML model 420. In other embodiments, the aggregator ML model 440 processes the received individual reduced dimensionality labeled CT images and estimates an aggregation of the individual reduced dimensionality labeled CT images.

[0095] The output of the aggregator ML model 440 is provided to a privacy standard noise addition module 450. The privacy standard noise addition module 450 adds noise to the aggregated information provided by the aggregator ML model 440 according to the selected privacy standard level. The privacy standard can include differential privacy, Renyi differential privacy, concentrated differential privacy, mutual information, conditional entropy, Fisher information, generative adversarial privacy, or k-anonymity. The privacy standard noise addition module 450 adds noise to the aggregated information by adding samples according to a Gaussian distribution, a Beta distribution, a Dirichlet distribution, or a Laplace distribution based on the privacy standard.

[0096] Differential privacy provides strict guarantees for database access mechanisms. Differential privacy is based on the concept of dataset adjacency (also called neighborhood): two datasets d, d' are defined as adjacent if they differ in the presence of a single dataset record. Differential privacy then requires that the output of the mechanism is indistinguishable with respect to adjacent inputs. If for any two adjacent inputs d, d'∈D and for any Pr[M(d)∈S]≤e ∈ Pr[M(d′)∈S]+δ, the random mechanism M:D→R satisfies (∈, δ)-differential privacy.

[0097] According to some embodiments, records are image-segment pairs and M is a random training algorithm. That is, M(d) is a random variable representing the model parameters of the ML model, trained on dataset d. In implementations, the randomness comes from the training algorithm, not from the data. Differential privacy is independent of the data distribution, so d is treated as a constant.

[0098] The output of the privacy criteria noise addition module 450 is provided to the dimensionality adjustment model 430B. The dimensionality adjustment model 430B can perform the opposite operation of the dimensionality adjustment model 430A. That is, the dimensionality adjustment model 430B can increase or restore the dimensionality of the now noisy aggregate information provided by the privacy criteria noise addition module 450. The output of the dimensionality adjustment model 430B is provided to the student ML model 460 to train the student ML model 460. That is, the student ML model 460 can now be trained based on the public training data 412 using the personalized noisy estimates of the radiation therapy treatment plan parameters provided by the trained teacher model 420. The privacy parameters ∈ and δ also directly translate into lower bounds on the false positive rate and false negative rate of any discriminator attempting to distinguish between d and d'. This may also be helpful in deciding which parameter values ​​can be considered sufficiently strong.

[0099] If and only if for any two adjacent inputs d, d'∈D and any rejection region The privacy criterion of the random mechanism M:D→R satisfies (∈, δ)-differential privacy when the following conditions are met:

[0100] P FP (d,d′,M,S)+e∈P FN (d,d′,M,S)≥1-δ

[0101] e ∈ P FP (d,d′,M,S)+P FN (d,d′,M,S)≥1-δ

[0102] Among them, P FP and PFN are the false positive and false negative rates of a classifier that outputs d when M∈S and d' otherwise. There are several basic mechanisms for adding differential privacy to existing non-private functions. These basic mechanisms are based on the concept of sensitivity of the function.

[0103] Function f: D → R n l p -Sensitivity S p (f) is defined as:

[0104]

[0105] where the maximum value is taken over adjacent d, d'∈D. Then, the standard mechanism adds noise to the function calibrated with respect to the sensitivity and privacy parameters. For the Laplace mechanism privacy standard, let f: D→R n is a function, and Lap(b) is the Laplace distribution with scale b. If And α=(α1,...,α n ) T , then the mechanism M(d) = f(d) + α is (∈, 0) is differentially private. For the privacy standard of the Gaussian mechanism, let f: D → R n is a function, and N(μ,σ 2 ) has a mean of μ and a variance of σ 2 Normal distribution. where c 2 >2 ln(1.25 / δ). If And α=(α1,...,α n ) T , then the mechanism M(d)=f(d)+α is (∈, δ) is differentially private. Differential privacy is a generalization of differential privacy. In a sense, each Differential privacy mechanisms all satisfy differential privacy, but not every differential privacy mechanism satisfies Differential privacy mechanism. Stricter requirements for differential privacy mechanisms, which allow for a clearer analysis of the cumulative privacy loss and for more general Divergence defines indistinguishability.

[0106] for Divergence privacy criterion, let X~p and Y~q be random variables. Their α-order The divergence is defined as

[0107]

[0108] for any α>1. Differential privacy requires that the output of the mechanism be The divergence is small. Differential privacy, if for any two adjacent inputs d, d'∈D Dα(M(d)||M(d'))≤∈, then the random mechanism M:D1→R satisfies (α,∈) Differential privacy.

[0109] Another privacy criterion is k-anonymity, a property of a database that requires that for a set of quasi-identifiers recorded in the database—such as a postal code or date of birth—any combination of values ​​for the quasi-identifiers present in the database occurs at least k times.

[0110] Another privacy metric is conditional entropy, for which X given Y is defined as:

[0111]

[0112] It intuitively represents the number of extra bits required to describe X after observing Y. In the context of machine learning, Y can refer to the model parameters, and X can refer to the training data. Thus, high conditional entropy limits an adversary's ability to reconstruct the training set based on the published model parameters. Conditional entropy implies a lower bound on the expected estimation error.

[0113] Another privacy metric is mutual information, where the mutual information between X and Y is defined as:

[0114]

[0115] and intuitively represents the amount of information shared between X and Y. Alternatively, we can describe mutual information as the amount of information about X gained by observing Y. While conditional entropy limits the certainty of the reconstruction of the training data, mutual information, on the other hand, limits the reduction of uncertainty. Conditional entropy and mutual information are directly related through entropy, which describes the degree of uncertainty an adversary has about X before observing Y. Mutual information is precisely the difference between entropy and conditional entropy.

[0116] Another privacy standard is generative adversarial privacy, a privacy definition inspired by generative adversarial networks. In a constrained minimax game, an adversarial model is trained alongside a privacy-preserving generative model. The goal of the former is to predict private attributes from public attributes, while the latter minimizes the adversary's prediction performance. Thus, the data owner implicitly learns a privacy-preserving scheme from the data. This scheme is data-dependent but does not require detailed knowledge of the data distribution, thus attempting to combine the advantages of both.

[0117] Figure 4BAn exemplary data flow 400B is shown for training and using one or more private and public machine learning models to generate radiation therapy treatment plans according to some examples of the present disclosure. Data flow 400B is operationally identical to data flow 400A, except that an aggregator ML model 440 is applied directly to the output of the trained teacher ML model 420 rather than to the output of the reduced dimensionality. In data flow 400B, after the output of the trained teacher ML model 420 is aggregated in a manner similar to that in data flow 400A, the aggregated information is provided to a dimension adjustment ML model 430A to reduce the dimensionality of the aggregated information. The dimension adjustment model 430A provides the aggregated information with reduced dimensionality to a privacy standard noise addition module 450. And, as in data flow 400A, the output of the privacy standard noise addition module 450 is provided to a student model 460 after passing through the dimension adjustment ML model 430B to restore or increase the dimensionality of the information.

[0118] Figure 5 is a flow chart illustrating example operations of treatment processing logic 120 in executing process 500 according to an example embodiment. Process 500 may be embodied as computer-readable instructions that are executed by one or more processors such that the operations of process 500 may be performed, in part or in whole, by functional components of treatment processing logic 120; therefore, process 500 is described below by way of example with reference to treatment processing logic 120. However, in other embodiments, at least some of the operations of process 500 may be deployed on various other hardware configurations. Thus, process 500 is not intended to be limited to treatment processing logic 120 and may be implemented, in whole or in part, by any other components. Some or all of the operations of process 500 may be performed in parallel, out of order, or omitted entirely.

[0119] At operation 510, the treatment processing logic 120 receives training data. For example, the treatment processing logic 120 receives training data 320, which may include paired training data sets 322 (eg, input-output training pairs).

[0120] At operation 520 , the treatment processing logic 120 receives constraints for training the model. For example, the treatment processing logic 120 receives the constraints 326 .

[0121] At operation 530 , the treatment processing logic 120 performs training of the model.

[0122] At operation 550 , the treatment processing logic 120 outputs the trained model. For example, the treatment processing logic 120 outputs the trained model 360 to operate on the new input data set 370 .

[0123] At operation 560 , the treatment processing logic 120 utilizes the trained model 360 to generate a radiation therapy plan. For example, the treatment processing logic 120 operates on a new input data set 370 utilizing the trained model 360 .

[0124] Figure 6 6 is a flow chart illustrating example operations of treatment processing logic 120 when executing process 600, according to an example embodiment. Process 600 may be embodied as computer-readable instructions that are executed by one or more processors such that the operations of process 600 may be performed, in part or in whole, by functional components of treatment processing logic 120; therefore, process 600 is described below by way of example with reference to treatment processing logic 120. However, in other embodiments, at least some of the operations of process 600 may be deployed on various other hardware configurations. Thus, process 600 is not intended to be limited to treatment processing logic 120 and may be implemented, in whole or in part, by any other component. Some or all of the operations of process 600 may be performed in parallel, out of order, or omitted entirely.

[0125] At operation 610 , the treatment processing logic 120 receives a medical image of a patient.

[0126] At operation 620 , the treatment processing logic 120 processes the medical image using a student machine learning model to estimate one or more radiation therapy planning parameters, wherein the student machine learning model is trained to establish a relationship between a plurality of common training medical images and corresponding radiation therapy planning parameters of the common training medical images.

[0127] At operation 630 , the treatment processing logic 120 processes the plurality of common training medical images using the plurality of teacher machine learning models to generate a set of radiation therapy plan parameter estimates.

[0128] At operation 640 , the treatment processing logic 120 reduces the respective dimensionality of a set of radiation therapy plan parameter estimates, wherein the radiation therapy plan parameters of the plurality of public training medical images are perturbed according to a privacy criterion.

[0129] At operation 650 , the treatment processing logic 120 generates a radiation therapy treatment plan for the patient based on the estimated one or more radiation therapy planning parameters of the medical image of the patient.

[0130] Figure 7700 is a flowchart illustrating example operations of treatment processing logic 120 when executing process 700 according to an example embodiment. Process 700 may be embodied as computer-readable instructions that are executed by one or more processors such that the operations of process 700 may be performed, in part or in whole, by functional components of treatment processing logic 120; therefore, process 700 is described below by way of example with reference to treatment processing logic 120. However, in other embodiments, at least some of the operations of process 700 may be deployed on various other hardware configurations. Thus, process 700 is not intended to be limited to treatment processing logic 120 and may be implemented, in whole or in part, by any other components. Some or all of the operations of process 700 may be performed in parallel, out of order, or omitted entirely.

[0131] At operation 710 , the treatment processing logic 120 receives common training medical images of a patient.

[0132] At operation 720 , the treatment processing logic 120 trains the student machine learning model to estimate one or more radiation therapy planning parameters of the common medical images by establishing a relationship between the plurality of common training medical images and corresponding radiation therapy planning parameters of the plurality of common training medical images.

[0133] At operation 730 , the treatment processing logic 120 processes the common training medical images using the plurality of teacher machine learning models to generate a set of radiation therapy plan parameter estimates.

[0134] At operation 740 , the treatment processing logic 120 reduces the respective dimensionality of a set of radiation therapy plan parameter estimates, wherein the radiation therapy plan parameters of the plurality of public training medical images are perturbed according to a privacy criterion.

[0135] As previously discussed, various electronic computing systems or devices may implement one or more of the methods or functional operations discussed herein. In one or more embodiments, the radiation therapy processing computing system 110 may be configured, adapted, or used to: control or operate the image-guided radiation therapy device 202; perform or implement the image-guided radiation therapy according to the image-guided radiation therapy device 202; Figure 3, training or prediction operations; operating the trained model 360; performing or implementing the operations of the flowcharts of processes 500 to 700; or performing any one or more of the other methods discussed herein (e.g., as part of the treatment processing logic 120). In various embodiments, such an electronic computing system or device operates as a standalone device or can be connected (e.g., networked) to other machines. For example, such a computing system or device can operate in the capacity of a server or client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. Features of the computing system or device can be implemented by a personal computer (PC), a tablet PC, a personal digital assistant (PDA), a cellular phone, a network appliance, or any machine capable of executing instructions (sequential or otherwise) that specify actions to be performed by the machine.

[0136] As also described above, the functionality discussed above can be implemented by instructions, logic, or other information storage on a machine-readable medium. Although a machine-readable medium may have been described in various examples with reference to a single medium, the term "machine-readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database and / or associated cache memories and servers) that store one or more transient or non-transient instructions or data structures. The term "machine-readable medium" should also be considered to include any tangible medium that can store, encode, or carry transient or non-transient instructions for execution by a machine and causing the machine to perform any one or more of the methods of the present disclosure, or any tangible medium that can store, encode, or carry data structures utilized by or associated with such instructions.

[0137] The above detailed description includes reference to the accompanying drawings, which form a part of the detailed description. The accompanying drawings show the specific embodiments in which the present disclosure can be put into practice by way of illustration, rather than by way of limitation. These embodiments are also referred to as "examples" in this article. Such examples may include elements other than those shown or described. However, the present disclosure is also intended to provide only examples of those elements shown or described. In addition, the present disclosure is also intended to use any combination or permutation of those elements (or one or more aspects of those elements) shown or described with respect to a particular example (or one or more aspects of a particular example) or with respect to other examples (or one or more aspects of other examples) shown or described in this article.

[0138] All publications, patents, and patent documents referenced in this document are incorporated herein by reference in their entirety, as if individually incorporated by reference. In the event of inconsistent usages between this document and those incorporated by reference, the usage in the incorporated references should be considered supplementary to that of this document; for conflicting inconsistencies, the usage in this document controls.

[0139] In this document, when introducing elements of the claims of the present disclosure or embodiments thereof, the terms "a," "an," "the," and "said" are used, as is common in patent documents, to include one or more than one or more of the elements, independent of any other instance or use of "at least one" or "one or more." In this document, unless otherwise indicated, the term "or" is used to refer to a non-exclusive or such that "A or B" includes "A but not B," "B but not A," and "A and B."

[0140] In the appended claims (aspects), the terms "including" and "in which" are used as the plain-English equivalents of the respective terms "comprising" and "wherein." Furthermore, in the appended claims, the terms "including," "comprising," and "having" are intended to be open-ended, to mean that there may be additional elements other than the listed elements, such that objects following such terms (e.g., comprising, including, having) in a claim are still considered to fall within the scope of the claim. Moreover, in the appended claims, the terms "first," "second," and "third," etc. are used merely as labels, and are not intended to impose numerical requirements on their objects.

[0141] The present disclosure further relates to a computing system adapted, configured or operated to perform the operations herein. The system can be specially constructed for the desired purpose, or the system can include a general-purpose computer selectively started or reconfigured by a computer program (e.g., instruction, code, etc.) stored in the computer. Unless otherwise stated, the order of the operations in the embodiments of the present disclosure shown and described herein is not necessary. That is, unless otherwise stated, operations can be performed in any order, and the embodiments of the present disclosure can include more or less operations than these operations disclosed herein. For example, it is contemplated that it is within the scope of the claims of the present disclosure to implement or perform specific operations before, simultaneously with, or after other operations, other operations.

[0142] In view of the above, it can be seen that some of the objects of the present disclosure are achieved and other advantageous results are obtained. Having described various aspects of the present disclosure in detail, it will be apparent that modifications and variations are possible without departing from the scope of the various aspects of the present disclosure as defined in the appended claims. As various changes can be made to the above-described constructions, products, and methods without departing from the scope of the various aspects of the present disclosure, it is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative and not in a limiting sense.

[0143] The examples described herein may be implemented in various embodiments. For example, one embodiment includes a computing device comprising processing hardware (e.g., a processor or other processing circuitry) and memory hardware (e.g., a storage device or volatile memory) including instructions embodied thereon, such that the instructions, when executed by the processing hardware, cause the computing device to implement, perform, or coordinate electronic operations for these techniques and system configurations. Another embodiment discussed herein includes a computer program product, such as may be embodied by a machine-readable medium or other storage device, that provides transient or non-transient instructions for implementing, performing, or coordinating electronic operations for these techniques and system configurations. Another embodiment discussed herein includes a method that is capable of operating on processing hardware of a computing device to implement, perform, or coordinate electronic operations for these techniques and system configurations.

[0144] In other embodiments, the logic, commands, or transient or non-transient instructions that implement various aspects of the above-described electronic operations may be provided in any number of form factors for computing systems, including distributed or centralized computing systems, including desktop or notebook personal computers, mobile devices such as tablet computers, netbooks, and smartphones, client terminals, and server-hosted machine instances. Another embodiment discussed herein includes incorporating the techniques discussed herein into other forms, including other forms of programmed logic, hardware configurations, or dedicated components or modules, including devices having corresponding means for performing the functions of such techniques. The various algorithms for implementing the functions of such techniques may include sequences of some or all of the above-described electronic operations or other aspects depicted in the accompanying drawings and the detailed description above.

[0145] The above description is intended to be illustrative, rather than restrictive. For example, the examples described above (or one or more aspects of the examples) can be used in combination with each other. In addition, without departing from the scope of the present disclosure, many modifications can be made to adapt specific situations or materials to the teachings of the present disclosure. Although the sizes, types of materials and example parameters, functions and implementations described herein are intended to limit the parameters of the present disclosure, they are by no means restrictive embodiments, but exemplary embodiments. After viewing the above description, many other embodiments will be apparent to those skilled in the art. Therefore, the scope of the present disclosure should be determined with reference to the appended claims and the full scope of equivalents enjoyed by such claims.

[0146] Furthermore, in the above detailed description, various features may be grouped together to simplify the disclosure. This should not be interpreted as intending that unclaimed disclosed features are essential to any claim. Rather, the subject matter of the invention may lie in fewer than all features of a particular disclosed embodiment. Accordingly, the appended claims are hereby incorporated into the detailed description, with each claim independently serving as a separate embodiment. The scope of the disclosure should be determined by reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

Claims

1. A method for performing privacy-based radiation therapy treatment planning, the method comprising: receiving, by the processor circuitry, a medical image of a patient; processing, by the processor circuitry, the medical image using a student machine learning model to estimate one or more radiation therapy planning parameters, wherein the student machine learning model is trained to establish a relationship between a plurality of common training medical images and corresponding radiation therapy planning parameters of the common training medical images, wherein the radiation therapy planning parameters of the plurality of common training medical images are generated by: Aggregating a plurality of radiation therapy plan parameter estimates, the plurality of radiation therapy plan parameter estimates being generated by: processing the plurality of common training medical images using a plurality of teacher machine learning models to generate a set of radiation therapy plan parameter estimates; and reducing the set of radiation therapy plan parameter estimates or the corresponding dimensionality of the plurality of common training medical images; and perturbing the aggregated multiple radiation therapy plan parameter estimates according to privacy criteria; increasing the dimensionality of the perturbed aggregated reduced-dimensionality radiation therapy planning parameter estimates to output the one or more radiation therapy planning parameters; and A radiation therapy treatment plan is generated, by the processor circuitry, for the patient based on the estimated one or more radiation therapy planning parameters of the medical image of the patient.

2. The method according to claim 1, wherein The student machine learning model and the multiple teacher machine learning models are implemented by corresponding neural networks or deep learning networks.

3. The method according to any one of claims 1 to 2, further comprising: The teacher machine learning model is trained based on private medical information including private medical images and private radiotherapy planning parameters of the private medical images, wherein the student machine learning model is trained on data that does not include the private medical information.

4. The method according to claim 3, wherein: Training the teacher machine learning model includes: generating disjoint datasets based on the private medical information; and The teacher machine learning model is trained based on corresponding datasets in the disjoint datasets.

5. The method according to any one of claims 1, 2 and 4, further comprising: generating a first radiation therapy plan parameter estimate from the plurality of radiation therapy plan parameter estimates by processing the plurality of common training medical images using a first trained teacher machine learning model from a plurality of trained teacher machine learning models; generating a second radiation therapy plan parameter estimate from the plurality of radiation therapy plan parameter estimates by processing the plurality of common training medical images using a second trained teacher machine learning model from the plurality of trained teacher machine learning models; reducing the dimensionality of each of the first radiation therapy plan parameter estimate and the second radiation therapy plan parameter estimate of the plurality of radiation therapy plan parameter estimates to produce a first radiation therapy plan parameter estimate and a second radiation therapy plan parameter estimate of reduced dimensionality; as well as The reduced-dimensional first radiation therapy plan parameter estimates and the second radiation therapy plan parameter estimates are aggregated.

6. The method according to claim 5, wherein: A first radiation therapy planning parameter estimate of the plurality of radiation therapy planning parameter estimates comprises a first number of entries, and wherein the first reduced-dimensional radiation therapy planning parameter estimate comprises a second number of entries that is less than the first number of entries.

7. The method according to claim 5, wherein: Aggregating the reduced-dimensionality first radiation therapy plan parameter estimate and the reduced-dimensionality second radiation therapy plan parameter estimate comprises: calculating the mean, trimmed mean, median or generalized f-mean of the first and second radiation therapy plan parameter estimates of the reduced dimension; and / or, Aggregating the reduced-dimensional first radiotherapy plan parameter estimates and the second radiotherapy plan parameter estimates includes estimating an aggregation of the reduced-dimensional first radiotherapy plan parameter estimates and the second radiotherapy plan parameter estimates by processing the reduced-dimensional first radiotherapy plan parameter estimates and the second radiotherapy plan parameter estimates using an aggregated machine learning model, wherein the aggregated machine learning model is trained to establish a relationship between a plurality of training individual radiotherapy plan parameter estimates and an aggregated result of the plurality of training individual radiotherapy plan parameter estimates, and to minimize the amount of perturbation required to meet the privacy standard.

8. The method according to claim 5, further comprising: The first reduced dimensionality radiotherapy plan parameter estimate and the second radiotherapy plan parameter estimate or the aggregated first reduced dimensionality radiotherapy plan parameter estimate and the second radiotherapy plan parameter estimate are perturbed by adding noise based on a selected privacy level to at least one of the first reduced dimensionality radiotherapy plan parameter estimate and the second radiotherapy plan parameter estimate or the aggregated first reduced dimensionality radiotherapy plan parameter estimate and the second radiotherapy plan parameter estimate.

9. The method according to claim 8, wherein Adding noise based on a selected privacy level includes adding samples according to a Gaussian distribution, a Beta distribution, a Dirichlet distribution, or a Laplace distribution to at least one of the first and second radiotherapy plan parameter estimates of the reduced dimension or the aggregated first and second radiotherapy plan parameter estimates of the reduced dimension.

10. The method according to claim 5, wherein Increasing the dimensionality of the perturbed aggregated reduced-dimensional radiotherapy plan parameter estimates includes processing the perturbed aggregated reduced-dimensional first radiotherapy plan parameter estimates and the second radiotherapy plan parameter estimates using a variational autoencoder, an autoencoder, principal component analysis, or homomorphic compression.

11. The method according to claim 5, wherein: Reducing the dimensionality of each of the first and second radiotherapy plan parameter estimates in the multiple radiotherapy plan parameter estimates includes: using a variational autoencoder, an autoencoder, principal component analysis, or homomorphic compression to process each of the first and second radiotherapy plan parameter estimates in the multiple radiotherapy plan parameter estimates.

12. The method according to claim 11, wherein The variational autoencoder, the autoencoder or the principal component analysis is trained based on a common segmentation atlas.

13. The method according to any one of claims 1, 2, 4 and 6 to 12, wherein The privacy standard includes at least one of differential privacy, Renyi differential privacy, centralized differential privacy, mutual information, conditional entropy, Fisher information, generative adversarial privacy, or k-anonymity; and / or, wherein the medical image comprises a magnetic resonance (MR) image or a computed tomography (CT) image, and wherein the one or more radiotherapy planning parameters comprise at least one of: a marking of the medical image, a three-dimensional volume corresponding to the medical image, a dose distribution, a synthetic CT image, the medical image, a processed version of the medical image, or radiotherapy device parameters.

14. A computer-readable medium comprising computer-readable instructions, the computer-readable instructions comprising instructions for performing the operations of the method of any one of claims 1 to 13.

15. A system comprising: a memory for storing instructions; as well as One or more processors configured to execute the instructions stored in the memory to perform the operations of the method according to any one of claims 1 to 13.

16. A method for training a machine learning model to perform privacy-based radiation therapy treatment planning, the method comprising: receiving, by the processor circuitry, a common training medical image of a patient; The student machine learning model is trained by the processor circuitry to estimate one or more radiation therapy planning parameters of a plurality of public training medical images by establishing a relationship between the plurality of public training medical images and corresponding radiation therapy planning parameters of the plurality of public training medical images, wherein the radiation therapy planning parameters of the plurality of public training medical images are generated by: Aggregating a plurality of radiation therapy plan parameter estimates, the plurality of radiation therapy plan parameter estimates being generated by: processing the common training medical images using a plurality of teacher machine learning models to generate a set of radiation therapy plan parameter estimates; and reducing the set of radiation therapy plan parameter estimates or the corresponding dimensionality of the plurality of common training medical images; and perturbing the aggregated multiple radiation therapy plan parameter estimates according to a privacy criterion; and The dimensionality of the perturbed aggregated reduced-dimensionality radiation therapy planning parameter estimates is increased to output the one or more radiation therapy planning parameters.

17. The method according to claim 16, further comprising: training the teacher machine learning model based on private medical information including private medical images and private radiotherapy planning parameters for the private medical images, wherein the student machine learning model is trained on data that does not include the private medical information, and / or Wherein, training the teacher machine learning model includes: generating disjoint datasets based on the private medical information; and The teacher machine learning model is trained based on corresponding datasets in the disjoint datasets.

18. The method according to claim 16 or 17, further comprising: generating a first radiation therapy plan parameter estimate from the plurality of radiation therapy plan parameter estimates by processing the plurality of common training medical images using a first trained teacher machine learning model from a plurality of trained teacher machine learning models; generating a second radiation therapy plan parameter estimate from the plurality of radiation therapy plan parameter estimates by processing the plurality of common training medical images using a second trained teacher machine learning model from the plurality of trained teacher machine learning models; reducing the dimensionality of each of the first radiation therapy plan parameter estimate and the second radiation therapy plan parameter estimate of the plurality of radiation therapy plan parameter estimates to produce a first radiation therapy plan parameter estimate and a second radiation therapy plan parameter estimate of reduced dimensionality; as well as The reduced-dimensional first radiation therapy plan parameter estimates and the second radiation therapy plan parameter estimates are aggregated.

19. The method according to claim 16 or 17, wherein: Training the student machine learning model includes performing the following operations for each of the plurality of public training medical images: obtaining a pair of a given common training medical image among the plurality of common training medical images and a given corresponding radiotherapy planning parameter among the corresponding radiotherapy planning parameters; applying the student machine learning model to the obtained public training medical images to generate estimates of radiation therapy planning parameters for the obtained public training medical images; calculating a deviation between the estimated value of the radiotherapy planning parameter for the obtained common training medical image and the obtained radiotherapy planning parameter; and One or more parameters of the student machine learning model are updated based on the calculated deviation.

Images