Updating a radiotherapy plan using information derived from whole slide images (WSI)

Abstract

Systems (600) and methods (S100) for adapting a treatment plan to the characteristics of an individual anatomopathology (e.g. a tumor), and systems, methods and devices for implementing an adaptive treatment workflow (220) that automatically updates (S103) a treatment plan based on whole slide images (WSI) of the target anatomopathology.

Description

Technical Field

[0001] This disclosure generally relates to adaptive radiotherapy, and more specifically, to systems, methods, and apparatus for adjusting treatment plans using information derived from whole-slice images (WSI), and for adjusting radiotherapy plans to suit individual anatomical pathology and patient anatomical variations. Background Technology

[0002] Radiation therapy involves using an external beam of radiation to treat anatomical pathologies (tumors, lesions, vascular malformations, neurological disorders, etc.) by delivering a prescribed dose of radiation (X-rays, gamma rays, electrons, protons, and / or ions) to the anatomical pathology while minimizing radiation exposure to surrounding tissues and critical anatomical structures.

[0003] Typically, radiation therapy treatment involves several phases. First, a precise three-dimensional (3D) map of the anatomical structures within the region of interest (head, body, etc.) is constructed using any one of the following techniques: computed tomography (CT), cone-beam computed tomography (CBCT), magnetic resonance imaging (MRI), positron emission tomography (PET), 3D rotational angiography (3DRA), or ultrasound (or a combination thereof). This determines the precise coordinates of the target within the anatomical structures, locating the tumor or abnormality within the body and defining its precise shape and size. Second, the path of the radiation beam is calculated to deliver a dose distribution deemed acceptable by radiation oncologists after considering various medical constraints. During this phase, a team of experts develops a treatment plan using specialized computer software to optimally irradiate the tumor and minimize the dose to surrounding normal tissues by designing radiation beams that converge to the target from different angles and planes. Third, the radiation therapy plan is executed. During this phase, the radiation dose is delivered to the patient according to the prescribed treatment plan. Typically, the treatment plan is delivered to the patient through a series of radiation treatments called fractions.

[0004] Many factors can cause a difference between the prescribed radiation dose distribution and the actual dose delivered (i.e., the actual dose delivered to the target during radiation therapy). One such factor is the uncertainty of the patient's position within the radiation therapy system. Other factors involve uncertainties introduced by changes that can occur during patient treatment. These changes can include random errors, such as small differences in patient setup position. Other sources can be attributed to physiological / anatomical changes that can occur if the patient's tumor regresses or if the patient loses weight during therapy. Another class of uncertainty includes motion. Motion can overlap with any of these categories, as some motions may be more random and unpredictable, while others may be more regular.

[0005] These changes can cause the target volume, as well as surrounding anatomy and organs, to move and change in size and shape during therapy. Therefore, continuing to follow the initial treatment plan can result in a different dose distribution actually received than planned, leading to a reduced dose to the target volume and / or an increased dose to organs at risk (OARs). Therefore, during the treatment delivery phase, the treatment plan can be adjusted based on the day's images to better reflect the current situation.

[0006] Adaptive radiotherapy can be performed on three different timescales: offline between treatment fractions, online immediately before treatment fractions, and in real-time during treatment fractions. In offline adaptive therapy, new images of the patient (e.g., CT or CBCT images) are acquired before or after each fraction within a fraction, and these images are evaluated to determine the multi-day location of the target volume. Based on this, new plans can be developed to better reflect the range of motion of the target volume. In online adaptive therapy, the radiotherapy system can be used before fractions to validate or adjust the patient's treatment plan for treatment delivery. Therefore, the imaging system can be used to simultaneously modify treatment delivery to reflect changes in the patient's anatomy. In real-time (treatment bed) adaptive therapy, the radiotherapy system can be used during treatment fractions. Adaptive radiotherapy on the treatment bed allows for adjustments to the treatment plan based on anatomical changes while the patient is on the treatment bed. This can involve performing a rapid CBCT scan on the patient prior to treatment delivery, comparing the target volume identified in the CBCT scan (i.e., the treatment session target volume) with the target volume identified during the treatment planning phase (i.e., the planning target volume), and automatically modifying the treatment plan based on the comparison results (i.e., adjusting the treatment plan). Because this comparison captures anatomical changes in the patient, the treatment plan is modified to match the new position and shape of the target volume and surrounding anatomy. The planning workflow associated with such modifications allows the treatment plan to be adjusted for the patient's anatomy at each fraction, while the patient is on the treatment bed.

[0007] The aforementioned treatment plan adjustments involve rescanning the patient prior to treatment delivery and modifying the treatment plan based on anatomical changes and / or patient movement. Because the rescanning of the patient does not capture changes related to the individual characteristics of the tumor itself, such as changes in the tumor's sensitivity to ionizing radiation, the resulting adjusted radiotherapy plan does not take into account tumor-specific variations. However, it is important to consider these variations when adjusting the treatment plan. For example, the tumor's sensitivity to ionizing radiation may have changed from the initial planning stage and / or between treatment fractions. Changes in the tumor's sensitivity to ionizing radiation can cause the tumor's response to ionizing radiation to differ from the response initially observed during the treatment plan. For example, since the initial treatment plan or from previous treatment fractions, the tumor's sensitivity and / or resistance to ionizing radiation may have increased or decreased. Therefore, continuing to follow the treatment plan may result in the tumor receiving a radiation dose higher or lower than the required dose.

[0008] Therefore, it is beneficial to adjust the treatment plan not only based on the anatomical / physiological changes that occur in the patient, but also based on the characteristics of the individual tumor.

[0009] Beneficially, the radiotherapy plan can also be adjusted based on the cellular characteristics of the tumor.

[0010] Beneficially, an adaptive radiotherapy workflow has been developed in which the radiotherapy plan is adjusted not only for the anatomical / physiological changes occurring in the patient, but also for the characteristics of the individual tumor. Summary of the Invention

[0011] According to a first aspect of the invention, a method for adjusting a treatment plan as defined in claim 1 is provided.

[0012] According to a second aspect of the invention, a treatment planning computer system as defined in claim 23 is provided.

[0013] According to a third aspect of the invention, an automated workflow for adaptive radiotherapy sessions is provided as defined in claim 46.

[0014] According to a fourth aspect of the invention, a non-transitory computer-readable storage medium as defined in claim 47 is provided, having stored thereon computer instructions for automatically updating a treatment plan.

[0015] Optional features are defined by the dependent claims.

[0016] The embodiments of the disclosed subject matter enable the adjustment of radiotherapy plans based on the cellular characteristics of the tumor.

[0017] In embodiments, adjustments include: obtaining / receiving / accessing a treatment plan or generating a treatment plan based on radiographic image data containing anatomical structures of interest including the tumor; obtaining a whole-slice image (WSI) of the tumor; and updating the treatment plan based on information extracted from the WSI, wherein the extracted information includes information specific to tumor cell characteristics. For example, a whole-slice image (WSI) can include a digital image of a specimen of cancerous tissue to be treated. A whole-slice image (WSI) represents a dataset of medical images obtained by scanning a specimen on a glass slide and converting it into a digital file.

[0018] Adjustments may include extracting feature vectors from the WSI. Adjustments may also include generating an average feature vector.

[0019] In one embodiment, the update includes using the treatment plan and WSI as input to a plan adjustment module, which includes optimization parameters generated based on feature vectors extracted from the WSI; and generating an updated treatment plan by optimizing the treatment plan based on the average feature vector.

[0020] In an embodiment, the feature vector includes biomarkers encoding morphological information about the tumor, biomarkers including information about the tumor’s proliferative or immune status, and biomarkers indicating the tumor’s resistance to ionizing radiation.

[0021] Embodiments of the disclosed subject matter also implement an adaptive process for seamlessly integrating radiotherapy workflows with digital pathology workflows.

[0022] Embodiments of the disclosed subject matter also enable the adaptation of radiotherapy plans to the characteristics of individual anatomical pathology (e.g., tumors) and anatomical changes in patients.

[0023] Embodiments of the disclosed subject matter also allow for more personalized and targeted radiographic protocols tailored to specific cancer types.

[0024] A system configured to perform the treatment adjustment process disclosed herein is also disclosed.

[0025] A system is also disclosed, including a computer processing device configured to execute a series of programming instructions embodied on a computer-readable storage medium, the execution of which causes the system to perform the treatment adjustment scheme disclosed herein.

[0026] A non-transitory computer-readable storage medium is also disclosed, on which are embodied a series of programming instructions for adjusting a treatment plan as disclosed herein, and a computer processing system for executing the series of programming instructions embodied on the computer-readable storage medium. Executing the series of programming instructions can cause the computer processing system to perform the adaptive workflow described herein.

[0027] In some embodiments, the step of obtaining a WSI may involve retrieving a previously determined WSI, such as from a computer storage device. Any biopsy required to obtain a WSI may be performed prior to performing the claimed method. The WSI determined by the biopsy may be stored in a computer storage device for subsequent use by the currently claimed method. Additionally or alternatively, existing biopsies, including specimens on slides, may be scanned and digitized to generate a WSI without performing any surgical steps on the patient.

[0028] The objects and advantages of embodiments of the disclosed subject matter will become apparent from the following description when considered in conjunction with the accompanying drawings. Attached Figure Description

[0029] Figure 1A This is a simplified schematic diagram of a radiotherapy system according to various embodiments of the disclosed subject matter.

[0030] Figure 1B This refers to the use of various embodiments of the disclosed subject matter. Figure 1A A simplified illustration of a radiotherapy system for adaptive radiotherapy.

[0031] Figure 2A This is a simplified illustration of a treatment plan update system based on various embodiments of the disclosed subject matter.

[0032] Figure 2B This is a simplified illustration of an adaptive workflow based on various embodiments of the disclosed subject matter.

[0033] Figures 3 to 5 This is a simplified illustration of a treatment plan generation system based on various embodiments of the disclosed subject matter.

[0034] Figure 6 This is a simplified illustration of a full slice image (WSI) analyzer module according to various embodiments of the disclosed subject matter.

[0035] Figures 7 to 8 This is a flowchart illustrating a process for adjusting a treatment plan based on information extracted from a corresponding WSI, according to various embodiments of the disclosed subject matter.

[0036] Figure 9 This is a simplified illustration of a 4D adaptive directive based on various embodiments of the disclosed subject matter.

[0037] Figures 10A to 10B This is a simplified illustration of a treatment plan adjustment module based on various embodiments of the disclosed subject matter.

[0038] Figures 11 to 13This is a flowchart illustrating the process of training and applying a deep learning-based treatment plan adjustment module according to various embodiments of the disclosed subject matter.

[0039] Figures 14 to 15 This is a flowchart illustrating a process for adjusting a treatment plan based on information extracted from the corresponding WSI and offsets in the patient's anatomical structure, according to various embodiments of the disclosed subject matter.

[0040] Figures 16A to 16E This is a flowchart illustrating a process for adjusting a treatment plan based on information and / or additional data extracted from a corresponding WSI, according to various embodiments of the disclosed subject matter.

[0041] Figure 17 It is a process flowchart for simulating a comparative treatment plan according to various embodiments of the disclosed subject matter. Detailed Implementation

[0042] refer to Figure 1A This shows that it can be used as... Figure 1B An exemplary radiotherapy system 100 for adaptive radiotherapy is shown. By implementing the various radiotherapy generation, adjustment, and validation protocols described herein, the radiotherapy system 100 can deliver radiation to a patient 110 located on a treatment bed 112. Radiotherapy may include photon-based radiotherapy, particle therapy, electron beam therapy, or any other type of treatment therapy.

[0043] In an embodiment, the radiotherapy system 100 may include a radiotherapy device 101, such as, but not limited to, a linear accelerator (LINAC) operable to generate one or more beams of megavolt (MV) X-ray radiation for treatment. The LINAC may also be operable to generate one or more beams of kilovolt (kV) X-ray radiation, for example, for patient imaging. The system 100 has a gantry 102 supporting a radiotherapy head 114, which carries one or more radiation sources 106 and various beam modulation elements, such as, but not limited to, a homogenizing filter 104 and a collimation assembly 108. The collimation assembly 108 may include, for example, a multi-leaf collimator (MLC), upper and lower jaws, and / or other collimation elements. The collimation assembly 108 and / or the homogenizing filter 104 may be positioned within the radiation beam path by their respective actuators (not shown), which may be controlled by a controller 200.

[0044] The gantry 102 can be a ring gantry (i.e., it extends through a complete 360° arc to create a complete ring or circle), but other types of mounting arrangements are also possible. For example, a static beam, a C-arm, a partially ring gantry, or a robotic arm can be used. Any other frame capable of positioning the treatment head 114 in different rotational and / or axial positions relative to the patient 110 can also be used.

[0045] In this embodiment, the radiotherapy device is an MV intensity modulated radiotherapy (IMRT) device. In such a system, the intensity distribution is customized according to the individual patient's treatment requirements. The IMRT field is delivered using an MLC 108, which may be a computer-controlled mechanical beamforming device attached to the treatment head 114 and includes an assembly of metal fingers or blades. For each beam direction, the optimized intensity distribution is achieved by sequentially delivering various subfields with optimized shapes and weights. From one subfield to the next, the blades may move when the radiation beam is on (i.e., dynamic multi-leaf collimation (DMLC)) or when the radiation beam is off (i.e., segmented multi-leaf collimation (SMLC)).

[0046] Alternatively or additionally, the radiotherapy device 101 may be a helical tomotherapy device, wherein intensity modulation is achieved using a binary collimator (not shown) that is opened and closed under computer control. As the gantry 102 rotates continuously around the patient 110, the exposure time of a narrow beam can be adjusted using the opening and closing of the binary collimator, allowing radiation 120 to be directed to a portion of the patient 110's body and delivered to a region of interest 122 by the most desired orientation and location. The region of interest is a two-dimensional region and / or three-dimensional volume from which radiation is desired to be received, and may be referred to as a target, target domain, or target volume. Another region of interest is a risk region. If a portion includes a risk region, radiation is diverted from that risk region. The patient 110 may have more than one target domain requiring radiation therapy.

[0047] Alternatively or additionally, the radiotherapy device may be a helical tomotherapy device, or a simplified intensity modulated arc therapy (SIMAT) device, a volumetric intensity modulated arc therapy (VMAT) device, or a volumetric high-definition (or super-arc) therapy (HDRT) device. In fact, any type of IMRT device can be used as the radiotherapy device 101 of system 100, and may also include onboard volumetric imaging, which can be used to generate in-treatment image data generated during a treatment session.

[0048] For example, embodiments of the disclosed subject matter can be applied to image-guided radiotherapy (IGRT) devices that use cross-sectional images of the patient's internal anatomy taken during a radiotherapy treatment session (i.e., in-treatment images) to provide information about the patient's position. Two-dimensional or three-dimensional imaging is frequently performed during radiotherapy to guide the treatment using imaging coordinates from the actual radiotherapy plan. This ensures that the patient's positioning within the radiotherapy system is identical to that during planning, and that the patient is correctly aligned during treatment. Although the IGRT process involves conformal radiotherapy guided by specialized imaging tests taken during the planning phase, it does rely on the imaging patterns from the planning process as reference coordinates for positioning the patient 110 during treatment. Therefore, each image-guided radiotherapy system is associated with an imaging system to provide in-treatment (treatment session) images used to set up the radiation delivery procedure.

[0049] Intra-treatment images can include one or more two-dimensional or three-dimensional images (typically X-rays) acquired at one or more different points during treatment. There are various ways to acquire intra-treatment images. In some methods, different independent imaging systems and / or imaging methods are used to acquire pre-treatment and intra-treatment images, respectively. For example, 3D IGRT may include locating a cone-beam computed tomography (CBCT) dataset with a planning computed tomography (CT) dataset, while 2D IGRT may include matching a planar kilovolt (kV) radiograph or megavolt (MV) image with a digitally reconstructed radiograph (DRR) obtained from the planning CT.

[0050] Alternatively, system 100 may include a kilovolt or megavolt detector operable to receive a radiation beam 120. The radiotherapy device 101 and the detector can operate as a computed tomography (CT) system to generate CT images of the patient. These images can illustrate the patient's body tissues, organs, bones, soft tissues, blood vessels, etc.

[0051] Each type of radiotherapy equipment can be accompanied by a corresponding radiation plan and radiation delivery procedure.

[0052] The controller 200 may be, but is not limited to, a graphics processing unit (GPU), and may include a computer with suitable hardware (such as a processor) and an operating system for running various software programs and / or communication applications. The controller 200 may include software programs for communicating with the radiotherapy equipment 101, which are operable to receive data from external software programs and hardware. The computer may also include any suitable input / output (I / O) devices 210 adapted to allow communication between the controller 200 and users of the radiotherapy system 100 (e.g., medical personnel). For example, the controller 200 may be provided with I / O interfaces, a console, storage devices, memory, a keyboard, a mouse, a monitor, a printer, a scanner, and a departmental information system (DIS), such as a communication and management interface (DICOM) for storing and transmitting medical imaging information and related data, and enabling the integration of medical imaging equipment such as scanners, servers, workstations, printers, network hardware, etc.

[0053] Alternatively or additionally, I / O device 210 may provide access to network 700 for data transfer between controller 200 and remote systems. For example, controller 200 may network with other computers and radiotherapy systems via I / O 210. Radiotherapy system 100, radiotherapy device 101, and controller 200 may also communicate via network 700 (cloud-based or otherwise) with databases and servers in different locations, such as hospitals, clinics, field researchers, different databases containing electronic health records (EHRs) 900, different databases storing whole-slice images (WSIs) 800, and / or other databases and servers providing clinical data, pathological data, omics data, electronic medical imaging (EMR) data, and any other supplementary data, including but not limited to paraclinical measurements (i.e., blood, urine, laboratory tests, etc.) and expert conclusions (tumor characterization, radiologist tumor size measurements, gene mutations, etc.) relating to patients and / or tumors, cancer types, or subtypes. The radiotherapy system 100, radiotherapy device 101, and controller 200 can also aggregate clinical information from these different sources and store it in a digital pathology database, analyze the data through different processing methods, and provide an integrated platform for fully automated data access, processing, viewing, and display.

[0054] The radiotherapy system 100, radiotherapy device 101, and controller 200 can also communicate with a dose calculation server (e.g., a distributed dose calculation framework) and a computing system 600 to generate and adjust radiotherapy plans to take into account the patient’s anatomical / physiological changes and other factors affecting the tumor, including but not limited to the tumor’s sensitivity to ionizing radiation and other characteristics of the tumor that can be derived from pathological data.

[0055] The computing system 600 may include a treatment planning system / module 300, a whole slice image (WSI) analyzer system / module 400, and a treatment planning adapter system / module 500. Although the computing system 600 is shown as a standalone unit, the computing system 600 and / or portions thereof may be integrated into the system 100 (e.g., as part of the controller 200, or as a standalone module within the system 100, or integrated into other components of the system 100).

[0056] System 100 may also include multiple modules containing programming instructions (e.g., as part of controller 200, as standalone modules within system 100, or integrated into other components of system 100) that, when executed, cause system 100 to perform various functions related to adaptive radiotherapy or other radiotherapy, as discussed herein. For example, system 100 may include: a patient positioning module operable to position and align patient 110 with respect to a desired location (such as an isocenter of the gantry) for a specific radiotherapy treatment; and an image acquisition module operable to instruct the radiotherapy system and / or imaging equipment to acquire images of patient 110 prior to radiotherapy treatment (i.e., pre-treatment / reference images for treatment planning and patient positioning) and / or during radiotherapy treatment (i.e., in-treatment session images), and to instruct the radiotherapy system 100 and / or imaging equipment 101 or other imaging equipment or systems used to acquire images of patient 110.

[0057] System 100 may further include: a radiation dose prediction module operable to predict the dose to be delivered to patient 110 before the start of radiotherapy treatment; a dose calculation module operable to calculate the actual dose delivered to patient 110 during radiotherapy; a treatment delivery module operable to instruct radiotherapy equipment 100 to deliver a treatment plan to patient 110; an association module operable to associate a planned image with an image acquired during treatment of radiotherapy; a calculation module operable to reconstruct a three-dimensional target volume from the image acquired during treatment; an analysis module operable to calculate displacement measurements; and a feedback module operable to instruct the controller to stop radiotherapy in real time based on a comparison of the calculated displacement with a predetermined threshold (range).

[0058] System 100 may further include: one or more contour generation modules operable to generate contours of target volume and other structures in pre-treatment (planning) and during-treatment (treatment session) images; an image registration module operable to register pre-treatment images with subsequent during-treatment images; a dose calculation module operable to calculate cumulative dose; a contour propagation module operable to propagate contours from one image to another; a contour verification module operable to verify the generated contours; and a registration deformation vector field generation module operable to determine the deformation vector field (DVF) generated during the image deformation process. System 100 may also include modules for electron density map generation, isodose distribution generation, dose-volume histogram (DVH) generation, image synchronization, image display, treatment plan generation, treatment plan optimization, automatic optimization parameter generation, updating and selection, and adaptive policy and treatment information transmission. These modules may be written, for example, in C or C++ programming languages. The computer program code used to perform the operations described herein may be written in any programming language, such as C or C++.

[0059] like Figure 1A , 1B As shown in Figures 2A and 2B, the computational system 600 may include: a treatment planning system / module 300 configured to generate a treatment plan based on radiological image data; a WSI analyzer system / module 400 configured to extract feature vectors from WSI images; and a treatment plan adapter system / module 500 configured to update (adjust) the treatment plan generated by the treatment planning system / module 300 based on the extracted feature vectors and optionally based on additional radiological and / or pathological data. By taking into account WSI and other pathological data, the radiotherapy plan can be further tailored to the cellular characteristics of the tumor, allowing for more personalized and targeted radiotherapy protocols adapted to specific cancer types.

[0060] Alternatively, the treatment planning system / module 300 is not part of the computing system 600. In this case, the treatment plan is generated outside the computing system 600, and the computing system 600 is configured to receive and / or obtain and / or access the treatment plan from the treatment planning system / module 300. Alternatively, the computing system 600 can receive / obtain / access previously generated treatment plans and associated treatment parameters / instructions / guidelines from a database / server that stores treatment plans and associated treatment parameters / instructions / guidelines. Such a database / server can be external to the radiotherapy system 100.

[0061] A general workflow for adjusting treatment plans based on the cellular characteristics of the tumor, such as Figure 2BAs shown, the process may involve: a first step, in which a radiotherapy plan is created, which may involve obtaining radiographic image data of the body parts of the patient to be treated, determining the treatment isocenter, and inputting both into a treatment plan generation algorithm; a second step, in which a whole-slice image (WSI) of the cancer tissue specimen to be treated is obtained, which may involve querying a digital pathology database to find any existing WSIs and / or preparing an order WSI (including indication of biopsy coordinates, staining patterns, etc.); a third step, in which the WSIs and the radiotherapy plan are input into a treatment plan update algorithm, which has been trained to update the radiotherapy plan by automatically analyzing the corresponding WSIs; and a fourth step, in which the updated radiotherapy plan is provided to the user for further evaluation or to the radiotherapy system 100 for execution.

[0062] Alternatively, a general workflow for adjusting a treatment plan based on the cellular characteristics of a tumor may involve: a first step, in which a previously generated radiotherapy plan is obtained / received / accessed, which may involve communicating with the treatment planning system / module 300 and / or an external database / server to access the treatment plan; a second step, in which a whole-slice image (WSI) of the specimen of the cancerous tissue to be treated is obtained, which may involve querying a digital pathology database for any existing WSI and / or preparing a WSI (including indication of biopsy coordinates, staining patterns, etc.); a third step, in which the WSI and the radiotherapy plan are input into a treatment plan update algorithm, which has been trained to update the radiotherapy plan by automatically analyzing the corresponding WSI; and a fourth step, in which the updated radiotherapy plan is provided to the user for further evaluation or to the radiotherapy system 100 for execution.

[0063] Treatment plans can be treatment plans generated in the initial planning phase and / or adjustments to earlier treatment plans (i.e., treatment plans adjusted for previous / different treatment sessions).

[0064] An exemplary treatment plan generation system 300 that can be used to generate treatment plans Figures 3 to 5 As shown in the diagram. In a typical planning process, a qualified medical professional (physician) manually draws contours on one or more planning images (e.g., CT images, i.e., radiographic images). These contours depict the malignant tumor to be irradiated (the target), as well as one or more other structures, such as organs, tissues, etc., susceptible to severe damage from radiation exposure (OARs). Planning radiographic images can also be drawn semi-automatically or automatically using the contour drawing and evaluation module 300A of a computer processing system 1000, which may be part of a computing system 600 or a controller 200. The accepted target contours 311, 312 on the planning images (i.e., CT images) represent the planning patient model 310.

[0065] To generate a treatment plan, medical personnel (physicians) generate a list of treatment parameters, such as, but not limited to, targets to maximize radiation, targets to minimize radiation, and other parameters and guidelines related to the specific radiation system 100 to which the treatment plan will be delivered to patient 110. The physician also specifies a preferred dose distribution for the target structure. This dose distribution can be represented as a set of clinical goals (CGs) or a template of clinical goals, which are appropriate radiation dose targets for treating the patient. These clinical goals (CGs) can be given, for example, in the form of the average radiation dose (Grey) of the target structure and the dose that a specific volume of an organ (such as an organ at risk (OAR)) must not exceed. However, clinical goals can also be given in other dimensions, not limited to the form of radiation dose for the target structure and dose for organ volume. Each of the given goals can also be prioritized, describing the importance of achieving the goal relative to another goal. Such a set is called a prioritized set of clinical goals. Each clinical goal can be represented as a quality metric Q and its associated target value. An exemplary prioritized set of clinical goals is:

[0066] Objective 1: The target (PTV) must receive 50 Gy: Priority 1

[0067] Objective 2: Organ at Risk X (OARx) must receive less than 25 Gy: Priority 2

[0068] Objective 3: Organ at Risk Y (OARy) must receive an average dose of less than 30 Gy: Priority 3

[0069] Then, the treatment plan generation module 300B automatically generates an optimized plan for the planning patient model 310 via an optimization process using one of the rule-based algorithms or machine learning networks, by minimizing a cost function that defines the dose distribution for the priority-ordered set of clinical objectives (CGs). Many algorithms can be used to minimize the cost function, including but not limited to calculating the gradient of the cost function. The solution of the optimization process results in treatment parameters determined for the treatment plan candidates. To optimize the treatment plan, at the start of the treatment planning process, multiple control points (CPs) are also specified for the beam trajectory, which takes into account the beamforming elements of the radiotherapy system 100. Each control point (CP) is associated with a set of treatment parameters, including but not limited to a set of (MLC) blade positions, (MLC) shape, gantry rotation speed, gantry position, dose rate, and / or any other parameters. The number and location of the control points (CPs) can be set in any convenient manner, such as, but not limited to, by using treatment planning software, or by the system operator. Based on treatment parameters, the dose distribution within the treatment volume can be calculated for each control point (CP) using any number of techniques (such as, but not limited to, pencil beam convolution or any other suitable algorithm), and the radiation dose distribution for each (CP) can be associated with the corresponding gantry angle, (MLC) configuration, and monitoring unit (MU). Therefore, during treatment delivery, the extracted (CP) parameters can be associated with the corresponding calculated dose distribution for each (CP).

[0070] The physician can also evaluate treatment plan candidates. The accepted treatment plan candidate becomes treatment plan 320. Treatment plan 320 and associated parameters 321 (i.e., clinical goals, dose specifications, clinical target values, 3D dose of treatment plan, how the plan is optimized, control points, etc.) are stored in the storage device of treatment planning system 300 or the storage device of computer processing system 1000 for later retrieval by radiotherapy system 100 for use by operating system 100 to deliver radiotherapy according to the selected radiotherapy plan.

[0071] Rule-based algorithms, as described in U.S. Publication No. 2019 / 0299025 A1, and machine learning networks, as described in U.S. Publication No. 2020 / 0121951 A1, can both be used to generate treatment plans 320 and corresponding treatment parameters 321, both of which are incorporated herein by reference in their entirety.

[0072] Figure 4An exemplary WSI analyzer system / module 400 is illustrated, configured to extract feature vectors (i.e., tissue and / or cell morphology classification or regression) from whole-slice images (WSI) 410 of a patient 110. A whole-slice image (WSI) represents a dataset of medical images obtained by scanning a specimen on a glass slide and converting it into a digital file. WSIs are digital representations of thin-layer stained tissue from various patient sources, such as biopsies, excisions, exfoliations, and bodily fluids. WSIs contain complete histological sections, allowing pathologists to see individual cells and subcellular structures and extract genetic, molecular, and histopathological information from them. These WSIs can be stored in a digital pathology database that can be accessed, analyzed, and shared among scientists, oncologists, and other professionals in the field to develop appropriate cancer treatments.

[0073] Target WSI 410 is a digital image of a specimen of cancerous tissue to be treated in patient 110. Target WSI 410 can be obtained, for example, from a digital pathology database 810, if it has been previously prepared for patient 110. Otherwise, an order may need to be placed with a hospital, clinic, pathology laboratory, etc., for the preparation of target WSI 410. This order may include specific requirements for WSI preparation, including but not limited to specific biopsy coordinates, staining protocols, specific image analysis algorithms, etc.

[0074] The computer processing system 1000 uses a trained machine learning architecture 420, which includes: a classification engine 421 for generating a grid comprising N tiles / patches 422, each tile / patch in the N tiles / patches 422 corresponding to a descriptive vector; and a trained classifier model 423 trained to determine tissue and / or cell morphology classification or regression (i.e., feature vector) 424 at the WSI level for each tile / patch in the corresponding N tiles / patches. Various machine learning architectures can be used to generate the feature vectors, including but not limited to Inception-v3, ResNet34, ResNet50, ResNet152, DenseNet169, DenseNet201, or other deep learning or machine learning architectures such as fully convolutional networks (FCNs), convolutional neural networks (CNNs), or ensembles thereof. The trained classification model 423 can be selected from a model database 1100 for execution by the computer processing system 1000.

[0075] WSI-level tissue and / or cell morphology classification or regression (i.e., feature vectors) may include, but are not limited to, visual biomarkers that can encode morphological information such as any combination of one or more of the following: shape characteristics of tumor cells or tumor infiltration, specific markers based on specific staining (such as markers used to provide information about proliferative or immune status), cancer type / subtype, cancer grade, percentage of true lymphocytes, percentage of tumor-infiltrating lymphocytes, RNA expression, mutational burden, allele frequency, etc.

[0076] An exemplary method for extracting feature vectors using a trained machine learning classification architecture is disclosed in U.S. Publication No. 2022 / 0180626 A1, which is incorporated herein by reference in its entirety.

[0077] To compute the average eigenvector (AFV) 440, a computation system 430 is used, which includes a computation module 431 that computes the average of 432 N extracted eigenvectors 424.

[0078] Figure 7 An exemplary treatment plan adapter system / module 500 is shown. The treatment plan adapter system / module 500 includes an adjusted treatment plan generation module 510, an evaluation module 550, and a decision-making module 560.

[0079] The treatment plan generation module 510 can be a rule-based module. Figure 10A ) or deep learning-based modules ( Figure 10B The system includes a treatment plan generation algorithm 511, which takes treatment plan 320' and extracted feature vector 440 as input to generate a current plan 513, which is optimized using optimization parameters 512. The optimization parameters 512 are determined based on feature vector 440. These optimization parameters 512 are used to obtain a new treatment plan, namely an adjusted treatment plan 530, which achieves a similarity to the original clinical goal as treatment plan 320'.

[0080] In an exemplary embodiment, the treatment plan 320' is a treatment plan 320 generated by the treatment plan generation system 300, which may be a treatment plan generated during the planning phase or during a previous treatment session.

[0081] Alternatively, treatment plan 320' is a previously generated treatment plan for patient 110 that is accessible by computing system 600.

[0082] Alternatively, treatment plan 320' is an adjustment to an earlier treatment plan (i.e., a treatment plan adjusted for a previous / different treatment session) and is stored in a database / server for receipt / acquisition / access by computing system 600.

[0083] Optimization parameters 512, such as radiation / dose parameters, can be used to recalculate the radiotherapy plan 320' using the treatment plan generation algorithm 511. Optimization parameters 512 can then be determined based on visual biomarkers 440 extracted from the target WSI 410. These visual biomarkers can be normalized across tumors to indicate the resistance of the corresponding tumor type to ionizing radiation. Furthermore, these biomarkers can encode morphological information, such as the shape characteristics of tumor cells or tumor infiltration, and / or specific markers based on specific staining, such as those used for information about proliferative or immune status. The listed biomarkers are merely exemplary; any other molecular, physiological, histological, and radiographic biomarkers extracted from the WSI can be used to generate optimization parameters 512 to recalculate the radiation / dose parameters of the treatment plan.

[0084] In the rule-based adjustment module 510, the treatment parameters 540 can be deterministically modified based on the presence of certain readings in the feature vector 440. For example, the radiation dose or radiation dose rate can be increased or decreased based on certain readings in the feature vector 440, such as proliferation level, percentage of necrotic cells, sensitivity to ionizing radiation, etc. This set of rules can be a pre-determined set of rules, which can be stored in the memory of the computer processing system 1000 and accessed when the adjustment algorithm is executed.

[0085] In the deep learning-based module 510, a trained deep neural network (DNN) is employed, which is trained to update treatment parameters 540 based on readings from the target WSI, such as... Figure 13 As shown. DNN refers to a class of computer-based machine learning algorithms that utilize data processing across many layers or stages (specifically at least two "hidden" layers between the input and output layers) for feature learning, pattern analysis, and / or classification. Typically, these DNN models consist of hierarchical networks of processing elements (called neurons or nodes) interconnected by connections (called synapses or weights). The node layers are trained end-to-end (i.e., from the input layer to the output layer) to extract (multiple) features from the input and classify (multiple) features to produce an output (e.g., a classification label or category).

[0086] Each hidden layer has its own nodes that perform specific computations and are interconnected with nodes in adjacent layers. For example, each node may include a weighting function (providing weights to its respective input) and an activation function (processing the weighted input to generate its respective output). Different hidden layers may include, but are not limited to, final loss layers, nonlinear operator layers, pooling layers, subsampling layers, fully connected layers, and convolutional layers.

[0087] Typically, the operation of each DNN model involves a training phase and an inference phase. During the training phase, the DNN model uses a training dataset to generate a specific output. For example, ... Figure 11 As shown, the training dataset S501 may include a dataset of multiple patients (>1000) with a specific cancer type / subtype. Each patient's dataset may contain at least one WSI (Weighted Injection Sequence) taken prior to radiotherapy. Each patient's dataset may also include information about subsequent (successful) radiotherapy as ground truth information. Optionally, the dataset may additionally include radiographic image data taken prior to radiotherapy. These datasets may be stored on a remote server and accessed via Network 700.

[0088] As used in this article, "training" refers to determining one or more parameters of nodes in the hidden layers of a DNN model, for example through an iterative process that changes the parameters so that the DNN model output more closely matches the corresponding real-world situation. For example, nodes in the hidden layers may include filters or kernels, whose parameters (such as kernel weights, size, shape, and / or structure) can be adjusted during the training process.

[0089] During the training phase, the process can proceed to S502, where a treatment plan is generated for each training dataset in S501. The treatment plan can be generated using any of the treatment plan generation methods described herein. Alternatively, each treatment plan can be a default treatment plan following case-specific guidelines. Alternatively, each treatment plan can be generated by applying a known and validated treatment plan generation algorithm. In S503, a trained classification algorithm (such as ResNet50) is applied to the WSI of the training dataset to generate corresponding feature vectors. Next, the treatment plan generated in S502 and the corresponding feature vectors extracted from the WSI in S504 are fed into the DNN model in S505 to train the DNN model to generate adjusted treatment plans based on the feature vectors.

[0090] The training of DNN models can follow the following steps: Figure 12The process flow is shown below. Specifically, in S515, the parameters of the treatment plan generated in S502 and the corresponding feature vectors obtained in S504 are input into the DNN model, for example, into its input layer. In S516, the DNN model processes the input data by propagating it between nodes in its hidden layers. In S517, the DNN model produces an output (i.e., the treatment parameters as output data) by providing the treatment parameters generated in the hidden layers to the output layer nodes of the DNN model. The output of S517 can be compared with the real situation (i.e., the parameters of the actual treatment plan used in radiotherapy) in S518 via a loss function. For example, this loss function can be mean squared error, DICE loss, cross-entropy-based loss, or any other loss function known in the art.

[0091] During training, the DNN model (via the loss function) is given feedback on how well its output S517 matches the correct output (i.e., the parameters of the true situation). Once the iteration criteria are met at S519 (e.g., the loss function reaches a predetermined threshold, the threshold number of iterations has been reached, or no further improvement is seen between iterations), the DNN model is fixed at S520. Otherwise, training proceeds to S521, where the DNN model is modified by adjusting the hidden layer node parameters of the DNN model itself or the hidden layers of the ResNet to improve the match between the output at S517 and the desired output. The training process can be iterated repeatedly until the desired iteration criteria are met at S519.

[0092] In some embodiments, the training dataset may include an additional subset. For example, the dataset may include a validation set, used during DNN model training (e.g., in...). Figure 12 (S519) tracks the quality of the DNN model, but is not used as input to the DNN model during training. Alternatively or additionally, the training dataset may include a subset of the test dataset, used only after training, to quantify the quality of the trained DNN model (e.g., accuracy, DICE score) and / or to verify that the DNN model has not been over-learned or under-learned.

[0093] During the inference phase, the trained DNN model S520 operates on the input treatment plan S531 to provide an adjusted treatment plan in S533 based on the corresponding feature vector input in S532, such as... Figure 13 As shown. Therefore, when the treatment plan 320' and the corresponding feature vector 440 are input into the trained DNN model S520 of the deep learning-based module 510, the DNN model outputs the corresponding adjusted treatment plan (i.e., the parameters of the output treatment plan).

[0094] Each corresponding DNN model can run on a corresponding DNN engine, which refers to any suitable hardware and / or (multiple) software components of a computer system capable of executing algorithms based on any suitable deep learning model. In embodiments, the (multiple) deep learning models can be based on any existing or future-developed neural network or a combination thereof. Exemplary neural networks include, but are not limited to, convolutional neural networks (ConvNet or CNN) (e.g., U-Net, Deep CNN, LeNet, V-Net, AlexNet, VGGNet, Xception, DenseNet, GoogLeNet / Inception, etc.), residual neural networks (ResNet), recurrent neural networks (RNN) (e.g., Hopfield, Echo State, Independent RNN, etc.), long short-term memory (LSTM) neural networks, recurrent neural networks, generative adversarial neural networks (GAN), and deep belief networks (DBN).

[0095] By execution Figure 7 The adjusted treatment plan 530 obtained by the rule-based or deep learning-based module 510 can be accepted as an adjusted treatment plan executed by the radiotherapy system 100 to irradiate the patient 110.

[0096] Optionally, the adjusted treatment plan 530 can be further evaluated in the evaluation module 550 before being delivered to the patient 110. This evaluation may be based on additional information / data regarding the patient and / or the amount of adjustment made to the treatment parameters in 540.

[0097] For example, the evaluation module 550 can compare the optimized radiation dose (i.e., the radiation dose required by the adjusted treatment plan 530) with the radiation dose required by the treatment plan 320'. If the difference exceeds an acceptable amount (e.g., the amount of modification in 541 is higher or lower than an acceptable threshold, thus causing the optimized radiation dose to require an excessive dose increase or an excessive dose decrease), the adjusted treatment plan 530 can be determined to be unsuitable for delivery.

[0098] Alternatively or additionally, the adjusted radiotherapy plan 530 may also be evaluated based on additional omics data 910 and / or EMR data 920 retrieved from the HER storage device 900. For example, omics data, i.e., data obtained by measuring biomolecules in the fields of proteomics, transcriptomics, genomics, metabolomics, lipidomics, epigenomics, and radiomics, such as, but not limited to, comprehensive analyses of proteins, RNA, genes, metabolites, lipids, and methylated DNA or modified histones in chromosomes, may be used to evaluate the adjusted radiotherapy plan 530. Additional EMR data, such as, but not limited to, electronic medical records indicating the patient's medical and treatment history, and / or additional data retrieved from the EHR storage device 900, such as, but not limited to, demographic data, previous diagnoses, reports, comorbidities, dose limitations, etc., may also be used to evaluate the adjusted treatment plan 530.

[0099] Additionally or alternatively, in order to extract patient-related information from omics data 910 and EMR data 920, neural networks (NNs), such as large language models (LLMs), can be applied to large datasets.

[0100] Evaluation can be based on the application of omics-based predictive models, for example, models trained to predict adaptive radiotherapy eligibility for patients with specific cancers. By applying omics-based predictive models to omics and EMR datasets, the efficacy of the adjusted treatment plan 530 on the tumor can be predicted. For example, by applying a trained omics-based predictive model, tumor shrinkage, and / or treatment perturbation, and / or treatment intensity, and / or the geometry and morphology of the patient's anatomy can be predicted if the adjusted radiotherapy plan 530 is applied. Based on these predictions, the adjusted radiotherapy plan 530 can be accepted as the radiotherapy plan to be delivered to patient 110, or it can be... Figure 8 Further decisions will be made based on the 560.

[0101] Based on the assessment results in 550, a decision can be made in 560 to revise (561) the treatment plan 320' and re-execute the treatment plan adjustment process in (563) 510, and / or perform additional examinations / tests in 562. This may involve arranging new comprehensive radiological imaging examinations, new biopsies, preparing additional WSIs, laboratory tests, etc. In addition, additional, existing WSIs (different from the WSI being analyzed) may be included and analyzed to ensure the outcome.

[0102] Additional or alternative therapies may be suggested based on available information and assessment results. These alternative therapies may include chemotherapy, immunotherapy, surgery, etc. Additional testing may also be performed, such as any combination of one or more of the following: laboratory tests, sequencing, additional radiological imaging, additional digital pathology, etc., to support such decisions. Newly acquired WSI data for the patient can be used to recalculate the eigenvector and re-execute the treatment plan adjustment process in 510 in 563.

[0103] like Figure 9 As shown, the physician also develops an adaptive policy set 230, which is a list of parameters / policies / information describing the intent of adaptive treatment, i.e., a 4D description of the planned treatment for patient 110. This adaptive policy set may include information on: the planned dose specification (i.e., Rx prescription), whether adaptive or standard IGRT treatment is used, specified clinical goals (such as, but not limited to, target dose coverage and (OAR) risk dose limits), planned clinical target values, planning images, supporting images and their corresponding registration information (PET, MRI, etc.), the planned patient model (i.e., the outline of target structures (such as target volume, OAR, and other internal dose-derived structures) on the planning images), the treatment plan (RT Plan), the treatment plan 3D dose (i.e., the RT 3D dose), a list of target structures (target volume, OAR, influencing structures, body contours), a list of influencing structures at different treatment sites, information on the shape and location of planned structures on the planning images, WSI, pathological data, eigenvectors, and any information on how the treatment plan is optimized.

[0104] The radiological data 564 of the revised treatment plan 561, as well as the new WSI and accompanying digital pathology data obtained by performing new tests, examinations, etc. in 562, may also be stored in the computer processing system 1000 and / or transferred to the EHR database 900.

[0105] The adjusted radiotherapy plan 530 can be further evaluated in 570 based on the determination of the offset between the patient 110 when generating the treatment plan 320' and when the patient 110 is preparing for the first treatment session and / or subsequent treatment sessions in the radiotherapy system 100.

[0106] like Figure 14As shown, to assess offsets in the anatomical structure, the patient 110 is first positioned on the treatment bed 112 and moved to the imaging position using the imaging console 210 (S571). After patient setup, the next step is to acquire one or more treatment session images of the area of ​​interest in the patient 110 using the radiographic imaging device 101 before treatment (S572). In an exemplary embodiment, the treatment session image is, for example, a 3D or 4D CBCT scan obtained before the treatment session by irradiating the area of ​​interest in the patient 110 with radiation 120. This treatment session image (i.e., the CBCT image) may show the bony structure of the patient 110 but does not include any depiction of target volume or other structures. Then, in S573, the outline of the target structure (S573') is generated on the treatment session image (CBCT image) to obtain a treatment session patient model 573''. The treatment session patient model 573'' is a CBCT image with outlined targets 311', 312'.

[0107] Then, in S574, the treatment session patient model (i.e., the CBCT image with outlined targets) 573'' is compared with the planning patient model 310 (i.e., the CT image with outlined targets 311, 312) to determine the differences between the targets in the planning image and the targets in the CBCT. This is done by overlaying the two images and using information such as the shape and position of targets 311, 312 in the planning patient model 370 to calculate the localization, shape, and position of the target structures 311', 312' on the treatment session image (CBCT image). Based on this calculation, in S575, it can be determined whether the target structures represent the same anatomical region of patient 110 as represented by the target structures in the planning image, and whether the differences in the shape / localization / position of the target structures in the two images are different, or whether the differences in the shape / localization / position of the target structures in the two images are different. The differences in S575 represent the offset of the target anatomical structures. Exemplary generation of a patient model for a treatment session and exemplary determination of patient anatomical offsets are described in U.S. Publication No. 2020 / 0121951 A1, which is incorporated herein by reference in its entirety.

[0108] If the deviation of the target anatomy is determined to be below a predetermined threshold in S579, the adjusted treatment plan (530 or 530A) can be accepted in S580 and executed in 580.

[0109] On the other hand, if the anatomical deviation exceeds a predetermined threshold (S576), the adjusted treatment plan (530 or 530A) is not accepted in S577 because treatment plan 320' is no longer acceptable. Therefore, a decision can be made in S560 to revise treatment plan 320' and / or arrange additional tests, examinations, etc., to determine whether the tumor morphology has been altered. The treatment plan adjustment process is then repeated in S578 to generate the adjusted treatment plan 530B. The adjusted treatment plan 530B can then be implemented in S580.

[0110] The adjusted treatment plans (530, 530A, 530B) can be further revised based on changes in the patient's anatomy, as shown in Figure 16. To adapt the treatment plans to changes in the patient's anatomy, the treatment session target structure 573' and the adjusted treatment plans (530, 530A, or 530B) are used as inputs to the automated plan generation algorithm in 511'. This plan generation algorithm combines several components from existing components (the photon optimization algorithm (PO-GPU) for VMAT and IMRT, the SmartLMC algorithm for leaf sequences, the RapidPlan for DVH estimation, the FTDC-GPU for optimized dose calculation, and the AcurosXP-GPU for final dose calculation). Additionally, to support the automated adaptive workflow 220, an additional component is included that allows for the automatic generation, selection, and continuous modification of optimization parameters 512', through which the algorithm 511' and the final generated plan 513' are optimized based on clinical targets stored in the 4D guidelines. The information on treatment parameters for the adjusted treatment plans (530, 530A, 530B) included in the 4D guidelines is used as optimization parameters in 512', and these parameters are automatically modified in 540' with the goal of obtaining a dose distribution similar to that of treatment plan 320'.

[0111] When the evaluation module 520' determines that the dose distribution of the generated plan 513' is within an acceptable threshold, the generated treatment plan is accepted as the adjusted treatment plan 530C. Therefore, the adjusted treatment plan 530C combines optimization based on variations in the patient's anatomy with optimization based on WSI. Thus, the adjusted treatment plan 530C provides a comprehensively optimized treatment plan.

[0112] The treatment planning adjustment module 510 can also be used to adjust treatment plans between different treatment sessions / therapies. For example, in the case of patient 110 who has already undergone radiotherapy and will now undergo further radiotherapy, a new WSI can be scheduled before the further radiotherapy to reflect the effect of radiotherapy on the tumor. For example, the percentage of necrotic tissue in the WSI can indicate the sensitivity of the tumor to radiotherapy. To ensure that the WSI includes the necessary information, the WSI can be specifically scheduled in terms of staining and pretreatment to best reflect the effect of (multiple) previous treatments.

[0113] Information obtained from the new WSI can be used to further adjust the treatment plan used for the first radiation therapy. For example, the radiation dose can be increased or decreased.

[0114] Additionally or alternatively, a prior WSI taken before the first radiation therapy may be available. This prior WSI can be compared with a WSI taken after radiation therapy. The comparison results between the two WSIs can also be used to further adjust the radiation therapy plan. In order to enable a proper comparison of the two WSIs, WSIs obtained after the second radiation therapy or after the first radiation therapy may need to be specifically arranged to be comparable to the prior WSI in terms of staining and / or pretreatment.

[0115] An exemplary process S100 for adjusting a radiotherapy plan based on the specific characteristics of the target (tumor) is as follows: Figure 16A , 16B As shown. In the first optional step S101, a treatment plan is generated for the patient based on radiographic image data of the anatomical structures of interest containing the target. Alternatively, in step S101, instead of generating a treatment plan, a previously generated treatment plan is received / obtained from a treatment plan generation system / database / server. Then, in S102, a whole-slice image (WSI) of the target is obtained. This WSI can be obtained from any external digital database containing the WSI. Next, in S103, an adjusted treatment plan is generated based on information extracted from the WSI. In S104, the adjusted treatment plan is evaluated using one or more additional data and / or information, including but not limited to omics data, EMR data, radiographic data, and pathological data. Based on the evaluation results, a decision is made regarding the next step in S105. This may involve revising the treatment plan and / or suggesting further examinations / studies / alternative treatments, etc.

[0116] An exemplary process for adjusting radiotherapy plans based on the specific characteristics of the target (tumor) and changes in the patient's anatomy, as shown in S200. Figure 16CAs shown. In the first optional step S201, a treatment plan is generated for the patient based on radiographic image data containing the anatomical structure of interest (AIO) of the target. Alternatively, in step S201, instead of generating a treatment plan, a previously generated treatment plan is received / obtained from the treatment plan generation system 300 / database / server. Then, in S202, a whole-slice image (WSI) of the target is obtained. This WSI can be obtained from any external digital database containing the WSI. Next, in S203, an adjusted treatment plan is generated based on information extracted from the WSI. In S204, a treatment session image (CBCT) containing the AIO of interest of the target is generated. In S205, the adjusted treatment plan generated in S203 is updated based on information determined from the CBCT images. The information determined from the CBCT images includes information about the offset of the AIO of interest. Then, in S206, the updated adjusted treatment plan is evaluated, and in S207, a decision is made regarding the further steps to be taken. This may involve revising the treatment plan and / or suggesting further examinations / studies / alternative treatments, etc.

[0117] An exemplary process S300 for adjusting a radiotherapy plan based on information contained in a new WSI taken during radiotherapy, such as... Figure 16D As shown. In step S301, radiotherapy is delivered to the patient according to the radiotherapy plan. A new WSI is obtained for the patient in S302 before further radiotherapy is administered. Based on the information extracted from the new WSI, the treatment plan is updated in S303. The updated treatment plan is evaluated in S304, and a decision is made regarding the further steps to be taken in S305.

[0118] An exemplary process S400 for adjusting a radiotherapy plan based on a comparison between information contained in a new WSI taken during radiotherapy and information contained in a WSI corresponding to the first radiotherapy, as shown below. Figure 16E As shown. In step S401, radiotherapy is delivered to the patient according to the radiotherapy plan. A new WSI is obtained for the patient in S402 before further radiotherapy is administered. In S403, the information extracted from the new WSI is compared with the information contained in the WSI corresponding to the delivered radiotherapy plan. In S404, the treatment plan is updated based on the comparison. In S405, the updated treatment plan is evaluated, and in S406, a decision is made regarding the further steps to be taken.

[0119] The adjusted treatment plans (530, 530A, 530B, 530C, etc.) generated based on the methods described throughout this specification, and the associated pathology data stored in the EHR 900, can also be applied to, for example... Figure 17The dose calculation algorithm S601 shown simulates dose accumulation and tumor progression in the session target volume according to the adjusted treatment plan. The simulated dose and tumor progression can be compared in S602 with the dose simulation and tumor status in the treatment plan 320'. A decision regarding further action is then made in S603 based on the comparison results. This provides additional checks and validation prior to radiation delivery.

[0120] Based on the results: (i) alternative or concomitant therapies, such as but not limited to chemotherapy, immunotherapy, surgery, etc., may be performed in S604, and / or (ii) additional examinations, such as laboratory tests, sequencing, additional radiological imaging examinations, additional digital pathology examinations, etc., may be performed in S605, and / or (iii) a modified treatment plan may be performed in S606.

[0121] Digital pathology data used in radiotherapy can be fed back into the digital pathology workflow in the following ways:

[0122] • The preparation of biopsies and / or WSIs can be scheduled and dispatched in response to the generation of a treatment plan.

[0123] • Coloring can be specified automatically.

[0124] • You can request staining / analysis of adjacent slices.

[0125] • The required preprocessing can be specified, for example, by applying a specific image analysis algorithm.

[0126] Therefore, it is evident that the disclosed subject implements an adaptive process 220 that seamlessly integrates radiotherapy workflows with digital pathology workflows.

[0127] It is equally evident that the disclosed subject matter enables the system to execute adaptive workflows as described in this paper 220.

[0128] Therefore, it is equally evident that the disclosed topics enable the adjustment of treatment plans based on the characteristics of individual tumors and the patient's anatomical changes.

[0129] Therefore, it is equally evident that the disclosed subject matter enables the adjustment of treatment plans based on the cellular characteristics of the tumor.

[0130] Equally evident is that the disclosed subject matter allows for more personalized and targeted radiographic protocols tailored to specific cancer types.

[0131] It will be understood that various aspects of the disclosed subject matter can be implemented, in whole or in part, in hardware, hardware programmed by software, software instructions stored on a computer-readable medium (e.g., a non-transitory computer-readable medium), or any combination thereof.

[0132] For example, components of the disclosed subject matter, including components, processes, or any other features such as controllers, may include, but are not limited to, personal computers or workstations or other such computing systems, which include processors, microprocessors, microcontroller devices, or control logic comprising integrated circuits, such as application-specific integrated circuits (ASICs).

[0133] The features discussed herein can be executed on a single or distributed processor (single-core and / or multi-core), by components distributed across multiple computers or systems, or by components commonly located within a single processor or system. For example, aspects of the disclosed subject matter can be implemented via: a programmable general-purpose computer, an integrated circuit device (e.g., an ASIC), a digital signal processor (DSP), an electronic device programmed using microcode (e.g., a microprocessor or microcontroller), hard-wired electronic or logic circuits, programmable logic circuits (e.g., programmable logic devices (PLDs), programmable logic arrays (PLAs), field-programmable gate arrays (FPGAs), programmable array logic (PALs)), software stored on a computer-readable medium or signals, optical computing devices, network systems of electronic and / or optical devices, special-purpose computing devices, semiconductor chips, and software modules or objects stored on a computer-readable medium or signals.

[0134] When implemented in software, functionality can be stored on or transmitted via a computer-readable medium as one or more instructions or code. The steps of the methods or algorithms disclosed herein can be embodied in a processor-executable software module that may reside on a computer-readable medium. Instructions can be compiled from provided source code instructions according to a programming language. A set of programming instructions and associated data can be stored in a computer-readable medium (e.g., a non-transitory computer-readable medium), such as computer memory or a storage device, which can be any suitable memory device, such as, but not limited to, read-only memory (ROM), programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), flash memory, disk drives, etc.

[0135] As used herein, computer-readable media includes both computer storage media and communication media, encompassing any medium that facilitates the transfer of a computer program from one place to another. Therefore, a storage medium can be any available medium that can be accessed by a computer. By way of example and not limitation, such computer-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

[0136] Furthermore, any connection is properly referred to as a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using a transmission medium (such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave), that transmission medium is included in the definition of a computer-readable medium. Additionally, the operation of a method or algorithm may reside as a set of code and / or instructions (or any combination thereof) on a machine-readable medium and / or a computer-readable medium, which may be incorporated into a computer program product.

[0137] It will be readily understood by those skilled in the art that the above description is not exhaustive, and that various aspects of the disclosed subject matter may be implemented in ways different from those specifically disclosed above. In fact, embodiments of the disclosed subject matter can be implemented in hardware and / or software by those skilled in the art using any known or future-developed systems, structures, devices, and / or software based on the functional descriptions provided herein.

[0138] In this application, unless otherwise expressly stated, the use of the singular includes the plural, and the separate use of "or" and "and" includes the other, namely "and / or". Furthermore, the use of the terms "comprising" or "having" and other forms such as "include", "has included", "has" or "has" is intended to have the same effect as "comprising" and therefore should not be construed as restrictive.

[0139] Any ranges described herein shall be understood to include the endpoints and all values ​​in between. Whenever “substantially,” “approximately,” “essentially,” “nearly,” or similar language is used in combination with a particular value, unless otherwise expressly stated, it is intended to include at most and within 10% of the variation of that value.

[0140] The terms “system,” “device,” and “module” are used interchangeably herein, and the use of one term in the description of an embodiment does not preclude the application of other terms to that embodiment or any other embodiment.

[0141] This disclosure enables numerous alternatives, modifications, and variations. While specific examples have been shown and described in detail to illustrate the application of the principles of the invention, it should be understood that the invention may be embodied in other ways without departing from these principles. For example, disclosed features may be combined, rearranged, omitted, etc., to produce additional embodiments, and certain disclosed features may sometimes be used advantageously without a corresponding use of other features. Therefore, the applicant intends to cover all such alternatives, modifications, equivalents, and variations within the scope of this invention.

Claims

1. A method for adjusting a treatment plan, comprising: Obtain treatment plans based on radiographic image data of anatomical structures of interest containing targets; Obtain a whole slice image (WSI) of the target; as well as The treatment plan is updated based on the information extracted from the WSI.

2. The method according to claim 1, wherein the method is an offline adaptive radiotherapy treatment planning process.

3. The method of claim 1 or 2, wherein obtaining the WSI comprises receiving the WSI from a database storing existing WSIs.

4. The method according to any of the preceding claims, wherein obtaining the WSI comprises obtaining a new WSI from tissue previously excised from a living organism.

5. The method according to any of the preceding claims, wherein obtaining the WSI includes generating data specifying how the WSI is obtained, such as determining biopsy coordinates, staining schemes, and / or image analysis algorithms.

6. The method according to any one of the preceding claims, wherein the extracted information includes information on cell characteristics specific to the target.

7. The method according to any one of the preceding claims, wherein the target is a tumor and the extracted information includes one or more biomarkers.

8. The method of claim 7, wherein the biomarker comprises a biomarker encoding morphological information about the tumor, a biomarker including information about the proliferative or immune status of the tumor, and / or a biomarker indicating the tumor's resistance to ionizing radiation.

9. The method according to any one of the preceding claims, wherein the extracted information is presented as an average feature vector.

10. The method of claim 9, further comprising extracting the average feature vector from the WSI by: The trained machine learning system is applied to the WSI to convert the WSI image data into feature vectors; and Calculate the average value of the eigenvectors. The machine learning system described therein has been trained to predict tissue or cell morphology classification, regression, or resistance at WSI levels.

11. The method of claim 10, wherein the conversion comprises: The WSI is divided into multiple tiles / patches; as well as Assign a corresponding feature vector to each of the tiles / patches.

12. The method of claim 10 or 11, wherein the trained machine learning system comprises a trained ResNet 50 deep learning neural network model.

13. The method according to any one of claims 9 to 12, wherein the update comprises: The treatment plan and the WSI are used as inputs to a plan adjustment module, which includes optimization parameters generated based on the feature vector extracted from the WSI. as well as An updated treatment plan is generated by optimizing the treatment plan based on the average feature vector.

14. The method of claim 13, wherein the optimization parameters include radiation and / or radiation dose-related parameters.

15. The method of claim 14, wherein the optimization comprises: The radiation and / or radiation dose-related parameters are modified based on the information contained in the average eigenvector.

16. The method according to any one of claims 13 to 15, wherein the plan adjustment module is one of the following: a rule-based module and a deep learning-based module.

17. The method of claim 16, wherein the deep learning-based module is trained to generate treatment plan parameters based on feature vectors.

18. The method of claim 17, wherein the training comprises: A neural network model is trained using a dataset of multiple patients with a specific disease type / subtype and associated WSI and radiotherapy plans, wherein the trained neural network model is configured to predict treatment plan parameters based on feature vectors.

19. The method according to any of the preceding claims, wherein the update is further based on additional data, the additional data including one or more of the following: omics data, EMR data, and one or more additional radiological image data.

20. The method of claim 19, wherein the additional radiographic image data includes information about the offset of the anatomical structure of interest containing the target.

21. The method according to any one of the preceding claims, further comprising: Evaluate the updated treatment plan; as well as The next step will be selected based on the results of the assessment.

22. The method of claim 21, wherein: The assessment includes simulating cumulative radiation dose and disease progression for the updated treatment plan and comparing it with simulated cumulative radiation dose generated for the treatment plan; and The options include one or more of the following: revising the treatment plan, arranging new tests, obtaining a new WSI, proposing alternative or concomitant therapies, performing additional examinations, and implementing the updated treatment plan.

23. A treatment planning computer system, configured as follows: Obtain treatment plans based on radiographic image data of anatomical structures of interest containing targets; Obtain a whole-slice image (WSI) of the target; and The treatment plan is updated based on the information extracted from the WSI.

24. The computer system of claim 23, configured to obtain the treatment plan, obtain the WSI, and update the treatment plan as an offline adaptive radiotherapy treatment planning process.

25. The computer system of claim 23 or 24, wherein obtaining the WSI comprises receiving the WSI from a database storing existing WSIs.

26. The computer system according to any one of claims 23 to 25, wherein obtaining the WSI comprises obtaining a new WSI from tissue excised from a living body.

27. The computer system of any one of claims 23 to 26, wherein obtaining the WSI includes generating data specifying how the WSI is obtained, such as determining biopsy coordinates, staining schemes, and / or image analysis algorithms.

28. The computer system according to any one of claims 23 to 27, wherein the extracted information includes information on cell characteristics specific to the target.

29. The computer system according to any one of claims 23 to 28, wherein the target is a tumor and the extracted information includes one or more biomarkers.

30. The computer system of claim 29, wherein the biomarker includes a biomarker encoding morphological information about the tumor, a biomarker including information about the proliferative or immune status of the tumor, and / or a biomarker indicating the tumor's resistance to ionizing radiation.

31. The computer system according to any one of claims 23 to 30, configured such that the extracted information is presented as an average feature vector.

32. The computer system of claim 31 is further configured to extract the average feature vector from the WSI in the following manner: The trained machine learning system is applied to the WSI to convert the WSI image data into feature vectors; and Calculate the average value of the eigenvectors. The machine learning system described therein has been trained to predict tissue or cell morphology classification, regression, or resistance at WSI levels.

33. The computer system of claim 32, wherein the conversion comprises: The WSI is divided into multiple tiles / patches; as well as Assign a corresponding feature vector to each of the tiles / patches.

34. The computer system of claim 32 or 33, wherein the trained machine learning system comprises a trained ResNet 50 deep learning neural network model.

35. The computer system according to any one of claims 31 to 34, wherein the update comprises: The treatment plan and the WSI are used as inputs to a plan adjustment module, which includes optimization parameters generated based on the feature vector extracted from the WSI. as well as An updated treatment plan is generated by optimizing the treatment plan based on the average feature vector.

36. The computer system of claim 35, wherein the optimization parameters include radiation and / or radiation dose-related parameters.

37. The computer system of claim 36, wherein the optimization includes: The radiation and / or radiation dose-related parameters are modified based on the information contained in the average eigenvector.

38. The computer system according to any one of claims 35 to 37, wherein the plan adjustment module is one of the following: a rule-based module and a deep learning-based module.

39. The computer system of claim 38, wherein the deep learning-based module has been trained to generate treatment plan parameters based on feature vectors.

40. The computer system of claim 39, wherein the training comprises: A neural network model is trained using a dataset of multiple patients with a specific disease type / subtype and associated WSI and radiotherapy plans, wherein the trained neural network model is configured to predict treatment plan parameters based on feature vectors.

41. The computer system according to any one of claims 23 to 40, wherein the update is further based on additional data, the additional data including one or more of the following: omics data, EMR data, and one or more additional radiological image data.

42. The computer system of claim 41, wherein the additional radiographic image data includes information about the offset of the anatomical structure of interest containing the target.

43. The computer system according to any one of claims 23 to 42, further configured as follows: Evaluate the updated treatment plan; and The next step will be selected based on the results of the assessment.

44. The computer system according to claim 43, wherein: The assessment includes simulating cumulative radiation dose and disease progression for the updated treatment plan and comparing it with simulated cumulative radiation dose generated for the treatment plan; and The options include one or more of the following: revising the treatment plan, arranging new tests, obtaining a new WSI, proposing alternative or concomitant therapies, performing additional examinations, and implementing the updated treatment plan.

45. The computer system according to claim 23, comprising: user interface; A storage device that stores software instructions for a treatment plan generation module configured to generate a treatment plan for delivering a specified radiation dose to the target based on a specified clinical goal received via the user interface; A conversion module configured to convert the whole slice image (WSI) data into a feature vector, the feature vector including cell information about the target; as well as A treatment plan adjustment module is configured to automatically update the treatment plan based on the feature vector. The treatment plan adjustment module is configured to use the average of the treatment plan and the feature vector as input to the plan adjustment algorithm, and to generate an updated treatment plan by optimizing the treatment plan based on the average feature vector.

46. ​​An automated workflow for an adaptive radiotherapy treatment session, comprising: Obtain a treatment plan based on radiological image data targeting specific points within the patient's body; Obtain a whole slice image (WSI) of the target; Feature vectors containing cellular information about the target are extracted from the whole slice image (WSI). The treatment plan is updated based on the feature vector; The updated treatment plan was evaluated using additional pathological and radiological information about the target; and The next step will be selected based on the results of the assessment. The additional pathological information mentioned above includes information retrieved from the EHR database and / or from the patient's EMR. The additional radiological information mentioned above includes information obtained from CBCT scans of the target, and The next steps mentioned therein include one or more of the following: revising the treatment plan, arranging new tests, obtaining a new WSI, proposing alternative or concomitant therapies, performing additional examinations, and implementing the updated treatment plan.

47. A non-transitory computer-readable storage medium storing computer instructions for automatically updating a treatment plan based on whole-slice images (WSI), the computer instructions, when executed by a processor, causing the processor to: Whole-slice image (WSI) data is converted into an average feature vector, which includes cellular information about the target; and The treatment plan is automatically updated based on the average feature vector. The updates mentioned include: The treatment plan and the average vector are used as inputs to a trained treatment plan adjustment algorithm, and an updated treatment plan is generated by optimizing the treatment plan based on the average feature vector.

Images