Artificial intelligence model for VMAT sector calculation

Abstract

Disclosed herein are systems (100) and methods (200) for iteratively training an artificial intelligence model (530) using reinforcement learning techniques, including a method, the method includes executing (250) an artificial intelligence model (530) configured to predict a plurality of control points and a plurality of dose calculation sectors of a volume modulated arc therapy (VMAT) treatment using at least one patient attribute, a plurality of arcs, and at least one clinical target attribute associated with the VMAT treatment of the patient, the artificial intelligence model (530) has been iteratively trained using a reinforcement learning method wherein, with each iteration, the artificial intelligence model (530) sends (210) one or more test control points and one or more test dose calculation sectors to a plan optimizer model configured to predict a treatment plan; and training (240) the policy according to the calculated (230) reward.

Description

Technical Field

[0001] This application generally relates to generating, training, and operating artificial intelligence (AI) computer models to predict radiotherapy treatment plans. Background Technology

[0002] Radiation therapy (radiation-based therapy or simply radiation therapy) is a cancer treatment in which high doses of radiation capable of killing cancer cells or shrinking tumors are applied to the affected area. The target area of ​​a patient's body to be irradiated (e.g., a tumor) is called the planned target volume (PTV). The goal is to deliver enough radiation to the PTV to kill cancer cells (also referred to herein as a treatment plan or radiation therapy). However, other organs or anatomical regions adjacent to or surrounding the PTV may be in the path of the radiation beam and may receive enough radiation to be damaged or harmed. These organs or anatomical regions are called organs of danger (OARs). Typically, physicians or radiologists identify both the PTV and OARs before radiation therapy using images such as computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), images acquired via other imaging modalities, or combinations thereof.

[0003] Using the patient's medical images and the identified PTV and OAR, a team of medical professionals (e.g., internists, radiologists, oncologists, radiation technicians, other medical personnel, or a combination thereof) determines the radiation parameters to be used during radiotherapy treatment. These radiation parameters may include, for example, the type, angle, intensity, and / or shape of each radiation beam. In determining these parameters, the team attempts to achieve a radiation dose distribution to the patient that conforms to predetermined criteria (also referred to herein as planning goals or clinical goals).

[0004] To optimize radiation parameters in a manner that meets predetermined criteria, software solutions or computer models (often called "planning optimizers") are used. These optimizers can run multiple simulations with various radiation parameters and select a final set of parameters to use based on the simulation results. These radiation parameters are commonly referred to as a radiotherapy treatment plan (RTTP). However, this process can be very inefficient because planning optimizers typically use iterative, trial-and-error processes to determine the various properties of the RTTP. For example, after the therapist inputs the target, the software solution iteratively analyzes different possibilities (e.g., different iterations of changing properties within a large search space) to identify one or more beam geometry properties that produce optimal or acceptable results. As a result, the software solution may require significant computational resources and may not produce timely results.

[0005] For example, radiation parameters can include volumetric modulated arc therapy (VMAT) techniques, such as control points (CPs). VMAT arc optimization is a non-convex problem constrained within a very large search space for CP parameters. Solving this problem requires an efficient spatial exploration scheme. Commonly used computer models search for optimal CP parameters within this large search space until the model converges to a result. This process is undesirable because it requires high computational power and may not produce timely results. Summary of the Invention According to a first aspect of the invention, a method as defined in claim 1 is provided. According to a second aspect of the invention, a system as defined in claim 8 is provided. According to a third aspect of the invention, a system as defined in claim 15 is provided. Optional features are defined by the dependent claims.

[0006] For the reasons mentioned above, there is a need for methods and systems that do not rely on the subjective interpretation of technicians or conventional computer modeling techniques to generate computer models of therapeutic attributes (specifically, VMAT arc attributes or CP attributes). There is also a need for an improved AI modeling / training technique that does not require extensive training datasets and is computationally efficient, cost-effective, and produces timely results.

[0007] In some embodiments, certain computer models (configured for VMAT arc optimization) construct a search for optimal CP parameters based on an asymptotic multiresolution (MR) method. In this method, at the beginning of each MR level, (1) the arc field is divided into multiple dose calculation sectors (DCs); (2) an iterative subset of DCS candidates is selected for calculating dose gradient updates; and (3) the MR level to continue is determined based on the convergence state of the optimizer.

[0008] The aforementioned sector tuning task has a positive impact on the final plan quality and the convergence speed of the plan optimizer. For example, the method of dividing the arc field may affect the speed at which a beneficial dose delivery angle is found. Furthermore, the solution to the determined optimization problem may be heavily biased towards the DCS candidate selected early in the optimization process. Finding the optimal strategy in performing the sector tuning task requires solving complex subproblems. This can slow down the optimization of the treatment plan. Typically, computer models used for VMAT arc optimization employ strategies that do not produce suitable results, such as selecting a fixed number of DCSs (sometimes randomly). This may provide a reasonable solution for some treatment cases, but may lead to suboptimal solutions with slower convergence speeds in other treatment cases.

[0009] The methods and systems discussed in this paper address this technical problem by using modern machine learning approaches, where a trained AI policy is used to optimally delegate sectors for each specific treatment use case. Furthermore, the agent (the policy of the AI ​​model) performs the delegation tasks at a much faster rate than traditional methods.

[0010] Using the methods and systems discussed herein, a computer (or processor or server) can be trained (e.g., using reinforcement learning methods) and executed to create an AI model that predicts the optimal way to divide a field arc into multiple sectors, each sector having a certain number of CPs (depending on the complexity of the fluence map within that sector). Some conventional methods, such as manual methods, identify sectors without considering patient-specific data (e.g., anatomical geometry). In contrast, the AI ​​model discussed herein divides the arc based on patient-specific data (at least partially). In this way (e.g., custom segmentation based on patient-specific data), a better segmentation of the arc is obtained, through which the planning optimizer can achieve convergence on RTTP with less computational power and / or in less time. Furthermore, the methods and systems discussed herein can be used to optimize the number of CPs. For example, in some embodiments, the AI ​​model can generate fewer CPs, which can lead to the planning optimizer achieving better results in less time.

[0011] In one embodiment, a method includes: executing an artificial intelligence model by a processor using at least one patient attribute, the number of arcs, and at least one clinical target attribute associated with volumetric modulated arc therapy (VMAT) treatment of a patient; the artificial intelligence model being configured to predict the number of control points and the number of dose calculation sectors for VMAT treatment; the artificial intelligence model having been iteratively trained using a reinforcement learning method, wherein, with each iteration, the artificial intelligence model: sends one or more test control points and one or more test dose calculation sectors to a plan optimizer model, the plan optimizer model being configured to predict a treatment plan; and identifying a reward for the one or more test control points and one or more test dose calculation sectors based on the performance value of the plan optimizer model using the one or more test control points and one or more test dose calculation sectors, wherein the processor iteratively trains the artificial intelligence model based on a strategy of maximizing the reward.

[0012] The method may also include: the processor sending the number of control points and the number of dose calculation sectors for VMAT treatment to the planning optimizer model.

[0013] Rewards can be based on whether the plan optimizer model converges to the predicted treatment plan.

[0014] Rewards can be based on the execution time of the plan optimizer model.

[0015] The artificial intelligence model is configured to predict the degree of association with at least one control point.

[0016] The method may further include: displaying a visual representation of at least one dose calculation sector by a processor.

[0017] The method may also include: displaying a visual representation of at least one control point by the processor.

[0018] In another embodiment, the system may include: a server communicating with an artificial intelligence model, wherein the server is configured to: execute the artificial intelligence model using at least one patient attribute, the number of arcs, and at least one clinical goal attribute associated with volumetric modulated arc therapy (VMAT) treatment of a patient, the artificial intelligence model being configured to predict the number of control points and the number of dose calculation sectors for VMAT treatment, the artificial intelligence model having been iteratively trained using a reinforcement learning method, wherein with each iteration, the artificial intelligence model: sends one or more test control points and one or more test dose calculation sectors to a plan optimizer model, the plan optimizer model being configured to predict a treatment plan; and identifies a reward for one or more test control points and one or more test dose calculation sectors based on the performance value of the plan optimizer model using one or more test control points and one or more test dose calculation sectors, wherein the processor iteratively trains the artificial intelligence model based on a strategy of maximizing the reward.

[0019] The server can also be configured to send the number of control points and the number of dose calculation sectors for VMAT treatment to the plan optimizer model.

[0020] Rewards can be based on whether the plan optimizer model converges to the predicted treatment plan.

[0021] Rewards can be based on the execution time of the plan optimizer model.

[0022] The artificial intelligence model is configured to predict the degree of association with at least one control point.

[0023] The server can also be configured to display a visual representation of at least one dose calculation sector.

[0024] The server can also be configured to display a visual representation of at least one control point.

[0025] In another embodiment, the system may include one or more processors; and a non-transitory memory configured to store computer code instructions that, when executed, cause the one or more processors to: execute an artificial intelligence model using at least one patient attribute, the number of arcs, and at least one clinical goal attribute associated with volumetric modulated arc therapy (VMAT) treatment of a patient, the artificial intelligence model being configured to predict the number of control points and the number of dose calculation sectors for VMAT treatment, the artificial intelligence model having been iteratively trained using a reinforcement learning approach, wherein, with each iteration, the artificial intelligence model: sends one or more test control points and one or more test dose calculation sectors to a plan optimizer model, the plan optimizer model being configured to predict a treatment plan; and identifies a reward for the one or more test control points and the one or more test dose calculation sectors based on the performance value of the plan optimizer model using the one or more test control points and the one or more test dose calculation sectors, wherein the processor iteratively trains the artificial intelligence model based on a strategy of maximizing the reward.

[0026] Computer code instructions can also cause one or more processors to send the number of control points and the number of dose calculation sectors for VMAT treatment to the plan optimizer model.

[0027] Rewards can be based on whether the plan optimizer model converges to the predicted treatment plan.

[0028] Rewards can be based on the execution time of the plan optimizer model.

[0029] Artificial intelligence models can be configured to predict the degree of association with at least one control point.

[0030] The computer code instructions also cause one or more processors to display a visual representation of at least one dose calculation sector. In some embodiments, the AI ​​model at least partially divides the arc based on patient-specific data. In some embodiments, the method includes identifying treatable sectors, for example, to limit the search space of control points. The search space can be limited or modified using attributes of the treatable sectors. The method may include identifying attributes and characteristics of the treatable sectors based on patient-specific data and / or treatment attributes. Attached Figure Description

[0031] Non-limiting embodiments of the present disclosure are described by way of example with reference to the accompanying drawings, which are schematic and not intended to be drawn to scale. Unless indicated as background art, the drawings illustrate aspects of the present disclosure.

[0032] Figure 1 Components of an AI-enabled VMAT arc computing system according to an embodiment are shown.

[0033] Figure 2 A flowchart illustrating the process performed in an AI-enabled VMAT arc computing system according to an embodiment is shown.

[0034] Figure 3 Different dose sectors according to different embodiments are shown.

[0035] Figure 4 Different dose sectors according to different embodiments are shown.

[0036] Figure 5 Different flowcharts of different processes performed in the AI-enabled VMAT arc computing system according to an embodiment are shown.

[0037] Figure 6 Different flowcharts of different processes performed in the AI-enabled VMAT arc computing system according to an embodiment are shown. Detailed Implementation

[0038] Reference will now be made to the illustrative embodiments depicted in the accompanying drawings, and specific language will be used herein to describe the same processes. However, it should be understood that this is not intended to limit the scope of the claims or the present disclosure. Changes and further modifications to the inventive features shown herein, as well as additional applications of the principles of the subject matter shown herein, will be considered within the scope of the subject matter disclosed herein, and such changes and further modifications are likely to be conceived by those skilled in the art and possessing the present disclosure. Other embodiments may be used and / or other changes may be made without departing from the spirit or scope of the present disclosure. The illustrative embodiments described in the detailed description are not intended to limit the subject matter presented.

[0039] As described below, the central server (referred to as the analytics server in this paper) can retrieve data and train AI models using the various methods described in this paper to identify appropriate VMAT arcs. Figure 1 This is a non-restricted example of a system component in which the server operates.

[0040] Figure 1 Components of an AI-enabled VMAT arc computing system (system 100) according to an embodiment are shown. System 100 may include an analytics server 110a, a system database 110b, a VMAT arc computing model 111, end-user devices 120a to 120d (collectively referred to as end-user device 120), and an administrator computing device 150. Figure 1The various components depicted may belong to a radiation therapy clinic where patients can receive radiation therapy via one or more radiation therapy machines (e.g., medical device 140). These components may be connected via network 130. Examples of network 130 may include, but are not limited to, private or public LANs, WLANs, MANs, WANs, and the Internet. Network 130 may employ wired and / or wireless communication according to one or more standards and / or via one or more transmission media.

[0041] Communication on network 130 can be performed according to various communication protocols such as Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), and IEEE communication protocols. In one example, network 130 may include wireless communication according to the Bluetooth specification set or another standard or proprietary wireless communication protocol. In another example, network 130 may also include communication on a cellular network, such as GSM (Global System for Mobile Communications), CDMA (Code Division Multiple Access), or EDGE (Global Evolution Enhanced Data) networks.

[0042] System 100 is not limited to the components described herein, and may include additional or other components not shown for brevity, which will be considered within the scope of the embodiments described herein.

[0043] The analytics server 110a can generate and display an electronic platform configured to identify attributes of the RTTP used for patient treatment using various computer models 111 to 112. The electronic platform may include a graphical user interface (GUI) displayed on an end-user device 120 and / or an administrator computing device 150. Examples of electronic platforms generated and hosted by the analytics server 110a may be web-based applications or websites configured to be displayed on different types of electronic devices, such as mobile devices, tablets, personal computers, etc.

[0044] In a non-limiting example, a physician operating physician device 120b can access the platform, input patient attributes, characteristics, and other data, and further instruct analysis server 110a to generate RTTP (including beam angle). Additionally or alternatively, analysis server 110a can optimize only one specific attribute of the RTTP (e.g., only the beam angle).

[0045] Analysis server 110a can recommend RTTPs (e.g., beam angles and other radiation parameters and / or treatment plan attributes) for proton, photon, and electron radiation. Specifically, analysis server 110a can use the methods and systems described herein to execute VMAT arc calculation model 111 to identify treatable sectors. Analysis server 110a can then use the generated treatable sectors to limit the search space used by the plan optimizer model 112. Furthermore, analysis server 110a can send beam angles and other radiation parameters and / or treatment plan attributes to one or more other servers. Additionally or alternatively, analysis server 110a (or another server) can adjust the configuration of one or more devices (e.g., medical device 140) based on the optimized beam angle. For example, RTTPs can be sent directly to medical device 140 and / or to computer 142, which functionally controls medical device 140.

[0046] Medical device 140 can be any medical device used for radiotherapy treatment of a patient (such as a CT scanner, radiotherapy machine (e.g., a linear accelerator, particle accelerator (including a circular accelerator), or a cobalt machine)). Medical device 140 may also include imaging equipment capable of emitting radiation, enabling it to acquire images using various methods to accurately image the patient's internal anatomy. For example, medical device 140 may include a rotational system (e.g., a static or rotating multi-view system). Non-limiting examples of multi-view systems may include stereoscopic systems (e.g., two systems arranged orthogonally).

[0047] Analysis server 110a can host websites accessible to users (e.g., end users or medical professionals) operating any electronic device described herein, where the content presented via various web pages can be controlled based on each specific user's role or viewing permissions. Analysis server 110a can be any computing device including a processor and non-transitory machine-readable storage capable of performing the various tasks and processes described herein. Analysis server 110a can employ various processors, such as central processing units (CPUs) and graphics processing units (GPUs). Non-limiting examples of such computing devices can include workstation computers, laptop computers, server computers, etc. While system 100 includes a single analysis server 110a, analysis server 110a can include any number of computing devices operating in a distributed computing environment such as a cloud environment.

[0048] Analysis server 110a can execute software applications configured to display electronic platforms (e.g., hosting websites), which can generate various web pages and serve them to each end-user device 120. Different users can use the website to view and / or interact with recommended (optimized) results. Different servers, such as clinic server 120c, can also use the recommended results in downstream processing.

[0049] Analysis server 110a can be configured to require user authentication based on a set of user authorization credentials (e.g., username, password, biometrics, password certificate, etc.). Analysis server 110a can access system database 110b configured to store user credentials, and analysis server 110a can be configured to refer to the user credentials to determine whether the input set of credentials (purely identifying the user) matches the appropriate set of credentials for the authenticated user.

[0050] Analysis server 110a can generate and host web pages based on the roles of specific users within system 100. In such an implementation, a user's role can be defined by a data field and an input field in user records stored in system database 110b. Analysis server 110a can authenticate users and identify user roles by executing access directory protocols (e.g., LDAP). Analysis server 110a can generate web page content customized according to the user roles defined by user records in system database 110b.

[0051] Analysis server 110a can receive various clinical goals, patient data, and treatment data from end-user device 120. For example, a physician can access the platform provided by analysis server 110a using physician device 120b. The physician can use one or more input elements of the platform to input various patient attributes and / or clinical goals. Analysis server 110a can then perform the various methods discussed herein and display RTTP on the platform.

[0052] End-user device 120 can be any computing device including a processor and a non-transitory machine-readable storage medium capable of performing the various tasks and processes described herein. Non-limiting examples of end-user device 120 may include workstation computers, laptop computers, tablet computers, and server computers. During operation, various users can use end-user device 120 to access a GUI operated by analytics server 110a. Specifically, end-user device 120 may include clinic computer 120a, doctor device 120b, clinic server 120c, or clinic database 120d.

[0053] In order to generate RTTP, the analysis server 110a can then execute various models (e.g., VMAT arc calculation model 111 and / or plan optimizer model 112) to analyze the retrieved / received data.

[0054] Administrator computing device 150 may represent a computing device operated by a system administrator. Administrator computing device 150 may be configured to display beam angles, radiation parameters, and / or other radiotherapy treatment attributes generated by analysis server 110a; monitor VMAT arc calculation model 111 used by analysis server 110a and / or end-user device 120; review feedback; and / or facilitate the training or retraining (calibration) of models 111 to 112 maintained by analysis server 110a.

[0055] Analysis server 110a can communicate (in real-time or near real-time) with medical device 140 (or its computer 142), enabling the server / computer hosting medical device 140 to adjust medical device 140 based on RTTP (e.g., beam angle, treatment attributes, and / or radiation parameters determined by analysis server 110a). For example, a radiotherapy machine can adjust its gantry, beam blocking device (e.g., multi-leaf collimator MLC), and treatment bed based on an optimized beam angle, where the optimized beam angle is the angle at which medical device 140 emits radiation in the direction of PTV. Analysis server 110a can send instructions to the radiotherapy machine indicating any number or type of radiation parameters and / or treatment attributes to facilitate this adjustment.

[0056] VMAT arc computation models 111 and / or 112 can represent any set of algorithmic logic and / or artificial intelligence models (e.g., using various machine learning techniques). For example, VMAT arc computation model 111 can include various algorithms to identify the attributes and characteristics of treatable sectors based on patient data and / or treatment attributes. Using VMAT arc computation model 111, analysis server 110a can execute plan optimizer model 112. For example, analysis server 110a can execute models in combination with each other and then modify one or more configuration parameters of plan optimizer 112. Non-limiting examples of configuration parameters modified by analysis server 110a can include limiting / modifying the search space using the attributes of treatable sectors. As a result, the efficiencies discussed herein can be achieved without modifying plan optimizer 112 itself. For example, VMAT arc computation model 111 can be executed independently and used in conjunction with existing plan optimizers. Therefore, the methods and systems discussed herein can be used to retrofit plan optimizers (or system infrastructure). This minimal disruption to existing infrastructure allows for improvements to the plan optimizer without modifying its source code.

[0057] In some embodiments, the analysis server 110a can... Figure 1 Patient data is collected from various sources, not shown. For example, analysis server 110a can monitor and collect patient and treatment attributes associated with previously treated radiotherapy patients to include in the training dataset. Additionally or alternatively, analysis server 110a can collect / aggregate other patient data (e.g., data associated with how a patient set or clinical participants received their respective radiotherapy treatments). Using the collected data, analysis server 110a can generate a training dataset, which it can then use to train the VMAT arc calculation model 111. Thus, the VMAT arc calculation model 111 can be used to limit the search space used by the planning optimizer 112.

[0058] Figure 2 A flowchart illustrating a process performed in a plan generation system according to an embodiment is shown. Method 200 includes steps 220 through 250. However, other embodiments may include additional or alternative execution steps, or one or more steps may be omitted entirely. An analysis server may train an AI model before implementing / executing the AI ​​model to predict appropriate dose sectors. Method 200 describes how the AI ​​model can be trained (210) and then executed (250). Method 210 is described as being executed by the AI ​​model. However, one or more steps of method 200 may also be performed by... Figure 1 The distributed computing system described herein can be operated by any number of computing devices to execute. For example, one or more user computing devices can execute locally. Figure 2 Some or all of the steps described in the document.

[0059] Method 200 may include two phases. First, the analytics server may train an AI model using a method such as method 210, which includes steps 220 through 240. Once training is complete, the analytics server may then deploy the model by executing the model (step 250).

[0060] At step 220, the analysis server may send one or more test control points and one or more test dose calculation sectors to the plan optimizer model configured to predict treatment plans.

[0061] The analytics server can use reinforcement learning methods, where an agent can be trained in conjunction with another model. Specifically, the agent (or AI model) can be trained by computing rewards for different chosen DCS and control points, where the rewards correspond to a secondary model, such as a plan optimizer model. For example, the analytics server could first retrieve patient data and then identify the rewards associated with different attributes (e.g., different DCS and / or control points) by sending different iterations of values ​​to the plan optimizer and determining how the sent values ​​affect the operation of the plan optimizer and / or the overall RTTP.

[0062] The analytics server can retrieve (via the platform and / or by querying a database) patient data required to calculate a treatment plan for a patient. Therefore, as used herein, patient data can refer to any data associated with a patient (e.g., the patient's physical data, diagnostic data, and any data entered by a healthcare professional). In some embodiments, the analytics server may use different patient identifiers to retrieve patient data. For example, the analytics server may query one or more databases to identify medical data associated with a patient. The analytics server may query data associated with the patient's anatomy, such as physical data (e.g., height, weight, and / or body mass index) and / or other health-related data (e.g., blood pressure or other data related to receiving radiation therapy). The analytics server may also retrieve data associated with the patient's current and / or previous medical treatments (e.g., data associated with the patient's previous surgeries).

[0063] In some embodiments, the analytics server can also analyze a patient's medical data records to identify desired patient attributes. For example, the analytics server can query a database to identify a patient's BMI. However, because many medical records are not digitized, the analytics server may not be able to receive a patient's BMI value using simple query techniques. As a result, the analytics server can retrieve the patient's electronic health data and can execute one or more analytics protocols (e.g., natural language processing) to identify the patient's body mass index. In another example, if the analytics server does not receive tumor data (e.g., endpoints), the analytics server can execute various image recognition protocols and identify the tumor data.

[0064] The analytics server can also receive additional data from one or more healthcare professionals. For example, a therapeutic oncologist can access a platform generated / hosted by the analytics server and can add, remove, or modify patient-specific data such as patient attributes, treatment attributes, tumor attributes, primary treatment site, tumor stage, endpoint, and whether the primary tumor has expanded. Because tumor staging and endpoint attributes are critical information influencing patient treatment, this information is typically input by the therapeutic oncologist. In some embodiments, an AI model (e.g., a separate AI model trained to identify tumor information) can identify this information, and the therapeutic oncologist can reject, approve, or modify the predicted outcome. In another example, the therapeutic oncologist can explicitly indicate whether treatment should be unilateral or bilateral.

[0065] Another example of patient data / attributes may include a specific tumor location. More specifically, this data may indicate the location of the primary tumor relative to the patient's centerline. This data may be input by the treating oncologist or analyzed using various image recognition or segmentation methods performed on the patient's medical images. In some embodiments, if this information is not input by the treating oncologist (or is otherwise received by an analytics server), it may also be used to predict using an AI model. Another patient attribute may indicate whether and how close the tumor is to other non-disease organs. For example, a tumor to be eradicated may be a few millimeters away from another organ. This information may alter the field geometry (generated by the optimizer) because other organs must be avoided.

[0066] The analytics server can deploy a patient plan optimizer to generate RTTPs for each patient based on patient data. However, to generate RTTPs, the plan optimizer can also use CP attributes and DCS (Distributed Cost Scheme) values. Therefore, to train the model, the analytics server and / or the AI ​​model's agent can first send test CP attributes and / or test DCS attributes to the plan optimizer. The plan optimizer can then use the test values ​​to generate RTTPs. The AI ​​model can monitor the plan optimizer's operations, enabling it to measure the test values ​​sent to the plan optimizer.

[0067] In some embodiments, a single test control point and / or a single test dose calculation sector may be sent to the planning optimizer. Alternatively, more than one (multiple) test control points and / or more than one test dose calculation sector may be sent at a time. Therefore, in some embodiments, batch test values ​​can be performed to obtain more timely results.

[0068] At step 230, the reward for the one or more test control points and the one or more test dose calculation sectors is identified based on the performance value of the plan optimizer model using one or more test control points and one or more test dose calculation sectors. At step 240, the processor iteratively trains the policy of the artificial intelligence / machine learning model until the policy satisfies an accuracy threshold based on maximizing the reward.

[0069] As discussed in this paper, training agents can use reinforcement learning methods to train themselves / AI models. As used herein, "agent" refers to a learner or trainer (e.g., an analytics server training an AI model or the AI ​​model itself). When training an AI model, the agent can use reinforcement learning techniques to determine when and which data points are used for training purposes. Therefore, the agent can choose between different actions and receive rewards based on the chosen actions. Using this approach, the agent does not need a large training dataset, thus eliminating the need for data labeling.

[0070] Because the agent is using reinforcement learning, it can choose multiple actions (e.g., simultaneously) and compute the rewards corresponding to them. The agent can then compute at least one action (or sequence of actions) that yields the maximum reward. Through multiple iterations, the agent can learn the optimal action to take.

[0071] In the reinforcement learning approach described in this paper, the agent can interact with a potentially unknown environment (e.g., data used in conjunction with an optimizer). Therefore, the agent can perform different actions and receive corresponding rewards (e.g., the agent performs actions on the environment and receives new observations and rewards from the environment). The agent can then train itself by selecting (multiple) actions that provide the highest set of cumulative rewards. Throughout the sequential decision-making process, the agent learns a strategy that maximizes the cumulative reward. To achieve the best results at the end of the training process, the agent learns how to optimally determine the sequence of actions.

[0072] Throughout the training process, the agent can develop a policy (also known as the agent's strategy). A policy is a mapping from the observation space to the action space (represented by various internal algorithms and mathematical formulas). The optimal policy guarantees receiving the most (or best-level) reward at the end of the training process. Because the agent considers multiple variables, the policy dictates how best the agent must decide to take actions (or sequences of actions) such that when calculating rewards using the considered variables, the agent receives a higher (or highest) reward. For example, the agent might sacrifice one reward to obtain a higher cumulative total reward. By taking actions and calculating rewards, the agent can interact with the observation space (e.g., an unknown environment with data points that are not necessarily constrained for the agent) and learn from experience. Specifically, the agent learns by taking actions and analyzing the received rewards (e.g., reconfiguring its policy).

[0073] In the example, the agent can train itself using the action reward system described in this paper. For example, the agent learns from the environment (s t The agent receives the current observation. Then, the agent bases its current policy (π) on the observation. t ) Performing an action (a t This strategy is also called a policy or agent's policy. During the execution of an action, the agent receives new observations (s...). t+1 And receive rewards (r) t The agent then updates its policy to maximize the cumulative sum of future rewards. In this example, the environment follows a Markov decision process:

[0074] P[s t+1 |s t ]=P[s t+1 |s1,...s t ].

[0075] When determining which action yields the greatest reward, the agent can use either an exploratory or exploitative approach. The exploratory approach, as used in this paper, allows the agent to train the AI ​​model and improve its knowledge of each data point and its ability to predict accurate outcomes. Conversely, the exploitative approach selects greedy actions (with limitations below) and seeks the maximum possible reward by utilizing the agent's current action value estimate. Therefore, when the agent uses the exploratory approach, it achieves a more accurate estimate of action value.

[0076] Analysis servers can use the epsilon-greedy method to train AI models. As used in this paper, epsilon-greedy refers to an AI training method that balances exploration and exploitation. In the epsilon-greedy method, the agent can first calculate the epsilon(ε) value. The ε value can refer to the probability that the agent chooses to explore or exploit a data point or action. Therefore, the ε value represents the ratio of the probability of how the agent explores to the probability of how the agent exploits. In the example, the ε value can be set to 50% or 0.5. In this example, the agent explores 50% of the time and exploits 50% of the time. Conversely, an ε value of 20% or 0.2 indicates that the agent explores 20% of the time and exploits 80% of the time (and vice versa, depending on how the epsilon-greedy algorithm is constrained).

[0077] The analysis server can determine an ε value associated with a data point. If the ε value meets a threshold, the analysis server can generate random, dependent results associated with the analyzed individual data points. The actions taken by the analysis server in this embodiment are also referred to herein as the exploration phase.

[0078] Using the reinforcement learning methods discussed herein, an agent can train itself to predict DCSs that produce higher rewards than other DCSs. For example, the agent can compute rewards for the test CP attribute and test DCS attribute sent to the plan optimizer (at step 220). The reward can be a quantitative evaluation of the plan optimizer's performance. For example, the reward can correspond to the speed at which the plan optimizer converges on the RTTP. In another example, the reward can correspond to the quality of the RTTP. In other embodiments, the reward can correspond to any other attribute of the RTTP and / or the plan optimizer's operating parameters.

[0079] Conventional software solutions / models typically divide the arc field into multiple equally sized sectors according to predetermined rules and thresholds, which can be hard-coded into the computer model or other algorithms. For example, the rules might indicate that each DCS will have 16 CPs at MR level 0, 8 CPs at level 1, 4 CPs at level 2, and 2 CPs at the final MR level 3. This can be performed regardless of patient-specific data or the treatment cases at hand. Furthermore, the positions of the sectors in the arc can always remain the same; a subset of DCS can then be selected (sometimes randomly) in different iterations and can be replaced with the original DCS. Typically, to ensure the reproducibility of the optimization process, the same random seed is used, which results in the selection of the same set of sectors for all treatments. A predetermined scheduler can decide (using various hard-coded algorithms, which can be static) whether to continue with the determined MR level, for example, based on the convergence state of the optimizer or a threshold of the maximum number of iterations allowed for each MR level. While such strategies may provide reasonable results for some treatment cases, they may also lead to suboptimal solutions with slower convergence rates for other treatment cases.

[0080] Figure 3 A representation of VMAT arc optimization is depicted. In the depicted embodiment, arc field 300 is represented as being divided into 360 degrees from DCS 302 to DCS 324. Each arc sector can be divided into different CPs (e.g., CPs are designated every 2 degrees) using a predetermined algorithm. In some embodiments, intermediate CPs can be identified as primary CPs. For example, the processor can identify 302A to 324A as primary CPs.

[0081] Figure 4 A representation of VMAT arc optimization using an AI model trained using methods and systems discussed herein (such as method 210) is depicted. In the depicted embodiment, the arc field 400 is represented as being divided into 360 degrees of DCS 402 to DCS 324 having 402A to 418A. The arc field 400 can be divided using computer models discussed herein. For example, using methods and systems discussed herein (such as method 210) Figures 1 to 2 The trained computer model can generate the boundaries of DCS 402 to DCS 418. As depicted, the DCSs are not generated uniformly. For example, DCS 414 has a different size than other DCSs (such as DCS 402 or DCS 408). This is because the analytics server can use AI training to intelligently generate DCSs (tailored to the patient) rather than dividing them into DCSs of equal size (e.g., Figure 3 ).

[0082] Figure 5The representation of training the AI ​​model discussed in this paper is depicted. Using method 500, the analysis server can train the AI ​​model (e.g., AI model 530). When trained, AI model 530 can determine / predict MR rank (524), intelligently partition the field into different DCSs (526), ​​and select a subset of DCSs (528). Method 500 can include reinforcement learning methods because these methods do not require the analysis server to collect and process the training dataset. Monitoring various properties of radiotherapy, collecting data, processing data, and labeling data to train an AI model can be a time-consuming effort requiring high computational power.

[0083] Figure 5 This paper describes how a reinforcement learning agent is integrated into the VMAT arc optimizer to optimize and intelligently predict DCS. During the training process, the agent can learn by trying multiple sector settings for each specific treatment case in the training dataset. The agent's observations may include the arc field (510) and the dose gradient (508), as well as the calculated reward (518) towards the planning goal (504) and / or treatment / patient data 502 at different iterations. The agent can provide a non-overlapping DCS candidate set (522) to be used to update the dose specified by the planning optimizer (516). Each sector can be specified by the start CP number and end CP number within the arc. The agent can receive a reward (506) based on the reduction of the optimization cost function for the provided DCS candidate set. That is, the reward can also be evaluated and observed according to the cost function 506, which indicates how certain attributes negatively affect the patient's treatment plan (e.g., radiointerference OAR). As training continues, the agent can learn an optimal strategy by maximizing the cumulative sum of rewards. Then, the agent can test situations that it cannot see in the trial sector and evaluate its generalization ability.

[0084] With each iteration, the agent can optimize one or more parameters or properties of how the DCS is predicted. As used herein, dynamically optimized parameters can refer to optimization parameters that can change from one iteration to another. Non-limiting examples of optimization parameters may include target flux plot (512), leaf and MU value optimization (514), dose gradient 508, number of iterations, convergence rate, etc. At each iteration, the AI ​​model 530 can observe the feature set extracted from the data and can provide a set of non-overlapping, optimally sized DCSs with favorable / suitable locations. In some embodiments, the AI ​​model 530 may not necessarily maintain any predetermined set of DCSs, but the non-overlapping DCSs can have different sizes and be located anywhere within the arc field. This allows for a fast and comprehensive search in the solution space of the CP parameters.

[0085] Training the AI ​​model can also include a separate training phase where the agent observes and experiments with various possible sector settings (e.g., different permutations of DCS segments) and can receive reward signals specifying the success rate of the DCS debugging task. Upon learning the optimal policy, reward signals may no longer be needed to execute the AI ​​model 530. As a result, the agent can perform tasks without any iterative loops, indicating that the AI ​​model 530 is ready for deployment.

[0086] As an example, considering different examples of the RapidArc algorithm, observations may include the target throughput map, the history of previously selected sectors, the history of how sector leaf sequencing has found improved leaf sequences, organ geometry (3D or projected images), and / or dosimetry status (e.g., how well clinical goals are met). The reward used to train the agent may be one or a combination of convergence speed and solution quality relative to some baseline optimization results.

[0087] The AI ​​model can be trained in conjunction with the planning optimizer 530. Therefore, the AI ​​model can be implemented in a way that does not interfere with the operation of the planning optimizer. How the DCS is generated (e.g., how the entire field is divided into different DCSs) can also affect the optimizer's operation. For example, depending on how the DCS is generated and partitioned (at least initially), the optimizer may take different paths. Therefore, providing new data for training the AI ​​model during the optimizer's operation is beneficial for training the AI ​​model itself. Furthermore, the optimizer can use an iterative process, and reinforcement learning can provide opportunities to train the AI ​​model while the optimizer executes this process.

[0088] Return to reference Figure 2 At step 250, the analysis server may use at least one patient attribute, number of arcs, and at least one clinical target attribute associated with the patient’s VMAT treatment to execute an artificial intelligence model configured to predict the number of control points and the number of dose calculation sectors for VMAT treatment.

[0089] Using the methods and systems discussed herein, AI models can delegate / predict DCS (Distributed Control System) accordingly using patient-specific data. Examples of patient data may include structural volume, dose, planning targets, non-target organs, dose-related calculations (e.g., radiation dose distribution within the patient's anatomical region), etc. An analytics server can retrieve patient identifiers, the patient's electronic health data record, medical images (e.g., CT scans, 4DCT scans, MRI and X-ray images), treatment-specific data (e.g., arc information or treatment type), target organs (e.g., specification and location data for identifying the tumor to be eradicated), and then transmit the retrieved data to a trained AI model.

[0090] In a non-limiting example, the analytics server can retrieve patient data and other treatment goals (e.g., the number of arcs and other clinical objectives). The analytics server can then execute a trained AI model and receive / predict the number and / or various attributes associated with the CP. As used herein, the attributes of the CP can indicate how and where the CP should be placed. For example, attributes can indicate the degree associated with each CP. Furthermore, the trained AI model can indicate how the DCS should be segmented (e.g., the start and end degrees of each DCS). The predicted data can then be displayed to the clinician. For example, an arc field can be displayed, indicating different CPs and DCS, such as... Figure 4 Alternatively, the predicted data can be ingested by the planning optimizer, so that the planning optimizer no longer selects CP and DCS randomly or using predetermined (and usually static) rules.

[0091] Now for reference Figure 6 This document illustrates a non-limiting visual example of a workflow utilizing the methods and systems described herein. In this non-limiting example 600, an analysis server provides dose calculation sector data to a planning optimizer 630 to generate a recommended RTTP optimized for the patient and their treatment. The analysis server may first collect patient data 610. Patient data may include patient anatomical data 610a (e.g., medical images, PTV, OAR), user input 610b (clinical goals or rules received from the treating oncologist via a user interface, such as tumor data, PTV identifiers, etc.). Other non-limiting examples of clinical goals may include dosimetry goodness functions, robustness measures, biological effects of radiation, measures of linear energy transfer, etc. Patient data 610 may also include rules 610c for patient treatment (e.g., clinical / treatment goals or standards identified by a medical professional, or any other specific treatment required by the medical professional).

[0092] In some configurations, the analysis server can access the patient's internal / external files and retrieve / extract the required patient data 610. The analysis server can then execute a dose calculation sector model 620 to identify various attributes of one or more dose calculation sectors of the patient using the patient data 610. The results generated via the dose calculation sector model 620 can be ingested by a treatment planning optimizer 630. The treatment planning optimizer 630 can be a treatment planning and / or monitoring software solution. The treatment planning optimizer 630 can analyze various factors associated with the patient and the patient's treatment to generate and optimize the patient's RTTP (e.g., the field geometry, treatment mode, and radiation parameters required to treat the patient).

[0093] One of the factors considered by the planning optimizer 630 can be the dose calculation sector attribute identified by the dose calculation sector model 620. For example, using the dose calculation sector attribute, the planning optimizer 630 can limit its search space. As a result, the planning optimizer 630 can calculate the final beam geometry that conforms to the values ​​predicted by the dose calculation sector model 620.

[0094] The treatment plan optimizer 630 can iteratively modify the patient's RTTP, whereby the optimizer 630 iteratively modifies different attributes of the RTTP (e.g., field geometry). In some configurations, the optimizer 630 can send new treatment plan data back to the dose calculation sector model 620, whereby the dose calculation sector model 620 can recalculate / repredict data based on the revised treatment data generated by the optimizer (iteration 622). The optimizer 630 and the dose calculation sector model 620 can repeat iteration 622 until the patient's RTTP is optimized.

[0095] When the treatment plan optimizer 630 completes the patient's RTTP, it can send a suggested treatment plan 640 to one or more electronic devices, where a user (e.g., a clinician) can view the suggested plan. For example, the suggested treatment plan 640 (or any attributes predicted by the dose calculation sector model 620) can be displayed on a clinic computer, where radiation therapy technicians or treatment oncologists can view the data.

[0096] The various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps have been described above in general terms of their functionality. Whether this functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of this disclosure or the claims.

[0097] Implementations in computer software can be implemented in software, firmware, middleware, microcode, hardware description languages, or any combination thereof. Code segments or machine-executable instructions can represent procedures, functions, subroutines, programs, routines, subroutines, modules, software packages, classes, or any combination of instructions, data structures, or program statements. Code segments can be coupled to other code segments or hardware circuitry by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., can be passed, forwarded, or transmitted via any suitable component, including memory sharing, message passing, token passing, network transmission, etc.

[0098] The actual software code or dedicated control hardware used to implement these systems and methods does not limit the claimed features or this disclosure. Therefore, since the operation and behavior of systems and methods are described without reference to any specific software code, it should be understood that the software and control hardware can be designed to implement the systems and methods based on the description herein.

[0099] When implemented in software, functionality can be stored as one or more instructions or code on a non-transitory computer-readable or processor-readable storage medium. The steps of the methods or algorithms disclosed herein can be embodied in a processor-executable software module, which can reside on a computer-readable or processor-readable storage medium. Non-transitory computer-readable or processor-readable media include both computer storage media and tangible storage media that facilitate the transfer of computer programs from one place to another. Non-transitory processor-readable storage media can be any available medium accessible to a computer. By way of example and not limitation, such non-transitory processor-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, disk storage devices or other magnetic storage devices, or any other tangible storage medium that can be used to store desired program code in the form of instructions or data structures and is accessible to a computer or processor. As used herein, disks and optical discs include compact optical discs (CDs), laser discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media. Additionally, the operation of a method or algorithm may reside as one or any combination or set of code and / or instructions on a non-transient processor-readable medium and / or a computer-readable medium, which may be incorporated into a computer program product.

[0100] The foregoing description of the disclosed embodiments is provided to enable those skilled in the art to make or use the embodiments described herein and variations thereof. Various modifications to these embodiments will readily be apparent to those skilled in the art, and the principles defined herein can be applied to other embodiments without departing from the spirit or scope of the subject matter disclosed herein. Therefore, this disclosure is not intended to be limited to the embodiments shown herein, but is accorded the widest scope consistent with the appended claims and the principles and novel features disclosed herein.

[0101] While various aspects and embodiments have been disclosed, other aspects and embodiments are contemplated. The disclosed aspects and embodiments are for illustrative purposes and are not intended to be limiting; their true scope and spirit are indicated by the appended claims.

Claims

1. A method comprising: An artificial intelligence model is executed by a processor using at least one patient attribute, the number of arcs, and at least one clinical target attribute associated with a patient's volumetric modulated arc therapy (VMAT) treatment. The AI ​​model is configured to predict the number of control points and the number of dose calculation sectors for the VMAT treatment. The AI ​​model has been iteratively trained using reinforcement learning methods, wherein, with each iteration, the AI ​​model: One or more test control points and one or more test dose calculation sectors are sent to the treatment plan optimizer model, which is configured to predict treatment plans. as well as The reward for the calculated sector is identified based on the performance value of the plan optimizer model using the one or more test control points and the one or more test doses. The processor therein employs a strategy of iteratively training the artificial intelligence model based on maximizing the reward.

2. The method according to claim 1, further comprising: The processor sends the number of control points and the number of dose calculation sectors for the VMAT treatment to the planning optimizer model.

3. The method according to claim 1 or claim 2, wherein the reward is based on whether the plan optimizer model converges to the predicted treatment plan.

4. The method according to any one of the preceding claims, wherein the reward is based on the execution time of the plan optimizer model.

5. The method according to any one of the preceding claims, wherein the artificial intelligence model is configured to predict the degree of association with at least one control point.

6. The method according to any one of the preceding claims further comprises: The processor displays a visual representation of at least one dose calculation sector.

7. The method according to any one of the preceding claims further comprises: The processor displays a visual representation of at least one control point.

8. A system comprising: A server that communicates with an artificial intelligence model, wherein the server is configured as follows: The artificial intelligence model is executed using at least one patient attribute, the number of arcs, and at least one clinical target attribute associated with the patient's volumetric modulated arc therapy (VMAT) treatment. The AI ​​model is configured to predict the number of control points and the number of dose calculation sectors for the VMAT treatment. The AI ​​model has been iteratively trained using reinforcement learning methods, wherein, with each iteration, the AI ​​model: One or more test control points and one or more test dose calculation sectors are sent to the treatment plan optimizer model, which is configured to predict treatment plans. as well as The reward for the plan optimizer model is identified based on the performance value of the plan optimizer model using the one or more test control points and the one or more test dose calculation sectors. The processor therein employs a strategy of iteratively training the artificial intelligence model based on maximizing the reward.

9. The system of claim 8, wherein the server is further configured to: The number of control points and the number of dose calculation sectors for the VMAT treatment are sent to the planning optimizer model.

10. The system of claim 8 or claim 9, wherein the reward is based on whether the plan optimizer model converges to the predicted treatment plan.

11. The system according to any one of claims 8 to 10, wherein the reward is based on the execution time of the plan optimizer model.

12. The system according to any one of claims 8 to 11, wherein the artificial intelligence model is configured to predict the degree of association with at least one control point.

13. The system according to any one of claims 8 to 12, wherein the server is further configured to: Display a visual representation of at least one dose calculation sector.

14. The system according to any one of claims 8 to 13, wherein the server is further configured to: Display a visual representation of at least one control point.

15. A system comprising: One or more processors; as well as Non-transitory memory, configured to store computer code instructions that, when executed, cause the one or more processors to: An artificial intelligence (AI) model is executed using at least one patient attribute, the number of arcs, and at least one clinical target attribute associated with a patient's volumetric modulated arc therapy (VMAT) treatment. The AI ​​model is configured to predict the number of control points and the number of dose calculation sectors for the VMAT treatment. The AI ​​model has been iteratively trained using reinforcement learning methods, wherein, with each iteration, the AI ​​model: One or more test control points and one or more test dose calculation sectors are sent to the treatment plan optimizer model, which is configured to predict treatment plans. as well as The reward for the calculated sector is identified based on the performance value of the plan optimizer model using the one or more test control points and the one or more test doses. The processor therein employs a strategy of iteratively training the artificial intelligence model based on maximizing the reward.

16. The system of claim 15, wherein the computer code instructions further cause the one or more processors to: The number of control points and the number of dose calculation sectors for the VMAT treatment are sent to the planning optimizer model.

17. The system of claim 15 or claim 16, wherein the reward is based on whether the plan optimizer model converges to the predicted treatment plan.

18. The system according to any one of claims 15 to 17, wherein the reward is based on the execution time of the plan optimizer model.

19. The system according to any one of claims 15 to 18, wherein the artificial intelligence model is configured to predict the degree of association with at least one control point.

20. The system according to any one of claims 15 to 19, wherein the computer code instructions further cause the one or more processors to: Display a visual representation of at least one dose calculation sector.

Images