System and method for automatic radiotherapy treatment plan generation using machine learning language processing model

Abstract

Embodiments of the present disclosure relate to systems and methods for automated radiotherapy treatment plan generation using a machine learning language processing model. Embodiments described herein provide radiation therapy plan generation using a machine learning language processing model. The processor may present a user interface that provides an interactive interface between the user and the machine learning language processing model. The processor may receive a first input including a first patient attribute of a patient. The processor may execute a machine learning language processing model using the first patient attribute as input to generate a response requesting the second patient attribute. The processor may present a response requesting a second patient attribute of the patient. The processor may receive a second input including a second patient attribute of the patient. The processor may transmit a first patient attribute of the patient and a second patient attribute of the patient to a radiation therapy plan optimizer. The radiation therapy plan optimizer may be configured to generate a radiation therapy plan for a patient.

Description

Technical Field

[0001] The present application generally relates to using machine learning language processing models to generate radiation therapy treatment plans. Background Art

[0002] Radiation therapy (or radiotherapy) treatment planning (RTTP) is a complex process involving the use of specific guidelines, protocols and instructions by different medical professionals (such as clinicians, medical device manufacturers, etc.). Typically, identification and application of guidelines for implementing radiotherapy treatments are performed by complex computer models that receive treatment goals from the attending physician and identify appropriate properties of the RTTP. For example, the attending physician can identify the treatment mode (e.g., choosing between volume modulated arc therapy (VMAT) or intensity modulated radiation therapy (IMRT)). The attending physician can then input various goals and objectives to be achieved via treatment, such as dose targets to be achieved for one or more structures of the patient. The software solution can then use various methods to calculate the properties of the patient's treatment, such as determining beam limiter angles and radiation emission properties. In the case of IMRT, the beam transmission direction and the number of beams are specific related variables that must be determined, while for VMAT, the software solution may need to select the number of arcs and their corresponding start and end angles.

[0003] The creation of an RTTP often involves multiple professionals collaborating with different skill sets. Particularly important in this process is the interaction between the radiation oncologist and a plan generator, which can be a computer model configured to automatically generate a radiation treatment plan for a patient based on patient attributes. It is common that after an oncologist first generates a treatment plan using such a computer model, the oncologist may perform multiple additional rounds of interaction with the computer model because there were ambiguities in the original request, aspects of the plan need to be improved, or the oncologist wants to verify whether a better plan is available. Summary of the invention

[0004] The computer model may require specific (mostly numerical) precise parameters as input to generate a radiation therapy treatment plan for a patient. Such parameters may be initially input by a user (e.g., an oncologist or other clinician) and then adjusted multiple times based on interactions between the patient and the user. The computer model may generate a new radiation therapy treatment plan for each adjustment and execution.

[0005] Computer models are needed to generate treatment properties of radiation therapy treatment plans using methods and systems that reduce the repetitiveness of multiple iterations of generating and updating radiation therapy treatment plans. Improved AI modeling / training techniques are also needed to train such models that do not require large training data sets and are computationally efficient, cost-effective, and produce timely results.

[0006] Using the methods and systems discussed herein, a processor can use a machine learning language processing model (e.g., a large language model) to automatically generate a radiation therapy treatment plan. The processor can provide an interactive interface (e.g., an interactive interface) to a computing device accessed by a user (e.g., an oncologist or other clinician), wherein the user can provide input (e.g., a prompt) indicating different patient attributes of the patient, which can be used to generate a radiation therapy treatment plan. The patient attribute can be data related to the patient or the patient's tumor, data related to the past process experienced by the patient, data characterizing or describing the patient or the patient's tumor, or data characterizing the treatment options (e.g., available treatment options) of the patient or the patient's tumor. Examples of such patient attributes can include, but are not limited to, potential methods for treating the patient, radiotherapy machine attributes or treatment attributes that can be used to treat the patient (e.g., machine motion and / or dose attributes of the treatment, treatment frequency, time period between treatments, etc.), characteristics about the patient (e.g., age, sex, height, weight, etc.), characteristics about the patient's tumor (e.g., location, size, width, etc.), the patient's treatment history, etc. The processor can receive the input patient attributes and use the patient attributes as input to execute the machine learning language processing model to generate a response. In certain specific cases, the response can be a further query of the patient attributes. The processor may compare the received patient attributes to templates, each template including a different set of patient attributes that can be used to generate a radiation therapy treatment plan. In response to determining that the patient attributes of the patient satisfy the conditions of the template (e.g., determining that there is sufficient data to generate a radiation therapy plan for the patient), the processor may input the patient attributes into a radiation therapy plan optimizer to generate a radiation therapy treatment plan for the patient. The processor may transmit the generated radiation therapy treatment plan back to a user accessing the interactive interface. Thus, a processor implementing the systems and methods described herein may automatically generate an optimized radiation therapy treatment plan using a machine learning language processing model without iteratively updating or generating a new radiation therapy treatment plan.

[0007] In an embodiment, the method may include presenting a user interface by a processor, which provides an interactive interface between a user and a machine learning language processing model; receiving a first input including a first patient attribute of a patient from the interactive interface by the processor; executing the machine learning language processing model by the processor using the first patient attribute of the patient as input to generate a response requesting a second patient attribute of the patient; presenting the response requesting the second patient attribute of the patient at the interactive interface by the processor; receiving a second input including the second patient attribute of the patient from the interactive interface by the processor; and in response to determining that the first patient attribute and the second patient attribute satisfy a planning condition, transmitting the first patient attribute of the patient and the second patient attribute of the patient to a radiotherapy plan optimizer by the processor, wherein the radiotherapy plan optimizer is configured to generate a radiotherapy treatment plan for the patient based on the first patient attribute and the second patient attribute.

[0008] The method may include receiving, by a processor at a user interface, input indicating a patient, wherein executing a machine learning language processing model includes retrieving, by the processor, stored patient data about the patient from a memory based on the input indicating the patient; and determining, by the processor, a response based on the input first patient attribute and the retrieved patient data.

[0009] The method may include retrieving, by a processor, a template from a memory, the template comprising a list of types of patient attributes, wherein executing the machine learning language processing model comprises determining, by the processor, a response based on inputting a first patient attribute and the template.

[0010] Determining a response may include determining, by the processor, a first type of patient attribute of a type list of patient attributes of the template based on the machine learning language processing model not receiving a value of the first type of patient attribute from the interactive interface; and determining, by the processor, a response to the request for a value of the first type of patient attribute.

[0011] The method may include comparing, by a processor, the first patient attribute and the second patient attribute to a template; and in response to determining that the first patient attribute and the second patient attribute satisfy a template condition of the template, determining, by the processor, that the first patient attribute and the second patient attribute satisfy a planning condition.

[0012] The method may include receiving, by a processor, input indicating a patient at a user interface; presenting, by the processor, patient data of the patient at the user interface based on the input indicating the patient, the patient data including one or more patient attributes of the patient; and receiving, by the processor, a selection of a first patient attribute from the one or more patient attributes presented.

[0013] The method may include generating, by a processor, a labeled vector representing a first patient attribute and a second patient attribute from a machine learning language processing model, wherein transmitting the first patient attribute and the second patient attribute of the patient to a radiation therapy plan optimizer includes transmitting, by the processor, the labeled vector representing the first patient attribute and the second patient attribute to the radiation therapy plan optimizer.

[0014] The method may include converting, by a processor, a tag vector representing a first patient attribute and a second patient attribute into a structured data set, wherein transmitting the first patient attribute of the patient and the second patient attribute of the patient to a radiation therapy plan optimizer includes transmitting, by the processor, the converted structured data set to the radiation therapy plan optimizer.

[0015] The radiation therapy plan optimizer may be configured to generate a radiation therapy treatment plan by generating radiation dose information for treating the patient based on at least the first patient attribute and the second patient attribute.

[0016] The method may include receiving, by a processor, input from a user interface indicating that a radiotherapy treatment plan is incorrect; in response to the input indicating that the radiotherapy treatment plan is incorrect, executing, by the processor, a machine learning language processing model to generate a second response requesting a third patient attribute of the patient; presenting, by the processor, the second response requesting the third patient attribute of the patient at an interactive interface; receiving, by the processor, a third input including the third patient attribute of the patient from the interactive interface; and transmitting, by the processor, the first patient attribute of the patient, the second patient attribute of the patient, and the third patient attribute of the patient to a radiotherapy plan optimizer, wherein the radiotherapy plan optimizer is configured to generate a second radiotherapy treatment plan for the patient based on the first patient attribute, the second patient attribute, and the third patient attribute.

[0017] The method may include storing, by a processor, a training data set comprising a set of text regarding radiation therapy treatments for a patient; and training, by the processor, a machine learning language processing model using the set of text regarding radiation therapy treatments for the patient in the training data set as input.

[0018] The method may include presenting, by a processor, the radiation therapy treatment plan on a user interface.

[0019] In another embodiment, a system may include a processor coupled to a memory and configured to present a user interface providing an interactive interface between a user and a machine learning language processing model; receive a first input including a first patient attribute of a patient from the interactive interface; execute the machine learning language processing model using the first patient attribute of the patient as input to generate a response requesting a second patient attribute of the patient; present the response requesting the second patient attribute of the patient at the interactive interface; receive a second input including the second patient attribute of the patient from the interactive interface; and in response to determining that the first patient attribute and the second patient attribute satisfy a planning condition, transmit the first patient attribute of the patient and the second patient attribute of the patient to a radiation therapy plan optimizer, wherein the radiation therapy plan optimizer is configured to generate a radiation therapy treatment plan for the patient based on the first patient attribute and the second patient attribute.

[0020] The processor may also be configured to receive input indicating a patient at a user interface, wherein the processor is configured to execute the machine learning language processing model by retrieving stored patient data about the patient from a memory based on the input indicating the patient; and determine a response based on the input first patient attribute and the retrieved patient data.

[0021] The processor may also be configured to retrieve a template from the memory, the template comprising a list of types of patient attributes, wherein the processor is configured to execute the machine learning language processing model by determining a response based on the input first patient attribute and the template.

[0022] The processor can be configured to determine a response by determining a first type of patient attribute in a list of patient attribute types for the template based on the machine learning language processing model not receiving a value for the first type of patient attribute from the interactive interface; and determining a response to request a value for the first type of patient attribute.

[0023] The processor may be further configured to compare the first patient attribute and the second patient attribute to a template; and in response to determining that the first patient attribute and the second patient attribute satisfy a template condition of the template, determine that the first patient attribute and the second patient attribute satisfy a planning condition.

[0024] The processor may also be configured to receive input indicating a patient at the user interface; present patient data of the patient based on the input indicating the patient at the user interface, the patient data including one or more patient attributes of the patient; and receive a selection of a first patient attribute from the one or more patient attributes presented.

[0025] The processor may also be configured to generate a labeled vector representing a first patient attribute and a second patient attribute from the machine learning language processing model, wherein the processor is configured to transmit the first patient attribute of the patient and the second patient attribute of the patient to the radiation therapy plan optimizer by transmitting the labeled vector representing the first patient attribute and the second patient attribute to the radiation therapy plan optimizer.

[0026] The processor may also be configured to convert the tag vector representing the first patient attribute and the second patient attribute into a structured data set, wherein the processor is configured to transmit the first patient attribute of the patient and the second patient attribute of the patient to the radiation therapy plan optimizer by transmitting the converted structured data set to the radiation therapy plan optimizer. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Non-limiting embodiments of the present disclosure are described by way of example with reference to the accompanying drawings, which are schematic and are not intended to be drawn to scale. Unless indicated to represent background art, the drawings represent various aspects of the present disclosure.

[0028] Figure 1 Components of a machine learning language processing plan generation system according to an embodiment are shown.

[0029] Figure 2 A flow chart of a process performed in a machine learning language processing plan generation system according to an embodiment is shown.

[0030] Figure 3 An example user interface for facilitating generation of radiation therapy treatment plans using machine learning language processing is shown, according to an embodiment.

[0031] Figure 4 The data flow within a machine learning language processing plan generation system according to an embodiment is shown.

[0032] Figure 5 A flow chart of a process performed in a machine learning language processing plan generation system according to an embodiment is shown.

[0033] Figure 6 A visual representation of processing data to train a machine learning language processing model is shown, according to an embodiment. DETAILED DESCRIPTION

[0034] Reference will now be made to the illustrative embodiments shown in the accompanying drawings, and specific language will be used herein to describe the same. However, it should be understood that it is not intended to limit the scope of the claims or the present disclosure thereby. Changes and further modifications to the inventive features shown herein, as well as additional applications of the principles of the subject matter shown herein, will be conceivable to those skilled in the relevant art and those skilled in the art who possess the present disclosure, and will be considered to be within the scope of the subject matter disclosed herein. Other embodiments may be used and / or other changes may be made without departing from the spirit or scope of the present disclosure. The illustrative embodiments discussed in the detailed description are not meant to limit the subject matter presented.

[0035] Figure 1 Components of a machine learning language processing plan generation system 100 according to an embodiment are shown. The system 100 may include an analysis server 110a, a system database 110b, a machine learning language processing model 111, end-user devices 120a to 120f (collectively referred to as end-user devices 120), a medical device 150, a medical device computer 152, a database 160, and a radiation therapy plan optimizer 162. Figure 1 The various components shown in may belong to a radiation therapy treatment clinic where, in some cases, a patient may receive radiation therapy treatment via one or more radiation therapy machines (eg, medical device 150).

[0036] The system 100 is not limited to the components described herein, and may include additional or other components, which are not shown for the sake of brevity, but are to be considered within the scope of the embodiments described herein.

[0037] The above components can be interconnected through a network 130. Examples of network 130 may include, but are not limited to, private or public local area networks (LANs), wireless local area networks (WLANs), metropolitan area networks (MANs), wide area networks (WANs), and the Internet. Network 130 may include wired and / or wireless communications according to one or more standards and / or via one or more transmission media. Communications on network 130 may be performed according to various communication protocols such as transmission control protocol and Internet protocol (TCP / IP), user datagram protocol (UDP), and IEEE communication protocols. In one example, network 130 may include wireless communications according to a Bluetooth specification set or another standard or proprietary wireless communication protocol. In another example, network 130 may also include communications through a cellular network, including, for example, a GSM (Global System for Mobile Communications), a CDMA (Code Division Multiple Access), or an EDGE (Enhanced Data for Global Evolution) network.

[0038] The analysis server 110a may generate and display an electronic platform configured to communicate between a user and the machine learning language processing model 111, and for receiving patient information and outputting the execution results of the machine learning language processing model 111 and the radiation therapy plan optimizer 162. The electronic platform may include a graphical user interface (GUI) displayed on each end-user device 120, medical device 150, and / or medical device computer 152. An example of an electronic platform generated and hosted by the analysis server 110a may be a web-based application or website configured to be displayed on different electronic devices, such as mobile devices, tablet computers, personal computers, etc.

[0039] The information displayed by the electronic platform may include, for example, input elements for receiving data associated with the patient to be treated (e.g., planning goals), and displaying predicted results generated by the machine learning language processing model 111 (e.g., text, images, or videos generated in response to input received by the electronic platform) and / or the radiation therapy plan optimizer 162 (e.g., predicted radiation therapy plans). The analysis server 110a may then display the results to the medical professional and / or directly modify one or more operational properties of the medical device 150. In some embodiments, the medical device 150 may be a diagnostic imaging device or a treatment delivery device.

[0040] The analysis server 110a can be any computing device, including a processor and non-transitory machine-readable memory capable of performing the various tasks and processes described herein. The analysis server 110a can use various processors such as a central processing unit (CPU) and a graphics processing unit (GPU). Among them, non-limiting examples of such computing devices can include workstation computers, laptop computers, server computers, etc. Although the system 100 includes a single analysis server 110a, the analysis server 110a may include any number of computing devices operating in a distributed computing environment (e.g., a cloud environment).

[0041] The end-user device 120 may be any computing device including a processor and a non-transitory machine-readable storage medium capable of performing the various tasks and processes described herein. Non-limiting examples of the end-user device 120 may be a workstation computer, a laptop computer, a tablet computer, and a server computer. In operation, various users may use the end-user device 120 to access the GUI operated and managed by the analysis server 110a. Specifically, the end-user device 120 may include a clinic computer 120a, a clinic server 120b, and a medical professional device 120c. Although referred to herein as "end-user" devices, these devices may not always be operated by end users. For example, the clinic server 120b may not be used directly by the end user. However, the results stored on the clinic server 120b may be used to fill in various GUIs accessed by the end user via the medical professional device 120c.

[0042] The medical device 150 may be a radiotherapy machine configured to administer radiation therapy treatment to a patient. The medical device 150 may also communicate with a medical device computer 152 configured to display various GUIs discussed herein. For example, the analysis server 110a may display the results predicted by the machine learning language processing model 111 and / or the radiation therapy plan optimizer 162 on a computing device as described herein. In a non-limiting example, the GUI may display the responses generated by the machine learning language processing model 111 and the radiation therapy treatment plan (RTTP) generated by the radiation therapy plan optimizer 126 at the medical device computer 152 and / or the end-user device 120.

[0043] The machine learning language processing model 111 may be stored in the system database 110b. The machine learning language processing model 111 may be configured or trained to automatically generate text, image, or video responses based on input received at the user interface. The machine learning language processing model 111 may be configured to receive patient attributes of a patient as input, and automatically generate responses (e.g., questions or commands) requesting patient attributes based on the received patient attributes. The machine learning language processing model 111 may maintain or simulate a conversation with a user who provides input requesting more and / or different patient attributes over time. After receiving the input patient attributes, whether it is the initial input or each input iteration, the analysis server 110a (e.g., via the machine learning language processing model 111) may compare the patient attributes received by the analysis server 110a for the patient with one or more templates stored in the system database 110b by the analysis server 110a. In response to determining that the patient attributes meet the conditions of the template (e.g., the patient attributes include enough patient attributes to exceed or meet the threshold, or include a specific combination of patient attributes to meet the template), the machine learning language processing model 111 may stop responding to the user's input. The analysis server 110 a may use the received patient attributes as input to the radiation therapy plan optimizer 162 .

[0044] The analysis server 110a may execute a radiation therapy plan optimizer 162 to generate one or more treatment attributes of the RTTP that meet any radiation therapy plan goals based on patient attributes of the patient for whom the radiation therapy plan is being generated. The radiation therapy plan optimizer 162 may be stored in the database 160. The radiation therapy plan optimizer 162 may generate the one or more treatment attributes, for example, by iteratively calculating the one or more treatment attributes, wherein with each iteration, the radiation therapy plan optimizer 162 may modify the one or more treatment attributes of the RTTP according to the cost value.

[0045] Analysis server 110a can deploy radiation therapy plan optimizer 162, to generate patient's RTTP based on patient's patient attribute.Radiotherapy plan optimizer 162 can iteratively calculate one or more treatment attributes of RTTP.For example, with each iteration, radiation therapy plan optimizer 162 can generate candidate RTTP with various attributes.Plan optimizer 162 can then use one or more loss functions to calculate the cost value of generated candidate RTTP.Cost value can indicate the possibility that candidate RTTP violates rule set (whether internal rule and / or external rule).For example, cost value can indicate whether candidate RTTP violates any plan target.Radiotherapy plan optimizer 162 can analyze cost value.If necessary (for example, when cost value meets threshold value), radiation therapy plan optimizer 162 can modify candidate RTTP and re-execute its loss function to generate new cost value.According to whether the new cost function increases or decreases, plan optimizer computer model can modify candidate RTTP again and recalculate cost value.Radiotherapy plan optimizer 162 can continue this iterative method until it converges to RTTP (or final RTTP) with cost value that meets threshold value.

[0046] The analysis server 110a may identify the treatment attributes determined by the radiation therapy plan optimizer 162 to have a cost value that satisfies a threshold. The analysis server 110a may present the treatment attributes as RTTP at an end-user device 120 being accessed by a user generating a radiation therapy treatment plan. The analysis server 110a may present the treatment attributes in the same interactive interface that the analysis server 110a previously used to communicate the user with the machine learning language processing model 111, or at a different portion of the user interface. The user may implement the RTTP or the analysis server 110a, or the end-user device 120 may use the RTTP to automatically control the medical device 150 to treat the patient based on the attributes of the RTTP.

[0047] refer to Figure 2 , method 200 illustrates an operational workflow performed in a machine learning language processing plan generation system according to an embodiment. Method 200 may include steps 202 to 212. However, other embodiments may include additional or alternative steps, or may omit one or more steps entirely. Method 200 is described as being performed by, for example, Figure 1 However, one or more steps of method 200 may be performed by a server such as the analysis server described in Figure 1 For example, one or more computing devices may execute locally Figure 2 Some or all of the steps described in .

[0048] Using method 200, the analysis server can implement a combination of a machine learning language processing model and a radiation therapy plan optimizer to generate a radiation therapy treatment plan. To this end, the analysis server can provide an interactive interface (e.g., a partition or portion of a user interface, which the analysis server can also provide) to a computing device that is being accessed by a user (e.g., an oncologist or other clinician). Through the interactive interface, the user can enter patient attributes of the patient or any other data. The analysis server can receive the input and provide the input to the machine learning language processing model. The machine learning language processing model can process the input and generate a response (e.g., text, image, video, etc.) based on the input. The response can be a question that further requests the patient attributes of the patient. The machine learning language processing model and the user can communicate through the interactive interface in this manner until the analysis server determines that there is enough data or patient attributes to generate a radiation therapy treatment plan for the patient. The analysis server can input the data or patient attributes into the radiation therapy plan optimizer. Based on the input data or patient attributes, the radiation therapy plan optimizer can generate the treatment attributes of the radiation therapy treatment plan for the patient. The analysis server can present the treatment attributes to the user in a user interface, such as in an interactive interface. In some cases, the analysis server can transmit the treatment attributes to the radiation therapy machine itself to implement the radiation therapy treatment plan.

[0049] At step 202, the analysis server may present a user interface at the computing device. The analysis server may generate the user interface and transmit the user interface to the computing device over a network. When providing an electronic platform that facilitates generating a radiation therapy treatment plan for a patient, the analysis server may generate and transmit the user interface to the computing device based on input from a user (such as an oncologist or other clinician).

[0050] The analysis server can generate a user interface with an interactive interface. The interactive interface can be an interface for communication between a user who can view and provide input to the interactive interface and an analysis server or a machine learning language processing model. The machine learning language processing model can be a large language model that is trained to generate text, images and / or videos in response to input text, images and / or videos. The machine learning language processing model can be or include a neural network, such as a neural network with a converter architecture. The interactive interface can implement a form of communication similar to a conversation between two humans, which can involve a back-and-forth message exchange between the participants in the conversation. In this case, the two participants can be users, such as oncologists or other clinicians, and a machine learning language processing model.

[0051] The interactive interface can simulate an instant messaging application dialogue between a user and a machine learning language processing model. For example, a user can enter text in the interactive interface. The interactive interface can be a feed (e.g., a user feed). The text can be or include patient attributes of a patient receiving radiotherapy treatment. For example, a user can select a send button to transmit text to an analysis server. The analysis server can input the text including the patient attributes into the machine learning language processing model and execute the machine learning language processing model. Based on the input, the machine language learning model can output a response (e.g., a question or command) in text form. The analysis server can present a response in the interactive interface, such as below the user's latest input text (e.g., directly or sequentially below). The user can respond to the response from the machine learning language processing model with the text below the response, and submit the user input response. This process can be repeated any number of times to simulate an instant messaging application dialogue.

[0052] The interactive interface may cover part of the user interface, or it may be a widget of the user interface. The user interface may include other parts or widgets that provide different functions. For example, although the interactive interface may enable communication between the user and the machine learning language processing model, the analysis server may also include parts of the user interface that display different patient attributes of the patient. For example, a user accessing the user interface may access the user interface to generate a radiotherapy treatment plan for the patient. The user may enter the patient's identifier (e.g., name) into the user interface or platform provided by the analysis server. In response to receiving the input, the analysis server may retrieve patient data (e.g., one or more patient attributes) about the patient from the memory. Examples of patient attributes that the analysis server may retrieve include computed tomography (CT) of the patient or the patient's tumor, images of the patient or the patient's tumor, previously collected patient attributes (such as data collected from previous health tests), etc. The analysis server may present the retrieved patient data in another widget or part of the user interface that is separated or adjacent to the interactive interface on the user interface. In some cases, the part or widget of the user interface including the retrieved patient data may be separated from the interactive interface on the user interface by a line (e.g., a vertical line across the width or length of the user interface (e.g., along the x-axis or y-axis of the user interface)).

[0053] In a non-limiting example, reference is now made to Figure 3, the analysis server can generate a user interface 300 including an interactive interface 302 and a patient data interface 304. The interactive interface 302 can be a portion of the user interface 300 that a user can use to interact with the machine learning language processing model. The user can input text, images and / or videos into the interactive interface 302 and receive a response from the machine learning language processing model in the interactive interface 302. The patient data interface 304 can be an area of ​​the user interface 300 through which the analysis server can present the retrieved patient data 306 for the patient. The patient data interface 304 can be configured to present any type of data, such as text, images, or videos. The analysis server can generate the user interface 300 and transmit the user interface to a computing device accessed by the user.

[0054] The user interface may include an auxiliary interface 308. The analysis server may include an auxiliary interface 308 on the user interface 300 to explain to the user additional information related to the dialogue on the interactive interface 302, and to perform the retrieved data on the patient data interface 304 using the user interface 300. The auxiliary interface 308 may include a task list, such as a current task. A task may refer to a task that a user completes by communicating with a machine learning language processing model. For example, a user may input an option to select a task to generate a radiation therapy treatment plan for patient A. In response to the input, the machine learning language processing model may identify the input and use the input to determine a response to collect patient data about patient A. Another example of a task is to show a CT scan of a patient in the patient data interface 304. In response to input identifying such a task, the machine learning language processing model may identify the input, and the analysis server may retrieve a CT scan of the patient and display the patient scan on the patient data interface 304.

[0055] Reference again Figure 2 At step 204, the analysis server may receive a first input. The analysis server may receive the first input from an interactive interface of the user interface. The first input may include a first patient attribute of the patient. The first input may include the patient attribute and any number of other patient attributes. The first patient attribute and the other patient attributes may be any type of patient attributes, such as height, gender, weight, treatment options for the patient, treatment attributes (e.g., gantry motion, gantry position, etc.), treatment goals, attributes about the tumor (e.g., size or shape), images of the patient or the tumor, tumor stage, primary site of treatment, endpoint, whether the tumor has expanded, body mass index, blood pressure, medical history (e.g., medical treatments previously received by the patient), etc. For example, a user may provide the first input and select a send or submit button on the user interface. The analysis server may receive the first input from a computing device presenting the user interface.

[0056] In one example, a user may provide a first input by typing a first patient attribute and any other patient attributes of the first input into the interactive interface. For example, a user may type a current height, age, weight, and blood pressure into the interactive interface. Typing the patient attributes into the interactive interface may cause text to appear on the interactive interface. The user may select a submit button in response to typing the first input into the interactive interface to submit the first input to the analysis server.

[0057] In another example, a user may provide a first input by selecting a first attribute and any other patient attributes from the patient data presented on the user interface. For example, a user may select a tumor image and / or one or more other patient attributes of a patient from the patient data. The user may select an option to move the selected patient attributes to the interactive interface, or otherwise drag the selected patient attributes to the interactive interface. After moving the selected patient attributes to the interactive interface, the user may select an option to submit or send the selected patient attributes to the analysis server. In some cases, the user may select an option to submit or send the selected patient attributes without moving the selected patient attributes to the interactive interface.

[0058] In some cases, the user may provide the first input by typing one or more patient attributes in the interactive interface and selecting one or more patient attributes from patient data for the patient displayed on the user interface. The user may type one or more patient attributes in the interactive interface and select one or more patient attributes from the patient data, and select a submit button to transmit the first input to the analysis server.

[0059] Reference again Figure 3 , in a non-limiting example, a user may provide a first input to the interaction interface 302. For example, the user may do so by typing a first patient attribute and any number of other patient attributes into the interaction interface 302. In some cases, in addition to or instead of typing the first patient attribute into the interaction interface 302, the user may select the first patient attribute and / or any number of other patient attributes from the patient data interface 304. The user may submit the patient attributes by selecting a submit button on the user interface 300 or by selecting an input / output button to send the first patient attribute and any other patient attributes to the analysis server.

[0060] Reference again Figure 2, at step 206, the analysis server may execute the machine learning language processing model. The analysis server may execute the machine learning language processing model using the first input (e.g., the first patient attribute of the patient and / or any other patient attribute of the first input) as the input of the machine learning language processing model. The machine learning language processing model may generate a response based on the execution. The response may be or include text. The text may be a string or word requesting a second patient attribute of the patient. For example, the response may be a request for the user to provide the location of a tumor on the patient. The machine learning language processing model may generate a response by implementing the learned weights and / or parameters on the data of the first input. The response may be a request for any type of patient attribute.

[0061] In some cases, the analysis server may include patient data stored in a memory in the input of a machine learning language processing model having a first input. For example, the analysis server may receive an input identifier (e.g., a name) of a patient from a computing device. The analysis server may query the memory based on the identifier to retrieve patient data about the patient from the memory. The retrieved patient data may include one or more patient attributes of the patient. In addition, in some cases, the analysis server may retrieve a treatment protocol that can be used to treat the patient. The analysis server may identify one or more patient attributes and / or treatment protocols retrieved from the memory, and input one or more patient attributes or treatment protocols into the machine learning language processing model in place of or in addition to the first input received from the user interface. The machine learning language processing model may determine a response based on the first input (e.g., the first patient attribute and any other patient attributes) and the retrieved patient data.

[0062] The machine learning language processing model may use one or more templates to determine a response to the first input. In some cases, one or more templates may include a list of patient attribute types. Patient attributes may include data or characteristics of a patient and / or treatment method (e.g., one template may correspond to one treatment method, while another template may correspond to another treatment method). The machine learning language processing model may compare one or more patient attributes of the first input, and in some cases, any stored data about the patient or available treatment protocols retrieved from memory, with the patient attributes and / or treatment methods of the template. Based on the comparison, the machine learning language processing model may determine that one or more patient attributes are needed to determine a radiation therapy treatment plan for the patient based on a template containing the required patient attributes that were not included in the first input or retrieved from memory. The machine learning language processing model may identify missing patient attributes (e.g., types of missing patient attributes) and generate a text response indicating that the user input is missing one or all of the patient attributes.

[0063] A template may correspond to one or more template conditions. A template condition may be a rule that, once satisfied, indicates that the template is satisfied and that there is sufficient data to generate a radiation therapy treatment (plan) device for a patient. In one example, a template condition of a template may indicate that the template satisfies a machine learning language processing model that receives patient data for each patient attribute type of the template as input. In another example, a template may correspond to one or more template conditions, each template condition indicating a different set of specific patient attributes. One of such template conditions may be satisfied in response to the machine learning language processing model receiving patient data for each patient attribute type for the template condition.

[0064] In one non-limiting example, a template may include a list of patient attribute types that includes gender, height range, tumor size, tumor location, and chemotherapy treatment availability. The template may include a condition indicating that the machine learning language processing model must receive data for each patient attribute type on the list to satisfy the condition. The template may include a condition indicating that the machine learning language processing model must receive data for gender, height range, and radiation therapy availability to satisfy the condition. The template may include conditions that are satisfied based on any permutation or combination of the patient attribute types on the list.

[0065] In some cases, a template may correspond to specific patient attributes. For example, a template may include a list of specific patient attributes, instead of or in addition to having a list of patient attribute types, the template may include a list of specific patient attributes. An example of a specific list of specific patient attributes is as follows: male, tumor on stomach, patient's tumor is 5 inches in diameter. A list of specific patient attributes may include any patient attribute and any number of patient attributes. A template having a list of specific patient attributes instead of a list of patient attribute types may include conditions similar to those described above, except that the condition is satisfied based on whether the machine learning language processing model includes a specific patient attribute for the condition instead of a patient type for the condition.

[0066] In some cases, a template may correspond to both a specific patient attribute and a patient attribute type. For example, a template may include both a list of specific patient attributes and a list of patient attribute types. A template having both a list of specific patient attributes and a list of patient attribute types may include one or more conditions similar to the above conditions, except that the conditions are satisfied based on whether the machine learning language processing model includes both the specific patient attribute and the patient attribute type.

[0067] The machine learning language processing model can receive a first input and retrieve one or more templates from a memory. The machine learning language processing model can compare patient attributes of the first input with the one or more templates. Based on the comparison, the machine learning language processing model can identify one or more patient attributes or patient attribute types (e.g., first type of patient attributes) of the template that are missing from the first input, and if the values ​​of the one or more patient attributes or types of patient attributes are in the first input and / or retrieved patient data for the patient, the template will satisfy the condition. The machine learning language processing model can generate a response with a question or command for a user accessing the interactive interface to enter a value for the requested one or more missing patient attributes.

[0068] The analysis server can generate or train a machine learning language processing model. The analysis server can generate or train a machine learning language processing model using supervised learning, unsupervised learning, or semi-supervised learning techniques. For example, the analysis server can use a labeled training data set to train a machine learning language processing model. The labeled training data set can include different sentences or text paragraphs corresponding to the patient's radiotherapy treatment. For example, a sentence or paragraph of text can correspond to the patient's radiotherapy treatment, because the sentence or paragraph can include the value of one or more patient attributes, and can include specific keywords corresponding to radiotherapy treatment (e.g., radiotherapy, radiation, radiotherapy, oncology, tumor, radiation oncology, linear accelerator, radiation dose, external beam radiation, etc.). In some cases, the labeled training data set can correspond to radiotherapy based on the source of the training data. For example, the labeled training data set can include sentences or texts from clinical guidelines for radiotherapy treatment plans, case studies of radiotherapy treatment, medical journals and research papers on radiotherapy treatment, structured radiotherapy treatment plans, etc. The labeled training data set can include annotations indicating the rationale behind certain decisions in the treatment plan or any specific considerations considered. The labeled training data set may additionally or alternatively include tags indicating correct responses to different sentences or texts. The analysis server may automatically label the training data set, or a human reviewer may label the training data set. Such text may be fed (e.g., by the analysis server) into a machine learning language processing model for training.

[0069] The analysis server can use back propagation techniques to train the machine learning language processing model. For example, the analysis server can insert different entries (e.g., prompts, such as sentences or texts) in the training data set into the machine learning language processing model, and execute the machine learning language processing model for each entry. When executing the model of the entry, the machine learning language processing model can generate or output words or word sequences based on the words, images and / or videos in the entry. The analysis server can determine the difference between the output of the machine learning language processing model and the tag of the entry according to the loss function. The analysis server can use back propagation techniques to adjust the parameters and / or weights of the machine learning language processing model based on the difference. The analysis server can train the machine learning language processing model in this way with different tag entries over time. The analysis server can train the machine learning language processing model until the machine learning language processing model is accurate to an accuracy threshold, at which point the analysis server can deploy the machine learning language processing model for collecting patient attributes through an interactive interface.

[0070] During training, the analysis server may iteratively execute the machine learning language processing model to generate new predicted text, images, and / or videos based on the training data set (e.g., each entry of text, image, and / or video). If the predicted result does not match the actual result, the analysis server may continue training until the computer-generated recommendation meets one or more accuracy thresholds and is within an acceptable range. For example, the analysis server may divide the training data set into three groups (i.e., training, validation, and testing). The analysis server may train the machine learning language processing model based on the first group (training). The analysis server may then execute (at least partially) the trained machine learning language processing model to predict the results of the second group of data (validation). The analysis server then verifies whether the prediction is correct. Using the above method, the analysis server may evaluate whether the machine learning language processing model is properly trained. The analysis server may use this method to continuously train and improve the machine learning language processing model. The analysis server may then use the remaining data points in the training data set (testing) to measure the accuracy (e.g., area under the curve, accuracy, and recall) of the machine learning language processing model.

[0071] At step 208, the analysis server may present a response. The response may be text requesting a second patient attribute of the patient. The second patient attribute determined by the machine learning language processing model will cause the patient's data to satisfy the template condition of the template. The analysis server may present the response requesting the second patient attribute of the patient at the interactive interface being accessed by the user.

[0072] Reference again Figure 3In a non-limiting example, the analysis server may present a response at user interface 300. The response may be a request for the patient's current body mass index. The analysis server may present the response at interaction interface 302 at user interface 300. The analysis server may present the response below the first input initially provided by the user at interaction interface 302.

[0073] Reference again Figure 2 At step 210, the analysis server may receive a second input from the user interface. The second input may include a value of a second patient attribute presented at the interactive interface in response to the request. A user accessing the user interface may input the value of the second attribute into the interactive interface (e.g., via text, image, or video). The response presented at the interactive interface may be text of the second patient attribute of the requested patient. The computing device presenting the user interface may transmit the value of the second patient attribute to the analysis server. The analysis server may receive the value as the second input.

[0074] The analysis server may receive a second input. The analysis server may execute the machine learning language processing model using the second input, the first input, and / or any patient data retrieved from the memory to determine whether any template conditions are met. In response to determining that there are no satisfied template conditions, the machine learning language processing model may generate a request for another patient attribute (e.g., a third patient attribute). The analysis server may transmit the request for the third patient attribute to the computing device for display on the user interface. A user accessing the user interface may enter a value for the third patient attribute, and the computing device may transmit the value of the third patient attribute to the analysis server. The analysis server and the computing device presenting the user interface may repeat the process until the machine learning language processing model has received patient attributes that meet the planned conditions.

[0075] A planning condition may be or include one or more rules that, when satisfied, indicate that sufficient data or patient attributes have been collected to generate a radiation therapy treatment plan for the patient. A planning condition may be or include a template condition or any other set of rules.

[0076] Reference again Figure 3 In a non-limiting example, the analysis server may receive a second input including a second patient attribute from the user interface 300. A user accessing the user interface 300 may enter a value for the second patient attribute into the interactive interface 302. The second patient attribute may be, for example, a potential treatment option for the patient. The user may enter the value of the second patient attribute and the value of any other patient attribute as the second input and select a submit button. Selection of the submit button may cause the computing device presenting the user interface 300 to transmit the second input to the analysis server.

[0077] In response to determining (e.g., via a machine learning language processing model) that the analysis server has received patient attributes that satisfy a plan condition (e.g., determining that at least a first and a second patient attribute satisfy a template condition), at step 212, the analysis server may transmit a first patient attribute of the patient and a second patient attribute of the patient to a radiation therapy plan optimizer. The radiation therapy plan optimizer may be a computer model (e.g., an optimization computer model) configured to generate one or more treatment attributes of a radiation therapy plan based on the patient attributes of the patient for which the radiation therapy treatment plan is being generated, the treatment attributes meeting the radiation therapy plan goals for the patient (e.g., the goals in the patient attributes of the patient). The radiation therapy plan optimizer may generate the one or more treatment attributes, for example, by iteratively calculating the one or more treatment attributes, wherein for each iteration, the radiation therapy plan optimizer modifies the one or more attributes of the radiation therapy plan according to a cost value. The radiation therapy plan optimizer may receive the first patient attribute of the patient and the second patient attribute of the patient, in some cases in combination with other patient attributes for satisfying a template or plan condition. Based on the patient attributes, the radiation therapy plan optimizer can generate or determine one or more attributes of a radiation therapy treatment plan for the patient (e.g., radiation dose or other information, field geometry settings, arc settings, treatment frequency, treatment type, radiation parameters, etc.) to generate a radiation therapy treatment plan.

[0078] The analysis server can generate a tag vector from the patient attributes. The analysis server can generate the tag vector using a tokenization algorithm (e.g., WordPiece or Byte Pair Encoding). In some cases, the analysis server can generate treatment attributes for the radiotherapy treatment plan using the same tokenization algorithm used to train the radiotherapy plan optimizer. When generating the tags, the analysis server can generate an array or vector of numbers based on the patient attributes. In some cases, each attribute can correspond to a different number (e.g., mapped to a different number), and the analysis server can determine the number or tag of each attribute based on the mapping. The analysis server can generate a tag vector based on the patient attributes and transmit the tag vector to the radiotherapy plan optimizer.

[0079] In some cases, the analysis server may convert the marker vector into a structured data set. The structured data set may include data in the marker vector in a format that can be processed by the radiation therapy plan optimizer. For example, the analysis server may convert the marker vector into a traditional structured cost function or field geometry definition. For example, the analysis server may store a mapping of the marker vector to the value of the cost function or field geometry setting. The analysis server may convert the marker vector into a traditional structural cost function or field geometry definition based on the mapping. The analysis server may transmit the converted marker vector to the radiation therapy plan optimizer. In another example, the marker vector may include a reference to one or more predefined treatment protocols (e.g., a treatment protocol template). Such markers may be used to construct instructions to the radiation therapy plan optimizer. For example, in some cases, the marker may instruct the plan optimizer to exclude and / or include certain treatment attributes from the radiation therapy treatment plan.

[0080] The radiation therapy plan optimizer may receive patient attributes (e.g., raw patient attributes, labeled vectors representing patient attributes, or transformed labeled vectors, as described above). A computer storing the radiation therapy plan optimizer (e.g., an analysis server or another computer) may execute the radiation therapy plan optimizer using the patient attributes as input. The radiation therapy plan optimizer may output one or more treatment attributes of a radiation therapy treatment plan for the patient based on the patient's attributes. The radiation therapy plan optimizer may transmit the one or more treatment attributes to the analysis server.

[0081] The analysis server may receive the one or more treatment attributes. The analysis server may present the radiation therapy treatment plan (e.g., one or more treatment attributes of the radiation therapy treatment plan) on a user interface. The analysis server may present the radiation therapy treatment plan on an interactive interface of the user interface or on another portion of the user interface. In some embodiments, the analysis server may transmit the one or more treatment attributes to a radiation therapy treatment machine for use in providing treatment to the patient. The radiation therapy treatment machine may operate (e.g., automatically) based on the one or more treatment attributes.

[0082] In some cases, a user at the user interface can adjust the radiation therapy treatment plan. For example, a user viewing the radiation therapy treatment plan at the user interface can provide an incorrect input (e.g., a text input or a button selection) on the user interface that the radiation therapy treatment plan is incorrect. The analysis server can receive an input indicating that the radiation therapy treatment plan is incorrect. In response to the input indicating that the radiation therapy treatment plan is incorrect, the analysis server can use the input or an indication of the input to execute a machine learning language processing model to generate a second response requesting a third patient attribute or a fourth patient attribute of the patient. The analysis server can present a second response requesting a third patient attribute or a fourth patient attribute of the patient at the interactive interface. The analysis server can receive a third input including the third patient attribute or the fourth patient attribute of the requested patient from the interactive interface. The analysis server can transmit the first patient attribute of the patient, the second patient attribute of the patient, and the third patient attribute or the fourth patient attribute of the patient to the radiation therapy plan optimizer (e.g., a marker vector generated based on the patient attribute or a converted marker vector). The radiation therapy plan optimizer can generate a second radiation therapy treatment plan for the patient based on the first patient attribute, the second patient attribute, and the third patient attribute or the fourth patient attribute. The radiation therapy plan optimizer can transmit a second radiation therapy treatment plan to the analysis server. The analysis server may present the second radiation therapy treatment plan at the user interface and / or transmit the second radiation therapy treatment plan to the radiation therapy machine.

[0083] Figure 4 The data flow within the machine learning language processing plan generation system 400 according to an embodiment is shown. The system 400 may include an analysis server 402 and a language learning model (LLM) server 404. The LLM server 404 may be stored locally in the memory of the analysis server 402, or the LLM server 402 may be stored remotely at another computing device. The LLM service 404 may be or include a machine learning language processing model, which is configured to generate and output a text, image, or video response to an input text, image, or video, as described herein. The analysis server 402 may include a workspace generator 406, a plan template selector 408, a database server 410, a plan optimization input converter 412, a descriptor function 414, a plan optimizer 416, and an LLM agent 418. Components 402 to 418 may operate together to generate a radiotherapy treatment plan using machine learning language processing technology. The LLM agent 418 may be a program, such as an application programming interface, which is configured to communicate to the LLM server 404 or otherwise send a message and / or receive a message from the LLM service 404. In doing so, the LLM agent 418 may provide input to, and receive output from, the machine learning language processing model.

[0084] A predefined prompt template 420 can support any component 402 to component 418, which can process language information into a "grammatically stricter" serialized list of tokens. The prompt template 420 can be or include an LLM prompt template. The prompt template 420 can be configured to make a selection or identify an identification string for database retrieval. The prompt template 420 can embed a language description or a grammatically stricter description into a request that instructs the machine learning language processing model to provide a specific type of output. For example, the template can create a feature vector from patient attributes and any other text input retrieved from a user interface and / or from a memory, and the machine learning language processing model can be configured to process these texts and provide a response. In turn, such a machine learning language processing model can be configured to output a response in an indicated format (e.g., text, image, video, etc.), which can be serialized or not serialized (e.g., a serialized list of requested patient attributes).

[0085] The workspace generator 406 can generate a user interface for a user (e.g., an oncologist or other clinician). In one example, the workspace generator 406 can generate the user interface 300 or generate a user interface similar to the user interface 300. The workspace generator 406 can be configured to generate and present a user interface component or widget for presenting a language feed that can be used for language communication between the user and the LLM service 404. The workspace generator 406 can generate a user interface in which radiation therapy related objects can be presented or visualized on the user interface. Examples of such objects can include radiation therapy plan templates, patient data (CT images, structure sets, etc.), plan data (e.g., dose distribution, the status of selected clinical targets, etc.), etc. Radiation therapy related objects can be patient attributes of individual patients. The workspace generator 406 can present such a user interface component on a user interface for communicating with the analysis server 402.

[0086] The workspace generator 406 can generate a user interface (e.g., the interaction interface 302) including at least one interaction pane that can present a text feed of responses from the LLM service 404 and other components. A user at the workspace generator 406 can type a user response to the presented response from the LLM service 404. In some cases, the communication can be accomplished using a separate speech recognition tool and a separate speech generation tool.

[0087] The workspace generator 406 may include a data pane (e.g., the patient data interface 304) to present other clinical data on the user interface. Such clinical data may be image data (e.g., CT images, structure sets, planning field geometry, or planned doses), or may be presented in the form of a table or table (e.g., a list of possible choices, templated planning instructions, how to achieve clinical goals, etc.). The interactive pane and the data pane may interact with each other. For example, the workspace generator 406 may load components of data from the interactive pane to the data pane, or a selection of a data element in the data pane may create a corresponding tag for the selected data element in the interactive pane.

[0088] The workspace generator 406 may include one or more auxiliary panes in the user interface. The one or more auxiliary panes may provide additional information, such as a list of tasks assigned to the user, an indication of (multiple) active tasks, and / or any status information related to the active tasks. For example, if the task requires filling out a form or selecting a template, the auxiliary pane may include the current options for the form content or template.

[0089] The workspace generator 406 can generate a user interface so that the user does not need to leave the interactive pane during a standard planning session and can view relevant information about the planning session presented on the user interface. In some cases, the user interface can include user interface controls (e.g., buttons) that can call different tasks, such as starting a planning session for a new patient. In some cases, a task can be instantiated or started by a request in the interactive pane of the user interface. When a task is requested through an interactive page, special prompts may be required to process the request and search whether a new task is to be started.

[0090] The database service 410 can allow for retrieval of radiation therapy related objects (e.g., patient attributes) based on an identification string or keyword. The database service 410 can be different from existing database services in that the database service 410 can store descriptions written in the language for each radiation therapy object. These descriptions can facilitate responses (e.g., queries) by the LLM service 404 by providing additional context for the objects in the feed that can be processed by the LLM service 404. Portions of the description written in the language can be serialized tagged strings that follow stricter "grammar rules" than the language (such as JSON or some key-value pairs), or can be processed by more traditional algorithms, such as the plan optimizer described below.

[0091] The database service 410 can store written descriptions associated with different data objects. The database service 410 can write algorithmically generated serialized descriptions (which can be created using the descriptor function 414, as described below). Additionally or alternatively, the database service 410 can store previous discussions (or summaries of such discussions) that are recorded from the user's interaction with the user interface generated by the workspace generator 406. The conversation description can be added as an optional element to the prompt or input of the machine learning language processing model to obtain more contextual information for the response (e.g., to create a machine learning language process that "remembers" the appearance of the previous discussion). For patient-related data objects (e.g., patient attributes), the scope of the stored discussion can be related to the planning session of the specific patient (e.g., the discussion stored from the corresponding planning session record to the specific plan). For more general data objects (e.g., for planning templates), the stored discussion summary can contain comments collected from a larger set of tasks (e.g., the database service 410 can store the user's or clinician's specific preferences for a certain planning template). Such preferences can be collected from notes (e.g., input provided by the user to the interaction pane during several planning tasks).

[0092] Descriptor functions 414 may be sets of instructions that, when executed by an analysis server, may construct written descriptions of different data objects. Descriptor functions 414 may construct written descriptions of patient attribute data objects, such as patient data (e.g., CT images or sets of structures) and treatment attributes (e.g., dose distribution). The written description may or may not be a language. For example, the written description may be or include a serialized list of tags describing the details so that the information can be used by a machine learning language processing model.

[0093] The descriptor function 414 can be used in the conversational treatment planning system described herein. The descriptor function 414 can, for example, convert a radiotherapy object into a serialized description. The descriptor function 414 can be or include a traditional algorithm, or the descriptor function 414 can be based on a generative machine learning model that has been trained to generate a serialized description. The descriptor function 414 can be configured to generate a written description of data in various formats (e.g., a three-dimensional image), which can be added to the input of the machine learning language processing model. The descriptor function 414 can enable a user to input an image into an interactive pane to provide data to the machine learning language processing model to generate a radiotherapy treatment plan for the patient. In some cases, a prompt template 420 can be used to generate input to the machine learning language processing model. The prompt template 420 can include instructions for creating an output format similar to or identical to the output of the descriptor function 414 (e.g., if the descriptor function 414 outputs text, the input of the machine learning language processing model can include instructions for outputting text).

[0094] In one example, the descriptor function 414 may be or include a patient descriptor function. The patient descriptor function may be configured to receive patient information, such as a CT image (or other imaging modality) and structure set information (e.g., delineation of organs at risk (OARs) and target structures). The patient descriptor function may output a list of statements describing the patient geometry based on the input. For example, the patient descriptor function may output a list of statements, such as the following statements:

[0095] "Treatment_Site":"HEAD_NECK"}

[0096] "GTV_location":"oropharyngeal"

[0097] "CTV_overlap":"oral_cavity

[0098] "CTV_low_volume":"320cc"

[0099] "CTV_low_proximity":"parotid gland left"

[0100] The patient descriptor function can create such statements using a template. In one example, the template can be configured to systematically test the overlap between the target structure and various OARs. In another example, the template can be configured to calculate the distance between structures.

[0101] In some cases, a supervised learning model (e.g., a self-organizing supervised learning model) can be used to create partial statements. A supervised learning model can be trained to assign certain labels to CT or structure set data. For example, a supervised learning model can be trained to identify the location of the gross tumor volume (GTV) or to generate a general statement of whether a case is unilateral or bilateral.

[0102] In some cases, the sentences can be generated using a general generative machine learning model that has been trained to generate a free-form description of a given image. In this case, the free-form description can be converted into a list of sentences using a machine learning language processing model (e.g., together with a prompt generator that generates prompts for the machine learning language processing model using a prompt template).

[0103] In some embodiments, the patient descriptor may receive a treatment plan template as input. The patient descriptor may use the treatment plan template to make statements related to the predicted dose distribution (eg, generate predicted doses) and add statements such as '{"Predicted_goal_violation":"(parotid gland left,mean_dose,<30Gy)"}'.

[0104] In another example, the descriptor function 414 can be or include a plan descriptor function. The plan descriptor function can be configured to receive as input the created plan and the plan directives (e.g., the minimum requirements for the plan) and evaluate the plan relative to the plan directives. The plan descriptor function can be or include a conventional algorithm that evaluates how different plan goals (e.g., explicitly stated or implicitly added, such as some normal organizational goals) are met or violated. For example, the constructed list of statements that the plan descriptor function can output can be or include

[0105] {"Goal_met":"(PTV_high,dose_coverage_at_prescription_level,>95%)"}

[0106] {"Acceptable_variation_met":"(Spinal_cord,dose_max,<42Gy)"}

[0107] {"Goal_violation":"(Lung Ipsi,D_5cc,<7Gy)"}

[0108] {"Prediction_violation":"(Lung Contra,mean_dose,10%)"}.

[0109] The sentence list can contain spatially constrained observations, such as '{"Regional_Prediction_violation":"(region_proiximal_to_spine,10%)"}'. In some cases, a generative model can be trained to create sentences that are free-form descriptions.

[0110] The plan optimizer 416 may be configured to create plan optimization instructions (e.g., cost function generation or field geometry settings) based on patient attributes or marker vectors generated based on patient attributes. The plan optimizer 416 may be configured to receive a structured cost function or field geometry definition from the plan optimization input converter 412 and generate treatment attributes for a radiation therapy treatment plan based on the structured cost function and field geometry definition.

[0111] The plan optimizer 416 may be configured to receive as input serialized plan input information in addition to patient geometry information (e.g., and beam data information). The plan optimizer 416 may operate in a pre-processing step in which a generated statement list is converted to a cost function and, in some cases, to field geometry information. The statement list may contain elements of which plan template to use to generate a radiation therapy treatment plan. In an embodiment, the statement list may include a plan template as a first statement, and then each successive statement after the plan template may be a modification to the plan template. The plan optimizer 416 may process the modifications in the order of the statement list. For example, the serialized plan instructions may be input to the plan optimizer 416 in the following type of statement list:

[0112] {“Use_template”:“Prostate_lowrisk_hypofractionation_VMAT”}

[0113] {"Increase_priority":"Femural Head left"}

[0114] {"lower_priority":"Bladder"}

[0115] {"Abandon_goal":"Meet_predicted_dose"}

[0116] {"Add_goal":"(Max_MU,<2000)"

[0117] The plan optimizer 416 can generate one or more treatment attributes of a radiation therapy treatment plan based on the statement list. The plan optimizer 416 combined with the plan descriptor function can form an optimization system that takes a feed (e.g., input from an interactive pane of a user interface generated by the workspace generator 406) as input and returns a feed (e.g., a radiation therapy treatment plan including treatment attributes such as a radiation dose matrix) as output. The feed can be embedded into the conversational plan creation process (e.g., displayed on an interactive pane generated by the workspace generator 406). In some cases, the plan template can be split into sub-templates. In this case, the statement list can have separate references to different templates (e.g., there can be separate templates for clinical goals and non-clinical goals (e.g., maximum monitoring units (MU))), field geometry can be specified with different templates, there can be additional sets of goals used in the case of replanning in different templates, and so on.

[0118] The plan template selector 408 can be configured to select the most relevant treatment plan template based on the language description. For example, once the user describes the planned task, one or more prompts can be used to determine which treatment plan template is most suitable for performing the planned task and covering possible subtasks related to template selection, such as filling in a form. The plan template selector 408 can do this by accessing a database of defined templates, where the language description is attached to each template. The prompt template can be used to embed the user's description, the serialized patient description, and the plan template description to create a request for a machine learning language processing model to indicate how the current template is related to the description of the oncologist or other user and the serialized patient description. After performing such an analysis on each plan, another prompt template 420 can be used to create a query (e.g., an LLM query) to select the best matching plan template. For efficiency reasons, it may be beneficial to combine traditional filtering to first reject plan templates that are not obviously related before using the first prompt template 420 or after using the prompt template. For example, filtering can be based on keyword searches.

[0119] In a non-limiting example, an oncologist (e.g., a user) can invoke a conversational plan creation session in a user interface generated by the workspace generator 406. The oncologist can select a plan task by selecting a patient or from a list of assigned tasks, and verbally describe what he wants to achieve as a result. The database service 410 can obtain patient data (and other data related to the task) for the selected patient. In some cases, the descriptor function 414 can create a verbal patient description of the patient's anatomy. The patient description and the oncologist's description of the patient can be combined to generate a prompt (e.g., using a prompt template 420). The prompt can instruct the LLM agent 418 to request additional information or use the plan template selector 408 to retrieve the template that best matches the situation at hand (e.g., including data that is most similar to the data of the prompt template 420 compared to other templates). The oncologist can provide input to the user interface in multiple iterations, each iteration indicating different patient attributes that can be used to determine the patient's radiation therapy treatment plan. When sufficient information is available to generate a radiation therapy treatment plan for the patient, the interactive process can end.

[0120] The descriptor function 414 may generate a serialized plan instruction based on the patient's information. The serialized plan instruction may not necessarily define anything other than a plan template. The plan optimizer 416 may generate treatment attributes for the radiation therapy plan. The descriptor function 414 may create a serialized plan description based on the treatment attributes output by the plan optimizer 416. The serialized plan description may be input into a machine learning language processing model to convert the serialized plan description into a language radiation therapy treatment plan.

[0121] The workspace generator 406 can present the radiation therapy treatment plan on a user interface accessed by the oncologist. The workspace generator 406 can request feedback on the radiation therapy treatment plan. The radiation therapy treatment plan can be presented to the oncologist and feedback can be requested. The oncologist can reject the radiation therapy plan and provide a description of what is wrong with the plan and / or what the oncologist wishes to change. The description can be packaged as a prompt and used to create a new set of plan input information (e.g., adding more detailed instructions on top of the previous plan information). The plan optimizer 416 and the descriptor function 414 can generate an updated radiation therapy treatment plan based on the feedback. The workspace generator 406 can output the radiation therapy treatment plan on the user interface. The process can be repeated until the oncologist provides input indicating acceptance of the radiation therapy plan or not otherwise rejecting the radiation therapy treatment plan.

[0122] In some cases, an oncologist may input a list of assigned tasks (e.g., requests for plans for different patients). Tasks may be divided into phases with clearly defined subtasks that have specific outcomes that need to be communicated (e.g., selecting the correct plan template, filling out a form used in template refinement, or determining the next plan variation request). Each user feed may be processed with multiple prompts to determine the appropriate action.

[0123] Figure 5 5 shows a process flow chart of a process performed in a machine learning language processing plan generation system according to an embodiment. Method 500 may include steps 504 to 524. However, other embodiments may include additional or alternative steps, or may omit one or more steps entirely. Method 500 is described as being performed by, for example, Figure 1 However, one or more steps of method 500 may be performed by a server such as the analysis server described in Figure 1 For example, one or more computing devices may execute locally Figure 5 The analysis server may perform steps of method 500 whenever a user provides input or initiates a dialog with the machine learning language processing model of the analysis server to generate a radiation therapy treatment plan. Steps 504, 508, 510, 512, 516, 518, and 522 may be performed using LLM prompts (e.g., inputs to the machine learning language processing model). Steps 506, 514, and 524 may be performed using conventional business logic or algorithms.

[0124] At step 504, the analysis server generates a task selection prompt. The task selection prompt can answer the question whether the input at the user feed is a continuation of the current topic of the conversation or introduces a new topic (or returns to the old topic). If a topic change is detected at step 506, the appropriate change to the active topic stack is performed. At step 508, a request detection prompt can be used to evaluate whether the input at the user feed is a direct answer to the latest response of the machine learning language processing model, or whether the input is processed as an interrupt request (e.g., a question for additional information or a request to present certain data). If a request is detected, the process will continue to the request processing pipeline, in which LLM prompts can be used to first detect the actual request type, and then help format the request into a shape when necessary. At step 514, traditional business logic or algorithms can be used to resolve the request. After the request is resolved, at step 516, one or more LLM feeds can be used to present the continuation as the next LLM feed (or the previous LLM feed can be repeated). If, on the contrary, at step 508, the request detection does not find an interrupt request, at step 518, the user feed can be processed in the stage continuation prompt, with the intention of processing the input at the user feed as a continuation of the current active discussion. The result at the stage continuation step 518 can be that the user needs to provide more information before completing the stage at step 522, in which case more than one LLM feed can be used to create the next LLM feed (e.g., the next response of the machine learning language processing model), or the next LLM feed can be generated as part of the previous stage continuation LLM prompt. Alternatively, stage completion can be detected, and at step 524, task-related business logic can be used to determine the next stage or task.

[0125] The steps of method 500 may be performed in any order and / or combined in any combination. For example, request detection step 508 and request selection step 510 may be combined, task selection step 504 may be omitted, and so on.

[0126] As described herein, a task may include one or more atomic stages. Each stage may include the following information: (1) task / stage specific information known at the start of the task / stage, such as patient information involved; (2) task / stage completion conditions (e.g., a form that needs to be filled out or a choice that needs to be made before the system can proceed to the next stage or complete the entire task); (3) a stage continuation LLM prompt template. The stage continuation LLM prompt template may be a template text that includes language that can be used to embed certain user input into an LLM input feed that provides more context for controlling the LLM output. An example of a stage continuation LLM prompt template is as follows:

[0127] “Your task is to answer the following question as well as possible.Give the

[0128] answer in the

[0129] Following format:

[0130] Question:the question to be answered

[0131] Answer:your detailed answer

[0132] When forming your answer, consider also the following information:

[0133] {context Info}

[0134] Question:{question}

[0135] Answer:”

[0136] where {context info} and {question} are variable data that can be populated into the prompt. This template is a non-limiting illustrative example; (4) any additional conventional business logic for processing the user feed and operating in a manner that ultimately completes the stage. Such logic may include: (i) matching patterns for retrieving particularly interesting or relevant parts of the answer, which are usually at the beginning of the output feed, but in some cases it is important to only obtain a certain part of the output and use it at a later stage; and / or (ii) decision patterns, such as conventional functions that convert a certain matched output into an action. These actions may be or include, for example:

[0137] -invoking another LLM Powered task,

[0138] -invoking new LLM query with augmented input feed,

[0139] -waiting for additional information from the user,

[0140] -invoke a non-LLM task such as planning,or replanning,

[0141] -present certain data in the data pane,

[0142] -initiate new planning task,or end current planning task,invoke any associated

[0143] database

[0144] calls; and

[0145] (5) In some cases, LLM prompts include task / phase-specific components. For example, a request selection prompt may consider certain requests only when a specific phase is active. For example, a request to display doses is valid only if at least one radiation therapy treatment plan is available for the task or phase.

[0146] The computer may iteratively perform method 500 by performing the following tasks:

[0147] -When the computer is in a waiting state (e.g., in a standby state with no active tasks), any user-given instruction can call the generic LLM task to determine what the most appropriate action is. In the waiting state, the computer can support activities other than plan creation;

[0148] - When the computer determines that the instruction given by the user is a request to start a new planning session, the same set of instructions is used as input to the LLM to determine when the planning template selection can begin. The LLM can perform an iterative decision process to ask more questions to the user or collect information and invoke planning template selection. In some cases, another LLM can be called to generate a serialized patient description, such as by retrieving a serialized patient description from a database or by using a patient descriptor function;

[0149] -The computer can select a planning template for radiotherapy treatment planning;

[0150] - After selecting a plan template, the computer can begin the plan creation phase, such as using an LLM query to prepare the required input for the plan optimizer. Once the computer determines the best plan, the computer can present the radiation therapy treatment plan to the user (e.g., an oncologist);

[0151] - In some cases, the user can select an option to reject the radiation therapy treatment plan. In doing so, the user can enter further data about the treatment and / or the patient. The computer can provide input to the machine learning language processing model to repeat the plan generation process using the plan optimizer.

[0152] There may be auxiliary tasks (eg, LLM tasks or non-LLM tasks) that support the interaction. For example, each time a portion of LLM output is provided to the user, a specialized LLM prompt or non-LLM algorithm may be called to determine whether to provide the data in the data pane.

[0153] The described plan / replan task can be part of a larger more supportive task system. In this case, each user-given input can first pass through a dedicated LLM task to determine whether to continue the current task or abandon the task and go to a new task.

[0154] In some cases, specialized LLM prompts can be used to detect whether any portion of the user feed contains general comments that are to be stored for future use and then added to the stage continuation LLM prompt in future similar tasks.

[0155] In summary, a computer implementing the systems and methods described herein can store and provide customized LLM powered tasks for radiation therapy planning for use. The computer can use LLM powered queries for radiation therapy planning. For example, the computer can use LLM powered queries that include LLM prompt templates that can be used to construct LLM query input feeds, pattern matching algorithms that allow retrieval of specific portions of LLM query output feeds, and / or decision patterns that indicate impending reactions to specific portions of LLM query outputs, the pattern matching algorithms can include possible re-queries using enhanced LLM input feeds.

[0156] Examples of LLM queries may include queries for detecting whether a user-given language feed should initiate a new planning task, queries for detecting when a planning problem is described well enough to initiate planning template selection, queries for selecting the planning template that best fits a given language description, queries for converting given language planning instructions into a grammatically restricted list of tokens, queries for presenting an automatically generated grammatically restricted list of tokens describing a plan in a user-friendly format, queries for detecting when a replanning problem is described well enough to initiate replanning, and queries for converting a replanning description into planning instructions using a grammatically restricted list of tokens. The computer may organize different queries into predefined pipelines so that a single user feed may be processed in stages.

[0157] The computer can provide a user interface designed to support conversational interaction with a machine learning language processing model. The user interface can include a data pane presenting relevant (e.g., non-natural language) data, such as a CT image, a structural outline, or plan-related information, such as field geometry, dose distribution, and a clinical target table. The computer can use language to control which data is presented in the data pane on the user interface and which tags are presented in the interactive pane of the user interface (e.g., an identification string for a selection made in some embedded selection dialog box, combo box, etc.).

[0158] The computer may store in memory a database storing language descriptions mapped to radiation therapy data (e.g., non-serialized radiation therapy data). For example, these descriptions may be language text indicating any previous "conversations" associated with a particular RT data object, a list of grammatically restricted statements, etc. These descriptions may be programmatically created from the associated radiation therapy data object.

[0159] The computer can store and execute a descriptor function that can provide a serialized text (e.g., language text) description for a particular type of radiation therapy data (e.g., patient and plan data). The patient data descriptor function can be configured to generate a description based on a selected treatment plan template, such as by analyzing the results of a dose prediction model).

[0160] The computer can store and execute an optimizer configured to generate a radiation therapy treatment plan based on the serialized text description (e.g., generate treatment properties of the radiation therapy treatment plan). The optimizer can use a preformatted list of statements (or a similar list of tokens that can be parsed using a predefined parser) or language as input to generate such a radiation therapy treatment plan.

[0161] In some cases, the computer can provide suggestions for plan templates based on language descriptions provided by a user (e.g., a clinician) and / or derived from patient data. The computer can provide such suggestions based on LLM queries generated by the computer from input and using prompt templates. A system that provides plan template suggestions based on language descriptions provided by an oncologist and / or derived from patient data.

[0162] Training a machine learning language processing model for radiation therapy treatment planning

[0163] The interaction between human experts and radiation therapy (RT) treatment planning systems can be inefficient and require specialized knowledge to understand the impact of different data and user interface components. While large language models (LLMs) offer a new way to interpret language, standard general-purpose LLMs may not be as good as RT experts at collecting patient attributes for RT treatment planning or generating RT treatment plans.

[0164] Radiation therapy treatment planning may require interaction between the oncologist and a treatment planner using a planning tool that may require a significant amount of learning to use correctly. Recently, because the manual planning process can be slow, significant efforts have been made to increase automation of the treatment planning process to improve its efficiency. However, these efforts do not necessarily reduce the communication time between different specialists or require oncologists to adopt new skill sets in order to interact directly with automated planning systems.

[0165] LLMs have received a lot of attention recently and seem to be able to perform a variety of tasks related to processing language or code. For example, LLMs can be used to "answer" questions, where the answer is generated entirely based on parameters stored by the LLM itself. Unfortunately, the answers obtained through this "zero-shot" approach are often wrong. Current LLMs do not seem to know when the proposed answer is completely "hallucination", which makes it unfeasible to use it in situations where high-precision answers are required.

[0166] A more appropriate approach in the field of radiation therapy is to use a "few-shot" learning approach, where information relevant to the current question is given as part of the LLM feed. In practice, the relevant information is provided to the LLM along with the question or command. The LLM can handle relatively long feeds (e.g., thousands of words). Thus, the oncologist's verbal interactions can be converted into instructions for the automated planning system in a way that takes into account the current context (e.g., current patient information or previous discussions with the patient).

[0167] However, in some cases, RT experts may differ from the majority of the training corpus used to train the standard LLM, potentially reducing the accuracy of the LLM text continuation. In order for the LLM to learn RT treatment plans, the information needs to be in linguistic form rather than in the traditional way of segmenting the data into data fields in a database.

[0168] Computers implementing the systems and methods described herein can overcome the above technical problems. For example, a computer can provide interactions in RT treatment planning that are enhanced by specialized training to collect specialized LLMs of patient attributes for radiotherapy treatment planning. The LLM can also be trained using a given additional training data set to improve its accuracy. The LLM can support continued training of the model through additional training data. The systems and methods described herein can collect the necessary data required to fine-tune the LLM model, and such a fine-tuned LLM model can be used as part of completing the radiotherapy treatment planning task.

[0169] For example, Figure 6As shown, a treatment planning system 600 operating in a training environment can be configured to train and / or use a machine learning language processing model 602. The machine learning language processing model 602 can be an LLM, or otherwise similar to the machine learning language processing model 111 described above. The system 600 can also include a natural text collector unit 604. The natural text collector unit 604 can collect data that is later used in the transfer learning of the machine learning language processing model 602. The collected data can be stored in a corpus 606 of training texts, which can be a database for storing training texts. The LLM training unit 608 can use text from the training text corpus 606 to update or otherwise train the machine learning language processing model 602.

[0170] The data collected can vary, and the natural text collector unit 604 itself can have a variety of possible implementations. For example, the natural text collector unit 604 can be completely independent of any use of the LLM as part of a treatment planning activity. For example, the collected data may only contain language information related to the patient that was created before the treatment plan began. The collected data can contain, for example, a doctor's examination report, a radiologist's statement on the diagnostic CT scan, a pathologist's statement on any tissue samples, minutes of a tumor board meeting, or any information related to the patient's medical history. All of this information can be connected to the description provided as input to the treatment planning system 600.

[0171] Data collection can be based on a new communication tool between oncologists and planners. The tool can have two functions. First, the tool can assist and facilitate communication. Second, the tool can record the language interaction between two human experts and store the language interaction as an example of the radiation therapy field language they use.

[0172] In some cases, the natural language interactions can be collected by the natural text collector unit 604, and the oncologist can mark which parts of the interactions are most relevant. This may mean recording the complete interaction between the LLM-powered treatment planning system 600 and the user, or focusing on cases where the user edits the input to the automatic planning process themselves (e.g., cases where the AI ​​agent fails to provide the correct action based on the language information or converges too slowly to the correct conclusion).

[0173] The scope of training data collection for tuning purposes may vary. For example, for RT purposes, there may be a globally improved LLM, and there may also be a nationwide (or hospital group) tuned LLM model. In the case of a nationwide tuned LLM model, if the training data cannot leave the hospital, the training process may involve a federated learning approach. For example, LLM models can be trained locally in different hospitals or any other clinics based on training data generated in individual hospitals. The global LLM model can be trained based on updates to the weights and / or parameters of different local LLM models. In some cases, training data collection can be for specific hospitals or clinicians to generate a machine learning language processing model that is trained to handle local cases. Other ways to divide the training data can be specific types of patient cases, such as pelvic region cases or head and neck cases.

[0174] The transfer learning training steps for an LLM can follow supervised machine learning model training techniques. For example, when a start section is given as input to an LLM, a neural network presenting the current state of the LLM can be used to predict the rest of the individual training data situation. The predicted continuation is compared to the ground truth, and a loss function is applied to evaluate how close the replication of the correct continuation is. The error is propagated back to the neural network to determine how the state of the LLM needs to change. As is usually the case in transfer learning, the training of the LLM does not start with random parameter initialization, but from the state of the original model. This training can also be based on a previously tuned LLM. Training events can be started manually when enough data is available, or they can be carried out at a regular pace. It is also possible to use only part of the data if one wants to create a specialized model. In some cases, a joint training approach can be adopted, in which the training corpus is split between several databases and the training process only accesses one database at a time.

[0175] The natural text collector unit 604 can provide automatic anonymization for the collected patient data. For example, the natural text collector unit 604 can replace all tokens related to the patient's name with a common token (e.g., "Patient A") before storing the collected patient data in the corpus of training text 606. The replacement can be performed using classic pattern matching techniques and / or LLM hints.

[0176] Advantageously, by implementing the systems and methods described herein, LLM models can be better trained to collect patient attributes using specific medical language patterns and / or words related to the field of radiation therapy. Doing so may be important in the field of radiation therapy because the medical language used by radiation therapists may be brief compared to the language used in other contexts that are typically used to train general LLMs, such as novels, educational books, or scientific articles. The discussion patterns of radiation therapists may be different from those in books and more similar to the dialogue interaction patterns in theater or movies.

[0177] In a non-limiting example, an oncologist may access an electronic platform provided by an analysis server. Through the electronic platform, the analysis server may provide a user interface with which the oncologist may interact to perform radiation therapy treatment planning. For example, the analysis server may provide a user interface including an interactive pane in which the oncologist may provide input text, images, or videos. The analysis server may also include an auxiliary pane in the user interface that shows different tasks to be performed, such as generating radiation therapy plans for different patients. The analysis server may also include a data pane in the user interface that may display patient data for different patients.

[0178] The oncologist can communicate with the analysis server by providing input to any pane of the user interface. For example, the oncologist can write a text input of "show me the CT scan of patient A" in the interaction pane. The analysis server can receive the text input and use the text input to execute the machine learning language processing model to determine the action of retrieving the CT scan of patient A. The oncologist can move the CT scan of patient A to the interaction pane for use as input to the interaction pane.

[0179] Through the interaction pane, the oncologist can communicate with the machine learning language processing model to generate a radiation therapy treatment plan. For example, the oncologist can input a request to generate a radiation therapy treatment plan for patient B. The machine learning language processing model can receive the input request and generate a response requesting patient attributes of patient B at the interaction pane. In some cases, the oncologist can input different patient attributes into the interaction pane after retrieving the patient attributes from the data pane of the user interface. The oncologist can make such patient attributes appear on the data pane by inputting a request to view data through the interaction pane. The machine learning language processing model can process the patient attributes and output a response requesting different patient attributes until it is determined that enough patient attributes have been collected to generate a radiation therapy treatment plan (e.g., the patient attributes of patient B meet the template or plan conditions).

[0180] In response to determining that sufficient patient data or patient attributes have been collected for patient B, the analysis server may generate a token vector from the patient attributes. The analysis server may input the token vector into a plan optimizer. The plan optimizer may generate a radiation therapy treatment plan for patient B by generating optimized treatment attributes for the radiation therapy treatment plan. The analysis server may display the treatment attributes on a user interface being accessed by the oncologist.

[0181] The oncologist may review the radiation therapy treatment plan. The oncologist may determine that the radiation therapy treatment plan may be improved, and provide input to the interaction pane indicating that the plan may be improved. In response to doing so, the machine learning language processing model may generate queries for further and other patient attributes of patient B. The oncologist may provide other patient attributes, and the plan optimizer may generate a new radiation therapy treatment plan based on all patient attributes collected for patient B. The analysis server may display the radiation therapy treatment plan on the user interface. The oncologist may accept and implement the radiation therapy treatment.

[0182] The various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been generally described above according to their functions. Whether such functions are implemented as hardware or software depends on the specific application and the design constraints imposed on the entire system. Those skilled in the art may implement the described functions in different ways for each specific application, but such implementation decisions should not be interpreted as causing deviations from the scope of the present disclosure or claims.

[0183] Embodiments implemented in computer software can be implemented in software, firmware, middleware, microcode, hardware description language or any combination thereof. Code segments or machine executable instructions can represent any combination of processes, functions, subroutines, programs, routines, subroutines, modules, software packages, classes or instructions, data structures or program statements. Code segments can be coupled to another code segment or hardware circuit by transmitting and / or receiving information, data, independent variables, parameters or memory contents. Information, independent variables, parameters, data, etc. can be transmitted, forwarded or transmitted via any suitable means, including memory sharing, message transmission, tag transmission, network transmission, etc.

[0184] The actual software code or specialized control hardware used to implement these systems and methods does not limit the claimed features or the present disclosure. Therefore, while the operation and behavior of the systems and methods are described without reference to specific software code, it should be understood that software and control hardware can be designed to implement the systems and methods based on the description herein.

[0185] When implemented in software, the function can be stored as one or more instructions or codes on a non-transient computer-readable or processor-readable storage medium. The steps of the method or algorithm disclosed herein can be embodied in a processor executable software module, which can reside on a computer-readable or processor-readable storage medium. Non-transient computer-readable or processor-readable media, including computer storage media and tangible storage media that facilitate the transfer of computer programs from one place to another. Non-transient processor-readable storage media can be any available medium that a computer can access. As an example and not limitation, such non-transient processor-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, or any other tangible storage medium that can be used to store the required program code in the form of instructions or data structures and can be accessed by a computer or processor. Disks and optical disks as used herein include compact disks (CDs), laser disks, optical disks, digital versatile disks (DVDs), floppy disks and blue optical disks, wherein disks usually reproduce data magnetically, and optical disks reproduce data optically with lasers. The above combination should also be included in the scope of computer-readable media. Furthermore, the operations of a method or algorithm may reside as one or any combination or set of codes and / or instructions on a non-transitory processor-readable medium and / or computer-readable medium, which may be incorporated into a computer program product.

[0186] The foregoing description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the embodiments described herein and variations thereof. Various modifications to these embodiments will be apparent to those skilled in the art, and the principles defined herein may be applied to other embodiments without departing from the spirit or scope of the subject matter disclosed herein. Therefore, the present disclosure is not intended to be limited to the embodiments shown herein, but should be consistent with the following claims and the broadest scope of the principles and novel features disclosed herein.

[0187] While various aspects and embodiments have been disclosed, other aspects and embodiments are also contemplated. The various aspects and embodiments disclosed are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.

Claims

1. A method comprising: A processor presents a user interface, wherein the user interface provides an interactive interface between a user and the machine learning language processing model; receiving, by the processor, from the interactive interface, a first input including a first patient attribute of a patient; executing, by the processor, the machine learning language processing model using the first patient attribute of the patient as input to generate a response requesting a second patient attribute of the patient; presenting, by the processor, at the interactive interface, the response requesting the second patient attribute of the patient; receiving, by the processor, from the interactive interface, a second input including the second patient attribute of the patient; as well as In response to determining that the first patient attribute and the second patient attribute satisfy a planning condition, transmitting, by the processor, the first patient attribute and the second patient attribute of the patient to a radiation therapy plan optimizer, Wherein the radiation therapy plan optimizer is configured to generate a radiation therapy treatment plan for the patient based on the first patient attribute and the second patient attribute.

2. The method according to claim 1, further comprising: receiving, by the processor at the user interface, input indicative of the patient, wherein executing the machine learning language processing model comprises: retrieving, by the processor, stored patient data regarding the patient from a memory based on the input indicative of the patient; and The response is determined by the processor based on the input first patient attribute and the retrieved patient data.

3. The method according to claim 1, further comprising: retrieving, by the processor, from a memory a template, the template comprising a type list of patient attributes, Wherein executing the machine learning language processing model includes: the processor determining the response based on the input first patient attribute and the template.

4. The method of claim 3, wherein determining the response comprises: determining, by the processor, the first type of patient attribute of the type list of patient attributes of the template based on the machine learning language processing model not receiving a value for the first type of patient attribute from the interactive interface; as well as The response to requesting a value for the first type of patient attribute is determined by the processor.

5. The method according to claim 3, further comprising: comparing, by the processor, the first patient attribute and the second patient attribute to the template; as well as In response to determining that the first patient attribute and the second patient attribute satisfy a template condition of the template, determining, by the processor, that the first patient attribute and the second patient attribute satisfy the plan condition.

6. The method according to claim 1, further comprising: receiving, by the processor at the user interface, input indicative of the patient; presenting, by a processor, at the user interface, patient data for the patient based on the input indicative of the patient, the patient data comprising one or more patient attributes of the patient; as well as A selection of the first patient attribute is received, by the processor, from the one or more presented patient attributes.

7. The method according to claim 1, further comprising: generating, by a processor, a labeled vector representing the first patient attribute and the second patient attribute from the machine learning language processing model, The transmitting the first patient attribute of the patient and the second patient attribute of the patient to the radiation therapy plan optimizer includes: transmitting, by the processor, the label vector representing the first patient attribute and the second patient attribute to the radiation therapy plan optimizer.

8. The method according to claim 7, further comprising: converting, by the processor, the tag vector representing the first patient attribute and the second patient attribute into a structured data set, Wherein transmitting the first patient attribute of the patient and the second patient attribute of the patient to the radiation therapy plan optimizer includes: transmitting, by the processor, the converted structured data set to the radiation therapy plan optimizer.

9. The method of claim 1, wherein the radiation therapy plan optimizer is configured to generate the radiation therapy treatment plan by generating radiation dose information for treating the patient based on at least the first patient attribute and the second patient attribute.

10. The method according to claim 1, further comprising: receiving, by the processor, input from the user interface indicating that the radiation therapy treatment plan is incorrect; executing, by the processor, the machine learning language processing model to generate a second response requesting a third patient attribute regarding the patient in response to input indicating that the radiation therapy treatment plan is incorrect; presenting, by the processor, at the interactive interface, the second response requesting the third patient attribute of the patient; receiving, by the processor, from the interactive interface, a third input including the third patient attribute of the patient; as well as transmitting, by the processor, to the radiation therapy plan optimizer, the first patient attribute of the patient, the second patient attribute of the patient, and the third patient attribute of the patient, Wherein the radiation therapy plan optimizer is configured to generate a second radiation therapy treatment plan for the patient based on the first patient attribute, the second patient attribute, and the third patient attribute.

11. The method according to claim 1, further comprising: storing, by the processor, a training data set comprising a set of text regarding radiation therapy treatments of patients; as well as The machine learning language processing model is trained by the processor using as input the set of text regarding radiation therapy treatments of patients in the training data set.

12. The method according to claim 1, further comprising: The radiation therapy treatment plan is presented, by the processor, at the user interface.

13. A system comprising: A processor coupled to the memory and configured to: Presents a user interface that provides an interactive interface between the user and the machine learning language processing model receiving, from the interactive interface, a first input including a first patient attribute of a patient; executing the machine learning language processing model using the first patient attribute of the patient as input to generate a response requesting a second patient attribute of the patient; presenting, at the interactive interface, the response requesting the second patient attribute of the patient; receiving, from the interactive interface, a second input including the second patient attribute of the patient; as well as In response to determining that the first patient attribute and the second patient attribute satisfy a planning condition, transmitting the first patient attribute and the second patient attribute of the patient to a radiation therapy plan optimizer, Wherein the radiation therapy plan optimizer is configured to generate a radiation therapy treatment plan for the patient based on the first patient attribute and the second patient attribute.

14. The system of claim 13, wherein the processor is further configured to: receiving, at the user interface, input indicative of the patient, The processor is configured to execute the machine learning language processing model in the following manner: retrieving stored patient data regarding the patient from a memory based on the input indicative of the patient; and The response is determined based on the input first patient attribute and the retrieved patient data.

15. The system of claim 13, wherein the processor is further configured to: retrieving a template from memory, the template comprising a type list of patient attributes, Wherein the processor is configured to execute the machine learning language processing model by determining the response based on the input first patient attribute and the template.

16. The system of claim 15, wherein the processor is configured to determine the response by: determining the first type of patient attribute of the type list of patient attributes of the template based on the machine learning language processing model not receiving a value for the first type of patient attribute from the interactive interface; and The response to requesting a value of the first type of patient attribute is determined.

17. The system of claim 15, wherein the processor is further configured to: comparing the first patient attribute and the second patient attribute to the template; and In response to determining that the first patient attribute and the second patient attribute satisfy the template condition of the template, determining that the first patient attribute and the second patient attribute satisfy the planning condition.

18. The system of claim 13, wherein the processor is further configured to: receiving, at the user interface, input indicative of the patient; presenting, at a user interface, patient data for the patient based on the input indicative of the patient, the patient data comprising one or more patient attributes of the patient; and A selection of the first patient attribute is received from the presented one or more patient attributes.

19. The system of claim 13, wherein the processor is further configured to: generating a labeled vector representing the first patient attribute and the second patient attribute from the machine learning language processing model, Wherein the processor is configured to transmit the first patient attribute of the patient and the second patient attribute of the patient to the radiation therapy plan optimizer by transmitting the marker vector representing the first patient attribute and the second patient attribute to the radiation therapy plan optimizer.

20. The system of claim 19, wherein the processor is further configured to: converting the label vector representing the first patient attribute and the second patient attribute into a structured data set, Wherein the processor is configured to transmit the first patient attribute of the patient and the second patient attribute of the patient to the radiation therapy plan optimizer by transmitting the converted structured data set to the radiation therapy plan optimizer.

Images