Generative motion modeling using external and internal anatomical information

Through deep learning models to analyze and predict patient movements, the problems of real-time dose calculation and anatomical motion visualization in IGRT are solved, and a more accurate radiation therapy process is achieved.

CN118922894APending Publication Date: 2024-11-08SIEMENS HEALTHINEERS INTERNATIONAL AG
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202380025795.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-07
Filing Date
2023-03-06
Publication Date
2024-11-08

Smart Images

  • Figure CN118922894A_ABST
    Figure CN118922894A_ABST
Patent Text Reader

Abstract

Provided herein are methods and systems for training and executing a motion model that uses artificial intelligence methodology (e.g., deep learning) to learn and predict the location of an internal structure of a patient. A method (200) includes receiving (202) respiration data of a patient from an electronic sensor in addition to a medical image, such as a kV image; executing (204) an artificial intelligence model (111) using the respiration data and predicting deformation data of at least one internal structure of the patient, where the artificial intelligence model (111) is trained according to a training data set comprising a set of participants, their corresponding respiration data, and their corresponding deformation data; and outputting (206) the predicted deformation data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application generally relates to the use of data analysis techniques to model and predict patient attributes during radiation therapy treatment, and to control radiation therapy machines. Background Art

[0002] One of the major challenges in image-guided radiation therapy (IGRT) is to account for various types of patient motion. The motion can be both circulatory (e.g., components of breathing and cardiac motion) and irregular (e.g., gastrointestinal events including peristalsis, swallowing and air bubble passage, muscle relaxation during breath holding, and body and limb movement).

[0003] IGRT attempts to mitigate the effects of motion in many ways, but there are two major drawbacks to IGRT. First, an estimate of the delivered dose (relative to the planned dose) is not usually calculated. Second, there is no real-time three-dimensional (3D) volumetric depiction of the patient's anatomical structure / motion during treatment to visualize it. Although there are imaging techniques that attempt to address the motion of the respiratory or cardiac cycle, conventional imaging equipment and systems do not provide real-time resolved 3D information about the patient at each time instance. In addition, conventional four-dimensional (4D) imaging techniques (e.g., positron emission tomography (PET), magnetic resonance (MR) imaging, computed tomography (CT) and / or cone beam computed tomography (CBCT)) typically rely on retrospective reconstruction of data and cannot be performed in real time or actively. Summary of the invention

[0004] According to a first aspect of the present invention, a method as defined in claim 1 is provided.

[0005] According to a second aspect of the invention, there is provided a computer system as defined in claim 11 .

[0006] Optional features are defined in the dependent claims.

[0007] For the above reasons, a system is desired that can quickly and accurately analyze patient information and provide the projected location of the patient's internal structures. Using the methods and systems discussed herein, a computer model (e.g., an artificial intelligence (AI) model) can take into account patient movement. Using the methods and systems discussed herein, a processor can use deep learning to train intra-patient and / or inter-patient motion models. Applications of these computer models can be in the following areas: real-time tissue tracking during radiation beam delivery, real-time motion visualization, retrospective and / or real-time delivery dose calculation, organ / segmentation (e.g., gross tumor volume (GTV)), specific dose tracking, outcome prediction, and / or image reconstruction.

[0008] The methods and systems discussed herein (unlike classical tracking methods) can allow AI models to learn the space of anatomically feasible deformations and therefore have the ability to infer 3D anatomical structures from limited data (e.g., single kV projections or appropriate initialization with proxy signals from respiration, stereoscopic kV imaging, kV+MV imaging, and digital tomosynthesis (kV and MV), or combinations of these, proxy signals such as patient 3D surface, real-time position management (RPM), or electrocardiogram (ECG) traces). AI models can be trained to analyze and predict patient motion (both external and internal). These AI models can be trained to use real-time input received from the patient and / or external sensors.

[0009] One or more AI models can be trained to predict physiologically effective deformations. In the field of radiation therapy, physiologically effective deformations can be key to several applications, such as dose accumulation, structure propagation, and tracking of tissues with little to no contrast. The AI ​​models discussed herein can be used to predict various deformations, and the results of the predictions can be transmitted to one or more downstream applications, where other software technology solutions can use the predicted data and calculate / predict other properties required to achieve and execute the patient's treatment.

[0010] One or more AI models can learn and exploit correlations between moving tissues of a patient. By learning the relative motion of tissues (based on a population of patients within a training data set, and extrapolating data based on the specific patient being treated), the AI ​​model can infer / predict local deformations from limited information. The information can be (but not limited to) one or a combination of the following: proxy signals for breathing or heartbeat (e.g., RPM, 3D surface, ECG), acquired kV projections (e.g., during treatment triggering or fluoroscopic images and / or tomosynthesis), high contrast objects detected in projections (such as metal markers), ultrasound or radar measurements, or other. The model can be fit to the data or correlated to the corresponding signal.

[0011] One or more AI models can incorporate and analyze data associated with the patient's physiological dynamics. Temporal analysis of biomechanics and modeling thereof can additionally be used to predict patient anatomy.

[0012] One or more AI models can be trained as multimodal motion models. Since motion models encode possible deformations within the body, they can in principle be independent of the underlying imaging technology. This means that it becomes feasible to jointly use image sequences from different sources to build such models or to adapt them for each patient individually.

[0013] One or more AI models can be trained to correspond to a common reference anatomy. The AI ​​model can encode deformations relative to a certain reference anatomy, which can be a defined patient-specific template (e.g., CT simulation) or a subject-to-subject patient atlas. This allows novel intra-fraction and / or inter-fraction dose accumulation, structural propagation, and / or outcome prediction methods derived from population statistics.

[0014] One or more AI models can constrain motion compensated image reconstruction. Since motion models can provide prior information about possible deformations in the body in the form of probabilities, they can be applied during image reconstruction, where deformations are predicted based on limited information.

[0015] A motion model based on deep learning can be trained on a motion-resolved volume image across different patients. The model can then be used to predict the current deformation of a patient relative to a volume static image (e.g., planned CT or CBCT) of the same patient based on one or more proxy signals (e.g., RPM, ECG, 3D surface, one or more kV projections, digital tomosynthesis and / or ultrasound, which can also include stereoscopic imaging). The AI ​​model discussed herein can be applied in at least two different ways to help patient treatment. First, the AI ​​model can be applied during post-treatment verification. Second, the AI ​​model can be applied during treatment for real-time or near real-time prediction. The prediction of deformation can be used for visualization purposes, tracking of structures identified on static images, deformation of planned doses (beamlets), and / or recalculation of delivered doses. In particular, deformation-related dose calculations enable tracking of PTV and / or organ-at-risk (OAR) doses, with the goal of achieving dose constraints using the dose actually delivered.

[0016] The AI ​​model can utilize data associated with a group of patients (data across patients or groups of patients), combined with patient-specific information. This allows the model to be trained using a group of participants (e.g., previously treated patients). The AI ​​model can then use the training to predict deformations associated with the patient. The methods and systems discussed herein (e.g., using deep learning to train AI models) enable the combination of patient-specific information in conjunction with training obtained from a group of patients. This overcomes the limitation (overfitting) of patient-specific models representing novel patient motion states (outside the scope of the model).

[0017] Models can learn temporal and / or biomechanical knowledge associated with patients and groups of patients. Due to the high complexity of classical methods (e.g., finite elements) for modeling biodynamics, it has not been possible to exploit certain temporal and / or biomechanical properties in clinical practice to date. The data-driven methods described herein can enable the use of temporal and / or biomechanical knowledge. AI models can learn anatomical and physiological properties of patients and / or groups of patients, which was previously impossible / infeasible (e.g., using deformable image registration (DIR), followed by sequential application of modeling).

[0018] Compared to classical approaches, where modeling is usually based on standard deformable image registration, the use of AI models trained in an unsupervised or semi-supervised model has the potential to learn anatomical properties such as sliding interfaces. In this way, physiological deformations of the bone can be blocked, which makes the model suitable for dose calculations.

[0019] The trained AI model can also provide anatomically valid interpolation. In order to predict continuous anatomical deformations, it may be necessary to interpolate between discrete time steps represented by the AI ​​model (for example, piecewise linear interpolation between phases of 4D CT / CBCT can be used). The AI ​​model discussed in this article can utilize a trained autoencoder so that interpolation in the latent space produces anatomically valid intermediate steps.

[0020] Compared to heuristically designed or classically optimized methods, the AI ​​models discussed in this article allow for much more complex loss / objective functions. This allows the model predictions to be optimized directly with respect to clinical applications (e.g., dose calculation or segmentation). Therefore, the predictions provided by the AI ​​model can be used as the basis for downstream dose calculation software technical solutions. The AI ​​model can be trained so that it is an imaging modality agnostic model. In addition to utilizing information across patients, the AI ​​model can also be trained across different modalities. This allows the prediction of reasonable deformations in areas where kV imaging does not show tissue contrast. As a result, the trained model can be applied to image reconstruction (due to additional expressibility).

[0021] Compared to classical motion models (e.g., those based on principal component analysis (PCA)), deep learning-based models can be nonlinear and generalize better to novel anatomical structures. Therefore, the model can be used as a guide during (e.g., motion-compensated) reconstruction or treatment without introducing additional / inappropriate deformations.

[0022] The AI ​​model can provide predictions of physiologically effective deformations. Conventional methods such as the DIR method map the intensities onto each other while constraining the deformation with a smoothness term. This simple smoothness assumption is insufficient for various tasks in radiation therapy such as dose deformation and accumulation. This is because for a given image modality, certain organs show uniform grayscale values, and therefore the estimated deformation within this region is completely defined by the smoothness assumption. On the other hand, the smoothness assumption does not allow for discontinuities in deformation such as at the sliding interface between the abdominal cavity and the liver.

[0023] The AI ​​model can be trained by applying high-level loss functions, model constraints, or using supervised methods with known effective deformations. The loss function and / or model constraints can be defined so that they take into account deformability (e.g., rigid bone and / or compressible lung tissue) or tissue-related changes in sliding interfaces. The model can take into account the cyclic nature of periodic motion, and the analysis server can constrain such data. For example, features with high contrast but no meaningful correspondence (such as moving air cavities) can be ignored. Knowing the deformation of the patient's anatomy in this way allows dose accumulation on the reference anatomy. For example, physiologically constrained deformations can allow artifacts in dose to be avoided.

[0024] The AI ​​model can allow for accumulation due to incorrect deformations. Accumulation of deformed doses can be applied to deformation sequences during or after treatment. This allows dose monitoring of different structures during treatment and taking actions (e.g., beam holding) or after beam delivery to adapt or verify / record the treatment.

[0025] The AI ​​model can also allow outcome prediction by mapping the applied dose to a single atlas per patient or across patients and extrapolating using data collected during follow-up. The AI ​​model can also use prior knowledge for motion compensation and / or image reconstruction.

[0026] The AI ​​model can provide learning and exploiting the correlation between moving tissue and proxy signals. Certain conventional systems (e.g., conventional standard of care using motion management) assume a strong correlation between the tumor and the proxy signal (such as RPM signal or 3D surface). Since the AI ​​model discussed in this article can predict the patient anatomy (especially the correlation between moving tissues), it can be enabled in addition to the prediction of GTV motion relative to the proxy, for example, the position estimation of OARs (allowing motion management strategies that minimize the dose to healthy tissue).

[0027] Because the AI ​​model discussed in this article can encode the correlation between any structure in the body, it can be applied to improve tracking through constraints (e.g., the detected tumor location based on detected landmarks). For example, the AI ​​model can identify that the tumor can only move tangentially with the rib cage or move almost in line with the diaphragm or marker. Modeling of anatomical parts also enables the use of a combination of agents such as kV projections, ECG, RPM, 3D surfaces, radar, or ultrasound. For example, in non-coplanar beam incidence, the model and its predictions can temporarily prevent imaging during treatment (or provide an option for medical professionals to stop imaging during treatment), and the model can predict patient anatomical changes based solely on agents. During deformable image registration, known correlations between points can also be used to constrain possible technical solutions (e.g., within an organ, at a sliding interface, or at a rigid bone).

[0028] The AI ​​model can provide a data-driven incorporation of physiological dynamics. In some embodiments, the AI ​​model can learn dynamic processes from training data. The AI ​​model can learn anatomically relevant constraints on acceleration. In addition, the AI ​​models discussed herein can utilize recurrent networks such as LSTM models to learn time-related / sequential or periodic aspects.

[0029] The AI ​​model can provide multimodal motion services. The AI ​​model can also be trained to transform images into other modalities. This technology can be used to correspond images of different modalities and use them for model construction based on different modalities such as 4D MRI and 3D / 4D CT. In addition, the analysis server can train a modality-agnostic predictive deformable registration algorithm. This can be done based on a suitable similarity metric that can compare different modalities, based on image pairs with the same geometry / anatomy so that the correspondence is known, or based on simulated images of different modalities.

[0030] The AI ​​model can be generated and trained relative to one (or more) common reference anatomical structures / atlases. The population-based model has the ability to factorize patient images into anatomical appearance (e.g., patient-specific tissues and / or fillings of the gastrointestinal tract) and / or shape. The AI ​​model can use this technique to compare images (patient images and / or predicted or reconstructed images) with a reference (e.g., a common reference image). Using this technique, the AI ​​model can establish comparability between different subjects and therefore statistically evaluate them (e.g., the distribution of lung tumors relative to clinical consequences).

[0031] AI models can also be used to constrain motion-compensated image reconstruction. One way to improve image quality in image reconstruction is to estimate the motion that occurs throughout the image acquisition and compensate for it (e.g., by warping to a common reference). Already pathological reconstruction becomes more challenging due to the need to additionally estimate deformations for image content. Using classical optimization algorithms as iterative reconstruction or deformable image registration means that the complexity of such constraints is limited to allow convergence. Finite element methods introduce more accurate physiological models, but are computationally intensive and difficult to implement. AI models can be trained in the form of probabilistic models that capture prior information to learn the rationality of deformations. This prior information can then be used throughout the reconstruction process to constrain the space of possible solutions. This can allow effective prediction of deformations based on limited data captured throughout the acquisition (e.g., due to time or dose constraints).

[0032] In one embodiment, a method includes: receiving, by a processor, breathing data of a patient from an electronic sensor; executing, by the processor, an artificial intelligence model using the breathing data and predicting deformation data of at least one internal structure of the patient, wherein the artificial intelligence model is trained based on a training data set comprising a group of participants, their corresponding breathing data, and their corresponding deformation data; and outputting, by the processor, the predicted deformation data.

[0033] The method may also include receiving, by a processor, a medical image of the patient, wherein the processor executes the artificial intelligence model using the medical image.

[0034] The respiratory data received from the electronic sensor may be at least one of chest position, chest movement, or respiratory cycle data of the patient.

[0035] The deformation data may correspond to movement of at least one internal structure of the patient.

[0036] The method may also include adjusting, by the processor, at least one property of the radiation therapy machine based on the predicted deformation data.

[0037] The at least one property may correspond to a multi-leaf collimator opening.

[0038] The output predicted deformation data may correspond to a simulated medical image depicting the anatomical region of the patient.

[0039] Outputting the predicted deformation data may correspond to transmitting the predicted deformation data to a dose calculation software solution or a tissue tracking software solution.

[0040] The artificial intelligence model may generate predicted breathing data associated with the patient, the predicted breathing data comprising at least one of a property of a breathing cycle or chest movement.

[0041] The electronic sensor can be a wearable breath sensor or an optical breath sensor.

[0042] In another embodiment, a computer system includes a server comprising a processor and a non-transitory computer-readable medium containing instructions, which when executed by the processor causes the processor to perform operations including: receiving breathing data of a patient from an electronic sensor; using the breathing data to execute an artificial intelligence model and predict deformation data of at least one internal structure of the patient, wherein the artificial intelligence model is trained based on a training data set, the training data set comprising a group of participants, their corresponding breathing data, and their corresponding deformation data; and outputting predicted deformation data.

[0043] The instructions may also cause the processor to receive a medical image of a patient, wherein the processor executes the artificial intelligence model using the medical image.

[0044] The respiratory data received from the electronic sensor may be at least one of chest position, chest movement, or respiratory cycle data of the patient.

[0045] The deformation data may correspond to movement of at least one internal structure of the patient.

[0046] The instructions may also cause the processor to adjust at least one property of the radiation therapy machine based on the predicted deformation data.

[0047] The at least one property may correspond to a multi-leaf collimator opening.

[0048] The output predicted deformation data may correspond to a simulated medical image depicting the anatomical region of the patient.

[0049] Outputting the predicted deformation data may correspond to transmitting the predicted deformation data to a dose calculation software solution or a tissue tracking software solution.

[0050] The artificial intelligence model may generate predicted breathing data associated with the patient, the predicted breathing data comprising at least one of a property of a breathing cycle or chest movement.

[0051] The electronic sensor can be a wearable breath sensor or an optical breath sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Non-limiting embodiments of the present disclosure are described by way of example with reference to the accompanying drawings, which are schematic and are not intended to be drawn to scale. Unless indicated to represent background art, the drawings represent aspects of the present disclosure.

[0053] Figure 1 Components of an artificial intelligence motion modeling system according to one embodiment are illustrated.

[0054] Figure 2 A process flow diagram of an artificial intelligence motion modeling system according to one embodiment is illustrated.

[0055] Figure 3 Illustrated is a visual representation of respiratory data for a group of patients according to one embodiment.

[0056] Figure 4 Illustrated is a visual representation of respiratory data and a corresponding medical image depicting movement of one or more internal structures according to one embodiment.

[0057] Figure 5 Illustrated is a visual representation of a vectorized medical image according to one embodiment.

[0058] Figure 6 Illustrated is a visual representation of a trained AI model according to one embodiment.

[0059] Figure 7 Illustrated is a visual representation of a medical image analyzed and generated by an AI model according to one embodiment.

[0060] Figure 8 Illustrated is a visual representation of deformation vectors according to one embodiment. DETAILED DESCRIPTION

[0061] Reference will now be made to the illustrative embodiments depicted in the accompanying drawings, and specific language will be used herein to describe the illustrative embodiments. However, it will be understood that it is not intended to limit the scope of the claims or the disclosure thereby. Changes and further modifications to the inventive features illustrated herein and additional applications of the principles of the subject matter illustrated herein that would occur to a person skilled in the relevant art and in possession of the disclosure are to be considered within the scope of the subject matter disclosed herein. Other embodiments may be used, and / or other changes may be made without departing from the spirit or scope of the disclosure. The illustrative embodiments described in the detailed description are not meant to limit the subject matter presented.

[0062] Figure 1 Components of a system 100 for an artificial intelligence motion modeling system according to one embodiment are illustrated. The system 100 may include an analysis server 110a, a system database 110b, an AI model 111, electronic data sources 120a-d (collectively, electronic data sources 120), end-user devices 140a-c (collectively, end-user devices 140), an administrator computing device 150, and a medical device 160, (one or more) medical device computers 162, and / or a respiratory sensor 163. Figure 1The various components depicted in the figure may belong to a radiation therapy clinic where a patient may, in some cases, receive radiation therapy treatment via one or more radiation therapy machines (e.g., medical device 160) located within the clinic. Additionally or alternatively, the AI ​​model 111 may be implemented using any 4D image, such as a 4D-MRI, that has been acquired for any other purpose that may not be connected to radiation therapy.

[0063] The system 100 is not limited to the components described herein, and may include additional components or other components (not shown for brevity) that are considered to be within the scope of the embodiments described herein.

[0064] The above components can be connected to each other through a network 130. Examples of network 130 may include, but are not limited to, private or public local area networks (LANs), wireless LAN (WLAN) networks, metropolitan area networks (MANs), wide area networks (WANs), and the Internet. Network 130 may include wired and / or wireless communications according to one or more standards and / or via one or more transport media. Communications through network 130 may be performed according to various communication protocols such as transmission control protocol and Internet protocol (TCP / IP), user datagram protocol (UDP), and IEEE communication protocols. In one example, network 130 may include wireless communications according to a Bluetooth specification set or another standard or proprietary wireless communication protocol. In another example, network 130 may also include communications through a cellular network, including, for example, GSM (Global System for Mobile Communications), CDMA (Code Division Multiple Access), EDGE (Enhanced Data for Global Evolution) networks.

[0065] The analysis server 110a may generate and display an electronic platform configured to use various AI models 111 (including artificial intelligence and / or machine learning models) to receive patient information and output the results of the execution of the AI ​​models 111. The electronic platform may include a graphical user interface (GUI) displayed on each electronic data source 120, end-user device 140, medical device 160, and / or administrator computing device 150. An example of an electronic platform generated and hosted by the analysis server 110a may be a web-based application or a website configured to be displayed on different electronic devices, such as mobile devices, tablet computers, personal computers, etc.

[0066] The information displayed by the electronic platform may include, for example, input elements to receive data associated with a patient being treated, synchronize one or more sensors such as patient sensors 163, and display results of predictions generated by the AI ​​model 111 (e.g., a reconstructed image of a patient showing the location of a tumor predicted by the AI ​​111). For example, the analysis server 110a may execute the AI ​​model 111 (e.g., a machine learning model trained to generate predicted tumor locations and / or breathing patterns of a patient being treated via the medical device 160). The analysis server 110a may then display the results to a medical professional and / or directly correct one or more operational properties of the medical device 160. In some embodiments, the medical device 160 may be a diagnostic imaging device or a treatment delivery device.

[0067] The analysis server 110a may be any computing device including a processor and a non-transitory machine-readable storage device capable of performing the various tasks and processes described herein. The analysis server 110a may employ various processors, such as a central processing unit (CPU) and a graphics processing unit (GPU), etc. Non-limiting examples of such computing devices may include a workstation computer, a laptop computer, a server computer, etc. Although the system 100 includes a single analysis server 110a, the analysis server 110a may include any number of computing devices operating in a distributed computing environment, such as a cloud environment.

[0068] The electronic data source 120 may represent various electronic data sources that contain, retrieve, and / or access data associated with the medical device 160, such as operational information associated with previously performed radiation therapy treatments (e.g., electronic log files or electronic configuration files), data associated with previously monitored patients (e.g., breathing patterns, tumor locations, deformation information), or data associated with participants in a study for training the AI ​​models discussed herein. For example, the analysis server 110a may use a clinic computer 120a, a medical professional device 120b, a server 120c (associated with a physician and / or clinic), and a database 120d (associated with a physician and / or clinic) to retrieve / receive data associated with the medical device 160. The analysis server 110a may retrieve data from the end-user device 140, generate a training data set, and train the AI ​​model 111. The analysis server 110a may execute various algorithms to convert raw data received / retrieved from the electronic data source 120 into a machine-readable object that may be stored and processed by other analysis processes as described herein.

[0069] The end-user device 140 may be any computing device including a processor and a non-transient machine-readable storage medium capable of performing the various tasks and processes described herein. Non-limiting examples of the end-user device 140 may be a workstation computer, a laptop computer, a tablet computer, and a server computer. In operation, various users may use the end-user device 140 to access the GUI that is operationally managed by the analysis server 110a. Specifically, the end-user device 140 may include a clinic computer 140a, a clinic server 140b, and a medical professional device 140c. Even though referred to herein as "end-user" devices, these devices may not always be operated by end users. For example, the clinic server 140b may not be used directly by the end user. However, the results stored on the clinic server 140b may be used to fill in various GUIs accessed by the end user via the medical professional device 140c.

[0070] The administrator computing device 150 may represent a computing device operated by a system administrator. The administrator computing device 150 may be configured to: display radiation therapy treatment attributes generated by the analysis server 110a (e.g., various analytical metrics determined during training of one or more machine learning models and / or systems); monitor various models 111 utilized by the analysis server 110a, electronic data sources 120, and / or end-user devices 140; review feedback; and / or facilitate training or retraining (calibration) of the AI ​​models 111 maintained by the analysis server 110a.

[0071] The medical device 160 may be a radiotherapy machine configured to implement radiotherapy treatment of a patient. The medical device 160 may also include an imaging device capable of emitting radiation, so that the medical device 160 can perform imaging according to various methods to accurately image the internal structure of the patient. For example, the medical device 160 may include a rotation system (e.g., a static or rotational multi-view system). A non-limiting example of a multi-view system may include a stereoscopic system (e.g., two systems may be arranged orthogonally). The medical device 160 may also communicate with a medical device computer 162, which is configured to display the various GUIs discussed herein. For example, the analysis server 110a may display the results predicted by the AI ​​model 111 on the computing device described herein.

[0072] The medical device 160 may also include one or more sensors configured to monitor the patient being treated. For example, the medical device 160 may include a 3D surface mechanism and a sensor (e.g., an optical sensor) configured to monitor the patient's movement (e.g., how the patient is moving and / or breathing). In some embodiments, the medical device 160 may communicate with a respiratory sensor 163. The respiratory sensor 163 may be any sensor configured to monitor the patient's breathing. For example, the respiratory sensor may be a strap configured to monitor the patient's chest position and movement, whereby a processor (e.g., a processor inside the respiratory sensor 163 or analysis server 110a) may analyze to identify how the patient is breathing. The data received from the respiratory sensor 163 may also be referred to as a proxy signal.

[0073] The AI ​​model 111 may be stored in the system database 110b. The AI ​​model 111 may be trained using data received / retrieved from the electronic data source 120 and may be executed using data received from the end-user device, the medical device 160, and / or the sensor 163. In some embodiments, the AI ​​model 111 may reside in a local or dedicated data repository of the clinic. In various embodiments, the AI ​​model 111 uses one or more deep learning engines to generate predicted breathing patterns and organ deformities for the patient being treated. For example, the analysis server 110a may transmit patient attributes from the sensor 163 and execute the AI ​​model 111 accordingly.

[0074] It should be understood that any (one or more) alternative and / or additional machine learning models may be used to implement a similar learning engine. The deep learning engine may include a processing pathway that is trained during a training phase. Once trained, the deep learning engine may be executed (e.g., by the analysis server 110a) to generate predicted patient attributes.

[0075] As described herein, the analysis server 110a can store an AI model 111 (e.g., a neural network, a random forest, a support vector machine, a regression model, a recursive model, etc.) in an accessible data repository. The analysis server 110a can retrieve the AI ​​model 111 and train the AI ​​model 111 to predict a deformity associated with one or more of the patient's structures / organs.

[0076] Various machine learning techniques may involve "training" a machine learning model to predict (e.g., estimate) (the likelihood of) a patient attribute, including supervised learning techniques, unsupervised learning techniques, or semi-supervised learning techniques, among others. In one non-limiting example, the predicted patient attribute may indicate a predicted breathing pattern of the patient and / or a deformity of a structure (e.g., a tumor) of the patient. Thus, the AI ​​model 111 may be used to predict the real-time position and orientation of the PTV. As a result, the analysis server 110a may display the projected position of the tumor, and / or modify the patient's treatment accordingly, such as by changing the MLC opening.

[0077] One type of deep learning engine is a deep neural network (DNN). A DNN is a branch of a neural network and consists of a stack of layers, each of which performs a specific operation, such as convolution, pooling, loss calculation, etc. Each intermediate layer receives the output of the previous layer as its input. The starting layer is an input layer that is directly connected to or receives an input data structure including data items in one or more machine-readable objects, and may have a number of neurons equal to the data items in one or more machine-readable objects provided as input. For example, a machine-readable object may be a data structure, such as a list or a vector, that includes multiple data fields that include data received from a sensor 163. Each neuron in the input layer may accept the contents of a data field as input. The analysis server 110a may preprocess the machine-readable object (e.g., through an encoding process) so that the data field may be accepted as input to the AI ​​model 111 described herein.

[0078] The next set of layers may include any type of layer that may be present in a DNN, such as a convolutional layer, a fully connected layer, a pooling layer, or an activation layer. Some layers (such as convolutional neural network layers) may include one or more filters. Filters, usually called kernels, have arbitrary sizes defined by the designer. Each neuron can only respond to a specific area (called a receptive field) of the previous layer. The output of each convolutional layer can be considered an activation map, which highlights the effect of applying a specific filter on the input. The convolutional layer may be followed by an activation layer to apply nonlinearity to the output of each layer. The next layer may be a pooling layer that helps reduce the dimensionality of the output of the convolution. In various implementations, high-level abstractions are extracted by fully connected layers. The weights of the neural connections and kernels can be continuously optimized during the training phase.

[0079] In fact, training data can be generated by the user through observation and experience, so as to facilitate supervised learning. For example, training data can be received and monitored during past radiotherapy treatments provided to previous patients. In another example, the training data can be a data set that includes the patient's breathing pattern while being treated and their corresponding movement (e.g., the patient's chest position and movement and their breathing pattern and their corresponding medical images with timestamps). The training data can be pre-processed via any suitable data enhancement method (e.g., standardization, encoding, any combination thereof, etc.) to produce a new data set with modified characteristics to improve model generalization using ground truth. The methods and systems described herein are not limited to training AI models based on previously treated patients. For example, instead of previously treated patients, the training data set can include data associated with any group of participants (not patients) willing to be monitored for the purpose of generating a training data set. Therefore, untreated participants in the study can be connected to one or more electronic sensors, where the analysis server includes the data collected from the sensors in the training data set.

[0080] Training the AI ​​model 111 can be performed, for example, by analyzing historical patient data (e.g., the patient's movements and their corresponding medical images and breathing patterns). For example, a training data set may include 100 patients and their breathing data collected via a breathing sensor. The training data may also include corresponding medical images associated with the patient. Each medical image may include a timestamp corresponding to the timestamp of the patient's breathing cycle. This raw information can be converted into a machine-readable object using the process described herein and associated with ground truth failure information (if applicable), which can be operated as a label. The input to the model 111 includes a set of machine-readable objects generated by the analysis server (received from the sensor 163). The model output may include a confidence score indicating the likelihood of the location of a particular structure.

[0081] refer to Figure 2 , depicts an example data flow diagram 200 showing how an AI model may be trained and executed to predict patient attributes according to one embodiment. The method 200 may include steps 202-206. However, other embodiments may include additional or alternative steps, or may omit one or more steps entirely. The method 200 is described as being performed by a server (such as Figure 1 However, one or more steps of method 200 may be performed by Figure 1 For example, one or more computing devices may execute locally Figure 2 Some or all of the steps described in .

[0082] Conventional motion modeling methods obtain various images of the patient's internal structure while the patient is breathing, and those images are binned according to the patient's breathing pattern. Conventional methods then analyze the patient's internal structure (e.g., PTV or GTV) based on the binned images to identify the location of the tumor and / or how the tumor moves as the patient moves (e.g., breathing). Conventional methods then assume that the patient continues to breathe in the same way and predict the location of the tumor during treatment accordingly. This method is prone to error. In order to correct the inefficiency and inaccuracy of conventional methods, some medical professionals constrain patients (e.g., abdominal bandages, use of ventilation devices to force patients to breathe in a certain pattern, or audio / visual guidance to patients to breathe regularly) in order to limit the movement of the tumor. However, this is very undesirable because it limits the patient's comfort during treatment. In another example, some other methods require additional kV imaging, which is also undesirable because the patient receives a higher dose.

[0083] Using method 200, an AI model can be trained based on a training data set to predict how a patient breathes and therefore how the patient's breathing data affects the movement of the patient's internal structures. In implementation, using method 200, an AI model can be executed to predict how the patient's internal structures deform based on the patient's projected breathing data. In addition, unlike conventional methods, method 200 allows an AI model that can be executed using minimal patient data (e.g., a patient's proxy signal and an initial medical image).

[0084] Training of AI models

[0085] Before executing the AI ​​model, the analysis server can first train the AI ​​model and ensure its accuracy. The analysis server can train the AI ​​model using a training data set, which includes two sets of data associated with a group of patients (or participants in a clinical trial).

[0086] First, the training data set may include respiratory data associated with a group of patients and / or participants. Specifically, the training data set may include data received from a respiratory sensor associated with each patient. Such data is also referred to as a proxy signal herein. The respiratory sensor may detect the respiratory pattern and / or movement (e.g., chest position and / or movement) of each patient. The sensor may (or sometimes a separate processor or computing device may) transmit a proxy signal to a processor (e.g., an analysis server), which records the proxy signal and further analyzes it. The proxy signal may be used to identify the respiratory cycle associated with each patient and the corresponding patient movement. In operation, in order to generate a training data set, participants and patients may be required to wear a respiratory sensor (or sometimes agree to monitor via an optical or 3D surface mechanism). As a result, the respiratory sensor may generate respiratory data (or proxy signals) associated with each patient (e.g., respiratory cycle, chest position and movement, etc.).

[0087] Second, the training data set may include structural deformation data associated with the group of patients. For example, the training data set may include medical images associated with the group of patients. The medical images may be periodically acquired while the patient is breathing and / or being treated. Each image may be taken from a specific anatomical region of the patient. For example, in operation and in order to prepare the training data set, medical images of patients and participants are periodically acquired. Each image may include a timestamp that may be used to identify corresponding respiratory data associated with the medical image.

[0088] In one embodiment, the training data set may include medical images (e.g., CT or 4DCT) depicting the patient's internal organs and corresponding sensor data identifying the patient's respiratory data (e.g., respiratory cycle). The training data set may not only rely on external data and deformation of the patient. For example, when preparing a training data set for a patient being treated, the analysis server may also include kV projections in the training data set. In some embodiments, the training data set may also include MR and / or time-resolved MR data associated with the patient.

[0089] The training dataset may also include additional data associated with the patient (in addition to the agent signals or medical images). For example, the AI ​​model may take into account demographic information and / or other biomarkers (e.g., age, weight, or BMI) for each patient. As a result, the model may also take into account the patient's attributes when considering and correlating how the patient's internal structures deform / move and / or how the patient breathes. In operation, some patients may lose weight during (and as a result of) treatment. Thus, their deformation and / or breathing cycle may change slightly due to their weight loss. For example, a tumor may shrink under treatment, which may then result in a changed motion pattern, even if the external agent remains unchanged.

[0090] The training data set may also include each patient's medical history so that the AI ​​model can connect any corresponding data to how the patient's internal structure has been or is being deformed. For example, the training data set may include whether the patient suffers from chronic obstructive pulmonary disease (COPD) or sleep apnea. If so, the training data set may also include a value associated with the severity of the patient's COPD or sleep apnea. In another example, the training data set may include an indication of whether the patient is a smoker. In another example, the patient may suffer from a disease (e.g., cancer). The training data set may include an indication of the disease and a corresponding grade of the patient's disease.

[0091] The patient's medical information may indicate how the patient breathes, and therefore how the patient's structure deforms / moves. For example, the model may learn that patients with COPD breathe faster than patients without COPD. As a result, patients with COPD may have different deformation data and may have internal structures that deform at different rates.

[0092] When reviewed as a whole, the training data set may include information that may indicate how each patient's breathing affects their internal organs and structures. Specifically, the training data set may indicate how breathing deforms or moves one or more internal structures of each patient. In operation, while a chest sensor is monitoring / recording respiratory data (e.g., chest movement and respiratory cycle data) for a group of patients, they may be asked to breathe normally. In addition, the medical imaging device is used to periodically capture images depicting how the patient's internal organs are moving and deforming. When reviewed together, each image may be analyzed based on its timestamp and the patient's corresponding respiratory cycle.

[0093] The analysis server can then aggregate the various data sets associated with the group of patients and include the aggregated data sets in the training data set. Using the training data sets, the analysis server can train one or more AI models discussed herein. In various embodiments, the AI ​​models can use one or more deep learning engines to perform automatic segmentation of received images and / or correlate data within the training data sets so that they reveal patterns connecting how patients breathe and how their breathing / movement deforms or moves their organs or internal structures.

[0094] The AI ​​model can first analyze the proxy signal data and determine the breathing cycle associated with each patient in the training data set. Figure 3As depicted in , the AI ​​model can identify that different patients have different breathing properties and cycles. Each breathing cycle depicts two phases (inhalation or inhalation and exhalation or exhalation). For example, each row 300-310 depicts breathing data associated with different patients. As depicted, different patients have different breathing properties (e.g., different patients breathe at different rates and have different breathing volumes). In addition, patients exhibit changes in their breathing cycles over time (e.g., during treatment). For example, some patients may relax as they adapt to the radiation therapy environment and may change their breathing patterns (e.g., breathe slowly). The AI ​​model can also learn that there are some common elements across breathing patterns (e.g., there may be detected periodicity). For example, the AI ​​model can learn common breathing phases and / or how different patients change their breathing patterns over time. As a result, it can learn how to predict a patient's breathing data at a given time.

[0095] Using various machine learning techniques, the model can identify how each patient (given its attributes) breathes. In addition, the AI ​​model can also ingest deformation data (such as medical images) and connect each deformation to its corresponding breathing data / cycle. Figure 4 Correlations between a patient's internal structures and corresponding proxy signals are depicted. Proxy signal graph 400 depicts a proxy signal for a particular patient that corresponds to their breathing data captured using a breathing sensor. The AI ​​model may determine that medical image 402 corresponds to point 404 within proxy signal graph 400, and that medical image 406 corresponds to point 408 within proxy signal graph 400. As depicted via medical images 402 and 406, the particular patient's internal structures move and deform slightly based on the patient's breathing (as indicated by the different points within proxy signal graph 400).

[0096] The AI ​​model can learn how internal structures deform or move. Using methods such as conditional variational autoencoders (conditional VAEs), the AI ​​model can identify how one or more structures will deform based on the patient's breathing data. The model can vectorize the received medical image and prepare the vectorized position for each point within the received medical image, such as Figure 5 The AI ​​model can then associate the vector with the proxy signal. The AI ​​can predict a vector after being properly trained that corresponds to how each point within the medical image will move / deform as the patient breathes. This predicted vector is also referred to herein as a deformation vector.

[0097] The AI ​​model can also analyze the proxy signal and identify the corresponding latent signal corresponding to the proxy signal. The AI ​​model can use the latent signal to reconstruct the medical image associated with the patient's movement (even in the absence of the proxy signal). Taking into account the patient's breathing data, the AI ​​model can train itself based on the detected differences between the patient's time-stamped medical images.

[0098] In unsupervised learning methods, the proxy signal may not always be known to the AI ​​model (as opposed to supervised learning methods where the proxy is known and can be labeled as ground truth). In some embodiments, the AI ​​model can predict the proxy signal itself. For example, the AI ​​model can predict how a patient may breathe during treatment. Because some patients change their breathing patterns (e.g., some patients relax and breathe more normally as treatment progresses), predicted breathing data can be used instead of using the patient's initial breathing cycle.

[0099] The AI ​​model can analyze images (including how internal structures move) and their corresponding respiratory data to train itself, so that the trained AI model can ingest proxy signals and initial medical images of a new patient and predict how the new patient's internal structures will move and deform. The AI ​​model can use various methods to reveal hidden patterns, such as using deep learning methods. The AI ​​model can use unsupervised or semi-supervised methods, in which moving images are automatically analyzed and deformations are highlighted. For example, when the AI ​​model receives a moving medical image (4DCT), the AI ​​model can change the moving medical image into a fixed medical image and identify various deformations and differences between the images. Figure 6 As depicted in , the AI ​​model may use a conditional VAE method 600, where the AI ​​model ingests a moving medical image 602. The goal of the AI ​​model may be defined as transforming the moving medical image 602 into one or more fixed images 604. However, the model (at this point in training) may not ingest data identifying how various internal structures are deformed within the moving medical image 602. That is, the AI ​​model may not receive data (e.g., supervised training) that labels how the structure has moved or deformed. This is primarily because the analysis server may utilize an unsupervised approach, where training is not limited to known techniques and known limitations of deformation vectors of different deformable image registration methods. The AI ​​model may calculate the difference between the images and test the identification of the difference by comparing pixel intensity values ​​(e.g., Hounsfield units (HU) for CT / CBCT) between different medical images. Specifically, each image may be divided into different segments (e.g., pixels or sets of pixels), and the pixel intensity values ​​of the corresponding segments may be compared to determine how the internal structure has moved.

[0100] The AI ​​model can test its own accuracy and recalibrate itself accordingly. For example, the AI ​​model can monitor the patient during treatment and determine whether the patient's actual internal structure positions match the patient's predicted structural positions (within a tolerable threshold). For example, one or more medical imaging devices can periodically provide images of the patient's internal structures, and the captured images (actual) can be compared with the patient's predicted images. In another example, the analysis server can compare Kv imaging captured as a result of the patient's treatment with simulated images / data predicted / simulated by the model and verify the accuracy of the model.

[0101] The AI ​​model can also be trained to generate predicted deformed medical images of the patient. For example, the AI ​​model can be used to generate moving or fixed medical images that represent how the patient's internal structures will move / deform. Figure 7 A medical image predicting how a patient's lungs will deform is depicted. Using the methods and systems discussed herein, the AI ​​model can predict the deformation vector depicted in the medical image 700. The AI ​​model can also consider the proxy signal and can learn from the proxy signal (e.g., learn how to replicate the signal, which is called a latent signal or latent code), as depicted in graph 702. The AI ​​model can replace the proxy signal received from the respiratory sensor with the latent signal to predict how the internal structure will move / deform. As depicted, the latent signal and the proxy signal are very similar to each other.

[0102] The AI ​​model can simulate medical images to convey predicted deformation data. For example, the AI ​​model can then generate image 708, which identifies a deformation of the projection of the patient's lungs. The AI ​​model can also use various segmentation protocols to segment one or more structures within the medical image. For example, image 708 can be segmented to highlight the location of the patient's lungs. The AI ​​model can also use image distortion to identify differences / deformations as depicted by images 704 and 706 (e.g., digitally manipulate the image). In some embodiments, the analysis server can use a distortion protocol to warp the image to a common reference / atlas.

[0103] Fitting and training AI models for individual patients (e.g., some conventional methods) do not utilize data that considers the dynamics of breathing or the psychology of breathing. For example, some patients can dramatically change their breathing patterns once they are more relaxed or more adapted to treatment (e.g., muscle relaxation or respiratory drift). In contrast, AI models trained using the methods and systems discussed herein take into account how patients breathe during their treatment.

[0104] In some configurations, the analysis server may pre-train or partially train the AI ​​model. For example, the analysis server may train the AI ​​model based on a group of cohort patients. The analysis server may then train (fine-tune) the AI ​​model using data specific to a particular patient. For example, when the AI ​​model is pre-trained, the analysis server may fine-tune the AI ​​model by feeding it information about a particular patient (e.g., respiratory data and / or medical images) and customize it to that patient. This allows customization of the AI ​​model without the risk of overfitting.

[0105] Using a cohort of patients to train an AI model allows the AI ​​model to learn various properties that are common among that group of patients, such as the stiffness of a bone or a sliding interface. The AI ​​model can then fine-tune these learnings for a specific patient.

[0106] Execution and implementation of AI models

[0107] During training, the analysis server may iteratively generate new predictions (recommendations) based on the training data set (e.g., for each patient in the data set and its corresponding data). If the prediction does not match the actual outcome, the analysis server continues training unless and until the computer-generated recommendation meets one or more accuracy thresholds and is within an acceptable range. For example, the analysis server may split the training data set into three groups (i.e., training, validation, and testing). The analysis server may train the AI ​​model based on the first group (training). The analysis server may then execute (at least partially) the trained AI model against the second set of data to predict the outcome (validation). The analysis server then verifies whether the prediction is correct. Using the above method, the analysis server may evaluate whether the AI ​​model is properly trained. The analysis server may use this method to continuously train and improve the AI ​​model. The analysis server may then use the remaining data points in the training data set to measure the accuracy of the AI ​​model (e.g., area under the curve, precision, and recall) (testing).

[0108] After the model is trained, the AI ​​model can predict deformation vectors. The deformation vector identifies how each point within the image will move / deform.

[0109] Return to reference Figure 2, at 202, the analysis server may receive respiratory data of the patient from an electronic sensor. The analysis server may communicate with one or more sensors configured to monitor the movement of the patient. The electronic sensor may identify the patient's respiratory rate by counting the number of breaths by counting how many times the patient's chest rises. In one example, the electronic sensor may be a wearable (e.g., a chest strap or patch on the chest) respiratory monitoring system that monitors the patient's breathing pattern. The electronic sensor may detect small changes in the patient's breathing pattern, chest position, tidal volume, and / or other vital signs. In another example, a fiber optic respiratory rate sensor may be used to monitor the patient. In yet another example, various 3D surfacing methods may be used to determine how the patient breathes. In addition, the analysis server may retrieve one or more medical images (e.g., CT or 4DCT) of the patient.

[0110] At 204, an artificial intelligence model is executed using the respiratory data and deformation data of at least one internal structure of the patient is predicted, wherein the artificial intelligence model is trained based on a training data set, the training data set including a group of participants (e.g., previously treated patients or participants in a clinical trial), their corresponding respiratory data, and their corresponding deformation data. As used herein, deformation data refers to any data predicted by the AI ​​model. Non-limiting examples of deformation data may include any data (e.g., deformation vectors, numbers, and / or simulated medical images) that convey how one or more internal structures will move or deform at a given time.

[0111] The analysis server may use the data received in step 202 to execute the AI ​​model discussed herein. In addition, the analysis server may receive an initial medical image of the patient. The AI ​​model may be trained according to the methods and systems discussed herein. As a result of this execution, the AI ​​model may predict deformation data associated with one or more organs or internal structures of the patient. Specifically, the AI ​​model may predict deformation vectors indicating how each point within the medical image of the patient will move / deform. The deformation vectors may indicate the distance and direction each point within the medical image will move. For example, as shown in deformation vector 800( Figure 8 ), vector 802 indicates that its corresponding point within the medical image will move upward (e.g., by 1 mm), and vector 804 indicates that its corresponding position will not move. In contrast, vector 806 indicates that its corresponding position will move downward (e.g., by 0.5 mm). Using the deformation vectors, the analysis server can predict the position and orientation of one or more internal structures of the patient.

[0112] At step 206, the analysis server may output the data predicted by the AI ​​model (deformation data). The analysis server may output the deformation data in a variety of ways. In one embodiment, the analysis server may output a deformation vector. For example, a GUI accessed by a medical professional may display the deformation vector corresponding to the deformation vector. Figure 8 A similar image to that depicted in , where different deformation vectors and their corresponding magnitudes and directions are depicted.

[0113] In another example, the analysis server can use the AI ​​model to generate a moving or fixed medical image depicting how the patient's internal structures will move / deform. For example, a GUI accessed by a medical professional can display a projected 4DCT of the patient that depicts how the patient's internal structures will move / deform.

[0114] In another example, the analysis server can use the data predicted by the AI ​​model to correct one or more properties of the patient's radiotherapy treatment. For example, the analysis server can correct the properties of the multi-leaf collimator (MLC), move the couch, or pause the beam, or any combination of these examples. Specifically, in combination with one or more other software technology solutions, the analysis server can correct the opening of the MLC so that the radiation delivery is directed to the projected location of the PTV (e.g., projected using the AI ​​model). In this way, the analysis server provides a dynamic MLC correction method in which the MLC opening can be corrected in real time or near real time.

[0115] Effectively, the analysis server can enable the gating of the beam to match the motion of the patient's tumor. Because the analysis server can predict / estimate the tumor location, the analysis server can control one or more properties of the radiotherapy machine. For example, the analysis server can control (e.g., review and modify) the MLC opening, timing, and / or dose rate.

[0116] In another example, the analysis server can transmit data predicted by the AI ​​model to a downstream software solution. For example, the results of executing the AI ​​model can be transmitted to a dose calculation software solution. In another example, the analysis server can transmit the deformation data to a downstream tissue tracking application.

[0117] In one non-limiting example, when the patient is positioned on the bed of a radiotherapy machine (before receiving treatment), the patient is asked to wear a breathing sensor. The analysis server retrieves a proxy signal received from the breathing signal. In addition, the analysis server retrieves an initial medical image (CT) of the patient. Using the proxy signal and the medical image, the analysis server executes an AI model that has been trained using the methods and systems discussed herein. As a result, the AI ​​model generates deformation vectors and simulates new medical images that predict how the patient's internal structures will move and deform. The simulated medical image (e.g., a simulated 4DCT) is displayed on a GUI accessed by a medical professional treating the patient. When the patient's treatment begins, the analysis server also corrects the MLC opening of the radiotherapy machine in real time based on the results predicted by the AI ​​model. The analysis server also communicates with the tissue tracking and dose calculation software technology solution and transmits the predicted results to the software technology solution.

[0118] The various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functions. Whether such functions are implemented as hardware or software depends on the specific application and the design constraints imposed on the entire system. The technician may implement the described functions in different ways for each specific application, but such implementation decisions should not be interpreted as resulting in a departure from the scope of the present disclosure or claims.

[0119] Embodiments implemented with computer software can be implemented with software, firmware, middleware, microcode, hardware description language or any combination thereof. Code segments or machine executable instructions can represent any combination of processes, functions, subroutines, programs, routines, subroutines, modules, software packages, classes, or instructions, data structures or program statements. Code segments can be coupled to another code segment or hardware circuit by transmitting and / or receiving information, data, independent variables, parameters or memory contents. Information, independent variables, parameters, data, etc. can be transmitted, forwarded or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc.

[0120] The actual software code or dedicated control hardware used to implement these systems and methods does not limit the claimed features or the present disclosure. Therefore, the operation and behavior of the systems and methods are described without reference to specific software code, and it should be understood that software and control hardware can be designed to implement the systems and methods based on the description herein.

[0121] When implemented in software, the function can be stored as one or more instructions or codes on a non-transient computer-readable or processor-readable storage medium. The steps of the method or algorithm disclosed herein can be embodied in a processor-executable software module, which can reside on a computer-readable or processor-readable storage medium. Non-transient computer-readable media or processor-readable media include both computer storage media and tangible storage media that facilitate the transfer of computer programs from one place to another. Non-transient processor-readable storage media can be any available medium that can be accessed by a computer. As an example and not limitation, such non-transient processor-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other tangible storage medium that can be used to store desired program codes in the form of instructions or data structures and can be accessed by a computer or processor. As used herein, disks and discs include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks and blue-ray discs, wherein the discs usually reproduce data magnetically, and the discs reproduce data optically with lasers. The above combination should also be included in the scope of computer-readable media. Additionally, the operations of a method or algorithm may reside as one or any combination or set of codes and / or instructions on a non-transitory processor-readable medium and / or a computer-readable medium, which may be incorporated into a computer program product.

[0122] The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the embodiments described herein and variations thereof. Various modifications to these embodiments will be apparent to those skilled in the art, and the principles defined herein may be applied to other embodiments without departing from the spirit or scope of the subject matter disclosed herein. Therefore, the present disclosure is not intended to be limited to the embodiments shown herein, but is to be given the widest scope consistent with the appended claims and the principles and novel features disclosed herein.

[0123] While various aspects and embodiments have been disclosed, other aspects and embodiments are foreseen. The various aspects and embodiments disclosed are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.

Claims

1. A method comprising: receiving, by a processor, respiration data of the patient from the electronic sensor; executing, by the processor, an artificial intelligence model using the respiratory data and predicting deformation data of at least one internal structure of the patient, wherein the artificial intelligence model is trained based on a training data set, the training data set comprising a group of participants, their corresponding breathing data, and their corresponding deformation data; as well as The predicted deformation data is output by the processor.

2. The method according to claim 1, further comprising: A medical image of the patient is received by the processor, wherein the processor executes the artificial intelligence model using the medical image.

3. The method of claim 1 or claim 2, wherein the respiratory data received from the electronic sensor is at least one of chest position, chest movement, or respiratory cycle data of the patient.

4. A method according to any preceding claim, wherein the deformation data corresponds to movement of at least one internal structure of the patient.

5. The method according to any preceding claim, further comprising: At least one property of a radiation therapy machine is adjusted by the processor based on the predicted deformation data. 6 . The method of claim 5 , wherein the at least one property corresponds to at least one of a multi-leaf collimator opening, pausing the beam, or moving the couch.

7. A method according to any preceding claim, wherein the output predicted deformation data corresponds to a simulated medical image depicting an anatomical region of the patient.

8. The method according to any preceding claim, wherein outputting the predicted deformation data corresponds to transmitting the predicted deformation data to a dose calculation software solution or a tissue tracking software solution.

9. A method according to any preceding claim, wherein the artificial intelligence model generates predicted breathing data associated with the patient, the predicted breathing data comprising at least one of a property of a breathing cycle or chest movement.

10. The method of any preceding claim, wherein the electronic sensor is a wearable breath sensor or an optical breath sensor.

11. A computer system: A server comprising a processor and a non-transitory computer-readable medium containing instructions that, when executed by the processor, cause the processor to perform operations comprising: receiving respiratory data of the patient from an electronic sensor; executing an artificial intelligence model using the respiration data and predicting deformation data of at least one internal structure of the patient, wherein the artificial intelligence model is trained based on a training data set including a group of participants, their corresponding respiration data, and their corresponding deformation data; and The predicted deformation data is output.

12. The computer system of claim 11, wherein the instructions further cause the processor to receive a medical image of the patient, wherein the processor executes the artificial intelligence model using the medical image.

13. A computer system according to claim 11 or claim 12, wherein the respiratory data received from the electronic sensor is at least one of chest position, chest movement or respiratory cycle data of the patient.

14. A computer system according to any one of claims 11 to 13, wherein the deformation data corresponds to movement of at least one internal structure of the patient.

15. The computer system of any one of claims 11 to 14, wherein the instructions further cause the processor to adjust at least one property of a radiation therapy machine based on the predicted deformation data.

16. The computer system of claim 15, wherein the at least one property corresponds to at least one of a multi-leaf collimator opening, pausing the beam, or moving the couch.

17. A computer system according to any one of claims 11 to claim 16, wherein the output predicted deformation data corresponds to a simulated medical image depicting an anatomical region of the patient.

18. The computer system of any one of claims 11 to 17, wherein outputting the predicted deformation data corresponds to transmitting the predicted deformation data to a dose calculation software solution or a tissue tracking software solution.

19. A computer system according to any one of claims 11 to claim 18, wherein the artificial intelligence model generates predicted respiratory data associated with the patient, the predicted respiratory data comprising at least one of a property of a breathing cycle or chest movement.

20. The computer system of any one of claims 11 to 19, wherein the electronic sensor is a wearable breath sensor or an optical breath sensor.

Citation Information

Cited By

  • Intelligent magnetic resonance respiratory training system, training method and electronic equipment

    CN120323954A