Method and system for automated x-ray imaging system orientation using image features

Abstract

A method for predicting an optimal orientation of an imaging device. The method comprises the following steps: obtaining an initial image of an anatomy body; extracting at least one image feature of the anatomy from the initial image; and predicting an optimal orientation of one or more adjustable components of the imaging device for subsequent imaging of the anatomy based on the extracted at least one image feature. The prediction of the optimal orientation may include at least one of: (i) maximizing alignment of an imaging system with the anatomy; (ii) minimizing areas around the anatomy to be excluded from X-ray imaging; (iii) maximizing the alignment of the imaging device to collimate out portions of the anatomy to be excluded from subsequent imaging; and (iv) maximizing the amount of the anatomy to be included in the subsequent imaging.

Description

Technical Field

[0001] This disclosure generally relates to methods and systems for predicting the optimal orientation of one or more adjustable components of an imaging system. Background Technology

[0002] Physicians or technicians typically need to make several adjustments to the X-ray imaging system in order to capture the desired field of view (FOV) in the image. These adjustments can include both out-of-plane rotation and in-plane rotation of the X-ray imaging system. As just one example, when obtaining a radial artery access, the patient's arm is rotated at approximately... The angle of the arm extends away from the body, and imaging the radial artery up to the shoulder requires the imaging system to also be rotated to accommodate the arm's rotation. This rotation can be a specific combination of L-arm rotation, detector rotation, etc. This combination of rotations balances various user needs, including aligning the imaging system with the anatomy to be imaged while maintaining the ability to collimate areas that don't need imaging (e.g., the air around the arm), or maximizing the amount of anatomy to be imaged included in the field of view (FOV) (so that, for example, only one contrast agent injection is needed to navigate through the arm and shoulder). These various evaluations are not always intuitive, can be difficult and time-consuming to perform, and can lead to underutilization of system capabilities (e.g., underutilization of detector rotation or collimation), suboptimal image acquisition (e.g., a smaller FOV than expected for a second acquisition that requires a different FOV), additional contrast agent usage (e.g., two DSAs with a smaller field of view instead of one DSA with a larger FOV), etc. These evaluations can also be further complicated in imaging systems where the source can also rotate independently (e.g., Philips® Azurion® FlexArm, CX-free imaging systems, etc.). Summary of the Invention

[0003] Therefore, there is a persistent unmet need for methods and systems for efficiently and accurately automating the orientation of imaging equipment (e.g., X-ray imaging equipment) to optimally align the equipment with the anatomy to be imaged. The imaging equipment is also referred to herein as an “imaging system” (e.g., “X-ray imaging system” or “X-ray system”).

[0004] Various embodiments and implementations relate to a method and system for predicting the optimal orientation of one or more adjustable components of an imaging device using an orientation system. The methods and systems described herein or otherwise contemplated generally relate to in-plane rotation of the imaging / orientation system.

[0005] According to one aspect, a system is provided configured to acquire an initial image of an anatomical body and extract at least one image feature of the anatomical body from the initial image. The system is configured to predict an optimal orientation of one or more adjustable components of an imaging device for subsequent imaging of the anatomical body based on the extracted at least one image feature. In some embodiments, the one or more adjustable components of the system can then be automatically or manually oriented to the predicted optimal orientation.

[0006] According to one aspect, a method for predicting the optimal orientation of an imaging device is provided. The method includes: obtaining an initial image of an anatomy; extracting at least one image feature of the anatomy from the initial image; and predicting, based on the extracted at least one image feature, the optimal orientation of one or more adjustable components of the imaging device for subsequent imaging of the anatomy.

[0007] According to an embodiment, the method includes applying a trained imaging orientation model configured to analyze at least one extracted image feature to predict the optimal orientation of one or more adjustable parts.

[0008] According to an embodiment, the method further includes obtaining information including identification of the anatomy to be imaged by the imaging device, and analyzing the information to predict the optimal orientation of one or more components.

[0009] According to an embodiment, the method further includes receiving approval from a user via a user interface for orienting one or more adjustable components to a predicted optimal orientation; and orienting one or more adjustable components to a predicted optimal orientation by an imaging device.

[0010] According to an embodiment, the received approval includes a fine-tuning of the predicted optimal orientation from the user, and the orientation includes orienting one or more adjustable components to the fine-tuned predicted optimal orientation.

[0011] According to an embodiment, the prediction of the optimal orientation of one or more adjustable components includes at least one of the following operations: (i) maximizing the alignment of the imaging device with the anatomy; (ii) minimizing the area around the anatomy to be excluded from subsequent imaging; (iii) maximizing the alignment of the imaging device to collimate the portion of the anatomy to be excluded from subsequent imaging; and (iv) maximizing the amount of anatomy to be included in subsequent imaging.

[0012] According to an embodiment, the method further includes: providing a user with a predicted optimal orientation of one or more adjustable components as an instruction via a user interface, the instruction being used to orient one or more adjustable components to the optimal orientation or request approval for the imaging device to orient one or more adjustable components to the predicted optimal orientation; or for the imaging device to automatically orient one or more adjustable components to the predicted optimal orientation.

[0013] According to an embodiment, providing a user with a predicted optimal orientation of one or more adjustable components via a user interface includes providing a predicted image of the anatomy if the one or more adjustable components are adjusted to the predicted optimal orientation.

[0014] According to an embodiment, providing a predicted optimal orientation of one or more adjustable components via a user interface includes providing a visual display of the optimal orientation of one or more adjustable components.

[0015] According to another aspect, a system for predicting the optimal orientation of an imaging device is provided. The system includes: an imaging device comprising one or more adjustable components and configured to acquire an initial image and perform subsequent imaging; and a processor configured to predict, based on the initial image, the optimal orientation of the one or more adjustable components for subsequent imaging of an anatomy.

[0016] According to an embodiment, the system further includes a processor configured to apply a trained imaging orientation model, the trained imaging orientation model being configured to analyze initial images of the anatomy to predict the optimal orientation of one or more adjustable parts.

[0017] According to an embodiment, the processor is also configured to automatically orient one or more adjustable components to a predicted optimal orientation.

[0018] According to an embodiment, the prediction of optimal orientation includes at least one of the following operations: (i) maximizing the alignment of the imaging device with the anatomy; (ii) minimizing the area around the anatomy to be excluded from subsequent imaging; (iii) maximizing the alignment of the imaging device to collimate the portions of the anatomy to be excluded from subsequent imaging; and (iv) maximizing the amount of anatomy to be included in subsequent imaging.

[0019] According to an embodiment, the system further includes a user interface configured to provide a user with a predicted optimal orientation of one or more adjustable components.

[0020] According to an embodiment, providing a predicted optimal orientation of one or more adjustable components includes at least one of the following operations: (i) providing a predicted image of the anatomy if one or more adjustable components are adjusted to the predicted optimal orientation; and (ii) providing a visual display of the optimal orientation of one or more adjustable components.

[0021] According to another aspect, a non-transient computer-readable storage medium is provided. The non-transient computer-readable storage medium stores a computer program including instructions that, when executed by a processor, cause the processor to: obtain an initial image of an anatomy; extract at least one image feature of the anatomy from the initial image; and predict, based on the extracted at least one image feature, an optimal orientation of one or more adjustable components of the imaging device for subsequent imaging of the anatomy. In some embodiments, one or more adjustable components of the system can then be automatically or manually oriented to the predicted optimal orientation.

[0022] It should be understood that all combinations of the foregoing concepts and the additional concepts discussed in more detail below (assuming that such concepts do not contradict each other) are considered part of the inventive subject matter disclosed herein. In particular, all combinations of the claimed subject matter appearing at the end of this disclosure are contemplated as part of the inventive subject matter disclosed herein. It should also be understood that terms expressly adopted herein that may also appear in any disclosure incorporated by reference should conform to the meaning most consistent with the specific concepts disclosed herein.

[0023] These and other aspects of the various embodiments will become apparent from the embodiments described below and will be set forth with reference to the embodiments described below. Attached Figure Description

[0024] In the accompanying drawings, similar reference numerals generally refer to the same parts throughout the different views. The drawings, which illustrate features and methods of implementing various embodiments, should not be construed as limiting oneself to other possible embodiments falling within the scope of the appended claims. Moreover, the drawings are not necessarily drawn to scale, but generally focus on illustrating the principles of the various embodiments.

[0025] Figure 1 This is a flowchart of a method for predicting the optimal orientation of one or more adjustable components of an X-ray system, according to an embodiment.

[0026] Figure 2 This is a schematic representation of an orientation X-ray system according to an embodiment.

[0027] Figure 3A These are fluorescence fluoroscopic images acquired by an interventional X-ray imaging system according to an embodiment.

[0028] Figure 3B It is an RGB image captured by an optical camera according to an embodiment.

[0029] Figure 4 This is a flowchart of a method for training an X-ray orientation model according to an embodiment.

[0030] Figure 5A An X-ray source is provided together with the embodiment. and X-ray detector A schematic representation of the optimal orientation of an X-ray orientation system.

[0031] Figure 5B An L-shaped arm is provided according to an embodiment. Provide detectors after optimal orientation A schematic representation of the optimal orientation of an X-ray orientation system.

[0032] Figure 6 This is a schematic representation of the slider of the X-ray orientation system according to an embodiment. Detailed Implementation

[0033] This disclosure describes various embodiments of systems and methods configured to predict the optimal orientation of one or more adjustable components of an imaging system (e.g., an X-ray system). More generally, the applicant has recognized and understood that it would be beneficial to provide a method and system for efficiently optimizing X-ray image acquisition (or acquisition of other imaging modalities). An X-ray orientation system acquires an initial image of an anatomy of a subject and extracts at least one image feature of the anatomy of the subject from the initial image. The X-ray orientation system analyzes (one or more) the extracted image features to predict the optimal orientation of one or more adjustable components of the X-ray system, wherein the optimal orientation optimizes one or more aspects of X-ray imaging for subsequent imaging of the anatomy of the subject by the X-ray system. The one or more adjustable components of the system can then be automatically or manually oriented to the predicted optimal orientation.

[0034] The embodiments and implementations disclosed herein or otherwise contemplated can be used in conjunction with any system that can utilize or benefit from the optimized orientation of the imaging components. For example, one application of the embodiments and implementations disclosed herein or otherwise contemplated is X-ray imaging, such as during routine imaging, catheter insertion, and many other patient procedures. One application is improving the functionality of Philips® Azurion® or Zenition® imaging systems and software (manufactured by Koninklijke Philips, NV) and other products. However, this disclosure is not limited to these devices or systems, and therefore the disclosure and embodiments herein can cover any system that can utilize or benefit from the optimized orientation of the imaging components.

[0035] refer to Figure 1 In one embodiment, a flowchart of a method 100 for predicting the optimal orientation of one or more adjustable components of an X-ray system using an X-ray orientation system is provided. The method described in conjunction with the accompanying drawings is provided by way of example only and should not be construed as limiting the scope of this disclosure. The X-ray orientation system can be any system described herein or otherwise contemplated. The X-ray orientation system can be a single system or multiple different systems.

[0036] At step 110 of the method, an X-ray orientation system 200 is provided. (See reference...) Figure 2 An embodiment of the depicted X-ray orientation system 200, for example, includes one or more of the following: a processor 220, a memory 230, a user interface 240, a communication interface 250, and a storage device 260 interconnected via one or more system buses 212. It should be understood that... Figure 2 In some respects, it constitutes an abstraction, and the actual organization of the components of system 200 may differ from and be more complex than those illustrated. Furthermore, X-ray orientation system 200 may be any system described herein or otherwise conceived. Other elements and components of X-ray orientation system 200 are disclosed and / or conceived elsewhere herein.

[0037] According to embodiments, the X-ray orientation system 200 includes an X-ray system or device 270 or communicates directly or indirectly with the X-ray system or device 270. The X-ray system or device 270 can be any system or device configured to obtain X-ray images of a patient (such as a human or animal, and other targets). The X-ray system or device 270 can be a fixed system or a mobile system. According to embodiments, the X-ray system or device 270 includes one or more adjustable components, wherein one or more adjustable components can be adjusted to achieve the target of X-ray imaging as described herein or otherwise contemplated. For example, a fixed single-plane / dual-plane C-arm X-ray imaging system may include one or more adjustable components, such as adjustment (rotation, etc.) of the L-arm of the device and / or adjustment (rotation, etc.) of the detector of the system. The imaging system may alternatively be a mobile C-arm X-ray imaging system with adjustable components or a C-arm-less imaging system. The X-ray system may include other adjustable components.

[0038] According to embodiments, the X-ray orientation system 200 includes an electronic medical record system and / or an electronic medical record (EMR) database 280, or communicates directly or indirectly with the electronic medical record system and / or the electronic medical record (EMR) database 280, and can obtain or receive information about a patient from the electronic medical record system and / or the electronic medical record database, including demographic, diagnostic, and / or treatment information. For example, the EMR database may include information about imaging or treatment procedures for a patient, including anatomy that will be imaged during the procedure. According to embodiments, the electronic medical record system 280 may be a local or remote database and communicates directly and / or indirectly with the system 200. Therefore, according to embodiments, the system includes an electronic medical record database or the system 280.

[0039] At an optional step 120 of the method, the X-ray orientation system 200 receives or obtains information that may include identification of the anatomy of the object to be imaged by the X-ray system, which may be utilized by downstream steps of the system to appropriately orient one or more adjustable components of the system, as described herein or otherwise contemplated. Information may be received from an electronic medical record database or system 280. Alternatively, information may be received via a user interface or other input from a physician or technician. As another alternative, information may be extracted from images of the patient taken. In some cases, the received information may include not only identification of the anatomy of the object to be imaged by the X-ray system, but also demographic information, treatment information, diagnosis, and / or other information. Once received, the information may be readily available and / or temporarily or permanently stored in local and / or remote memory for future use.

[0040] At step 130 of the method, the X-ray orientation system 200 receives or acquires one or more images of the patient. According to an embodiment, the one or more images are X-ray images received from the X-ray device or system 270, which are taken of the patient at the beginning or during an imaging session or procedure. However, according to other embodiments, the one or more images may be acquired via another imaging modality, provided the images are sufficient for use in downstream steps of the method as described herein or otherwise contemplated.

[0041] For example, in one embodiment, reference Figure 3A and 3B One or more images of the patient can be obtained through imaging modalities for optimization of subsequent imaging via an X-ray orientation system. Figure 3A In this process, the initial images are fluorescence fluoroscopic images acquired by interventional X-ray imaging systems (such as Philips Azurion or Zenition, and many other possible X-ray systems). Figure 3BIn this process, the initial image is an RGB image captured by an optical camera such as the Philips ClarifEye® (manufactured by Koninklijke Philips, NV), which, among many other possible optical cameras, can be mounted on the interventional imaging system. Other imaging modalities are possible.

[0042] Once obtained, one or more initial images of the patient can be used immediately, and / or they can be temporarily or permanently stored in local and / or remote memory for future use, including as training data.

[0043] At step 140 of the method, the X-ray orientation system 200 extracts one or more image features about the anatomy of the object from one or more received initial images. The X-ray orientation system will utilize the extracted one or more image features to predict the optimal orientation of one or more adjustable parts, and therefore the one or more extracted image features can be any features suitable for use by the X-ray orientation system for optimal orientation prediction. According to embodiments, the features can be information about bony structures in fluorescence fluoroscopy images, external anatomy from RGB images, or any other available image features. Any feature extraction method can be used to extract one or more relevant image features from one or more initial input images, including but not limited to feature extractors (e.g., Scale Invariant Feature Transform (SIFT) or Accelerated Robust Feature Transform (SURF)), edge extractors (e.g., Canny edge detectors or Hough transforms), etc. Alternatively, a deep learning-based feature extractor can be used to extract one or more relevant image features from one or more initial input images. Other methods of feature extraction are possible.

[0044] Once extracted, one or more of the extracted image features can be used immediately, and / or they can be temporarily or permanently stored in local and / or remote memory for future use, including as training data.

[0045] At step 150 of the method, the X-ray orientation system analyzes at least one extracted image feature to determine or predict the optimal orientation of one or more adjustable components of the X-ray system. The X-ray orientation system can analyze image features in various ways to predict or determine the optimal orientation of one or more adjustable components of the X-ray system. For example, the processor of the X-ray orientation system can be programmed to receive and analyze the extracted image features to identify or determine the angle or position of an anatomical structure within an initial image, and then use that identified or determined angle or position to reorient the adjustable components of the X-ray system such that the structure within the initial image is captured in subsequent images with the optimal orientation of the X-ray system. Therefore, subsequent images obtained by the X-ray system will include different views of the anatomical structure within the image.

[0046] According to embodiments, this optimal orientation of one or more adjustable components of the X-ray system depends on one or more factors. For example, optimal orientation may be an orientation that optimizes one or more aspects of X-ray imaging for subsequent imaging of the anatomy of the subject by the X-ray system. These aspects of X-ray imaging may be, for example, one or more of the following: (i) maximizing the alignment of the X-ray imaging system with the anatomy of the subject; (ii) minimizing the area around the anatomy of the subject that is preferably excluded from the X-ray imaging, such that the area is minimized and the anatomy of the subject is maximized; (iii) maximizing the alignment of the X-ray imaging system to collimate the anatomy to be excluded from the X-ray imaging (such as maximizing collimation shutter alignment and other alignments); and (iv) maximizing the amount of the anatomy of the subject to be included in the X-ray imaging; and other aspects. It is worth noting that achieving one aspect may simultaneously achieve another. These aspects do not need to be satisfied in every imaging process or situation. For example, in some cases (e.g., during the disposal delivery phase of the process), it may only be necessary to include a small region of interest in the field of view (FOC). An X-ray orientation system can be programmed or designed to know what aspects or balances are needed for the patient during stages of the procedure, during imaging of a particular anatomical body, and / or at other points or times, and thus orient one or more adjustable components of the X-ray system based on those aspects or balances. Therefore, the optimal orientation of the system at one point in time during the procedure may differ from the optimal orientation at another point in time for the same procedure. Similarly, the optimal orientation of the system for one patient may differ from the optimal orientation of the system for another patient. Many other variations are possible.

[0047] According to an embodiment, the X-ray orientation system analyzes at least one extracted image feature to predict the optimal orientation of one or more adjustable components of the X-ray system, and also includes patient information received from an EMR database or system 280 or other sources (such as via a user interface or other input from a physician or technician). This information may include, for example, demographic information, treatment information, diagnosis, and / or other information. Therefore, the received information (such as information about the procedure to be performed, the patient's weight or age, and other information) can be informative when the system determines the optimal orientation for the patient. Procedure information may inform the system about the structure(s)(s) to maximize the image, the optimal orientation of the X-ray machine relative to the anatomy, and other useful information. Patient information (such as the patient's weight or age) may be related to, for example, the patient's dimensions, and thus may inform the system about the optimal orientation of the X-ray machine relative to the anatomy, and other useful information.

[0048] According to an embodiment, the X-ray orientation system uses a trained X-ray orientation model to analyze at least one extracted image feature (as input to the model) to predict the optimal orientation of one or more adjustable parts of the X-ray system (as the model's output). The trained X-ray orientation model can be any model that can be trained to generate an output from the input, as described herein or otherwise contemplated. For example, the trained X-ray orientation model can be a neural network or other trained machine learning model. Therefore, according to an embodiment, the X-ray orientation system includes a trained X-ray orientation model that receives input data and outputs the optimal orientation of various one or more orientable portions of the X-ray imaging system, such that an appropriate field of view (FOV) of the region of interest (ROI) is imaged.

[0049] X-ray orientation models can be trained in various ways. According to one embodiment, an X-ray orientation model is trained in an unsupervised manner by designing a loss function that captures some or all of the user's needs. That is, the model is trained to optimize or achieve one or more target aspects of an X-ray system, procedure, and / or imaging session.

[0050] According to one embodiment, for example, similar to the radial artery approach, one or more relevant image features (e.g., bony structures in (one or more) fluorescence fluoroscopic images, the arm in RGB images) can be extracted from one or more initial input images, and a loss function can be formulated. For example, a loss function can be formulated where the following equation is minimized: (Equation 1). Here, The y-axis represents the angle between a vector representing one or more principal components of the extracted features and a vector parallel to the y-axis. Principal components are vectors pointing in directions that maximize the variance of the extracted feature set and can be computed using known methods such as principal component analysis. Orienting various orientable parts of an X-ray imaging system to minimize... Aligning relevant image features with the y-axis, and also aligning them with the collimator, maximizes the ability to collimate out air or other unwanted or unnecessary areas around the target ROI.

[0051] As another example, the loss function can be formulated where the following equation is minimized: (Equation 2). Here, The angle between the vector representing the principal components of (one or more) extracted features and the vector representing the diagonal axis through the image space. Orienting various orientable parts of the X-ray imaging system to minimize... Aligning relevant image features with the diagonal axis maximizes the amount of relevant image features included in the field of view (FOV), thereby reducing the need for additional acquisitions, and so on. Such alignment is optimal for elongated features such as long, thin bony structures and long, thin vascular structures.

[0052] According to another embodiment, the X-ray orientation model is trained in a supervised manner. In this embodiment, the optimal orientation of one or more adjustable components of the X-ray system can be configured to minimize the difference between a ground truth or gold standard image or angle and an image or angle identified in an initial image via extracted image features(s). The ground truth or gold standard can be available in the form of what the final generated image should look like or in the form of the ideal orientation of each of the one or more adjustable components of the imaging system. Therefore, optimization can minimize the difference between the predicted image and the ground truth or gold standard image, and / or minimize the difference between the angle found in the predicted orientation and the angle found in the ground truth or gold standard orientation. The angles found in the predicted and ground truth / gold standard orientations can be, for example, the angles of principal components of features extracted from one or more of the predicted and ground truth / gold standard images, respectively. Alternatively, the angles can represent the angles of various orientable components of the X-ray system that generate the predicted standard image and the ground truth / gold standard image, respectively.

[0053] This is not an exhaustive list of loss functions or other formulas that can be used to analyze the extracted image features and generate information used to determine or predict the optimal orientation of one or more adjustable components of an X-ray system.

[0054] According to embodiments, depending on the scenario, the orientation of various directional or adjustable components of the X-ray system can be optimized together. For example, in the absence of a C-system, the orientation of both the X-ray source and the X-ray detector can be optimized simultaneously. (Reference) Figure 5A In one embodiment, the trained X-ray orientation model can also provide an X-ray source. and X-ray detector The optimal orientation. Similarly, X-ray orientation models can provide the optimal orientation to be applied to the acquired images. The post-processing transformation. The trained X-ray orientation model can be, for example, a trained neural network (NN).

[0055] According to another embodiment, the orientation of various directional or adjustable components of the X-ray system can be optimized individually or separately. For example, in the case of a C-arm system, this can be achieved first by minimizing... (See Equation 1 above) to optimize the orientation of the L-arm, and once the optimal orientation of the L-arm is found and fixed, it can be minimized if necessary. (See Equation 2 above) to optimize the detector's orientation. Reference Figure 5B In one embodiment, a trained X-ray orientation model (neural network NN1) can determine the L-shaped arm as described herein or otherwise conceived. The optimal orientation is determined, and then the trained X-ray orientation model (neural network NN2) can determine the detector orientation as described in this paper or otherwise envisioned. The optimal orientation. It is worth noting that neural networks NN1 and NN2 can be the same neural network or a single trained X-ray orientation model configured to perform orientation prediction or determine both. According to another embodiment, neural networks NN1 and NN2 can be different neural networks or different trained X-ray orientation models, each individually trained to perform a specific associated orientation prediction.

[0056] The trained X-ray orientation model can be of any type, including convolutional neural networks (CNNs), transformer networks, or other neural networks. For example, a neural network can be a CNN trained to regress the orientations of various orientable parts of a system by: estimating the orientations of these orientable parts, generating corresponding transformed images that will result from the new orientations of the system, and calculating an appropriate loss function to evaluate whether an appropriate system orientation has been achieved. During training, the value of the loss function is typically minimized, and this value can be calculated using functions such as mean absolute error (MAE), mean squared error (MSE), etc. Various methods are known for solving the loss minimization problem, such as stochastic gradient descent (SGD), adaptive moment estimation (Adam), etc., which calculate the derivative of the loss function with respect to the model parameters. These derivatives inform how the algorithm must tune the model parameters to minimize the loss function. These steps of estimating orientation, generating corresponding transformed images, calculating the loss function, and updating the model parameters are repeated several times during training until the model's parameters produce an output with a small loss function value.

[0057] refer to Figure 4 In one embodiment, a flowchart is provided for a method 400 for training an X-ray orientation model for an X-ray orientation system 200. This method may be performed by the X-ray orientation system itself, or by another system such as a dedicated machine learning model training system.

[0058] At step 410 of the method, the training system receives training data to be used to train the model. The training data can be any data sufficient to train the model to generate the described output using the described input data. For example, the training data may include X-ray images for multiple patients and each item in the process, including images before and after optimization (including optimization using a benchmark ground truth). In some orientations, the training data may include X-ray images identifying different structures (e.g., bony or vascular structures) in the anatomy of multiple patients, and corresponding benchmark ground truth optimal orientations for one or more adjustable imaging components to (i) maximize the alignment of the imaging device with the anatomy; (ii) minimize the area around the anatomy to be excluded from the imaging; (iii) maximize the alignment of the imaging device to collimate portions of the anatomy to be excluded from the imaging; and / or (iv) maximize the amount of anatomy to be included in the imaging. For example, in a supervised learning context, structures and structures (e.g., bony or vascular structures) and such corresponding optimal orientations for imaging the anatomy using the structures can be annotated in the training data used to train the model. The training data, which can be used in a supervised or unsupervised manner, may include X-ray images of hundreds or thousands of patients and / or procedures, and may be updated using new images. The training data may also include other information. This training data may be obtained and created by an expert such as a clinician, or may be obtained and created under the supervision of a clinician, or may be obtained and used without creation. Training data can be received from any source. For example, it may be received from an electronic medical record database or system 280, or from any other component of the X-ray orientation system or training system. According to an embodiment, the X-ray orientation system 200 includes or communicates directly or indirectly with an imaging database, which includes some or all of the training dataset.

[0059] According to embodiments, the training system may include a data preprocessor or similar component or algorithm configured to process received training data. For example, the data preprocessor analyzes the training data to remove noise, bias, errors, and other potential problems. The data preprocessor may also analyze the input data to remove low-quality data. Many other forms of data preprocessing or data point identification and / or extraction are possible.

[0060] At step 420 of the method, the training system uses training data to train an X-ray orientation model to determine the optimal orientation of one or more adjustable components of the X-ray system based on input data. For example, in some embodiments, the trained model can determine a specific bony or vascular structure in an anatomy based on an input image of the anatomy, and predict the optimal orientation of one or more adjustable components for imaging the anatomy based on that specific structure. The X-ray orientation model can be trained using any method used to train such a model. The trained X-ray orientation model is a unique model based on the training data used to train the model. After training, the system includes the trained X-ray orientation model.

[0061] Therefore, after training, the X-ray orientation model is a specialized model configured to receive specialized input (i.e., one or more image features extracted from the patient's initial image) and generate a highly specific output: the predicted optimal orientation of one or more adjustable components of the X-ray system, where optimal optimization or achievement of one or more target aspects of the X-ray system, procedure, and / or imaging session. Alternatively, in the case where the X-ray orientation model is a neural network, the model can be configured to receive specialized input of the patient's initial image, which can be preprocessed, and to use layers of the neural network to extract relevant image features.

[0062] At step 430 of the method, the trained X-ray orientation model is stored for future use. According to an embodiment, the trained X-ray orientation model can be stored in a local or remote storage device.

[0063] Return to Figure 1 In method 100, at step 160, the X-ray orientation system utilizes a predicted optimal orientation of one or more adjustable components of the X-ray system generated in step 150. The X-ray orientation system can utilize the predicted optimal orientation in a variety of ways.

[0064] According to an embodiment, at step 160 of the method, the X-ray orientation system itself can automatically orient one or more adjustable components of the system to a predicted optimal orientation. Therefore, the X-ray orientation system can be programmed, designed, or otherwise configured such that it automatically responds to an optimal orientation prediction by orienting one or more adjustable components of the system to the predicted optimal orientation. Thus, the one or more adjustable components of the system can include mechanics that allow for automatic adjustment of the components. Motors, servo systems, or other components can provide this automation functionality.

[0065] According to another embodiment, the system can use a new image acquired from its updated orientation to fine-tune one or more adjustable components of the system. For example, when optimizing detector orientation, the system can compare a vector representing the principal components of one or more extracted features from the new image with a vector representing the diagonal through the image space, and can then perform several optimization steps to further minimize the angle between the two vectors. This can be an automated process and is useful if the first predicted optimal orientation may be inaccurate (due to noise or other artifacts in the image) or if the patient moves or is moved after the first image is acquired and the predicted optimal orientation is outdated.

[0066] According to another embodiment, at step 160 of the method, the X-ray orientation system provides a user with a predicted optimal orientation of one or more adjustable components of the X-ray system via the system's user interface. The user interface can be any display or interface that enables the predicted optimal orientation to be displayed in text, visual, or other formats.

[0067] The predicted optimal orientation of one or more adjustable components of an X-ray system can be provided in a variety of different ways via a user interface. According to an embodiment, the predicted optimal orientation of one or more adjustable components of the X-ray system is provided to the user as an instruction to orient the one or more adjustable components to the optimal orientation.

[0068] According to another embodiment, the predicted optimal orientation of one or more adjustable components of the X-ray system is provided to the user as a request for approval for the X-ray system to orient the one or more adjustable components to the predicted optimal orientation. Therefore, at step 170 of the method, the system can receive approval from the user via a user interface for orienting the one or more adjustable components to the predicted optimal orientation. And at step 180 of the method, after receiving the approval, the X-ray system can orient the one or more adjustable components to the predicted optimal orientation.

[0069] According to another embodiment, a preview of the predicted optimal orientation of one or more adjustable components of the X-ray system, along with a preview of the corresponding image generated based on the optimal orientation suggested by the imaging system, is provided to the user. Transformations (e.g., rotations) that must be applied to one or more adjustable components of the X-ray system to move them to the predicted optimal orientation can be applied to the input image to generate the preview. This can be performed as a post-processing step. Alternatively, the transformation can be applied by an X-ray orientation model. Therefore, an X-ray orientation model can also be trained to generate a predicted X-ray image predicted to be produced by the optimal orientation suggested by the imaging system. This can include, for example, optimization of one or more aspects, as described herein or otherwise contemplated.

[0070] According to an embodiment, the X-ray system allows a user to interact with a preview image, enabling them to provide further input on how the system orientation should be adjusted so that they can acquire images in an orientation optimal for them. Their system can optionally preserve user input and also fine-tune a trained neural network to user preferences. Therefore, according to an embodiment, received approval to orient one or more adjustable components to a predicted optimal orientation includes fine-tuning of the predicted optimal orientation from the user, and orientation includes orienting one or more adjustable components to the fine-tuned predicted optimal orientation.

[0071] In this embodiment, the user can additionally provide their preferences as input, which can be used to weight the loss, for example, during fine-tuning of the inference time. For example, refer to... Figure 6 This is an example slider that allows users to indicate a preference that controls the final orientation of the system and thus the acquired images. If a user wants the vascular system aligned with the navigation direction rather than maximizing the amount of relevant image features in the field of view (FOV), they can provide this preference as input. Therefore, a slider can be used to provide input, where if the slider goes all the way to the right, a weight of 0 is applied during fine-tuning or optimization. And weight 1 is applied If the slider remains to the left, weight 1 is applied during fine-tuning or optimization. And weight 0 is applied The weights can change linearly as the slider moves between the two extremes.

[0072] exist Figure 1 At step 190 of method 100 described herein, the X-ray system may acquire one or more subsequent X-ray images after the imaging system has been optimally positioned. This results in improved X-ray imaging because one or more components are optimally oriented to optimize one or more aspects of the X-ray imaging. These aspects of the X-ray imaging may be, for example, one or more of the following: (i) maximizing the alignment of the X-ray imaging system with the anatomy of the subject; (ii) minimizing the area around the anatomy of the subject that is preferably excluded from the X-ray imaging, such that the area is minimized and the anatomy of the subject is maximized; (iii) maximizing the alignment of the X-ray imaging system to collimate the anatomy to be excluded from the X-ray imaging (such as maximizing collimation shutter alignment and other alignments); and (iv) maximizing the amount of the anatomy of the subject to be included in the X-ray imaging; and other aspects.

[0073] According to another embodiment, images acquired after one or more adjustable components of the X-ray system have been oriented to optimal alignment can be adjusted in post-processing to further optimize imaging. In this embodiment, additional rotation parameters can be calculated to be applied to the acquired images in post-processing. For example, if the user wants to maximize the amount of relevant image features in the field of view (FOV) and also orient the image so that the vascular system is aligned with the navigation direction, the detector of the X-ray system can be oriented to maximize the amount of relevant image features in the FOV, and rotation can be applied to (one or more) new images in post-processing after image acquisition so that the vascular system in the image is aligned with the navigation direction. Since image rotation may result in areas of no information in the rotated image, deep learning or other techniques can be used to fill in this missing information via extrapolation or repair (e.g., using GANs, diffusion models, etc.).

[0074] According to another embodiment, 3D images may be available. For example, if 3DRA is acquired during treatment planning, optimizing the orientation of the C-arm to best visualize the treatment target can be combined with optimizing the L-arm to best collimate the anatomy around the treatment target that does not need imaging, and / or with optimizing the detector to maximize the amount of relevant image features in the FOV, or orienting the image so that the vascular system is aligned with the direction of navigation, etc.

[0075] refer to Figure 2 This is a schematic representation of an X-ray orientation system 200. System 200 can be any system described herein or otherwise contemplated, and can include any components described herein or otherwise contemplated. It should be understood that... Figure 2 In some respects, it constitutes an abstraction, and the actual organization of the components of system 200 may differ from and be more complex than that shown in the figure.

[0076] According to an embodiment, system 200 includes a processor 220 capable of executing instructions stored in memory 230 or storage device 260 or otherwise processing data to, for example, perform one or more steps of the method. Processor 220 may be formed from one or more modules. Processor 220 may take any suitable form, including but not limited to a central processing unit (CPU), graphics processing unit (GPU), tensor processing unit (TPU), neural processing unit (NPU), microprocessor, microcontroller, multiple microcontrollers, circuitry, field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), single processor, or multiple processors.

[0077] Memory 230 can take any suitable form, including non-volatile memory and / or RAM. Memory 230 may include various types of memory, such as L1, L2, or L3 caches or system memory. Thus, memory 230 may include static random access memory (SRAM), dynamic RAM (DRAM), flash memory, read-only memory (ROM), or other similar memory devices. Memory may store an operating system, etc. RAM is used by the processor for temporary data storage. According to embodiments, the operating system may contain code that, when executed by the processor, controls the operation of one or more components of system 200. It will be apparent that in embodiments where the processor implements one or more of the functions described herein in hardware, software described in other embodiments corresponding to such functions may be omitted.

[0078] User interface 240 may include one or more devices for enabling communication with a user. The user interface may be any device or system that allows the transmission and / or reception of information, and may include a display, mouse, and / or keyboard for receiving user commands. In some embodiments, user interface 240 may include a command-line interface or graphical user interface that can be presented to a remote terminal via communication interface 250. The user interface may be located alongside one or more other components of the system, or it may be located remotely from the system and communicate via wired and / or wireless communication networks.

[0079] Communication interface 250 may include one or more devices for enabling communication with other hardware devices. For example, communication interface 250 may include a network interface card (NIC) configured to communicate according to the Ethernet protocol. Furthermore, communication interface 250 may implement a TCP / IP stack for communication according to the TCP / IP protocol. Various alternatives or additional hardware or configurations for communication interface 250 will be apparent.

[0080] Storage device 260 may include one or more machine-readable storage media, such as read-only memory (ROM), random access memory (RAM), disk storage media, optical storage media, hard disk drive (HDD), solid-state drive (SSD), flash memory device, or similar storage media. In various embodiments, storage device 260 may store instructions for execution by processor 220 or data that processor 220 can operate on. For example, storage device 260 may store operating system 261 for various operations of system 200.

[0081] It will be apparent that the various information described as being stored in storage device 260 may additionally or alternatively be stored in memory 230. In this respect, memory 230 may also be considered as constituting a storage device, and storage device 260 may be considered as memory. Various other arrangements will be apparent. Furthermore, both memory 230 and storage device 260 may be considered as non-transient machine-readable media. As used herein, the term non-transient will be understood to exclude transient signals but include all forms of storage devices, including volatile and non-volatile memories.

[0082] While system 200 is shown as including one of each described component, various components may be replicated in various embodiments. For example, processor 220 may include multiple microprocessors configured to independently execute the methods described herein, or configured to execute steps or subroutines of the methods described herein, such that multiple processors cooperate to achieve the functions described herein. Furthermore, where one or more components of system 200 are implemented in a cloud computing system, various hardware components may belong to separate physical systems. For example, processor 220 may include a first processor in a first server and a second processor in a second server. Many other variations and configurations are possible.

[0083] According to embodiments, system 200 includes an X-ray system or device 270 or communicates directly or indirectly with an X-ray system or device 270. The X-ray system or device 270 can be any system or device configured to obtain X-ray images of a patient (such as a human or animal, and other targets). The X-ray system or device 270 can be a fixed system or a mobile system. According to embodiments, the X-ray system or device 270 includes one or more adjustable components, wherein the one or more adjustable components can be adjusted to achieve the desired X-ray imaging objective as described herein or otherwise contemplated.

[0084] According to embodiments, the X-ray orientation system 200 includes an electronic medical record system and / or an electronic medical record (EMR) database 280, or communicates directly or indirectly with the electronic medical record system and / or the electronic medical record (EMR) database 280, and can obtain or receive information about a patient from the electronic medical record system and / or the electronic medical record (EMR) database 280, including demographic, diagnostic, and / or treatment information. For example, the EMR database may include information about imaging or treatment procedures for a patient, including anatomy that will be imaged during the procedure. According to embodiments, the electronic medical record system 280 may be a local or remote database and communicates directly and / or indirectly with the system 200. Therefore, according to embodiments, the system includes an electronic medical record database or system 280. As described herein, the electronic medical record also includes information that can be described as an electronic health record (EHR), patient health information (PHI), etc.

[0085] According to an embodiment, the storage device 260 of system 200 may store one or more algorithms, modules, and / or instructions to perform one or more functions or steps of the methods described herein or otherwise contemplated. For example, storage device 260 may include a trained X-ray orientation model 262, training instructions 263, application instructions 264, and / or reporting instructions 265, as well as other instructions or data.

[0086] According to an embodiment, a trained X-ray orientation model 262 of the X-ray orientation system 200 is trained to analyze extracted image features (as input to the model) to predict the optimal orientation of one or more adjustable parts of the X-ray system (as output of the model). The trained X-ray orientation model can be any model that can be trained to generate an output from the input, as described herein or otherwise contemplated. For example, the trained X-ray orientation model can be a neural network or other trained machine learning model. Thus, according to an embodiment, the X-ray orientation system includes a trained X-ray orientation model that receives input data and outputs the optimal orientation of various one or more orientable parts of the X-ray imaging system, such that an appropriate field of view (FOV) of the region of interest (ROI) is imaged. The trained X-ray orientation model is unique and based on the training data used to train the model. Once generated, the trained X-ray orientation model 262 can be used immediately or stored in local and / or remote memory for future use.

[0087] According to an embodiment, training instruction 263 instructs the system to train an X-ray orientation model 262 of the X-ray orientation system 200. The instruction instructs the system to retrieve, acquire, or receive training data. The training data can be any data sufficient to train the model to generate the described output using the described input data. For example, the training data may include X-ray imaging for each of multiple patients and procedures, including images before and after optimization (including optimization using a baseline truth value). Training instruction 263 also instructs the system to train the X-ray orientation model using the acquired training data. Various different training methods can be used to train the X-ray orientation model. Training instruction 263 also instructs the system to store the trained X-ray orientation model for future use.

[0088] Similarly, application instruction 264 guides the system to apply the trained X-ray orientation model 262 of the X-ray orientation system 200 to new data acquired at the application or inference time. The application or inference time refers to the application of the trained model. The instruction guides the system to retrieve, acquire, or receive data acquired at the application or inference time and predict the optimal orientation of one or more adjustable components of the X-ray system, as described herein or otherwise envisioned.

[0089] According to an embodiment, the X-ray orientation system 200 is configured to process thousands or millions of data points in the input data used to train the X-ray orientation model 262, such as via training instructions 263. For example, generating a functional and proficient trained X-ray orientation model from a corpus of training data requires processing millions of data points from the input data and the generated features. This may require millions or billions of computations to generate a novel trained X-ray orientation model based on these millions of data points and millions or billions of computations. Therefore, each trained X-ray orientation model is novel and different based on the input data and the model's parameters, thus improving the system's functionality. Generating functional and proficient trained X-ray orientation models involves a process with a large amount of computation and analysis that the human brain cannot perform in a lifetime or multiple life cycles.

[0090] According to an embodiment, Reporting Instruction 265 instructs the system to provide system output to a user (such as a clinician) via a user interface. The output provided can be any information as described herein or otherwise contemplated. The system can provide information to the user via any mechanism, including but not limited to visual displays, auditory notifications, pages, or any other notification method. This information can be transmitted to another device via wired and / or wireless communication. For example, the system can transmit information to a monitor, screen, mobile phone, computer, laptop computer, wearable device, and / or any other device configured to allow the display and / or other transmission of information.

[0091] All definitions used herein should be understood to govern the general meaning of terms beyond dictionary definitions, definitions in referenced literature, and / or the general meaning of the defined terms.

[0092] Unless explicitly indicated to the contrary, the words “a” and “an” as used in this specification and claims shall be understood to mean “at least one”.

[0093] As used in this specification and claims, the phrase “and / or” should be understood to mean “any one or both” of the elements so combined, that is, elements that exist together in some cases and separately in others. Multiple elements listed using “and / or” should be interpreted in the same way, that is, “one or more” of the elements so combined. In addition to the elements specifically identified by the “and / or” clause, other elements may optionally be present, whether related to or unrelated to those specifically identified.

[0094] As used in this specification and claims, “or” should be understood to have the same meaning as “and / or” as defined above. For example, when separating items in a list, “or” or “and / or” should be interpreted as inclusive, i.e., including at least one, but also including more than one of a plurality of elements or a list of elements, as well as optional additional items not listed. Terms that clearly indicate the opposite, such as “only one of…” or “exact one of…”, or, when used in the claims, “consisting of…”, will refer to including a plurality of elements or an exact one of a list of elements. Generally, the term “or” as used herein should only be interpreted as indicating an exclusive alternative (i.e., “one or another but not both”) when preceded by an exclusive term (e.g., “any one of…”, “one of…”, “only one of…”, or “exact one of…”).

[0095] As used in this specification and claims, the phrase "at least one" referring to a list of one or more elements should be understood to mean at least one element selected from any one or more elements in the list of elements, but not necessarily including at least one element from each and every element specifically listed in the list of elements, and does not exclude any combination of elements in the list of elements. This definition also allows for the optional presence of elements other than those specifically identified in the list of elements referred to by the phrase "at least one," whether related to or unrelated to those specifically identified elements.

[0096] It should also be understood that, unless expressly indicated to the contrary, in any method claimed herein that includes more than one step or action, the order of the steps or actions of the method is not necessarily limited to the order in which the steps or actions of the method are described.

[0097] In the claims and the preceding description, all transitional phrases, such as “comprising,” “including,” “carrying,” “having,” “containing,” “involving,” “holding,” “including,” etc., should be understood as open-ended, that is, meaning including but not limited to. Only the transitional phrases “consisting of…” and “consisting substantially of…” should be closed or semi-closed transitional phrases, respectively.

[0098] Although several embodiments of the invention have been described and illustrated herein, those skilled in the art will readily conceive of various other modules and / or structures for performing functions and / or obtaining results and / or one or more of the advantages described herein, and each such variation and / or modification is considered to be within the scope of the embodiments of the invention described herein. More generally, those skilled in the art will readily understand that all parameters, dimensions, materials, and configurations described herein are intended to be exemplary, and actual parameters, dimensions, materials, and / or configurations will depend on one or more specific applications using the teachings of the invention. Those skilled in the art will recognize or be able to determine many equivalents of the specific embodiments of the invention described herein using experimental means no more than conventional. Therefore, it should be understood that the foregoing embodiments are presented by way of example only, and that embodiments of the invention may be practiced in ways different from those specifically described and claimed within the scope of the claims and their equivalents. The embodiments of the invention disclosed herein relate to each individual feature, system, article, material, kit, and / or method described herein. In addition, any combination of two or more such features, systems, articles, materials, kits, and / or methods is included within the scope of the invention disclosed herein, provided that such features, systems, articles, materials, kits, and / or methods are not contradictory.

Claims

1. A method (100) for predicting the orientation of an imaging device, the method comprising: Obtain initial images of the (130) anatomical body; Extract at least one image feature of the anatomy from the initial image (140); as well as (150) Predict the optimal orientation of one or more adjustable components of the imaging device for subsequent imaging of the anatomy based on at least one extracted image feature.

2. The method according to claim 1, wherein, Predicting the optimal orientation of the one or more adjustable components includes applying a trained imaging orientation model configured to analyze at least one extracted image feature.

3. The method according to claim 1, further comprising: Obtaining (120) includes information on the identification of the anatomical body to be imaged by the imaging device, and Predicting the optimal orientation of the one or more adjustable components includes analyzing the obtained information to predict the optimal orientation of the one or more components.

4. The method according to claim 1, further comprising: (170) Obtain approval from the user via the user interface for orienting the one or more adjustable components to the predicted optimal orientation; as well as The imaging device orients (180) the one or more adjustable components to the predicted optimal orientation.

5. The method according to claim 4, wherein: The approval includes fine-tuning of the predicted optimal orientation from the user, and Orienting the one or more adjustable components includes adjusting the one or more adjustable components to a finely tuned, predicted optimal orientation.

6. The method according to claim 1, wherein, The prediction of the optimal orientation of the one or more adjustable components includes at least one of the following operations: (i) maximizing the alignment of the imaging device with the anatomy; (ii) minimizing the area around the anatomy to be excluded from X-ray imaging; (iii) Maximize the alignment of the imaging device to collimate the portions of the anatomy to be excluded from the subsequent imaging; and (iv) Maximize the amount of the anatomy to be included in the subsequent imaging.

7. The method according to claim 1, further comprising: The user is provided with a predicted optimal orientation of one or more adjustable components of the (160) X-ray system via a user interface as an instruction to orient the one or more adjustable components to the optimal orientation or to request approval for the imaging device to orient the one or more adjustable components to the predicted optimal orientation, or The imaging device automatically orients (160) the one or more adjustable components to the predicted optimal orientation.

8. The method according to claim 7, wherein, Providing the predicted optimal orientation via a user interface includes providing a predicted image of the anatomy if the one or more adjustable components are adjusted to the predicted optimal orientation.

9. The method according to claim 7, wherein, Providing the predicted optimal orientation via a user interface includes providing a visual display of the optimal orientation for one or more adjustable components.

10. A system (200) for predicting the optimal orientation of an imaging system, the system comprising: A processor that communicates with the memory, the processor being configured to: Obtain initial images of the anatomical body; Extract at least one image feature of the anatomy from the initial image; as well as The optimal orientation of one or more adjustable components of the imaging device is predicted based on at least one extracted image feature for subsequent imaging of the anatomy.

11. The system according to claim 10, wherein, The processor is configured to apply a trained imaging orientation model to predict the optimal orientation of the one or more adjustable components based on at least one extracted image feature.

12. The system according to claim 10, wherein, The processor is also configured to automatically orient the one or more adjustable components to a predicted optimal orientation.

13. The system according to claim 10, wherein, In order to predict the optimal orientation, the processor is configured to perform at least one of the following operations: (i) maximizing the alignment of the imaging device with the anatomy; (ii) minimizing the area around the anatomy to be excluded from the subsequent imaging; (iii) maximizing the alignment of the X-ray system to collimate the portions of the anatomy to be excluded from the subsequent imaging; and (iv) maximizing the amount of the anatomy to be included in the subsequent imaging.

14. The system of claim 10, further comprising a user interface configured to provide a user with a predicted optimal orientation of the one or more adjustable components of the X-ray system.

15. The system according to claim 14, wherein, The processor is configured to provide the optimal orientation as at least one of the following: (i) a predicted image of the anatomy with the one or more adjustable components of the imaging system adjusted to the predicted optimal orientation; and (ii) a visual display of the optimal orientation of the one or more adjustable components.

16. The system of claim 10, further comprising: The imaging device includes the one or more adjustable components and is configured to acquire the initial image of the anatomy and perform the subsequent imaging of the anatomy.

17. A non-transitory computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a processor, cause the processor to: Obtain initial images of the anatomical body; Extract at least one image feature of the anatomy from the initial image; and The optimal orientation of one or more adjustable components of the imaging device is predicted based on at least one extracted image feature for subsequent imaging of the anatomy.

Images