Simulated 4D CT localization views using 3D localization views and RGB-D cameras to improve scan planning

Abstract

Systems (FS) and related methods are used to facilitate planning of raw data acquisition by a tomography imaging device (IA). The system (FS) comprises an input interface (IN) via which a 3D localization view image (V0) of an input movable target (OI) occupying a first part of a 3D space can be received, the localization view image (V0) being previously obtained by the tomographic imaging device on the basis of raw data, the raw data is acquired at a lower quality than the quality expected for the planned raw data acquisition, and the localization view image represents a first motion state of the target motion. A predictor component (PC) predicts a predicted 3D localization view image (Vj, j > 0) based on the input localization view image (V0), said predicted 3D localization view image representing: targets in other predicted motion states, and targets in other portions of the footprint of said motion states. An output interface (OUT) provides the predicted localization view image (Vj) for the raw data acquisition facilitating the planning.

Description

TECHNICAL FIELD

[0001] The present invention relates to a system for facilitating planning of raw data acquisition by a tomographic imaging apparatus, a related method, an imaging arrangement, a computer program element, and a computer readable medium. BACKGROUND

[0002] CT (Computed Tomography) imaging typically starts with the acquisition of scout view images (also referred to as scout views or simply "scout views"). Such scout view images are acquired prior to a diagnostic scan. Such scout views can be used to help a user adjust a field of view ("FOV") of an imaging apparatus (e.g., a CT scanner). Such a field of view can be understood as a subset in the image domain (also referred to as a "scan frame" for a given target anatomy).

[0003] Compared to conventional 2D scout views (coronal and sagittal views), nowadays 3D scout view images can also be acquired which similarly have a low X-ray dose (compared to the high dose required for a diagnostic scan). Compared to 2D projection images, 3D scout view images provide more detailed anatomical information. This information can be used for manual / automatic planning of a scan (e.g., selection of a planning frame and a safety margin).

[0004] Referring back to 3D scout views, these are static images and do not provide temporal tissue motion, e.g., caused by respiratory motion or cardiac motion. Thus, planning of a scan frame is only valid for a specific breathing state during acquisition of projection data. For example, when a user plans a chest or cardiac anatomy, the user can not be aware of the true anatomy in some intermediate motion state in the breathing cycle. To compensate for this, a default safety margin is typically added to a tight scan frame, assuming that the organ of interest is also kept covered by such a larger scan frame during the changes caused by breathing. However, a wrongly estimated margin can either increase unnecessary X-ray dose (too large scan frame), or result in unsatisfactory image quality with an incomplete anatomical field of view (too small scan frame).

[0005] On the other hand, for workflow improvements in patient positioning, several vendors of tomographic modalities such as CT or MRI (Magnetic Resonance Imager / Imaging) provide automatic positioning based on RGB-D camera technology. The cameras are typically mounted on the ceiling directly above the patient couch to monitor the patient position and its changes during scan preparation. In addition to the ceiling mounted cameras for positioning, in-rack or in-bore cameras can also be used to monitor the patient during raw data acquisition. SUMMARY

[0006] Thus, there can be a need for improving planning in tomographic raw data acquisition.

[0007] The object of the invention is achieved by the subject matter of the independent claims, wherein further embodiments are included in the dependent claims. It should be noted that the aspects of the invention described below are equally applicable to related methods, computer program units, and computer-readable media.

[0008] According to a first aspect of the invention, a system is provided to facilitate the planning of raw data acquisition performed by a tomographic imaging apparatus, the system comprising: An input interface is provided for receiving a 3D positioning view image (V0) of a movable target occupying a first part of 3D (three-dimensional) space. The positioning view image is obtained by a tomographic imaging device or based on raw data acquired at a lower quality than expected for planned raw data acquisition. The positioning view image represents a first motion state of the target. The predictor component is capable of predicting one or more predicted 3D positioning view images (V) based at least on the input positioning view images. j (j>0), the one or more predicted 3D positioning view images represent: a target in one or more other predicted motion states, and a target occupying one or more other portions of the space in the one or more predicted motion states, and An output interface capable of providing at least one or more predicted positioning view images for use in the raw data acquisition that facilitates the planning.

[0009] The motion of the target of interest can be interpreted in a generalized way. Therefore, the motion of the target of interest can include any one or more of the following: translation, rotation, and deformation (e.g., compression, contraction, expansion, etc.). Such motion can include motion states. In each or some of these states, the target can move to different spatial locations (relative to a reference position), thus at least a portion of the target occupies different parts of 3D space.

[0010] In a medical context, a target of interest can be at least a part of the whole of a target anatomical feature, such as at least a part of the heart of a mammal (human or animal), or at least a part of at least one of the two lungs, etc.

[0011] In one embodiment, the system includes a planner module capable of calculating planning data based on one or more predicted 3D positioning view images.

[0012] In one embodiment, the planning data includes specifications of a target portion of the 3D space, the target portion comprising one or more other portions of the space, and wherein the target portion of the space is a subset of the input positioning view image.

[0013] The target portion of the space (in some imaging setups referred to as the “scan frame”) includes / covers all voxel locations over which projection data is to be acquired.

[0014] The target portion is preferably "closely" defined, that is, the smallest part of the space that includes (all) one or more other parts of the space. Therefore, the target portion ensures that the moving target remains within the boundaries of the target portion during (all) movement.

[0015] In one embodiment, the system includes a graphics display generator capable of generating a graphic display for display on a display device, the generated graphic display being able to visualize one or more motion states of the target based on one or more predicted 3D positioning view images.

[0016] The display device can be integrated into the operator's console of the imaging device.

[0017] In one embodiment, the graphic display includes a video animation capable of representing the different portions of space occupied by the target as it transitions between the different motion states.

[0018] In one embodiment, the system includes a user interface that allows a user to specify the target portion in space based on the graphical display.

[0019] In an embodiment, the predictor component is capable of calculating one or more predicted 3D positioning view images based on additional input data, which includes one or more of the following: i) video footage of surrogate features associated with the target acquired by a camera based on non-ionizing radiation, ii) a motion model of the target's motion. The camera may be optical, IR (infrared), NIR (near-IR), LIDAR (light detection and ranging), laser, etc. The camera may be a depth-sensing or ranging camera capable of recording the depth distribution of surrogate features over time.

[0020] Alternative features can be anatomical structures surrounding the anatomical feature, such as the skin portion of the chest. Generally, the target can be the patient's internal parts (anatomical features of interest), while the alternative features are the patient's external parts.

[0021] In an embodiment, the motion model is a trained machine learning model trained on training data, such as a convolutional artificial neural network.

[0022] ML models can be generative models, allowing the prediction of “virtual” localized views even without such camera channels. However, non-generative ML, such as supervised learning-type regressors, can also be used. Therefore, two main implementations are envisioned in this paper: a 1-channel input type (which uses the localized view as input) and a 2-channel input type (which uses the localized view as input and the associated motion of a camera shot with alternative features). ML models can be used with either the 1-channel or 2-channel implementation.

[0023] For example, training data may include instances (samples) of historical or synthetically generated images, including: i) historical or synthetically generated 3D localization view images of examples of targets and surface lenses based on non-ionizing radiation for corresponding example surrogate features of the example targets.

[0024] Alternatively or additionally, training data may include sequences of historical or synthetically generated localization views from different patients representing targets of interest in different motion states. Sequences can be split into subsequences and paired with another localization view sample to obtain training pairs, where one such localization view serves as the training input and the corresponding subsequence serves as the target. Such training pairs can be used to train regression models, particularly regression models configured for ordinal data. Alternatively or additionally, generative models can be trained based on such data.

[0025] In an embodiment, the motion model represents at least one motion component of the motion that can be extracted from the video shot, and wherein the predictor component applies the extracted motion component to the positioning view image to extrapolate the predicted 3D positioning view image.

[0026] Rigid or flexible registration can be used to spatially register an input positioning view with a camera lens. This allows for the association of two motions: motion based on the target (target anatomical features) in the positioning view, and motion based on alternative features of the camera lens.

[0027] In one embodiment, the system may include a transducer capable of issuing an alarm signal in the event of a deviation between the motion state so imaged by the video lens of the non-ionizing radiation-based camera and the model, wherein the deviation violates a predefined acceptableness condition.

[0028] In another aspect, an imaging arrangement is provided, the imaging arrangement comprising: a system according to any one or more of the preceding claims, and any one or more of the following: a tomographic imager, the camera, and the display device.

[0029] In another aspect, a method is provided to facilitate the computational implementation of planning for raw data acquisition performed by a tomographic imaging device.

[0030] In another aspect, a method is provided for training a machine learning model based on training data for computational implementation of planning for raw data acquisition by a tomographic imaging device, wherein the model, after being trained, is able to predict one or more such predicted positioning view images for a target in one or more different motion states, given a positioning view image of a movable target in a first motion state.

[0031] In another aspect, a method is provided for obtaining training data for computational implementation of training a machine learning model based on the training data to be obtained, the machine learning model being used for planning raw data acquisition by a tomographic imaging device, the model being trained to predict one or more such predicted positioning view images for a target in one or more different motion states, given a positioning view image of a movable target in a first motion state.

[0032] Training data can be obtained by searching medical databases such as PACS for patients with records including alternative features of the positioning view and at least one shot over time. Alternatively, such positioning view vs. alternative feature video camera shot pairs can be generated in engineering projects at one or more hospitals, where the camera is mounted in a hole / rack, or such cameras are repurposed to obtain such positioning view vs. camera shot pairs.

[0033] In another aspect, a computer program unit is provided, which, when executed by at least one data processing system, is adapted to cause the data processing system to perform any of the methods.

[0034] In another aspect, at least one computer-readable medium is provided on which the program unit is stored, or on which the model is stored.

[0035] In some embodiments, this document proposes combining information from a static 3D positioning view image with temporal motion information provided by a camera channel (e.g., the depth channel of a depth-sensing camera), which may or may not include an optical RGB channel. Such a camera may already be adapted for locating or monitoring a patient during scanning. It is proposed to use the lens from such a camera for a new purpose (i.e., scan frame definition).

[0036] By using explicit or implicit motion models (e.g., respiratory motion models), we propose combining distance / range information from potentially installed cameras with information from 3D positioning view images to generate an “animated” 4D positioning view. This “animated” 4D positioning view is a prediction of how the initial 3D positioning view image will change during the respiratory cycle. The initial static positioning view image is thus replaced by this animation, providing the user with additional valuable information for more accurate planning of diagnostic scans.

[0037] In some aspects, a system is provided for facilitating the planning of projection data acquisition by a tomographic imaging apparatus, the system comprising: An input interface is provided, through which a 3D positioning view image of a target occupying a first portion of 3D space can be received. This positioning view image is based on projection data previously acquired at a lower dose than that acquired for planned projection data, and the positioning view image represents a first motion state of the target capable of withstanding motion. The predictor component is capable of predicting planning data based at least on an input positioning view image, the planning data including a specified target portion in 3D space, the imager acquiring the projection data at a higher dose for the specified target portion.

[0038] "User" refers to a person who operates the imaging device or supervises the imaging process (e.g., a medical professional or other person). In other words, generally speaking, a user is not a patient. However, in some autonomous imaging settings, a patient can perform some functions that are typically done by medical professionals or technicians.

[0039] The terms “target of interest,” “region of interest,” or “target anatomical feature” will be used interchangeably herein. This refers to an actual part of the human body; or, in non-medical applications, to the object or purpose of imaging, particularly for diagnostic scans / high-dose projection data acquisition. Features imaged by a non-ionizing camera are not diagnostically significant herein. Such features (e.g., portions of anatomical structures, such as the head, chest, trunk, legs, etc.) are merely substitutes for the target anatomical feature (the primary interest and purpose of diagnostic scans). Such features may be referred to herein as substitute features or said substitute features, for example, the surrounding portion of the anatomical feature (e.g., skin) or any other (external) anatomical structure that can be imaged by a non-ionizing camera. However, the motion state of the substitute feature can be associated with the motion state of the target anatomical feature.

[0040] A "motion state" refers to the transformation of a target (e.g., a TAF or SF) from one position to another in 3D space. Motion can include any one, two, or all three of translation, rotation, and deformation. When a target (e.g., a TAF or SF) moves, it will occupy at least temporarily a portion of 3D space that differs from the portion it previously occupied. This 3D space is defined by 3D coordinates (voxels). Thus, different voxels occupy different portions of the image domain / inspection area in different motion states. The moment when a portion of the space is thus occupied is the "motion state." Therefore, each motion state has a temporal component (the time when the target occupies the corresponding portion of the space) and a spatial component (the space thus occupied / filled (a subset in 3D)). The occupied space can be identified using the corresponding voxels, i.e., using the set of spatial coordinates of points in the occupied portion of the space.

[0041] A “scan frame” is a 3D subset of the image domain. It includes all voxels from which projection data / raw tomographic data will be collected. The term “frame” as used in “scan frame” can actually refer to the cuboid shape of such a subset; this is not required, and “scan frame” should be interpreted as any subset, regardless of its shape. Preferably, the shape can be a function of the imaging geometry or the type of coordinate system used in the image domain. The scan frame, or the voxel locations it includes, can be defined in any form (defined as a subset of coordinates, a geometric trajectory defined as an equation, or defined in any form of parameterization), whichever is convenient and suitable for the imaging task at hand.

[0042] "Raw data," or tomographic raw data, is a measurement result obtained based on an interrogation signal that interacts with patient tissue. Such a signal can include X-rays in CT, RF signals in MRI, or gamma rays in PET. Such measurements are made at the detector device (X-ray or gamma-ray sensitive detector or TRF coil) of a tomographic imaging modality such as CT, MRI, or MRI. Such a detector device defines a data space (not another space, the image domain) in 3D, composed of voxels (3D locations), called the projection or raw data domain. These voxels are filled with tomographic image values ​​that constitute a reconstructed cross-sectional image or volume in 3D by a tomographic algorithm (e.g., FBP or other algorithms). Projection data is one such example of raw data acquired in a tomographic setup, and this is the primary assumption herein. In the following text, reference is primarily made to such projection data; however, the principles described herein regarding data planning / scan frame, etc., are equally applicable to other such tomographic imaging setups.

[0043] A “localization view” is a general 3D and reconstructed volume reconstructed from raw data such as projections using a tomographic reconstruction algorithm. The image quality of such a localization view is lower than that of a (target) volume reconstructed from raw data with higher IQ. While such a localization view has a lower IQ compared to a diagnostic volume, some coarse details of anatomical structures (at-risk organs, abnormalities, and prominent landmarks) can still be discerned in, for example, low-dose images. In a predicted virtual localization view, when run as an animation, the user may be able to discern coarse anatomical motion details sufficient for planning the scan box.

[0044] Generally, the term "machine learning" encompasses a computerized arrangement (or module) that implements machine learning ("ML") algorithms. Some such ML algorithms operate on machine learning models configured to perform ("learn") tasks. Other MLs operate directly on training data, in which case the training data can form at least a portion of the model. This adjustment or updating of the model is called "training." Generally, the task performance of an ML module can be measurably improved using training experience. Training experience can include appropriate training data and exposing the model to such data. Task performance can improve the effectiveness of the data representation of the task to be learned. Training experience helps improve performance if the training data well represents the distribution of examples used to measure the performance of the final system. See, for example, TM Mitchell's "Machine Learning" (page 2, section 1.1, page 6, 1.2.1, McGraw-Hill, 1997). Performance can be measured by objective testing based on the output produced by the model in response to test data fed to it. Performance can be defined by a specific error rate to be achieved for a given test data. Attached Figure Description

[0045] Exemplary embodiments of the invention will now be described with reference to the following accompanying drawings, which are not drawn to scale unless otherwise stated: Figure 1 An imaging arrangement including a tomographic imaging device is shown; Figure 2 This is a diagram illustrating the concept of a scanning frame for planning data, which can be used in this article. Figure 3 A schematic block diagram of a facilitator system for predicting planning data, including, for example, scan boxes, is shown. Figure 4 It uses explicit motion modeling. Figure 3 Illustration of an embodiment of the facilitator system; Figure 5 Is Figure 3 A block diagram of a training system for training a machine learning model for implicit motion modeling, as described in an embodiment of the facilitator system. Figure 6 A flowchart illustrating a method for the computer / computational implementation of defining planned data for raw data acquisition by a tomographic imaging apparatus is shown; and Figure 7 A flowchart is shown illustrating a method for training machine learning models in implicit motion modeling to be used for planning computer / computational implementations of data definitions. Detailed Implementation

[0046] First refer to Figure 1 , Figure 1 A schematic block diagram of a medical imaging setup (MAR) as envisioned herein in the embodiments is shown. The setup of the MAR may include a medical imaging device (IA) (referred to simply as an "imager"), preferably based on a type of tomographic X-ray; therefore, the imager may be a computed tomography (CT) scanner, but interventional systems (e.g., C-arm / U-arm type interventional systems) are not excluded herein. In some other embodiments, other tomographic modalities (e.g., MRI, PET, etc.) are also not excluded herein.

[0047] The MAR setup also includes a computing system (CS), which is broadly operable to process data (including image data) provided by the imager. The computing system can also allow control of the imager's operation. The computing system can be positioned remotely from the imaging device (IA) or close to it (e.g., integrated into the operator console (CS), from which the user can operate the imaging device; specifically, the user can control imaging operations to obtain medical images for diagnosis, treatment, planning (e.g., in radiotherapy)).

[0048] Broadly speaking, and as will be explained more fully below, the computational system CS includes a facilitator system FS that can be used to facilitate planning data. Planning data can be used in diagnostic scans to acquire cross-sectional or volumetric data V of the target of interest (OI) (e.g., target anatomical feature TAF). Planning data may include the definition of the scan frame SB, as described below. Figure 2 This concept is explored in more detail. The scan frame is the data that allows the imager IA to acquire the correct diagnostic tomographic raw data (e.g., projection data of the correct and appropriate dose for the imaging task at hand).

[0049] Broadly speaking, imaging can include two phases: a first planning (or preparation) phase, and a subsequent second operational phase. The facilitator system FS primarily operates during the planning phase to acquire scan frame data. The scan frame data can then be used during the operational phase to control the imager IA to acquire diagnostic projection data λ of the target image V during a diagnostic scan.

[0050] The computing system CS (particularly its facilitator system FS) can be implemented as a cloud solution running on one or more servers. The imaging device IA can be installed in a clinical facility (e.g., in a hospital). The computing system can be installed or used in a control room adjacent to the imaging room where the imaging device IA is located. In some embodiments, the computing system can be integrated into the imaging device IA. The imaging device IA can be communicatively coupled to the computing system CS via a wired or wireless (or partially both) communication channel CC. The computing system CS can be arranged as a fixed computing system (e.g., a desktop computer), or as the server, or as a mobile device (e.g., a laptop computer, smartphone, tablet, etc.). For example, the computing system CS can be arranged on a workstation WS associated with the imaging device IA.

[0051] Before explaining the operation of the facilitator system FS in more detail, let’s first refer to the components of the imager IA to explain the two-stage imaging process in more detail, so as to help with the explanation of the facilitator system FS later.

[0052] The imaging device IA is operable to generate (specifically acquire) projection data λ, which is forwarded via a communication channel to the computing system CS for reconstruction into a tomographic (sectional) image. The computing system CS runs one or more reconstructor units RECON, which implement one or more reconstruction algorithms. Generally, the reconstruction algorithm implements a mapping that maps the projection data λ, located in the projection domain, to the image domain. The image domain is a portion of 3D space and is located within the examination area ER of the imaging device, while the projection domain is 2D and is located at the (X-ray) detector XD of the imaging device IA.

[0053] The image domain is conceptually composed of a 3D grid of 3D points (voxels), each 3D point having its own 3D coordinates v(x, y, z) relative to a reference 3D coordinate system (X, Y, Z) with an origin. The 3D coordinates of each 3D point are assumed to be defined and given in this paper. Such a reference coordinate system with such an origin is typically set when the imager IA is mounted during the calibration process. Some voxel positions in v are populated with values ​​by a reconstruction algorithm to construct a localization view volume image V0, or virtually any other scan (e.g., a subsequent diagnostic target volume V').

[0054] As described above, the imaging device IA is preferably of the tomographic type and is preferably configured for multi-directional raw data acquisition (e.g., projection image acquisition). Therefore, the imaging device IA is capable of acquiring projection images λ along different projection directions α relative to the examination area ER and thus relative to the target anatomical feature TAF in the patient PAT. In an embodiment, acquisition can be accomplished via a rotation system, wherein at least the X-ray source XS is arranged in a movable gantry MG.

[0055] A movable gantry MG (and, in this embodiment, an X-ray source XS) can rotate within a stationary gantry SG around the examination area ER, where the patient / ROI resides during imaging. Opposite the X-ray source in the movable gantry is an X-ray detector XD, which can rotate with the gantry and X-ray source around the examination area ER to achieve different projection orientations α.

[0056] like Figure 1 As schematically indicated, the patient's longitudinal axis, or imaging axis Z, may extend into the examination region ER during imaging. The patient (PAT) may lie on a patient support (PS) (e.g., a bed), which is at least partially positioned within the examination region ER during imaging. In some, but not all, embodiments, a helical imaging protocol is contemplated herein, wherein there is relative lateral movement along the longitudinal axis Z between the X-ray source (XS) and the patient (PAT). For example, during multi-directional projection image acquisition (e.g., during rotation of the X-ray source (XS) around the patient), the patient support (PS) may be advanced through the examination region ER.

[0057] like Figure 1 The CT scanner setup shown is merely an example of one embodiment and other tomographic imaging instruments (e.g., C-arm or U-arm scanners, cone-beam CT setups, mammography scanners, etc.) are not excluded herein. In some embodiments, a C-arm cone-beam imager is preferred. Additionally, multi-directional acquisition capability may not necessarily be achieved by, for example... Figure 1 The rotating system shown is generated. Non-rotating imaging systems are also envisioned (e.g., in 4th and 5th generation CT scanners), in which multiple X-ray sources are arranged in a source ring, for example, around the examination area. Additionally or alternatively, the detector XD can be arranged as a detector ring around the examination area. Therefore, in such a system, there is no rotation of the X-ray source XS or the detector XD, or both.

[0058] An operator console (OC) may exist, through which users (e.g., medical personnel) control imaging operations. For example, a user can request image acquisition, reconstruction, or other operations, initiate data transfer to the computing system (CS), or stop such transfers as needed. The operator console (OC) may integrate a display device for viewing control parameters, etc.; however, the console display device can also be used by the facilitator system (FS) in this paper, as will be explained later.

[0059] During imaging, a corresponding X-ray beam XB is emitted from the focal spot of one or more X-ray sources XS along different projection directions α. The corresponding beam XB passes through the examination area and through the patient within the examination area. The X-ray radiation interacts with the patient's tissue. The X-ray beam XB is modified due to this interaction. Generally, such modification of the X-ray beam XB involves attenuation and scattering of the initially incident X-ray beam. The modified X-ray radiation is then detected as a spatial distribution of various intensities at the X-ray sensitive pixels of the detector XD.

[0060] In this paper, it is not necessary to acquire the projected image λ over the entire 360° angular range surrounding the inspection area ER. Acquisition over partial angular ranges such as 270°, 180°, or even smaller may be sufficient. The X-ray detector is preferably configured to acquire 2D projected images with rows and columns of intensity values ​​registered by the detector pixels. That is, the detector pixels themselves can be arranged in a matrix layout. Such a 2D layout can be used for divergent imaging geometries (e.g., cone or fan-shaped beams or other beam shapes). However, one-dimensional detector pixel layouts (e.g., along a single line) and beam geometries where neither is parallel are excluded in this paper.

[0061] The reconstructor RECON implements one or more reconstruction algorithms to process diagnostic projection images λ. Specifically, RECON can compute a tomographic target image V of the examination area (where the patient is located) for diagnostic, therapeutic, or other purposes. RECON may be able to generate cross-sectional volumetric image data (“(one or more) image volumes”) V. However, this does not preclude the generation of individual image slices in the examination area as needed. Therefore, the reconstructed image can be represented herein as V, which may include the entire volume, a partial volume, or through a specific portion thereof. Volumetric reconstruction can be facilitated by the helical movement and / or 2D layout of the X-ray detector XD.

[0062] In embodiments, the scan frame SB (data) can be defined automatically or (particularly through user input UI functionality supported by the facilitator system FS) by the user. Broadly speaking, the scan frame CB is a portion of 3D space in the §2 image domain. Therefore, the scan frame CB is a 3D target, but can be visualized as a rectangle in a 2D view, hence the name scan frame. The scan frame defines the volume to be scanned in order to acquire projection data from which the spatial volume marked by the scan frame can be reconstructed.

[0063] The reconstructed volumetric image V can be stored in memory MEM, or it can be processed in other ways as needed. A visualizer VIZ can visualize the reconstructed volumetric image V. The visualizer VIZ can generate a graphical display of the volume or a given slice. The graphical display can be displayed on a display device DD. The visualizer VIZ can map a portion or the entire image volume V to grayscale values ​​or a color palette. The visualizer VIZ can control video circuitry via a suitable interface to enable the display of the graphical display on the display device DD. Alternatively or in lieu of such display, the reconstructed image V can be stored in memory for later close examination or for other types of processing. Such memory may include an image library (e.g., a database (e.g., PACS)) or other (preferably) non-volatile data storage arrangements.

[0064] The reconstructed volumetric image V can be manipulated, for example, by reformatting to define a cross-sectional plane that differs from the cross-sectional plane defined by the imaging geometry. Such reformatting can allow medical users to better identify the type of tissue or anatomical details within a patient for their specific medical purpose (e.g., for diagnosis or preparation of a particular type of treatment).

[0065] Based on the information from the scan frame SB, the target volume V is reconstructed from the diagnostic projection data λ during the operational phase. This reconstructed image V is envisioned as a diagnostic image that represents the target anatomical features TAF (e.g., target anatomical structures, organs, parts of organs or groups of organs, different tissue types, etc.) and has sufficient contrast to safely inform treatment or diagnostic decisions or other medical decisions to complete further imaging sessions (possibly using other imaging modalities) or other tasks (tests, etc.) as needed based on the image.

[0066] A (target) projection image λ of the reconstructed target volume V is acquired at a sufficient dose by appropriately controlling the dose to be used via the operator console OC. This can be accomplished by controlling the voltage and / or ampere setting of the tube of the X-ray source XS to ensure the specific diagnostic image quality. Such a target projection image acquired at a sufficiently high dose may also be referred to herein as a diagnostic projection image λ in order to follow established terminology. However, this nomenclature does not preclude the use of the projection data λ and its reconstructed V for tasks other than diagnosis (e.g., for treatment (in catheterization labs, etc.), planning tasks, or any other task).

[0067] The target anatomical feature (TAF) is related to the medical purpose of the imaging. Therefore, if a patient's liver needs to be examined, the target anatomical feature (TAF) is the liver: the purpose and target of the abdominal scan to be performed is the liver.

[0068] Each target anatomical feature (TAF) is typically associated with an imaging protocol (a set of specifications) that preferably defines the patient's biological characteristics (age, weight, sex, height, BMI, medical records, etc.), certain preferred imaging settings, the desired image contrast to be achieved, the radiation dose for the target anatomical feature TAF for a given purpose, the voltage / ampere setting of the source X to be used, collimation, etc. In short, the imaging protocol encapsulates medical knowledge for any given imaging task, purpose, target anatomical feature TAF, etc.

[0069] Before the operational phase of obtaining the target volume V, the facilitator system FS operates in the planning or preparation phase.

[0070] During the planning phase, an initial positioning view image V0 is obtained. This initial image V0 is preferably a 3D (image domain) positioning view volume, which is reconstructed by the reconstructor RECON (or by a different reconstructor) based on a set of first (non-diagnostic) projection images λ'. However, this set of first / initial projection images λ0 is acquired by the imager IA in the first acquisition during the planning phase, and the image quality of this first / initial projection image λ0 is lower than the image quality of the projection image λ to be acquired later in the operation phase and used to reconstruct the target image V. Therefore, there are two acquisitions: the acquisition in the planning phase is used to acquire low-dose non-diagnostic projection data λ', while the subsequent acquisition in the operation phase is used to acquire higher-dose diagnostic projection data λ.

[0071] Specifically, the radiation dose incurred during λ0 acquisition is lower than the dose that will decrease later (due to the diagnostic scan used to acquire the diagnostic projection image λ). This is to conserve dose on the patient's PAT, and also because the 3D localization view volume V0 is used for a completely different purpose than the target volume V: the purpose of the localization view volume / image V0 is essentially one of navigation, which will be expanded upon here: the localization view image V0 is used to define the scan frame against the target anatomical feature TAF. Therefore, the scan frame controls the imager to acquire higher dose / higher dose diagnostic projection data λ for a smaller field of view (FOV) based on the scan frame SB focused on the target anatomical feature TAF. Thus, in the localization view, a much larger FOV is used, but with a lower dose. No planning is required here. However, once the facilitator FS has defined the scan frame, a diagnostic scan is run with a smaller FOV, but with a higher dose to obtain the target volume of the target anatomical feature TAF at a higher diagnostic level of IQ.

[0072] Referring now in more detail to the facilitator system FS, which is configured to facilitate the definition of the scan frame SB, with respect to which diagnostic quality projection data λ will be acquired in a new prospective diagnostic acquisition performed by the imager IA. This diagnostic quality projection data λ is reconstructed by the reconstructor RECON into the target volume V with an image quality (“IQ”) higher than that of the positioning view image V0 based on the low-dose projection data λ0. During the operational phase, it is anticipated that the new diagnostic acquisition performed by the imager IA will occur in the same examination room for the same patient (for whom the facilitator has already calculated the scan frame) after the positioning view volume V0 has been acquired. This is the preferred workflow. Therefore, for both the positioning view scan and the subsequent diagnostic scan, the patient is preferably held approximately in the same position within the imager aperture BR. However, theoretically, if desired, for example using a suitable registration technique, a positioning view scan could be acquired in the first imaging session of the day, and the patient could return on another day or the same day for a subsequent imaging session, followed by a diagnostic scan. However, such a segmented workflow is less preferred herein, and because the former option (where the patient PAT remains within the imager aperture BR for both scans) is preferred.

[0073] The target volume V assigns a diagnostic quality view to the target anatomical feature (TAF) or any other target of interest (OI). The target OI will primarily be referred to herein as the target anatomical feature (TAF), but the two references will be used interchangeably. The term "target OI" as used herein can also be understood to refer to foreign bodies in the body (if any) (e.g., implants, etc.) and can also be applied to non-medical applications (as also mentioned in the examples herein).

[0074] Although the following text will primarily refer to (e.g., using, for example) Figure 1 The donut-shaped X-ray scanner shown is described in this paper as a form of computed tomography (CT) imaging, or using a C-arm scanner, or in any other design. However, the principles of scan frame finding and definition envisioned herein are not limited to X-ray-based imaging (e.g., CT), but can also be beneficially practiced in other computed tomography modalities (e.g., MRI, PET / SPECT, or any other computed tomography modalities that require planning data).

[0075] Before explaining the operation of the promoter FS or "scan box definer" in more detail, let's first refer to... Figure 2 , Figure 2 The concept of a scanning frame, as can be envisioned in this paper, is illustrated in an illustrative manner.

[0076] First refer to Figure 2 A, Figure 2 A schematic diagram illustrates, for example, the examination region ER defined by the aperture BR of the imager IA. When the CT imager is the instrument primarily envisioned herein, the examination region ER has a cylindrical shape. The shape depends on the construction of the imaging device and therefore may not be cylindrical for other modalities. In MRI, the examination region ER is also typically cylindrical. The examination region ER can also be referred to as the imaging domain. As previously mentioned, the imaging domain is a portion of 3D space, conceptually composed of a grid of 3D locations (voxels), as previously described.

[0077] The patient typically resides within the examination area, or at least within the target anatomical feature TAF. The scan frame is the planning data that can be used for diagnostic (e.g., high-dose) imaging of the target anatomical feature TAF. The scan frame SB is a definition of a portion of 3D space. The scan frame is a 3D sub-part of the image domain. The scan frame is a (mathematically real) sub-volume of the examination area ER. Therefore, the scan frame SB is a (mathematically real) subset of the examination area. The scan frame is positioned and sized such that it includes (covers) at least the target anatomical feature TAF. In other words, the scan frame is at least sometimes a set of voxel locations where at least a portion of the target anatomical feature TAF resides. The scan frame can be defined by any geometry. It does not have to be as... Figure 2 The box shown (cubic prism) can be defined as an ellipsoid, a sphere, or any other shape, depending on the implementation. For example, the shape can be specifically determined by the general image domain coordinate system (X, Y, Z). This is often the case if the image domain coordinate system is Cartesian, and a cuboid definition may be preferred to promote computational simplicity and low latency performance of the FS system. However, if a polar coordinate system or similar is used, a spherical shape definition or the like may be more suitable.

[0078] The scan frame SB defines a set of voxel locations, which are to be collected during the upcoming diagnostic projection data acquisition. The voxel locations form a 3D subset defined by the spatial boundaries of the scan frame SB.

[0079] Once the scan frame is fully defined, it can be passed to the control interface CL. The control interface CL translates spatial information from the scan frame SB into imaging control commands or imaging parameters to control the machine of the imaging apparatus IA to acquire corresponding projection data of all voxels within the scan frame SB. Therefore, the scan frame defines such imaging control commands or imaging parameters. Such control operations can involve the (relative) translation of the patient table PS and / or the rotation of the X-ray source XS around the imaging axis Z. Imaging control commands or imaging parameters can also define other aspects of the imaging geometry, such as the beam shape to be used (e.g., its collimation), the starting point of the X-ray tube rotation, the pitch (the ratio of translation along axis Z to the number of rotations around axis Z), and other imaging parameters or commands.

[0080] The facilitator FS can provide the scan frame SB to the imaging control interface CL in any of a variety of different representations or specifications, all of which are envisioned in different embodiments and depend on the control circuitry / code at hand. Generally speaking, and as will be seen from... Figure 2 Obviously, the scan frame is a specification of voxel locations. For example, the scan frame SB can be provided as a set of all voxel locations forming the scan frame. Alternatively, and preferably for low latency performance, the scan frame can be provided with appropriate parameterization. For example, in addition to size values, the coordinates of a reference point of the scan frame can be provided. More specifically, for example, for a cuboid-shaped scan frame, corner points or center points and the lengths of the diagonals of the cuboid can be provided. If a spherical shape is preferred, the radius / diameter and center point, etc., can be provided. Alternatively, the reference point (e.g., corner point) and the length, width, and height of the scan frame can be provided. Any other parameterization or definition is contemplated, whatever its kind, as long as it is well-defined for each voxel location in the image domain / inspection area, regardless of whether it forms part of the scan frame.

[0081] The size of the scan frame should be such that it encompasses the entire target anatomical feature (TAF). However, if the target anatomical feature TAF is subject to motion (e.g., one or more parts of a human lung, heart, etc.), care must be taken to define the size and position of the scan frame accordingly to account for such motion. Therefore, the anatomical feature TAF may be capable of motion (e.g., translation, rotation, deformation), and different portions of the space within the examination area can be assumed under different motion states. The motion may or may not be periodic.

[0082] In this paper, it is preferred to define the scan frame tightly, such that the scan frame always includes all target feature TAFs during different motion states of the target feature TAF, but it should preferably not exceed the necessary condition, as this may cause unnecessary exposure of patient tissues unrelated to the imaging task to X-ray dose. Therefore, such tightness is a space minimization condition.

[0083] Figure 2 B Figure 2 Figure C illustrates potential errors when defining the scan frame SB of the target anatomical feature (TAF) undergoing motion. Motion states are indicated by different line styles. Motion states indicated by dashed lines represent motion states different from those indicated by solid lines. Therefore, Figure 2 B Figure 2 The state of motion represented in C can be understood, for example, as the compression of the human heart, which delivers its pumping function. Figure 2 C) vs. Expansion Figure 2 Change B). From Figure 2 B Figure 2 C also shows that as the target anatomical feature (TAF) transitions between different motion states, the TAF occupies different portions of the surrounding space in the image domain ER, thus occupying (or not occupying) certain voxel locations. For clarity, the diagram is shown along the Z direction (i.e., only along one spatial dimension). Figure 2 The scan frame SB in B-2D.

[0084] Selected as Figure 2 The example scan box SB shown in B is too small. The target anatomical feature TAF extends beyond the boundaries of the scan box during its motion (e.g., expansion). On the other hand, in Figure 3 In the case shown in C, the scan box is defined too freely, leaving undesirable spatial gaps Δ, because the dose is consumed on voxel points in the gaps, and these voxel points do not provide any spatial information about the target anatomical features TAF.

[0085] Figure 3 Figure D illustrates the correctly selected scan frame following the minimization or compactness principle described above. Therefore, the boundary surfaces of the scan frame SB are preferably selected such that they closely contact the boundary points on the target anatomical feature, particularly those boundary points on the target anatomical feature TAF that travel the furthest as the target anatomical feature TAF changes its motion state. The "dead space" Δ of voxels that the target anatomical feature TAF never visits during its motion is preferably approximately zero. Therefore, few or no such points are visited within the scan frame SB.

[0086] Therefore, the facilitator FS is configured to facilitate the spatial definition of a scan box to ensure that all points of the target anatomical feature remain within the bounds of the scan box SB, as defined, without any voxels that are never accessed by the target anatomical feature TAF during transitions through different motion states. Thus, while the scan box itself remains static, its size and position are defined such that the target anatomical feature TAF remains within the box, where the "dead space" Δ is approximately zero, as... Figure 3 An example scan box SB is shown, which is a tight definition of D.

[0087] In a broader sense, the facilitator system FS helps the user define the voxel positions of the scan frame. This is accomplished by defining the size and position of the bounding box in the image domain ER. The facilitator achieves this by processing the localization view image V0 of the target anatomical feature TAF, acquired at a given (“initial” or “reference”) motion state (i.e., the initial position and extent of the target anatomical feature TAF). Thus, the target anatomical feature TAF in this initial motion state occupies the initial portion of the 3D space in the examination region ER. The furthest motion state of the target anatomical feature TAF mentioned is obtained relative to this initial motion state as recorded in the (initial) localization view image V0. The localization view image V0 is volumetric data and thus captures the examination region ER in 3D to a reasonably large extent (either over the entire examination region ER (whole-body scan) or over portions of the examination region ER that are a priori known to include the target anatomical feature TAF but excluding other surrounding anatomical structures with a large (in any case non-zero) dead space Δ), thereby demonstrating localization view anatomical information about how the target anatomical feature TAF is embedded in the patient’s surrounding whole / larger anatomical structures.

[0088] The facilitator system FS processes this positioning view image V0 as input data and generates multiple "virtual" positioning view images V based on the positioning view image V0. j The prediction results. Preferably, each virtual positioning view V j This represents the target anatomical features (TAF) under different predicted motion states compared to the initial motion state based on V0. In the following text, for simplicity, we will simply refer to the real / initial and virtual positioning view images as "(one or more) positioning views," discarding "volume" or "image." The virtual one or more positioning views V are calculated by the facilitator FS. j Preferably, it has the same dimensions and dimensions as the initial positioning view. Therefore, the virtual positioning view V jThis also represents the volume of 3D space within the examined area, thus capturing the image portion with dead space Δ > 0. However, while the initial positioning view V0 is based on measured projection data, the virtual positioning view is a prediction, and therefore is not, or at least not directly, based on such measured projection data. Therefore, when generating the virtual positioning view V... j When (j > 0), no further dose will be administered. The predicted positioning view image V j It can be conceptualized as an extrapolation of the positioning view volume V0 from the initial input.

[0089] Preferably, the virtual positioning view V is predicted in this way for different times (represented by index j). j The number makes the virtual positioning view V j This covers all motion states of the target, even those furthest from the predefined reference motion states represented / captured in the actual reconstructed input positioning view V0. Theoretically, and indeed in some embodiments, it is envisioned to predict only one such virtual positioning view image V. j *This may be sufficient. If such a single positioning view is predicted, it is preferably such that such a single positioning view represents the motion state (extreme motion state) of the target anatomical feature, which is spatially furthest from the initial motion state of the target anatomical feature captured in the initial measurement positioning view V0. However, preferably, and in some embodiments, the facilitator system calculates more than one such predicted positioning view image to represent the initial state according to the initial positioning view V0 and as in the extreme positioning view V0. j The intermediate motion states between the predicted and represented extreme motion states can be appropriately referenced in this paper. Multiple such positioning views V can be used. j The media footage (as "animation") is sequentially replayed to be displayed on a display device DD. Therefore, animation can be understood as conveying a visual sense of "swaying" of the target's anatomical features, sufficient to allow the user to estimate a sufficiently large but tight scan frame based solely on observation of the animation. Often, as needed, the animation can be replayed once or multiple times in a playback loop. Thus, the media footage V displayed as video / animation... j The display shows the user how the target anatomical feature (TAF) is expected to move as it transitions through different motion states, thus occupying different spatial portions of the examination area. The display device can be integrated into the operator console (OC) of the imager (IA).

[0090] The facilitator system FS may be able to predict the localization view V based on such output. jTo calculate the scan frame without any user intervention, and especially without displaying such a calculated prediction or virtual positioning view V in the media / video footage as described above. j Any one of them. However, preferably, some user participation is also present through the use of a user interface (UI). In such a more user-centric embodiment, the predicted positioning view V is displayed in conjunction with or instead of the initial V0. j One or more images are provided, and the user can then use graphical tools, which can be supported by the user interface UI, to allow the user to position themselves according to the displayed predicted view V. j Spatial information is appropriately used to essentially draw / delineate scan frames, etc. In particular, the aforementioned media shots / animations can be rendered for the view display device DD to provide the user with an illustration of the spatial portion occupied by the target anatomical feature during the transition of the target anatomical feature's motion state.

[0091] After one or more repetitions of the video shot, the user can stop the animation at the point where the farthest position is reached in the associated extreme motion state, and then draw the scan box accordingly. The user interface (UI) can be implemented via a touchscreen or through pointer tools (mouse, stylus, etc.). The user can draw geometry such as a rectangle in 2D or a cuboid in 3D to tightly enclose the extreme motion state of the target anatomical feature (TAF), where the boundary surface or boundary line of the scan box SB is drawn tangent to one or more of the spatial extreme points of the target anatomical feature (TAF), as in the predicted extreme localization view V. j As indicated in *, the coordinates specified by the drawing action are captured and stored as scan frame specifications, which can then be forwarded to the control interface CL automatically or upon request. A diagnostic scan based on the scan frame thus specified can then begin. Similarly, the scan can be started automatically when the user assigns a specified scan frame, or initiated when the user issues a forward signal. Due to the 3D nature envisioned herein, a predicted positioning view V can be completed along each spatial dimension X, Y, Z. jThe video footage is replaced, and the scan frame specifications are completed in each view to capture extreme motion states in all dimensions. The three partial scan frame specifications thus obtained can then be combined in a set manner (one such specification per view along the corresponding axes of the coordinate system X, Y, and Z) for a complete spatial definition of the scan frame, and then dispatched to the control interface CL. For each partial specification along the corresponding standard axis X, Y, or Z, the user interface UI with drawing tools can be used again. For most purposes, a simple 2D display of the predicted positioning view in static or animated form (e.g., on the existing display unit of the imager's operation console OC) may suffice. Alternatively, 3D rendering (preferably rotating the target anatomical features in any state) based on algorithms such as marching cubes or others allows the user to define the scan frame perimeter in such a 3D view at once for all three dimensions. The definition of each dimension in the partial specification may not necessarily be based on visualization along the standard axes X, Y, and Z. Alternatively, such per-view specifications for the scan frame SB can be based on a reformulated positioning view, and can be replaced by any new coordinate system with its axes X', Y', Z' instead of the standard coordinate system X, Y, Z.

[0092] The user interface (UI) can utilize freehand drawing tools, or guided tools that assist users in defining preset shapes for the scanframe. Such scanframes are drawn based on user changes to one or two dimensions, allowing users to center the scanframe accordingly while dynamically adjusting and expanding the third dimension to capture the appropriate predicted positioning view V. j The extreme motion state is indicated by * (at the point of maximum spatial deviation from the initial motion state). Therefore, such guided drawing tools limit the automatic completion of user input in certain directions and in other directions relative to the geometry (angles between edges, etc.) of the preset scan frame shape.

[0093] Now for reference Figure 4 , Figure 4 A schematic block diagram of an facilitator system FS as envisioned in the embodiments herein is shown.

[0094] In a broader sense, input data is received at the input interface IN. The input data includes at least one positioning view V0. This is optionally processed by the predictor component PC along with additional information provided by the camera DSC (explained in more detail below). The predictor component PC can use a motion model M to calculate one or more virtual positioning views V0 based on the input data. j This one or more virtual positioning views V j It can be used for output via the output interface OUT. Based on the output virtual positioning view V jThe scan frame specifications can be obtained and passed to the control interface for diagnostic / more detailed imaging, but in any case, with an IQ higher than the input positioning view V0. The planning module PM can automatically calculate the scan frame SB specifications, or define them through interaction with the user interface (UI) as described above. In this case, the planning module PM can still be used specifically to translate user input (e.g., drawing instructions as mentioned above) into scan frame SB specifications. For example, the planning module PM can combine the specifications of each view portion into the desired 3D specifications of the scan frame SB.

[0095] Different virtual positioning views V for output at external or internal output interface OUT j Preferably, it has the same dimensions (length, height, width) as the initial input positioning view V0. The planning module PM can combine data from the virtual positioning view V0. j The information is used to calculate the scan frame SB, for example, as a definition of a geometry such as a cuboid, sphere, ellipsoid, or any other volume, as previously described. In the automated implementation, the planning module can identify extreme value positioning views V. j *And fits a predefined geometric shape to closely cover, as in the extreme value positioning view V. j The extreme points of the estimated target anatomical features are shown in the figure. Segmentation can be used with tangential fitting of a preset shape to geometrically define the scan frame. The scan frame can be displayed and stored for later use in memory MEM; or the scan frame can be passed to the aforementioned control interface CL to control subsequent operations in diagnostic scans of the imager IA, thereby acquiring a second, higher IQ projection data λ in diagnostic scans targeting the scan frame voxels.

[0096] The newly collected, higher-quality projection data λ can then be reconstructed into a diagnostic volume V of the target anatomical features with low noise and good contrast using the RECON reconstructor. This image can then be used for treatment, diagnosis, training, or any other medical task requiring it. This process flow can be called an automated process flow because it does not require user intervention.

[0097] exist Figure 4 The lower left shows a second embodiment of user engagement via the user interface described above, wherein the graphics display generator GDG displays a virtual motion state positioning view V in the graphics display object GD. j One or more of the images, preferably a virtual motion state positioning view V, are displayed in the motion picture / animation. j One or more images from the image. Positioned views V can be selectively displayed on the display device. j This allows the predicted positioning view V to be displayed, either alone or in combination with the initial volume V0. jOne or more images and / or, as needed, a virtual positioning view V. j V j Two or more images from the original volume V0. Preferably, the virtual positioning views V are displayed alternately in a given order, preferably in combination with the initial volume V0. j To present the animation. The display order is preferably, for example, a expected natural sequence of motion states, but this may not be necessary. Therefore, the predictor component outputs the positioning views V in the correct order. j Or output the positioning view V in any order. j Then it is possible to achieve this by using the virtual positioning view V j The target anatomical feature location is compared with the initial location recorded in the initial input localization view V0 to determine the localization view V. j Reordered into anatomical sequence. First, those virtual positioning views V that are spatially closer to the original view are displayed. j Then, those virtual positioning views V, which are spatially further away from the original view, are displayed. j This replicates natural animation that adheres to anatomical reality. However, as far as the scan frame is defined, the display order may be sufficient, as long as extreme views (one or more) can be identified individually in 3D or along each dimensional axis as needed. j That's all.

[0098] Referring now to embodiments of the facilitator system FS in more detail, and with regard to input data, this document envisions two main embodiments, which will be explained in more detail below.

[0099] One such embodiment relies solely on a single or multiple localization views V0 collected from previous measurements (projection data) gathered during exploratory image acquisition at low doses. The predictor component PC is then able to generate virtual localization views V0 for additional motion states of the target anatomical features based solely on the single or multiple input localization view volumes V0 received as input. j The prediction. However, in another embodiment (which can be called a dual-channel embodiment), in contrast to the single-channel embodiment just described, there is a second channel (the first channel is one or more input positioning views V0), in which video footage of alternative features SF of the patient's PAT is acquired by a suitable non-ionizing radiation-based camera DSC.

[0100] A lens camera DSC can be mounted in the gantry SG or aperture BR of the imager IA to acquire images of the surrounding anatomical structures SF (e.g., the patient's skin surface). However, such an arrangement within the aperture or gantry is not mandatory, and the lens camera DSC can be mounted anywhere in the examination room where the imager is installed, such that images are acquired before the patient is introduced into the aperture. For example, a video camera can be mounted on a bracket, wall, or ceiling as needed. A camera DSC can be specifically adapted, or an existing such camera can be reused for scan frame planning. Such existing cameras may already be used for patient positioning or monitoring.

[0101] The video footage over time t This includes some dynamic information about how the motion state of the target anatomical features changes. However, this information is not directly captured because the target anatomical features are internal anatomical structures and the camera's DSC is non-ionizing. Instead, it is captured by the camera through video footage over time. t External features are involved (e.g., surrogate features SF such as the patient's chest, which also undergo motion states, but different from the motion states of the target anatomical features). However, the motion states of surrogate features SF can be correlated with the motion states of the target anatomical features, and such correlations can be utilized in some of these two-channel embodiments described herein.

[0102] Then, the predictor component PC combines and processes the information from the positioning view image V0 and the video footage m from the camera DSC. t (It may include multiple frames m1, m2, and m3, etc.) to calculate the virtual positioning view V. j The user and / or planning module (PM) can use this virtual positioning view (V) j To obtain the scan frame SB. As described above, the camera is preferably operable based on non-ionizing radiation (e.g., near-infrared (NIR), IR) or optical (visible) spectrum. A depth-sensing camera can be used, which acquires the spatial distribution of the surrogate feature SF (e.g., chest) and how its motion state changes over time (e.g., respiratory circulation, which may be related to the motion state of (one or more) lungs or heart, etc.). The depth-sensing image can be conceptualized as a 2D set of point measurements, where each pixel represents a depth measurement from the corresponding focal spot to a corresponding point on the patient surface (e.g., a point on the patient's skin or clothing covering the chest) along a geometric ray extending from the focal spot of the sensor of the camera DSC, where different pixels correspond to different such rays and points on the patient. The chest is an example of a surrogate feature SF, whose motion state is used herein as a spatial information substitute for at least part of the actual motion state of the target anatomical feature TAF.

[0103] As an example of non-ionizing camera DSC, depth-sensing camera DSC acquires a sequence of depth distributions from a camera reference location on the skin of the patient's PAT (e.g., the patient's chest, etc.). Depth values ​​will vary over time according to the patient's (specifically, alternative features, skin pathways, chest, etc.) movement. Therefore, the movement states of external alternative features within the observable patient PAT, or the movement of (internal) target anatomical features otherwise associated with them, cannot be observed in this manner as described. However, without consuming additional doses, the movement states of alternative anatomical features (SRs) are observed and used. For example, an alternative feature (SF) (skin of the chest) may at least partially surround a target anatomical feature (TAF) (lung, heart), or may otherwise be spatially related. Movement of the target anatomical feature (TAF) may or may not cause movement of the anatomical feature, such as in the case of lung vs. chest skin or heartbeat vs. chest skin. However, such causality is not required herein as long as some correlation exists between the movement (state) of the target anatomical feature (one or more) and the movement (state) of the alternative feature (one or more) (SF).

[0104] Therefore, it is possible to correlate spatial information about the motion of the alternative feature SF (e.g., patient skin near the target anatomical feature), and then extract motion components from this information and apply them to the initial input localization view V0 in order to predict / extrapolate virtual localization views V for other motion states of the target anatomical feature TAF. j Similarly, a single view of such a location / motion state might be sufficient to define the scan box in this paper, but it is preferable to acquire multiple such localization views, each representing a different intermediate motion state, to obtain a more robust and realistic understanding of the motion state of the target anatomical features. However, when calculating multiple such localization views V... j In such cases, extreme motion states should preferably be included. Similarly, more robust results can be achieved in this way.

[0105] Now turning more specifically to the dual-channel embodiment, the predictor component PC can use a motion model M, as will be described in more detail, to calculate virtual positioning views V for different motion states of the target anatomical feature TAF. j .

[0106] The virtual positioning view V can be calculated by the predictor component PC. j The motion model M can be either of these two types. The model can be an explicit motion model or an implicit motion model. In an explicit motion model, the motion components are explicitly computed as vectors, which can then be applied to the initial localization view image V0 to calculate the predicted volume V. jFor example, a single such vector can be used to translate the image representation of the target anatomical feature TAF according to the input positioning view V0. More than one such vector can be computed for different points across three dimensions, and these vectors can be applied at different points to the image representation m(TAF) of the target anatomical feature TAF to achieve, for example, deformation. Tensor or matrix representations can be used for more complex motion patterns. Rotation matrices, etc., can be used. The intra-image representation m(TAF) of the target anatomical feature TAF can be geometrically defined using segmentation. One or more vectors can be applied at points on the boundaries of the intra-image representation m(TAF).

[0107] In another embodiment (particularly in the single-channel embodiment), a machine learning model is used (e.g., an artificial neural network that does not require dedicated computation of motion components). In such a machine learning approach, the model may be able to extrapolate suitable patterns learned from a large number of training data instances based on the input positioning view volume V0 and known motion patterns (regardless of how they are obtained, the aim is to learn the general relationship between the input positioning view image V0 and motion information). Therefore, in this machine learning embodiment, even based on a single positioning view input image V0, a ​​virtual positioning view V can be computed. j The sequence. In some embodiments, more than one positioning view image V0 can be used as input. And as previously mentioned, the machine learning model M can be configured to compute only a single virtual positioning view, particularly a single virtual positioning view V for extreme motion states. j *

[0108] In an alternative embodiment, a machine learning model can be used in a dual-channel approach. In this embodiment, the model processes video footage of SF images of the patient's PAT surface acquired by a non-ionizing camera. t As explained above in conjunction with the localization view input V0, the video shot captures the temporal evolution of alternative anatomical features SF (e.g., features surrounding the target anatomical feature TAF). Such alternative features can specifically include the surface of human skin, or any other body part whose motion can be correlated with the motion state of the internal target anatomical feature. The model M is the training data used to learn the relationship between the localization view image and such camera shots of the alternative features SF. For each frame m received from the camera DSC... j，t＝j Given initial information from the measured input positioning view V0, the corresponding virtual positioning view V representing the motion state of the target anatomical feature TAF can be calculated. j Therefore, conceptually, the localization view input V0 can be understood as a data-based regularizer for model M, while the surface image stream provided by the camera provides motion information. Thus, the input localization view V0 and the surface video stream m...j，t＝j The combined information helps the "manipulation" model M provide the "correct" virtual positioning view for each or some shot frame moments. j .

[0109] Turning more specifically to the explicit motion modeling embodiment, the facilitator FS may include one or more various components, such as a calibrator CAL, a registration unit REG, and / or a motion part extractor MX. Such components may be part of a motion model adapter MP, which is operational to adapt the motion model M to the details of a given patient PAT and the current imaging setup, including the imaging geometry.

[0110] In a broader sense, the calibrator CAL uses imaging geometry to calibrate the acquired camera lens m t The registration unit REG uses the selected motion model M to register the lens m. t More specifically, the calibrator utilizes the imaging geometry of the imaging equipment to calibrate the camera images. The camera and CT scanner are typically mounted in fixed locations within the scanner room, and their respective distances, angles, and other geometric parameters are known according to system specifications. Additionally, a registration process performed by the registration unit (REG) can be executed during installation, and the registration process can optionally be validated periodically. For example, it is possible to identify and use markers placed in known locations, such as the patient's bedside (PS) or elsewhere, within the camera's RGB images for this purpose. Other techniques can be employed.

[0111] Explicit motion modeling can include motion extraction from moving shots by a motion extractor MX. t Extract motion vectors. Then, the registration unit REG can utilize the video footage m t The timing is used to register the extracted motion information. Once the motion components (one or more vectors, matrices, tensors, etc.) are properly registered in time and / or space, they can be applied to the localized view input V0, which serves as a part of the model in this case. For example, vector fields can be applied to expand or compress or otherwise move (e.g., rotate, translate, etc.) the voxels of the localized view image V0. This allows for the extrapolation of different motion states. Therefore, a virtual localized view V can be extrapolated. j For each new video frame m t There is a virtual positioning view V j Extract the corresponding motion vector from it. At this point, all voxels in the positioning view V0 (including the voxels representing m(TAF) within the image that constitute the target anatomical feature TAF) can undergo the extracted motion. However, motion information can still be obtained by using a segmenter (not shown). Focusing is applied only on the in-image representation m(TAF) of the target anatomical feature TAF. However, this may require a properly trained segmenter capable of handling low-signal, high-noise data and low-dose localization view images V0. A machine learning-based segmenter (trained on ground-value data of training input images that may include artificial noise artifacts) can be used. The planning model PM can also use a similarly trained segmenter when calculating / fitting the scan box shape using such segmentation results s(m(TAF)) around the in-image representation m(TAF) of the target anatomical feature TAF.

[0112] Additionally or alternatively, such a segmenter can be used in user interface (UI)-based embodiments to better assist the user in drawing scan boxes sufficiently close around the segmented structure s (m(TAF)). Regarding the density requirement, a dead space Δ = 0 is preferred herein, but this may not be applicable in all embodiments, and a more relaxed dead space requirement Δ' (Δ' > 0) with some appropriately small error margin is also acceptable.

[0113] As an optional feature, the system FS may include a motion validator (VAL). The VAL compares extracted motion with pattern-predicted motion. If a deviation exceeds a preset threshold, abnormal patient motion may be present. This can be flagged, and control signals can be transmitted to a transducer (TR) (e.g., a display device, speaker, light, etc.). The transducer TR can respond by issuing sensor alarm signals to alert the user to potentially abnormal patient motion that may be detrimental to IQ.

[0114] The operation of some components of the facilitator FS will now be described in more detail regarding an embodiment of explicit motion modeling (rather than implicit modeling (based on machine learning) as will be described later). Explicit motion modeling is preferably the motion of the alternative feature SF (the patient's surface, e.g., the skin surface of the chest, etc.). The modeling motion of the alternative feature SF can be known to be related to the motion of the target anatomical feature TAF.

[0115] Figure 4 This is a diagram illustrating the operation of the registration unit REG and calibrator CAL used for explicit motion modeling. As an example, respiratory cycle-based modeling is used, but the principles described in this paper are easily applicable to other scenarios.

[0116] Generally, as previously stated, spatial and temporal registration of motion models is preferred in this paper. Spatial registration can be understood as matching which part of the moving portion in the positioning view corresponds to the motion recorded using a camera (e.g., a depth-sensing camera). Temporal registration can be understood as the identification of phase information (e.g., inspiratory and expiratory phases or cardiac phases, etc.). The latter (i.e., temporal registration) can be achieved by telling the patient whether they are inhaling or expiring before taking the positioning view.

[0117] Now, we turn more specifically to spatial registration, where the registration unit REG can match the actual patient position on the bed with the camera image m t Registration is performed. Therefore, registration-based methods can be used for this purpose. For example, certain landmarks LMs of the patient (e.g., eyes, mouth, hands, joints, etc.) identified in the RGB channels of the video stream can be used to identify the patient's position on or relative to the bed PS. These landmark LMs can be used to spatially align the patient PAT with the explicit motion model M. Depending on the alternative feature SF or the target anatomical feature TAF, a respiratory prediction model can be used. As an example of the alternative feature SF, it is preferably known that the video stream m should be used. t Which 2D region of the D channel (depth channel) is used to observe distance changes caused by breathing? Therefore, within the shot, the target area representing the motion of the alternative feature SF can be identified.

[0118] Patient-specific respiratory motion models can be selected or generated. The model preparer (MP) can operate to calibrate / parameterize the model M using the patient's actual respiratory cycles over time or whatever alternative feature SF. For example, the camera's D-channel and target region can be used. The minimum and maximum distances of the patient's surface SF within the respiratory cycle are identified and used to calibrate the motion model in the time domain. Once the model M is thus prepared, the motion extractor (MX) extracts the motion components. The predictor component can apply these motion components to some or each voxel of the 3D localization view V0 for some or each time point. For example, an explicit model can be used to calculate the translation of the target anatomical feature TAF voxel due to respiration to obtain a virtual localization view V for a given time t = j. j This operation can be repeated over time to generate sequence V. j Then it is possible to convert sequence V j Displayed as an animation to impart a degree of motion, thereby facilitating the definition of a tight scan box covering the target anatomical features TAF in most (if not all) motion states.

[0119] Continue to refer to Figure 4Furthermore, within the context of explicit motion modeling, more details about the model preparer MP are provided, which can utilize the RGB camera DSC to identify the patient landmarks to spatially align the 3D breathing model with the patient, specifically, aligning it with the target area to be observed in the camera's D channel for calculating distance changes due to the patient's breathing. Figure 5 This illustration shows a method related to the target region. For example... Generative The rectangle shown illustrates at least a portion of model M. This rectangle illustrates the spatial alignment of a region (“target region”) in the D channel based on its correlation with the landmark LM identified in the RGB channels of the camera. For example, the rear position is aligned with the patient table PS. The target region is motion information representing the motion state of the surrogate feature SF in the shot. Therefore, the spatial registration of the motion model (by...) Adversarial Nets (The rectangle shown in the diagram) can be based on landmarks identified in the camera's RGB channels. Temporal breathing state is determined using only the pixels within the rectangle. The breathing state uses the value of the D channel. Using an RGB-D camera that combines optical imagery with depth information is not required herein. These two channels can alternatively be provided by separate cameras. The optical channel / optical camera allows for the identification of landmarks using image recognition techniques, while the depth channel / depth camera allows for the re-encoding of motion state information from alternative features (SF). Again, the optical channel is optional and may not be required in some embodiments.

[0120] Instead of using the correlation with the landmark LM, the input 3D localization view V0 itself can be used for registration. The input 3D image V0 provides some tissue and bone structures that can be registered with the 3D atlas of the human body, and it can itself be "flexibly" matched using the contours of the patient and the contours of the atlas (as can be identified in the RGB channels).

[0121] In another embodiment, precise spatial alignment of the landmark LM and motion model between the camera data and the positioning view data may not be necessary. Instead, a temporal motion model can be extracted from the camera data and applied directly to the positioning view data V0 to generate breathing-induced deformation.

[0122] Temporal alignment of the 3D breathing model can be based on the identification of inspiratory and expiratory motion states. These inspiratory and expiratory motion states are examples of the aforementioned extreme motion states, which exist not only for the target anatomical feature TAF but also in the relevant motion states of the surrogate feature SF. Preferably, for robustness, some intermediate motion states are also identified.

[0123] Depending on which alternative (anatomical) feature SF is used, there are several options for selecting a suitable respiratory model. For changes in the motion state of such an alternative feature SF due to respiration, the following options can be considered.

[0124] Such a model allows for movement in the anterior / posterior direction and can identify linear interpolation between (anterior) maximum inspiratory / expiratory states and (anterior) minimum inspiratory / expiratory states in the D-channel of the target region. The posterior position is fixed.

[0125] Such a model can be refined by automatically segmenting the 3D localization view V0 within the bone and soft tissue structures. Interpolation can then distinguish between rigid and more flexible moving parts of the respiratory cycle.

[0126] The model is preferably spatially aligned with the 3D positioning view V0 and matched with the respiratory cycle. Therefore, it is beneficial to record, as described above, which motion state (inhalation / exhalation) the 3D positioning view V0 was acquired during. For example, such matching can be easily achieved when the camera also detects respiratory motion during the acquisition of the positioning view.

[0127] In another embodiment, and now turning more specifically to implicit motion modeling based on machine learning models, this paper envisions various methods (either in a two-channel embodiment where the input data includes camera lens m) t And input positioning view V0), or in a single-channel embodiment (where the input data includes positioning view V0 without camera lens m) t Such ML methods can use training data. For example, in... Generative Adversarial Nets The training system TS illustrated in the diagram can use machine learning algorithms based on such training data. This is used to adjust the parameters θ of model M. The machine learning model can be general and may not require explicit analytical modeling of motion components as explained above in the embodiment based on explicit motion modeling.

[0128] For example, in one embodiment, the query facility Q is used to locate historical patient records of various patients in a medical database MD (e.g., HIS (Hospital Information System) or PACS (Picture Archiving and Communication System)). For those cases where a locational view image including the TAF (Transient Aspect of Interest) of interest was acquired at a point in time, the historical patient record varies in terms of age and sex as well as cross-demographic information.

[0129] Some imaging setups used in such historical examinations may have included video-lens camera DSCs as described above, which, as previously mentioned, are suitably mounted in the examination room, on a gantry, or within an aperture. Such camera DSCs may have acquired images of the surrounding anatomical structures SF (e.g., the patient's skin surface may have been acquired during the acquisition of projection data for the positioning view V0). Such cameras may have been used for purposes other than those intended herein (e.g., patient supervision, etc.). However, such images can be reused to train a machine learning model M to predict the virtual positioning view V. j The current goal.

[0130] By capturing alternative features SF (e.g., chest skin) during the acquisition of projection data for the input localization view image V0 using a camera shot, it is possible to collect data pairs of naturally associated training input data (video footage) and their targets (corresponding localization view V0). Therefore, it is possible to associate the reconstructed localization view V0 with certain surface images (e.g., depth-sensing distributions acquired during projection data acquisition) using timestamps. Then, it is possible to pair the training data... Such moments The training data is fed into the ML model M for training. Thus, model M can be trained to associate surface images of the surrounding structures SF of different patients with different associated localization view instances V0. Therefore, the dynamic correlation between such video shots and corresponding localization view images V0 can be learned. It may be difficult to locate such data pairs of surface shots and associated localization views across historical patient records. In such cases, experimental data collection projects can be set up at different medical facilities. Such projects may include mounting cameras at tomographic imaging instruments such as CT or MRI to acquire a sufficiently large accumulation of such training data pairs over time. Alternatively, existing intra-port / gantry cameras originally intended for patient monitoring during imaging can be repurposed in this way.

[0131] Therefore, in this 2-channel ML method, it is possible to learn from training data the correspondence between the temporal motion states of a substitute feature SF (e.g., on the surface of the chest) measured by the camera and the internal organ motion states of the target anatomical feature TAF obtained according to the positioning view V0. The training data may include the data pairs of multiple 2D or 3D tomographic images (MRI, CT, US) acquired during respiration and the associated camera lenses.

[0132] Regressive machine learning models (e.g., neural network models, preferably of convolutional type) can be used to learn this relationship between surface imagery and associated positioning views. Once trained, and during the deployment / inference phase, the machine learning model takes the current positioning view and the currently acquired surface video frame as input to compute a virtual positioning view and an initial (projection-based) positioning view V0 for a given video frame. Thus, the above is an example of a dual-channel application with a machine learning implementation.

[0133] In another approach, generative machine learning modeling is used. Such an approach allows for setting up a single channel with only the input localization view V0 required. Camera DSC is not needed in this paper, but it can still be used if necessary. In this generative ML modeling approach, a database queryer Q is used to localize historical images including different kinds of localization view images capturing various anatomical structures of interest from various patients. It is impossible to collect all these localization view images under the same motion state. Therefore, the motion states encoded by those historical localization views V0 can encode different motion states, thus effectively providing samples of localization view shots across different demographic information. Given any input initial localization view image V0, this training shot may be sufficient for the generative ML network to learn virtual localization view shots V. j Then, after training, during deployment, the positioning view can be obtained from the output. (wherein, index) (This refers to the number of times the generative model M is applied iteratively) to generate virtual localization views V under different motion states of the target anatomical feature TAF. j Different instances. Alternatively, the model can be trained to output a sequence of predicted localized views given an input localized view. For example, for any given instance j, the historical localized view v0 (lowercase) can be split into input... Furthermore, two or more historical location views can be used as targets. This can be divided into different combinations to form a training dataset of appropriate size. “m” determines the “length” of the sequence of localized views that one wants to predict (and therefore animate, if needed).

[0134] Generative ML modeling can be conceptualized as sampling from the underlying probability distribution of the space from which training data is extracted.

[0135] Generative machine learning models can include GAN-type (Generative Adversarial Networks), Variational Autoencoders (VAEs), and hybrid models (e.g., Hidden Markov Models (HMMs), Markov Random Fields (MRFs), etc.). Some such generative models can be based on ANNs (Artificial Neural Networks) (preferably based on convolutional ANNs (CNNs)) because such models have been observed to produce good results on spatially relevant data (e.g., images, such as the localized view images of which this paper is of primary interest). As an extension or alternative to CNN models, transformer-type models with attention mechanisms can be used, where convolutional filters are adapted to act differently on different parts of the intermediate feature map input.

[0136] Now turning to an example of a GAN-type ML model M, Ian Goodfellow et al. previously already in " Figure 5 Figure 7 The document describes such ML model types (the "..."). Figure 6 "Available on the preprint server at arXiV:1406.2661v1, released on June 10, 2014."

[0137] GAN-type learning can include two neural network models: a discriminator network (“D”) and a generator network G. The discriminator D can be set up as a neural network classifier. The discriminator D can be arranged as a deep fully connected network, a deep fully convolutional network, or a hybrid network including one or more layers of both types.

[0138] During training, the GAN operates as follows: The generator G generates sample localization views G(r) based on a seed signal (r) applied to it (e.g., random noise). The discriminator D attempts to distinguish the generator output G(r) from the true localization view sequence; the discrimination results can be stored in the training database. The ground truth consists of localization views representing target anatomical features (TAFs) at different motion states in different patients, collected from medical databases through queries as described above. The historical sequence.

[0139] A specially configured cost function can be used to adjust the parameters of the generator G and the discriminator D. For example, Goodfellow et al. (ibid.) described the binary cross-entropy cost function on page 3 of their formula (1). This cost function represents two opposing objectives similar to a zero-sum game: the objective for the discriminator D is to maximize the probability of it being correctly labeled, while the objective for the generator G is to produce an output such that the discriminator D cannot statistically distinguish between the real localized view V0 extracted from the training data and the virtual localized view generated by the generator.

[0140] The parameters of models G and M can be adjusted alternately in separate iterative runs within a min-max flow. Iterations cause the parameters of models G and D to converge to a Nash equilibrium type, at which point the generator can produce realistic "pseudo-locality views." At this point, the discriminator D can no longer distinguish between the pseudo-locality views and the baseline truth, and this distinction is no longer necessary. The generator G can therefore be used to generate virtual locality views based on a given locality view V0.

[0141] As an extension of GAN-type generative ML models, conditional GANs (cGANS) can be used.

[0142] However, generative modeling is not the only option for a single-channel implementation. Additionally, this paper envisions any regression-type ML model setup (e.g., a recurrent neural network) configured to handle ordinal data. One type of recurrent neural network could include LSTM (Long Short-Term Memory) networks, etc.

[0143] For more details, please refer to [link / reference]. Figure 6 The training system (TS) in ML models can be broadly categorized into three processing phases: the initial training / learning phase, an optional subsequent testing phase, and the subsequent deployment / inference phase. The training phase involves adjusting the model's parameters based on the training dataset. Training can be implemented using training / learning algorithms. In testing, the model's performance is validated based on test data. In the deployment / inference phase, the model is used in the intended environment (e.g., in clinical practice, when the model processes deployment data to compute virtual positioning views). Test data differs from training data, and deployment data differs from both test and training data. The training system (TS) implements the training phase. The training phase can be a one-time event or can be repeated using an earlier trained version of the model as a starting point. For example, training can be repeated once new training data becomes available.

[0144] The training system TS can be implemented on one or more computing systems and configured to train the ML model M. The training data is shown as data pairs used for the supervised setup. ,in, Indicate the training input image, and It is its associated target, and the index " " is the index of the training data pair, which constitutes the training dataset.

[0145] During the training phase, the architecture of the machine learning model M (e.g., a CNN) is pre-populated with an initial set of weights. The weights θ of the model NN represent parameterization. Weights can include filter parameters from the convolution operator (CV). The purpose of training the system TS is to base it on training data. To optimize and thus adapt parameters In other words, learning can be mathematically formulated as an optimization scheme where the cost function F is minimized, but alternatively, a double formula maximizing the utility function can be used. Instead of such an initialization model, a pre-trained model can be used.

[0146] Now assume the paradigm of the cost function F, which measures (one or more) aggregated residuals, i.e., the summation error between the data estimated by the neural network model NN and the target in some or all of k based on the training data in the batch or all of the training data: (1) In formula (1), the function The model M is represented as being applied to the training input. The result is often different from the associated target ỹ. This is achieved through distance metrics. This is used to measure the difference or the corresponding residual for each training data pair i. Therefore, the cost function F can be pixel / voxel based, for example, an L1 or L2 norm cost function, or any other norm Lp. Specifically, when the output layer regresses to one or more virtual localized views, the Euclidean cost function (e.g., least squares or similar methods) in (1) can be used for the regression task described above. The cost function in (1) is merely exemplary. Other cost functions can be used, such as the cross-entropy-based cost function for the generator G and discriminator D in the GAN or cGAN models described above.

[0147] Output training data It is the input training image data that is applied. Related goals The estimation results. As mentioned above, this is typically the output. The target associated with each data pair k Errors exist between them. Then, an optimization process (e.g., back / forward propagation or other gradient-based methods) can be used to adapt the parameters θ of the model M to reduce the error for the considered data pairs. The residuals, or preferably the sum of the residuals in a batch (subset) of training data pairs from the full training dataset.

[0148] The optimization process can be performed iteratively. After one or more iterations in the first inner loop, the training system TS enters the second outer loop, in which the model's parameters θ are updated by the updater UP with respect to the current batch of data. Update the data accordingly, and in this second outer loop, process the next training data pair accordingly. Or the next batch. The structure of the updater UP depends on the optimization process used. For example, the inner loop managed by the updater UP can be implemented based on the gradient of F, via one or more forward and backward propagations in a forward / backward propagation algorithm or other gradient-based setup. When tuning the parameters, the aggregated residuals (e.g., summation residuals) of all training data pairs are considered. It is possible to form aggregated residuals by configuring the objective function F as the sum of squared residuals (e.g., some or all of the residuals considered in each batch in Equation (1)). Other algebraic combinations instead of sums of squares are also envisioned. In general, the outer loop proceeds through batches (sets) of training data items. Each set (“batch”) including multiple training data items and the sum in (1) is expanded over the entire corresponding batch, rather than iterating through training data pairs one by one, but this paper does not exclude the latter option.

[0149] Optionally, one or more batch normalization operators (not shown) can be used. Batch normalization operators can be integrated into the model M, for example, coupled to one or more convolution operators (CVs) within a layer. The BN operator allows mitigation of the vanishing gradient effect—the gradual decrease in gradient magnitude experienced during repeated forward and backward propagations in gradient-based learning algorithms during the learning phase of the model M. The batch normalization operator (BN) can be used for training, but it can also be used for deployment.

[0150] Able to consider all learning schemes (especially supervised schemes) such as Figure 7 The training system is illustrated. In alternative embodiments, an unsupervised learning scheme may also be envisioned herein. One or more GPUs may be used to implement the training system TS. A fully trained machine learning module M may be stored in one or more memories MEM and can be used as a trained machine learning model in the system FS.

[0151] Alternatively or additionally, historical location view imagery can be procured, and the training data generator system TDGS can artificially generate training data. For example, a generative model (e.g., GAN) can be used, which is capable of being trained on some ground-valued benchmark data to produce realistic surface shots that represent the correct motion states of the surrogate feature SF of interest. Historical location views can be more easily procured from existing inventory. Shots thus generated can then be paired with corresponding location views to obtain a training dataset. .

[0152] The ML model envisioned in this paper can also use a parametric statistical model.

[0153] For example, based on MRI, a series of baseline 3D positioning views of volunteers are acquired under different motion states within the respiratory cycle. By using image registration methods, patient-specific respiratory models can be derived from the baseline volunteer images, and generalized using other such volunteer studies. In this way, parametric statistical models can be obtained.

[0154] Now for reference Figure 7 , ​ A flowchart illustrates a computer-implemented method for facilitating the collection of planning data, particularly for predicting scan frames based on localized view volumes. This planning data can be used to control the acquisition of projection data in tomographic imaging.

[0155] At step S610, input data is received. The input data may include a 3D localization view volume. The input localization view V0 includes a representation of the target anatomical feature or target of interest (OI). The localization view has been reconstructed based on low-dose projection data previously collected by the imaging device during exploratory image acquisition. Therefore, the input localization view is based on measurements from the collected low-dose projection data. When collecting low-dose projection data, the field of view (FOV) is large (not compact) and covers anatomical structures wider than the target anatomical feature (TAF). In particular, the FOV is not compact because the target anatomical feature (TAF) may be subject to motion. In extreme cases, the localization view V0 may be based on a whole-body scan or on a scan of a wider anatomical portion (e.g., the entire torso), even if the target anatomical feature (TAF) is only one of, for example, lung, heart, or abdominal structures.

[0156] Typically, the localization view covers a large field of view from the neck to the pelvic bones. This localization view is a static image acquired before the actual diagnostic procedure. Just before acquiring the localization view, the patient is asked to hold their breath. Therefore, the respiratory status of the 3D localization view is known, and this information is used for the following processing.

[0157] In some embodiments, the input data is multi-channel and includes, in addition to the input positioning view V0, video camera lenses collected from the external portion of the patient (referred to herein as surrogate features SF). The camera used to record the lenses is based on non-ionizing radiation, such as an optical camera, IR, NIR, or LIDAR. A depth-sensing camera (e.g., RGB-D) can be used to collect the depth distribution of the surrogate features SF over time. The surrogate features SF undergo motion that may be associated with the internal target anatomical feature TAF. Therefore, the video lens records such motion of the surrogate features SF over time. The surrogate features SF may include, for example, a patient surface (e.g., skin) at least partially surrounding the TAF of interest. In the case of periodic motion, the length of the lens over time ensures that all expected cycles of motion of the target anatomical feature TAF / surrogate features SF are recorded. However, even for non-periodic motion, the lens should be recorded for a sufficiently long period to obtain better robustness, thereby ensuring that sufficient motion information is collected. The exact length of the lens will depend on the available resources. The camera can be mounted in the scanning room, typically at a position where the camera can observe the patient's PS and the patient. The lens can record part or all of the patient over time.

[0158] At step S620, the input data is preferably processed based on a motion model M to calculate one or more virtual predicted positioning view images representing the target anatomical structure in motion states different from those represented by the input positioning view V0. As described above, it is technically possible to output a single volume representing the extreme motion state with the farthest spatial reach compared to the motion state of the target anatomical feature TAF captured by the input positioning view. However, preferably, multiple virtual positioning views V0 are calculated, including those representing the initial motion state according to the initial positioning view V0 and the extreme positioning view V0. j A virtual positioning view of one or more intermediate motion states between extreme motion states.

[0159] If the dual-channel method is used as described above, step S620 may include video footage m captured from a camera with alternative feature SF. t Extracting (e.g., in the form of a vector, matrix, or tensor) one or more motion components. These extracted motion components can be applied to an initial volume V0 to, for example, perform translation, rotation, deformation (e.g., deformation or compression during cardiac motion) as needed, thereby simulating different motion states and obtaining different predicted virtual volumes V. j Therefore, this embodiment is an example of explicit motion modeling. Such an embodiment may include a motion model setup phase, including calibration of the geometric / positioning view V0 regarding the lens and imager IA. Model preparation may also include registration, segmentation, etc.

[0160] In some embodiments, a single-channel method is used, wherein only the positioning view input image V0 is fed into the model to predict one or more virtual positioning views V0 representing multiple different motion states different from the motion state in the initial volume. j In this single-channel approach, implicit modeling, such as machine learning models, can be used. The model can be used to predict the virtual positioning view V. j Generative models of sequences. Modeling can be based on neural networks, particularly CNNs. Recursive models, such as LSTMs or other models, can be used. In this case, model preparation involves using machine learning algorithms (ML) to tune the parameters of the ML model based on the training data.

[0161] Use the input positioning view V0 and the surface lens m t The dual-channel approach can also be used with machine learning models as described above.

[0162] The calculation of the virtual positioning view can be completed sequentially, or the calculation of the virtual positioning view can be timed using the timing of camera capture. Therefore, in video frame m t Calculate the corresponding virtual positioning view at some or every time t. Therefore, there may be some delay until all virtual positioning views V are calculated within a sufficient timeframe. j So far. This is the virtual positioning view V. j The per-frame computation can be used for explicit or implicit modeling (ML-based modeling). Information from more than one video frame can be used to compute any given virtual positioning view V. j .

[0163] In an optional step (not shown), the motion of the alternative features extracted from the lens is compared with a motion model. Deviations between the model predictions and the actual recorded lens can be observed in the validation step. If such deviations exceed a threshold, the fact is flagged as an anomaly, as this indicates unintended patient movement. An alert can be issued to the user. The transducer (TR) can be used to issue alerts (e.g., visual, tactile, auditory, or any other). Therefore, lens collection can continue during image acquisition for diagnostic scans as a motion monitoring measure for unintended patient movements that may cause motion artifacts.

[0164] At step S630, the predicted virtual positioning view V is then output. j And make it available for further processing.

[0165] For example, in step S640, the positioning view V is based on one or more projections. jTo calculate the scan frame. At the tight boundary surface of any given geometry or irregular shape, for example, forming a convex structure to include the positioning view V as predicted. j The text represents all the different parts occupied by the anatomical structures of the target of interest.

[0166] The scan frame can be calculated automatically in this way. The scan frame can then be further processed (e.g., stored), or it can be used to control diagnostic acquisition at step S650.

[0167] The scan frame calculated in this paper is a patient-specific scan frame that takes into account relative patient movement. In addition to the scan frame that has already been manually defined by the technician at the console, the calculated scan frame can also be displayed.

[0168] In another embodiment, at step S660, the predicted virtual positioning view V is... j The animation is displayed sequentially. Alternatively or additionally, after or before playing back such an animation, a virtual positioning view V is displayed (optionally in combination with the entered initial positioning view V0 as needed). j The selected one or more virtual positioning views in the image. Based on the virtual positioning view V j Displayed as one or more static or time-lapse animations (videos), the user can then use the graphical user interface or any other user interface at step S670 to adjust the displayed positioning views V0, V... j Mark the scan frame.

[0169] At step S660, the virtual positioning view V j The animation can replay moving images showing the transitions between different motion states of the target anatomical feature (TAF). The user can then easily identify the extreme motion state with the furthest positional deviation compared to the motion states recorded in the initial input image V0, and can then use this extreme motion state to define tight scan box boundaries in one or more spatial views. The boundary surfaces of the scan boxes can be tangentially selected from multiple views to position view V according to the extreme motion states of the target anatomical feature. j The * indicates that the content is in close contact with the surface of the target anatomical feature. This then defines the scan box in a tight manner to ensure that it includes all motion states, but excludes any irrelevant voxels that the target anatomical feature TAF never accesses. This allows for dose savings and allows for good image quality.

[0170] The scan box can be defined one after another in different views in the X, Y, and Z directions, and then these definitions can be combined to ensure that motion in different (e.g., all) spatial dimensions is taken into account.

[0171] Therefore, steps S660 and S670 represent defining the scan frame based on user input, while in the fully automatic embodiment, step S640 is based on the virtual positioning view V. j The scan box can be calculated without such a display. Segmentation results of various in-image representations m(TAF) for target anatomical features can be used in manual and / or automated implementations.

[0172] In step S640, even if the scan frame is automatically calculated, the virtual positioning view V is... j It can be displayed as additional information, either when stationary or as part of an animation. The visualization of the calculated scan frame can be displayed separately, or preferably in conjunction with the initial positioning view V0 or the virtual positioning view V. j Combine the visualization of the scan frame. For example, the scan frame can be visualized on the initial positioning view V0 or on the virtual positioning view V. j On any one or more virtual positioning views (especially in extreme virtual positioning views V) j The overlay or 3D rendering result (above). Various visualization and display methods are preferably used in conjunction with the user-input-based embodiments according to steps S660 and S670. However, this can also be used based on the predicted virtual positioning view V. j An automated implementation of calculating the scan frame.

[0173] Now for reference ​ , ​ A flowchart illustrating an example of machine learning-based modeling (implicit motion modeling) is shown. In such a model, motion information is learned implicitly from patterns in the training data, rather than through explicit analytical modeling.

[0174] In step S710, training data is obtained.

[0175] In step S720, a machine learning model is adapted based on the training data, and in step S730, the model thus trained is made available for deployment or testing, for example, in clinical practice. The input positioning view V0 encountered during deployment, as well as optional shots, differs from the training data.

[0176] Obtaining S710 training data can be done synthetically by a generative machine learning model, or it can be done by querying existing medical data records. The latter can include querying records that are large enough to include corresponding localization views of the anatomical features of the target of interest. The patient's history may also include footage of suitable alternative features of the patient that are associated with the movement of the target anatomical features.

[0177] In another embodiment, obtaining training data may include mounting a surface camera within a aperture of a tomographic imaging apparatus to acquire corresponding surface images of, for example, a patient's chest or any other body part while acquiring projection data of a positioning view image V0.

[0178] The collected footage can then be correlated with different instances of the patient-specific view image V0.

[0179] In another embodiment, localization view images from different patients are collected under the assumption that some or each patient represents a different motion state. Then, any one of the localization views collected in this way can be used as training input, wherein the remaining one, two or more localization views represent a target sequence, and the target sequence can then be used to train a generative model (preferably a generative model that can be modulated based on the input localization view V0, e.g., cGAN).

[0180] Generative ML models include preferably tunable GANS, such as cGAN, VAE, etc.

[0181] You can use a supervised training scheme with labeled data, or an unsupervised or self-supervised training scheme.

[0182] Any of the systems FS and methods described above can be equivalently applied to other tomographic imaging modalities where raw tomographic data (e.g., projection data) is required or acquired, such as in MRI, SPECT / PET, or other nuclear imaging modalities. Similar to the CT modality, in addition to the CT modality, a first low-IQ scan can be used, based on which imaging planning data (e.g., scan frame) can be derived, and then this imaging plan can be used to control a second, higher-IQ scan. The low-IQ scan can use a larger volume / area / FOV than that used for the subsequent (higher) IQ scan. For example, in MRI, compared to the case in a diagnostic scan, the localization view V0 can correspond to a faster scan with lower resolution, thicker slices, lower signal-to-noise ratio, but a larger FOV, in order to obtain the overview needed to plan the diagnostic scan.

[0183] The components of the imager promoter system FS can be implemented as one or more software modules that run on one or more general-purpose processing units (PUs) (e.g., workstations associated with the imager promoter system FS) or on a server computer associated with a group of imagers.

[0184] Alternatively, some or all of the components of the promoter system FS can be arranged in hardware, such as a suitably programmed microcontroller or microprocessor, such as an FPGA (Field Programmable Gate Array), or as a hard-wired IC chip, application-specific integrated circuit (ASIC), integrated into the imaging system IA. In another embodiment, the promoter system FS can be implemented partly in software and partly in hardware.

[0185] Different components of the facilitator system FS can be implemented on a single data processing unit (PU). Alternatively, some or more components can be implemented on different processing units (PUs), which may be remotely deployed in a distributed architecture and connected to a suitable communication network (e.g., in a cloud setup or a client-server setup).

[0186] One or more features described herein can be configured or implemented as, or implemented using, circuitry encoded in a computer-readable medium, or combinations thereof. The circuitry may include discrete circuitry and / or integrated circuits, system-on-a-chip (SoC) and combinations thereof, machines, computer systems, processors and memories, and computer programs.

[0187] In another exemplary embodiment of the present invention, a computer program or computer program unit is provided, characterized in that it is adapted to perform the method steps of the method according to one of the foregoing embodiments on a suitable system.

[0188] Therefore, a computer program unit may be stored on a computer unit, which may also be part of an embodiment of the present invention. The computing unit may be adapted to perform or cause the execution of the steps of the described methods. Furthermore, the computing unit may be adapted to operate components of the described apparatus. The computing unit is capable of automatically operating and / or executing user commands. The computer program may be loaded into the working memory of a data processor. Therefore, a data processor may be configured to execute the methods of the present invention.

[0189] This exemplary embodiment of the invention covers both computer programs that use the invention from the outset and computer programs that convert existing programs into programs that use the invention by means of an update date.

[0190] In addition, the computer program unit may be able to provide all the necessary steps to complete the process of the exemplary embodiments of the method described above.

[0191] According to another exemplary embodiment of the invention, a computer-readable medium, such as a CD-ROM, is proposed, wherein the computer-readable medium has computer program units stored thereon as described in the preceding section.

[0192] Computer programs can be stored and / or distributed on suitable media (in particular, but not necessarily non-transient media), such as optical storage media or solid-state media supplied together with or as part of other hardware, but can also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.

[0193] However, computer programs can also be presented via networks like the World Wide Web and can be downloaded from such networks into the working memory of a data processor. According to another exemplary embodiment of the invention, a medium is provided for making a computer program unit available for download, the computer program unit being arranged to perform a method according to one of the embodiments previously described according to the invention.

[0194] It should be noted that embodiments of the invention have been described with reference to different subjects. In particular, some embodiments are described with reference to method claims, while other embodiments are described with reference to apparatus claims. However, those skilled in the art will understand from the above and following description that, unless otherwise stated, any combination of features related to different subjects, in addition to any combination of features belonging to one type of subject, is also considered to utilize the disclosure of this application. However, it is possible to combine all features to provide a synergistic effect greater than the sum of the features.

[0195] While the invention has been described and illustrated in detail in the accompanying drawings and the foregoing description, such description is to be considered illustrative or exemplary rather than restrictive. The invention is not limited to the disclosed embodiments. Other variations of the disclosed embodiments will be understood and implemented by those skilled in the art in practicing the claimed invention upon studying the drawings, the disclosure, and the appended claims.

[0196] In the claims, the word "comprising" does not exclude other elements or steps, and the words "a" or "an" do not exclude multiple. A single processor or other unit can perform the functions of several items recited in the claims. The mere fact that certain measures are recited only in dissimilar appended claims does not indicate that combinations of these measures cannot be advantageously used. Any reference numerals / characters used in the claims should not be construed as limiting the scope.

Claims

1. A system (FS) for facilitating the planning of raw data acquisition performed by a computed tomography (IA) imaging apparatus, comprising: The input interface (IN) receives a 3D positioning view image (V0) of a movable target (OI, TAF) occupying a first portion of 3D space. This positioning view image (V0) is previously acquired by a tomographic imaging device based on raw data acquired at a lower quality than expected for planned raw data acquisition. The positioning view image represents the first motion state of the target. Predictor component (PC) capable of predicting one or more predicted 3D positioning view images (V0) based at least on the input positioning view image (V0). j (j>0), the one or more predicted 3D positioning view images represent: a target in one or more other predicted motion states, and a target occupying one or more other portions of the space in the one or more predicted motion states, and The output interface (OUT) is capable of providing at least one or more predicted positioning view images (V). j This is used for the acquisition of raw data to facilitate the planning process.

2. The system according to claim 1, comprising: The planner module (PM) is capable of predicting one or more 3D positioning view images (V) based on the predicted images. j ) to calculate planning data.

3. The system according to claim 1 or 2, wherein, The planning data includes the specifications of the target portion (SB) of the 3D space, the target portion including one or more other portions of the space, and wherein the target portion of the space is a subset of the input positioning view image (V0).

4. The system according to any one of the preceding claims, the system comprising: A graphics display generator (GDG) capable of generating graphic objects (GDs) for display on a display device (DD), wherein the generated graphic objects (GDs) are based on one or more predicted 3D positioning view images (V). j Visualize one or more motion states of the target among its different motion states.

5. The system according to claim 4, wherein, i) The graphical display (GD) includes a video animation capable of representing the target occupying different portions of space as it transitions between the different motion states, and / or ii) wherein the system includes a user interface (UI) capable of allowing a user to specify the portion of the target in space based on the graphical display (GD).

6. The system according to any one of the preceding claims, wherein, The predictor component (PC) is capable of calculating one or more predicted 3D positioning view images based on i) additional input data and / or ii) a motion model (M) of the target motion, wherein the additional input data includes video footage of surrogate features (SF) associated with the target acquired by a non-ionizing radiation-based camera (DSC).

7. The system according to claim 6, wherein, The motion model (M) is a trained machine learning model trained on training data.

8. The system according to claim 6, wherein, The motion model (M) represents at least one motion component of the motion that can be extracted from the video shot, and wherein the predictor component (PC) applies the extracted motion component to the positioning view image (V0) to extrapolate the predicted 3D positioning view image (V0). j ).

9. The system according to any one of the preceding claims, comprising: A transducer (TR) is capable of issuing an alarm signal in the event of a deviation between the motion state so imaged by the video lens of the camera based on non-ionizing radiation and the model, wherein the deviation violates a predefined acceptableness condition.

10. An imaging arrangement (MAR) comprising: The system according to any one or more of the preceding claims, and any one or more of the following: a tomographic imager, the camera, and the display device.

11. A method for facilitating the computational implementation of planning for raw data acquisition performed by a tomographic imaging apparatus, comprising: Receive (S610) a 3D positioning view image (V0) of a movable target (OI) occupying a first part of 3D space, the positioning view image (V0) being previously obtained by a tomographic imaging device or the tomographic imaging device based on raw data acquired at a lower quality than expected for planned raw data acquisition, and the positioning view image representing a first motion state of the target motion. Predict (S620) one or more predicted 3D positioning view images (V0) based at least on the input positioning view image (V0). j (j>0), the one or more predicted 3D positioning view images represent: a target in one or more other predicted motion states, and a target occupying one or more other portions of the space in the one or more predicted motion states, and At least one or more predicted positioning view images (V) as described in (S630) are provided. j This is used for the acquisition of raw data to facilitate the planning process.

12. A method for computationally implementing training a machine learning model based on training data for planning raw data acquisition by a computed tomographic imaging apparatus (IA), said model, once trained, being able to predict one or more such predicted positioning view images for a target in one or more different motion states, given a positioning view image of a movable target in a first motion state.

13. A method for acquiring training data for training a machine learning model based on the training data to be acquired, the machine learning model being used for planning raw data acquisition by a tomographic imaging apparatus (IA), the model, after being trained, being able to predict one or more such predicted positioning view images for a target in one or more different motion states, given a positioning view image of a movable target in a first motion state.

14. A computer program unit, when executed by at least one data processing system, is adapted to cause the data processing system (FS) to perform the method according to any one of claims 11, 12 or 13.

15. At least one computer-readable medium (MEM) having stored thereon a program unit as claimed in claim 14, or having stored thereon a model as claimed in any one of claims 6 to 8.

Images