Computer-implemented method for medical imaging

Abstract

The invention provides a computer-implemented method for medical imaging, the method comprising: obtaining X-ray image data from one or more X-ray images acquired by an imaging system comprising an X-ray source, an X-ray detector, and a collimation shutter arranged between the X-ray source and the X-ray detector and configured to collimate X-ray radiation, wherein the X-ray image data is obtained using initial imaging settings, the initial imaging settings comprising initial imaging position settings and initial shutter settings, wherein the initial imaging position settings describe the position of the X-ray source and the X-ray detector in space, wherein the initial shutter settings describe the configuration of the collimation shutter, wherein the X-ray image data comprises collimated image data for a first detector area of the X-ray detector, the first detector area corresponding to an initial field of view (initial FOV) defined by an imaging geometry of the imaging system including the configuration of the collimation shutter, and wherein the X-ray image data comprises non-collimated image data for a second detector area outside of the first detector area, the non-collimated image data resulting from out of focus (OOF) radiation; processing the non-collimated image data to obtain extracted image information; and outputting data based on the extracted image information.

Description

[0001] COMPUTER-IMPLEMENTED METHOD FOR MEDICAL IMAGING

[0002] FIELD OF THE INVENTION

[0003] The present invention relates to a computer-implemented method for medical imaging, a corresponding computer program product and computer-readable medium, and system.

[0004] TECHNICAL BACKGROUND

[0005] Medical imaging may involve collimating radiation from the radiation source. Collimation, while being advantageous in reducing radiation dose by only irradiating small portions, poses some challenges and drawbacks, such as limiting the size of the field of view and difficulties arising therefrom, e.g. for object detection and FOV tracking, and / or artifacts, e.g. due to shutter penumbra.

[0006] An object of the present invention is to provide a method for medical imaging allowing for improved results in collimated imaging scenarios.

[0007] The present invention can be used for medical imaging, e.g. in connection with a Loop-X imaging system, or navigation in medical procedures and / or together with systems for image-guided radiotherapy such as VERO® and ExacTrac®, both products of Brainlab AG.

[0008] Aspects of the present invention, examples and exemplary steps and their embodiments are disclosed in the following. Different exemplary features of the invention can be combined in accordance with the invention wherever technically expedient and feasible.

[0009] EXEMPLARY SHORT DESCRIPTION OF THE INVENTION

[0010] In the following, a short description of the specific features of the present invention is given which shall not be understood to limit the invention only to the features or a combination of the features described in this section.

[0011] The invention provides a computer-implemented method for medical imaging, the method comprising: obtaining X-ray image data from one or more X-ray images acquired by an imaging system comprising an X-ray source, an X-ray detector, and a collimation shutter arranged between the X-ray source and the X-ray detector and configured to collimate X-ray radiation, wherein the X-ray image data is obtained using initial imaging settings, the initial imaging settings comprising initial imaging position settings and initial shutter settings, wherein the initial imaging position settings describe the position of the X-ray source and the X-ray detector in space, wherein the initial shutter settings describe the configuration of the collimation shutter, wherein the X-ray image data comprises collimated image data for a first detector area of the X-ray detector, the first detector area corresponding to an initial field of view, initial FOV, defined by an imaging geometry of the imaging system including the configuration of the collimation shutter, and wherein the X-ray image data comprises non-collimated image data for a second detector area outside of the first detector area, the non-collimated image data resulting from out of focus, OOF, radiation; processing the non-collimated image data to obtain extracted image information; and outputting data based on the extracted image information.

[0012] GENERAL DESCRIPTION OF THE INVENTION

[0013] In this section, a description of the general features of the present invention is given for example by referring to possible embodiments of the invention.

[0014] The invention provides a computer-implemented method for medical imaging, a corresponding computer program product and computer-readable medium, and a system according to the independent claims. Preferred embodiments are laid down in the dependent claims.

[0015] The invention provides a computer-implemented method for medical imaging, the method comprising obtaining X-ray image data from one or more X-ray images acquired by an imaging system comprising an X-ray source, an X-ray detector, and a collimation shutter arranged between the X-ray source and the X-ray detector and configured to collimate X-ray radiation. The X-ray image data is obtained using initial imaging settings, the initial imaging settings comprising initial imaging position settings and initial shutter settings. The initial imaging position settings describe the position of the X-ray source and the X-ray detector in space. The initial shutter settings describe the configuration of the collimation shutter. The X-ray image data comprises collimated image data for a first detector area of the X-ray detector, the first detector area corresponding to an initial field of view, initial FOV, defined by an imaging geometry of the imaging system including the configuration of the collimation shutter. The X-ray image data comprises non-collimated image data for a second detector area outside of the first detector area, the non-collimated image data resulting from out of focus, OOF, radiation. The method further comprises processing the non-collimated image data to obtain extracted image information and outputting data based on the extracted image information, particularly the outputted data obtained by means of processing the extracted image information using an image processing technique.

[0016] It will be understood from the above, that the initial imaging settings define the initial FOV. Updating the imaging settings may thus lead to an updated FOV, as will be discussed in more detail below in the context of some of the use cases of the present method.

[0017] The shutter settings of the present disclosure may comprise a position of the shutter in space, e.g., relative to the detector, and a setting of the shutter jaws, or, in other words, the opening of the collimation shutter. The latter influences the size of the FOV.

[0018] The first detector area, hereinbelow, is also referred to as collimated (detector or image) area and the second detector area as non-collimated (detector or image) area.

[0019] Known methods strive to reduce OOF radiation and any image data from residual OOF radiation will be discarded. The method of the present disclosure, however, makes use of said OOF data instead. Image information is extracted from said OOF radiation. This information can be advantageously employed for different purposes that will be explained in detail below, such as for tracking medical instruments, FOV adjustment, or creating extended-view images without increasing the actual FOV. The method of the present disclosure allows for some use cases that are not possible with known methods. For other use cases, known methods may need to increase the actual FOV or otherwise work on a trial-and-error basis, which is not necessary when employing the method of the present disclosure, and thereby allows for reducing the overall dose.

[0020] According to the present disclosure, the method may comprise using the extracted image information for determining the outputted data. This may particularly comprise processing the extracted image information by means of an image processing technique, as explained above.

[0021] According to the present disclosure, the outputted data may comprise control data for a device, particularly a medical navigation device, a robot, and / or a medical device, e.g. an X-ray imaging device, particularly comprising the X-ray source and X-ray detector.

[0022] These use cases particularly profit from the improvements brought about by the method outlined above. Particularly, the outputted data may comprise control data for the X-ray imaging device, particularly, for controlling positioning of the X-ray imaging device or components thereof, such as the source, detector, and / or collimation shutter, and / or controlling imaging operation. The use of non-collimated image data, as will be seen further below, allows for a more targeted selection of position and settings with respect to objects or regions of interest, which in turn allows for low overall X-ray dosage, e.g., where known methods would have required a larger FOV.

[0023] According to the present disclosure, the extracted image information may comprise information on an object that is arranged at least partially between the X-ray source and the second detector area, and processing the non-collimated image data may comprise detecting the object. The object may be at least one of: a medical device, such as a guidewire, an anatomical structure, particularly of a subject of the medical imaging and / or of a person other than the subject of the medical imaging, an implant, a contrast agent, particularly a contrast bolus.

[0024] That is, the object being arranged between the X-ray source and the second detector area may be understood as the object being projected into the second detector area (non-collimated area). This means that the object is arranged in such a manner that it is not or not entirely projected onto the initial FOV. Using the non-collimated image data allows for detecting the object or parts thereof even if they are not projected onto the initial FOV. This is advantageous because the object might be detected in cases where this would not be possible with known methods.

[0025] As mentioned above, the object may be a medical device, such as a guidewire. It is often important to depict at least part of the object in the medical image. However, the initial FOV may not or not to a sufficient degree depict the medical device. Other methods might require increasing the FOV size or otherwise searching for the medical device by repeated image acquisitions. The method of the present disclosure overcomes this and allow for finding the medical device with reduced trial and error and reduced overall dose. As mentioned above, the object may be an anatomical structure, such as a subject of the medical imaging and / or a person other than the subject. In the former case, anatomy surrounding the anatomy that is projected onto the field of view may provide valuable information on anatomical features, orientation and position thereof. In the latter case, it may be of interest to detect when part of another person is in the vicinity of, but not yet visible in, the initial FOV. Since this person should not be exposed to radiation and also might obstruct the actual subject of the radiation, this allows for issuing a warning to allow for avoiding the object to enter into the initial FOV.

[0026] Where the object is an implant, this may be of similar interest as anatomical features of the subject of the medical imaging or medical devices. Accordingly, similar considerations apply here.

[0027] As mentioned above, the object may also be a contrast agent, particularly a contrast bolus. For the purpose of not using too much contrast agent and nonetheless taking contrast medical images at the time when the contrast agent is at the correct position, contrast bolus tracking may be employed. While a bigger field of view might help with tracking the contrast bolus, the method of the present disclosure allows for reducing the dose by using a smaller FOV while still allowing for accurate contrast bolus tracking.

[0028] According to the present disclosure, the method, particularly the processing the non-collimated image data to obtain extracted image information, may further comprise determining out-of-focus object detection data based on the non-collimated image data, and determining object characterization data describing the object’s position and / or the object’s shape in the X-ray image data based on the out-of-focus object detection data, particularly based only on the out-of-focus object detection data.

[0029] According to the present disclosure, out-of-focus object detection data may be image data acquired in the second detector area. Object detection data may comprise at least one of a) data indicative of whether the object is there or is not there, b) a coordinate describing the position of at least one part of the object, c) a bounding box enclosing the object or the like, d) a mask containing pixel positions related to significantly all parts of the object, e) any of point a) to d) in combination with image data acquired in the second detector area. As an example, out-of-focus object detection data may comprise at least part of a projection of the object onto the second detector area. The out-of-focus object detection data may be part of or can be derived from the non-collimated image data.

[0030] According to the present disclosure, object characterization data may be any data describing the object’s position and / or the object’s shape in the X-ray image data, for example comprising positions in a coordinate system and / or an envelope of the object. The object’s position and shape may be considered to be characteristics of the object.

[0031] As mentioned above, the object characterization data may be determined based on the out-of- focus object detection data. In one example, the object characterization data may be determined based only on the out-of-focus object detection data. In other words, no collimated image data may be used for determining the object characterization data. Or, in yet other words, the object might be found only in the non-collimated image data.

[0032] It will become apparent from the above how the method may advantageously be used to leverage non-collimated image data in the context of objects not or only partially depicted in the initial FOV. Known methods would not be able to provide object characteristics without moving or increasing the FOV.

[0033] The above may, in particular, be used for tracking the object and potentially to catch up with an object, particularly also a moving object, for example where the object comprises a guidewire, a contrast bolus, or the like.

[0034] According to the present disclosure, the method, particularly the processing the non-collimated image data to obtain extracted image information, may further comprise determining collimated object detection data based on the collimated image data, and determining whether the collimated object detection data indicates that the object is not present in the collimated image data. In case it is determined that the collimated object detection data indicates that the object is not present in the collimated image data, the method may comprise determining out-of-focus object detection data based on the non-collimated image data, and determining object characterization data describing the object’s position and / or the object’s shape based on the out- of-focus object detection data, particularly based only on the out-of-focus object detection data.

[0035] In other words, it is checked to see whether an object of interest is found in the collimated image area and, if the object of interest is not found within the collimated image area, the non-collimated image information is to be analyzed with similar means as mentioned above to characterize the object. In yet other words, it may be determined that the object is found only in the non-collimated image data and said non-collimated data may be used to characterize the object.

[0036] According to the present disclosure, collimated object detection data may be image data acquired in the first detector area. Object detection data may comprise at least one of a) data indicative of whether the object is there or is not there, b) a coordinate describing the position of at least one part of the object, c) a bounding box enclosing the object or the like, d) a mask containing pixel positions related to significantly all parts of the object, e) any of point a) to d) in combination with image data acquired in the first detector area. As an example, collimated object detection data may comprise at least part of a projection of the object onto the first detector area. The collimated object detection data may be part of or can be derived from the collimated image data.

[0037] A determination may be carried out whether the collimated object detection data indicates that the object is not present in the collimated image data, in other words, whether there is a projection of the object in the collimated area. In case it is determined that the collimated object detection data indicates that the object is not present in the collimated image data, or, in other words, if there is no data representative of the object present in the collimated image data, then the method may comprise determining out-of-focus object detection data based on the non- collimated image data, and determining object characterization data describing the object’s position and / or the object’s shape based on the out-of-focus object detection data, particularly based only on the out-of-focus object detection data. This may be done in the manner described further above in detail.

[0038] It will become apparent from the above how the method may advantageously be used to leverage non-collimated image data in the context of objects that cannot be found in the initial FOV. Known methods would not be able to provide object characteristics without moving or increasing the FOV. The above may, in particular, be used for tracking the object and potentially to catch up with an object, particularly also a moving object, for example where the object comprises a guidewire, a contrast bolus, or the like.

[0039] According to the present disclosure, the method, particularly the processing the non-collimated image data to obtain extracted image information, may comprise determining collimated object detection data based on the collimated image data, determining out-of-focus object detection data based on the non-collimated image data, and determining object characterization data describing the object’s position and / or the object’s shape in the X-ray image data based on the collimated object detection data and the out-of-focus object detection data.

[0040] Reference is made to the terminology as explained above in terms of object detection data and object characterization data. In the present aspect, the object is found in both, the collimated image data and the non-collimated image data, and both are used to characterize the object. This is particularly advantageous because higher quality information from the collimated image data can be supplemented with the slightly lower quality information from the non-collimated image data to extend the FOV and the processing of the information from the non-collimated data can be supported by the collimated data, e.g. in case the object extends from the FOV outwards.

[0041] According to the present disclosure, the method may further comprise determining, based on the object characterization data, updated imaging settings describing an updated FOV, wherein, when a new X-ray image is acquired using the updated imaging settings, the object is depicted at least partially in the updated FOV. For example, where the initial FOV does not depict the object, the updated FOV may depict at least part of the object.

[0042] Alternatively, the method may further comprise determining, based on the object characterization data, updated imaging settings describing an updated FOV, wherein, when a new X-ray image is acquired using the updated imaging settings, at least part of, in particular all of the object, is not depicted in the updated FOV. For example, the FOV may be updated to no longer depict the object, which is particularly advantageous where an object should or need not be exposed to radiation and / or where an object obstructs the view.

[0043] Alternatively, the method may further comprise determining, based on the object characterization data, updated imaging settings describing an updated FOV, wherein, when a new X-ray image is acquired using the updated imaging settings, a larger or a smaller portion of the object is depicted at least partially in the updated FOV than the initial FOV. This may be employed similarly as the two alternatives described above, except that the object, after updating the FOV, need not be completely inside or outside the updated field of view, but the portion that is depicted is increased or decreased, respectively in the updated field of view.

[0044] The updated imaging settings may comprise updated imaging position settings and / or updated shutter settings and / or updated X-ray exposure settings, such as dosage or energy.

[0045] In terms of the exposure settings, these are settings to be used for automatic exposure control, for example. The updating of the exposure settings may employ the method, as it allows for better determination of patient characteristics, such as body diameter, and prediction of adequate exposure settings.

[0046] According to the present disclosure, the updating of the exposure settings may be based on the non-collimated image data, as this e.g. allows to determine the body diameter of a patient being imaged, and hence a prediction or determination of the exposure settings. In other words, the non-collimated image data may provide information about the e.g. body diameter, shape, etc.

[0047] The updated imaging position settings describe the position of the X-ray source and the X-ray detector in space.

[0048] The updated shutter settings may describe the configuration of the collimation shutter. This may comprise where the shutter is located in detector space and / or the shutter opening.

[0049] Additionally or alternatively, the shutter settings may be described based on the mechanical possible extreme positions, e.g. based on an encoder, e.g. in combination with a mechanical or geometric description of the shutter configuration.

[0050] Additionally or alternatively, the shutter settings may be described in relation to a plane that is rotating with the source arm and is also normal to the line from the source’s focal spot to the mechanical isocenter, i.e., the rotation center of source and / or detector.

[0051] As will be understood from the above, the imaging system can be adjusted to the present need based on the object characterization data and, thereby, based at least in part on the non- collimated image data from which the object characterization data was obtained.

[0052] According to the present disclosure, the method may comprise determining object envelope data describing a geometric area in the X-ray image data representing at least parts of the object, in particular a region of interest of the object, based on the object characterization data.

[0053] Any suitable method for determination of object envelope data from X-ray image data may be employed.

[0054] The method may further comprise determining updated imaging settings describing an updated FOV based on the object envelope data, wherein the updated FOV contains the geometric area described by the object envelope data, in particular such that, when a new X-ray image is acquired using the updated imaging settings, the object is depicted at least partially in the updated FOV. For example, where the initial FOV does not depict the object, the updated FOV may depict at least part of the object.

[0055] Alternatively, the method may further comprise determining updated imaging settings describing an updated FOV based on the object envelope data, wherein the updated FOV does not contain the geometric area described by the object envelope data, in particular such that, when a new X- ray image is acquired using the updated imaging settings, the object is at least partially not depicted in the updated FOV. For example, the FOV may be updated to no longer depict the object, which is particularly advantageous where an object should or need not be exposed to radiation and / or where an object obstructs the view.

[0056] Alternatively, the method may further comprise determining updated imaging settings describing an updated FOV based on the object envelope data, wherein the updated FOV contains a larger or a smaller portion of the geometric area described by the object envelope data than the initial FOV, in particular such that, when a new X-ray image is acquired using the updated imaging settings, a larger or a smaller portion of the object is depicted in the updated FOV than the initial FOV. This may be employed similarly as the two alternatives descried above, except that the object, after updating the FOV, need not be completely inside or outside the updated field of view, but the portion that is depicted is increased or decreased, respectively in the updated field of view.

[0057] The updated imaging settings may comprise updated imaging position settings and / or updated shutter settings and / or updated X-ray exposure settings, such as dosage or energy.

[0058] The updated imaging position settings may describe the position of the X-ray source and the X- ray detector in space.

[0059] The updated shutter settings may describe the configuration of the collimation shutter. This may comprise where the shutter is located in detector space and / or the shutter opening.

[0060] Additionally or alternatively, the shutter settings may be described based on the mechanical possible extreme positions, e.g. based on an encoder, e.g. in combination with a mechanical or geometric description of the shutter configuration.

[0061] Additionally or alternatively, the shutter settings may be described in relation to a plane that is rotating with the source arm and is also normal to the line from the source’s focal spot to the mechanical isocenter, i.e., the rotation center of source and / or detector.

[0062] As will be understood from the above, the imaging system can be adjusted to the present need based on the object characterization data and, thereby, based at least in part on the noncollimated image data from which the object characterization data was obtained.

[0063] In particular, a tracking mechanism as described in other parts of the present disclosure can be implemented making use of the above steps. As an example, based on the object detection, the determined object position on the detector’s full extent may be used to derive a, for example circular or polygonal, X-ray collimation that increases the likelihood of capturing the object of interest in the new collimated area of the detector in the next X-ray image(s).

[0064] According to the present disclosure, the method may comprise determining motion pattern data describing the object’s motion and / or shape changes. In particular, a motion pattern may comprise a translation of the object and / or rotation of the object and / or deformation of the object and / or dilution of the object. The method may further comprise determining updated imaging settings describing an updated field-of-view, updated FOV, based on the determined motion pattern data, in particular additionally based on the object characterization data.

[0065] That is, based on a motion pattern it may be possible to determine where the object or portion thereof will likely be and the FOV can be updated, for example in the manner described above, i.e., to include or not include the object or to increase or decrease the portion of the object included in the FOV.

[0066] In one particular example, a tracking of the object may be implemented leveraging the motion pattern data and a current position and / or shape of the object. The motion pattern, thus, allows for determining where the object is likely going to be, for example.

[0067] According to the present disclosure, determining the motion pattern data may comprise determining object characterization data comprising object position data based on multiple X-ray images taken at different points in time. For example, a motion pattern may then be extrapolated or otherwise calculated or modelled.

[0068] In particular the method may comprise determining the motion pattern data by acquiring two X- ray images at two different points in time, determining first object position data P_t1 from the image data at a first point in time t1 , determining second object position data P_t2 from the image data at a second point in time t2 which is after the first point in time t1 , and determining the motion pattern data based on deriving a motion pattern based on the first object position data P_t1 and the second object position data P_t2.

[0069] As an example for determining a motion pattern, an optical flow technique may be employed, i.e. the motion between two frames may be determined at every pixel position. Known optical flow techniques from the field of motion detection may be employed.

[0070] Alternatively to deriving the motion pattern from the X-ray images, motion patterns may also otherwise be captured, e.g., from optical camera images of the object.

[0071] According to the present disclosure, determining motion pattern data may comprise acquiring the X-ray image using a long exposure. Alternatively, determining motion pattern data may comprise using a model for the motion, wherein the model is based on an object type of the object and, additionally or alternatively, based on external measurement data describing the motion of the object. Length of the long exposure may depend on the application, and generally ranges from milliseconds to seconds. For example, the period of a heart beat or the period of a breath cycle may determine the length of the long exposure. It may also be derived from an expected distance that would be travelled at a certain speed. A long exposure image shows motion artifacts that can be used for determining a motion pattern.

[0072] According to the present disclosure, determining updated imaging settings may comprise determining a geometric center of gravity of the geometric area described by the envelope data and / or determining a size of the geometric area described by the envelope data, and determining updated imaging positing settings based on the geometric center of gravity and / or determining updated shutter settings based on the size of the geometric area, so as to obtain an updated FOV that substantially centers the geometric area containing the object on the detector.

[0073] That is, optionally, the overall X-ray imaging device can be moved either as alternative or in addition to the collimation shutter movements, specifically if the object of interest is found close to the detector borders.

[0074] According to the present disclosure, determining updated imaging settings may comprise determining the geometric center of gravity of the geometric area described by the envelope data, and / or determining a size of the geometric area described by the envelope data, determining updated shutter settings based on the size of the geometric area and based on the geometric center of gravity while maintaining the initial imaging positioning settings. This is advantageous where the object is located close to the detector center. In this case, it may be advantageous to avoid moving the entire imaging system.

[0075] In an example, determining the updated imaging settings may comprise determining whether the updated FOV meets predetermined criteria, in particular, whether the updated FOV is substantially centered around the geometric area, and otherwise additionally determining updated imaging position settings. That is, it may be determined whether the updated FOV is in a suitable position, such as close to the center of the detector and only if this is not the case the imaging position settings may be changed and otherwise only the shutter settings may be changed.

[0076] In other words, optionally, the overall X-ray imaging device can be moved either as alternative or in addition to the collimation shutter movements. Specifically movement of the imaging system may be avoided if the object is close to the center.

[0077] According to the present disclosure, the method may comprise artifact correction based on the non-collimated image data.

[0078] In particular, the artifact correction based on the non-collimated image data may comprise correction of collimator shutter penumbra artifacts. As an example, the extracted information from the non-collimated area may be used to correct jaw penumbra in the projection.

[0079] Alternatively or in addition, the artifact correction based on the non-collimated image data may comprise correction of truncation artifacts of volumetric reconstruction images based on the image data, in particular cone beam CTs. That is, the information from the non-collimated area may be used for estimation of image data in said area and using said information for use prior to filtering and 3D back projection to avoid truncation artifacts.

[0080] According to the present disclosure, the method may comprise generating an extended-view X- ray image based on the collimated image data and the extracted image information, the extended-view X-ray image depicting the initial or an updated FOV and areas outside of the initial or updated FOV, particularly the extended-view X-ray image being a full-detector-size image or larger-than-detector-size image.

[0081] Such an extend-view X-ray image may, in particular, be used for supporting tracking. In addition, it may be useful for image processing, e.g. by means of machine learning or for image registration techniques. It may also be useful when rendered for a user to obtain a better overview over the imaged subject.

[0082] It is noted that here the extended-view X-ray image, particularly, refers to an image that has the visual properties of an image that has actually been acquired with an extended FOV, particularly acquired with the full detector size, e.g., as if there had been no collimation. In other words, image data acquired with a given FOV is processed to output an image that looks like it was taken with a larger FOV.

[0083] It is noted that, while the acquired image data for the part of the image depicting an area outside of the initial or updated FOV may not be of the same quality as that within the initial or updated FOV, it nonetheless can be processed to generate extended-view X-ray images. As an example, in the process of generating the extended-view X-ray images, image enhancement techniques may be employed to improve image quality also outside of the initial or updated FOV. Alternatively or in addition, machine learning techniques, e.g. as outlined below, may be used for obtaining the extended-view X-ray images.

[0084] According to the present disclosure, the method may comprise the use of a machine learning technique, in particular a trained machine learning model, trained based on X-ray image pairs of a collimated X-ray image and a corresponding extended-view X-ray image, particularly full-size X- ray image.

[0085] As an example, the machine learning technique, particularly the trained machine learning model, may be used for predicting an extended-view X-ray image from a collimated X-ray image based only on the non-collimated image data or based on the collimated image data and the noncollimated image data of the collimated X-ray image. This is one of the ways in which the extended-view X-ray imaged described further above may be obtained.

[0086] As another example, the machine learning technique, particularly the trained machine learning model, may be used for predicting a collimated X-ray image including collimated image data and non-collimated image data from an extended-view X-ray image, particularly for use in training object detection algorithms and / or models. As another example, the machine learning technique, particularly the trained machine learning model, may be used for predicting, from the non-collimated image data of a collimated X-ray image, image data for the second detector area in a corresponding extended-view X-ray image. This is one of the ways in which the extended-view X-ray imaged described further above may be obtained.

[0087] As another example, the machine learning technique, particularly the trained machine learning model, may be used for predicting non-collimated image data of a collimated X-ray image from collimated image data of the collimated X-ray image.

[0088] As another example, the machine learning technique, particularly the trained machine learning model, may be used for correcting artifacts such as collimation shutter penumbra artifacts. Thus, an improved image can be provided.

[0089] As another example, the machine learning technique, particularly the trained machine learning model, may be used for generating collimated Digitally Reconstructed Radiographs, DRRs, from CT, the DRRs including predicted Out Of Focus, OOF, information.

[0090] According to the present disclosure, the method may comprise using the extracted image information, particularly the outputted data, e.g. control data, for visualization for object detection, particularly for navigation and / or tracking of the object and / or for image registration.

[0091] As explained above, different objects, such as medical instruments or a contrast bolus or anatomical structures may be detected and, for example, tracked. Moreover, objects may be used in navigation or image registration techniques. In each of these cases, it is advantageous to not be limited to information from the FOV.

[0092] According to the present disclosure, the method may comprise using the extracted image information, particularly the outputted data, e.g. control data, for imaging geometry adjustment, particularly adjustment of X-ray source and / or X-ray detector position and / or collimation shutter configuration, particularly for FOV tracking of an object, and / or for dose reduction by reducing FOV size. As explained above, for example, medical instruments or a contrast bolus may be tracked (or chased).

[0093] According to the present disclosure, the method may comprise using the extracted image information, particularly the outputted data, e.g. control data, for FOV extension. For example, as explained above, an extended-view X-ray image may be provided.

[0094] According to the present disclosure, the method may comprise using the extracted image information, particularly the outputted data, e.g. control data, for FOV tracking, detection and / or tracking of an object. Reference is made to the above explanation thereof.

[0095] According to the present disclosure, the method may comprise using the extracted image information, particularly the outputted data, e.g. control data, for artifact reduction, particularly correction of shutter penumbra and / or correction of truncation artifacts. Reference is made to the above explanation thereof.

[0096] As mentioned above, the present disclosure also provides a system comprising a processing device configured to carry out and / or control the method of the present disclosure, particularly as described above. Features and advantages outlined above in the context of the method similarly apply to the system.

[0097] According to the present disclosure, the system may further comprise an imaging system comprising an X-ray source, an X-ray detector, and a collimation shutter arranged between the X- ray source and the X-ray detector and configured to collimate X-ray radiation, the imaging system configured to acquire collimated X-ray images and optionally configured to acquire extended- view, in particular full-size, X-ray images.

[0098] According to the present disclosure, the X-ray source and / or the X-ray detector may be configured to be, e.g. robotically, movable based on control data, particularly at least in part based on control data obtained by the method of the present disclosure, particularly as described above.

[0099] According to the present disclosure, a robotic patient support may be provided that is configured to move the patient relative to the imaging system and / or the imaging system may be mounted on a robotic structure configured to move the imaging system relative to a patient support.

[0100] As an example, the imaging system may be mounted on wheels so as to be freely and independently movable with respect to a patient support.

[0101] The X-ray imaging system may comprise a CT (computed tomography) imaging system. In particular, the medical imaging system may comprise a CBCT (cone beam CT). Specifically, the imaging system may comprise an, e.g. 2D, X-ray scanner, which in particular, may be a cone beam computed tomography, CBCT, scanner.

[0102] The system may comprise a wheeled device, the wheeled device particularly being a non-rail- borne wheeled device. The X-ray scanner may be mounted on the wheeled device and the wheeled device may be configured to move the X-ray scanner. The movement, in particular a translational movement, of the viewing axis of the X-ray scanner may comprise moving the wheeled device relative to the patient table.

[0103] The wheeled device may comprise four independently steerable wheels, in particular with rear wheels having active drive, also referred to as traction.

[0104] The wheeled device may, for example, be an automated guided vehicle, AGV.

[0105] The imaging system may be an autonomously movable system. The imaging system may have a tiltable gantry or C-arm. As an example, the imaging system may be configured such that several degrees of freedom are possible, including one or more of C-arm tilt, gantry-tilt, C-arm-yaw, gantry-yaw, longitudinal translational movements of the gantry or C-arm. A trajectory of the gantry or C-arm combining two or more of these motions of the gantry or C-arm may be referred to as saddle trajectories.

[0106] According to the present disclosure, the imaging system may comprise a tiltable gantry or a tiltable C-arm, particularly a gantry configured such that its rotation plane is tiltable or a C-arm configured such that its rotation plane is tiltable.

[0107] An example for such an imaging system is the Loop-X, which is an X-ray imaging system. In general, medical imaging systems allow for rotation of the source and detector in a plane.

[0108] In addition, some imaging systems allow for a translational movement, e.g., in a direction perpendicular to the plane. According to the present disclosure, the gantry or C-arm may be configured and mounted such that the plane itself may be tilted. For example, the gantry may rotate around an axis that is parallel to the longitudinal axis of a patient support, such as a patient bed. The gantry, in addition, may describe a translational movement in a direction parallel to the longitudinal axis. The gantry may also tilt around a horizontal axis that is perpendicular to the longitudinal axis, e.g., towards and against the translational movement direction.

[0109] For example, a tilt rotation of the C-arm or gantry may be a rotation around hinges used for mounting the C-arm or gantry, e.g., mounted to a fixed or movable support structure like one or more feet, particularly a support structure movable on wheels.

[0110] According to the present disclosure, the X-ray imaging system may comprise a gantry or a C-arm, particularly a tiltable gantry or a tiltable C-arm as described above, and the X-ray imaging system may be configured such that the gantry or the C-arm are movable to describe a yaw rotation. For example, a yaw rotation may be achieved by a wheeled support structure to which the C-arm or gantry is mounted, wherein particularly the wheeled support structure may be driven via traction, e.g., using back wheels. As an example, all wheels might be set to a 45 ° so as to form a circle yaw rotation.

[0111] The present disclosure also provides a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to control and / or carry out any of the steps of the method according to the present disclosure, particularly of the method claims.

[0112] The present disclosure also provides a computer-readable medium comprising instructions which, when executed by a computer, cause the computer to control and / or carry out any of the steps of the method according to the present disclosure, particularly of the method claims.

[0113] The present disclosure also relates to the use of the method and / or system of the present disclosure for medical imaging.

[0114] For the sake of completeness, it is noted that for example the invention does not involve or in particular comprise or encompass an invasive step which would represent a substantial physical interference with the body requiring professional medical expertise to be carried out and entailing a substantial health risk even when carried out with the required professional care and expertise. For example, the invention does not comprise a step of positioning a medical implant in order to fasten it to an anatomical structure or a step of fastening the medical implant to the anatomical structure or a step of preparing the anatomical structure for having the medical implant fastened to it. More particularly, the invention does not involve or in particular comprise or encompass any surgical or therapeutic activity. The invention is instead directed as applicable to medical imaging, navigation, positioning and / or alignment. For this reason alone, no surgical or therapeutic activity and in particular no surgical or therapeutic step is necessitated or implied by carrying out the invention.

[0115] The features and advantages outlined above in the context of the method similarly apply to the system, use, computer program product and computer readable medium of the present disclosure.

[0116] DEFINITIONS

[0117] In this section, definitions for specific terminology used in this disclosure are offered which also form part of the present disclosure.

[0118] Computer implemented method

[0119] The method in accordance with the invention is for example a computer implemented method. For example, all the steps or merely some of the steps (i.e. less than the total number of steps) of the method in accordance with the invention can be executed by a computer (for example, at least one computer). An embodiment of the computer implemented method is a use of the computer for performing a data processing method. An embodiment of the computer implemented method is a method concerning the operation of the computer such that the computer is operated to perform one, more or all steps of the method.

[0120] The computer for example comprises at least one processor and for example at least one memory in order to (technically) process the data, for example electronically and / or optically. The processor being for example made of a substance or composition which is a semiconductor, for example at least partly n- and / or p-doped semiconductor, for example at least one of II-, III-, IV-, V-, Vl-sem iconductor material, for example (doped) silicon and / or gallium arsenide. The calculating or determining steps described are for example performed by a computer. Determining steps or calculating steps are for example steps of determining data within the framework of the technical method, for example within the framework of a program. A computer is for example any kind of data processing device, for example electronic data processing device. A computer can be a device which is generally thought of as such, for example desktop PCs, notebooks, netbooks, etc., but can also be any programmable apparatus, such as for example a mobile phone or an embedded processor. A computer can for example comprise a system (network) of "sub-computers", wherein each sub-computer represents a computer in its own right. The term "computer" includes a cloud computer, for example a cloud server. The term "cloud computer" includes a cloud computer system which for example comprises a system of at least one cloud computer and for example a plurality of operatively interconnected cloud computers such as a server farm. Such a cloud computer is preferably connected to a wide area network such as the world wide web (WWW) and located in a so-called cloud of computers which are all connected to the world wide web. Such an infrastructure is used for "cloud computing", which describes computation, software, data access and storage services which do not require the end user to know the physical location and / or configuration of the computer delivering a specific service. For example, the term "cloud" is used in this respect as a metaphor for the Internet (world wide web). For example, the cloud provides computing infrastructure as a service (laaS). The cloud computer can function as a virtual host for an operating system and / or data processing application which is used to execute the method of the invention. The cloud computer is for example an elastic compute cloud (EC2) as provided by Amazon Web Services™. A computer for example comprises interfaces in order to receive or output data and / or perform an analogue- to-digital conversion. The data are for example data which represent physical properties and / or which are generated from technical signals. The technical signals are for example generated by means of (technical) detection devices (such as for example devices for detecting marker devices) and / or (technical) analytical devices (such as for example devices for performing (medical) imaging methods), wherein the technical signals are for example electrical or optical signals. The technical signals for example represent the data received or outputted by the computer. The computer is preferably operatively coupled to a display device which allows information outputted by the computer to be displayed, for example to a user. One example of a display device is a virtual reality device or an augmented reality device (also referred to as virtual reality glasses or augmented reality glasses) which can be used as "goggles" for navigating. A specific example of such augmented reality glasses is Google Glass (a trademark of Google, Inc.). An augmented reality device or a virtual reality device can be used both to input information into the computer by user interaction and to display information outputted by the computer. Another example of a display device would be a standard computer monitor comprising for example a liquid crystal display operatively coupled to the computer for receiving display control data from the computer for generating signals used to display image information content on the display device. A specific embodiment of such a computer monitor is a digital lightbox. An example of such a digital lightbox is Buzz®, a product of Brainlab AG. The monitor may also be the monitor of a portable, for example handheld, device such as a smart phone or personal digital assistant or digital media player.

[0121] The invention also relates to a program which, when running on a computer, causes the computer to perform one or more or all of the method steps described herein and / or to a program storage medium on which the program is stored (in particular in a non-transitory form) and / or to a computer comprising said program storage medium and / or to a (physical, for example electrical, for example technically generated) signal wave, for example a digital signal wave, carrying information which represents the program, for example the aforementioned program, which for example comprises code means which are adapted to perform any or all of the method steps described herein.

[0122] Within the framework of the invention, computer program elements can be embodied by hardware and / or software (this includes firmware, resident software, micro-code, etc.). Within the framework of the invention, computer program elements can take the form of a computer program product which can be embodied by a computer-usable, for example computer-readable data storage medium comprising computer-usable, for example computer-readable program instructions, "code" or a "computer program" embodied in said data storage medium for use on or in connection with the instruction-executing system. Such a system can be a computer; a computer can be a data processing device comprising means for executing the computer program elements and / or the program in accordance with the invention, for example a data processing device comprising a digital processor (central processing unit or CPU) which executes the computer program elements, and optionally a volatile memory (for example a random access memory or RAM) for storing data used for and / or produced by executing the computer program elements. Within the framework of the present invention, a computer-usable, for example computer-readable data storage medium can be any data storage medium which can include, store, communicate, propagate or transport the program for use on or in connection with the instruction-executing system, apparatus or device. The computer-usable, for example computer- readable data storage medium can for example be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, apparatus or device or a medium of propagation such as for example the Internet. The computer-usable or computer-readable data storage medium could even for example be paper or another suitable medium onto which the program is printed, since the program could be electronically captured, for example by optically scanning the paper or other suitable medium, and then compiled, interpreted or otherwise processed in a suitable manner. The data storage medium is preferably a non-volatile data storage medium. The computer program product and any software and / or hardware described here form the various means for performing the functions of the invention in the example embodiments. The computer and / or data processing device can for example include a guidance information device which includes means for outputting guidance information. The guidance information can be outputted, for example to a user, visually by a visual indicating means (for example, a monitor and / or a lamp) and / or acoustically by an acoustic indicating means (for example, a loudspeaker and / or a digital speech output device) and / or tactilely by a tactile indicating means (for example, a vibrating element or a vibration element incorporated into an instrument). For the purpose of this document, a computer is a technical computer which for example comprises technical, for example tangible components, for example mechanical and / or electronic components. Any device mentioned as such in this document is a technical and for example tangible device.

[0123] Acquiring data The expression "acquiring data" for example encompasses (within the framework of a computer implemented method) the scenario in which the data are determined by the computer implemented method or program. Determining data for example encompasses measuring physical quantities and transforming the measured values into data, for example digital data, and / or computing (and e.g. outputting) the data by means of a computer and for example within the framework of the method in accordance with the invention. The meaning of "acquiring data" also for example encompasses the scenario in which the data are received or retrieved by (e.g. input to) the computer implemented method or program, for example from another program, a previous method step or a data storage medium, for example for further processing by the computer implemented method or program. Generation of the data to be acquired may but need not be part of the method in accordance with the invention. The expression "acquiring data" can therefore also for example mean waiting to receive data and / or receiving the data. The received data can for example be inputted via an interface. The expression "acquiring data" can also mean that the computer implemented method or program performs steps in order to (actively) receive or retrieve the data from a data source, for instance a data storage medium (such as for example a ROM, RAM, database, hard drive, etc.), or via the interface (for instance, from another computer or a network). The data acquired by the disclosed method or device, respectively, may be acquired from a database located in a data storage device which is operably to a computer for data transfer between the database and the computer, for example from the database to the computer. The computer acquires the data for use as an input for steps of determining data. The determined data can be output again to the same or another database to be stored for later use. The database or database used for implementing the disclosed method can be located on network data storage device or a network server (for example, a cloud data storage device or a cloud server) or a local data storage device (such as a mass storage device operably connected to at least one computer executing the disclosed method). The data can be made "ready for use" by performing an additional step before the acquiring step. In accordance with this additional step, the data are generated in order to be acquired. The data are for example detected or captured (for example by an analytical device). Alternatively or additionally, the data are inputted in accordance with the additional step, for instance via interfaces. The data generated can for example be inputted (for instance into the computer). In accordance with the additional step (which precedes the acquiring step), the data can also be provided by performing the additional step of storing the data in a data storage medium (such as for example a ROM, RAM, CD and / or hard drive), such that they are ready for use within the framework of the method or program in accordance with the invention. The step of "acquiring data" can therefore also involve commanding a device to obtain and / or provide the data to be acquired. In particular, the acquiring step does not involve an invasive step which would represent a substantial physical interference with the body, requiring professional medical expertise to be carried out and entailing a substantial health risk even when carried out with the required professional care and expertise. In particular, the step of acquiring data, for example determining data, does not involve a surgical step and in particular does not involve a step of treating a human or animal body using surgery or therapy. In order to distinguish the different data used by the present method, the data are denoted (i.e. referred to) as "XY data" and the like and are defined in terms of the information which they describe, which is then preferably referred to as "XY information" and the like.

[0124] Registering

[0125] The n-dimensional image of a body is registered when the spatial location of each point of an actual object within a space, for example a body part in an operating theatre, is assigned an image data point of an image (CT, MR, etc.) stored in a navigation system.

[0126] Image registration

[0127] Image registration is the process of transforming different sets of data into one co-ordinate system. The data can be multiple photographs and / or data from different sensors, different times or different viewpoints. It is used in computer vision, medical imaging and in compiling and analysing images and data from satellites. Registration is necessary in order to be able to compare or integrate the data obtained from these different measurements.

[0128] Marker

[0129] It is the function of a marker to be detected by a marker detection device (for example, a camera or an ultrasound receiver or analytical devices such as CT or MRI devices) in such a way that its spatial position (i.e. its spatial location and / or alignment) can be ascertained. The detection device is for example part of a navigation system. The markers can be active markers. An active marker can for example emit electromagnetic radiation and / or waves which can be in the infrared, visible and / or ultraviolet spectral range. A marker can also however be passive, i.e. can for example reflect electromagnetic radiation in the infrared, visible and / or ultraviolet spectral range or can block x-ray radiation. To this end, the marker can be provided with a surface which has corresponding reflective properties or can be made of metal in order to block the x-ray radiation. It is also possible for a marker to reflect and / or emit electromagnetic radiation and / or waves in the radio frequency range or at ultrasound wavelengths. A marker preferably has a spherical and / or spheroid shape and can therefore be referred to as a marker sphere; markers can however also exhibit a cornered, for example cubic, shape.

[0130] Marker device

[0131] A marker device can for example be a reference star or a pointer or a single marker or a plurality of (individual) markers which are then preferably in a predetermined spatial relationship. A marker device comprises one, two, three or more markers, wherein two or more such markers are in a predetermined spatial relationship. This predetermined spatial relationship is for example known to a navigation system and is for example stored in a computer of the navigation system.

[0132] In another embodiment, a marker device comprises an optical pattern, for example on a two- dimensional surface. The optical pattern might comprise a plurality of geometric shapes like circles, rectangles and / or triangles. The optical pattern can be identified in an image captured by a camera, and the position of the marker device relative to the camera can be determined from the size of the pattern in the image, the orientation of the pattern in the image and the distortion of the pattern in the image. This allows determining the relative position in up to three rotational dimensions and up to three translational dimensions from a single two-dimensional image.

[0133] The position of a marker device can be ascertained, for example by a medical navigation system. If the marker device is attached to an object, such as a bone or a medical instrument, the position of the object can be determined from the position of the marker device and the relative position between the marker device and the object. Determining this relative position is also referred to as registering the marker device and the object. The marker device or the object can be tracked, which means that the position of the marker device or the object is ascertained twice or more over time.

[0134] Marker holder

[0135] A marker holder is understood to mean an attaching device for an individual marker which serves to attach the marker to an instrument, a part of the body and / or a holding element of a reference star, wherein it can be attached such that it is stationary and advantageously such that it can be detached. A marker holder can for example be rod-shaped and / or cylindrical. A fastening device (such as for instance a latching mechanism) for the marker device can be provided at the end of the marker holder facing the marker and assists in placing the marker device on the marker holder in a force fit and / or positive fit.

[0136] Pointer

[0137] A pointer is a rod which comprises one or more - advantageously, two - markers fastened to it and which can be used to measure off individual co-ordinates, for example spatial co-ordinates (i.e. three-dimensional co-ordinates), on a part of the body, wherein a user guides the pointer (for example, a part of the pointer which has a defined and advantageously fixed position with respect to the at least one marker attached to the pointer) to the position corresponding to the coordinates, such that the position of the pointer can be determined by using a surgical navigation system to detect the marker on the pointer. The relative location between the markers of the pointer and the part of the pointer used to measure off co-ordinates (for example, the tip of the pointer) is for example known. The surgical navigation system then enables the location (of the three-dimensional co-ordinates) to be assigned to a predetermined body structure, wherein the assignment can be made automatically or by user intervention.

[0138] Reference star

[0139] A "reference star" refers to a device with a number of markers, advantageously three markers, attached to it, wherein the markers are (for example detachably) attached to the reference star such that they are stationary, thus providing a known (and advantageously fixed) position of the markers relative to each other. The position of the markers relative to each other can be individually different for each reference star used within the framework of a surgical navigation method, in order to enable a surgical navigation system to identify the corresponding reference star on the basis of the position of its markers relative to each other. It is therefore also then possible for the objects (for example, instruments and / or parts of a body) to which the reference star is attached to be identified and / or differentiated accordingly. In a surgical navigation method, the reference star serves to attach a plurality of markers to an object (for example, a bone or a medical instrument) in order to be able to detect the position of the object (i.e. its spatial location and / or alignment). Such a reference star for example features a way of being attached to the object (for example, a clamp and / or a thread) and / or a holding element which ensures a distance between the markers and the object (for example in order to assist the visibility of the markers to a marker detection device) and / or marker holders which are mechanically connected to the holding element and which the markers can be attached to.

[0140] Navigation system

[0141] The present disclosure may be applied in the context of a navigation system for computer- assisted surgery. This navigation system preferably comprises the aforementioned computer for processing the data provided in accordance with the computer implemented method as described in any one of the embodiments described herein. The navigation system preferably comprises a detection device for detecting the position of detection points which represent the main points and auxiliary points, in order to generate detection signals and to supply the generated detection signals to the computer, such that the computer can determine the absolute main point data and absolute auxiliary point data on the basis of the detection signals received. A detection point is for example a point on the surface of the anatomical structure which is detected, for example by a pointer. In this way, the absolute point data can be provided to the computer. The navigation system also preferably comprises a user interface for receiving the calculation results from the computer (for example, the position of the main plane, the position of the auxiliary plane and / or the position of the standard plane). The user interface provides the received data to the user as information. Examples of a user interface include a display device such as a monitor, or a loudspeaker. The user interface can use any kind of indication signal (for example a visual signal, an audio signal and / or a vibration signal). One example of a display device is an augmented reality device (also referred to as augmented reality glasses) which can be used as so-called "goggles" for navigating. A specific example of such augmented reality glasses is Google Glass (a trademark of Google, Inc.). An augmented reality device can be used both to input information into the computer of the navigation system by user interaction and to display information outputted by the computer.

[0142] The invention also relates to a navigation system for computer-assisted surgery, comprising: a computer for processing the absolute point data and the relative point data; a detection device for detecting the position of the main and auxiliary points in order to generate the absolute point data and to supply the absolute point data to the computer; a data interface for receiving the relative point data and for supplying the relative point data to the computer; and a user interface for receiving data from the computer in order to provide information to the user, wherein the received data are generated by the computer on the basis of the results of the processing performed by the computer.

[0143] Surgical navigation system

[0144] A navigation system, such as a surgical navigation system, is understood to mean a system which can comprise: at least one marker device; a transmitter which emits electromagnetic waves and / or radiation and / or ultrasound waves; a receiver which receives electromagnetic waves and / or radiation and / or ultrasound waves; and an electronic data processing device which is connected to the receiver and / or the transmitter, wherein the data processing device (for example, a computer) for example comprises a processor (CPU) and a working memory and advantageously an indicating device for issuing an indication signal (for example, a visual indicating device such as a monitor and / or an audio indicating device such as a loudspeaker and / or a tactile indicating device such as a vibrator) and a permanent data memory, wherein the data processing device processes navigation data forwarded to it by the receiver and can advantageously output guidance information to a user via the indicating device. The navigation data can be stored in the permanent data memory and for example compared with data stored in said memory beforehand.

[0145] Landmarks

[0146] A landmark is a defined element of an anatomical body part which is always identical or recurs with a high degree of similarity in the same anatomical body part of multiple patients. Typical landmarks are for example the epicondyles of a femoral bone or the tips of the transverse processes and / or dorsal process of a vertebra. The points (main points or auxiliary points) can represent such landmarks. A landmark which lies on (for example on the surface of) a characteristic anatomical structure of the body part can also represent said structure. The landmark can represent the anatomical structure as a whole or only a point or part of it. A landmark can also for example lie on the anatomical structure, which is for example a prominent structure. An example of such an anatomical structure is the posterior aspect of the iliac crest. Another example of a landmark is one defined by the rim of the acetabulum, for instance by the centre of said rim. In another example, a landmark represents the bottom or deepest point of an acetabulum, which is derived from a multitude of detection points. Thus, one landmark can for example represent a multitude of detection points. As mentioned above, a landmark can represent an anatomical characteristic which is defined on the basis of a characteristic structure of the body part. Additionally, a landmark can also represent an anatomical characteristic defined by a relative movement of two body parts, such as the rotational centre of the femur when moved relative to the acetabulum.

[0147] Imaging geometry

[0148] The information on the imaging geometry preferably comprises information which allows the analysis image (x-ray image) to be calculated, given a known relative position between the imaging geometry analysis apparatus and the analysis object (anatomical body part) to be analysed by x-ray radiation, if the analysis object which is to be analysed is known, wherein "known" means that the spatial geometry (size and shape) of the analysis object is known. This means for example that three-dimensional, "spatially resolved" information concerning the interaction between the analysis object (anatomical body part) and the analysis radiation (x-ray radiation) is known, wherein "interaction" means for example that the analysis radiation is blocked or partially or completely allowed to pass by the analysis object. The location and in particular orientation of the imaging geometry is for example defined by the position of the x-ray device, for example by the position of the x-ray source and the x-ray detector and / or for example by the position of the multiplicity (manifold) of x-ray beams which pass through the analysis object and are detected by the x-ray detector. The imaging geometry for example describes the position (i.e. the location and in particular the orientation) and the shape (for example, a conical shape exhibiting a specific angle of inclination) of said multiplicity (manifold). The position can for example be represented by the position of an x-ray beam which passes through the centre of said multiplicity or by the position of a geometric object (such as a truncated cone) which represents the multiplicity (manifold) of x-ray beams. Information concerning the above- mentioned interaction is preferably known in three dimensions, for example from a three- dimensional CT, and describes the interaction in a spatially resolved way for points and / or regions of the analysis object, for example for all of the points and / or regions of the analysis object. Knowledge of the imaging geometry for example allows the location of a source of the radiation (for example, an x-ray source) to be calculated relative to an image plane (for example, the plane of an x-ray detector). With respect to the connection between three-dimensional analysis objects and two-dimensional analysis images as defined by the imaging geometry, reference is made for example to the following publications:

[0149] 1 . "An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision", Roger Y. Tsai, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Miami Beach, Florida, 1986, pages 364-374

[0150] 2. "A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses", Roger Y. Tsai, IEEE Journal of Robotics and Automation, Volume RA-3, No. 4, August 1987, pages 323-344.

[0151] 3. "Fluoroscopic X-ray Image Processing and Registration for Computer-Aided Orthopedic Surgery", Ziv Yaniv 4. EP 08 156 293.6

[0152] 5. US 61 / 054,187

[0153] Fixed (relative) position

[0154] A fixed position, which is also referred to as fixed relative position, in this document means that two objects which are in a fixed position have a relative position which does not change unless this change is explicitly and intentionally initiated. A fixed position is in particular given if a force or torque above a predetermined threshold has to be applied in order to change the position. This threshold might be 10 N or 10 Nm. In particular, the position of a sensor device remains fixed relative to a target while the target is registered or two targets are moved relative to each other. A fixed position can for example be achieved by rigidly attaching one object to another. The spatial location, which is a part of the position, can in particular be described just by a distance (between two objects) or just by the direction of a vector (which links two objects). The alignment, which is another part of the position, can in particular be described by just the relative angle of orientation (between the two objects).

[0155] Medical Workflow

[0156] A medical workflow comprises a plurality of workflow steps performed during a medical treatment and / or a medical diagnosis. The workflow steps are typically, but not necessarily performed in a predetermined order. Each workflow step for example means a particular task, which might be a single action or a set of actions. Examples of workflow steps are capturing a medical image, positioning a patient, attaching a marker, performing a resection, moving a joint, placing an implant and the like.

[0157] BRIEF DESCRIPTION OF THE DRAWINGS

[0158] In the following, the invention is described with reference to the appended figures which give background explanations and represent specific embodiments of the invention. The scope of the invention is however not limited to the specific features disclosed in the context of the figures, wherein

[0159] Figs. 1a and 1b illustrate methods according to the present disclosure;

[0160] Fig. 2 is a schematic illustration of a system according to the present disclosure;

[0161] Fig. 3a illustrates an optical setup for collimated imaging;

[0162] Fig. 3b illustrates an object and its envelope;

[0163] Fig. 4 illustrates the effects associated with OOF radiation;

[0164] Fig. 5 illustrates examples of images for different scenarios;

[0165] Fig. 6 illustrates images depicting a guidewire; Fig. 7 illustrates a machine learning application;

[0166] Fig. 8 illustrates another machine learning application;

[0167] Fig. 9. illustrates another machine learning application.

[0168] DESCRIPTION OF EMBODIMENTS

[0169] Figure 1a illustrates the basic steps of the method according to the present disclosure. The computer-implemented method for medical imaging according to the present disclosure comprises, in step S11, obtaining X-ray image data from one or more X-ray images acquired by an imaging system comprising an X-ray source, an X-ray detector, and a collimation shutter arranged between the X-ray source and the X-ray detector and configured to collimate X-ray radiation.

[0170] In step S12, non-collimated image data resulting from out of focus, OOF, radiation are processed to obtain extracted image information, e.g. by image processing, and, in step S13, the method comprises outputting data based on the extracted image data.

[0171] The X-ray image data is data obtained using initial imaging settings. Said initial imaging settings comprise initial imaging position settings and initial shutter settings.

[0172] The initial imaging position settings describe the position of the X-ray source and the X-ray detector in space. The initial shutter settings describe the configuration of the collimation shutter, which, for example may comprise the shutter opening size and the shutter position in space.

[0173] The X-ray image data comprises collimated image data for a first detector area of the X-ray detector, the first detector area corresponding to an initial field of view, initial FOV, defined by an imaging geometry of the imaging system including the configuration of the collimation shutter.

[0174] The X-ray image data also comprises the non-collimated image data, i.e. non-collimated image data for a second detector area outside of the first detector area.

[0175] Further optional steps, also shown in Figure 1 b, will be provided below for illustrating possible aspects of the method of the present disclosure.

[0176] In some aspects, the method of the present disclosure, particularly the processing the non- collimated image data to obtain extracted image information, further comprises determining in step S14a out-of-focus object detection data based on the non-collimated image data and determining in step S14b object characterization data describing the object’s position and / or the object’s shape in the X-ray image data based on the out-of-focus object detection data, particularly based only on the out-of-focus object detection data. Thus, object-identification and particularly tracking can be implemented using the non-collimated image data. In another aspect, the method, particularly the processing the non-collimated image data to obtain extracted image information, may comprise determining in step S15 collimated object detection data based on the collimated image data, and in step S16, determining whether the collimated object detection data indicates that the object is not present in the collimated image data. In case it is determined that the collimated object detection data indicates that the object is not present in the collimated image data, the method may proceed with step S17 of determining out-of-focus object detection data based on the non-collimated image data, and, in step S18, determining object characterization data describing the object’s position and / or the object’s shape based on the out-of-focus object detection data, specifically based only on the out-of-focus object detection data. That is, the out-of-focus image data may advantageously be used where objects are not found in the collimated image data. With known methods, this scenario would rather lead to trial and error and increased radiation dosage. Reference is made to Fig. 3, where an example of out-of-focus object detection data 7b-2 is shown. In case in step S16 it is determined that the object is present in the collimated image data, then the method may use said collimated image data for obtaining object characterization data and then, for example, proceed with any one of steps S22 to S25, as indicated by the second arrow from step S16.

[0177] In yet another aspect, the method, particularly the processing the non-collimated image data to obtain extracted image information, may comprise determining, in step S19, collimated object detection data based on the collimated image data, determining, in step S20, out-of-focus object detection data based on the non-collimated image data, and determining, in step S21, object characterization data describing the object’s position and / or the object’s shape in the X-ray image data based on the collimated object detection data and the out-of-focus object detection data. Thus, the object can be better characterized in view of using not only the collimated detection data, but also the out of focus object detection data. Reference is made to Fig. 3, where an example of out-of-focus object detection data 7b-2 is shown.

[0178] In some aspects, the method may comprise determining, in step S22, based on the object characterization data, e.g. obtained in step S14b or step S18 or step S21, updated imaging settings describing an updated FOV, wherein, when a new X-ray image is acquired using the updated imaging settings, the object is depicted at least partially in the updated FOV.

[0179] In an alternative aspect, in step S23, the method may comprise determining, based on the object characterization data, updated imaging settings describing an updated FOV, wherein, when a new X-ray image is acquired using the updated imaging settings, at least part of, in particular all of the object, is not depicted in the updated FOV.

[0180] In yet another alternative, in step S24, the method may comprise determining, based on the object characterization data, updated imaging settings describing an updated FOV, wherein, when a new X-ray image is acquired using the updated imaging settings, a larger or a smaller portion of the object is depicted at least partially in the updated FOV than in the initial FOV. In said steps S22 to S24, the updated imaging settings comprise updated imaging position settings and / or updated shutter settings and / or updated X-ray exposure settings, such as dosage or energy. The updated imaging position settings describe the position of the X-ray source and the X-ray detector in space, and the updated shutter settings describe the configuration of the collimation shutter, e.g. position and / or opening.

[0181] As another aspect, the method may comprise step S25 and subsequently one of steps S26 to S28. That is, the method may comprise, in step S25, determining object envelope data describing a geometric area in the X-ray image data representing at least parts of the object, in particular a region of interest of the object, based on the object characterization data.

[0182] The method may comprise, in step S26, determining updated imaging settings describing an updated FOV based on the object envelope data, wherein the updated FOV contains the geometric area described by the object envelope data, in particular such that, when a new X-ray image is acquired using the updated imaging settings, the object is depicted at least partially in the updated FOV. This is similar to step S22.

[0183] Alternatively, the method may comprise, in step S27, determining updated imaging settings describing an updated FOV based on the object envelope data, wherein the updated FOV does not contain the geometric area described by the object envelope data, in particular such that, when a new X-ray image is acquired using the updated imaging settings, the object is at least partially not depicted in the updated FOV. This is similar to step S23.

[0184] Yet alternatively, the method may comprise, in step S28, determining updated imaging settings describing an updated FOV based on the object envelope data. The updated FOV may contain a larger or a smaller portion of the geometric area described by the object envelope data than the initial FOV, in particular such that, when a new X-ray image is acquired using the updated imaging settings, a larger or a smaller portion of the object is depicted in the updated FOV than the initial FOV. This is similar to step S24.

[0185] In said steps, S25 to S28, the updated imaging settings may comprise updated imaging position settings and / or updated shutter settings. The updated imaging position settings describe the position of the X-ray source and the X-ray detector in space and the updated shutter settings describe the configuration of the collimation shutter, e.g. position and / or opening.

[0186] The method of the present disclosure, in some aspects and particularly before, after or for use in any of steps described above, may comprise determining, in step S29, motion pattern data describing the object’s motion and / or shape changes, in particular wherein a motion pattern may comprise a translation of the object and / or rotation of the object and / or deformation of the object and / or dilution of the object. In step S30, the method may then comprise updated imaging settings describing an updated field-of-view, updated FOV, based on the determined motion pattern data, in particular additionally based on the object characterization data. In some aspects of the present disclosure, the method may comprise step S31 and optionally steps S32 to S34. In this case, determining the motion pattern data comprises determining, in step S31, object characterization data comprising object position data based on multiple X-ray images taken at different points in time. For example, the motion pattern data may be determined by acquiring, in step S32, two X-ray images at two different points in time, determining, in step S33a, first object position data P_t1 from the image data at a first point in time t1 , and determining, in step S33b, second object position data P_t2 from the image data at a second point in time t2 which is after the first point in time t1. Then, in step S34, the motion pattern data is determined based on deriving a motion pattern based on the first object position data P_t1 and the second object position data P_t2.

[0187] In other aspects, the determining motion pattern data may comprise, in step S35a, acquiring the X-ray image using a long exposure. Alternatively, determining motion pattern data may comprise using, in step S35b a model for the motion, wherein the model is based on an object type of the object and, additionally or alternatively, based on external measurement data describing the motion of the object.

[0188] When carrying out one of the above-described steps S26, S27 or S28, the method may comprise, for example prior to step S29, comprise steps S36 to S38 or steps S39 to S41.

[0189] That is, as an example, determining updated imaging settings may comprise, in step S36, determining a geometric center of gravity of the geometric area described by the envelope data and / or determining a size of the geometric area described by the envelope data, and determining, in step S37, updated imaging positing settings based on the geometric center of gravity and / or determining, in step S38, updated shutter settings based on the size of the geometric area, so as to obtain an updated FOV that substantially centers the geometric area containing the object on the detector. S37 and S38 are here shown subsequently, but may also be carried out concurrently or in reverse order.

[0190] As another example, determining updated imaging settings may comprise, in step S39, determining the geometric center of gravity of the geometric area described by the envelope data, and / or determining a size of the geometric area described by the envelope data. Determining updated imaging settings may further comprise determining, in step S40 updated shutter settings based on the size of the geometric area and based on the geometric center of gravity while maintaining the initial imaging positioning settings. Optionally, determining updated imaging settings may also comprise determining, in step S41, whether the updated FOV meets predetermined criteria, in particular, whether the updated FOV is substantially centered around the geometric area, and otherwise additionally determining, in step S42, updated imaging position settings. S39 to S42 are here shown subsequently, but where appropriate may also be carried out concurrently or in a different order.

[0191] In any of the above methods, the present disclosure may comprise, in optional step S43, artifact correction based on the non-collimated image data, particularly correction of collimator shutter penumbra artifacts and / or truncation artifacts of volumetric reconstruction images based on the image data, in particular cone beam CTs. Specifically, extracted information from the noncollimated area may be used for correcting jaw penumbra in the projections.

[0192] In some aspects, the present disclosure may comprise, in optional step S44, generating an extended-view X-ray image based on the collimated image data and the extracted image information, the extended-view X-ray image depicting the initial or an updated FOV and areas outside of the initial or updated FOV. Particularly the extended-view X-ray image being a fulldetector-size image or larger-than-detector-size image.

[0193] In further aspects, the method of the present disclosure comprises using, in optional step S45, a machine learning technique, in particular a trained machine learning model, trained based on X- ray image pairs of a collimated X-ray image and a corresponding extended-view X-ray image, particularly full-size X-ray image. For example, the machine learning technique may be used for at least one of predicting an extended-view X-ray image from a collimated X-ray image based only on the non-collimated image data or based on the collimated image data and the noncollimated image data of the collimated X-ray image, predicting a collimated X-ray image including collimated image data and non-collimated image data from an extended-view X-ray image, particularly for use in training object detection algorithms and / or models, predicting, from the non-collimated image data of a collimated X-ray image, image data for the second detector area in a corresponding extended-view X-ray image, predicting non-collimated image data of a collimated X-ray image from collimated image data of the collimated X-ray image, correcting artifacts such as collimation shutter penumbra artifacts, generating collimated Digitally Reconstructed Radiographs, DRRs, from CT, the DRRs including predicted Out Of Focus, OOF, information.

[0194] In further aspects, the method comprises, in optional step S46, using the extracted image information, particularly the outputted data, e.g. control data, for at least one of visualization for object detection, particularly for navigation and / or tracking of the object and / or for image registration, imaging geometry adjustment, particularly adjustment of X-ray source and / or X-ray detector position and / or collimation shutter configuration, particularly for FOV tracking of an object and / or for dose reduction by reducing FOV size, FOV extension, FOV tracking, detection and / or tracking of an object, artifact reduction, particularly correction of shutter penumbra and / or correction of truncation artifacts.

[0195] Steps S43 to S46 may be carried out in any combination and order. One or more of them may be omitted.

[0196] Figure 2 illustrates a system 1 according to the present disclosure. The system may be configured to carry out the methods of the present disclosure, particularly as outlined in the context of Figures 1a and 1 b. Several optional system features are shown for illustration and may be omitted or replaced. The system may comprise an imaging system 2, such as an X-ray imaging system. An exemplary source 2a and exemplary detector 2b arranged on an optional gantry 2c are shown. Moreover, a collimation shutter 2d is shown.

[0197] The imaging system is illustrated, exemplarily, as a system having an optional gantry 2c, e.g. a tiltable gantry, on which the source and detector are mounted. As an example, the gantry may have a tiltable rotation plane. If a gantry is provided, the gantry may, for example, have a loop shape. However, other configurations of the medical imaging system are conceivable. For example, a C-arm configuration may be adopted instead of a gantry.

[0198] As an example, the imaging system may comprise a 2D X-ray scanner configured to carry out X- ray image data acquisition. The 2D X-ray scanner may correspond to the medical imaging system. Accordingly, the 2D X-ray scanner is also referred to as imaging system hereinbelow. Particularly, the imaging system may be a CT imaging system, particularly a cone beam CT (CBCT). As an example, a Loop-X imaging system may be employed for X-ray image data acquisition.

[0199] The system may also comprise a processing device 5 configured to carry out and / or control any of the method steps of the method of the present disclosure, e.g. as illustrated in Figures 1a and 1b. The processing device is shown as being part of the imaging system comprising the X-ray scanner, but it may also be a separate processing device or be a partially separate distributed processing system.

[0200] The system may comprise a wheeled device, for example an automated guided vehicle (AGV), the scanner being mounted on the wheeled device and the wheeled device being configured to move the scanner. For example, a movement of the X-ray viewing axis, specifically a translational movement, may comprise moving the wheeled device relative to the patient table. The wheeled device may, for example, comprise four independently steerable wheels, in particular with rear wheels having active drive / traction. The wheeled device may be non-rail-borne to allow for independent translational movement.

[0201] The system may optionally further comprise a tracking device 3, in this example an infrared tracking camera. The tracking device may be rigidly attached to the imaging system, optionally in a detachable manner. The Figure also illustrates that the system may optionally comprise a surgical navigation system 4. These optional tools may be used for navigation and for providing spatial information for even further improved image reconstruction. A portion 7a of the patient 7 inside the region of interest 8 can comprise an object to be imaged by the imaging system. Alternatively or in addition, a medical tool 9 may be the object to be imaged by the imaging system.

[0202] A tracking device of the surgical navigation system is denoted by reference sign 4a. The tracking device 3 and the medical imaging system 2 may communicate via data connection 10a. The surgical navigation system and the medical imaging system may communicate via a data connection 10b. The respective data connection may be wired or wireless.

[0203] Figure 3a illustrates an optical setup for collimated X-ray imaging to illustrate the method of the present disclosure.

[0204] An X-ray source 2a is provided. Moreover, a detector 2b is provided, which acquires X-ray image data. Between the source and the detector, a collimation shutter 2d is provided. The optical settings, also referred to as imaging settings, comprise, for example shutter settings.

[0205] An object arranged between the source and the collimation shutter, will have a projection onto the detector, referred to as object projection. Its position can, for example, be determined in a detector coordinate system.

[0206] Shown is, for initial imaging settings, a first detector area 11a, also referred to as collimated area or (initial) FOV. In this area, collimated image data are acquired by the detector, for example collimated object projection data 7a-1. An object 7b outside the FOV is also shown.

[0207] Also shown is, for initial imaging settings, a second detector area 11b, which represent an out-of- focus area. In this area, out-of-focus object detection data 7b-1 , as part of out-of-focus image data, will be acquired for object 7b.

[0208] According to the method of the present disclosure, image information extracted from the X-ray image data will at least comprise out-of-focus image data.

[0209] The FOV may be updated based on the extracted image information. Thus, an updated FOV 12a may be obtained. The FOV may be updated to include the collimated object projection 7b-1.

[0210] When at least one of the collimation shutters is visible in the image acquired by the detector, the resultant image is referred to as a collimated X-ray image. Where X-ray image data comprises an X-ray image with a smaller extent of the shutter visible in the image acquired by the detector, e.g., where no shutters are visible at all, this referred to as an extended-view X-ray image.

[0211] Figure 3b illustrates an object 7 with an envelope 7c.

[0212] Further features and advantages associated with the method and system of the present disclosure are outlined below.

[0213] As an example, the method may aid in saving dose during X-ray imaging of a patient by collimating to a small field of view but still using sparse / approximate image information from the collimated parts to enhance the image information from the non-collimated part.

[0214] To provide some context, scattering can lead to difficulties, as will be explained below making reference to an X-ray imaging system.

[0215] For example, a rotating anode X-ray source of an X-ray imaging system may provide headscatter radiation. Head-scatter is composed of off focus radiation, but also additional scattering downstream of the X-ray path (e.g. caused by physical filters in the beam path for beam hardening). The amount of off focus radiation of an X-ray source (the effect of secondary, back- scattered electrons causing additional photons with tendentially lower energies and not following the focal path) is also a function of the X-ray system’s design; i.e., it depends on the inner shielding of the X-ray tube that is supposed to largely block the off focus radiation. In the method of the present disclosure, an X-ray source and X-ray imaging system design is used, which does not fully avoid off focus radiation.

[0216] The X-ray imaging system, as explained further above, has a collimator and a detector. The collimator enables collimation of the X-ray beam to the detector (i.e. to the full extent of the detector or a polygonal or circular subregion of it) using two or more movable, radio-intransparent collimator shutters that enable to restrict the X-ray beam.

[0217] According to the present disclosure, the method may comprise acquiring an X-ray image (fluoro, single or cumulated projection image) that includes the collimated (i.e. non-blocked) and the noncollimated (i.e. blocked) area, e.g., using fullsize detector read-out.

[0218] Optionally, multiple images may be acquired and used to improve detection of low energy / low quantity photons to improve signal to noise ratio. For example, this can be done by postprocessing, e.g., by cumulating and averaging detector images of the same imaged object with or without motion-compensation applied to the acquired images.

[0219] As another optional aspect, the method of the present disclosure may comprise adaptive contrast enhancement and denoising of the acquired and optionally post-processed image(s). As an example, this may be accomplished using contrast-limited adaptive histogram analysis (CLAHE) or unsharp masking or any other local contrast enhancement strategy, optionally in combination with edge-preserving denoising filtration (e.g. anisotropic diffusion). Such additional postprocessing may be applied to the fullsize detector image or exclusively to the non-collimated areas of the image.

[0220] Different use cases for the method of the present disclosure are conceivable.

[0221] In an exemplary first use case is the tracking of an object of interest (e.g. guidewire, contrast bolus, any surgical instrument). This may be referred to as “catching up” with an object.

[0222] To that end, an object detection mechanism may be employed as follows:

[0223] (i) A first step tries to detect an object of interest based on intensity and / or shape in the collimated (standard) area of the image using image delineation or template matching. This may entail approaches with or without machine learning means.

[0224] (ii) In a second step, if the object of interest is not found within the collimated image area, the non-collimated (blocked) image information, which shows slightly geometrically distorted and more noisy image information, is to be analyzed with similar means as mentioned in (i). (iii) Even if in (i) the object is found, the reliability of the object detection can be improved by additionally incorporating (ii) if the object is also visible in the non-collimated (blocked) area.

[0225] A tracking mechanism may, for example, be implemented as follows:

[0226] (i) Based on the object detection, e.g. using the object detection mechanism described above, the determined object position on the detector’s full extent is used derive a circular or polygonal X-ray collimation that increases the likelihood of capturing the object of interest in the new collimated area of the detector in the next X-ray image(s).

[0227] (ii) For the derivation of the next collimation, besides the object of interest’s detected position on the detector, also optionally captured motion patterns over time of the object of interest may be incorporated.

[0228] (iii) Finally, the collimation shutters are moved according to the derivation in (ii) and thus applied to the next acquisition of X-ray images.

[0229] (iv) Optionally, the overall X-ray imaging device can be moved either as alternative or in addition to the collimation shutter movements of (iii), specifically if the object of interest is found close to the detector borders.

[0230] (v) For imaging device motion, a means for motorized movement of relevant parts of the X-ray system, e.g. source and detector, including structures that align them in defined relative position to each other, on the floor or other mounting structures (e.g. floor- / ceiling- mounted rails or robotic arm) may be provided.

[0231] The method of the present disclosure, particularly the first use case described above, may be used for different applications.

[0232] As an example application, dose-saving contrast bolus chasing or surgical instrument or guidewire tracking in fluoroscopic sequences may be provided by employing the object detection and tracking mechanisms. The method also enables continuous tracking of the object(s) of interest, even if intermediately not visible in the collimated (non-blocked) area. Moreover, tracking of known, rigid instruments or tools that extend beyond the collimated (non-blocked) area, but are not or at least not reliably detectable in the collimated area due to occlusions (e.g. by absorbing materials such as metal or cortical bone) can be enabled. These applications may be achieved, for example, by employing the object detection mechanism in the non-collimated (blocked) area and registering the known model of the object of interest according to the image information using optional post-processing, e.g. as described above.

[0233] A second potential use case of the method of the present disclosure entails different machine learning approaches, e.g. machine learning from collimated fullsize X-ray images.

[0234] The machine learning approach may entail using, for training of a machine learning model, a set of X-ray image records of a certain class of objects (e.g. specific anatomical region such as thorax). Each X-ray image record may consist of at least one fullsize detector image (i.e. full detector area is collimated) and at least one partially collimated X-ray image, where the imaged object does either not change or changes in a known manner throughout the record. The X-ray image records may or may not comprise different X-ray viewing angles (projection geometries) to enhance representativity and generalization for the selected object class.

[0235] The machine learning model may be trained on the set of X-ray image records by one of the following approaches:

[0236] (a) Using the fullsize detector images as ground truth (GT) and trying to predict the corresponding non-collimated (blocked) area of it from the non-collimated (blocked) area of the partially collimated X-ray images. The image information of the collimated area of the partially collimated X-ray images may additionally be incorporated into the prediction model. This is illustrated in Figure 7, which will be explained in more detail below.

[0237] (b) Using the non-collimated (blocked) area of the partially collimated X-ray images as ground truth (GT) and trying to predict it using the corresponding collimated (non-blocked) area of the fullsize detector images. This is illustrated in Figure 8, which will be explained in more detail below.

[0238] (c) Using the fullsize detector images as ground truth (GT) and trying to predict the corresponding collimated (non-blocked) area including the collimator shutter penumbra boundaries from a partially collimated image that includes collimator shutter penumbra artifacts and distorted, noisy image information in its non-collimated (blocked) area. This is illustrated in Figure 9, which will be explained in more detail below.

[0239] There are different exemplary applications for which the trained machine learning model may be used. The model is applied to unseen X-ray images, e.g. of the underlying class of objects. Example applications comprise:

[0240] (i) Prediction of fullsize detector images from partially collimated images by employing machine learning models as specified in (a). This may, for example, facilitate visual user experience in image presentation of technically partially collimated X-ray images and / or facilitate subsequent image processing activities such as spatial image registration or object detection.

[0241] (ii) Prediction of partially collimated images from fullsize detector images using machine learning models as specified in (b) and assuming a desired collimation; this may, for example, facilitate training of object detection algorithms and models (see first use case).

[0242] (iii) Prediction of artifact-free partially collimated images from technically partially collimated images using machine learning models as specified in (c). This may be used for eliminating the collimation jaw penumbra artifacts of partially collimated images and can also be employed for 3D CT / CBCT reconstruction, for example in the third use case described below. A third use case may comprise using the method of the present disclosure for improved 3D reconstruction of truncated CT or CBCT datasets. This may apply to X-ray projection datasets of 3D CTs or CBCTs where at least one projection is a partially collimated X-ray image, thereby, not fully capturing the boundaries of the imaged object (i.e. truncated projection data). As one example, the third use case may entail using for all partially collimated X-ray images a machine learning model (a) of the second use case to predict the corresponding fullsize detector image and to specifically improve image quality in the non-collimated (non-blocked) areas. In the third use case, subsequently this predicted fullsize detector image may be considered during ramp filtering to replace or combine with heuristic extrapolation algorithms to avoid truncation artifacts due to ramp filter extrema. The method may allow to minimize truncation artifacts in technically truncated 3D CBCT / CT data, and to better reconstruct the real-world attenuation coefficients or Hounsfield units, respectively, of the imaged object(s).

[0243] A fourth potential use case is training a machine learning model, e.g. using data from the machine learning approaches described above, to obtain DRRs from CTs, specifically to train a machine learning model to prepare a DRR from CT data, the DRR looking like the collimated images.

[0244] As an example, referring to the second use case, either ground truth (GT) or technically acquired X-ray data may be replaced with DRRs that mimic the projection geometry of the imaging device accurately (optionally incorporating further physical effects such as scatter or beam hardening) and use an a priori CT as volumetric information for DRR rendering (or other suited modality) of the object of interest, which is spatially registered with the object of interest as currently observed in the X-ray images.

[0245] Even further aspects concerning the system and method of the present disclosure will be provided below.

[0246] In an ideal scenario, emitted radiation, such as X-rays would be emitted by a focal spot and, in spite of travelling through a collimator, there would only be one point source.

[0247] However, in reality, radiation, like X-rays, also originates outside of the focal spot. Moreover, backscattering occurs, e.g. secondary and tertiary backscatter electrons. Moreover, photons can be scattered at different system components, such as collimation shutters, also referred to as collimation jaws, a source exit window, monoblock components, filter insets, or the like. Accordingly, off focus radiation occurs, also referred to as out of focus radiation, extra focal radiation, or head scatter.

[0248] This results in the detected image data in a primary detector area, where the image should otherwise be focused, in some blurriness. In a secondary detector area to the sides of the primary detector area, a lower signal intensity image data (compared to the primary detector area is present). Usually systems will be constructed to avoid these phenomena as much as possible and only the primary detector area data will generally be used.

[0249] This is illustrated schematically in Fig. 4. In Fig. 5, from left to right, an uncollimated image, a primary detector area of a collimated image, and the primary and secondary detector area image are shown. In Fig. 6, it is shown exemplarily for a guidewire that the guidewire is to some degree visible in the secondary detector area. In the first image, it does not extend through the primary detector area, whereas in the second image it extends also through the first detector area. The present disclosure, by making use of data from the secondary detector area, may move the FOV to bring it from the first to the second position, i.e., to find the guidewire.

[0250] Some potential use cases of the method of the present disclosure will be described below.

[0251] In a use case, already mentioned above, among others in the context of Figure 6, the present method may be used for tracking of objects of interest. Described in the following is out-of-sight guidewire tracking. As an example, an object like a guidewire or the like may be detected in the in non-collimated region, e.g. using deep learning or template matching. Collimation may be adjusted and / or the imaging device may be repositioned (e.g. using wheels, or by changing the gantry angle). This allows for dose saving by targeted (minimum) collimation. A similar principle may be used for contrast bolus chasing (vascular use case in fluoro / DSA sequences) and / or for chasing of any surgical instrument of interest. Similarly, detection of other objects, e.g. a practitioner’s hands, may be detected and it may be determined whether it is too close to the X- ray FOV. Overall, it will be understood that the above allows for dose savings and for providing safety features.

[0252] Another use case, illustrated in Figure 7, is a machine learning application, wherein a model is trained to output a full image from a partial image. The model may learn from corresponding collimated and non-collimated X-ray pairs. The model may predict a full size X-ray or large portion thereof from a small FOV collimated X-ray, particularly using also non-collimated area only or full image as input. This may allow for improved user experience and preparation for object detection (see above) or other analysis (e.g. image registration).

[0253] Another use case illustrated in Figure 8 is a machine learning application, wherein a model is trained to simulate the effects of scattering and the like. To that end, the model learns from corresponding collimated and non-collimated X-ray pairs and, when trained, predicts collimated X-ray images from full-size images. This allows for training object detection algorithms and can aid in planning of tracking procedures on CT.

[0254] Another use case, illustrated in Figure 9, is a machine learning application, wherein a model is trained to eliminate penumbra or other artifacts from image data. To that end, the model learns from corresponding collimated and non-collimated X-ray pairs. The trained model may then eliminate collimation jaw penumbra from collimated images. This allows for better visualization. This can be employed, for example, for 3D CBCT reconstruction. This also allows for slightly increasing the FOV with the same dose.

[0255] Yet another use case, as already mentioned above, is reconstruction of truncated CBCTs. Out Of Focus Radiation (OOFR) generated image information may be used to further improve limited FOV CBCT reconstruction (i.e. extend FOV), either by using the machine learning predicted full FOV image from collimated X-rays with standard filtered back-projection or by regularizing an iterative reconstruction method with the OOFR information to further improve the reconstruction results. This allows for dose savings and improved patient overview (extended FOV).

[0256] Yet another use case, as mentioned above, is use with Digitally Reconstructed Radiographs (DRR). This may involve training a Machine Learning model to compute realistic, collimated DRRs from CT, including predicted, corresponding OOFR information. Such DRRs (optionally with further physical effects modeled) may be incorporated into the above-mentioned machine learning use cases for training purposes using spatially registered CT datasets. This allows for training efficiency, education, and simulation.

[0257] While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered exemplary and not restrictive. The invention is not limited to the disclosed embodiments. In view of the foregoing description and drawings it will be evident to a person skilled in the art that various modifications may be made within the scope of the invention, as defined by the claims.

Claims

CLAIMS1. A computer-implemented method for medical imaging, the method comprising: obtaining (S11) X-ray image data from one or more X-ray images acquired by an imaging system comprising an X-ray source, an X-ray detector, and a collimation shutter arranged between the X- ray source and the X-ray detector and configured to collimate X-ray radiation, wherein the X-ray image data is obtained using initial imaging settings, the initial imaging settings comprising initial imaging position settings and initial shutter settings, wherein the initial imaging position settings describe the position of the X-ray source and the X- ray detector in space, wherein the initial shutter settings describe the configuration of the collimation shutter, wherein the X-ray image data comprises collimated image data for a first detector area of the X- ray detector, the first detector area corresponding to an initial field of view, initial FOV, defined by an imaging geometry of the imaging system including the configuration of the collimation shutter, and wherein the X-ray image data comprises non-collimated image data for a second detector area outside of the first detector area, the non-collimated image data resulting from out of focus, OOF, radiation; processing (S12) the non-collimated image data to obtain extracted image information; and outputting (S13) data based on the extracted image information, particularly the outputted data obtained by means of processing the extracted image information using an image processing technique.

2. The method of claim 1 , wherein the outputted data comprises control data for a device, particularly a medical navigation device, a robot, and / or a medical device, e.g. an X-ray imaging device, particularly comprising the X-ray source and X-ray detector.

3. The method of any of the preceding claims, wherein the extracted image information comprises information on an object that is arranged at least partially between the X-ray source and the second detector area, and processing the non-collimated image data comprises detecting the object, the object being at least one of: a medical device, such as a guidewire, an anatomical structure, particularly of a subject of the medical imaging and / or of a person other than the subject of the medical imaging, an implant, a contrast agent, particularly a contrast bolus.

4. The method of any of the preceding claims, wherein the method, particularly the processing the non-collimated image data to obtain extracted image information, comprises: determining (S14a) out-of-focus object detection data based on the non-collimated image data, determining (S14b) object characterization data describing the object’s position and / or the object’s shape in the X-ray image data based on the out-of-focus object detection data, particularly based only on the out-of-focus object detection data.

5. The method of any of claims 1 to 3, wherein the method, particularly the processing the non- collimated image data to obtain extracted image information, comprises: determining (S15) collimated object detection data based on the collimated image data, and determining (S16) whether the collimated object detection data indicates that the object is not present in the collimated image data, in case it is determined that the collimated object detection data indicates that the object is not present in the collimated image data: determining (S17) out-of-focus object detection data based on the non-collimated image data, and determining (S18) object characterization data describing the object’s position and / or the object’s shape based on the out-of-focus object detection data, particularly based only on the out-of-focus object detection data.

6. The method of any of claims 1 to 3, wherein the method, particularly the processing the non- collimated image data to obtain extracted image information, comprises: determining (S19) collimated object detection data based on the collimated image data, determining (S20) out-of-focus object detection data based on the non-collimated image data, determining (S21) object characterization data describing the object’s position and / or the object’s shape in the X-ray image data based on the collimated object detection data and the out-of-focus object detection data.

7. The method of any of claims 4 or 5 or 6, the method further comprising: determining (S22), based on the object characterization data, updated imaging settings describing an updated FOV, wherein, when a new X-ray image is acquired using the updated imaging settings, the object is depicted at least partially in the updated FOV, or determining (S23), based on the object characterization data, updated imaging settings describing an updated FOV, wherein, when a new X-ray image is acquired using the updated imaging settings, at least part of, in particular all of the object, is not depicted in the updated FOV, ordetermining (S24), based on the object characterization data, updated imaging settings describing an updated FOV, wherein, when a new X-ray image is acquired using the updated imaging settings, a larger or a smaller portion of the object is depicted at least partially in the updated FOV than the initial FOV, wherein the updated imaging settings comprise updated imaging position settings and / or updated shutter settings and / or updated X-ray exposure settings, wherein the updated imaging position settings describe the position of the X-ray source and the X-ray detector in space, wherein the updated shutter settings describe the configuration of the collimation shutter.

8. The method of any of claims 4 or 5 or 6, the method further comprising: determining (S25) object envelope data describing a geometric area in the X-ray image data representing at least parts of the object, in particular a region of interest of the object, based on the object characterization data, determining (S26) updated imaging settings describing an updated FOV based on the object envelope data, wherein the updated FOV contains the geometric area described by the object envelope data, in particular such that, when a new X-ray image is acquired using the updated imaging settings, the object is depicted at least partially in the updated FOV, or determining (S27) updated imaging settings describing an updated FOV based on the object envelope data, wherein the updated FOV does not contain the geometric area described by the object envelope data, in particular such that, when a new X-ray image is acquired using the updated imaging settings, the object is at least partially not depicted in the updated FOV, or determining (S28) updated imaging settings describing an updated FOV based on the object envelope data, wherein the updated FOV contains a larger or a smaller portion of the geometric area described by the object envelope data than the initial FOV, in particular such that, when a new X-ray image is acquired using the updated imaging settings, a larger or a smaller portion of the object is depicted in the updated FOV than the initial FOV, wherein the updated imaging settings comprise updated imaging position settings and / or updated shutter settings, wherein the updated imaging position settings describe the position of the X-ray source and the X-ray detector in space, wherein the updated shutter settings describe the configuration of the collimation shutter.

9. The method of any of the preceding claims, the method further comprising:determining (S29) motion pattern data describing the object’s motion and / or shape changes, in particular wherein a motion pattern may comprise a translation of the object and / or rotation of the object and / or deformation of the object and / or dilution of the object, determining (S30) updated imaging settings describing an updated field-of-view, updated FOV, based on the determined motion pattern data, in particular additionally based on the object characterization data.

10. The method according to claim 9, wherein determining the motion pattern data comprises determining (S31) the object characterization data comprising object position data based on multiple X-ray images taken at different points in time, in particular wherein the method comprises determining the motion pattern data by: acquiring (S32) two X-ray images at two different points in time, determining (S33a) first object position data P_t1 from the image data at a first point in time t1 ; determining (S33b) second object position data P_t2 from the image data at a second point in time t2 which is after the first point in time t1 , determining (S34) the motion pattern data based on deriving a motion pattern based on the first object position data P_t1 and the second object position data P_t2.

11. The method according to claim 9 or 10, wherein determining motion pattern data comprises acquiring (S35a) the X-ray image using a long exposure, or wherein determining motion pattern data comprises using (S35b) a model for the motion, wherein the model is based on an object type of the object and, additionally or alternatively, based on external measurement data describing the motion of the object.

12. The method according to any of claims 8 to 11 , wherein determining updated imaging settings comprises: determining (S36) a geometric center of gravity of the geometric area described by the envelope data and / or determining a size of the geometric area described by the envelope data, determining (S37) updated imaging positing settings based on the geometric center of gravity and / or determining (S38) updated shutter settings based on the size of the geometric area, so as to obtain an updated FOV that substantially centers the geometric area containing the object on the detector.

13. The method according to any of claims 8 to 11 , wherein determining updated imaging settings comprises:determining (S39) the geometric center of gravity of the geometric area described by the envelope data, and / or determining a size of the geometric area described by the envelope data, determining (S40) updated shutter settings based on the size of the geometric area and based on the geometric center of gravity while maintaining the initial imaging positioning settings, and optionally, determining (S41) whether the updated FOV meets predetermined criteria, in particular, whether the updated FOV is substantially centered around the geometric area, and otherwise additionally determining (S42) updated imaging position settings.

14. The method of any of the preceding claims, the method comprising artifact correction (S43) based on the non-collimated image data, particularly, based on the non-collimated image data, correction of collimator shutter penumbra artifacts and / or, based on the non-collimated image data, correction of truncation artifacts of volumetric reconstruction images that are based on the image data, in particular, use of the non- collimated image data for estimating image information prior to filtering and 3D back projection so as to avoid truncation artifacts.

15. The method of any of the preceding claims, comprising generating (S44) an extended-view X-ray image based on the collimated image data and the extracted image information, the extended-view X-ray image depicting the initial or an updated FOV and areas outside of the initial or updated FOV, particularly the extended-view X- ray image being a full-detector-size image or larger-than-detector-size image.

16. The method of any of the preceding claims the method comprising the use (S45) of a machine learning technique, in particular a trained machine learning model, trained based on X-ray image pairs of a collimated X-ray image and a corresponding extended-view X-ray image, particularly full-size X-ray image, for at least one of: predicting an extended-view X-ray image from a collimated X-ray image based only on the non-collimated image data or based on the collimated image data and the non- collimated image data of the collimated X-ray image, predicting a collimated X-ray image including collimated image data and non-collimated image data from an extended-view X-ray image, particularly for use in training object detection algorithms and / or models, predicting, from the non-collimated image data of a collimated X-ray image, image data for the second detector area in a corresponding extended-view X-ray image, predicting non-collimated image data of a collimated X-ray image from collimated image data of the collimated X-ray image, correcting artifacts such as collimation shutter penumbra artifacts,generating collimated Digitally Reconstructed Radiographs, DRRs, from CT, the DRRs including predicted Out Of Focus, OOF, information.

17. The method of any of the preceding claims, wherein the method comprises using (S46) the extracted image information, particularly the outputted data, particularly the control data, for at least one of: visualization for object detection, particularly for navigation and / or tracking of the object and / or for image registration, imaging geometry adjustment, particularly adjustment of X-ray source and / or X-ray detector position and / or collimation shutter configuration, particularly for FOV tracking of an object and / or for dose reduction by reducing FOV size,FOV extension,FOV tracking, detection and / or tracking of an object, artifact reduction, particularly correction of shutter penumbra and / or correction of truncation artifacts.18 A system (1 ) comprising a processing device (5) configured to carry out and / or control the method of any of the preceding claims.

19. The system of claim 18, further comprising an imaging system (2) comprising an X-ray source (2a), an X-ray detector (2b), and a collimation shutter (2d) arranged between the X-ray source (2a) and the X-ray detector (2b) and configured to collimate X-ray radiation, the imaging system (2) configured to acquire collimated X-ray images and optionally configured to acquire extended-view, in particular full-size, X-ray images.

20. The system of claim 19, wherein the X-ray source (2a) and / or the X-ray detector (2b) are configured to be, e.g. robotically, movable based on control data, particularly at least in part based on control data obtained by the method of any one of claims 2 to 17.

21. The system (1) of any of claims 18 to 20, a robotic patient support is provided that is configured to move the patient relative to the imaging system and / or the imaging system is mounted on a robotic structure configured to move the imaging system relative to a patient support.

22. A computer program product, comprising instructions which, when the program is executed by a computer, cause the computer to carry out and / or control the method of any of claims 1 to 17.

23. A computer-readable medium comprising instructions which, when executed by a computer, cause the computer to carry out and / or control the method of any of claims 1 to 17.

Images