Object visualization in X-ray imaging

Abstract

The X-ray imaging system 100 comprises an X-ray source 110 and an X-ray detector 120 separated by an examination region 150 for performing an X-ray imaging operation on an object 160. A processor 140 identifies one or more internal structures 180 based on a comparison of depth sensor data representative of a three-dimensional surface 170 of the object 160 to an anatomical model including one or more internal structures 180 within the object 160 (S120). The processor 140 also uses the depth sensor data and the identified one or more internal structures 180 to calculate a surface projection 190 of the one or more internal structures onto the three-dimensional surface 170 of the object 160 from a perspective of the X-ray source 110 (S130) and outputs an image representation of the surface projection 190 for display as an overlay on the three-dimensional surface 170 of the object 160 (S140).

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE The present disclosure relates to visualizing an object during x-ray imaging. An x-ray imaging system, a computer implemented method, and a computer program product are disclosed. [Background technology]

[0002] The X-ray imaging system comprises an X-ray source and an X-ray detector. The X-ray source and the X-ray detector are separated by an examination region. An object is placed in the examination region for performing an X-ray imaging operation on the object. To avoid the need for repeated X-ray imaging and the associated increase in X-ray dose, it is important that the object to be imaged is properly positioned with respect to the X-ray imaging system.

[0003] As an example, in skeletal imaging of the knee or ankle, it is important to ensure that the x-rays pass through the desired portion of the joint. An additional consideration is that the x-rays pass through the joint within a desired range of angles so that the joint, and potentially other anatomical structures, are aligned in a desired manner in the resulting image.

[0004] Positioning of an object relative to an X-ray imaging system is traditionally performed by eye or via a monitor that displays a visual or red-green-blue (RGB) camera image of the object. The coverage of the X-rays emitted by the X-ray source on the object is often indicated by projecting a light field onto the object that represents the collimation window of the X-ray source. The radiation sensitive area of ​​the X-ray detector is typically indicated as a marking on the radiation receiving surface of the detector. In use, an operator positions the object relative to the X-ray detector by eye or via a monitor using the light field and the markings on the surface of the detector. Summary of the Invention [Problem to be solved by the invention]

[0005] However, conventional approaches for positioning an object relative to an X-ray imaging system have drawbacks. In particular, the location of internal structures, such as skeletal features, are hidden. To ensure that the desired portion of the anatomy is imaged, an operator typically palpates the patient to confirm the location of such skeletal features. However, palpation is time consuming and impedes workflow. The operator also collects an initial X-ray image of the patient and uses the initial image to determine a more optimal positioning of the patient. However, repeated collection of X-ray images in this manner increases the X-ray dose to the patient.

[0006] Therefore, there is a need for improved positioning of an object relative to an x-ray imaging system.

[0007] The document US Patent Application Publication No. 2019 / 0183439(A1) discloses a method for positioning a patient's body region for radiography acquisition by a radiography system. The method includes the steps of providing examination requirements for the body region, pre-positioning the body region in the radiography system for radiography acquisition, pre-positioning an acquisition unit of the radiography system for radiography acquisition, generating three-dimensional positioning acquisition information of the body region using a 3D camera system, generating a preview image from the three-dimensional positioning acquisition information, where a patient model is generated from the three-dimensional positioning acquisition information, and the preview image is generated from the patient model, the preview image showing a representation as if it had been performed using the acquisition unit of the radiography system as intended in the employed pre-positioning, and outputting at least one of the preview image and the positioning information based on the preview image. An apparatus and a computer-readable medium are also disclosed.

[0008] Another document, US Patent Application Publication No. 2016 / 0213329(A1), discloses an X-ray recording system including an X-ray emitter for generating a beam used for imaging, an imaging X-ray detector with a two-dimensional or three-dimensional recording geometry for determining the attenuation of the rays of the beam, and a patient support and / or positioning device for the patient in a recording area of ​​the X-ray recording system between the X-ray emitter and the X-ray detector. A time-of-flight (TOF) camera for determining the contour of the patient is configured, and there is a computer having a memory and software stored therein, the computer unit being implemented in operation to generate a three-dimensional wire model of the patient from the contour of the patient recorded by the TOF camera together with the joint locations located therein, and to simulate and display at least one anatomical structure scaled to the wire model.

[0009] Another document, US Patent Application Publication No. 2020 / 0029919(A1), discloses that proper positioning of a patient in an X-ray imaging system can cause difficulties for medical professionals, on the one hand, due to the small size of important anatomical aspects that need to be captured in an X-ray image, and on the other hand, due to the large movements in the field of view exhibited by a typical patient. The document proposes acquiring an image of the patient's position in the field of view at approximately the same time that the initial X-ray image is acquired. If it is found necessary to acquire a subsequent X-ray image with updated field of view settings (e.g. collimation parameters), the patient's movements at the point of taking the second image are factored into the provision of the updated field of view settings. [Means for solving the problem]

[0010] According to one aspect of the present disclosure, an x-ray imaging system is provided. The x-ray imaging system includes an x-ray source, an x-ray detector, a depth sensor, and a processor. The x-ray source and the x-ray detector are separated by an examination region for performing an x-ray imaging operation on an object when the object is received within the examination region. The depth sensor generates depth sensor data representative of a three-dimensional surface of the object when the object is received within the examination region. The processor is configured to: Receiving depth sensor data; Identifying one or more internal structures within the subject based on a comparison of the depth sensor data to an anatomical model including the one or more internal structures; calculating, using the depth sensor data and the identified internal structure or structures, a surface projection of the internal structure or structures onto a three-dimensional surface of the object from a perspective of the x-ray source; outputting a pictorial representation of the surface projection for display as an overlay on the three-dimensional surface of the object; Do the following.

[0011] A surface projection of the internal structure is provided from the viewpoint of the X-ray source. The surface projection of the internal structure is therefore used to provide accurate guidance in locating the internal structure to be imaged under X-ray. As a result, the need to retake X-ray images is reduced, thereby limiting the X-ray dose to the subject.

[0012] Further aspects, features, and advantages of the present disclosure will become apparent from the following detailed description of examples which proceeds with reference to the accompanying drawings. [Brief description of the drawings]

[0013] [Figure 1] 1 is a schematic diagram illustrating a first view of an exemplary X-ray imaging system 100 including an X-ray source 110 and an X-ray detector 120, in accordance with some aspects of the present disclosure. [Diagram 2] 1 is a schematic diagram illustrating a second view of an exemplary X-ray imaging system 100 including an X-ray source 110 and an X-ray detector 120, in accordance with some embodiments of the present disclosure. [Diagram 3]FIG. 2 is a schematic diagram illustrating an example of an anatomical model 250 including an internal structure 180, according to some embodiments of the present disclosure. [Figure 4] 1 is a schematic diagram illustrating an exemplary configuration including an X-ray source 110, an X-ray detector 120, a depth sensor 130, and a processor 140 according to some aspects of the present disclosure. [Diagram 5] 1 is a schematic diagram illustrating an example of a surface projection 190 of an internal structure 180 onto a surface 170 of an object 160 from the perspective of an X-ray source 110, according to some embodiments of the present disclosure. [Figure 6] 1 is a schematic diagram illustrating a first example of a surface projection 190 displayed as an overlay on the surface of an object 160, according to some aspects of the present disclosure. [Figure 7] A schematic diagram showing a second example of a surface projection 190 displayed as an overlay on the surface of an object 160, according to some embodiments of the present disclosure. [Figure 8] 1 is a schematic diagram illustrating an example of an anatomical model including an internal structure 180 having an actual pose Pa and a desired pose Pd, according to some embodiments of the present disclosure. [Figure 9] 1 is a flowchart illustrating an example of a method for generating an image representation according to some aspects of the present disclosure. [Figure 10] 3 is a schematic diagram illustrating a first view of an exemplary X-ray imaging system 300 including an X-ray source 310 and an X-ray detector 320 according to a second set of aspects of the present disclosure. [Figure 11] 3 is a schematic diagram illustrating a second view of an exemplary X-ray imaging system 300 including an X-ray source 310 and an X-ray detector 320 according to a second set of aspects of the present disclosure. [Figure 12] 1 is a flowchart illustrating an example of a method for generating an image representation according to a second set of aspects of the present disclosure. [Figure 13] FIG. 4 is a schematic diagram illustrating an example of an anatomical model 430 including an internal structure 380 according to a second set of embodiments of the present disclosure. [Figure 14] 1 is a schematic diagram illustrating an example of a method for generating an image representation according to a second set of aspects of the present disclosure. [Figure 15]3 is a schematic diagram illustrating an exemplary configuration including an X-ray source 310, an X-ray detector 320, and a processor 340 according to a second set of aspects of the present disclosure. [Figure 16] A schematic diagram showing an example of a surface projection 390 of an internal structure 380 onto an estimated object surface 450 from the perspective of an X-ray source 310, according to a second set of aspects of the present disclosure. [Figure 17] 2 is a schematic diagram illustrating an example of an anatomical model including an internal structure 380 having an actual pose Pa and a desired pose Pd according to a second set of embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Examples of the present disclosure are provided with reference to the following description and figures. In this description, numerous specific details of several examples are described for the purpose of explanation. Reference herein to an "example," "implementation," or similar terminology means that a feature, structure, or characteristic described with respect to that example is included in at least that one example. It should also be understood that features described in relation to one example may also be used in other examples, and that for brevity, not all features are necessarily repeated in each example. For example, features described in relation to an X-ray imaging system may be implemented in a computer-implemented method and in a computer program product in a corresponding manner.

[0015] In the following description, reference is made to an X-ray imaging system, e.g., a DigitalDiagnost C90 sold by Philips Healthcare, Best, The Netherlands. Alternatively, the X-ray imaging system may be another type of X-ray imaging system. In some examples, the X-ray source of the X-ray imaging system may be mounted to a ceiling via a gantry, and the corresponding X-ray detector may be mounted to a stand and held in a vertical position. However, it should be understood that examples of the present disclosure are not limited to this particular configuration, and the X-ray source and X-ray detector may alternatively be mounted in different ways and held in different positions.

[0016] In the following description, reference is made to various methods implemented by a processor, i.e., a computer. It should be noted that the computer-implemented methods disclosed herein may be provided as a non-transitory computer-readable storage medium including computer-readable instructions stored thereon that, when executed by at least one processor, cause the at least one processor to perform the method. In other words, the computer-implemented methods may be implemented in a computer program product. The computer program product may be provided by dedicated hardware or by hardware capable of operating software in association with appropriate software. When provided by a processor, the operations performed in the methods may be provided by a single dedicated processor, or by a single shared processor, or by multiple individual processors, some of which may be shared. The operations performed in the methods may be provided by a processor shared within a networked processing architecture, such as, for example, a client / server architecture, the Internet, or the cloud.

[0017] Explicit use of the terms "processor" or "controller" should not be construed as referring solely to hardware capable of running software, and may implicitly include, but is not limited to, digital signal processor "DSP" hardware, read only memory "ROM" for storing software, random access memory "RAM", non-volatile storage devices, and the like. Furthermore, examples of the present disclosure may take the form of a computer usable storage medium, or a computer program product accessible from a computer readable storage medium, which provides program code for use by or in connection with a computer or any instruction execution system. For purposes of this description, a computer usable storage medium or a computer readable storage medium may be any apparatus that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. The medium may be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system or device or propagation medium. Examples of a computer-readable medium include semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory "RAM", a read-only memory "ROM", a rigid magnetic disk, and an optical disk. Current examples of optical disks include Compact Disk-Read Only Memory "CD-ROM", Compact Disk-Read / Write "CD-R / W", Blu-Ray™, and DVD.

[0018] As mentioned above, to avoid the need for repeated x-ray imaging and the associated increase in x-ray dose, it is important that the subject to be imaged is properly positioned relative to the x-ray imaging system.

[0019] FIG. 1 is a schematic diagram illustrating a first perspective of an exemplary X-ray imaging system 100 including an X-ray source 110 and an X-ray detector 120, according to some aspects of the present disclosure. The X-ray imaging system 100 also includes a depth sensor 130 and a processor 140. The X-ray source 110 and the X-ray detector 120 are separated by an examination region 150 for performing an X-ray imaging operation on an object 160 when the object is received within the examination region. The X-ray source and the X-ray detector are generally maintained in a stationary position during the imaging operation. The object may be, for example, a portion of a human body, or in fact, any object. In the illustrated example, the X-ray source 110 is attached to a ceiling via a gantry, and the X-ray detector 120 is attached to a stand and held in a vertical position. Alternative arrangements, mounting configurations, and positions of the X-ray source 110 and the X-ray detector 120 may also be used.

[0020] FIG. 2 is a schematic diagram showing a second perspective of an exemplary X-ray imaging system 100 including an X-ray source 110 and an X-ray detector 120 according to some aspects of the present disclosure. Compared to FIG. 1, the perspective shown in FIG. 2 more clearly shows the positions of the X-ray source 110 and the depth sensor 130. Also in FIG. 2, an exemplary object 160 in the form of a patient is received in an examination region 150 for performing an X-ray imaging operation on the patient. In FIG. 2, a chest X-ray imaging operation is being performed on the patient. The solid line extending between the X-ray source 110 and the X-ray detector 120 in FIG. 1 and FIG. 2 indicates a volumetric extent of overlap between the X-ray beam emitted by the X-ray source 110 and the X-ray radiation sensitive area of ​​the X-ray detector 120, within which X-ray image data is generated. This volumetric extent of overlap defines the examination region 150. The perimeter of the X-ray radiation sensitive area 230 of the X-ray detector 120 is marked on the radiation receiving surface of the X-ray detector as shown in FIG. 1. The X-ray detector 120 may also include one or more radiation dosimetry sensors for generating X-ray dose data. The locations of the radiation dosimetry sensors are shown as radiation dosimetry areas 240 on the radiation receiving surface of the X-ray detector 120. The radiation dosimetry sensors are sometimes referred to as automatic exposure control "AEC" chambers. In use, the radiation dosimetry sensors act to limit the exposure time of the X-ray exposure based on the detected radiation dose. In the illustrated example in FIG. 1, there are five radiation dosimetry areas 240, which have a circular shape. In other examples, the radiation dosimetry areas 240 may have a different shape, there may be a different number of such areas, or even none at all.

[0021] During use, it is desirable for the object 160 shown in Figure 2 to be properly positioned relative to the X-ray imaging system. More specifically, it is desirable for the object to be properly positioned relative to the X-ray radiation sensitive area 230 and / or one or more radiation dosimetry areas 240 of the X-ray detector 120 and the X-ray source 110. In the example shown in Figures 1 and 2, this is facilitated by the use of the depth sensor 130.

[0022] Generally, a depth sensor generates depth sensor data that represents the range between the depth sensor and a number of points on the surface of the object within the field of view of the depth sensor. Various types of depth sensors are known. These include depth cameras employing time-of-flight or LIDAR principles, depth cameras employing structured light principles, and depth cameras employing binocular stereo vision principles. The use of one or more of such depth sensors is contemplated for use in the examples shown in Figures 1 and 2. In some examples, a single depth sensor is used to observe the inspection area, thereby providing depth sensor data from a single viewpoint. However, the use of multiple depth sensors is also contemplated. Multiple depth sensors are positioned to observe the inspection area, thus providing depth sensor data from different viewpoints. Depth sensor data from different viewpoints may also be combined to provide depth sensor data from a single viewpoint, which may even be a different viewpoint than the viewpoints of the individual depth sensors.

[0023] In the time-of-flight or lidar principle, the time it takes for an emitted light pulse to travel from the camera's position in the scene to the object and back is used to generate depth sensor data in the form of image data representing the distance to the object's surface. The Azure Kinect DK depth camera and the Intel RealSense™ LiDAR Camera L515 are examples of depth cameras employing this principle. In the structured light principle, an optical pattern is projected onto the surface of an object in the scene, and the parallax between the original projected pattern and the pattern distorted by the object's surface is imaged by one or more cameras. In the binocular stereoscopic principle, different views of the scene are used to provide depth sensor data in the form of a depth map representing the scene. Some depth cameras are capable of generating optical image data representing a two-dimensional image of the object's surface, as well as depth sensor data. Such cameras are often referred to as RGB-D cameras. The optical image data represents the visible or infrared portion of the light spectrum. As will be described in more detail below, the use of such RGB-D cameras is also contemplated in the systems shown in Figures 1 and 2.

[0024] In the exemplary system 100 shown in Figures 1 and 2, the depth sensor 130 has a field of view whose minimum extent is indicated by a dotted line. In general, the field of view of the depth sensor 130 includes the X-ray detector and overlaps with a portion of the inspection region 150. In Figures 1 and 2, the depth sensor 130 is positioned such that a common surface of an object 160 received within the inspection region 150 is within the field of view of both the X-ray source 110 and the depth sensor 130. In doing so, the depth sensor 130 shown in Figures 1 and 2 is configured to generate depth sensor data representative of a three-dimensional surface 170 of the object 160 when the object is received within the inspection region 150.

[0025] In the exemplary configuration shown in Figures 1 and 2, the depth sensor 130 is mechanically coupled to the X-ray source 110. The depth sensor 130 is also offset with respect to an axis passing through the X-ray source 110 and the center of the X-ray detector 120. However, it should be noted that the depth sensor 130 may alternatively be positioned differently to generate depth sensor data representative of the three-dimensional surface 170 of the object 160 when the object 160 is received within the examination region 150. For example, the depth sensor 130 may be mechanically coupled to a wall or ceiling of a room in which the X-ray imaging system 100 is located. Alternatively, the depth sensor 130 may be mechanically coupled to a floor-mounted stand within the room. The depth sensor may alternatively be mobile. In some examples, the depth sensor may thus be movable around the room in which the X-ray imaging system 100 is located.

[0026] The processor 140 shown in Figures 1 and 2 is configured to receive depth sensor data. The processor 140 receives the depth sensor data via any form of digital communication. The processor 140 receives the depth sensor data directly or indirectly from the depth sensor 130. The processor 140 and the depth sensor 130 communicate with each other via a direct wired or wireless communication path, such as an electrical cable or Ethernet, or a wireless infrared or RF communication path, such as Bluetooth, as shown by the arrows connecting these components in Figures 1 and 2. Alternatively, the communication path between the depth sensor 130 and the processor 140 can be indirect. For example, the depth sensor 130 and the processor 140 communicate with each other via the Internet, via the cloud, or via a computer-readable storage medium.

[0027] Additionally, the processor 140 shown in FIGS. 1 and 2 also Identifying (S120) one or more internal structures 180 based on a comparison of the depth sensor data to an anatomical model including one or more internal structures 180 within the object 160; Using the depth sensor data and the identified internal structure(s) 180, calculating (S130) a surface projection 190 of the internal structure(s) onto the three-dimensional surface 170 of the object 160 from the perspective of the X-ray source 110; outputting (S140) a pictorial representation of the surface projection 190 for display as an overlay on the three-dimensional surface 170 of the object 160; Do the following.

[0028] The operations performed by the processor are illustrated in FIG. 9, which is a flow chart illustrating an example of a method for generating an image representation according to some aspects of the present disclosure. These operations result in a surface projection of a given internal structure from the viewpoint of the X-ray source. The surface projection is used to provide accurate guidance in locating the internal structure to be imaged under X-ray. As a result, the need to retake X-ray images is reduced, thereby limiting the X-ray dose to the object.

[0029] The operations performed by processor 140 are described in more detail below with reference to FIGS.

[0030] In operation S120, one or more internal structures 180 are identified based on a comparison between the depth sensor data and an anatomical model including one or more internal structures 180 within the object 160. FIG. 3 is a schematic diagram illustrating an example of an anatomical model 250 including an internal structure 180, according to some aspects of the present disclosure. The exemplary anatomical model 250 represents a knee and includes an internal bone structure 180. In general, however, the anatomical model may represent any part of the human body. The anatomical model may represent, for example, a limb, a torso, a hand, etc. The internal structure 180 may generally be any type of internal structure including bones and soft tissues such as organs, muscles, tendons, ligaments, etc. The anatomical model 250 includes surrounding tissue 260 that defines a surface of the anatomical model. The surrounding tissue may include, for example, muscle, fat, and skin. The depth sensor data represents a three-dimensional surface 170 of the object 160, and in operation S120, the internal structure 180 is identified by comparing the three-dimensional surface 170 of the object 160 represented by the depth sensor data with the surface of an anatomical model, or in other words, fitting the three-dimensional surface 170 of the object 160 represented by the depth sensor data to the surface of the anatomical model.

[0031] In some examples, the anatomical model 250 includes more than two dimensions. The anatomical model includes, for example, three spatial dimensions. The anatomical model may include three spatial dimensions and a fourth time dimension. In some examples, the anatomical model includes one or more kinematic joints, where the anatomical model includes at least one degree of freedom. The kinematic joints may have degrees of freedom such as rotation and sliding.

[0032] In some examples, the anatomical model 250 used in act S120 is provided by a plurality of X-ray images representing the object. The images represent different views of the object. In other examples, the anatomical model 250 is provided by a computer tomography image representing the object, or by a magnetic resonance image representing the object. In other words, the plurality of X-ray images, computer tomography images, and magnetic resonance images were generated from the object in the examination region, or the images were generated from the same type of object as the object in the examination region. In the former case, the X-ray images, computer tomography images, and magnetic resonance images are available from a previous imaging procedure on the object. Such images generally include those showing the surface of the anatomical form. Such images are available from a previous imaging procedure on the object. In these examples, the anatomical model is registered in act S120 to the depth sensor data representing the three-dimensional surface 170 of the object 160. More specifically, the comparison of the depth sensor data to the anatomical model includes registering the anatomical model to the depth sensor data representing the three-dimensional surface 170 of the object 160. The registration can be a rigid registration or a non-rigid registration. Registration thus includes operations such as translating, rotating, and scaling the size of the anatomical model to fit the anatomical model to the depth sensor data. The anatomical model is adapted to find the best fit between a portion of the object surface in the model and a corresponding portion of the object surface in the depth sensor data.

[0033] The anatomical model 250 used in act S120 is selected from a database of anatomical models representing the object or from an anatomical atlas. The anatomical model is selected based on various object parameters, such as the gender or dimensions of the object in the examination region 150. The anatomical model is selected automatically or by an operator. For example, the anatomical model is selected automatically by analyzing the depth sensor data to determine the object's dimensions and using the determined dimensions to select the anatomical model. The selected model is also registered in act S120 to the depth sensor data representing the three-dimensional surface 170 of the object 160, as described above for the example of the anatomical model 250 provided by a computed tomography image.

[0034] In one example, the operation S120 of identifying one or more internal structures in the object and the operation S130 of calculating the surface projection 190 of the one or more internal structures are performed using a machine learning algorithm. In this example, the machine learning algorithm is trained to predict the surface projection of the internal structures from data representing the surface of the object. The training data in this example is synthetically provided from magnetic resonance images. For example, the ground truth data may include synthetic depth sensor data corresponding to a view of the surface of the object from the position of the depth sensor and synthetic surface projection data generated by projecting the internal structures from the magnetic resonance images onto the surface of the object as viewed from the position of the x-ray source.

[0035] In doing so, act S120 results in the identification of internal structures that are not visible through mere visual inspection of the object's surface.

[0036] In operation S130, a surface projection 190 of one or more internal structures on the surface 170 of the object 160 is calculated using the depth sensor data and the identified one or more internal structures 180 from the perspective of the X-ray source 110. This operation is described with reference to FIG. 4, which is a schematic diagram illustrating an example configuration including the X-ray source 110, the X-ray detector 120, the depth sensor 130, and the processor 140 according to some aspects of the present disclosure. The configuration illustrated in FIG. 4 corresponds to a cross-sectional view through the schematic diagrams illustrated in FIGS. 1 and 2, and further includes an internal structure 180 within the object 160. The internal structure 180 may represent, for example, a bone or another type of internal structure. In the configuration illustrated in FIG. 4, the X-ray source and the X-ray detector are separated by an examination region 150, as in FIGS. 1 and 2. The X-ray source 110 and the X-ray detector 120 are similarly used to perform an X-ray imaging operation on the object 160 disposed within the examination region 150. The X-ray source 110 has a solid angle Ω x The X-ray source 110 generates X-ray radiation within a volume beam defined by x-ray radiation, which is detected by the X-ray detector 120 to generate X-ray image data. A depth sensor 130 is positioned at an offset position relative to an axis passing through the X-ray source 110 and the center of the X-ray detector 120. The depth sensor 130 has a minimum extent that subtends a solid angle Ω with the dotted line. ds 1 and 2, the depth sensor 130 is configured to generate depth sensor data representative of a three-dimensional surface 170 of the object 160 when the object is received within the inspection region 150.

[0037] 4, in operation S130, a surface projection 190 of one or more internal structures 180 from the viewpoint of the X-ray source 110 is calculated by back-projecting the positions of the internal structures 180 identified in the model onto the three-dimensional surface 170 of the object 160 given by the depth sensor data along the virtual path of the X-rays emitted by the X-ray source 110. This projection is performed in a mathematical sense and is shown as arrowed lines converging towards the X-ray source 110 in FIG. 4, the arrowed lines representing the virtual path of the X-rays. In one example, a correction of the offset position of the depth sensor 130 with respect to the X-ray source is also applied. In this example, the surface projection is calculated further based on a transformation related to the relative positions of the depth sensor 130, the X-ray source 110, and the X-ray detector 120. The determination of these relative positions is described in more detail below.

[0038] Figure 5 is a schematic diagram illustrating an example of a surface projection 190 of an internal structure 180 on a surface 170 of an object 160 from the perspective of an x-ray source 110, according to some embodiments of the present disclosure. The surface projection 190 shown in Figure 5 is produced by the configuration shown in Figure 4. Thus, two internal structures 180 that extend in the plane of the drawing in Figure 4 appear as horizontal bands in Figure 5 when projected onto the surface 170 of the object 160.

[0039] The effect of providing a surface projection from the perspective of the X-ray source is to provide an accurate depiction of the internal structure as it appears on the surface of the object as seen from the X-ray source 110. This type of view is sometimes called a beam eye view because it provides a projected view of the internal structure from the perspective of the beam emitted by the X-ray source. The surface projection 190 is used first to verify that the correct internal structure is within the field of view of the X-ray imaging system before exposing the object to X-rays. Secondly, the surface projection 190 is used to verify that the internal structures are aligned in the correct manner. For example, an operator can easily tell from the surface projection 190 in FIG. 5 whether the two internal structures 180 in FIG. 4 are separated or whether the object 180 should be rotated to prevent their surface projections from overlapping each other. In both cases, the surface projection is therefore used to limit the X-ray dose to the object by reducing the need to retake X-ray images.

[0040] In act S140, the image representation of the surface projection 190 is output for display as an overlay on the surface 170 of the object 160. The image representation 190 may be output, for example, to a display or to a projector.

[0041] In one example, the X-ray imaging system 100 shown in Figures 1, 2 and 4 further includes a display 200, and the processor 140 is further configured to output an overlay image to the display, the overlay image including the image representation of the surface projection 190 and the depth sensor data representing the surface of the object. By providing the overlay image with the image representation of the surface projection 190 and the depth sensor data, a topographical image of internal features on the surface of the object may be provided to the operator to improve understanding of the positioning of the object relative to the X-ray imaging system. The overlay image is generated by techniques such as setting pixel values ​​of one of the two images semi-transparent and combining corresponding pixel values ​​in the two images. Alternatively, corresponding pixel values ​​in the images are combined by replacing pixel values ​​in one image with corresponding pixel values ​​in the other image.

[0042] In another example, the X-ray imaging system 100 shown in Figures 1, 2 and 4 also includes a display 200. In this example, the depth sensor 130 is further configured to generate optical image data representative of a two-dimensional surface of the object. The depth sensor 130 is, for example, an RGB-D camera that provides depth data representative of the 3D surface of the object as well as optical image data representative of the two-dimensional surface of the object. In this example, the processor 140 is further configured to output an overlay image to the display, including the image representation of the surface projection 190 and the optical image data representative of the two-dimensional surface of the object. This example differs from the previous example in that instead of displaying the image representation as a 3D representation, it is displayed as a 2D representation. The 2D representation provides a view of the patient that is easier to interpret.

[0043] In another example, the X-ray imaging system 100 shown in Figures 1, 2 and 4 also includes a projector 210. In this example, the processor 140 is further configured to output an image representation of the surface projection 190 to the projector for display as an overlay on the surface of the object. Various optical image projectors may be used for this purpose, such as, for example, the VPLlaser projector sold by Sony Europe, BV, Weybridge, UK. This example is described with reference to Figure 6, which is a schematic diagram illustrating a first example of a surface projection 190 displayed as an overlay on the surface of the object 160, according to some aspects of the present disclosure. In the example shown in Figure 7, the object 160 is a patient and the internal structures are bones in the patient's torso. The surface projection 190 thus includes ribs, a portion of the spine, and a shoulder blade, which are projected onto the patient's torso. In the illustrated example, the surface projection 190 is provided in grayscale, but in other examples, one or more colors are used to indicate different internal structures.

[0044] In some examples, additional information is also included in the surface projection 190 to provide further guidance to the operator in aligning the object 160 with respect to the X-ray imaging system. This additional information may be output to the display 200 or to the projector 210 described above. For example, an image representation of the dosimetry region 240 and / or the radiation sensitive region 230 of the X-ray detector, and / or an image representation of the collimation window are also output. By way of example, FIG. 7 is a schematic diagram illustrating a second example of the surface projection 190 displayed as an overlay on the surface of the object 160, according to some aspects of the present disclosure. In this example, image representations of both the radiation sensitive region 230 and the dosimetry region 240 are projected onto the surface of the patient. Providing this additional information is useful because when the object is placed in the examination region 150, the object generally obscures the markings on the surface of the X-ray detector 120 indicating the extent of these regions. It is therefore ensured that the desired internal structures, such as the ribs in FIG. 7, are imaged during the subsequent X-ray imaging operation.

[0045] In the above example, the surface projection of the internal structure is provided from the perspective of the X-ray source 110. This type of view is sometimes referred to as a beam eye view, since it provides a view from the perspective of the beam emitted by the X-ray source. However, this type of view may not appear intuitive, especially when output to a display as an overlay image on an image generated by a camera offset with respect to an axis passing through the X-ray source and the center of the X-ray detector 120. A more intuitive view is provided by generating an image representation of the projected overlay image data 220 from the perspective of the depth sensor 130. Thus, in one example, the X-ray imaging system 100 shown above in Figures 1, 2 and 4 further includes a display 200 and a processor 140, and further includes: generating overlay image data representing the image representation of the surface projection 190 and depth sensor data representing the three-dimensional surface 170 of the object 160; projecting the overlay image data onto a radiation receiving surface of an X-ray detector from a perspective of an X-ray source; generating an image representation of the projected overlay image data 220 on the radiation receiving surface of the X-ray detector 120 from the perspective of the depth sensor 130; outputting the generated image representation of the projected overlay image data 220 to the display 200; Do the following.

[0046] The operations involved in this example are described in more detail with reference to FIG. 4. In this example, the first step of generating the overlay image data involves combining the depth sensor data with a surface projection 190. This overlay image data can be visualized as a thick curved black line on the surface of the object 160 in FIG. 4. The second step can be visualized as a projection of a thick curved black line onto the surface of the X-ray detector to give a thick straight black line on the surface of the detector 120. The third step can be visualized as an image representation of a thick straight black line from the perspective of the depth sensor. These projections are performed in a mathematical sense and are calculated based on the relative positions of the depth sensor 130, the X-ray source 110, and the X-ray detector 120.

[0047] The relative positions of the depth sensor 130, the X-ray source 110, and the X-ray detector 120 are determined from calibration data, or alternatively, they are determined from position sensor data. The calibration data represents discrete positions of each of the depth sensor 130, the X-ray source 110, and the X-ray detector 120. The discrete positions are selectable. For example, the height of the X-ray detector is selectable from one of an integer number of fixed mechanical positions, and the sensor indicates which of those positions is currently selected. Alternatively, different types of position sensors may be used to provide position sensor data representing the relative positions of the depth sensor 130, the X-ray source 110, and the X-ray detector 120. For example, position sensors employing optical, radio frequency "RF" or ultrasonic tracking techniques may be used. Examples of suitable position sensors include rotational and translational position encoders, laser-based optical range finders, RF and ultrasonic ranging transponders, and optical cameras configured to track the position of fiducial markers disposed on one or more of the X-ray source 110, the X-ray detector 120, and the depth sensor 130. Another example of a position sensor is a depth camera configured to track the position of one or more of the X-ray source 110, the X-ray detector 120, and the depth sensor 130.

[0048] In another example, the system 100 described above with reference to Figures 1, 2 and 4 outputs corrective actions to guide the operator to obtain a desired pose of the structure within the object. This allows the operator to more accurately position the object relative to the x-ray imaging system, thus helping to reduce the need to retake x-ray images. In this example, the processor 140 further comparing an actual pose of one or more identified internal structures within the object 160 to a desired pose of the one or more internal structures; Calculating one or more corrective actions to reduce a difference between the actual pose and the desired pose; Outputting corrective actions Do the following.

[0049] For this example, the actual pose P a and the desired posture P d Reference is now made to Fig. 8, which is a schematic diagram of an example of an anatomical model including an internal structure 180 having a P a represents an axis passing through the X-ray source and the center of the X-ray detector 120. In this example, the actual pose is determined from the position of one or more internal structures in the fitted anatomical model. The actual pose of the internal structure 180 is represented by the parameters of a six degree of freedom "6DOF" anatomical model, with an axis passing through the X-ray source as the reference orientation and the center of the detector as the reference position. Similarly, the desired pose P d is expressed as the desired position and orientation of the internal structures of the anatomical model. d is determined based on a clinical recommendation or based on a previous X-ray image of the object. The desired pose for the portion of the object is stored in a look-up table and selected by the operator during the X-ray imaging procedure. In the illustrated example, the desired pose P d is obtained by orienting the anatomical model over the angles Θ and Φ shown in FIG.

[0050] In this example, the operation of comparing the actual pose of one or more identified internal structures in the object 160 with the desired pose includes comparing parameters of a 6DOF model of the internal structures 180 in their current positions with parameters of the 6DOF model of the internal structures in their desired positions. The operation of calculating one or more corrective actions to reduce the difference between the actual pose and the desired pose includes determining the difference in one or more degrees of freedom represented by the model. These include translation and / or rotation of the internal structure and / or change in bending angle, the latter represented by the symbol δ in FIG. 8. The translation and / or rotation also include an associated magnitude and / or direction. These transformations are then output as corrective actions. For example, the transformations include corrective actions such as "rotate 10 degrees around the X axis" and "translate 20 centimeters along the Y axis". The operations of comparing the actual pose, calculating one or more corrective actions, and outputting the corrective actions may be performed once or they may be performed multiple times. For example, these operations are performed repeatedly until the difference between the actual pose and the desired pose is within a predetermined value.

[0051] As mentioned above, in some examples, the system 100 described above with reference to Figures 1, 2 and 4 includes a projector, and in other examples, the system 100 includes a display. The corrective action may be output to the display and / or to the projector. The corrective action may be output in textual form and / or in graphical form. A graphical form of the corrective action may include, for example, generating an arrow indicating the direction of the corrective action, with the size of the arrow indicating the magnitude of the corrective action. A graphical form of the corrective action may also be provided in the form of an animation. Instead of or in addition to outputting the corrective action to a display or projector, the corrective action may be output in audio form. In one example, one or more corrective actions are output to a projector for display on the object 160. In this example, the projector is aligned with the X-ray imaging system and thus with the object, such that the corrective action is displayed on the object itself. This provides the operator with more intuitive guidance in repositioning the object.

[0052] In one example, the positioning of the object is performed iteratively using an X-ray image representing the object. In this example, an operation S110 of receiving depth sensor data, an operation S120 of identifying one or more internal structures 180 in the object 160, an operation S130 of calculating a surface projection 190 of the one or more internal structures, and an operation S140 of outputting an image representation of the surface projection 190 are performed at least in a first iteration and a second iteration. In the first iteration, an anatomical model corresponding to the object is selected from an anatomical atlas or from a database of anatomical models representing the object 160, the selected anatomical model is registered to the object 160, and the first iteration further includes generating an X-ray image representing the object 160. In the second iteration, the anatomical model is provided by the X-ray image generated during the first iteration.

[0053] In this example, the X-ray image used in the second iteration is registered to the object as described above. In this example, the second iteration results in a more accurate depiction of the internal structure. This example therefore provides more accurate guidance, which further helps to limit the total number of X-ray images ultimately collected to obtain the desired X-ray image of the object. This therefore reduces the X-ray dose to the object and also improves workflow.

[0054] According to other examples, a computer-implemented method and a computer program product are also provided. An exemplary method is described with reference to Figure 9, which is a flow chart illustrating an example of a method for generating an image representation according to some aspects of the present disclosure.

[0055] 9, there is provided a computer-implemented method for generating an image representation using an X-ray source 110, an X-ray detector 120, and a depth sensor 130 separated by an examination region 150 for performing an X-ray imaging operation on an object 160 when the object is received within the examination region, and the depth sensor 130 generates depth sensor data representative of a three-dimensional surface 170 of the object 160 when the object is received within the examination region 150. The computer-implemented method comprises: A step S110 of receiving depth sensor data; A step S120 of identifying one or more internal structures 180 based on a comparison of the depth sensor data with an anatomical model including one or more internal structures 180 within the object 160; a step S130 of calculating, from the viewpoint of the X-ray source 110, a surface projection 190 of the one or more internal structures 180 onto the three-dimensional surface 170 of the object 160 using the depth sensor data and the identified one or more internal structures 180; a step S140 of outputting a pictorial representation of the surface projection 190 for display as an overlay on the three-dimensional surface 170 of the object 160; has.

[0056] 9, there is also provided a computer program comprising instructions that, when executed by one or more processors, cause the one or more processors to perform a method of generating an image representation using an X-ray source 110, an X-ray detector 120 and a depth sensor 130 separated by an examination region 150 for performing an X-ray imaging operation on an object 160 when the object is received within the examination region, and the depth sensor 130 generates depth sensor data representative of a three-dimensional surface 170 of the object 160 when the object is received within the examination region 150. The method comprises: receiving depth sensor data; Identifying one or more internal structures 180 based on a comparison of the depth sensor data to an anatomical model including one or more internal structures 180 within the object 160; using the depth sensor data and the identified internal structure(s) 180 to calculate a surface projection 190 of the internal structure(s) 180 onto the three-dimensional surface 170 of the object 160 from the perspective of the X-ray source 110; outputting a pictorial representation of the surface projection 190 for display as an overlay on the three-dimensional surface 170 of the object 160; has.

[0057] A second set of aspects of the present disclosure will now be described. These relate to Examples 1-12 listed below. The second set of aspects relates to an alternative system for improving the method of positioning an object relative to an X-ray imaging system, which will be described in more detail below with reference to Figures 10-17. In this second set of aspects, a surface projection of the internal structure is also generated. In contrast to the examples described above, this surface projection is generated without the need for a depth sensor.

[0058] Example 1. an X-ray source (310); An X-ray detector (320); Processor (340) and An X-ray imaging system (300) comprising: the x-ray source (310) and the x-ray detector (320) are separated by an inspection region (350) for performing an x-ray imaging operation on an object (360) when the object (360) is received within the inspection region (350); A processor (340): Receiving (S310) X-ray image data, the X-ray image data representing one or more internal structures (380) within an object (360) when the object (360) is received within an examination region (350); Receiving (S320) an anatomical model (430) representing an object (360), the anatomical model including one or more internal structures (380) and surrounding tissue (440) defining a surface of the anatomical model; Mapping (S330) one or more internal structures (380) from the anatomical model (430) to one or more corresponding internal structures (380) represented in the X-ray image data, such that a surface of the anatomical model (430) provides an estimated object surface (450) for the one or more internal structures (380) represented in the X-ray image data; Calculating (S340) a surface projection (390) of one or more internal structures (380) onto the estimated object surface (450) from the perspective of the X-ray source (310) using the anatomical model (430); outputting (S350) a pictorial representation of the surface projection (390) for display as an overlay on the surface (370) of the object (360); The X-ray imaging system (300) performs the above steps.

[0059] Example 2. The X-ray imaging system of Example 1, further comprising a projector (410), and the processor (340) further outputs an image representation of the surface projection (390) to the projector for display as an overlay on the surface (370) of the object (360).

[0060] Example 3. The X-ray imaging system (100) further comprises: A camera (330), Display (400) and Equipped with The camera (330) generates optical image data representative of a two-dimensional surface (370) of the object (360) when the object (360) is received within the inspection region (350); An X-ray imaging system as described in Example 1 or Example 2, wherein the processor (340) further outputs to the display (400) an overlay image including an image representation of the surface projection (390) and optical image data representing the two-dimensional surface (370) of the object (360).

[0061] Example 4. An X-ray imaging system as described in Example 1, wherein the operations of mapping one or more internal structures (S330) and calculating the surface projection of the one or more internal structures (S340) are performed using a machine learning algorithm.

[0062] Example 5. The processor (340) further comprises: comparing an actual pose of one or more identified internal structures (380) within the object (360) to a desired pose of the one or more internal structures; Calculating one or more corrective actions to reduce a difference between the actual pose and the desired pose; Outputting corrective actions The X-ray imaging system according to Example 1,

[0063] Example 6. The display (400) and / or projector (410) are further included, and the processor (340) further comprises: An X-ray imaging system as described in Example 5, which outputs one or more corrective actions to a display (400) and / or outputs one or more corrective actions to a projector (410) for display on the object (160).

[0064] Example 7. An X-ray imaging system as described in Example 5 or 6, wherein the one or more corrective actions represent one or more translations and / or one or more rotations, and the one or more corrective actions include a magnitude and / or a direction.

[0065] Example 8. The X-ray imaging system of any one of Examples 1 to 7, wherein the anatomical model includes three or more dimensions.

[0066] Example 9. The X-ray imaging system of example 8, wherein the anatomical model includes one or more kinematic joints, and wherein the anatomical model includes at least one degree of freedom.

[0067] Example 10. The X-ray imaging system of any one of Examples 1 to 9, wherein the anatomical model (430) is provided by one or more of a computed tomography image representative of the object and a magnetic resonance image representative of the object.

[0068] Example 11. The X-ray imaging system of any one of Examples 1 to 9, wherein the processor (340) further selects an anatomical model from an anatomical atlas or from a database of anatomical models that represents the subject.

[0069] As mentioned above, to avoid the need for repeated x-ray imaging and the associated increase in x-ray dose, it is important that the subject to be imaged is properly positioned relative to the x-ray imaging system.

[0070] FIG. 10 is a schematic diagram illustrating a first perspective of an exemplary X-ray imaging system 300 including an X-ray source 310 and an X-ray detector 320 according to a second set of aspects of the present disclosure. The X-ray imaging system 300 also includes a processor 340. The X-ray source 310 and the X-ray detector 320 are separated by an inspection region 350 for performing an X-ray imaging operation on an object 160 when the object is received within the inspection region 350. The X-ray source and the X-ray detector are generally maintained in a stationary position during the imaging operation. The object may be, for example, a portion of a human body, or in fact, any object. In the illustrated example, the X-ray source 310 is attached to a ceiling via a gantry, and the X-ray detector 320 is attached to a stand and held in a vertical position. Alternative arrangements, mounting configurations, and positions of the X-ray source 310 and the X-ray detector 320 may also be used.

[0071] FIG. 11 is a schematic diagram showing a second view of an exemplary X-ray imaging system 300 including an X-ray source 310 and an X-ray detector 320 according to a second set of aspects of the present disclosure. In comparison with FIG. 10, the view shown in FIG. 11 also includes an exemplary object 360 in the form of a patient. The patient is received in an examination region 350 for performing an X-ray imaging operation on the patient. In FIG. 11, a chest X-ray imaging operation is being performed on the patient. The solid line extending between the X-ray source 310 and the X-ray detector 320 in FIG. 10 and FIG. 11 indicates a volumetric extent of overlap between the X-ray beam emitted by the X-ray source 310 and the X-ray radiation sensitive region of the X-ray detector 320, within which the X-ray image data is generated. This volumetric extent of overlap defines the examination region 350. The periphery of the X-ray radiation sensitive region 530 of the X-ray detector 320 is marked on the radiation receiving surface of the X-ray detector, as shown in FIG. 10. The X-ray detector 320 may also include one or more dosimetry sensors for generating X-ray dose data. The locations of the dosimetry sensors are shown as dosimetry areas 540 on the radiation receiving surface of the X-ray detector 320. The dosimetry sensors are sometimes referred to as automatic exposure control "AEC" chambers. In use, the dosimetry sensors act to limit the exposure time of the X-ray exposure based on the detected radiation dose. In the illustrated example in FIG. 10, there are five dosimetry areas 540, which have a circular shape. In other examples, the dosimetry areas 540 may have a different shape, there may be a different number of such areas, or even none at all.

[0072] During use, it is desirable for the object 360 shown in Figure 11 to be properly positioned relative to the X-ray imaging system. More particularly, it is desirable for the object to be properly positioned relative to the X-ray radiation sensitive area 530 and / or one or more radiation dosimetry areas 540 of the X-ray detector 320 and the X-ray source 310. In the example shown in Figures 10 and 11, this is facilitated using the X-ray image data and an anatomical model. With reference to Figures 10 and 11, the processor 340: Receiving (S310) X-ray image data, the X-ray image data representing one or more internal structures 380 within an object 360 when the object 360 is received within an examination region 350; Receiving (S320) an anatomical model 430 representing an object 360, the anatomical model including one or more internal structures 380 and surrounding tissue 440 defining a surface of the anatomical model; Mapping (S330) one or more internal structures 380 from the anatomical model 430 to one or more corresponding internal structures 380 represented in the X-ray image data, such that a surface of the anatomical model 430 provides an estimated object surface 450 for the one or more internal structures 380 represented in the X-ray image data; Calculating (S340) a surface projection 390 of one or more internal structures 380 onto an estimated object surface 450 from the viewpoint of the X-ray source 310 using the anatomical model 430; outputting (S350) an image representation of the surface projection 390 for display as an overlay on the surface 370 of the object 360; Do the following.

[0073] The operations performed by the processor are illustrated in FIG. 12, which is a flowchart illustrating an example of a method for generating an image representation according to a second set of aspects of the present disclosure.

[0074] 12, in operation S310, X-ray image data is received. The X-ray image data represents one or more internal structures 380 in the object 360 when the object 360 is received in the examination region 350. The X-ray image data is generated using the X-ray imaging system 300. The X-ray image data is thus received from the X-ray imaging system 300. Alternatively, the X-ray image data may be provided by pre-recorded X-ray images generated for the object hours, days, weeks, months, or even years ago. Thus, the X-ray image data may be received from a database. In an example where the X-ray image data is generated using the X-ray imaging system 300, the X-ray image data is provided by a scout scan that is generated to provide guidance on how to generate a subsequent desired X-ray image. The scout scan may be generated using a lower X-ray dose than that used for the subsequent images. Alternatively, the x-ray image data is provided by an x-ray image using a standard radiation dose, from which the operator attempts to provide an improved x-ray image.

[0075] Referring to FIG. 12, in operation S320, an anatomical model representing an object is received. FIG. 13 is a schematic diagram illustrating an example of an anatomical model 430 including an internal structure 380 according to a second set of aspects of the present disclosure. The exemplary anatomical model 430 represents a knee and includes an internal bone structure 380. In general, however, the anatomical model may represent any part of the human body. The anatomical model may represent, for example, a limb, a torso, a hand, etc. The internal structure 380 may generally be any type of internal structure including bones and soft tissues such as organs, muscles, tendons, ligaments, etc. The anatomical model 430 includes surrounding tissue 440 that defines a surface of the anatomical model. The surrounding tissue may include, for example, muscle, fat, and skin.

[0076] In some examples, the anatomical model 430 includes more than two dimensions. The anatomical model includes, for example, three spatial dimensions. The anatomical model may include three spatial dimensions and a fourth time dimension. In some examples, the anatomical model includes one or more kinematic joints, and the anatomical model includes at least one degree of freedom. The kinematic joints may have degrees of freedom such as rotation and sliding. In some examples, the anatomical model is provided by a computed tomography image representative of the object. In other examples, the anatomical model is provided by a magnetic resonance image representative of the object. In other words, the computed tomography image and the magnetic resonance image were generated from the object in the examination region, or the images were generated from the same type of object as the object in the examination region. In the former case, the computed tomography image and the magnetic resonance image are available from a previous imaging procedure on the object.

[0077] In some examples, the processor 340 is also configured to select an anatomical model 430 from an anatomical atlas or from a database of anatomical models representing the subject. The anatomical model is selected based on various subject parameters, such as the gender or dimensions of the subject in the examination region 350. The anatomical model may be selected based on the received X-ray image data. For example, the X-ray image data is analyzed to determine dimensions of one or more of the internal structures represented in the X-ray image data, and the anatomical model is selected using the determined dimensions. In these examples, the anatomical model is selected automatically. However, in other examples, the model is selected based on user input. For example, an operator may specify that a pre-acquired computed tomography or magnetic resonance image of the subject should be used.

[0078] 12, in operation S330, one or more internal structures 380 from the anatomical model 430 are mapped to one or more corresponding internal structures 380 represented in the X-ray image data. The mapping performed in operation S330 includes the use of a rigid registration or a non-rigid registration. The registration thus includes performing one or more translation, rotation, or scaling operations on the anatomical model such that the internal structures in the anatomical model are matched to the corresponding structures in the X-ray image data. The mapping is performed such that the best match between the internal structures in the anatomical model and the anatomical structures represented in the X-ray image data is found. In doing so, bones in the anatomical model are matched to corresponding bones in, for example, the X-ray image data.

[0079] Operation S330 will be described with reference to FIG. 14, which is a schematic diagram illustrating an example of a method for generating an image representation according to a second set of aspects of the present disclosure. The operations illustrated in FIG. 14 correspond to those in the flowchart of FIG. 12. In operation S330 in FIG. 14, an internal structure 380 of an anatomical model 430 in the upper part of FIG. 14 is adapted to a corresponding internal structure 380 in the X-ray image data illustrated on the left side of FIG. 14. This results in an adapted anatomical model illustrated in the lower center part of FIG. 14. By performing this mapping operation, the surface of the anatomical model provides an estimated object surface 450 for one or more internal structures represented in the X-ray image data. The estimated object surface 450 is illustrated in dashed lines in the lower center part of FIG. 14.

[0080] 14 and 12, in operation S340, a surface projection 390 of one or more internal structures 380 on the estimated object surface 450 is calculated using the anatomical model 430. The surface projection 390 is calculated from the perspective of the X-ray source 310. An example of the surface projection 390 is shown in the right portion of FIG. 14, where the surface projection 390 is shown using a solid outline. Operation S340 is also described with reference to FIG. 15, which is a schematic diagram illustrating an exemplary configuration including an X-ray source 310, an X-ray detector 320, and a processor 340 according to a second set of aspects of the present disclosure. The configuration shown in FIG. 15 corresponds to a cross-sectional view through the schematic diagrams shown in FIGS. 10 and 11, and further includes an internal structure 380 within the object 360. The internal structure 380 represents, for example, bones and soft tissues such as organs, muscles, tendons, ligaments, etc. In the configuration shown in Figure 15, as in Figures 10 and 11, the X-ray source and X-ray detector are separated by an examination region 350. The X-ray source 310 and the X-ray detector 320 are similarly used to perform X-ray imaging operations on an object 360 disposed in the examination region 350. The X-ray source 310 is focused into a solid angle Ω x , which is detected by the X-ray detector 320 to generate X-ray image data.

[0081] 15, in operation S340, a surface projection 390 of one or more internal structures 380 is calculated on the estimated object surface 450 from the viewpoint of the X-ray source 310. The surface projection 390 is calculated by back projecting the position of the internal structures 380 in the model onto the estimated object surface 450 given by the anatomical model 430 along the virtual path of the X-rays emitted by the X-ray source 310. This projection is performed in a mathematical sense and is shown as arrowed lines converging towards the X-ray source 310 in FIG. 15, the arrowed lines representing the virtual path of the X-rays. The estimated object surface 450 is shown as a thick dashed curved black line and coincides with the actual surface 370 of the object shown as a thin black line around the object 360 in FIG. 15.

[0082] In this example, the plane of the X-ray detector is known from the X-ray image used in act S310, the position of the anatomical model relative to the X-ray image is known from the fitting process performed in act S330, and the solid angle Ω of the volume beam of the X-ray source is known. x is also known from the X-ray detector, so the virtual path of the X-ray can be traced.

[0083] Figure 16 is a schematic diagram illustrating an example of a surface projection 390 of an internal structure 380 onto an estimated object surface 450 from the perspective of an x-ray source 310 according to a second set of aspects of the present disclosure. The surface projection 390 shown in Figure 16 is generated by the configuration shown in Figure 15. Thus, two internal structures 180 that extend in the plane of the drawing in Figure 15 appear as horizontal bands in Figure 16 when projected onto the estimated object surface 450.

[0084] The effect of providing the surface projection 390 from the perspective of the X-ray source 310 is to provide an accurate depiction of the internal structure as it appears on the estimated object surface 450 as seen from the X-ray source 310. This type of view is sometimes referred to as a beam eye view, since it provides a projected view of the internal structure from the perspective of the beam emitted by the X-ray source. The surface projection 390 is used first to verify that the proper internal structure is within the field of view of the X-ray imaging system before exposing the object to X-rays. Secondly, the surface projection 390 is used to verify that the internal structures are aligned in the proper manner. For example, an operator can easily tell from the surface projection 390 in FIG. 16 whether the two internal structures 380 in FIG. 15 are separated or whether the object 380 should be rotated to prevent their surface projections from overlapping each other. In both cases, the surface projection is therefore used to limit the X-ray dose to the object by reducing the need to retake X-ray images.

[0085] In operation S350, the image representation of the surface projection 390 is output for display as an overlay on the surface 370 of the object 360. The image representation 190 is output, for example, to a display or to a projector. The surface projection of the internal structures provides guidance to the operator to obtain a desired x-ray image of the object. As a result, the need to retake x-ray images is reduced, thereby limiting the x-ray dose to the object.

[0086] In one example, the system 300 described with reference to Figures 10, 11 and 15 further includes a projector 410. In this example, the processor 340 is further configured to output an image representation of the surface projection 390 to the projector 410 for display as an overlay on the surface 370 of the object 360. Various optical image projectors may be used for this purpose, such as, for example, the VPLlaser projector sold by Sony Europe, BV, Weybridge, UK. In some examples, the surface projection is provided in grayscale. However, in other examples, it is contemplated to use one or more colors. For example, different colors are used to indicate different internal structures.

[0087] In one example, the operation S330 of mapping one or more internal structures and the operation S340 of calculating the surface projection of one or more internal structures are performed using a machine learning algorithm. In this example, the machine learning algorithm is trained to predict the surface projection of the internal structures from X-ray image data. The training data in this example is synthetically provided from computed tomography images or magnetic resonance images. The ground truth data may include synthetic X-ray projection data and synthetic surface projection data generated by projecting the internal structures from the computed tomography images or magnetic resonance images onto a virtual X-ray detector surface and onto the surface of the object, respectively, as viewed from the position of the X-ray source.

[0088] 10, 11 and 15 further includes a camera 330 and a display 400. In this example, the camera 330 is configured to generate optical image data representative of a two-dimensional surface 370 of the object 360 when the object 360 is received within the inspection region 350. The processor 340 is further configured to output an overlay image to the display 400, the overlay image including an image representation of the surface projection 390 and the optical image data representative of the two-dimensional surface 370 of the object 360.

[0089] In this example, the camera 330 has a field of view, the minimum extent of which is shown by the dotted lines in FIGS. 10, 11 and 15, and is also shown in FIG. 15 as a solid angle Ω cam 10, 11 and 15. Typically, the field of view of the camera 330 includes the x-ray detector and overlaps with a portion of the inspection region 350. In Figures 10, 11 and 15, the camera 330 is positioned such that a common surface of an object 360 received within the inspection region 350 is within the field of view of both the x-ray source 310 and the camera 330. In doing so, the camera 330 shown in Figures 10, 11 and 15 is configured to generate optical image data representative of a two-dimensional surface 370 of the object 360 when the object 360 is received within the inspection region 350.

[0090] In the exemplary configuration shown in Figures 10 and 11, the camera 330 is mechanically coupled to the X-ray source 310. The camera 330 is also offset with respect to an axis passing through the center of the X-ray source 310 and the X-ray detector 320. However, it should be noted that the camera 330 may alternatively be positioned differently to generate the optical image data. For example, the camera 330 may be configured to provide a view that is coaxial with the X-ray source by inserting a mirror in the path of the X-ray source 110. Alternatively, the camera 330 is mechanically coupled to a wall or ceiling of the room in which the X-ray imaging system 300 is located. Alternatively, the camera 330 may be mechanically coupled to a floor-mounted stand within the room. The camera 330 may alternatively be mobile. In some examples, the camera may thus be capable of being moved around the room in which the X-ray imaging system 300 is located.

[0091] The processor 340 shown in Figures 10, 11 and 15 is configured to receive optical image data generated by the camera. The processor 340 receives the optical image data via any form of digital communication. The processor 340 receives the optical image data directly or indirectly from the camera 330. The processor 340 and the camera 330 communicate with each other via a direct wired or wireless communication path, such as an electrical cable or Ethernet, or a wireless infrared or RF communication path, such as Bluetooth, as shown by the arrows connecting these components in Figures 10 and 11. Alternatively, the communication path between the camera 330 and the processor 340 can be indirect. For example, the camera 330 and the processor 340 communicate with each other via the Internet, via the cloud, or via a computer-readable storage medium.

[0092] In this example, the overlay image includes a representation of the surface projection 390 and the optical image data. The overlay image is generated by techniques such as setting pixel values ​​in one of the two images to semi-transparent and combining corresponding pixel values ​​in the two images. Alternatively, corresponding pixel values ​​in the images are combined by replacing pixel values ​​in one image with corresponding pixel values ​​in the other image.

[0093] In some examples, additional information is also included in the surface projection 390 to provide further guidance to the operator in aligning the object 360 with respect to the X-ray imaging system. This additional information may be output to the display 400 or to the projector 410 described above. For example, an image representation of the dosimetry area 540 and / or the radiation sensitive area 530 of the X-ray detector, and / or an image representation of the collimation window are also output. Providing this additional information is useful because when the object is placed in the examination area 350, the object will generally obscure the markings on the surface of the X-ray detector 320 that indicate the extent of these areas. Thus, it is ensured that the desired internal structures are imaged during the subsequent X-ray imaging operation.

[0094] In another example, additional information in the form of corrective actions is output. The corrective actions may be output to the display 400 or to the projector 410. Alternatively, the corrective actions are output as audio instructions. In one example, the corrective actions provide guidance to the operator to obtain a desired pose of the structures within the object. This allows the operator to more accurately position the object relative to the x-ray imaging system, thus helping to reduce the need to retake x-ray images. In this example, the processor 340 further comparing an actual pose of one or more identified internal structures 380 within the object 360 to a desired pose of the one or more internal structures; Calculating one or more corrective actions to reduce a difference between the actual pose and the desired pose; Outputting corrective actions Do the following.

[0095] For this example, the actual pose P a and the desired posture P d Reference is now made to Fig. 17, which is a schematic diagram of an example of an anatomical model including an internal structure 380 having a P a denotes an axis passing through the X-ray source and the center of the X-ray detector 320. In this example, the actual pose is determined by detecting the internal structure in the X-ray image data. The actual pose of the internal structure 380 is represented by parameters of a six degree of freedom "6DOF" anatomical model, with an axis passing through the X-ray source as the reference orientation and the center of the detector as the reference position. Similarly, the desired pose P d is expressed as the desired position and orientation of the internal structures of the anatomical model. d is determined based on a clinical recommendation or based on a previous X-ray image of the object. The desired pose for the portion of the object is stored in a look-up table and selected by the operator during the X-ray imaging procedure. In the illustrated example, the desired pose P d is obtained by orienting the anatomical model through the angles Θ and Φ shown in FIG.

[0096] In this example, the operation of comparing the actual pose of one or more identified internal structures in the object 360 with the desired pose includes comparing parameters of a 6DOF model of the internal structures 380 in their current positions with parameters of the 6DOF model of the internal structures in their desired positions. The operation of calculating one or more corrective actions to reduce the difference between the actual pose and the desired pose includes determining the difference in one or more degrees of freedom represented by the model. These include translation and / or rotation of the internal structure and / or change in bending angle, the latter represented by the symbol δ in FIG. 8. The translation and / or rotation also include an associated magnitude and / or direction. These transformations are then output as corrective actions. For example, the transformations include corrective actions such as "rotate 10 degrees around the X axis" and "translate 20 centimeters along the Y axis". The operations of comparing the actual pose, calculating one or more corrective actions, and outputting the corrective actions may be performed once or they may be performed multiple times. For example, these operations are performed repeatedly until the difference between the actual pose and the desired pose is within a predetermined value.

[0097] As mentioned above, in some examples, the system 300 described above with reference to Figs. 10, 11 and 15 includes a projector 410, and in other examples, the system 300 includes a display 400. The corrective actions can be output to the display and / or to the projector. The corrective actions can be output in textual and / or graphical format. A graphical form of the corrective action can include, for example, generating an arrow indicating the direction of the corrective action, with the size of the arrow indicating the magnitude of the corrective action. A graphical form of the corrective action can also be provided in the form of an animation. Instead of or in addition to outputting the corrective action to a display or projector, the corrective action can be output in audio format. In one example, one or more corrective actions are output to a projector for display on the object 360. In this example, the projector is aligned with the X-ray imaging system and thus with the object, such that the corrective actions are displayed on the object itself. This provides the operator with more intuitive guidance in repositioning the object.

[0098] According to other embodiments, a computer implemented method and a computer program product are also provided.

[0099] An exemplary method is described with reference to Figure 12. Referring to Figure 12, a computer-implemented method is provided for generating an image representation using an X-ray source 310 and an X-ray detector 320, the X-ray source 110 and the X-ray detector 120 being separated by an examination region 150 for performing an X-ray imaging operation on an object 160 when the object is received within the examination region. The computer-implemented method comprises: receiving S310 X-ray image data, the X-ray image data representing one or more internal structures 380 within the object 360 when the object 360 is received within the examination region 350; receiving S320 an anatomical model 430 representing an object 360, the anatomical model including one or more internal structures 380 and surrounding tissue 440 defining a surface of the anatomical model; a step S330 of mapping one or more internal structures 380 from the anatomical model 430 onto one or more corresponding internal structures 380 represented in the X-ray image data such that a surface of the anatomical model 430 provides an estimated object surface 450 for the one or more internal structures 380 represented in the X-ray image data; A step S340 of calculating a surface projection 390 of one or more internal structures 380 onto the estimated object surface 450 from the viewpoint of the X-ray source 310 using the anatomical model 430; a step S350 of outputting a pictorial representation of the surface projection 390 for display as an overlay on the surface 370 of the object 360; has.

[0100] 12, there is also provided a computer program comprising instructions that, when executed by one or more processors, cause the one or more processors to perform a method of generating an image representation using an X-ray source 310 and an X-ray detector 320, the X-ray source 110 and the X-ray detector 120 separated by an examination region 150 for performing an X-ray imaging operation on an object 160 when the object is received within the examination region. The method comprises: receiving S310 X-ray image data, the X-ray image data representing one or more internal structures 380 within the object 360 when the object 360 is received within the examination region 350; receiving S320 an anatomical model 430 representing an object 360, the anatomical model including one or more internal structures 380 and surrounding tissue 440 defining a surface of the anatomical model; a step S330 of mapping one or more internal structures 380 from the anatomical model 430 onto one or more corresponding internal structures 380 represented in the X-ray image data such that a surface of the anatomical model 430 provides an estimated object surface 450 for the one or more internal structures 380 represented in the X-ray image data; A step S340 of calculating a surface projection 390 of one or more internal structures 380 onto the estimated object surface 450 from the viewpoint of the X-ray source 310 using the anatomical model 430; a step S350 of outputting a pictorial representation of the surface projection 390 for display as an overlay on the surface 370 of the object 360; has.

[0101] The above examples should be understood as illustrating the present disclosure, but not as limiting. Further examples are contemplated. For example, the examples described with respect to an X-ray imaging system are also provided in a corresponding manner by a computer-implemented method, or by a computer program product, or by a computer-readable storage medium. It should be understood that features described with respect to any one example may be used alone or in combination with other described features, and may be used in combination with one or more features of another example or combination of other examples. Moreover, equivalents and modifications not described above may also be employed without departing from the scope of the invention as defined in the appended claims. In the claims, the word "comprises" does not exclude other elements or operations, and singular elements do not exclude a plurality. The mere fact that certain features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be advantageously used. Any reference signs in the claims should not be interpreted as limiting their scope.

Claims

1. An X-ray source, an X-ray detector, a depth sensor, and a processor, wherein the X-ray imaging system comprises: the X-ray source and the X-ray detector are separated by the inspection area for performing an X-ray imaging operation on the subject when the subject is received within the inspection area; the depth sensor generates depth sensor data representing a three-dimensional surface of the subject when the subject is received within the inspection area; the processor: receives the depth sensor data; identifies the one or more internal structures based on a comparison of the depth sensor data with an anatomical model including the one or more internal structures within the subject; uses the depth sensor data and the identified one or more internal structures to calculate a surface projection of the one or more internal structures on the three-dimensional surface of the subject provided by the depth sensor data from the viewpoint of the X-ray source; and outputs an image representation of the surface projection for display as an overlay on the three-dimensional surface of the subject. An X-ray imaging system that performs the above operations.

2. The X-ray imaging system according to claim 1, wherein the surface projection is calculated by back-projecting the position of the internal structure identified in the model along a virtual path of the X-rays emitted by the X-ray source onto the three-dimensional surface of the subject provided by the depth sensor data.

3. The X-ray imaging system according to claim 1, further comprising a display, wherein the processor further outputs an overlay image including the image representation of the surface projection and the depth sensor data representing the three-dimensional surface of the subject to the display.

4. The X-ray imaging system according to claim 1 or 2, further comprising a display, wherein the depth sensor further generates optical image data representing a two-dimensional surface of the subject, and the processor further outputs an overlay image including the image representation of the surface projection and the optical image data representing the two-dimensional surface of the subject to the display.

5. The X-ray imaging system according to claim 1, further comprising a projector, wherein the processor further outputs the image representation of the surface projection to the projector for display as an overlay on the surface of the subject.

6. The X-ray imaging system further comprises a display, and the processor further generates overlay image data representing the image representation of the surface projection and the depth sensor data representing the three-dimensional surface of the object, projects the overlay image data onto the radiation receiving surface of the X-ray detector from the viewpoint of the X-ray source, generates an image representation of the overlay image data projected onto the radiation receiving surface of the X-ray detector from the viewpoint of the depth sensor, outputs the generated image representation of the projected overlay image data to the display The X-ray imaging system according to claim 1, which performs the above operations.

7. The operation of identifying one or more internal structures within the object and the operation of calculating a surface projection of the one or more internal structures are executed using a machine learning algorithm. The X-ray imaging system according to claim 1.

8. The processor further compares the actual pose of the identified one or more internal structures within the object with the desired pose of the one or more internal structures, calculates one or more correction actions to reduce the difference between the actual pose and the desired pose, outputs the correction action The X-ray imaging system according to claim 1, which performs the above operations.

9. Further comprising a display and / or a projector, and the processor further outputs one or more correction actions to the display and / or outputs the one or more correction actions to the projector for display on the object. The X-ray imaging system according to claim 1.

10. The X-ray imaging system according to claim 8, wherein the one or more correction actions represent one or more translations and / or one or more rotations, and the one or more correction actions include magnitude and / or direction.

11. The X-ray imaging system according to claim 1, wherein the anatomical model is provided by one or more of a plurality of X-ray images representing the object, a computed tomography image representing the object, and a magnetic resonance image representing the object.

12. The processor further selects the anatomical model from a database of anatomical models representing the object or from an anatomical atlas, aligning the selected anatomical model with the depth sensor data representing the three-dimensional surface of the subject The X-ray imaging system according to claim 1, which performs the above operations. **Claim 13**: The operations of receiving the depth sensor data, identifying one or more internal structures within the subject, calculating surface projections of the one or more internal structures, and outputting an image representation of the surface projections are performed at least in a first iteration and a second iteration. In the first iteration, the anatomical model is selected from a database of anatomical models representing the subject or from an anatomical atlas, the selected anatomical model is aligned with the subject, and the first iteration further includes generating an X-ray image representing the subject. The X-ray imaging system according to claim 1, wherein in the second iteration, the anatomical model is provided by the X-ray image generated during the first iteration. **Claim 14**: A computer-implemented method for generating an image representation using an X-ray source, an X-ray detector, and a depth sensor, wherein the X-ray source and the X-ray detector are separated by the examination region for performing an X-ray imaging operation on the subject when the subject is received within the examination region, and the depth sensor generates depth sensor data representing the three-dimensional surface of the subject when the subject is received within the examination region. The computer-implemented method includes: receiving the depth sensor data; identifying the one or more internal structures based on a comparison between the depth sensor data and an anatomical model including the one or more internal structures within the subject; using the depth sensor data and the identified one or more internal structures to calculate surface projections of the one or more internal structures on the three-dimensional surface of the subject provided by the depth sensor data from the viewpoint of the X-ray source; outputting an image representation of the surface projections for display as an overlay on the three-dimensional surface of the subject The computer-implemented method having the above steps. **Claim 15**: A computer program comprising instructions which, when executed by one or more processors, cause the one or more processors to perform a method of generating an image representation using an X-ray source, an X-ray detector, and a depth sensor, wherein the X-ray source and the X-ray detector are separated by the inspection region for performing an X-ray imaging operation on a subject when the subject is received within the inspection region, the depth sensor generates depth sensor data representing a three-dimensional surface of the subject when the subject is received within the inspection region, and the method comprises: Receiving the depth sensor data; Identifying the one or more internal structures based on a comparison of the depth sensor data with an anatomical model including the one or more internal structures within the subject; Calculating a surface projection of the one or more internal structures on the three-dimensional surface of the subject provided by the depth sensor data from a viewpoint of the X-ray source using the depth sensor data and the identified one or more internal structures; Outputting an image representation of the surface projection for display as an overlay on the three-dimensional surface of the subject A computer program having the above.

Images