Patient motion detection in diagnostic imaging

Abstract

An apparatus (10), system, method and computer program product for detecting movement of an object in a diagnostic imaging examination are disclosed. The apparatus comprises a camera / 3D surface scanning system (11) and an input (14) for receiving a trigger indicating that a current spatial configuration of the object is to be maintained for inspection. The device comprises a processor (12) and an output (13), where the processor (12) is adapted to acquire reference data of the object using the camera / 3D system upon receipt of said trigger and to acquire further data of the object using the camera / 3D system after acquisition of said reference data. The processor is adapted to compare reference data representing a reference state of the object substantially upon receipt of the trigger with additional data representing an updated near state of the object. The processor is adapted to provide output data via said output (13), where the output data represents a comparison of further data with reference data in order to indicate a movement of the object relative to a reference state of the object.

Description

Technical Field

[0001] The present invention relates to the field of diagnostic imaging, and more specifically to an apparatus, a system, a workstation, a method and a computer program product for monitoring the positioning (spatial configuration) of an object (e.g., one or more body parts of a patient) in a diagnostic imaging environment, for example to check for unwanted movement of the patient before starting a medical imaging procedure (e.g., particularly after initial preparation of the patient) to ensure that correct positioning for the examination is achieved. Background Art

[0002] Preparing a patient for an examination by diagnostic imaging (e.g., to diagnose a medical condition and / or plan treatment) is often a complex task that relies heavily on manual intervention by trained and experienced personnel. Typically, a skilled operator (e.g., a medical technical assistant, MTA) needs to carefully perform preparation and setup for the session according to specific guidelines and / or established protocols regarding proper positioning of the patient and configuration of image acquisition (e.g., configuration of the diagnostic imaging system).

[0003] For example, in order to acquire an X-ray image, the patient needs to be appropriately positioned relative to the X-ray imaging system (and / or vice versa). This typically means that the patient needs to assume a specific posture relative to the image detector and the X-ray tube. For example, limbs and joints may need to be precisely positioned and / or oriented relative to the detector and tube, as indicated by the diagnostic purpose of the examination. After this positioning, the patient is typically instructed to avoid any additional movement, but X-ray images are typically not acquired immediately after positioning. For example, the operator may need to configure system parameters for acquisition and / or retreat to a safer distance and / or behind a radiation shield (or to a different room) to avoid exposure of the operator to potentially damaging radiation (even though the doses are typically very small, they may still accumulate over time in typical (e.g., daily) practice without appropriate protective measures).

[0004] However, if the patient moves after the correct posture has been assumed (e.g. as assisted and confirmed by an operator) but before image acquisition is performed, the X-ray image(s) may be unsuitable or suboptimal for the intended diagnostic purpose. Therefore, the image may need to be retaken in such a situation. Even if digital radiography technology fortunately allows rapid detection of whether the diagnostic quality of the acquired image is sufficient (e.g. within minutes, if not just seconds), it will be appreciated that repeating the image acquisition has several undesirable consequences: the patient is exposed to additional radiation doses that could have been avoided, and time and other resources (such as due to power drawn by the system in use and / or wear and tear on the system (e.g., X-ray tube)) will also be wasted unnecessarily.

[0005] Various methods that rely on optical (digital) camera imaging for improving the patient positioning process are known in the art. For example, it is known to display the target position of a limb or body part or more generally the target spatial configuration to the patient in combination with live camera imaging feedback in order to guide the patient to the correct position, see, for example, US2015 / 003674A1 and US2017 / 020469A1.

[0006] Thus, visual feedback can be provided to bring the patient to a desired position for a diagnostic imaging examination. However, such techniques may typically rely on specific processing to determine the correct (desired) position for the examination, such as in a reference frame associated with the diagnostic imaging system and / or optical camera (the correspondence between these coordinate systems is typically precisely established), to detect the current spatial configuration of the patient's (one or more) relevant body parts, and / or to compare the current position to the desired state. This may involve specific (potentially complex) processing techniques that may increase the cost of development, production and / or maintenance of the system, may limit the flexibility of the system (e.g., only supporting common examinations, typical body shapes and / or standard use cases), may create risks due to incorrect inference of information from the observed images (e.g., detection of one or more body parts), and / or may incorrectly determine the recommended spatial configuration for atypical patients (e.g., due to injuries, congenital conditions and / or other medical conditions).

[0007] Even though various methods are known in the art, including the examples above, to aid in the process of guiding a patient to a particular position required (or desired) for a particular imaging examination, there remains a need in the art to provide simple and robust methods for monitoring patient positioning, preferably without using (or at least minimally relying on) complex image recognition and / or processing techniques, and preferably also without relying on a predefined definition of and / or assumptions about the desired geometric state (i.e., "position" in its broadest sense), for example in a manner that is examination-agnostic and patient-agnostic, i.e., independent of the particular (type of) examination and / or patient.

[0008] When the operator has finished positioning the patient, other actions may be required before starting the acquisition process. For example, the X-ray equipment may be configured and / or fine-tuned for the examination, for example, the collimation and / or other settings of the X-ray beam may need to be adjusted. After the patient is set up and any configuration steps that need to be performed locally at the imaging site (e.g., where the X-ray tube and / or detector are located; it should be noted that remotely controllable settings and locally controllable settings may vary from system to system), the operator typically leaves the immediate vicinity of the patient (e.g., leaves the imaging site, such as in an acquisition room) to control the image acquisition process from a console (e.g., in a control room). Therefore, the actual X-ray image acquisition may occur after a non-negligible period of time has passed after the positioning setup. In addition, during the final steps before the acquisition is performed, the operator may not be able to observe the patient directly or in sufficient detail. Embodiments of the present invention can address the need for simple techniques to ensure that the patient does not move after the preparation steps (positioning of the patient), that is, to make the patient essentially frozen until the acquisition of (one or more) (e.g., X-ray) diagnostic images.

[0009] Document US2020 / 029919A1 discloses that proper positioning of a patient in an X-ray imaging system may present difficulties to medical professionals, both due to the small size of important anatomical aspects that need to be captured in the X-ray image and due to the significant movement in the field of view presented by a typical patient. Document US2020 / 029919A1 proposes obtaining an image of the patient's position in the field of view at approximately the same time as the initial X-ray image is obtained. If it proves necessary to obtain a subsequent X-ray image with an updated field of view setting (e.g., collimation parameters), the patient's movement when taking the second image is taken into account in providing the updated field of view setting.

[0010] Another document US2015 / 208981A1 discloses a method for providing information for magnetic resonance imaging "MRI", the method comprising: performing imaging of a patient by executing a sequence including multiple protocols; and providing threshold-related information about the patient's movement to the patient, the threshold-related information corresponding to a currently executed protocol among the multiple protocols.

[0011] Another document WO2021 / 136250A1 relates to a method for automatic image acquisition and imaging processing. The method may include obtaining imaging information of an object. The method may include at least determining target device positioning information of an imaging device based on the imaging information. The method may include positioning an imaging device to perform image acquisition based on the target device positioning information of the imaging device. The method may include providing guidance information based on the inspection information, the guidance information being configured to guide the positioning of the object. The method may also include obtaining a target image from an imaging operation by the imaging device. In addition, the method may include determining a target image processing algorithm for a medical image. Summary of the invention

[0012] It is an object of embodiments of the present invention to provide simple, good and / or efficient modules and methods for monitoring the spatial configuration (in a general sense, the "position") of an object (e.g., a patient, e.g., at least one or more body parts of the patient that are of particular interest for an imaging examination) in a diagnostic imaging environment, e.g., to detect undesired movement of the patient before starting a medical imaging procedure (such as an X-ray image acquisition). In particular, embodiments can be used to ensure that the correct positioning of the object for an imaging examination is maintained after initial preparations for setting up the patient have been performed (e.g., before performing the actual diagnostic image acquisition).

[0013] An advantage of embodiments of the present invention is that poor diagnostic image quality (e.g., insufficient for diagnostic purposes) and / or reshooting of diagnostic images due to movement of the patient after positioning and before acquisition can be avoided or reduced. It is also advantageous that additional radiation dose exposure of the patient due to reshooting of diagnostic images can be avoided or reduced (e.g., at least statistically). Similarly, it is also advantageous that radiation exposure of sensitive organs or tissues that are positioned outside the collimated radiation field (e.g., not of interest for the intended examination) but exposed due to undesired movement of the patient can be reduced or avoided. Waste of resources due to unnecessary reshooting of diagnostic images, such as waste of time, electricity, wear and tear of the system, etc., can also be reduced or avoided.

[0014] Furthermore, if the operator does not promptly detect suboptimal positioning of the patient at the time of diagnostic image acquisition, the risk of an incorrect or uncertain diagnosis may occur, which may also be reduced by embodiments of the present invention. It should be appreciated that the costs associated with inviting the patient back to the facility to repeat the examination at a later time (e.g., date) (in view of poor diagnostic image quality that was not noticed at the time of earlier acquisition) may be even higher than if an immediate retake of the image were to be performed.

[0015] An advantage of embodiments of the present invention is that a camera system (e.g., one or more cameras) can be used that does not need to be specifically and / or accurately aligned with a diagnostic imaging system, for example, without accurately determining the correspondence between (one or more) camera coordinate systems and the diagnostic imaging coordinate system, such as through a calibration process. For example, in a system that automatically determines a desired spatial configuration of a patient and / or guides the patient (possibly via interaction with an operator) to that desired configuration, an accurate mapping is typically required to relate camera observations to the coordinate system of the (one or more) diagnostic images that are intended to be captured. However, embodiments of the present invention can monitor for signs of (any substantial) movement of the patient after receiving a trigger indicating that the desired position has been achieved, such that knowledge of the coordinate system of the diagnostic imaging system (and therefore knowledge of the diagnostic images to be acquired) is not required. This may result in a method that is very robust (e.g., errors in coordinate mapping are avoided, and / or changes in camera alignment over time are not necessarily problematic) and simple to install and use (e.g., no detailed calibration process is required). In addition, the camera system does not need to be integrated into the diagnostic imaging system, for example, it can be installed as a simple add-on without the need for complex integration, or even any substantial co-integration (e.g., data connections, precise mechanical linkages, etc.) into the diagnostic imaging system.

[0016] An advantage of embodiments of the present invention is that a simple and robust method is applied to detect the movement of a patient (or one or more of his / her relevant body parts), which method does not rely on specific (e.g. complex) processing techniques to determine the correct (or at least expected) spatial configuration of the patient's body for a specific examination, to detect the current spatial configuration of the patient's body or its (one or more) relevant body parts (e.g., by fitting a model of the body or its parts to a live camera image), and / or to compare the current state with the expected state. By avoiding such complex processing techniques, the cost of development, production and / or maintenance of a system according to an embodiment can be kept low, and a high degree of flexibility can be provided, for example, the system does not (broadly) rely on a predefined expected (i.e., reference) state for a specific examination (e.g., the embodiment can be applied to essentially any examination without prior knowledge), nor on assumptions about the body shape, condition and / or other characteristics of a specific patient. It should be understood that a simple method that minimizes the use of assumptions and / or (machine-learned and / or encoded) prior knowledge can avoid or at least reduce the risks caused by incorrect estimation and / or inference of information (e.g., detection of body features, model fitting, ...). The flexibility thus provided advantageously allows the operator to use his / her experience and best judgment to position the patient without being constrained by reference conditions (patient position / spatial configuration) defined and / or determined by the system.

[0017] However, it should be noted that some (e.g., limited and / or simple) image processing is not necessarily excluded. For example, landmark features may be detected in the camera image(s) that may (preferably) be easily and robustly computed. The use of landmark features may allow for a more reproducible quantification and / or characterization of the patient's movement, e.g., checking the quantified degree of movement against a predetermined threshold of "tolerable" motion, without requiring detailed knowledge of the imaging procedure to be performed or the specific characteristics of the patient, such that methods according to embodiments may still be considered advantageously procedure-agnostic and patient-agnostic.

[0018] An advantage of embodiments of the present invention is that an arbitrary position (i.e., spatial configuration) of a patient (or one or more relevant body parts of him / her) can be used as a reference in order to detect movements, for example significant deviations of the patient's position from this reference, in a simple manner (e.g., without requiring programming, (machine) training and / or an explicit definition of a reference).

[0019] An advantage of embodiments of the present invention is that position information of the patient and / or position information of (one or more) specific body parts and / or movement information (e.g., changes in the position information) can be conveniently monitored by an operator, for example, using (one or more) overlays shown to the operator on a display. Additionally or alternatively, such information can be displayed to the subject, for example, in order to allow the subject to recognize changes in his / her position and correct such movements.

[0020] An advantage of embodiments of the present invention is that depth and / or 3D camera imaging can be used to allow changes in the spatial configuration of a patient to be detected in essentially three dimensions, for example, changes in position (and by extension, orientation, etc.) occurring in a direction perpendicular to the camera imaging plane (or at least having a substantial component in that depth direction) can be detected, in addition to in-plane changes that may also be detected using a conventional 2D camera.

[0021] Depth (e.g., 3D) imaging can also advantageously allow good detection of movement by increasing the separability and detectability of the patient (or, typically, his / her associated body part(s)) in the camera (depth / 3D) image(s), so that changes in spatial configuration can be more easily detected. Even if the body part(s) have poor image contrast with the background, they can typically be positioned at a distance from their background, as viewed from the camera's perspective, so that they can be easily detected and dynamically compared to a reference image (e.g., captured earlier upon receipt of a trigger to align the patient's position to a desired state for diagnostic image acquisition) (to detect changes in position, orientation, ...). It should be noted that this does not require any assumptions about the shape or other properties of the body part(s), only that it is sufficiently physically removed from the surrounding objects(s). Even if the body part is placed in contact with a surface (e.g., the plane of the detector or detector housing) (and the camera is viewing from a generally opposite direction, e.g., more or less from the vantage point of the x-ray tube, without limitation), it will be appreciated that the thickness of the (and in general any arbitrary) body part will cause it to figuratively and literally stand out in the acquired depth or 3D image. This also reduces false detections of movement due to, for example, moving shadows, lighting changes, and / or other such observable changes in a 2D-only (e.g., color or grayscale) image that are unrelated to actual changes in the position of the actual object (body part or parts).

[0022] Furthermore, it is an advantage of embodiments of the present invention that an objective assessment value may be determined to indicate whether repositioning of the object (e.g., returning to a patient positioning preparation phase) is required or would be advisable in order to obtain good diagnostic image quality, such as an X-ray image of sufficiently high quality for diagnostic purposes. Such an assessment may be performed by an operator (e.g., an MTA), but may also be determined automatically according to embodiments of the present invention (e.g., as an automatically generated recommendation to the operator). When the assessment as determined by a system or method according to an embodiment is negative, even the performance of the prepared imaging examination may be prevented, such as by disabling a button or other signal source to activate the acquisition process, so that the operator is not only alerted, but corrective action is also forced to be taken before the acquisition can be performed (it will be appreciated that in such a case an override option may be provided to allow the operator to proceed anyway, relying on his / her experience and professional judgment).

[0023] Even though the automatic generation of an assessment (i.e., a value indicating whether an observed change in patient positioning is likely to significantly degrade diagnostic image quality) may (e.g., optionally) take into account some (e.g., limited) a priori knowledge of the imaging procedure to be performed, either explicitly or implicitly, it should be understood that this is different from starting with a predefined procedure (e.g., using a geometric 3D model of the patient and / or an otherwise encoded definition of a preferred patient positioning for a particular procedure), automatically guiding the patient to a position prescribed for that procedure and comparing the patient's position to a reference, e.g., based on the model. Embodiments of the present invention allow an operator to position the patient in any manner he / she deems appropriate for the intended purpose, without any constraints imposed by an automated patient positioning guidance system. After the operator has indicated that the patient is correctly positioned, changes in position may be detected in an automated manner, and, for example, the operator (and / or the patient) may be alerted when the detected movement exceeds a predetermined threshold. A reference position (e.g., a snapshot image of the patient taken when the operator has indicated the end of positioning preparation) and a currently observed position (e.g., a live camera image) or information derived therefrom may be processed to determine the assessment value. For example, a machine learning algorithm (e.g., a trained artificial intelligence model) can use this information to estimate whether the diagnostic image taken at this time (i.e., the patient is positioned for observation) is (likely to be) of sufficient quality compared to the image originally intended by the operator (e.g., the reference image). However, this does not limit the flexibility or usability of systems and / or methods according to embodiments: the operator remains free to position the patient in any way that he / she deems appropriate in his / her best judgment when defining the reference state (e.g., the reference snapshot).

[0024] An advantage of embodiments of the present invention is that a partially automated (eg, machine-assisted) process is provided to routinely improve the efficiency of workflow and / or patient positioning tasks in diagnostic imaging examinations (eg, X-ray examinations, such as projection radiography examinations).

[0025] An advantage of embodiments of the present invention is that embodiments of the present invention may provide real-time monitoring of a patient's spatial configuration (e.g., position), such as substantially continuously updated (or at least updated at a reasonably high frequency) in order to detect movement (e.g., exceeding a predetermined threshold) away from an earlier spatial configuration of the patient selected as a reference.

[0026] An advantage of embodiments of the present invention is that the motion detection methods (or in other words, detection of changes in the spatial configuration of an object) provided by the embodiments can be applied in conjunction with any diagnostic imaging system (e.g., any generic, non-standard, third-party, custom and / or generally arbitrary imaging system) with no (or little) modification, registration, mathematical modeling and / or initial configuration. In other words, the method does not rely on knowledge of the characteristics, parameters, configuration, features and / or type of diagnostic imaging system, nor on knowledge of the diagnostic imaging procedure to be performed, nor on knowledge of the characteristics of the object. However, it should be understood that if deemed useful, some (e.g., limited) a priori knowledge about the system, examination and / or object may be used in accordance with some embodiments of the present invention (i.e., such embodiments are not necessarily excluded).

[0027] The method, computer program product, device, system and / or workstation according to the embodiments of the present invention achieve the above-mentioned objectives.

[0028] In a first aspect, the present invention relates to a method, such as a computer-implemented method, for detecting movement of at least one body part of a subject in a diagnostic imaging examination, such as an X-ray imaging examination, such as an X-ray projection radiography examination. The method comprises receiving a trigger signal to indicate that a current spatial configuration of the subject is to be maintained for the diagnostic imaging examination. The method comprises, when the trigger signal is received, acquiring a reference image and / or spatial data of the subject using a camera and / or a 3D surface scanning system. The method further comprises: after acquiring the reference image and / or spatial data of the subject using the camera and / or the 3D surface scanning system, acquiring a further image and / or spatial data of the subject using the camera and / or the 3D surface scanning system. The method further comprises: comparing the reference image and / or spatial data of the subject with further images and / or spatial data of the subject (or vice versa, further data with reference data), the reference image and / or spatial data representing a state of the subject substantially at the time the trigger was received, the further image and / or spatial data representing a more recent state of the subject. The method comprises providing an output to an operator and / or subject, the output representing said comparison of further image and / or spatial data with the reference image and / or spatial data to indicate movement of the subject relative to a reference state of the subject.

[0029] In a method according to an embodiment of the invention, an output may be provided via at least one display monitor and / or a human-machine interface device.

[0030] In a method according to an embodiment of the invention, the output may be provided as an image overlay, displaying the comparison in the form of a difference image as an overlay on the further image and / or spatial data.

[0031] In a method according to an embodiment of the present invention, additional images and / or spatial data of an object may be acquired repeatedly, periodically and / or substantially continuously to obtain a live stream of the additional images and / or spatial data, wherein the reference images and / or spatial data are repeatedly compared with the most recently acquired additional images and / or spatial data of the live stream, and the output is repeatedly provided to present a dynamic view of current changes in the spatial configuration of the object relative to an earlier state of the object.

[0032] In a method according to an embodiment of the present invention, a trigger signal may be received from an operator via a human interaction interface (eg, a button, a voice control interface, and / or a gesture detection system) to detect a gesture made by the operator.

[0033] A method according to an embodiment of the present invention may include detecting proximity of an operator to an object using radio frequency identification tag detection, an indoor positioning system, a beam gate, a sonar, a radar and / or a lidar system and / or another sensor system for presence, proximity and / or position detection, wherein the trigger signal is generated when the proximity detection indicates that the operator has left the vicinity of the object.

[0034] In a method according to an embodiment of the invention, a trigger signal may be received from an automated system for detecting a predetermined reference space configuration of an object as required for a diagnostic imaging examination in a live imaging stream from a camera and / or a 3D surface scanning system.

[0035] A method according to an embodiment of the present invention may include: acquiring a live imaging stream from a camera and / or a 3D surface scanning system, performing scene analysis of the live imaging stream to detect a predetermined condition, and generating the trigger in response, wherein the predetermined condition corresponds to an operator leaving the immediate vicinity of an object.

[0036] In a method according to an embodiment of the present invention, reference images and / or spatial data may be acquired at a predetermined short time at or after the moment of receiving a trigger, and / or reference images and / or spatial data may be selected from a buffer storing image and / or spatial data streams acquired from a camera and / or a 3D surface scanning system, wherein the selection corresponds to the most recent time point before receiving the trigger.

[0037] In a method according to an embodiment of the invention, acquiring reference images and / or spatial data of an object using a camera and / or a 3D surface scanning system may include acquiring conventional monochrome and / or color digital photographic images in the visible and / or infrared spectrum.

[0038] In a method according to an embodiment of the present invention, using a camera and / or a 3D surface scanning system to collect reference images and / or spatial data of an object may include: simultaneously using multiple cameras included in the camera system to collect reference images and / or spatial data so as to obtain images and / or spatial information of the object from different observation points.

[0039] In a method according to an embodiment of the invention, acquiring a reference image and / or spatial data of an object using a camera and / or a 3D surface scanning system may comprise acquiring 3D and / or depth information.

[0040] In a method according to an embodiment of the present invention, comparing a reference image and / or spatial data of an object may include: detecting at least one image feature and / or landmark in the reference image and / or spatial data and in another image and / or spatial data, and comparing the position of at least one image feature and / or landmark between the reference and another image and / or spatial data for determining the output.

[0041] In a method according to an embodiment of the invention, the image features and / or landmarks may include at least one anatomical landmark on a joint, bone, muscle, and / or other externally discernible anatomical feature of the subject's body.

[0042] In a method according to an embodiment of the invention, providing an output may include: determining a difference metric indicative of movement of the object relative to a reference state based on the comparison; and alerting an operator that the object is no longer in the expected reference position when the difference metric exceeds a predetermined threshold.

[0043] In a second aspect, the invention relates to a device for detecting movement of at least one body part of a subject in a diagnostic imaging examination (e.g. a diagnostic X-ray imaging examination, e.g. a projection X-ray radiography examination). The device comprises a camera and / or a 3D surface scanning system for acquiring images and / or spatial data of the subject and an input for receiving a trigger signal, the trigger signal being used to indicate that the current spatial configuration of the subject is to be maintained for the diagnostic imaging examination. The device comprises a processor and an output, wherein the processor is adapted to acquire a reference image and / or spatial data of the subject using the camera and / or the 3D surface scanning system upon receipt of the trigger signal, and to acquire further images and / or spatial data of the subject using the camera and / or the 3D surface scanning system after acquiring the reference image and / or spatial data of the subject. The processor is further adapted to compare the reference image and / or spatial data of the subject with further images and / or spatial data of the subject (or, equivalently, to compare the further data with the reference data, i.e., vice versa), the reference image and / or spatial data representing a reference state of the subject substantially at the time of receiving the trigger signal, and the further image and / or spatial data representing a newer state of the subject. The processor is adapted to provide output data to an operator and / or the subject via the output portion, wherein the output data represents the comparison of further image and / or spatial data with the reference image and / or spatial data so as to indicate movement of the subject relative to a reference state of the subject.

[0044] In a device according to an embodiment of the present invention, the processor may be adapted to repeat the steps of: acquiring additional images and / or spatial data via a camera and / or a 3D surface scanning system, comparing a reference image and / or spatial data with the additional images and / or spatial data, and providing updated output data to present a dynamic view of current changes in the spatial configuration of the object relative to an earlier reference state of the object.

[0045] In an apparatus according to an embodiment of the invention, the output may comprise a display monitor.The processor may be adapted to present a visual representation of the comparison via the display monitor.

[0046] In a device according to an embodiment of the invention, the processor may be adapted to present in the output data the shape, area and / or volume of the two-dimensional and / or three-dimensional differences determined by the comparison as an overlay and / or contour on a visualization of further image and / or spatial data and / or as a marker and / or annotation accompanying such a visualization.

[0047] In an apparatus according to an embodiment of the present invention, the processor may be adapted to determine a difference metric indicating movement of the object relative to a reference state based on the comparison, and to output an alarm to an operator and / or the object via the output unit and / or to output a signal to a diagnostic imaging system when the difference metric exceeds a predetermined threshold, so as to indicate that the object is no longer in the expected reference state.

[0048] In a device according to an embodiment of the present invention, the camera and / or 3D surface scanning system may include at least one camera and / or 3D surface scanning device, which is arranged to obtain a two-dimensional and / or three-dimensional view of at least one body part and / or object, wherein the at least one camera and / or 3D surface scanning device includes an optical camera for acquiring monochrome, color and / or multi-spectral two-dimensional images in the infrared and / or visible spectrum, and / or a plurality of such cameras arranged to view the object from different angles, and / or a depth camera and / or a 3D surface imaging system.

[0049] In a device according to an embodiment of the present invention, the input section may include a human interaction interface.

[0050] In a device according to an embodiment of the present invention, the input part (eg human interaction interface) may include buttons, a voice control interface and / or a gesture detection system for detecting gestures made by an operator.

[0051] In a device according to an embodiment of the invention, the input portion may include a proximity and / or position detection system to detect the proximity of the operator to the object, so as to generate the trigger signal when the operator leaves the vicinity of the object.

[0052] In an apparatus according to an embodiment of the invention, the input (e.g., a proximity and / or position detection system) may include an RFID tag in combination with at least one RFID tag detection sensor, and / or may include at least one light gate, and / or may include a sonar, radar, and / or lidar system, and / or may be implemented by using a processor to detect the position of an operator in a live stream of images and / or spatial data of an object captured by a camera and / or a 3D surface scanning system.

[0053] In an apparatus according to an embodiment of the invention, the input may include a connection for receiving a trigger signal from an automated system for detecting a predetermined reference space configuration of an object as indicated for a diagnostic imaging examination in a live imaging stream provided by a camera and / or a 3D surface scanning system.

[0054] An apparatus according to an embodiment of the invention may comprise said automated system for detecting a predetermined reference spatial configuration of an object.

[0055] The device according to an embodiment may include an artificial intelligence module for evaluating a trained machine learning model and generating the trigger signal (provided to the input portion) by considering the output of the evaluated model. The trained machine learning model uses the image and / or spatial data, and / or data from the diagnostic imaging system, and / or data derived therefrom as input. The data from the diagnostic imaging system (to be received as input in the operation of the device) may include device state information of the diagnostic imaging system and / or control input information received by the diagnostic imaging system from user interaction.

[0056] In a device according to an embodiment of the present invention, the processor may be suitable for comparing the further image and / or spatial data with the reference image and / or spatial data by detecting at least one image feature and / or landmark in the reference image and / or spatial data and at least one corresponding image feature and / or landmark in the further image and / or spatial data and comparing the position of at least one image feature and / or landmark between the reference image and / or spatial data and the further image and / or spatial data.

[0057] In a third aspect, the invention relates to a diagnostic imaging system, wherein the system is adapted to perform a method according to an embodiment of the first aspect of the invention and / or comprises an apparatus according to an embodiment of the second aspect of the invention.

[0058] A diagnostic imaging system according to an embodiment of the present invention may include an X-ray imaging system, such as a projection X-ray imaging system.

[0059] In a fourth aspect, the present invention relates to a workstation for a diagnostic imaging system (e.g. a diagnostic X-ray imaging system), wherein the workstation is suitable for performing a method according to an embodiment of the first aspect of the invention and / or includes an apparatus according to an embodiment of the second aspect of the invention.

[0060] In a fifth aspect, the invention relates to a computer program product for performing a method according to the first aspect of the invention when executed on a computer, such as an apparatus according to an embodiment of the second aspect of the invention.

[0061] Independent and dependent claims describe particular and preferred features of the invention. Features of the dependent claims may be combined with features of the independent claims and with features of other dependent claims as appropriate and not necessarily only as explicitly set out in the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A method according to an embodiment of the present invention is shown.

[0063] Figure 2An illustrative visualization of patient movement relative to a frozen reference image (eg, included in an output of a method and / or apparatus according to an embodiment) is shown to illustrate embodiments of the present invention.

[0064] Figure 3 A device according to an embodiment of the invention is schematically shown.

[0065] Figure 4 A diagnostic imaging system according to an embodiment of the present invention and a workstation for a diagnostic imaging system according to an embodiment of the present invention are shown.

[0066] The accompanying drawings are schematic and non-limiting. The elements in the accompanying drawings are not necessarily represented to scale. The present invention is not necessarily limited to the specific embodiments of the invention shown in the accompanying drawings. DETAILED DESCRIPTION

[0067] Although exemplary embodiments are described below, the present invention is limited only by the appended claims, which are hereby expressly incorporated into this Detailed Description, wherein each claim and each combination of claims permitted by the claim-defined dependency structure forms a separate embodiment of the present invention.

[0068] The word "comprising" used in the claims is not limited to the features, elements or steps described thereafter, and does not exclude additional features, elements or steps. Thus, this specifies the presence of the mentioned features, but does not exclude the further presence or addition of one or more features.

[0069] In this detailed description, various specific details are presented. Embodiments of the present invention may be performed without these specific details. In addition, for the sake of clarity and conciseness of this disclosure, well-known features, elements and / or steps are not necessarily described in detail.

[0070] In a first aspect, the present invention relates to a method for detecting movement of an object (e.g., at least one body part of the object) in a diagnostic imaging environment, for example, to detect unwanted movement of a patient before starting a medical imaging procedure, for example, in particular after completing initial preparation of bringing the object into a correct position (posture, or generally, desired spatial configuration) for examination. Thus, before performing diagnostic image acquisition, movement can be detected relative to an earlier position and spatial configuration of the object. The position, orientation, deformation and / or other spatial (geometric) properties of the object can be monitored to detect changes therein, such that the spatial relationship between the object and a diagnostic imaging system (e.g., a detector and an X-ray tube) remains substantially constant.

[0071] It should be understood that the principles of the present invention are equally applicable to various types of diagnostic imaging examinations and are not necessarily limited to X-ray imaging in general or X-ray projection radiography in particular. Therefore, references to X-ray imaging and the like throughout this disclosure are intended to be used for illustrative purposes only and are not necessarily limiting.

[0072] However, it will also be appreciated that embodiments of the present invention may be particularly advantageous and / or useful in view of the specific circumstances of X-ray imaging and / or X-ray projection imaging. For example, the configuration process for an X-ray examination (e.g., beam collimation, tube voltage (kVp), tube current, filter parameters and / or other settings) and the need for radiation protection precautions for the operator's safety may often result in a large time span between the moment when the patient has assumed the desired position for the examination and the actual acquisition moment, during which the operator may not be able to (easily) verify that the patient has maintained the desired position.

[0073] In addition, conventional (e.g., relatively simple) projection radiography systems may typically lack many automated features and / or support systems that allow for a high degree of remote control of the system from a control room (e.g., remote collimation, automatic insertion / configuration of beam filter plates, automated patient couches, etc.) and / or may lack modules for high-quality remote patient viewing (e.g., via high-resolution cameras and / or cameras for simultaneously viewing the patient from different viewing angles), making the amount of time that the patient remains in the intended position when the operator is distracted by other (typically manual) tasks and / or cannot easily visually confirm that the patient's position has not changed can be particularly problematic in conventional projection radiography examinations using such systems.

[0074] The method according to an embodiment of the present invention may be a computer-implemented method, i.e., an automated or semi-automated method that can be performed by dedicated processing hardware (e.g., using a dedicated integrated circuit) and / or configured / programmed processing hardware, such as a computer or other general-purpose processor programmed to perform a specific task of the method (e.g., by executing specific software designed for the purpose) and / or a configurable hardware platform (e.g., a field programmable gate array) configured to perform a specific task of the method. A combination of dedicated hardware (e.g., ASIC) and / or configured hardware (e.g., FPGA) and / or one or more programmed processing devices (e.g., using a CPU, GPU or other suitable processor, typically in combination with supporting hardware, such as is common in computers) is also possible. The method may also be performed by multiple processors, computers and / or other processing devices working in collaboration, such as using a host-client architecture, a web-based architecture, cluster processing and / or other forms of distributed processing, data collection, data presentation and / or data storage.

[0075] “Automated” and “semi-automated” may refer to computer-implemented methods implemented by a (e.g., digital) processor, controller, and / or other such hardware, wherein the methods are performed in an autonomous or supervised autonomous mode, e.g., requiring only limited input from and / or limited interaction with an operator, e.g., to select or enter relevant parameters and / or configuration options, to start, stop and / or interrupt a process, to supervise a process, and / or other such limited interaction.

[0076] Where a "camera", "depth camera", "3D camera" or similar component is mentioned, it should be understood that the referred device should not be confused with a diagnostic imaging system (or its diagnostic imaging component). Even though a "camera" may be implemented as a part of an imaging system (either by default or with appropriate modifications), the main purpose of a diagnostic imaging system will be understood to generally involve an image acquisition process by different means, such as by ionizing radiation, radio frequency signals, ..., and is not essentially (or at least not exclusively) based on "camera" observation. For example, even though the radiation detection component (i.e., image detector) of a PET or SPECT system may be referred to as a PET or SPECT "camera", it should be understood that this is different from a "camera" in the sense of the present disclosure.

[0077] In the sense of the present disclosure, a camera may, for example, include a conventional optical imaging camera and / or a depth imaging camera. The former (conventional camera) may generally relate to an imaging camera that detects (i.e., in particular for determining an image) light in the (human) visible spectrum and / or infrared spectrum. Conventional cameras may, for example, be suitable for monochrome, color and / or multispectral imaging. The latter (depth camera) generally relates to a technique that detects depth information in addition to in-plane information (e.g., a two-dimensional projection plane of an image detector), for example, using stereo imaging (or more generally, camera imaging from multiple different viewpoints at the same time), range cameras, LIDAR, RADAR and / or other such techniques to determine points and / or surfaces in three-dimensional space, for example, so that the surface contour of (one or more) objects (e.g., facing the camera) can be qualified in three dimensions at least to some extent. In this context, a depth camera is understood to generally refer to any suitable device for collecting spatial data representing an observed scene in three spatial dimensions, for example, so that the (3D) position in space of a point on the surface of an object in the observed scene (at least when not obstructed from the camera viewpoint) is determined. Note that where reference is made below to a camera system, this may equally refer to a 2D image camera, a depth or 3D camera or a 3D surface scanning system (or any combination thereof).

[0078] Some examples of optical and / or other distance sensing technologies that may be suitable for use with cameras and / or 3D surface scanning systems according to embodiments include, but are not limited to, structured light systems, stereo vision systems, and / or active stereo systems. An active stereo system refers to a stereo vision system that additionally uses structured light to improve the detection of surface details, for example, where a structured light pattern (e.g., using infrared light) is used to actively provide optically detectable details to homogeneous surface areas. Distance measurements may be collected at a single point, across a scan plane, and / or over an entire volume in space, such as a complete image with depth measurements at every point in the sensor's field of view.

[0079] Methods according to embodiments of the present invention may be used to assist in patient positioning during an imaging session (e.g., in a preparatory step for a projection X-ray imaging examination), in particular after positioning the patient (e.g., assisted by an operator) and before performing (one or more) diagnostic image acquisitions. Where a diagnostic (or medical) imaging session or environment is referred to, it should be understood that this may refer to a procedure (e.g., for its respective environment, such as an examination room), wherein a (e.g., human) object is positioned in a desired posture or generally desired spatial configuration relative to an imaging system (e.g., relative to an X-ray tube and an X-ray detector) to perform an imaging procedure, after which one or more diagnostic images may be acquired. Embodiments of the present invention may be particularly useful in situations where the patient should remain stationary in a desired posture until the acquisition is completed. This may be particularly problematic in situations where the patient is not assisted by a support or a specific positioning module, such as in routine examinations where the patient is standing (e.g., upright) or sitting (e.g., sitting on a stool) without a lot of mechanical support for the body but must remain stationary in a desired orientation and / or position relative to the detector plane. For example, this is often the case for projection radiographic examinations to diagnose fractures, etc. (but not limited to this).

[0080] refer to Figure 1 , an illustrative method 100 according to an embodiment of the present invention is shown. The method (e.g., a computer-implemented method) 100 can be used to assist in positioning of an object, e.g., to avoid movement from a desired position / pose (i.e., spatial configuration), e.g., during and / or after preparation steps for a diagnostic imaging session.

[0081] The method 100 comprises receiving 101 a trigger signal to indicate that a current spatial configuration of an object is to be maintained.

[0082] The trigger can be provided manually by an operator. For example, an operator (e.g., a medical technical assistant, MTA) can bring a patient to a position (typically, a spatial configuration) that is deemed suitable or desired (e.g., optimal for an intended diagnostic imaging examination). The operator can rely on his / her own experience and knowledge to make his / her own best judgment, e.g., preferably without being constrained by any substantial limitations imposed by a method or system according to an embodiment. Once the object is in the desired position, the operator provides a trigger to indicate that the position (i.e., spatial configuration) of the object is to be maintained.

[0083] However, alternatively or additionally (e.g., using an "or" logical connection), the method (e.g., a system integrating the method) may also be combined with different methods to automatically determine and match the correct positioning of the patient, such as a machine vision algorithm to detect when the patient is in a predetermined position (spatial configuration) suitable for examination. Although such algorithms are known and are generally outside the scope of this application, triggers can therefore be generated by another system to assist in the positioning of the patient. As shown in the examples in the background technology section above, various methods are known in the art to guide the patient to a posture suitable for examination, so they are not discussed in detail. The method according to the embodiment may also include such prior art methods to help locate the object, such as before the step of receiving 101 triggers. The trigger may be generated as an intermediate output by the (automatic or semi-automatic) positioning assistance step, or the positioning assistance step may be performed before the operator performs manual confirmation of the patient's position and manually generates the trigger. In other words, the method 100 may include a step of positioning assistance to enable the patient to enter a predetermined or automatically determined spatial configuration. Therefore, the trigger may be generated (e.g., automatically) by the positioning assistance step (the system implementing the positioning assistance step), or may be received as an external input (e.g., due to an action performed by a user (i.e., an operator)).

[0084] The trigger may be generated by a human interaction interface, such as by pressing (or more generally, using) a button or switch, by a voice interface, by gesture detection (e.g., detecting a gesture of an operator), by interaction of a keyboard and / or mouse with a (e.g., graphical) user interface, and / or by any other suitable means for interacting with a user (i.e., for receiving a signal (e.g., at least one bit signal of any suitable form) from an operator). Thus, a hardware button (possibly included in a common interface, such as a keyboard and / or mouse), gesture detection, or a voice control system may be used to indicate when an object is well positioned for inspection.

[0085] For example, a camera system (discussed in further detail below) may be used to detect gestures from an operator. Even if the operator is preferably not visible in the reference image and / or further spatial data (discussed below), the operator may instantaneously provide a (machine) detectable gesture, such as a sliding motion across the camera view (e.g., moving a hand across the camera lens from a short distance), or a more complex gesture, such as a thumbs-up gesture. For example, moving a hand across the camera, or generally occluding a significant portion of the camera's field of view by the hand (or equivalently, by an alternative body part or object) (e.g., in a short-range sliding, grabbing, or other gesture, motion, or hand pose) may be particularly easy to detect with good sensitivity, accuracy, and robustness. Thus, according to embodiments of the present invention, a trigger signal can be easily generated by detecting large changes in a camera image, such as a significant drop in the average image intensity of a conventional optical camera image (e.g., obtained by a 2D monochrome, color, or similar conventional camera), a significant increase in image intensity (e.g., when using a passive or active infrared camera), a significant change in average color value, a significant decrease in the average distance from the camera to points in the scene (e.g., the average depth value for each pixel on a depth image), and / or another similar simple detection strategy. Optionally, such change detection can be configured to detect a temporary change (e.g., a drop) associated with a test parameter (e.g., average image intensity) returning to approximately its previous value (e.g., returning to approximately the value before the drop). A technician can easily determine an appropriate threshold, score, or another suitable test parameter for generating a trigger signal based on such a "substantial" change, for example, by direct experimentation and / or trial and error.

[0086] It will be appreciated that for some gestures a (short) delay timer may be activated (or a short delay may be accommodated in other ways) to allow the operator to vacate the camera's view before a trigger is generated in response to a detected gesture. Gestures may also be detected via, for example, a separate camera from the camera system. Gestures may also be detected in other ways, such as by an accelerometer or machine-detectable tag attached to the operator's clothing (e.g., in / on a wristband or clothespin).

[0087] The method may include generating a trigger by acquiring an image of the camera system (e.g., a live image stream) and performing 110 scene analysis. Thus, a suitable time point can be automatically determined for which the patient is considered to be in a suitable position for the diagnostic imaging procedure, i.e., when the preparatory step of positioning the object is considered to be completed. However, this does not require detailed knowledge of the procedure to be performed. In particular, the scene analysis can detect the time point at which the operator has left the immediate vicinity of the patient. Advantageously, such intelligent automatic triggering can facilitate the patient movement detection provided by embodiments of the present invention without requiring changes to the diagnostic imaging workflow (e.g., except, of course, when substantial movement is subsequently detected by the method and the operator is implicitly or explicitly alerted to take action in response). The scene analysis may include detecting the shape of the operator and / or the operator's limbs (e.g., one or two arms) in the image acquired (e.g., a live stream) by the camera system, and generating a trigger when such a shape has moved outside the image frame (e.g., after a short predetermined delay) to avoid false detections. It should be understood that, for example, one or more arms of the operator can typically move from a central position in the image frame toward the edge of the frame and outside it. For example, the object and the operator's body part(s) may be spatially separated from each other at some point, and one (or more) spatially isolated segments may connect the edge of the image frame (if not already present) and decrease in size to zero (move outside the frame) over time.

[0088] The detection may even be as simple as detecting a dramatic reduction in image content (e.g., pixels, surface areas, or voxels) attributable to living matter. For the latter, the use of depth imaging (for clear separation of the patient and operator from the background by depth differences) and / or infrared imaging (using the thermal emissions of one or more bodies to obtain good contrast for algorithmic separation from the background) may be advantageous (but not strictly necessary).

[0089] Additionally or alternatively, the camera system may include one or more additional cameras to cover a larger volume of the scene, e.g. a wider view of the examination room, e.g. arranged to provide a (longer) image capture, such that the primary camera(s) can track the position of the object in detail, while the auxiliary camera(s) can detect when the operator leaves the vicinity of the object (e.g., so as to be observed from the wider overall view). In other words, a trigger may be generated by recognizing a "gesture" (see above) by the operator, where the "gesture" may include or involve leaving the volume of space monitored by (at least one camera of) the camera system.

[0090] Additionally or alternatively, a trigger may be generated by detecting proximity or spatial position of the operator using one or more sensors (e.g., radio frequency identification tags (on the operator's body; RFID)) in combination with an RFID reader positioned in or near the spatial area where the patient is positioned for examination. This can be used to detect when the operator is no longer present in the vicinity of the subject (or the state changes from being present (i.e., RFID detected) to not being present in the vicinity of the subject (i.e., no longer RFID detected)), for example, relying on RFID communications which are typically short range. Other suitable sensors may include, for example, beam gates (detecting when the operator crosses one or more beams while leaving the examination area), indoor positioning systems (e.g., using triangulation of the operator's position; for example, even if not completely accurate, this may be referred to as "GPS for indoor tracking"), sonar, lidar and / or radar technology and / or another such suitable sensor system.

[0091] Advantageously, operator interaction, e.g. actions to be performed by a diagnostic imaging technician, can be reduced and / or their workflow can be simplified. Ideally, the interaction steps are reduced to a minimum. Any type of trigger that the technician has to give explicitly will increase the complexity of the workflow, even if it is only in a minimal way. Voice or gesture interaction may be preferred for, e.g., pressing a button, as it can be given without constraining the operator's position to the position of the button. Likewise, automatic detection of triggers without requiring a conscious action by the operator may still be more preferred, e.g. using detection of the operator's position (particularly at the moment when the operator leaves the area where the object is set for examination).

[0092] The method may also include evaluating a trained machine learning and / or artificial intelligence model, such as an artificial neural network, to generate a trigger (or: considering the output of the ML / AI model to generate a trigger). For example, an image or image sequence acquired via a camera system (e.g., a live stream) may be provided as an input (directly or indirectly, for example via an intermediate processing such as a feature extraction step) to a trained ML / AI model. Thus, the step of performing 110 scene analysis may include evaluating a trained machine learning and / or artificial intelligence model. However, other input(s) may also be used (additionally or alternatively; directly or indirectly), such as motion sensors, (e.g., X-ray) device control inputs and / or outputs, sound inputs (e.g., microphones), etc.

[0093] The ML / AI model may be (or has been) trained on a dataset that includes multiple (e.g., typically a large number) of user-manipulated interactions with a diagnostic imaging modality, such as using an X-ray system (e.g., a system of the same or similar type as that of the intended application of the method according to the embodiments). For example, the positioning of a patient may be monitored for a large number of test cases, for example, via a camera system, and the time at which the patient has been positioned in a position to be maintained during the diagnostic imaging session may be added as an annotation (e.g., determined via a manually generated explicit trigger signal provided by an operator, or added after the fact when preparing the training dataset). Thus, the ML / AI model may receive this training input (e.g., camera images and / or other sensor data and / or information derived therefrom, such as extracted features) and a corresponding training output (trigger annotation) to be trained so as to generate a trigger at an appropriate time point or sufficiently close to an appropriate time point. It will be clear that the particular choice of input to the ML / AI model may be flexible, for example, many input sources may be conceivable, as long as sufficient information is at least implicitly inferable therefrom.

[0094] For example, user interactions with a diagnostic imaging (e.g., X-ray) system may be used as input (possibly in combination with other data (e.g., the camera data described above) to predict when the system (and user) is in a stable state, i.e., when the subject is ready for a diagnostic imaging session. For example, an ML / AI model may be trained to detect when the operator has completed the patient preparation process and moved on to other tasks, i.e., when a trigger should be generated, using implicit information that may be embedded in actions for repositioning of an X-ray tube, adjustment of beam parameters, movement of an actuated support aid (e.g., a patient platform), adjustment of collimation, activation / deactivation of an indicator light field (a visual representation of the X-ray beam), etc.

[0095] In addition, it should be understood that the trigger effectively used in performing the method can also be determined by combining trigger signals from multiple sources and / or generating methods. For example, different inputs, such as a general human interaction interface, a button, a voice control interface, a gesture detection system, a proximity / position detection system (e.g., to detect whether an operator and an object are close to each other), and / or other trigger signals can be combined, for example, in a logical "or", in a logical "and", in a combination thereof (e.g., combining different logical operations in a non-trivial expression), in a weighted combination, in a majority vote, and / or in any other suitable manner.

[0096] The method also includes acquiring 102 a reference image and / or spatial data of the subject using a camera system when the trigger is received 101. The camera system is typically aligned with the anatomical structure of interest of the subject, e.g., so that the subject's (one or more) relevant body parts can be fully observed through the data provided by the camera system. However, as discussed in detail below, accurate positioning and / or alignment of the camera is not required, e.g., an accurate mapping between a coordinate system of the camera system and a coordinate system of the diagnostic imaging system need not be reproduced or determined.

[0097] For example, the reference image / spatial data may be acquired at a predetermined time at or after the moment of receiving the trigger, for example preferably substantially simultaneously or shortly thereafter (e.g. delayed only by the time required to process the trigger and control the camera system accordingly to acquire the data, and / or by other such non-substantial delays in view of technical and practical limitations). It should be understood that the camera system may also be configured to continuously capture image and / or spatial data (e.g. update an image and / or spatial data buffer) such that upon receiving a trigger, an image and / or spatial data may be selected as said reference image and / or spatial data from the buffer, even though the selected data may represent the state of the object (shortly before) the trigger was received. It should be understood that this approach allows capturing a reference state of the object (i.e. represented by the reference image / spatial data) that is substantially simultaneous with the reference that the trigger (e.g. given by an operator) indicates is to be recorded, for example, the buffer may compensate for any short technical delays, and may even compensate for any non-technical delays, such as biological decision-making and / or muscle response times of the operator. However, it will also be understood that such technical and / or biological delays are typically of an order of magnitude that may not require any compensation at all, e.g. for most applications a delay of one or even several seconds (negative or positive, e.g. before or after the trigger time) is typically acceptable.

[0098] Therefore, "when the trigger is received" should be interpreted as the acquired reference image / spatial data representing the state of the object (e.g., having a specific spatial position, orientation and / or other geometric state in time) substantially at the time when the trigger is received (and therefore, also substantially at the time when the trigger is generated), for example less than 5 seconds before or after the time when the trigger is received, for example preferably less than 1 second before or after the trigger time, preferably within the range of half a second to half a second before the trigger is received.

[0099] After positioning the patient (wherein the operator has checked (or actively guided the patient) that the patient's spatial configuration (e.g., relative to the diagnostic imaging system) is correct), e.g., is (sufficiently) suitable for the examination to be performed, a trigger is generated (explicitly by the operator, implicitly through the operator's movements, or automatically through a dedicated algorithm), and reference image / spatial data is determined in response, e.g., camera image(s) of the object, or at least camera image(s) of the anatomical region of interest are taken, i.e., at a time sufficiently close to the moment indicated by the trigger, which is assumed to have positioned the subject in the correct (i.e., desired) posture.

[0100] Acquiring 102 reference images and / or spatial data of the object using the camera system may comprise conventional (digital) photo acquisition, e.g. capturing color or monochrome camera images. Using infrared imaging may also be advantageous, e.g. causing the camera system to comprise an infrared camera, e.g. a camera configured to obtain infrared image data (possibly in combination with one or more visible spectral components). Thus, the camera system may comprise, e.g., one or more optical cameras sensitive to one or more spectral bands in the visible (and / or infrared) range, e.g., (one or more) monochrome cameras and / or (one or more) color cameras. (One or more) color cameras will be understood to refer to (one or more) color cameras, without necessarily being limited to a specific choice of color components (e.g., RGB) or a choice of a specific combination thereof. It will also be understood that a "color" camera may also (unintentionally or intentionally) be sensitive to one or more infrared bands, possibly exclusively, but possibly also in combination with visible light bands. The camera system is also not necessarily limited to a "color" camera having a relatively small number of color bands (e.g., red, green, blue), but may also include (one or more) multispectral cameras, for example, which are adapted to more fully quantify the spectrum received at each pixel location, for example, by decomposing into a relatively large number of spectral bins (e.g., so that a substantial spectral image scale is formed in addition to 2D image coordinates).

[0101] Reference images and / or spatial data, such as one or more images and / or spatial information of any suitable form about an observed scene (including an object or an associated body part), may optionally include 3D and / or depth information. Thus, acquiring 102 reference images and / or spatial data may include acquiring 3D and / or depth information. For example, acquiring reference images / spatial data may alternatively or additionally) include acquiring a depth image and / or acquiring three-dimensional (3D) data (e.g., a 3D point cloud and / or a surface model constructed from such a 3D point cloud) by a camera system in another manner. For example, the camera system may include a depth camera and / or another suitable device for capturing 3D surface information (e.g., capturing a 3D (external) surface map of a patient's body (or at least one or more associated body parts)). Thus, the images and / or spatial data may include monochrome images, color images, infrared images, multispectral images, depth images, 3D images, and / or combinations thereof. In addition to the in-plane image information, depth information can also be collected, or at least three-dimensional (3D) information can be collected, which is not simply reduced to 2D data, for example, not just expressed in coplanar coordinates. For example, a 2D map (e.g., image) of depth and / or a collection of 3D points (e.g., 3D point cloud) can be collected. The depth image can be obtained, for example, by a range camera, which can produce a 2D image showing the distance from a predetermined reference point or plane (e.g., focus or other reference point) to a point in the scene. Depth information can be obtained by stereo imaging, where images are collected (e.g., substantially simultaneously) by two (conventional, such as monochrome or RGB) cameras from different observation points. Processing techniques known in the art can be applied to such stereo images (applied to pairs of images collected concomitantly) to derive depth information from them, for example, by parallax analysis. This principle can be easily extended to more than two cameras, which can also be generally referred to as stereo imaging (i.e., "stereo" should not be interpreted as the narrowest sense of using only two cameras). The advantage of stereo (or multi-camera) depth imaging is that the required equipment components can be easily obtained and can be relatively cheap. For example, such a system may be constructed using conventional (2D) cameras, or existing (2D) cameras may be easily upgraded by adding one or more further cameras.

[0102] The processing that may be required to calculate the corresponding depth pixel map (i.e., depth image) is relatively simple and can be easily implemented in software and / or hardware (potentially utilizing graphics processing unit GPU processing and / or dedicated hardware, such as application specific integrated circuits and / or field programmable gate arrays). An advantage of stereo or multi-camera depth imaging is that a scene, for example including objects, can be imaged without dedicated lighting and / or other direct manipulation. In other words, the scene can be imaged and processed to determine depth data (i.e., a third dimension in addition to the planar image coordinates) without actively perturbing or affecting the scene, i.e., essentially passively, except for possible general (common) lighting.

[0103] Depth images can also be acquired by active techniques. For example, a sheet of light can be scanned over a scene to image its reflections. From the shape and displacement of the imaged (line) reflections, the distance between the reflections (points on the line, e.g., each point in the image after fully scanning the scene) and a reference (e.g., a light source and / or camera) can be calculated relatively easily, e.g., using triangulation methods. Another type of active depth imaging that can be applied uses a structured light 3D scanner, which can be viewed as a more complex version of the light sheet triangulation mentioned above, with additional advantages, such as requiring fewer (or no) physical displacements of the camera and / or light source to fully define the depth value of the scene.

[0104] Depth information may be collected by a time-of-flight camera, LIDAR, or RADAR system. A possible advantage of time-of-flight camera depth imaging is that images may be collected essentially instantaneously, e.g., without the need to scan points, lines, structured light patterns, lasers, waves, or generally any "active" (e.g., time-varying, e.g., scanning) analyzer over the scene. Other potential principles for depth imaging may include interferometry techniques (e.g., relying on coherent light), coded aperture imaging, and / or possibly other techniques.

[0105] Furthermore, acquiring 102 reference image / spatial data may further comprise acquiring (e.g. substantially simultaneously) images (e.g. conventional 2D images and / or depth images and / or 3D data) of the object from different viewpoints (e.g. camera observation points). Thus, the camera system may also provide any of the aforementioned data from different observation points simultaneously, e.g. to acquire image / spatial data from different camera positions (e.g. using different cameras in the camera system), e.g. to view the object from at least two different angles. Thus, even without using depth or 3D data, explicitly by detecting movement in (any of) at least two different image planes, the position of the object may still be monitored.

[0106] It should be understood that the above comments about image / spatial data also apply to the reference data about the live tracking data discussed further below. Given that they are acquired by the same camera system and preferably have the same configuration settings, the reference data and the live tracking data are generally of the same type, nature and configuration.

[0107] Note that the camera system does not have to be connected to a diagnostic imaging system (e.g., an X-ray system), e.g., no mechanical, electronic and / or data processing is required to integrate the camera system into the diagnostic imaging system. The position and orientation of the camera system (or its individual camera components) does not need to be accurately determined relative to the diagnostic imaging system (e.g., relative to the radiation beam axis of a projection X-ray system), it is sufficient that the object or the object's related anatomical structure is observable by the camera, e.g., falls within an image frame. Therefore, no complex calibration and / or alignment procedures are required. In addition, the camera system does not require any input from the diagnostic imaging system, nor does it need to provide output to the diagnostic imaging system, e.g., via electronic signaling and / or input / output of interaction between software of the respective systems. Since the method does not necessarily explicitly determine the position of the object relative to the coordinate system of the diagnostic imaging system, e.g., to check whether the position is correct for a particular examination, accurate positioning and / or alignment of the camera system is not strictly necessary, nor is calibration of the camera coordinate system relative to the coordinate system of the diagnostic imaging system.

[0108] Method 100 also includes: after acquiring 102 the reference image and / or spatial data of the object using the camera system, (e.g., repeatedly 114) acquiring 103 additional images and / or spatial data of the object (e.g., which may also be referred to as live tracking data) using the camera system, for example periodically or substantially continuously acquiring additional image / spatial data, for example through video monitoring (e.g., so that the additional image / spatial data may correspond to or may include the latest camera image in one or more live video streams).

[0109] The method further comprises (e.g., also repeatedly) comparing a reference image and / or spatial data of the object (which represents the state of the object substantially when the trigger is received) with further image and / or spatial data of the object (which represents the most recent state of the object) 104. For example, the further image and / or spatial data may be compared with the reference substantially continuously, for example at a reasonable frequency, which may also correspond to a sampling frequency for updating the further image / spatial data (e.g., at least 0.1 Hz, preferably at least 1 Hz, such as at least 10 Hz). Even if the further data is updated continuously, or at least at a certain frequency to enable a more or less smooth monitoring of the current state of the object observed via the camera, it should be understood that this does not necessarily imply a constant refresh frequency, even if in a typical approach the conventional video capture and processing frequency can advantageously be used to update the further image / spatial data at such a constant refresh rate (frames per second, FPS), for example in the range of 0.1 Hz to 120 Hz, for example 10 Hz to 80 Hz, for example 20 Hz to 60 Hz, for example 25 Hz, 30 Hz; 40 Hz or 50 Hz, without placing any limitation on these illustrative ranges and / or values.

[0110] Hence, by comparing the current image and / or spatial data with the reference data, any (relevant and / or substantial) movement of the object may be detected in a simple and effective manner.

[0111] The method includes providing 105 an output to an operator and / or subject, the output representing such a comparison of further image and / or spatial data (e.g., a current image, such as a most recently acquired image) with a reference image and / or spatial data to indicate movement of an object (e.g., at least one body part of interest of the object) relative to an earlier state of the object represented by the reference image and / or spatial data. Thus, after completing the initial preparation of bringing the object into a correct posture (represented by the reference data), before starting a diagnostic (e.g., medical) imaging procedure, the operator's and / or subject's attention may be drawn to an undesired movement of the object (e.g., a patient). The output may be presented to the operator (and optionally also to the subject) via (one or more) display monitors or (one or more) other suitable human interface devices. Preferably, the output may be dynamic, for example by repeatedly updating the output based on a live data stream from a camera system. For example, the information to be output may be shown on a display by (one or more) simple numerical values, by a text string, and / or in another suitable form in a graphical representation (e.g., an overlay and / or other type of visualization).

[0112] Providing an output may also include, for example, displaying the results of the comparison in a display format that is continuously (e.g., at some reasonable frequency) updated. For example, a difference image may be displayed to an operator (and / or directly to an object to allow the object to automatically correct his / her position). Many different ways of visualizing the difference and / or the results of the comparison may be envisioned. For example, additional images and / or spatial data may be displayed on a display monitor (e.g., displayed to an operator and / or displayed to an object), on which a difference image (e.g., obtained by image subtraction) may be shown in an overlay, such as as a color overlay. Thus, the difference of the currently observed state relative to the reference state of the object may be highlighted, such as by marking the difference with a color. It should be understood that this may also be achieved by different types of overlays, such as changing the image intensity instead of the color (e.g., literally highlighting the difference), or creating a contour overlay to indicate the contour of the difference area. For example, the perimeter of an image area where the image difference exceeds a predetermined threshold may be marked by a line, such as a solid line or a dashed line.

[0113] Embodiments of the present invention provide (e.g., enable) a method to assist in positioning a patient for a diagnostic imaging examination, and the method allows for detecting the movement of the patient regardless of the intended examination, for example, by not requiring an explicit definition of the "correct" posture / positioning (e.g., other than a reference image of the subject recorded after being guided to this non-explicit state by an operator), and / or regardless of the patient's body (e.g., a specific body shape, anatomical parameters, possibly a specific medical condition). In principle, embodiments of the present invention do not even require the assumption that the patient is a human, for example, a method or device according to an embodiment may be suitable for assisting in the positioning of a human patient (e.g., in typical medical diagnostic applications), but may also be used to assist in the positioning of an animal patient (e.g., in veterinary medicine) without any modification. Even though a system that can easily switch between humans and veterinary medicine may not find widespread application, or may even have very limited use in practice, this clearly illustrates the versatility of the method. For example, a method according to an embodiment may be used for patients with non-standard body shapes or features, for example due to amputation, congenital conditions, etc., without the need for reconfiguration and / or modification.

[0114] However, in a method according to an embodiment of the invention, comparing 104 a reference image and / or spatial data of an object may also include detecting 106 at least one image feature and / or landmark in the reference image and / or spatial data and (one or more corresponding features) in the further image and / or spatial data. Thus, a relative displacement of at least one image feature and / or landmark between the reference and further image and / or spatial data may be determined for determining an output. For example, at least one anatomical feature and / or landmark may be automatically detected (e.g., detected and located based on camera image coordinates) in the reference image and / or spatial data and the further image and / or spatial data. The anatomical features and / or landmarks may, for example, include one or more predetermined landmarks on one or more body parts, such as a shoulder, hip, wrist, finger (e.g., in a general sense, or a specific finger, such as a pinky or thumb), knee, ankle, toe (e.g., generally or a specific toe) and / or landmark (one or more) on other anatomical parts of the body. The landmark(s) may include points (or, by extension, line segments and / or surface patches) that are (e.g., easily) identifiable by automated processing (e.g., with high specificity and / or sensitivity). Such points may be salient features of a particular body part, such as an easily identifiable ulnar head prominence of a wrist, a center point of a joint, and / or other such visible anatomical features. Such salient features may be readily detected in an image, for example, by a suitable algorithm (e.g., by a combination of image processing filters tuned to the particular shape, orientation, boundary shape, and / or other visual (and / or generally spatial) properties of the body part (and / or a particular region or portion thereof, such as the immediate vicinity of the salient feature to be detected), by a trained machine learning model, and / or by any other suitable method known in the art.

[0115] For example, methods (and / or devices) according to embodiments may be adapted to detect one or more specific image features (corresponding to anatomical features and / or landmarks) and to detect and / or quantify movement of an object based on a displacement of such image features (and / or a change in another spatial variable or multiple spatial variables, such as orientation), for example based on a change in the position of an anatomical feature and / or landmark in another image and / or spatial data relative to the position of the same feature and / or landmark in a reference image and / or spatial data.

[0116] While it can be seen that automatic detection of such at least one anatomical feature and / or landmark reduces versatility, such as the essentially universal applicability of patient positioning assistance in diagnostic imaging as described above, it should be noted that this is not necessarily the case, or may be so only in a limited sense. For example, embodiments may also be configured to fall back to simpler methods of quantifying (and / or detecting) movement, such as by image subtraction (another image from a reference image) and / or calculating at least one metric based on the subtracted image, such as the maximum absolute difference, the mean absolute difference, a predetermined percentile of (e.g., absolute) differences, a sum of squares, and / or any other such value to characterize the magnitude of the total displacement. However, when (one or more) landmark features can be detected, the detection of movement may take into account that changes in the position of such landmarks are more sensitive to associated motion rather than any arbitrary changes in image content, and / or the position of (one or more) landmarks may be indicated on the (e.g., overlay) image based on a simpler image comparison (e.g., based on the subtracted image) to further enhance the ease with which the operator can determine the correct position to hold the object. It should also be noted that such landmark features can be used to easily add 3D information to a 2D representation, for example, a 2D (image) representation can easily show the displacement of an object in the image plane (e.g., by using a color overlay to emphasize changes relative to a reference), while the landmark position can also be indicated on the visual representation along with an indicator showing the depth (out-of-plane position) of the image feature and / or its relative change relative to a reference.

[0117] Methods according to embodiments may be adapted to detect at least one anatomical feature and / or landmark, such as a set of different features and / or landmarks, without necessarily relying on (i.e. requiring) the presence of each feature and / or landmark, or relying on the presence of a particular subset or combination of anatomical features and / or landmarks in any particular reference image / spatial data. For example, one or more detection algorithms may be used to detect a variety of different anatomical landmarks, such as at least one significant point for each joint, bone, and / or other anatomical part of interest, such that at least one or at least several of the supported landmarks may be detectable (and therefore detected) regardless of the context of a particular examination. Thus, regardless of whether, for example, a patient's arm or a patient's head is being examined, at least one or at least several points may be detected (with a predetermined probability) corresponding to points associated with the location of the underlying anatomical structure. For example, for an imaging examination of the arm, bony protrusions on the wrist and elbow may be detected, wherein a detected displacement of at least one of them may be associated with a (substantial) movement of the arm as a whole. Similarly, for the head, landmark points on the chin, on the top of the skull, on the frontal tip of the nose, and / or other such easily detectable points may be used. The applied algorithm does not necessarily require input to define the correct set of landmarks to search for, e.g., the algorithm may attempt to detect all landmarks and retain only those for which detection does not fail or the estimated detection accuracy is sufficiently high. Landmark detection may also be performed by a trained machine learning algorithm, such that the training of the model may implicitly take into account the co-occurrence of specific features to determine an internally consistent set of detected features (e.g., corresponding to combinations commonly encountered in its training data).

[0118] The landmark / feature detection process is preferably agnostic, e.g. preferably does not take into account a specific geometrical model of the object and / or (one or more) body parts of the object. It should be noted that such a landmark detection algorithm can be adapted to detect landmarks (e.g. in any combination from a library of supported landmark features) without requiring manual selection of a procedure or any a priori knowledge of the specific procedure being performed. In fact, even if an incorrect point is identified, e.g. a landmark is detected as the center of the patella in a reference camera image intended to be the patient's elbow, this need not imply a problem. By far, the detection of this feature and / or landmark point is sufficiently stable, i.e. such that the same point can be detected in further images / spatial data, even if anatomically less relevant, and patient movement can still be sufficiently accurately defined and detected by changes in the position of this landmark point.

[0119] In practice, according to embodiments of the present invention, a method for detecting 106 image features and / or landmarks may be used that relies on good detectability and saliency of the image features, and not necessarily on a predetermined definition of the corresponding anatomical features. For example, a filter or combination of filters may be applied to detect, for example, the center of a more or less uniform zone of a particular size or scale (e.g., within a particular size range). While such a method may be tuned to specifically detect the center of the patella, for its simplicity and robustness, it may also be included as a general feature detector.

[0120] If optional detection of at least one anatomical feature and / or landmark is applied according to an embodiment, the position of at least one anatomical feature and / or landmark or each of them can be compared between the further image / spatial data and the reference image / spatial data to detect movement of the object. In other words, the step of comparing the reference image and / or spatial data of the object with the further image and / or spatial data of the object may comprise a comparison of (or each) anatomical feature and / or landmark in the further image / spatial data with that in the reference image / spatial data, such as a comparison of their positions. If multiple features are detected, the movement may be characterized by the sum, maximum, minimum and / or (one or more) other summary statistics of the individual displacements (e.g., the displacement of a particular feature expressed by the length of the displacement vector). A more direct image comparison, for example using a difference image, may also be used alternatively (e.g. without the need for any landmark detection), or in combination with such a landmark-based approach.

[0121] For example, the maximum absolute image difference or maximum displacement vector length of an optical flow map calculated to compare a reference image to another image can be used as a fallback when no landmarks are detectable. A combination of landmark-based motion quantization and direct image-based motion quantization can be used, for example, by expressing a value indicating the motion of an object (e.g., the magnitude of the motion) through a weighted sum or other type of combination of these individual metrics.

[0122] Thus, substantial changes in the position of point(s) indicative of, for example, the position(s) of an anatomical structure or landmark, such as a shoulder, hip, wrist, finger, knee, ankle, toe, etc., may be automatically detected. The output may show (corresponding to - i.e. with respect to the same landmark) the current detected position (in the further image / spatial data) and a reference position (in the reference image / spatial data) for the or each detected landmark and / or the difference between the position(s), e.g. the magnitude of displacement of the landmark.

[0123] Providing 105 an output may also include determining a difference metric indicative of movement of the object relative to a reference state based on the comparison 104, and alerting an operator that the object is no longer in the expected reference position, e.g., has moved substantially relative to the established reference data, when the difference metric exceeds a predetermined threshold. Thus, the difference metric may be based on direct image subtraction (or other type of image comparison calculation) and / or based on quantification of changes in position of (one or more) detected image features and / or landmarks.

[0124] As described above, the location and / or amount of movement may be displayed through the use of image overlays. Figure 2 An illustrative visualization of patient movement relative to a frozen reference image of a (correctly) positioned patient is shown. It should be noted that due to the limitations of grayscale representation, Figure 2 . Therefore, alternatively, the difference area is shown by a shaded area 21 with an outline. Although such a representation may be used in practice according to embodiments of the present invention, it will be appreciated that a color overlay (with a different color, such as a transparent color with sufficient saturation on a monochrome source background image (such as the current camera image in a live stream)) may in practice be a more convenient form of representing such an overlay.

[0125] Furthermore, anatomical structures or landmarks, or even just significant image features (e.g., points and / or regions in an image that can be detected in a stable manner throughout an image sequence, e.g., in a video stream, such as local extrema, cusps, centers or centroids of relatively homogeneous regions, etc.) may be automatically detected (see above). The positions of such landmarks may be shown individually and / or in an overlay, e.g., as both a current position (determined in the live stream) and a reference position, as a vector indicating the displacement between the reference position and the current position, and / or in another suitable form. Alternatively and / or additionally, substantial changes in the image (current versus reference) and / or landmark position(s) may be used to automatically detect substantial patient movement, e.g., to alert an operator that such substantial movement has occurred.

[0126] When using depth and / or 3D data (e.g., acquired by a depth, stereo and / or 3D camera system), patient movement can be detected when the depth (e.g., the coordinates of a landmark point in the direction normal to the image plane) changes, for example, even when the movement in the image plane is below a detection threshold (explicitly or implicitly by an operator's assessment). In addition, when using depth and / or 3D data, the detection of salient points, such as representing anatomical features, can be more robust and easier to compute. For example, convex and / or concave areas (of the 3D body surface) can be detected and their centers can be used as (possibly relevant) landmarks. Thus, the tip of the elbow, the tip of the nose, the center of the chin, the ulnar head visible at the wrist, and / or other such anatomical features can be easily detected by simple methods. Likewise, saddle points can be anatomically relevant and / or can allow robust detection of the same point on the body surface under movement (or at least under small movement, assuming that the subject is intended to remain stationary, as is typically required for diagnostic imaging procedures).

[0127] A similar effect may be achieved by performing 2D imaging in two non-coplanar image planes, such as presenting and / or analyzing and / or visualizing two orthogonal (but not limited to) views of an object acquired from at least two different cameras.

[0128] This output allows the operator to make an objective decision prior to actual diagnostic image acquisition whether repositioning of the object may be necessary to obtain a good diagnostic (e.g., X-ray) image. This decision can be made by the operator, or automatically, for example, by a simple decision algorithm (e.g., a metric expressing the magnitude of motion, such as the difference metric discussed above exceeding a predetermined threshold) and / or an artificial intelligence-based algorithm (e.g., a trained machine learning model).

[0129] In summary, the method may acquire at least one reference photographic image and / or depth (or 3D) image (or at least spatial data, such as 3D surface data in the form of a 3D point cloud or surface mesh) when the operator or controller decides that the positioning setup is complete (substantially acquiring reference data upon receiving the trigger discussed above). The actual video stream from the same camera system may then be compared (e.g., continuously) to the reference data and, for example, shown in an overlay. Movement may also be quantified and provided as an output, e.g., generating an alarm when a large amount of movement away from the reference occurs. Deviations from the current camera image(s) and the reference image(s) indicating a "good" positioning of the object may be evaluated by an AI system, conventional algorithms and / or may simply be displayed on a monitor for objective decision making by the operator as to whether repositioning of the patient is required when diagnostic image acquisition is to be performed.

[0130] Output may be provided, for example, via a display monitor of a console computer used to control the diagnostic imaging system (e.g., outside the acquisition room), or via a separate monitor dedicated to this purpose (e.g., thereby avoiding the need for any integration with the diagnostic imaging system).

[0131] When deviations from the original spatial configuration (frozen in the reference data) are too great to proceed with diagnostic image acquisition, an operator (e.g., a Medical Technical Assistant, MTA) may be signaled explicitly and / or implicitly (by allowing the operator to compare states in a convenient data presentation (e.g., an image overlay)).

[0132] In a second aspect, the invention relates to an apparatus for detecting movement of at least one body part of a subject in a diagnostic imaging examination. Figure 3 The apparatus 10 according to an embodiment of the present invention is schematically illustrated. For example, the diagnostic imaging examination may include an X-ray imaging examination, such as a projection radiography examination. Thus, the apparatus 10 may be included in a diagnostic imaging system, such as an X-ray imaging system, or in a workstation for such a system. For example, the X-ray imaging system may be an X-ray projection radiography system.

[0133] The device 10 can be adapted to detect changes in the position, orientation, and / or other spatial configuration of an object or at least an examination-related anatomical structure of the object relative to a reference state of the object, such as being correctly positioned for examination. Thus, the device can be adapted to assist in patient positioning in preparation steps for an imaging session, such as by detecting undesired movement of the object (e.g., patient) after completing initial preparations to place the object in the correct spatial configuration for examination and before actually performing a diagnostic imaging acquisition. For example, an operator (e.g., MTA) can bring the patient into a position (typically, spatial configuration) that is deemed suitable or desired (e.g., optimal for an intended diagnostic imaging examination).

[0134] The device 10 may include a processor, a data storage memory, (one or more) inputs, (one or more) outputs, a user interface and / or other modules generally known for performing the method as discussed above, such as when programmed and / or configured accordingly. Therefore, the device may include a computer and a computer program product according to an embodiment of the present invention (i.e., suitable for being executed by a computer). Additionally or alternatively, the device may include hardware specifically designed and / or configured to perform the method according to the first aspect of the present invention, such as a field programmable gate array including a dedicated integrated circuit and / or a configuration to perform the method according to the first aspect of the present invention. The device may include a computer. The device may also include multiple processors, computers and / or other processing devices such as using a host-client architecture, a web-based architecture, cluster processing and / or other forms of distributed processing, data collection, data presentation and / or data storage to collaborate on actions.

[0135] The device comprises a processor 12 and an output 13. The device further comprises an input 14 for receiving a trigger for indicating that the current spatial configuration of the subject is to be maintained for the diagnostic imaging examination.

[0136] The input unit 14 may include (or may be included in) a human interaction interface, for example, may include a button 17, a voice control interface 18, and / or a gesture detection system 19 for detecting gestures made by an operator. For example, once the object is in the desired position, the operator may provide a trigger to indicate that the current position (i.e., spatial configuration) of the object is to be maintained. It should be understood that the gesture detection system 19 may include the camera system 11 discussed below in combination with appropriate processing (gesture detection) performed by the processor 12. Similarly, the voice control interface 18 may include a microphone in combination with appropriate processing (voice command recognition) performed by the processor 12.

[0137] The input 14 may include a proximity and / or position detection system 16 to detect the proximity of the operator to the object. For example, the proximity and / or position detection system 16 may include an RFID tag on the operator's body that is combined with at least one RFID detection sensor (near the object, i.e., in, at, or near the inspection area of ​​the diagnostic imaging system). The proximity / position detection system 16 may include at least one light gate to detect when the operator leaves the vicinity of the object (i.e., the inspection area). The proximity / position detection system 16 may include a sonar, radar, and / or lidar system, and / or another sensor system for the presence, proximity, and / or position detection of the operator. Therefore, a trigger may be generated when the proximity detection indicates that the operator has left the vicinity of the object, or when the operator's position is detected to be substantially away from the object (e.g., at a distance that makes the operator's direct physical interaction with the object impossible and / or impractical for the purpose of assisting in locating the object). The proximity / position detection system 16 may also use the camera system (in conjunction with appropriate processing by the processor) to detect when an operator leaves an inspection zone for a process setting object, e.g., when the operator leaves a camera image frame of a camera(s) used to monitor the position of the object and / or is detected as being sufficiently far away from the object in a camera image frame captured by another (e.g., long-lens) camera dedicated to that purpose. Thus, the processor may be adapted to perform scene analysis of a live imaging stream from the camera system to detect a predetermined condition for generating a trigger and generating a trigger in response, wherein the predetermined condition corresponds to an operator leaving the immediate vicinity of the object.

[0138] Additionally or alternatively, the input 14 may be adapted to receive a trigger from an automated system for detecting a predetermined reference spatial configuration of an object as required for a diagnostic imaging examination in a live imaging stream provided by the camera system 11. Thus, as is known in the art, a device for determining a suitable spatial configuration of an object for a particular examination and for directing the object into a desired spatial configuration may be used to generate the trigger. Since such an automated system may be implemented by the same processing hardware, the trigger may be a virtual signal, for example transmitted via a memory flag, a file, a network socket, or a similar mechanism for inter-process communication. According to an embodiment, such an automated system may also be included in the device 10, but not necessarily.

[0139] The device 10 may also include an artificial intelligence module (e.g., a machine learning processor core) for evaluating a trained machine learning model. For example, the AI ​​module may be included in the automation system mentioned above. The AI ​​module is suitable for generating a trigger signal by considering the output of the evaluated model. The trained machine learning model uses, for example, images and / or spatial data obtained from a camera and / or a 3D surface scanning system 11 and / or data derived therefrom and / or data from a diagnostic imaging system received, for example, via an interface with a diagnostic imaging system as input. Such data received from the diagnostic imaging system may, for example, include device status information of the diagnostic imaging system and / or control input information received by the diagnostic imaging system from user interaction. Therefore, the AI ​​module may detect (implicit) laws indicating the completion of the patient positioning step in the use of the diagnostic imaging system. Such implicit laws and relationships may be pre-learned by the model, for example, based on a suitable training data set. It will be clear that the AI ​​module is not necessarily physically separated from the processor discussed below, for example, the processor 12 may be suitable for implementing the AI ​​module. Alternatively, the AI ​​module may be a dedicated hardware module (e.g., ML core) for evaluating a trained machine learning model (e.g., a trained neural network).

[0140] It will also be understood that the input may combine different information sources to determine the trigger signal to be relayed to the processor. For example, different user interfaces and / or detection modules may be combined, and their (trigger) outputs may be combined in any suitable (and / or user-configurable) manner to obtain a trigger for the processing discussed below.

[0141] The device 10 comprises a camera and / or 3D scanning system 11 for acquiring images and / or spatial data of an object. The camera and / or 3D scanning system may comprise at least one camera, which is arranged, for example, to obtain a two-dimensional and / or three-dimensional view of the object or at least (one or more) relevant body parts of the object. The camera and / or 3D scanning system may comprise at least one optical camera for acquiring a two-dimensional image. The camera and / or 3D scanning system may comprise a plurality of cameras, which are, for example, arranged at different positions and / or so as to view the object from different angles.

[0142] The camera and / or 3D scanning system may be adapted to acquire depth information. The camera and / or 3D scanning system may include a depth camera. The camera and / or 3D scanning system may include multiple two-dimensional optical cameras to acquire (e.g., substantially simultaneously) two-dimensional images from multiple different observation points. Thus, the camera and / or 3D scanning system (and / or the processor) may be adapted to determine depth information from the two-dimensional images, for example by applying an algorithm for depth inference from stereo or multi-camera images.

[0143] The processor 12 is adapted to acquire a reference image and / or spatial data of the object using the camera system upon receipt of the trigger. The reference image and / or spatial data may be acquired at or a predetermined short time after the moment of receipt of the trigger, and / or the reference image and / or spatial data may be selected from a buffer storing an image and / or spatial data stream acquired by the camera system, for example, wherein the selection corresponds to a (e.g., most recent) most recent time point before receipt of the trigger.

[0144] The processor is further adapted to acquire further images and / or spatial data of the object using the camera system after acquiring said reference image and / or spatial data of the object.

[0145] The reference image and / or spatial data (and further images and / or spatial data, typically of the same type and nature and typically corresponding to the same setup configuration) may include conventional monochrome and / or color digital photographic images in the visible and / or infrared spectrum. The reference (and further) images and / or spatial data may include multiple simultaneously (or at least concomitantly) acquired images from different cameras (e.g., from different observation points). The reference (and further) images and / or spatial data may include 3D and / or depth information.

[0146] The processor is adapted to compare a reference image and / or spatial data of the object (which represents a reference state of the object substantially at the time the trigger is received) with a further image and / or spatial data of the object (which represents a more recent state of the object).

[0147] The processor may be adapted to compare reference and further image and / or spatial data of the object by detecting at least one image feature and / or landmark in the reference image and / or spatial data and the further image and / or spatial data, and comparing the position of at least one image feature and / or landmark between the reference and further image and / or spatial data for determining the output. The image feature and / or landmark may include at least one anatomical landmark on a joint, bone, muscle and / or other externally discernible anatomical feature of the subject's body. The image feature and / or landmark may include one or more salient image features, for example, such that the position of a point on the imaged object (subject) corresponding to the salient image feature may be stably, accurately and robustly tracked in different images (particularly at least in the reference image / spatial data and the further image / spatial data).

[0148] The processor may be adapted to compare the reference data and the further image and / or spatial data of the object taking into account the three-dimensional information, for example by determining 3D points (or surface segments or spatial regions) for which corresponding acquired depth (and / or 3D) information in the reference image / spatial data differs significantly from corresponding depth (and / or 3D) information in the further image / spatial data. Such identified points (or surfaces or spatial regions) may be clustered or otherwise combined (e.g., summarized in a suitable data representation) to allow a 3D visual representation of the differences (e.g., in an output image and / or overlay) and / or to quantify the movement (displacement) of the patient in his / her current state relative to the reference state (e.g., for use as or in a difference metric discussed further below).

[0149] The processor is adapted to provide output data to the operator and / or the object via the output portion 13. The output data represents the comparison of the additional image and / or spatial data with the reference image and / or spatial data so as to indicate the movement of the object relative to the reference state of the object. For example, the shape, area and / or volume of the detected (2D or 3D) difference can be presented (i.e., visualized) as an (e.g., color) overlay and / or contour on another image (e.g., on the most recently acquired image of the object) and / or a 3D model visualization of the scene (e.g., a 3D visualization of the most recently acquired image / spatial data of the object). For example, the 3D outer surface of the object (or its associated body part) can be drawn, and the difference area can be marked with a different color or with other properties (different from the property value drawn for the main surface) to highlight the difference. In the case of a superimposed display of the difference on the additional (e.g., most recently acquired) image / spatial data, it should be understood that the same effect can be achieved by superimposing the difference on the reference image / spatial data, even if it may be preferred to visualize the live data stream with the difference annotation (overlay) in order to avoid any loss of available information due to processing artifacts when presented to the operator.

[0150] This output enables an operator to make an objective decision whether the subject needs to be repositioned to obtain a good diagnostic (e.g., X-ray) image prior to actual diagnostic image acquisition. This decision can be made by the operator or automatically, for example, by a simple decision algorithm (e.g., a metric expressing whether the magnitude of motion exceeds a predetermined threshold) and / or an artificial intelligence-based algorithm (e.g., a trained machine learning model) and / or a combination of the foregoing.

[0151] The processor 12 may be adapted to repeat the steps of acquiring additional images and / or spatial data via the camera system 11 (e.g., periodically and / or substantially continuously so as to obtain a live stream of additional images and / or spatial data), comparing the reference images and / or spatial data with the additional images and / or spatial data (e.g., comparing the most recently acquired additional data with the reference data), and providing output data. Thus, in each iteration, the output data may be updated to represent the current movement of the object relative to the reference state, e.g., so as to present a dynamic view of the current changes in the spatial configuration of the object relative to an earlier reference state of the object.

[0152] The device may include a user interface, for example, it may include a display monitor, a mouse, a keyboard and / or one or more similar human-machine interface devices known in the art. The user interface may be suitable for controlling the processes discussed above and below, for example to provide a trigger. The user interface may be suitable for interacting with an operator (e.g., a healthcare professional, a doctor, a nurse, an imaging technician, etc.). For example, an operator may use the user interface to control the device and / or monitor the positioning of a patient in the preparation phase of an imaging session.

[0153] The output 13 may for example comprise a display monitor 15. The processor 12 may be adapted to present a visual representation of the comparison via the display monitor. The processor 12 may be adapted to repeatedly perform the monitoring loop discussed above to dynamically update the operator with respect to the position (ie spatial configuration) of the object relative to the reference state.

[0154] The user interface may include an input 14 , but the input 14 is not necessarily (only) physically located at the same location as an output of the user interface (eg, display monitor 15 ).

[0155] The processor may be adapted to provide as said output data or part of said output data an image overlay showing said comparison in the form of a difference image as an overlay, such as a colour overlay, on said further image and / or spatial data.

[0156] The processor may be adapted to determine a difference metric indicative of movement of the object relative to a reference state based on the comparison, for example, based on a direct image comparison (e.g., a difference image) and / or corresponding pairs of image features and / or landmarks in the reference and further image / spatial data. The processor may be adapted to provide output data, wherein the output data comprises an alert to an operator when the difference metric exceeds a predetermined threshold so as to indicate that the object is no longer in the expected reference position. Additionally or alternatively, if the difference metric exceeds the predetermined threshold, a signal may be provided to the diagnostic imaging system to disable performance of diagnostic image acquisition until the object is repositioned (or until an operator overrides such a disable signal).

[0157] In a third aspect, the present invention relates to a diagnostic imaging system, wherein the system is adapted to perform a method according to an embodiment of the first aspect of the invention and / or comprises an apparatus according to an embodiment of the second aspect of the invention. In a fourth aspect, the present invention relates to a workstation for a diagnostic imaging system, wherein the workstation is adapted to perform a method according to an embodiment of the first aspect of the invention and / or comprises an apparatus according to an embodiment of the second aspect of the invention. The diagnostic imaging system may be a diagnostic (e.g. medical or veterinary) X-ray imaging system, such as a projection X-ray imaging system (e.g. for planar projection radiography).

[0158] For example, Figure 4 A diagnostic X-ray projection imaging system 30 according to an embodiment of the invention is schematically illustrated. Such an X-ray imaging system typically includes an X-ray detector 31 for acquiring an X-ray image of an object 32 by modulating the properties of an X-ray beam 33 emitted by an X-ray tube 34 as it propagates through the object. The operation of the system is typically controlled by a workstation 35 (e.g., a console), for example to configure the settings of the system, control the X-ray tube 34 (e.g., activation) and / or acquire images from the detector 31. For example, the workstation may also be configured to view the acquired X-ray images. The system includes an apparatus 10 according to an embodiment of the invention, which may (optionally) be integrated (e.g., at least partially) in the workstation 35, or may be provided substantially as a completely isolated system (e.g., for ease of installation, such as in an upgrade and / or portable installation). The camera system 11 of the apparatus 10 according to an embodiment is configured to monitor the object 32, for example so that at least the relevant anatomical structure of the object (in the example shown, the head) for the diagnostic imaging examination to be performed is included in the (one or more) viewing cones 37 of the (one or more) cameras of the camera system.

[0159] In a fifth aspect, the invention relates to a computer program product for performing a method according to the first aspect of the invention when executed on a computer, such as an apparatus according to an embodiment of the second aspect of the invention.

[0160] In view of the description provided above in relation to the method according to an embodiment of the present invention, other features or details of features of the apparatus (or computer program product, system and workstation) according to an embodiment of the present invention described above will be clear, and / or vice versa.

Claims

1. A device (10) for detecting movement of at least one body part of a subject in a diagnostic imaging examination, wherein: The device comprises: a camera and / or a three-dimensional 3D surface scanning system (11) for acquiring images and / or spatial data of the object, an input portion (14) for receiving a trigger signal for indicating that the current spatial configuration of the subject is to be maintained for the diagnostic imaging examination, a processor (12), and an output unit (13), Wherein, the processor (12) is adapted to: when the trigger signal is received, using the camera and / or the 3D surface scanning system to acquire a reference image and / or spatial data of the object such that the reference image and / or spatial data represent a reference state of the object substantially at the time when the trigger signal is received, After acquiring the reference image and / or spatial data of the object, acquiring further images and / or spatial data of the object using the camera and / or 3D surface scanning system, comparing the further image and / or spatial data with the reference image and / or spatial data, and Output data is provided to an operator and / or the subject via the output portion (13), wherein the output data represents the comparison so as to indicate movement of the subject relative to the reference state of the subject.

2. The device according to claim 1, wherein: generating the trigger signal prior to performing the diagnostic imaging examination such that the reference image and / or spatial data represents the reference state of the subject prior to performing the diagnostic imaging examination; and Wherein further images and / or spatial data of the object are acquired using the camera and / or 3D surface scanning system prior to performing the diagnostic imaging examination, such that the comparison indicates movement of the object prior to performing the diagnostic imaging examination.

3. The device according to claim 1 or 2, wherein: The processor (12) is adapted to repeat the following steps: acquiring the further image and / or spatial data via the camera and / or 3D surface scanning system (11); comparing the further image and / or spatial data with the reference image and / or spatial data; and providing the output data so as to present a dynamic view of the current changes of the spatial configuration of the object relative to the reference state of the object.

4. The device according to any one of the preceding claims, wherein The output unit (13) comprises a display monitor (15), and the processor (12) is suitable for presenting the shape, area and / or volume of the two-dimensional and / or three-dimensional difference determined by the comparison via the output data as an overlay and / or contour on the visualization of the further image and / or spatial data and / or as a marker and / or annotation accompanying the visualization.

5. The device according to any one of the preceding claims, wherein: The processor (12) is adapted to: determine, based on the comparison, a difference metric indicative of movement of the object relative to the reference state; and when the difference metric exceeds a predetermined threshold, output an alarm in the output data to the operator and / or the object and / or output a signal to the diagnostic imaging system to indicate that the object is no longer in the reference state.

6. The device according to any one of the preceding claims, wherein The camera and / or 3D surface scanning system (11) comprises at least one camera and / or 3D scanning device, and the at least one camera and / or 3D scanning device is arranged to obtain a two-dimensional and / or three-dimensional view of the at least one body part and / or the object, wherein the at least one camera and / or 3D scanning device comprises: an optical camera for acquiring monochrome, color and / or multi-spectral two-dimensional images in infrared and / or visible spectrum; and / or a plurality of such cameras configured to view the object from different angles; and / or a depth camera and / or a three-dimensional surface imaging system.

7. Apparatus according to any one of the preceding claims, wherein: The input unit (14) comprises a human interaction interface, a button (17), a voice control interface (18) and / or a gesture detection system (19).

8. Apparatus according to any one of the preceding claims, wherein The input (14) includes a proximity and / or position detection system (16) to detect the proximity of the operator to the object so as to generate the trigger signal by taking into account when the operator leaves the vicinity of the object.

9. The device according to claim 8, wherein: The proximity and / or position detection system (16) comprises an RFID tag in combination with at least one RFID tag detection sensor, and / or comprises at least one light gate, and / or comprises a sonar, radar and / or lidar system, and / or wherein the proximity and / or position detection system (16) is implemented by the processor (12) by detecting and / or tracking the position of the operator in a live stream of images and / or spatial data acquired by the camera and / or 3D surface scanning system (11).

10. The device according to any one of the preceding claims, wherein The input (14) comprises a connection for receiving the trigger signal from an automated system for detecting a predetermined reference space configuration of the object indicated for the diagnostic imaging examination in a live imaging stream provided by the camera and / or 3D surface scanning system (11).

11. The apparatus according to any one of the preceding claims, further comprising an artificial intelligence module for evaluating a trained machine learning model and generating the trigger signal by considering an output of the evaluated model, wherein The trained machine learning model uses the image and / or spatial data and / or data from the diagnostic imaging system as input, wherein the data from the diagnostic imaging system includes: equipment status information of the diagnostic imaging system and / or control input information received by the diagnostic imaging system from user interaction.

12. Apparatus according to any one of the preceding claims, wherein: The processor is suitable for comparing the additional image and / or spatial data with the reference image and / or spatial data by: detecting at least one image feature and / or landmark in the reference image and / or spatial data and in the additional image and / or spatial data; and comparing the position of the at least one image feature and / or landmark between the reference image and / or spatial data and the additional image and / or spatial data.

13. A diagnostic imaging system (30) comprising the apparatus (10) according to any one of the preceding claims.

14. The diagnostic imaging system of claim 13, wherein: The diagnostic imaging system is an X-ray imaging system or a projection X-ray imaging system.

15. A method (100) for detecting movement of at least one body part of a subject in a diagnostic imaging examination, the method comprising: receiving (101) a trigger signal indicating that a current spatial configuration of the subject is to be maintained for the diagnostic imaging examination; When the trigger is received (101), a reference image and / or spatial data of the object is acquired (102) using a camera and / or a 3D surface scanning system; After acquiring (102) the reference image and / or spatial data of the object, acquiring (103) further images and / or spatial data of the object using the camera and / or 3D surface scanning system; comparing the further image and / or spatial data with the reference image and / or spatial data (104), and An output is provided (105) to an operator and / or the subject, the output representing the comparison of the further image and / or spatial data with the reference image and / or spatial data to indicate movement of the subject relative to the reference state of the subject represented by the reference image and / or spatial data.

16. A computer program product for performing the method according to claim 15 when run on a computer.

Images