Patient / instrument positioning assistance by depth imaging

Abstract

An apparatus, system, method (100) and computer program product are disclosed for assisting in positioning an object including a subject and / or an assist device on a subject support in an imaging and / or treatment session. The method comprises acquiring (101), using a camera system, image / spatial data of the object support on which the object (s) are placed, the image / spatial data comprising depth / 3D information. The method comprises obtaining (111) reference data comprising reference depth / 3D information, the reference data representing the object support on which no object (s) are placed. Determining (104) at least one geometric attribute of the (or each) object from the image and / or spatial data, where the determining (104) comprises detecting (105) the object (s) by determining (110) 3D points of which the acquired depth / 3D information differs from the reference depth / 3D information and associating these points with the object. The geometric attribute for each object is determined by directly calculating sample statistic (s) for the point (s) associated with the object.

Description

Technical Field

[0001] The present invention relates to the field of diagnostic imaging and treatment systems, and more specifically to an apparatus, system, workstation, method and / or computer program product for monitoring and assisting in the positioning and / or orientation of an object and / or auxiliary instruments in an imaging and / or treatment environment, for example to support an operator in the preparation phase of an imaging and / or treatment procedure. Background Art

[0002] Preparing a patient for an examination that relies on diagnostic imaging techniques (e.g., to diagnose a medical condition and / or plan a treatment) and / or for a treatment procedure is typically a time-consuming task that relies heavily on manual intervention by specially trained personnel. Typically, a skilled operator needs to carefully perform the preparation and setup of the procedure according to established protocols and / or specific guidelines regarding the positioning of the patient and the use of auxiliary equipment.

[0003] For example, the operator may need to place a positioning device (such as a knee support, cushion, backrest, or headrest) to achieve a stable resting posture for the patient throughout the imaging or treatment procedure. To be consistent, the posture may need to conform as closely as possible to a reference position, or more generally, to a reference spatial configuration of the human body, which may depend on the specific type of examination or treatment intended, or even be specific to a specific patient, symptom, and / or patient condition, and / or other unique factors. It is also often very important to reproduce the exact patient setup over consecutive procedures for the same patient (e.g., during different imaging procedures and / or related treatment procedures).

[0004] For example, it is largely preferred to acquire (diagnostic) images of the same patient using different modalities and / or to acquire (diagnostic) images of the same patient representing different points in time in a way that makes the images as similar as possible to each other (e.g. so that the image contents are easily comparable or easily processed by dedicated software). Likewise, it may be important that the acquired images depict the spatial configuration of the body (or its relevant parts) as close as possible to a state that is reproduced in (one or more) future treatment procedures, e.g. so that radiotherapy (or different types of treatments) can be accurately and reliably planned and performed. For example, in radiotherapy, it is often necessary to reproduce as closely as possible the exact same patient setup in multiple (e.g. even up to forty) separate radiation delivery procedures.

[0005] In order to allow accurate positioning of auxiliary instruments, for example, positioning devices such as head and / or knee supports in a relatively quick and convenient manner, the patient support (i.e., a patient table or examination / treatment couch) may be equipped with mechanical means to provide a set of fixed discrete positions to which such auxiliary equipment can be mounted (e.g., by bolts, screws, hooks, clamps, etc.). For example, these discrete positions can be easily recorded and referenced by marking them and referencing (one or more) positions for use in patient files, guidelines, and / or other such information resource repositories. As an example, 12 equidistant linear positions along the longitudinal axis of the support can be labeled from H4 to F7 ("head 4", "head 3", ..., 0, "foot 1", ... "foot 7"). The range of such discrete indexes is shown, for example, in Figure 3 Thus, devices such as knee braces can be positioned at these specific discrete locations. The locations used or to be used can be conveniently noted in the patient record and / or in notes in the procedure guide.

[0006] A meticulous quality assurance process may typically require detailed reporting of all relevant setup information for each procedure, i.e. to be able to compare, evaluate and / or reproduce the setup. It should be understood that the manual workflow for positioning the patient and auxiliary positioning equipment and recording relevant information can be a time-consuming and error-prone task.

[0007] It is known in the art (e.g. in particular in radiotherapy planning) to use radio frequency identification tags (RFID) to check the presence of the correct type of localization devices and their respective positions. For example, such a system may use a plurality of RFID sensors and tags, which may be integrated in a set of localization devices and indexing and / or clamping devices. This allows the system to use a set of registration devices, i.e., they are equipped accordingly with the components of the system.

[0008] Although RFID can help to automate, reproduce and protect the positioning setup, reporting and reproduction workflow, this also has some limitations. For example, a typical patient setup may be recorded only to a certain extent, that is, some complexity of the configuration may be lost or may still need to be recorded manually. When customized or third-party devices are not equipped with the required RFID components, such systems are generally unable to handle these devices, and it is also relatively difficult to add the required RFID components to the customized or third-party devices for various technical (but also possible other) reasons (such as compliance, quality assurance, maintenance, manufacturer support, calibration and / or certification).

[0009] Since RFID-equipped systems typically rely on many components that work together, such as RFID tags and sensors, the potential for error generally increases with the number of devices over which the components are distributed, i.e., since each tag and sensor can fail independently. Another potential disadvantage is the limited range of RFID signals, which are essentially a technology for short-range communications, making position detection unusable over longer distances between tags and sensors.

[0010] Note also that some positioning aids are not suitable for installation along a fixed predefined linear and / or grid index, or the available (discrete) positions may be too limited for a particular case. Correct use of the device also implies other parameters that are not easily encoded in a single position parameter.

[0011] Examples of devices that are not easily positioned in a reproducible manner by relying solely on a discrete grid and / or linear coordinate system may include support wedges, flexible devices, pads, cushions, and custom patient-specific positioning devices. For example, when using prior art RFID-based systems, such devices are often not automatically included in the setup report, which increases the potential risk due to errors in (manual) reporting, can result in time inefficiencies due to additional manual reporting steps, and / or limits the ability to accurately reproduce the configuration.

[0012] More generally, complex patient setups can be difficult to reproduce, for example where detailed step-by-step instructions need to be followed to achieve a correct outcome, when using prior art RFID-based systems or even using any manual or (partially) automated methods that use a discrete positioning grid.

[0013] Examinations used to plan and / or monitor treatments may involve magnetic resonance imaging, so that systems for assisting in positioning patients and ancillary equipment will preferably be suitable for use in such MRI procedures (preferably in addition to imaging procedures that may be used for related treatments and / or using other imaging modalities). This illustrates another possible disadvantage of RFID-based systems. Such systems rely on radio frequency signal transmission, which may be interfered with by the magnetic fields and radio frequency signals used by the MRI system. This may also imply RF safety issues, as RFID technology may interfere with MR imaging and / or other medical equipment, placing patients and / or instruments at risk. Although additional safety measures and design choices may address this issue, this will inevitably lead to increased costs and efforts for its integration and implementation.

[0014] It is known in the art to position a movable patient table relative to an imaging system based on camera observations. For example, US10181074 B2 describes such a method, wherein a freeze-frame camera image is used, on which an operator can define reference position information. The examination table can then be moved to make the reference information consistent with the acquisition area of ​​the system. It is also known (see, for example, US2009 / 182221) to store the camera image positions of the patient and MRI receiving coil in a medical picture archiving and communication system (PACS) server, so that this information can be presented to the operator in a subsequent process to reproduce the setting. Although camera-based patient positioning can provide many advantages, there is still a need in the art for a method to provide a simple, highly automated, reliable, reproducible, versatile and accurate positioning of the patient and / or any auxiliary instrument, such as a positioning aid, and an accurate and / or reliable record of such position information. Preferably, the positioning system will be MRI-compatible and able to handle any instrument item, such as requiring little or no modification of the positioning device, even when provided by a different manufacturer or when it is patient-specific, such as 3D printed for a specific patient.

[0015] US2016262714A1 describes a method for determining a tube current curve for recording an X-ray image. An image of a body region of a patient to be depicted by an X-ray image is recorded via an optical sensor. Body-related information is determined from the image, and the tube current curve to be used is determined by taking into account the X-ray attenuation distribution determined from the body-related information.

[0016] US2019143145A1 discloses a method for real-time monitoring of patient positioning and / or position during a radiation treatment process. Images of the patient acquired during the treatment process can be used to calculate the positioning and / or position of the patient relative to components of a radiation therapy system. The patient monitoring imaging system may include a plurality of cameras mounted on a rotatable ring (e.g., mounted on a frame of a radiation therapy system).

[0017] US2018116518A1 describes a method for providing preparation information in magnetic resonance imaging. An inspection target (e.g., a patient) is supported on a patient support of an MRI system. Preparation information for preparation of magnetic resonance imaging is then generated based on a depth map of the target on the support acquired by a time-of-flight camera.

[0018] US2013342851A1 relates to another method for collecting information related to a target on a patient positioning device of a medical imaging device. The method includes collecting 3D image data of a target on the patient positioning device by an optical 3D imaging device, and determining information related to the target based on the 3D image data.

[0019] EP2727535A1 relates to a radiographic imaging device with a camera mounted on a frame, wherein volume data of the object is generated based on an image of the object captured by the camera, and an optimal radiation dose is calculated based on the volume data of the object. Summary of the invention

[0020] It is an object of embodiments of the present invention to provide a good and / or effective means and method to assist in positioning and / or orienting objects (e.g., patients) and / or instruments in diagnostic imaging and / or treatment systems (and / or to facilitate the recording of such positioning information).

[0021] The apparatus mentioned above may generally refer to one or more items that can be placed in the system (e.g., therefore in the imaging and / or treatment environment) to support, stabilize, orient, shape (e.g., deform, bend) or otherwise affect the spatial configuration of the patient, i.e., may refer to positioning devices such as pads, supports, cushions, masks, stereotactic reference devices, physical markers, etc. Optionally, the apparatus may also include (or even consist of) other devices that can be freely or at least variably placed, oriented, shaped, etc. relative to the object, such as sensor devices, electrocardiogram monitors, respiratory belt instruments, etc., which have other functions in addition to or not limited to only positioning assistance. The latter "other" devices will have a positioning and / or spatial relationship with the object during their use, which may be related to accurate recording and / or reproduction and / or configuration according to predetermined specifications.

[0022] The apparatus may exclude fixed components of the imaging system, i.e., movable (typically actuated or controlled) parts of the imaging / treatment system that are substantially stationary in the environment or fixed to the imaging / treatment system, such as fixed to a gantry, magnet bore attachment, linear accelerator, robotic arm, or similar other key components of the imaging / treatment device. Nevertheless, embodiments are not necessarily limited by this illustrative exclusion, for example in some applications, it may even be advantageous to assist in positioning such (e.g., important) system components (e.g., radiation sources, imaging detectors, etc.). This is the case, for example, in specific applications where such key components of the imaging / treatment system rely substantially on manual positioning (e.g., portable (non-mechanized) C-arm imaging systems, bedside projection radiography systems, isolated gamma radiation sources, etc.).

[0023] An advantage of embodiments of the present invention is that a patient and a positioning device may be placed in an imaging and / or treatment environment via a partially automated, machine-assisted procedure.

[0024] An advantage of embodiments of the present invention is that the position and / or spatial configuration (e.g. orientation, shape, relative position / orientation...) of a patient and / or a positioning aid can be easily and accurately recorded, for example so that the same configuration can be easily, accurately and / or reliably reproduced.

[0025] An advantage of embodiments of the present invention is that the burden of the time-consuming task of setting up a patient for an imaging or treatment procedure can be alleviated, for example, so that an operator can work more efficiently. For example, the disadvantages of a (completely or primarily) manual workflow, such as being prone to errors, time-consuming and / or requiring an operator to perform many quality checks completely manually, can be substantially reduced.

[0026] An advantage of an embodiment is that good reproducibility and / or consistency in positioning a patient (and / or an instrument) may help improve the efficiency and / or accuracy of diagnosis and / or treatment.

[0027] An advantage of embodiments of the present invention is that manual intervention required in positioning a patient and / or a device may be reduced and / or simplified.

[0028] An advantage of embodiments of the present invention is that the training requirements and / or experience of staff required for correct positioning of a patient (including the use of positioning aids) can be reduced or relaxed. An advantage of embodiments of the present invention is that errors and mistakes in positioning a patient and a positioning device can be reduced or avoided.

[0029] An advantage of embodiments of the present invention is that embodiments of the present invention can provide real-time (e.g., substantially continuously updated) position tracking, positioning guidance and / or position recording (and / or tracking, guidance and / or recording of other geometric parameters).

[0030] An advantage of embodiments of the present invention is that assistance can be provided in locating and recording the position (and possibly other spatial configuration properties) of an auxiliary positioning aid without unduly restricting the possible positions and / or configuration options, for example to a fixed, discrete set of selectable positions.

[0031] An advantage of embodiments of the present invention is that no mechanical means for attaching and / or fixing the positioning device to the patient support table is required to use methods and devices according to embodiments if this is not considered necessary or advantageous for medical and / or other practical reasons.

[0032] An advantage of embodiments of the present invention is that the position and / or other spatial configuration parameters of the patient and positioning device can be easily recorded for subsequent use and retrieved for reuse if previously stored. Not only the position of the patient and / or positioning device relative to the patient table can be recorded, but also other geometrical properties such as orientation, shape, elevation and / or deformation.

[0033] An advantage of embodiments of the present invention is that the positioning guidance provided by the embodiments can be applied to general, non-standard, third-party, customized and / or generally arbitrary auxiliary devices without modification, alignment, mathematical modeling and / or other initial configuration costs. For example, any positioning device can be used in combination with embodiments of the present invention as long as it can be observed and automatically detected by its camera system. There is no need for a learning phase or a large amount of setup or configuration for the system according to the embodiment to recognize a specific device, for example, there is no need to train or configure the system for a specific shape or appearance of the positioning device. Therefore, new devices that have not been used in combination with the system before, such as devices that have not been previously observed by the camera, can be used at any time. Therefore, there is no need for specific hardware modifications to the auxiliary device.

[0034] An advantage of embodiments of the present invention is that the positioning guidance provided is not limited to short-range wireless communication between the sensor and the tag.

[0035] An advantage of embodiments of the present invention is that embodiments of the present invention are compatible with, usable in, and / or easily integrated into magnetic resonance imaging systems, computed tomography imaging systems, and / or radiotherapy systems, for example without substantial interference and / or other safety risks. For example, in an MRI, a camera can be used to observe from outside the volume of space where the camera may interfere with MRI imaging (e.g., at a sufficient distance). For example, a camera or combination of cameras can be located outside or near the edge of the main magnet housing, such as near or on the flange of the bore, from which it can observe the patient and any instruments of interest without interfering with the sensitive radio frequency / magnetic gradient system.

[0036] An advantage of embodiments of the present invention is that positioning guidance, recording, quality assurance and / or management can be provided in combination with potentially any diagnostic imaging system, such as computed tomography (CT), magnetic resonance imaging (MRI), projection radiography, single photon emission tomography (SPECT), positron emission tomography (PET) and / or other imaging modalities, and / or in combination with any treatment modality in which accurate positioning of the patient (and / or instrument item) is important, such as radiotherapy systems, such as linear accelerator (LINAC)-based treatment systems, robotic arms and / or image-guided surgical suites, and potentially other treatment systems. This can also include combined imaging and treatment systems, such as where a CT and / or MRI system (not limited to said combination) is integrated in a treatment system, such as a MR-LINAC system.

[0037] An advantage of embodiments of the present invention is that good accuracy, efficiency and / or effectiveness can be achieved in radiation therapy and / or in general image-guided treatments (e.g., HIFU). For example, high-quality image-guided treatments, such as MRI or CT-guided radiation therapy, can be achieved. An advantage of embodiments of the present invention is that MRI and / or other types of imaging can be achieved with good image quality and a reproducible representation of the target anatomical structure. An advantage of embodiments of the present invention is that short procedure times can be achieved in MRI, CT, and / or other imaging modalities, for example, due to a shortened time requirement for initial setup of the patient.

[0038] Another advantage is that camera viewing is often used in MRI imaging to monitor patients. Therefore, existing camera systems can be easily upgraded and / or upgradeable to implement embodiments of the present invention, and / or suitable MRI-compatible camera systems can be readily used with embodiments of the present invention. Camera viewing can often be used in diagnostic imaging and / or treatment environments, such as CT, PET, SPECT, radiation therapy, ...

[0039] An advantage of embodiments of the present invention is that a pre-installed camera system of such an imaging or treatment kit can be readily adapted for use with the present invention, for example, to provide positioning guidance and / or recording in addition to existing patient viewing capabilities. For example, existing camera equipment can be replaced, expanded and / or upgraded to support previously unavailable depth imaging, for example, in combination with appropriate processing as further explained below.

[0040] An advantage of embodiments of the present invention is that, for example, different diagnostic images (2D, 3D or even time-sequential or grouped images) acquired for the same patient in different examinations on the same or heterogeneous imaging modality can be accurately, efficiently and / or easily registered with each other, for example due to good consistency and reproducibility of patient position, orientation and / or spatial configuration between different procedures.

[0041] An advantage of embodiments of the present invention is that image registration errors and / or artifacts in registered images may be reduced, and therefore the propagation of such errors / artifacts to further images, treatment plans, treatments and / or other types of information or final results derived therefrom or dependent thereon may also be reduced.

[0042] An advantage of embodiments of the present invention is that the use of depth imaging allows collecting information not only about the in-plane position (and / or orientation...) of the instrument (and the patient) relative to the (usually horizontal) patient table, but also about the height (orientation, shape...) in the direction normal to the plane, although the latter may also be relevant for accurate and repeatable positioning. For example, this allows determining, recording and / or accurately reproducing the elevation level and / or angle of, for example, a leg, foot, arm, hand, torso (or specific parts thereof) and / or head, for example when such body part(s) are supported by positioning device(s).

[0043] Depth imaging can also advantageously avoid false detections due to shadows, uneven lighting and / or irrelevant (substantially) thin objects (e.g., a piece of paper) on the patient table. Conventional two-dimensional photographs (or videos) (e.g., as can be obtained with a monochrome or color camera) would typically require more complex image processing and / or would have difficulty distinguishing such interfering objects and / or lighting artifacts from the object(s) of interest.

[0044] In embodiments of the present invention, this also implies that, for example, due to (or to some extent due to) the use of depth imaging techniques, objects can be detected by relying on the implicit volumetric properties of the device (instrument item), and possibly also distinguishing different objects through algorithms, for example, without the need to train or configure the system for specific properties or other properties of the object. For example, for devices that play an important role in positioning and / or immobilizing a patient, this volumetric property may be implicit, for example, this may generally involve at least some basic physical volume and / or weight to achieve the intended purpose (in a robust and stable manner).

[0045] The method, computer program product, device, system and / or workstation according to the embodiments of the present invention achieve the above-mentioned objects.

[0046] In a first aspect, the present invention relates to a method for assisting in positioning and / or orienting an object (such as an object and / or at least one auxiliary instrument item) on an object support (e.g. a patient couch, an imaging / treatment table) during an imaging and / or treatment procedure (i.e. using an imaging and / or treatment system). The method comprises acquiring, using a camera system, an image and / or spatial data of the object support on which at least one of the objects is placed, wherein the image and / or spatial data comprises depth (and / or 3D) information. The method further comprises obtaining reference data representing the object support on which the at least one object is not placed, the reference data comprising reference depth (and / or 3D) information.

[0047] The method comprises determining at least one geometric property of the at least one object (or each of the at least one object) based on the image and / or spatial data, wherein such determination of the at least one geometric property comprises detecting the at least one object by taking into account at least the reference depth (and / or 3D) information in the reference data and the depth (and / or 3D) information in the acquired image and / or spatial data.

[0048] Thus, the method may include detecting the position, orientation and / or other spatial properties of the object (e.g., patient) and / or (one or more) auxiliary instrument items, and / or detecting the spatial relationship between the object and the instrument and / or between different instrument items. This spatial configuration information may be used to assist in patient and / or instrument positioning in preparatory steps in an imaging and / or treatment procedure, and / or may be stored for future reference.

[0049] The step of detecting includes determining three-dimensional points in the corresponding acquired depth (and / or 3D) information that are substantially different from the corresponding reference depth (and / or 3D) information, and associating the points and / or clusters of the points (e.g., automatically determined) with one or more objects being detected.

[0050] Determining the at least one geometric property of each of the objects further comprises computing one or more geometric properties directly from the points and / or clusters of points associated with each object by computing, for example, one or more statistics, for example summary statistics, such as sample statistics, of the points and / or subsets thereof. In other words, direct computing refers to computing that operates on a set of points to produce a value of interest (output of the computing) in a substantially deterministic (e.g., algebraic) manner and / or without requiring additional input (e.g., without resorting to, for example, fitting a 3D model, parameter adjustment, numerical optimization, and / or other methods that rely on parameters, models, and / or other auxiliary information).

[0051] The at least one geometric attribute may include a centerline calculated from the three-dimensional points associated with the corresponding object. For example, the set of points associated with the object may be divided (e.g., partitioned) along a predetermined axis (e.g., a longitudinal axis) to define a plurality of subsets. The centroid locations of the subsets may be used to create the centerline.

[0052] The at least one geometric attribute may include a cluster center location calculated from the three-dimensional points associated with the corresponding object (eg, a 1, 2, or 3-coordinate location of a centroid of a cluster of points).

[0053] The at least one geometric attribute may comprise a patient support index calculated based on the cluster centroid positions, for example by discretizing (at least one coordinate component) of the cluster centroid, for example by selecting the nearest discrete position index from a set of (e.g. available / implemented / physically feasible) predetermined discrete position indices.

[0054] In a method according to an embodiment of the invention, the reference data may include a three-dimensional geometric model of the object support and / or at least one depth image of the object support.

[0055] For example, the reference data may comprise a three-dimensional model of the object support, such as a computer-aided design (CAD) model. Such a model may thus comprise, for example, a 3D geometrical model of the scanner couch, and optionally also other relevant components of the system, such as components visible (to the camera system). In particular, it is noted that, according to an embodiment, such model(s) may be used for substantially fixed and / or stationary (e.g. substantially involved in each procedure; and not motionless) (one or more) components of the system, while the objects of interest (those detected and located by the method) are analyzed via their detected point cloud representations, i.e. such geometrical models of the objects are not required (and therefore not used).

[0056] In a method according to an embodiment of the present invention, obtaining the reference data may include receiving a current position of the object support from an automatic object support actuation system, and / or determining the current position based on acquired images and / or spatial data. Obtaining the reference data may also include determining the reference depth (and / or 3D) information based on the current position combined with a three-dimensional geometric model of the object support (e.g., defined based on its position), and / or based on the current position and a set of reference depth (and / or 3D) images of the object support (e.g., corresponding to multiple different positions).

[0057] In a method according to an embodiment of the invention, the steps of acquiring the image and / or spatial data and detecting the at least one of the objects may be repeated. After each iteration (of the repetition), the reference data may be updated to include the detected object or objects, so that the updated reference data represents the object support having the one or more objects monitored up to the iteration. Then, in the next iteration, one or more additional objects may be placed on the object support to be detected, for example to guide an operator to place the additional object(s), while the previously detected object(s) remain substantially stationary.

[0058] In a method according to an embodiment of the present invention, the at least one geometric attribute may comprise a discrete object support index selected from a predetermined discrete ordinal set of object support indices (e.g. discrete reference longitudinal table positions for reference in a radiotherapy setting) to identify a position of a detected object associated with at least one geometric attribute along a (primarily, e.g. typically longitudinal) direction of the object support.

[0059] In a method according to an embodiment of the present invention, the at least one geometric attribute may include at least one value indicating the position and / or orientation of the object associated therewith, for example, in one predetermined direction (e.g., a longitudinal worktable coordinate), in two predetermined directions (e.g., a ordinate and a lateral coordinate) and / or in three predetermined directions (e.g., also including a height (e.g., vertical) coordinate) and / or its equivalent with respect to orientation (e.g., one or more rotations around one, two or three axes).

[0060] In a method according to an embodiment of the invention, the at least one geometric attribute may comprise a centre line of the or each detected object, for example a centre line along a predetermined axis (e.g. the longitudinal axis of the object support) or about a straight line detected as a principal axis of the object, for example to show a deformation of the object relative to an automatically determined predetermined, average, best fit or similar representative straight line axis of the object.

[0061] In a method according to an embodiment of the invention, the at least one geometrical attribute may comprise a straight line, a line segment or a curve, or a set of points indicating such a line, for example, which represents a centre line of the detected object.

[0062] The method according to an embodiment of the present invention may include presenting to an operator at least one geometric attribute determined for one or more detected objects (e.g., for each object). It should be understood that some distinctions may be made, such as determining and / or showing different attributes (one or more) for patients and for inanimate objects. It should also be understood that different (one or more) attributes may be determined and presented for presentation purposes, rather than for information recording purposes (data storage for subsequent use). This does not exclude the same set of attributes used for patients and for inanimate objects or for presentation and for storage, nor does it exclude possible overlap between such sets for different types of objects (patients vs. instruments) or different intended uses (presentation vs. recording).

[0063] In a method according to an embodiment of the present invention, the presentation may include presenting the at least one geometric attribute to an operator in a visual form on at least one captured camera image and / or in an overlay on a geometric model extracted from the captured image and / or spatial data and / or from obtained reference data.

[0064] In a method according to an embodiment of the present invention, the shape, area (e.g., projected area on the image plane) and / or volume of one or more detected objects can be presented to an operator as a (e.g., colored) overlay or outline on a model of the at least one camera image and / or scene.

[0065] In a method according to an embodiment of the invention, the centerline of the detected one or more objects may be presented to an operator on the at least one camera image and / or a model of the scene. Additionally, the centerline may be presented on two (e.g. orthogonal) views in 3D (e.g. a 3D model) by showing a height (vertical) profile in addition to a (horizontal) plane representation, or by another visualization allowing the operator to evaluate both in-plane alignment and normal-to-plane alignment simultaneously.

[0066] A method according to an embodiment of the present invention may include repeatedly performing image and / or spatial data acquisition, determining at least one geometric attribute, and presenting at least one geometric attribute (e.g., repeating this sequence of steps) to dynamically guide an operator to locate and / or orient one and / or more detected objects.

[0067] A method according to an embodiment of the present invention may include storing the (one or more) geometric attributes in a data store for future reference in another process, for quality assurance, for workflow archiving, and / or for further processing and / or evaluation of information collected in the image and / or treatment process.

[0068] A method according to an embodiment of the invention may comprise retrieving the said (one or more) geometrical attributes stored in a previous process and, for the purpose of comparison, including the information thus retrieved (not necessarily excluding any intermediate further processing, transformation or other type of digital operation of the retrieved information) in the said presentation.

[0069] In the method according to an embodiment of the present invention, the image and / or spatial data may be acquired from one or more camera positions arranged on the object support, thereby obtaining a two-dimensional and / or three-dimensional top view of the object support.

[0070] In a method according to an embodiment of the present invention, collecting the image and / or spatial data may further include collecting at least one two-dimensional image from an optical camera included in the camera system, wherein the optical camera is sensitive to one or more spectral bands in the infrared and / or human visible spectrum. The at least one two-dimensional image may include a monochrome image, a color image, and / or a multispectral image.

[0071] In a method according to an embodiment of the present invention, acquiring image and / or spatial data may include processing multiple two-dimensional images acquired substantially simultaneously from multiple different viewpoints by a stereo camera or a multi-camera setup to determine the depth (and / or 3D) information (e.g., using algorithms known in the art for inferring depth (and / or 3D) information based on multiple images acquired simultaneously from different perspectives).

[0072] In a method according to an embodiment of the present invention, the imaging and / or treatment process may include a magnetic resonance imaging process, a computed tomography imaging process, an imaging process using another diagnostic imaging modality (e.g., projection radiography, mammography, echography, etc.), a nuclear medicine imaging process (e.g., SPECT, PET, etc.), a radiation therapy process and / or different (usually automatic or machine-assisted) treatment processes.

[0073] In a second aspect, the invention relates to a device for assisting in positioning and / or orienting a plurality of objects including an object and / or at least one auxiliary instrument item on a subject support during an imaging and / or treatment procedure, the device comprising a camera system for acquiring images and / or spatial data of the subject support while at least one of the objects is placed on the subject support, wherein the camera system is adapted to acquire depth (and / or 3D) information. The device further comprises a processor and an output.

[0074] The processor is adapted to obtain reference data representing the object support on which the at least one object is not placed, wherein the reference data comprises reference depth (and / or 3D) information. The processor is further adapted to: detect the at least one object by taking into account at least the reference depth (and / or 3D) information in the reference data and the depth (and / or 3D) information in the acquired image and / or spatial data, for determining at least one geometric property of the detected (one or more) objects (or each thereof) placed on the object support; and for outputting the at least one geometric property via the output.

[0075] The processor is adapted to detect the at least one object by determining a three-dimensional point whose corresponding acquisition depth (and / or 3D) information is substantially different from the corresponding reference depth (and / or 3D) information. The processor is adapted to associate the point and / or the determined group of points with the at least one object (e.g., associating different groups with different detected objects).

[0076] The processor is suitable for directly calculating at least one geometric property of each of the multiple objects based on these points (and / or groups) associated with the objects, that is, by calculating one or more statistics (e.g. summary statistics, such as sample statistics) of the points and / or subsets of the points.

[0077] In a device according to an embodiment of the present invention, the processor may be adapted to repeat the steps of acquiring the image and / or spatial data via the camera system, and detecting at least one of the objects, wherein, after each iteration, the reference data is updated to include the detected one or more objects, so that the updated reference data represents the object support having the one or more objects detected up to the iteration, and so that in the next iteration, one or more additional objects can be placed on the object support to be detected.

[0078] In a device according to an embodiment of the present invention, the processor may be adapted to determine the at least one geometric attribute, wherein the at least one geometric attribute may include a discrete object support index selected from a predetermined discrete ordinal set of object support indices to identify a position of a detected object along a longitudinal direction of the object support associated with the at least one geometric attribute.

[0079] In a device according to an embodiment of the invention, the processor may be adapted to determine the at least one geometrical attribute, wherein the at least one geometrical attribute may comprise at least one value indicative of a position and / or orientation of a detected object associated therewith.

[0080] In an apparatus according to an embodiment of the invention, the processor may be adapted to determine the at least one geometrical property, wherein the at least one geometrical property may comprise a centre line of the detected object associated therewith.

[0081] In an apparatus according to an embodiment of the invention, the camera system may comprise at least one camera arranged above the object support so as to obtain a two-dimensional and / or three-dimensional top view of the object support.

[0082] In a device according to an embodiment of the invention, the camera system may comprise a depth camera (eg a device adapted to determine 3D information of an observed scene).

[0083] In an apparatus according to an embodiment of the invention, the camera system may comprise at least one optical camera for acquiring a two-dimensional image, wherein the optical camera is sensitive to one or more spectral bands in the infrared and / or human visible spectrum, for example, suitable for acquiring monochrome images, color images and / or multispectral images. Advantageously, the geometrical attribute(s) may be presented on the optical camera image in a visual manner that can be easily interpreted by an operator, for example similar to normal human vision.

[0084] In a device according to an embodiment of the present invention, the camera system may include multiple two-dimensional optical cameras to capture two-dimensional images from multiple different viewpoints essentially simultaneously, wherein the camera system is suitable for determining the depth (and / or 3D) information based on the two-dimensional images, for example using stereo or multi-camera image processing.

[0085] In an apparatus according to an embodiment of the invention, the reference data obtained by the processor may include a three-dimensional geometric model of the object support and / or at least one depth image of the object support.

[0086] The device according to an embodiment of the present invention may include an input for receiving a current position of the object support from an automatic object support actuation system, and / or the processor may be adapted to determine the current position based on the acquired image and / or spatial data. The processor may also be adapted to determine the reference depth (and / or 3D) information based on the current position and a three-dimensional geometric model of the object support defined according to its position and / or based on the current position and a set of reference depth (and / or 3D) images of the object support corresponding to a plurality of different positions.

[0087] In an apparatus according to an embodiment of the invention, the output may comprise a display monitor, wherein the processor may be adapted to present a visual representation of at least one geometric property determined for the detected one or more objects to an operator via the display monitor.

[0088] In a device according to an embodiment of the present invention, the processor may be adapted to repeatedly perform the image and / or spatial data acquisition, determine the at least one geometric attribute, and present the at least one geometric attribute, so as to dynamically guide the operator to locate and / or orient the detected one and / or multiple objects.

[0089] In a device according to an embodiment of the invention, the processor may be adapted to present the visual representation in an overlay on at least one camera image acquired by the camera system and / or on a geometric model extracted from the acquired image and / or spatial data and / or from obtained reference data.

[0090] In a device according to an embodiment of the invention, the processor may be adapted to visually present the shape, area and / or volume of the detected one or more objects. In a device according to an embodiment of the invention, (one or more) geometric attributes (e.g., center point, orientation vector, box outline (or another simple geometric 2D or 3D shape that can be fitted to the object)) and / or the principal axes of the object (or each object) may be shown in the visual representation.

[0091] In an apparatus according to an embodiment of the invention, the centerline may include and / or may be visually presented via the display monitor as a straight line, line segment or curve, or a set of points indicating such a line, wherein the line represents the centerline of the detected object.

[0092] In an apparatus according to an embodiment of the invention, the geometrical property(s) may be shown in the visual representation in 2D or 3D. For example, the centerline may be indicated in at least two (e.g. orthogonal) views, or a height (e.g. vertical) view of the centerline may be shown in addition to a plan (e.g. horizontal) view, such as a correspondingly annotated overhead camera image.

[0093] The device according to an embodiment of the present invention may comprise a data storage interface (and / or may comprise such a data storage), wherein the processor may be adapted to store the at least one geometric attribute in the data storage via the data storage interface for future reference.

[0094] In a device according to an embodiment of the present invention, the processor may be adapted to retrieve at least one of the geometric attributes stored in a previous process and, for comparison purposes, include the retrieved information (possibly after further processing of the information) in the visual representation (provided via the display).

[0095] In a third aspect, the present invention relates to a medical imaging system, such as a magnetic resonance imaging system or a computed tomography imaging system, or a (e.g. radio) therapy system, wherein the imaging and / or therapy system is suitable for performing a method according to an embodiment of the first aspect of the invention and / or comprises an apparatus according to an embodiment of the second aspect of the invention.

[0096] In a fourth aspect, the present invention relates to a workstation for a medical imaging system or a (eg radiation) therapy system, wherein the workstation is adapted to perform a method according to an embodiment of the invention and / or comprises an apparatus according to an embodiment of the invention.

[0097] In a fifth aspect, the present invention relates to a computer program product for performing the method according to the first aspect of the invention when said computer program product is run on a computer.

[0098] Independent and dependent claims describe particular and preferred features of the invention. Features of the dependent claims may be combined with features of the independent claims and with features of other dependent claims where deemed appropriate and not necessarily only as explicitly recited in a claim. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] Figure 1 A method according to an embodiment of the present invention is shown.

[0100] Figure 2 A device according to an embodiment of the invention is shown.

[0101] Figure 3 An example of using a detection of a knee positioning device and a position index thereof along a patient table direction for different positions thereof according to an embodiment of the present invention is illustrated.

[0102] Figure 4 An example of detection and visualization of a patient's body and the body's centerline along the table using two different spatial configurations of the body according to an embodiment of the present invention is illustrated.

[0103] The accompanying drawings are schematic and non-restrictive. The elements in the drawings are not necessarily shown to scale. The present invention is not necessarily limited to the specific embodiments of the invention shown in the drawings. DETAILED DESCRIPTION

[0104] Although exemplary embodiments are described below, the present invention is limited only by the appended claims, which are hereby expressly incorporated into this Detailed Description, with each claim and each claim combination as permitted by the claim-defined dependent structure forming a separate embodiment of the present invention.

[0105] The word "comprising" used in the claims is not limited to the features, elements or steps described thereafter, and does not exclude additional features, elements or steps. The word thus specifies the presence of the mentioned features, but does not exclude the presence or addition of one or more further features.

[0106] In this detailed description, various specific details are provided. Embodiments of the present invention can be implemented without these specific details. In addition, in order to make the present disclosure clear and consistent, well-known features, elements and / or steps are not necessarily described in detail.

[0107] In a first aspect, the present invention relates to a method for assisting in positioning an object and / or an auxiliary instrument in an imaging and / or treatment procedure, such as a magnetic resonance imaging or computed tomography imaging procedure (or using another imaging modality) and / or a radiotherapy procedure (or other treatment where accurate positioning is considered very important). Such assistance may include detecting the position, orientation and / or other spatial properties of the patient and / or the auxiliary instrument, and / or detecting the spatial relationship between the object and the instrument and / or between different instrument items. This information may be collected for presentation in a convenient form, for recording the information (e.g. for future reference in another procedure and / or as quality assurance or general annotation), for comparing the information with its reference (stored in an earlier procedure, such as cited above, and / or from the procedure description and / or obtained from any other suitable data source, such as a prescription, guide...), for confirming the correspondence between the current state and the relevant reference, for guiding the operator to achieve such consistency, and / or for other related functions.

[0108] In particular, the method may be a computer-implemented method, i.e., an automatic or semi-automatic method that may be performed by dedicated processing hardware (e.g., using one or more application-specific integrated circuits) and / or configured / programmed processing hardware (e.g., a computer or other general-purpose processor programmed to perform the specific tasks of the method) and / or a configurable hardware platform (e.g., a field programmable gate array) configured to perform the specific tasks of the method. A combination of dedicated hardware (e.g., ASIC) and / or configured hardware (e.g., FPGA) and / or one or more programmed processing devices (e.g., using a CPU, GPU or other suitable processor, possibly in combination with supporting hardware commonly found in, for example, a computer) is also possible.

[0109] “Automatic” and “semi-automatic” refer to computer-implemented methods implemented by (e.g., digital) processors, controllers and / or other such hardware, wherein the methods are performed in an autonomous or supervised autonomous mode, e.g., requiring only limited input from and / or limited interaction with an operator, such as selecting or entering relevant parameters and / or configuration options, to starting, stopping and / or interrupting the process, to supervising the process and / or other such limited interaction.

[0110] When referring to a "camera", "depth camera" or similar components, it should be understood that the devices mentioned are not to be confused with the diagnostic imaging system described above (nor with the therapeutic system described above). Although a "camera" may be implemented as part of an imaging system by default or with appropriate modifications, it should be understood that the main purpose of a diagnostic imaging system generally involves an image acquisition process by different means, such as by ionizing radiation, radio frequency signals, etc., and is not primarily (or at least not entirely) based on observations with the "camera" described above.

[0111] For example, although the radiation detection component (i.e., image detector) of a PET or SPECT system may be referred to as a PET or SPECT "camera", it should be understood that this is different from a "camera" in the sense of the present disclosure. For example, cameras in the sense of the present disclosure may include conventional optical imaging cameras and depth imaging cameras. The former (conventional cameras) may generally relate to imaging cameras that detect light in the (human) visible spectrum and / or infrared spectrum (i.e., in particular for determining images based on this light). Conventional cameras may, for example, be suitable for monochrome, color and / or multispectral imaging. The latter (depth cameras) generally relate to techniques for detecting depth information other than in-plane information (e.g., a two-dimensional projection plane of the image detector), such as using stereo imaging (or more generally, a camera that simultaneously images from multiple different viewpoints), range-finding cameras, LIDAR, RADAR and / or other such techniques for determining points and / or surfaces in three-dimensional space, such as to enable the surface contour of (one or more) objects (e.g., facing the camera) to be quantified in three dimensions, at least to some extent. In this context, a depth camera is generally understood to refer to any suitable means for collecting spatial data representing an observed scene in three spatial dimensions, for example enabling determination of the (3D) spatial position of points on the surface of an object (at least when the camera's viewpoint is not obstructed) in the observed scene.

[0112] The method according to an embodiment of the present invention can be used to assist in patient and / or instrument positioning in an imaging and / or treatment process (e.g., in the preparatory steps of a magnetic resonance imaging, computed tomography imaging, and / or radiotherapy process), but is not limited thereto. The method can also be applied to a system or environment in which an imaging modality is combined with a treatment modality, such as performing radiotherapy while imaging a patient (e.g., by CT, MRI, etc.). It should be understood that a magnetic resonance imaging (MRI) process refers to an examination using an MR imaging scanner, such as for diagnostic and / or medical purposes, such as for diagnosing the condition of an object, planning treatment, monitoring and / or controlling treatments performed simultaneously, and / or other related (e.g., medical) purposes. Similarly, a computed tomography imaging process refers to an examination using a computed tomography (CT) scanner.

[0113] When referring to an imaging (e.g. MRI) procedure, it should be understood that this can refer to a procedure in which a (e.g. human) subject is positioned in a (e.g. MR) imaging system to perform an imaging procedure and then one or more diagnostic images can be acquired (e.g. one or more MRI pulses and magnetic gradient sequences can be applied to acquire MRI data).

[0114] refer to Figure 1, an illustrative method 100 according to an embodiment of the present invention is shown. The method (e.g., a computer-implemented method) 100 can be used to assist in positioning an object and / or auxiliary instrument on a patient support during an imaging and / or treatment procedure (e.g., in particular, during a preparatory step of the procedure). Thus, the method is suitable for assisting in positioning and / or orienting a plurality of objects, wherein the objects may in particular include an object (e.g., a patient) to which a procedure is to be applied (e.g., an object to be subjected to diagnostic imaging and / or treatment) and / or at least one auxiliary instrument item, such as a positioning aid.

[0115] The method 100 includes using a camera system to acquire 101 images and / or spatial data of a patient support on which one or more objects are placed. The images and / or spatial data (e.g., one or more images and / or spatial information of any suitable form regarding an observed scene) specifically include depth information. Thus, acquiring 101 one or more images and / or spatial data includes acquiring 102 depth information.

[0116] The method comprises obtaining 111 reference data representing a patient support without one or more objects placed thereon, wherein the reference data comprises reference depth information. Thus, the reference data may represent a patient support without one or more objects (e.g., a patient, a positioning aid, ...) detected and characterized in the following steps.

[0117] The method further comprises determining 104 at least one geometrical property of the or each object (e.g. in the observed scene) based on the image and / or spatial data. This step of determining at least one geometrical property comprises detecting 105 one or more of the objects by taking into account at least reference depth information in the reference data and observed depth information in the acquired image and / or spatial data. Thus, the depth information and the reference depth information may be compared, subtracted, contrasted or analyzed in relation to each other by any suitable technique, e.g. such that the observed depth information is compensated on the basis of the reference depth information. Then, at least one geometrical property is determined for (e.g. each) detected object.

[0118] When one or more images and / or spatial data are acquired 101 by a camera system, the camera system may include one or more optical cameras, such as optical cameras sensitive to one or more spectral bands in the visible (and / or infrared) range, such as (one or more) monochrome cameras and / or (one or more) color cameras. (One or more) color cameras should be understood to refer to (one or more) color cameras, without being limited to a specific choice of color components (such as RGB) or a specific combination thereof. It should also be understood that a "color" camera may also (accidentally or intentionally) be sensitive to one or more infrared spectral bands (possibly exclusively), but also in combination with visible spectral bands. The camera system also need not be limited to "color" cameras with a relatively small number of color spectral bands (such as red, green, blue), but may also include (one or more) multispectral cameras, such as those suitable for more comprehensive quantization of the spectrum received at each pixel position, such as by decomposing into a relatively large number of spectral bins (e.g. so that a large number of spectral image dimensions are formed in addition to the 2D image coordinates).

[0119] Preferably, the camera system comprises or consists of a depth camera, i.e., to provide said depth information. For example, in addition to the in-plane image information, depth information may be collected, or at least three-dimensional (3D) information may be collected which cannot be simply reduced to 2D data, e.g., cannot be expressed only in coplanar coordinates. For example, a 2D map (e.g., an image) of depth and / or a collection of 3D points (e.g., a 3D point cloud) may be collected. The collection of 3D points may include (but is not limited to) at least 32 spatial points, but typically more, i.e., not limited to points on the same single plane or another simple geometric surface (e.g., a spherical or cylindrical surface, ...), or in other words, the 3D points cannot be simply reduced to 2D data. Thus, objects and / or auxiliary instruments in the observed scene may be characterized or characterizable in three-dimensional space (e.g., at least to a certain extent, with a certain level of accuracy, and / or within practical limitations) by means of the collected depth information.

[0120] Thus, one or more depth images may be acquired 102 by a depth camera or by a suitable alternative thereof. The depth information may be obtained in various possible forms, such as (3D, external) surface geometry and / or spatial features of objects in the field of view of the camera, a point cloud of 3D points (sampled points on the surface of the objects in the field of view), and / or an image (e.g. a matrix of pixel values, e.g. organized in rows and columns) in which each pixel has associated therewith at least one value (e.g. a scalar pixel value or one component of a vector value) representing the depth. For example, the depth value may indicate the distance from the camera to the sample point in space corresponding to the image coordinates in the projected space, or more generally, the distance in a direction pointing substantially out of the image plane. For example, such a depth image may be obtained by a range-finding camera, which may produce a 2D image showing the distance from a predetermined reference point or plane (e.g. a focal point or other reference point) to a point in the scene.

[0121] For example, depth information can be obtained by a stereo camera, in which two (conventional, e.g., monochrome or RGB) cameras capture images from different observation points (e.g., substantially simultaneously). Processing techniques known in the art can be applied to such stereo images (applied to pairs of images captured simultaneously) to derive depth information therefrom, e.g., by parallax analysis. This principle can be easily extended to more than two cameras, which can often also be referred to as stereo imaging (i.e., "stereo" is not interpreted in the narrowest sense as using only two cameras). Similarly, the camera configuration need not be limited to a setting that mimics human binocular vision (i.e., need not be limited to corresponding intraocular and intraaxial distances and / or inter-camera distances). For example, a depth image (e.g., a pixel map of depth values) can be obtained from a stereo camera system (generally a multi-camera system) by stereo triangulation. The advantage of stereo (or multi-camera) depth imaging is that the required equipment components are easily available and relatively cheap. For example, such a system can be constructed using conventional (2D) cameras, or existing (2D) cameras can be easily upgraded by adding one or more additional cameras. It should also be noted that even a single camera equipped with a mirror and / or lens assembly may be sufficient to obtain data for the same point(s) in space from observation points that are sufficiently spaced apart and / or from sufficiently different observation angles, for example by dividing the image detector into multiple areas that receive light via different optical paths, and / or using a switchable mirror assembly to quickly switch between different optical paths for continuous image acquisition.

[0122] The processing required to calculate the corresponding depth pixel map (i.e., depth image) is relatively simple and can be easily implemented in software and / or hardware (possibly using image processing unit GPU processing and / or dedicated hardware, such as application-specific integrated circuits and / or field programmable gate arrays). The advantage of stereo camera or multi-camera depth imaging is that the scene (e.g., patients and / or instruments of interest in the scene) can be imaged without the need for dedicated lighting and / or other direct manipulation. In other words, the scene can be imaged and processed to determine depth data (i.e., a third dimension in addition to the planar image coordinates) without actively interfering with or affecting the scene, i.e., essentially passively, except for possible general (universal) lighting.

[0123] Depth images can also be acquired by active techniques. For example, a light sheet can be scanned over a scene to image its reflections. Based on the shape and displacement of the imaged (line) reflections, the distance between the reflections (points on the line, e.g. each point in the image after a complete scan of the scene) and a reference (e.g. a light source and / or camera) can be relatively easily calculated, e.g. using triangulation. Another type of active depth imaging that can be applied uses a structured light 3D scanner, which can be viewed as a more complex version of the light sheet triangulation mentioned above, with additional advantages, such as requiring fewer (or no) physical displacements of the camera and / or light source (one or more) to fully quantify the depth value of the scene.

[0124] Depth information may be collected by a time-of-flight camera, LIDAR or RADAR system. A possible advantage of depth imaging with a time-of-flight camera is that images may be collected substantially instantaneously, e.g. without the need to scan points, lines, structured light patterns, lasers, waves or generally any "active" (e.g. time-varying, e.g. scanning) analyzer over the scene. Other potential principles for depth imaging may include interferometric techniques (e.g. relying on coherent light), coded aperture imaging and / or possibly other techniques.

[0125] Obviously, diagnostic (e.g. medical) imaging and / or therapeutic procedures may place restrictions on the technologies that are or can be used, or may create preferences for technologies with certain properties. For example, "active" scanning techniques may be preferably avoided, or restricted to non-visible forms of light, especially when high-intensity light is required, when the type of wave may damage the instrument or even the human body (e.g. lasers, for example, are particularly damaging to the eyes), and / or when scanning (or other dynamic) modes are required, which may confuse, disorient and / or induce fear or unpleasantness in the patient, may induce epileptic seizures in the patient or staff, and / or may limit the performance of the staff. Therefore, essentially passive technologies such as stereo (or multi-) camera imaging may be preferred. However, the mentioned technologies (including "active" 3D imaging) are not excluded.

[0126] In an embodiment of the present invention, acquisition 101 of an image may combine acquisition 102 of depth information (e.g. comprising a pixel map, whose values ​​represent depth, a 3D point cloud and / or a parametric description of a surface and / or in any other suitable form) with acquisition 103 of an optical 2D image of an object and / or an auxiliary instrument, the optical 2D image being, for example, a photograph of the human visible spectrum, the infrared spectrum, (one or more) sub-bands of any of the foregoing, and / or any combination of the foregoing. Preferably, the optical 2D image is (co-)registered to the depth information, e.g. such that the depth information for the same image coordinates corresponds to the optical image pixel value, or such that registration information associating the image content in the depth image with the image content in the optical image is determined by a suitable image registration method (as known in the art) and / or is predetermined in another way and available (e.g. to a processor implementing the method), e.g. by a calibration procedure associating different camera views with parameters to each other.

[0127] It is noted that the camera system may be adapted to simultaneously (or concomitantly) acquire 102 depth information and to acquire 103 optical image information. Thus, a depth image may be acquired, as well as, preferably substantially simultaneously, more conventional monochrome, color (e.g. RGB) or multispectral images in the visible and / or infrared spectral domain. For example, a multi-camera setup (e.g. a stereo camera system) may simultaneously acquire depth and optical (e.g. visual) image information (e.g. by capturing photographic image information and supplementing the image information with depth information obtained from processing said image information). Thus, the spatial relationship between the two types of image information is known per se, e.g. photographic information and depth information are implicitly co-registered (or the relationship and / or the corresponding transformation is determined in a straightforward manner). Other depth camera systems may also be adapted to simultaneously collect photographic information when alternatively used to acquire depth information, or (if not) may be combined with further cameras, so that photographic information and depth information can be combined in a method according to an embodiment.

[0128] For example, the camera system (or part thereof) may be positioned generally above the subject support (e.g., above the imaging and / or treatment couch, patient table, ...) to observe from above the area where interaction with the patient primarily occurs during an imaging / treatment procedure, e.g., to observe the subject and / or any ancillary equipment in the bore of an MRI scanner, on a CT patient table, on a radiotherapy couch, and / or other such environments. It is noted that for certain imaging and / or treatment environments, patient (and / or ancillary equipment) setup (i.e., in the preparation phase of the procedure) may occur near the interaction volume (of the imaging / treatment system), after which a simple, predictable, and well-defined (e.g., actuated) movement may position the patient support together with the patient and / or any ancillary equipment within the target interaction volume, e.g., by translating the patient support without (ideally) any substantial relative movement between the patient support, the patient, and / or any (one or more) equipment items set up for the procedure. In this case, it will be appreciated that the camera system may generally be positioned above the patient support, where it is in its (usually accessible) configuration for preparing the patient and the procedure (e.g., with operator assistance). The camera system may also be positioned above the patient support in a configuration intended for performing an imaging / treatment procedure, such as when the system is not configured to perform any substantial automatic translation between the preparation and execution phases of a procedure (e.g., as may be common in the case of a radiation therapy environment). It should also be understood that the camera system may include various components, such as multiple cameras and / or sensors, and / or may be configured for a relatively wide viewing angle, such that both the preparation and execution positions of the patient table (if substantially different at all) may be adequately observed and analyzed using the same static camera setup.

[0129] Thus, at least one camera comprised by the camera system (which generally refers to any device for collecting depth information and / or optical image information) can be positioned above the patient support and oriented toward the patient support so as to be able to collect depth (and optionally also optical) information in the field of view sufficient to observe the patient and / or instruments to be used near the patient from a top-down perspective. Alternative positions and / or orientations of the camera axis(es) are not necessarily excluded. Furthermore, it should be understood that the terms "above" and "looking down" do not (necessarily) require an exact vertical position or alignment of the camera(s) above the patient, nor do they (necessarily) require the camera axis(es) to point (precisely) vertically downward. For example, in an MRI environment, it may be advantageous to view the patient table at a relatively small angle (not limited to this) and / or from a point (camera position) relatively far from the patient's center of mass (e.g. a distance on a horizontal projection) (e.g. 1 m or more). Thus, interference between the camera system and the MRI system can be avoided or at least reduced, for example by positioning the camera(s) near or at the magnet bore flange, or even further away, such as on a wall of the scanner room or even outside the scanner room (e.g. through a viewing window observing the patient table).

[0130] The method 100 includes determining 104 at least one geometric property of a patient (and / or one or more body parts of the patient) and / or an auxiliary instrument based on a camera image and / or spatial information acquired via a camera system. For example, the at least one geometric property may be calculated by processing a segmentation, a set of identified pixels corresponding to an object of interest, and / or another suitable representation of the (each) detected object.

[0131] The detection of individual objects (which may include patients) and the processing of each detected shape or volume to determine its representative geometric property(s) has significant advantages when applied in the patient and / or equipment preparation phase of an imaging / therapy procedure. By detecting the different objects in a scene and recording the spatial configuration of each object and / or the spatial relationships between objects in a more representative and / or reproducible form, accurate reproduction of the same setup and / or accurate recording of the setup can be achieved for future reference.

[0132] One or more geometric properties determined for one or more objects (e.g., a patient, (one or more) body parts of a patient, and / or one or more instrument items (e.g., positioning aids)) may be implicitly obtained by a step of detecting 105 (one or more) objects (discussed further below). For example, a detection algorithm may be applied which implicitly determines characteristics of (one or more) different objects, such as centroids or other such metrics indicating positions (and / or orientations, and / or other geometric properties) thereof. However, it is also possible to determine, for example (independently) the geometric property(s) for each object based on the results of said detection 106, for example by explicit calculation thereof, for example based on spatial clustering and / or segmentation outputs.

[0133] The detection process is preferably agnostic, for example, preferably not taking into account the specific geometric model of the / each (type) object, but rather detecting objects by their isolation (in space and / or time, for example, there is a time gap between the first appearances of different objects in the scene), separation, distance relative to each other, (zero or finite) interface surface, volume or contour, and / or other such general properties (i.e., properties that are generally applicable to any object of interest). However, embodiments are not necessarily limited to this. For example, a patient or (one or more) body parts of a patient may be detected in a manner different from the / any instrument item of interest. Therefore, optionally, a step of determining whether the detected object corresponds to a living object (e.g., a patient) or to an inanimate object (e.g., a positioning device) may be included, or such a distinction may be implicitly included in the detection method. Some examples are provided below, but are not limited thereto, such as distinguishing patients from inanimate objects based on motion and / or body heat (infrared radiation) and / or simply by substantial differences in volume, surface, contour length, and / or generally physical size. This allows different (one or more) geometric properties to be determined and / or presented and / or stored for patients rather than for, for example, positioning aids.

[0134] It may be particularly advantageous to detect different instrument items in a substantially identical or similar manner, which allows the method to be applied to a wide range of such potential objects, and / or without requiring a priori object models (or other object class descriptors), and / or without modifying each such instrument item (e.g., by including physical identification marks, tags, and / or other means on / in the object to facilitate automatic detection). The (one or more) geometric attributes are preferably clearly and / or robustly defined for any instrument item (e.g., automatically computable), and for example may preferably involve properties that can be detected in a reproducible manner based on repeated application of the method, such as based on segmentation as discussed below in different imaging and / or treatment processes and / or in different environments (e.g., scanners or treatment systems / rooms). However, the geometric attributes associated with a patient may be different (or may be the same) from those associated with an instrument item. The (one or more) attributes determined for each detected instrument item may preferably be the same (e.g., of the same nature, and / or determined by the same algorithm / process). By treating all objects of interest in a scene in a substantially equivalent manner, difficulties in identifying different types of objects may be advantageously avoided or reduced, which may result in problems with incorrect identification and / or the need for complex modeling or means of specific identification of different objects. Alternatively, some distinctions may be made in treating different types of objects, such as based on broadly defined and / or robustly detectable characteristics, and / or simple and robust characteristics may be used to identify detected devices from an easily maintainable library of devices (or instructions for devices for a particular process). For example, objects (or types of objects) may be identified by their volume or projected area, such that the method may be easily configured to identify a particular device (type) by simple and rapid calibration or input of one (or several) identifying properties.

[0135] By determining the geometric properties of objects, the spatial relationships between these objects can be clearly determined, for example to facilitate the positioning setup process (e.g., when providing live updates of the properties as feedback to the operator during setup) and / or to support quality monitoring and documentation processes. Geometric properties can generally include any value that indicates the spatial configuration of an object (e.g., position, orientation, deformation, shape, etc.).

[0136] The (one or more) geometric properties may include simple summary statistics and / or other representative values, such as the center of mass (or at least the volume and / or area or surface observable in the camera's field of view), a measure of orientation and / or a measure of mass, volume and / or area.

[0137] For example, at least one geometric attribute may include a position in one, two or three directions (e.g., along non-collinear and / or orthogonal coordinate axes). For example, the position may correspond to the center of the patient and / or instrument item, such as the geometric center, centroid, centroid, or generally any point that can be used to represent the position of the object as a whole (patient, patient's body part of interest, instrument item...), preferably in an unambiguous and reproducible manner. For example, the position (and / or orientation) of a patient (i.e., generally a human or animal) and / or one or more devices (instrument items) used in the preparation workflow can be calculated.

[0138] The physical position of an object need not be completely quantified by the position contained in at least one geometric attribute. For example, the position may be defined along a single axis, such as the longitudinal axis of a patient table or an axis of a diagnostic imaging system or treatment system; the position may be defined relative to a plane, such as the (e.g., horizontal) plane of a patient table or a reference plane of an imaging / treatment system; or the position may be completely quantified, such as by three coordinates in a (e.g., Cartesian) coordinate system. The position of the (or each) device may be constrained to discrete values ​​from a set of known patient support indices, such as those commonly used in radiation therapy (without limitation), such as discrete indices taken from an ordinal set F6, ..., F1, 0, H1, H2, H3 (e.g., as discussed in the background section above). It should be understood that embodiments may equivalently use integer values ​​to represent such discrete indices, or may use different scales (and / or indices). It should also be understood that real-valued coordinates may be used instead of discrete indices, or suitable approximations thereof (fixed-point precision, floating-point values, ...). For example, the discrete position information (e.g., longitudinal position index) may also be extended by additional coordinates (which may be discrete or non-discrete) (e.g., in the lateral and / or vertical directions). The discrete (e.g., longitudinal position) index may also be supplemented with more precisely defined information (e.g., real-valued position) in the same direction.

[0139] For example, a location may be indicated (e.g., for display and / or storage purposes) by an annotation such as "pos: F2, x: -12.1, y: -25.3, z: 4.2" to facilitate access to approximate locations as well as more precisely defined coordinates. This example is not limiting; arbitrary units are used for illustration purposes; the illustrated mark F2 does not necessarily correspond to any real-world illustrated coordinates. Redundant information may be used as desired, such as by human and / or machine users. For example, a human user may rely on an index mark for quick reference, while a processing algorithm may use the detailed location coordinates for post-processing of the recorded data.

[0140] Likewise, at least one geometric property may include an orientation, for example (without limitation), expressed as one, two or three angles, for example, relative to suitable coordinate axes or reference planes. The orientation may be fully quantized, i.e., in the form of an orientation vector uniquely identifying the orientation in 3D space, or relative to only one axis or only one plane.

[0141] (One or more) geometric properties can be determined by fitting a 2D or 3D shape to (e.g., each) detected object. For example, a rectangle, ellipse, rectangular cube (or prism), ellipsoid, and / or another simple geometric 2D or 3D shape can be fitted to a detected surface / volume / pixel to generally represent an object detected in a scene, such as an object placed on a patient table. The length and width of the fitted rectangle, the major and minor axes of the ellipse, and / or other such parameters of the fitted (2D or 3D) shape can be used as geometric properties representing the orientation and / or scale of the object. Instead of fitting the shape (e.g., by optimization), similar parameters, such as maximum, minimum, or average width (and / or length and / or height) can be determined along one or more predetermined axes (e.g., along the longitudinal axis of the object support (and / or perpendicular to its lateral and / or vertical direction)). This requires less calculation than applying shape fitting, and may have sufficient information, be sufficiently accurate, and / or reproducible for at least some application scenarios (e.g., when (relatively large) changes in orientation are not expected).

[0142] Other examples of geometric properties include an overall estimated volume, an overall estimated surface area (e.g., in a camera projection view or taking into account surface angles), and / or a perimeter (e.g., of a contour in a camera projection view). Geometric properties may also include a representation of the contour in some suitable form, such as a lasso representation.

[0143] Geometric properties may include, among other things, a representation of a centerline of a detected object, as discussed in further detail below.

[0144] The method 100 comprises obtaining 111 reference data representing a patient support on which no (one or more) objects are placed (which were present when the data was acquired 101). The reference data comprises reference depth information. Furthermore, determining 104 the (one or more) geometrical properties from the image and / or spatial data comprises detecting 105 the object or objects. The detection uses at least the reference depth information in the reference data and the depth information in the acquired image and / or spatial data, but other information may also be considered, such as optical (e.g. 2D) image information, such as (one or more) infrared, color and / or monochrome images acquired simultaneously with the depth information.

[0145] One of the advantages of using depth data is that any object with non-negligible volume can be easily detected, regardless of its specific shape or specific object (or object type) properties. However, paper on an object support is generally not detected (i.e., does not cause a significant change in depth along the corresponding camera depth trajectory path) due to its thinness. For example, an operator may often use handwritten or printed notes to keep track of the (one or more) procedures being performed. Such notes may easily appear on the patient table (at least temporarily during the preparation phase), but will not easily interfere with the detection process 105 when considering depth information. The sensitivity of such depth-based detection 105 may also be adjustable, configurable or tunable, so that a lower threshold value (e.g., for depth difference or absolute depth difference, or any suitable equivalent value, such as a suitable metric / standard distance) can be defined in order to automatically reject other potentially nuisance or interfering (thin) objects, such as smartphones or tablet devices. Such a threshold value (explicit or through an equivalent adjustable algorithm parameter) can also effectively avoid false detections caused by small deformations of the patient table (e.g., when the patient is placed on a padded object support). It will be appreciated that an upper threshold may also be applied, for example to ignore atypical large / tall objects, such as the (eg temporary) presence of an operator in the scene.

[0146] The detection 105 of the object(s) of interest may implicitly or explicitly use background subtraction or compensation, or at least take into account reference depth information to detect the object(s) of interest. For example, a reference image or other reference data (including in particular depth information) representing an object support (patient table) without the object(s) to be detected (e.g. in the same imaging / therapy system and / or environment in which the method is intended to be used) may be used to detect the object(s) of interest. Thus, the reference data (e.g. reference image(s)) may include a representation of an object support (or more generally: in a represented reference scene) without a patient nor a positioning aid or other device of interest (i.e. to be detected by the method) on it, but the reference data may also represent an intermediate state in which a certain object(s) is already contained in the reference data, but at least one further object is to be detected and characterized in the next iteration, as discussed in more detail below.

[0147] The object detection process may be as simple as (or may include) subtracting reference data (e.g., a reference depth image or reference depth data, e.g., organized per camera viewing ray) from the image / spatial data currently acquired 101, and clustering the non-zero results (possibly after considering a predetermined threshold) to detect objects (e.g., properties of objects, volumes, pixels, and / or other descriptive spatial references) currently present in the scene but not in the reference scene, and / or to detect multiple individual entities (objects). The differences may be treated as absolute values ​​(or equivalent values, such as squared differences, ...), or alternatively may be considered only if the sign of the difference indicates an object (close to the camera) above the reference (e.g., empty) patient support.

[0148] Thus, detecting 105 the object(s) may include determining 110 a point at which corresponding acquired depth information is substantially different from corresponding reference depth information, i.e., a point in the observed space that does not fall on (or below) the surface(s) represented by the reference data (e.g., does not coincide with the object support).

[0149] The reference information obtained 111 may characterize the patient support in the form of one or more (e.g. depth) images of the table without the patient nor the instrument (still) to be detected. Alternatively (or additionally), the reference information may comprise a geometric model of the patient support calculated in the camera coordinate system, or another type of model to relate the camera coordinates and / or the image content received by the camera system (which may contain depth information) to the object support (patient table), e.g. in the absence of the patient or (another) instrument item of interest. Thus, based on the (image and / or model) reference information, the depth data and possibly further camera imaging information (e.g. monochrome or color digital photographs) may be used to detect any object placed on the patient support, e.g. one or more positioning devices (or other related objects) and / or a patient, e.g. by comparing the depth data of a point with a reference and calculating whether the point is above a corresponding point in the reference configuration. Thus, any arbitrary mass distribution placed on top of the reference configuration may be detected and its geometric properties may be calculated without requiring specific assumptions about the object itself (e.g. different geometric models for specific types of objects that may be encountered).

[0150] It will be appreciated that, according to some embodiments, the segmentation and / or clustering used in detection 105 may receive such reference compensation data as input, may provide reference information as an additional input to the camera observation data (e.g., as an additional input to a trained machine learning model), or may be applied to the camera observation data itself. In the latter case, the results of the detection (e.g., segmentation) may be compensated for the reference information, e.g., by rejecting segmented fragments corresponding to corresponding elements identified in a reference (e.g., a workbench and / or a static or unrelated instrument (for tracking purposes)). It will also be appreciated that iterative processes are not necessarily excluded, e.g., alternating segmentation (or other types of object detection steps) with background content removal steps.

[0151] Obtaining 111 reference data may include receiving a current position of the subject support (e.g., height, longitudinal and / or lateral position, such as three coordinates of an automatic table translation system), and calculating or updating a (e.g., parameterized) three-dimensional geometric model of the subject support. The current position (which may optionally also include, if relevant, an orientation or other geometric parameters (e.g., one or more azimuth angles)) may be provided by a control system of the patient support. Thus, this current position information may be used as input to an image and / or data processing algorithm. Alternatively, the position (and generally the spatial configuration of the subject support) may be calculated directly from the acquired image and / or spatial data, such as when direct input of position information is not available or readily available. This position information may be used in conjunction with other reference information, such as a geometric model of the patient support itself (which may use the position or spatial configuration information as a parameter) or reference image information, such as a reference image (2D and / or 3D) of the patient support indexed by a particular position (or generally the spatial configuration) or defined in another way as a function of a particular position. The reference information may comprise a geometrical model of the patient support, preferably a geometrical model defined in a camera coordinate system, in which case the current position of the patient support may be used to evaluate the model for a specific patient and the immediate procedure. The reference information may comprise a set of images used as a reference for a plurality of different positions, for example including a depth image. The set of images may be interpolated to the positions received as input if necessary, for example by interpolation of a nearest neighbor method, or a more detailed interpolation may be used (e.g. linear interpolation, non-linear interpolation, ...). It will be appreciated that interpolation may not be required if the set is sufficiently comprehensive and / or the positions of the worktable in use of the method are always limited to well-defined discrete positions (e.g. steps of a stepping or indexing motor system).

[0152] Detecting 105 the object(s) may comprise performing image segmentation and / or more generally spatial segmentation and / or clustering (e.g. on image data and / or data in the form of a 3D point cloud and / or in another form of representation of 3D data). The segmentation and / or clustering may preferably use a naive or agnostic algorithm, i.e. not relying on a priori knowledge about the specific object(s) to be detected, but taking into account some general assumptions that are reasonably applicable to substantially any possible object of interest (e.g. to all items of instrumentation that may reasonably be included in the observed scene, such as positioning devices). As already indicated, the detection may be relatively simple, i.e. when reference information is taken into account, a single object may be easily detected in a depth image, e.g. by background compensation, subtraction, etc., with high sensitivity and a relatively low risk of false detection. Suitable segmentation and / or clustering techniques may be used to detect multiple objects.

[0153] The method may be used in a sequential manner, e.g., to advantageously reduce complexity, increase robustness, and / or improve accuracy. Specifically, in each step, new objects may be added to the scene (e.g., by an operator), and corresponding changes may be detected relative to the previously built scene (in the previous step). Detecting 105 an object is thus relatively simple: objects that would be adjacent, overlapping, or indistinguishable when presented simultaneously may be readily distinguished by focusing on each incremental update in the built scene, wherein the previous object or objects remain substantially stationary, and (preferably) a single new object is introduced and moved in the scene until correctly positioned.

[0154] It should be understood that, for example, the next step can be triggered by an operator action, such as pressing a button, voice command or other suitable user interface interaction; or detected by processing, for example, a sudden substantial change in the overall scene content, while the tracked object remaining substantially stationary can be used as a signal indicating that a new object has been added, and the newly acquired scene (image data, 3D data, models extracted therefrom, or similar representations of images and / or spatial data acquired at a previous near point) can be used as a new reference when tracking the new object. As another example, when the detected (one or more) objects remain stationary for a predetermined time, it can be assumed that the operator intends to switch to a new step in which another object (such as a patient or another positioning aid) will be positioned. Therefore, this can also be used as a trigger to store the current scene content for reference and start a new iteration block for detecting another object or objects.

[0155] In other words, in a method 100 according to an embodiment of the present invention, the steps of acquiring 101 images and / or spatial data and detecting 105 at least one object based on the data may be repeated 114. After each iteration of the repetition 114, for example after determining 104 the geometrical properties of the newly detected object(s), the reference data may be updated 113 to include the detected object(s) (in that iteration) so that the updated reference data represents the patient support with the object(s) detected as of the completed iteration. Thus, one or more additional objects may be detected in the next iteration. Although this sequential approach may be particularly robust, simple, accurate and / or efficient, embodiments are not limited thereto. However, in many cases, the operator's workflow may be organized to add different objects to the scene in a sequential, step-by-step manner. For example, a knee support may be positioned at its intended location on a table, after which a patient may be placed on the table (and the knee support), while the operator ensures that the patient's position, orientation, and general spatial configuration meet any applicable requirements. Other paradigms can be easily found where the order in which objects are placed in a scene is determined by necessity, by trial and error, or by efficiency factors.

[0156] However, various methods can be applied to detect and distinguish several objects presented at the same time, so that sequential methods can also be avoided. For example, the method according to an embodiment can be adapted to allow detection and isolation of multiple objects relative to each other, even if none of these objects are represented in the background data. In other words, the method can be applied sequentially to detect a new object in each step, adding this information to the reference data at the end of each step, but can also be adapted to detect multiple objects relative to the reference data (simultaneously) in a single (or fewer) steps. Therefore, the method for segmenting and / or clustering data and thereby identifying multiple objects discussed later (for example, wherein each object is identified separately) can be used as an alternative to the sequential method discussed above, or can be used in combination with such a sequential method.

[0157] Note that the quality of segmentation (particularly agnostic segmentation (i.e., not based on an explicit object model)) can be improved by obtaining additional data for each pixel. For example, segmentation of a color image may be easier (e.g., more efficient to perform) and / or of higher quality (e.g., more accurate) than segmentation of a monochrome image. Similarly, depth information (e.g., expressed as a depth value for each pixel, but not limited to this form of representation) and possibly combined with other scalar values ​​for each pixel (e.g., depth or Z-values ​​combined with image intensity, different colors, spectral components, and / or other such properties for each pixel) can allow for fast, efficient, and accurate segmentation. Since depth information is available, this information can be advantageously included in segmentation and / or clustering methods to improve the ease of detecting different objects in space.

[0158] Isolated or minimally connected (e.g. relative to a predetermined threshold) objects can be detected by clustering 3D point data, image data (e.g. segmentation), or a combination of the two. Image segmentation and / or clustering algorithms suitable for use in methods according to embodiments are well known in the art. Segmentation and / or clustering can use "classical" computer vision algorithms and / or methods based on artificial intelligence, machine learning, deep learning, etc. Examples of classical (declarative) algorithms include simple (e.g. pixel value) thresholding, histogram-based methods, clustering (e.g. K-means and many others), compression-based methods, edge-based methods (e.g. edge detection), region-based methods (e.g. region growing, watershed transform), differential equations and / or variable-based algorithms, graph partitioning (e.g. Markov fields, posterior partitioning, expectation maximization ...) and / or multi-scale methods. This list of examples is not exhaustive, but combinations of such methods may also be considered.

[0159] Note also that the combination of depth information with more conventional 2D image information can provide the advantage of easily and accurately detecting isolated or only lightly touching objects (based on depth) while also allowing different objects that are touching or overlapping to be distinguished by their appearance (e.g., their color, intensity, and / or optical properties).

[0160] In this context, it is also noted that including (e.g., in addition to depth information and / or visible spectrum image information) infrared image information is particularly useful for distinguishing inanimate objects (e.g., instrument items, such as positioning tools) from living objects (e.g., patients and / or their body parts). Therefore, particularly advantageous embodiments may use depth information combined with infrared information, optionally also combined with image information in the visible spectrum.

[0161] Image and / or spatial data (e.g., depth data and optionally additional image data) may also be acquired as a live stream or generally in a dynamic manner. Since information about the observed scene may therefore be dynamic, e.g., updated at a predetermined (non-zero; but not necessarily constant) refresh rate, motion-based segmentation and / or clustering techniques may also be used, e.g., independently or in combination with any other segmentation or clustering techniques. For example, this may be particularly useful for detecting background (e.g., patient tables, scanner / treatment rooms, and / or other unrelated static scene content). Thus, a dynamic stream of image / spatial data may be acquired 101 and analyzed to determine reference data as discussed above, e.g., to determine reference depth information (and / or other reference image information) based on acquired data corresponding to an earlier point in time, e.g., when an operator provides (or automatically infers from the data) a "save state" trigger to indicate that one or more new objects are to be added to the scene. For completeness, note that a cache of data may be maintained such that such a trigger may be generated at some time after a point in time representing a reference state, such that when a new object is detected in the scene, a suitable previous point in time may be determined (e.g., retrieved from a cache) for which data represents a suitable reference state without the new object.

[0162] Even though the purpose of the process using the method according to the embodiment may be to require the patient to lie still, it will be appreciated that some residual motion (e.g., breathing and / or other involuntary movements) is generally unavoidable. This can be exploited to improve the detection of the patient or his / her body parts themselves. Similarly, although it may be intended that the positioning device or other device remain stationary during the process to be performed, it will be appreciated that the method can be applied during process preparation, wherein the operator adjusts the position, orientation, and / or other spatial properties of such equipment in the scene. Thus, motion analysis can be used to detect objects whose position or spatial configuration is to be monitored, because the image component that has a substantial change in the scene is the instrument item of interest, or the patient, or (optionally) the operator manipulating the scene. The patient's motion can be distinguished from the motion of such instrument items by substantial differences in their dynamics.

[0163] If the operator is explicitly considered (e.g., explicitly ignored in the processing of the scene), the dynamics of the operator's movements can also be easily distinguished from instrument objects and patients. For example, objects that move outside the camera's field of view, or at least objects that show a tendency to move dynamically between the periphery of the scene and the center of the observed scene (or between the edge and the position of the detected object and / or patient in the scene), can be identified as people and excluded from the detection process. By optionally applying motion tracking, such as based on optical flow or other such techniques, once the operator is identified, it can be tracked throughout the process of setting up the patient, for example if detected in the analysis of previous images obtained in the same process (i.e., images / spatial information previously acquired 101). Infrared data as discussed above can also be used in combination with motion analysis to further improve detection.

[0164] Depth data can be processed by a general segmentation algorithm, for example, it can be treated as an image of only values, without considering the meaning of the value as depth. For example, the image can be represented and processed as a vector-valued image, where one vector component can therefore represent depth or third dimension data. However, depth information can also be considered more explicitly, for example, it does not have to be processed in the same way as other image components (such as color components) (if taken into account). Depth data represented as a 3D point cloud can be processed by point clustering or another 3D technique to detect objects, such as dividing 3D shape or surface data into several minimally connected parts. For example, joint optimization of cost function components and / or constraints can be used to identify objects in 3D data. Illustrative cost function components may include a measure of the correspondence between the segmented model and the observed 3D data (such as the mean square error or other suitable measure, as is generally known for such optimization problems), the number of divided segments (such as biased towards fewer objects to reduce potential overfitting) and / or length, surface or other such measures indicating the intersection of different objects (or geometric interfaces between different objects), and / or a measure of the distance between objects, without limitation. Other 3D clustering, segmentation and / or similar processing techniques may be applied, as is generally known in the art, including the application of artificial intelligence techniques.

[0165] It should be noted that in particularly simple embodiments, even segmentation or similar detection of individual objects of interest may be avoided or may be implemented in a very basic manner. For example, merely comparing the depth data (and optionally the conventional image data) with reference depth data may be sufficient to identify individual objects, which correspond to differences in the observed scene relative to the reference scene.

[0166] The method may comprise repeating 106 the data acquisition 101 and the characterization of the scene and / or the objects therein by determining 104 at least one geometrical attribute and presenting the updated information to the operator (e.g. in a graphical representation of the scene showing the at least one geometrical attribute). For example, the determined at least one geometrical attribute may be presented to the operator in a visual form, for example in an overlay on at least one (e.g. most recent) camera image acquired 101. Examples include presentation of detected objects determined, for example, by segmentation, in an overlay on the (visible spectrum) camera image (e.g. in a transparent color overlay or as contour lines), and / or a graphical representation of the geometrical attribute(s), for example an indication of a center point, a center line, a fitted shape (e.g. a fitted box, a circle, an ellipse, etc.), etc., at a representative position in the image. The representation may comprise a numerical value of the attribute or another suitable form of display element. For example, the distance between the detected position and a reference position (from the instructions or an earlier stored procedure) may be shown in numerical form or as a bar, for example similar to a progress indicator. It will be appreciated that many alternative forms of presentation may also be envisaged.

[0167] Thus, the method may comprise the step of presenting 107 at least one geometric property to an operator, e.g. via a display monitor or other suitable human-computer interaction device. Preferably, such feedback to the user may be dynamic, e.g. by repeating 106 the detection and property calculations mentioned above. For example, a center point, a center line, one or more axes of orientation, a bounding box and / or other such visualization of the object(s) (whose geometric properties are determined) may be shown on the image (or geometric model) of the scene.

[0168] The method may also include storing 108 at least one attribute, for example in a data storage device, for future reference. For example, this allows the positioning of the patient and related instruments to be reviewed at a later time, for example for quality assurance and / or workflow archiving. This also allows the stored 108 information to be retrieved in future procedures, especially when the same (or similar) positioning of the patient and / or instrument is desired. For example, such information can be stored after the stage of guiding the positioning and / or orientation (generally spatial configuration) of the object has been completed, so that the information can be used for future reference, or the stored information can be updated throughout the process. The information can be stored, for example (one or more) DICOM headers and / or PACS systems as (one or more) annotations and / or metadata in (e.g. diagnostic) image data acquired by the imaging and / or treatment process (for example) (without limitation).

[0169] The method may further comprise retrieving 109 at least one attribute stored 108 in a different (e.g., previous) procedure for comparison purposes, if such previously stored information is available. The different procedures do not necessarily relate to the same type of imaging and / or treatment procedures. For example, without limitation, information regarding the positioning and spatial configuration of the scene may typically be recorded in the imaging procedure and used to replicate the scene (with sufficient and / or maximum achievable fidelity) in one or more treatment procedures (e.g., a radiation therapy procedure) and / or in an imaging procedure using a different imaging modality.

[0170] The presentation 107 (e.g., for guiding an operator to configure a scene) may take into account such retrieved 109 data by, for example, simultaneously displaying a current scene configuration and a previously recorded scene configuration; by showing one or more metrics representing the difference between the scene configurations, and / or by another guidance method to assist the operator in comparing previous and current conditions to assess deviations and / or reproduce a previous configuration. Procedure and / or patient information not explicitly stored in different procedures may also be retrieved. For example, procedure guidance(s), procedure planning, prescription, or similar specifications for an intended imaging and / or treatment procedure may define the type, quantity, and / or location of patient support equipment (and / or other related auxiliary instrument items) used in the current procedure (possibly accompanied by information about the intended patient posture, position, orientation, etc.). For example, such procedure information may include locations to be used to position equipment in patient preparation guided by the method.

[0171] To illustrate embodiments of the present invention, a possible workflow for preparation of an imaging examination or treatment procedure is described below. For specific procedures, such as where a patient support is in a known position, the method may include obtaining relevant reference information. In other words, an empty state of the patient support (e.g., an "empty" state that would be observed in images and / or other spatial data acquired by a camera and / or generally from one or more camera viewpoints) is determined. A geometric model and / or a reference camera image may be used to determine a representation of the (empty) patient support in its current position, wherein optionally, the received current position of the table support may be used. Alternatively, the camera system may be triggered to acquire this reference information on site, for example before placing a patient and / or related instruments.

[0172] The operator can position the patient and / or any required instrument items on the workbench. Optionally, the method can present information to the operator via a display or other interface device, for example based on the retrieved procedure and / or patient information, to assist the operator in selecting the positioning device(s). The presented information can also include information stored by previous use of the method, for example, positioning information of the patient and / or any (one or more) devices as recorded in a previous procedure for the patient.

[0173] For each instrument item placed by the operator on the patient support, the camera system can acquire image data, such as a depth image (or generally 3D data) and / or other (optical) image(s), e.g. in the visible and / or infrared range.

[0174] The method may then determine which pixels and / or spatial data in the acquired data correspond to points in space that are above the patient support, which may be done by using the aforementioned reference data representing the empty state of the patient support. Optionally, this may also take into account additional image data (e.g. in the visible light range), e.g. to improve the accuracy of the identified pixels and / or data points that are not (only) attributable to the reference state. For selected data, e.g. pixels determined in the depth image to be elevated relative to the table, a set of corresponding 3D coordinates (e.g. a point cloud), or an equivalent spatial representation, may be determined.

[0175] A clustering algorithm may then be applied to the determined points in the point cloud (or an equivalent process when the points and / or regions identified in space are in a different spatial representation), thereby computing one or more (e.g., point) clusters. Each cluster may be represented in a visualization provided to an operator, such as a camera image (or a specific image component thereof; or a composite image computed from the camera image data). For example, each point cluster may be marked on the camera observation image by a color overlay (e.g., a different color for each point cluster).

[0176] Geometric properties (e.g., centroid and / or principal orientation axis(es)) may be calculated for each point cluster. A class identification label (ID) may also be assigned to each cluster, allowing different objects to be distinguished (e.g., without confusion or interchange) throughout the process and / or in subsequent use of the information. This may be based on spatial properties, such as properties that are preferably substantially invariant, such as universal shape and / or volume descriptors, or simply numbers assigned by order of detection, or any other suitable strategy.

[0177] The geometric attribute(s) may also include a patient support index calculated based on the detected position (e.g., cluster centroid), such as a closest patient support index value (which may be a discrete scale on the vertical axis). The patient support index may be particularly useful for archiving and storing the position of the device in a quality control system (e.g., a radiation therapy quality system). For example, the closest (discrete) index to the cluster centroid may be calculated. The index may be displayed for operator convenience and / or stored for subsequent reference, such as for archiving workflows.

[0178] This information (e.g. a patient support index and / or other (one or more) geometrical properties) may be presented to an operator in numerical form or in a graphical representation on the camera observation image, e.g. displayed to the operator in (substantially) real time, to guide him / her, e.g. in applying corrections to the positioning, orientation and / or generally spatial configuration of objects in the scene (possibly including a patient).

[0179] If several devices are to be located, these devices can be processed synchronously. This can avoid confusion and can improve the stability and accuracy of the method. Unlike detecting multiple clusters at the same time, by adding only a single item (potentially including a patient) at a time, the algorithm according to the embodiment can easily track (gradual) changes in the setting. After confirming the correct placement of the object (e.g., equipment or patient), the operator can proceed to the next item, at which time the method can use the current (confirmed) state to determine a new reference for the next stage. Thus, for example, a depth image representing a reference state (e.g., an initially empty support) can be updated based on the current image (e.g., including a newly placed object), so that changes indicating new objects added to the scene can be easily detected and processed. Alternatively, the geometric reference model (initially representing only 3D points of the table support) can be updated to include the 3D coordinates (or other spatial descriptors) of (one or more) objects that were also located in (one or more) previous steps.

[0180] In a second aspect, the present invention relates to an apparatus for assisting in positioning and / or orienting a plurality of objects (including a subject and / or at least one ancillary instrument item) on a subject support during an imaging and / or treatment procedure. Figure 2 The apparatus 10 according to an embodiment of the present invention is schematically illustrated. For example, the imaging and / or treatment procedure may include a magnetic resonance imaging procedure, a computed tomography imaging procedure, and / or a radiotherapy procedure. The apparatus may be adapted to detect the position, orientation, and / or other spatial properties of a patient and / or an auxiliary instrument, and / or to detect the spatial relationship between an object and an instrument and / or between different instrument items. Thus, the apparatus may be adapted to assist in patient and / or instrument positioning (and / or orientation) in a preparatory step of an imaging and / or treatment procedure.

[0181] The device 10 may include a processor, a data storage memory, (one or more) inputs, (one or more) outputs, a user interface and / or other means generally known to perform the methods discussed above (e.g. when programmed and / or configured accordingly). Thus, the device may include a computer and a computer program product (i.e. suitable for being run by a computer) according to an embodiment of the present invention. Additionally or alternatively, the device may include hardware specifically designed and / or configured to perform the methods according to an embodiment of the first aspect of the present invention, such as a field programmable gate array including a dedicated integrated circuit and / or a configuration to perform the methods according to the first aspect of the present invention. The device 10 may be included in a workstation or in an imaging and / or treatment system.

[0182] The device 10 includes a camera system 11 for collecting images and / or spatial data of an object support on which at least one object is placed, wherein the camera system is suitable for collecting depth information. The camera system may include at least one camera arranged above the object support, so as to obtain a two-dimensional and / or three-dimensional top view of the object support. The camera system may include a depth camera. The camera system may include at least one optical camera for collecting two-dimensional images. The camera system may include multiple two-dimensional optical cameras to collect two-dimensional images from multiple different observation points (e.g., substantially simultaneously). Therefore, the camera system (and / or the processor) may be suitable for determining depth information based on the two-dimensional image, for example by applying an algorithm for inferring depth based on stereo or multi-camera images.

[0183] The device comprises a processor 12 and an output 13. The processor 12 is adapted to obtain reference data representing an object support on which at least one object is not placed, wherein the reference data comprises reference depth information. The reference data may comprise a three-dimensional geometric model of the object support and / or at least one depth image of the object support.

[0184] The device may further comprise an input 14 for receiving a current position of the object support from the automated object support actuation system, or the processor may be adapted to determine such current position from the acquired image and / or spatial data, for example by fitting a model of the object support to the acquired data. Thus, the processor may be adapted to determine reference depth information based on the current position of the object support and a three-dimensional geometric model (e.g. defined according to its position) and / or based on the current position of the object support and a set of reference depth images (e.g. corresponding to a plurality of different possible positions).

[0185] The processor 12 is adapted to detect (one or more) objects (on the support) by taking into account at least reference depth information in the reference data and depth information in the acquired image and / or spatial data. The processor 12 is adapted to determine at least one geometric property of the or each object (on the support) and to output the at least one geometric property in a suitable form via the output 13.

[0186] Without limitation, the determined (one or more) geometric attributes may include a discrete object support index selected from a predetermined discrete ordinal set of object support indices to identify the position of the detected object associated with the at least one geometric attribute along the longitudinal direction of the object support. The determined (one or more) geometric attributes may include at least one value indicating the position and / or orientation of the detected object associated therewith. The determined (one or more) geometric attributes may include a centerline of the detected object associated therewith.

[0187] For example, the processor 12 may be adapted to detect the object(s) by determining 3D points whose corresponding acquired depth information is substantially different from the corresponding reference depth information. It may then associate the identified points and / or clusters of the points (e.g., determined by a clustering algorithm) with the object(s) being detected.

[0188] The processor may be adapted to repeat the steps of acquiring images and / or spatial data and detecting object(s) via the camera system 11, wherein after each iteration, the reference data is updated to include the detected object(s), such that the updated reference data represents the object support having the object(s) detected up to that iteration, and such that in a next iteration, one or more additional objects may be placed on the object support to be detected.

[0189] The device may include a user interface, for example, it may include a display monitor, a mouse, a keyboard and / or one or more similar human-computer interaction devices known in the art. The user interface may be suitable for controlling the processes discussed above, for example, to configure and / or control the detection of (one or more) objects and the properties determined thereby. The user interface may be suitable for interacting with an operator, such as a healthcare professional, such as a doctor, a nurse, a radiologist, an imaging technician, etc. For example, the user interface may be used by an operator to control the device and / or monitor the positioning of the patient and instrument items in the preparation phase of an imaging / treatment process.

[0190] For example, the output 13 may include a display monitor 15. The processor 12 may be adapted to present a visual representation of at least one geometric property determined for one or more detected objects to an operator via the display monitor. The processor 12 may be adapted to repeatedly perform image and / or spatial data acquisition, determine at least one geometric property, and present at least one geometric property, thereby dynamically guiding the operator to locate and / or orient the detected one and / or more objects.

[0191] For example, the shape, area and / or volume of (one or more) detected objects may be presented (i.e., visualized) as a colored overlay or outline on at least one camera image and / or model of the scene. The processor 12 may be adapted to present a visual representation on at least one camera image acquired by the camera system and / or in an overlay on a geometric model extracted from the acquired image and / or spatial data and / or from the obtained reference data. For example, at least one geometric attribute may include a center line of the (or each) detected object, which may be represented by a straight line, line segment, or curved line or a set of points indicating such a line. Other examples include a center point of an object indicated at its corresponding position in the image or model, an orientation vector, and / or another suitable marker indicating the determined attribute.

[0192] The device may further comprise a data storage interface 16. The processor may be adapted to store at least one geometric attribute in a data storage device via the data storage interface for future reference. For example, the device may comprise a data storage device, i.e. the data storage device may be integrated in the device, or may comprise an interface for accessing an external data storage, e.g. via a data communication network. For example, the data storage unit may comprise a data storage disk, a hard disk drive, a database, a network drive or any other suitable device for storing digital information.

[0193] For example, processor 12 may be adapted to retrieve at least one geometric property stored during a previous (eg, imaging and / or treatment) procedure and include the retrieved information in the visual representation for comparison purposes.

[0194] In a third aspect, the present invention relates to a medical imaging and / or radiotherapy system, wherein the system is adapted to perform a method according to an embodiment of the present invention and / or comprises an apparatus according to an embodiment of the present invention. In a fourth aspect, the present invention relates to a workstation for a medical imaging and / or radiotherapy system, wherein the workstation is adapted to perform a method according to an embodiment of the first aspect of the present invention and / or comprises an apparatus according to an embodiment of the second aspect of the present invention.

[0195] Therefore, embodiments of the present invention also relate to a magnetic resonance imaging workstation or a magnetic resonance imaging system, a computer tomography workstation or a computer tomography system, a radiotherapy workstation or a radiotherapy system and / or similar imaging and / or therapy workstations or systems, wherein the workstation or system comprises an apparatus according to embodiments of the present invention.

[0196] In a fifth aspect, the invention relates to a computer program product for executing the method according to the first aspect of the invention when run on a computer, such as an apparatus according to an embodiment of the second aspect of the invention.

[0197] In view of the description provided above relating to the method according to an embodiment of the invention, the above-mentioned other features or details of features of the apparatus (computer program product, system and workstation, respectively) according to an embodiment of the invention should be apparent, and vice versa.

[0198] As an example for illustrating an embodiment of the present invention, Figure 3 A process for using a knee positioning device in an imaging or treatment procedure (e.g., an MRI or radiation therapy procedure) is shown. According to an embodiment of the present invention, image information, such as visible spectrum image data and corresponding depth imaging data, is collected (or a setting that implicitly combines (e.g., using stereo imaging) to collect such information is used). For example, the process preparation can be performed by first placing the knee device in the correct position on the workbench, and then positioning the patient on a prepared patient support in a next step. The method discussed above can be applied to detect the knee positioning device, for example, by detecting a point cloud elevated relative to the workbench, and determining its associated (one or more) geometric properties.

[0199] Different possible positions in the longitudinal direction are indicated in different images, e.g. from H3 ("head 3") to F6 ("foot 6"). The detected shape 202 of the device may be filled with a (e.g. transparent) color in the view presented to the operator. For example, the (e.g. live) camera image may be presented in monochrome (e.g. grayscale), while the detected object 202 (e.g. the segmented shape or volume of the knee support) may be displayed as a colored overlay, e.g. purple.

[0200] The centroid determined by method 100 may be indicated by a marker 201, such as a colored dot, for example in a different color than the color used to show the detected shape 202 (e.g., yellow). A position indicator (H3-F6, such as the discrete object support volume index discussed above) may also be calculated, for example as the position of the set of discrete possible positions that is closest to the detected centroid. Thus, the position indicator may also be displayed and / or used for other purposes, such as being stored in an annotation.

[0201] When the object is positioned on the prepared patient support, a point cloud corresponding to the object can be similarly detected. Then, the geometric properties of the object can be calculated based on the detected point cloud. The calculation of the center line of the point cloud may be particularly useful as such geometric information to facilitate the correct alignment of the object. This can generally be used for objects that have great flexibility or can be non-rigidly deformed, such as patients (but not necessarily limited thereto).

[0202] exist Figure 4 , positioning of a patient on a patient support is assisted by a method according to an embodiment. In the situation shown on the left, the centerline of the table and the centerline of the patient substantially overlap. The detected shape 202 of the patient (e.g. determined by segmentation or any other suitable detection algorithm) may be represented in a manner similar to Figure 3 The exemplary manner of the knee support in FIG. 1 is shown in a (eg transparent) colored overlay (eg purple) on the camera image. Alternative solutions can also be easily envisaged, such as showing the outline of the shape on the image.

[0203] The (one or more) geometric properties determined based on the detected body (2D or 3D) shape may include (one or more) values ​​representing the centerline of the body. In the case where multiple objects are detected, the method (e.g., detection and / or display of the centerline) may be applied to each detected object of interest, however, in other embodiments of the invention, the centerline detection and / or its presentation (e.g., display on the camera image) may be specifically applied to only the patient's body and not to other objects (e.g., positioning aids), which may be described by other geometric properties, such as position coordinates, orientation vectors, one or more size indicators and / or other quantities indicating spatial properties of an object, such as applicable to any general physical object.

[0204] Because the patient's body can be easily distinguished from other detected objects, differential processing can be easily applied when processing and / or presenting information related to the patient and information related to auxiliary equipment (such as a positioning device), see some of the possible methods discussed above (but not limited to). It should be understood that, other than differential detection of patients and other inanimate objects, the method can also rely on operator input to identify the type of object (such as a selector indicating that a patient or another object is being placed in the scene), can be configured to expect a sequence of objects (including patients) in a predetermined order, or can avoid the need for differential detection altogether by treating all detected (or detectable) objects equally.

[0205] Figure 4The centerline of the body is shown in the form of points 207 along a (generally curved and / or segmented) line, for example based on a corresponding segmented (2D or 3D) shape. At least in this example, the centerline is considered to be the centerline of the body in the longitudinal direction (e.g., the upper and lower body axis) and / or the centerline relative to the longitudinal table axis, or a suitable approximation or replacement thereof. For example, a simple method to determine such a centerline is to consider a plurality of lines (for 2D imaging-based methods) or planes (for 3D-based methods) orthogonal to the centerline of the patient table, such as equally spaced lines or planes, and for each such line or plane, determine the center point (line segment or cross-sectional area) of the intersection between the line or plane and the detected body (e.g., the body component of the segmentation map or other suitable representation of the detected object). Therefore, these center points (which can be calculated as the geometric center (i.e., centroid) of the line segment or cross-sectional area) can form the point 207 of the centerline representation. Alternatives can be easily found to determine and / or present the body centerline more accurately and / or in a manner that is less dependent on the table orientation. For example, optimization or other suitable techniques may be used to find a curved and / or segmented line (e.g., a segmented continuous line) that maximizes the symmetry of the detected body relative to the line (e.g., mirror or planar point symmetry, such as left-right mirror symmetry). However, it will be appreciated that methods based on sampling of center points along predetermined planes / lines (e.g., easily determined from a (known) longitudinal table axis) (e.g., as discussed above) may be particularly simple and efficient, and may provide sufficient accuracy for the intended purpose in many application scenarios.

[0206] As another example, let X denote the primary longitudinal axis of the patient support. This may typically correspond at least generally to the patient's feet-head (inferior-superior) orientation. First, the X coordinates of the extracted points (e.g., point cloud clusters) corresponding to the object may be calculated. Then, a set of discrete X coordinates may be defined. b The positions are from the minimum to the maximum X coordinate of the points detected in the patient support or in the point cloud representing the body to define the number of bins. Each 3D point in the point cloud representing the patient can thus be assigned to a bin based on its X coordinate. For each bin, the average Y and / or Z coordinate of the points associated with that bin can be calculated and used and In particular, the centerline of an object can be calculated as a set of points (X b , ), for example one such point per bin, and can be shown on the (e.g. camera) image guiding the operator during accurate alignment of the patient on the object support.

[0207] An ideal centerline may also be displayed, e.g. corresponding to the centerline 205 of the patient support, e.g. for visual reference. The table centerline 205 may be static, e.g. unchanged between different procedures using the same system and / or in the same imaging / treatment environment, or may be associated in a predetermined manner to system parameters that may be obtained via a suitable interface, e.g. taking into account the position of an automated couch actuator system received. Thus, the table centerline 205 may be preconfigured, e.g. defined in configuration data and / or as an element of reference data describing an "empty" table (e.g. reference data obtained by a calibration procedure). This may be a static definition, or a definition of parameters that define an association with available system parameters, e.g. defining the position and orientation of a line to be displayed in / on the image in terms of table height and / or lateral and / or longitudinal coordinates (of an automated table actuator system), without being limited to this example. It will be appreciated that the table centerline may also be determined as part of segmentation and / or other camera image / data processing, even though the aforementioned predetermined configuration of the centerline and / or the predetermined relationship to readily available system parameters will generally be sufficient.

[0208] For example, by determining the centerline of the patient (e.g., visualized as point 207), the patient's alignment can be easily checked. Figure 4 In the left-hand photo shown in , the patient's body is shown aligned substantially along a straight line, wherein the centerline also (substantially) coincides with the table axis. In at least some cases, this is the preferred alignment of the patient for imaging or treatment procedures. Thus, the operator can easily detect any substantial deviation from this straight line and proper lateral centering of the body.

[0209] However, Figure 4 The photo on the right in shows a significant deviation from the target centerline position. The centerline representation 207 of the body is obviously not straight and is not along the centerline of the workbench. Some potential reasons for this can be easily seen in the visual presentation: the patient's head is slightly tilted to the left (upward direction of the page), and the overall orientation of the upper body also seems to be biased to the left side of the workbench (i.e., the displacement to the left side of the patient's head is increased), rather than straight and aligned with the longitudinal axis. In many cases, without the assistance of such visualization, this problem will be less obvious, even if there is an original overlooking observation camera to provide a bird's-eye view for the operator. However, according to an embodiment of the present invention, the operator can easily see that the centerline of the object and the workbench does not coincide, and the corrective action can be to rotate the upper body slightly to the right and remind attention that the head may be slightly tilted to the side. Similarly, it can be easily seen that the right leg is not fully extended, and the position and orientation of the feet are asymmetrical.

[0210] although Figure 4Only the XY (horizontal) plane is shown, but it will be appreciated that due to the use of depth data, for example the depth (i.e. the vertical (Z) component) of the detected centerline of the body may also be visualized. Thus, an advantage is that embodiments of the present invention enable guiding and / or verifying the positioning of a patient (and / or any associated equipment) in a horizontal plane as well as in the vertical direction in a convenient manner. For example, the Z component of the detected centerline may be indicated on a similar XZ image (synthesized or obtained by a camera at different suitable observation points) (as a simple graph of Z according to X) and / or in another suitable form. For example, the points representing the centerline may also be annotated with height information, shown as colors according to corresponding Z values, or as changing sizes according to Z, among other possibilities.

Claims

1. A method (100) for assisting in positioning and / or orienting a plurality of objects including a subject and / or at least one auxiliary instrument item on a subject support during an imaging and / or treatment procedure, the method comprising: - using a camera system to acquire (101) images and / or spatial data of the object support on which at least one of the objects is placed, wherein the images and / or spatial data include depth and / or three-dimensional 3D information, - obtaining (111) reference data representing said object support on which said at least one object is not placed, said reference data comprising reference depth and / or 3D information, and - determining (104) at least one geometric property of said at least one object based on said image and / or spatial data, wherein determining (104) the at least one geometrical property comprises detecting (105) the at least one object by taking into account at least the reference depth and / or 3D information in the reference data and the depth and / or 3D information in the acquired image and / or spatial data, wherein the detecting (105) comprises: determining (110) a three-dimensional point corresponding to the acquired (101) depth and / or 3D information different from the corresponding reference depth and / or 3D information, and associating the point and / or the determined cluster of the points with the at least one object, and Wherein determining (104) the at least one geometric property of each of the plurality of objects comprises directly computing one or more geometric properties based on the points and / or clusters of points associated with each object by computing one or more statistics of the points and / or subsets thereof.

2. A method according to any one of the preceding claims, wherein: The steps of acquiring (101) the image and / or spatial data and detecting (105) at least one of the objects are repeated (114), wherein after each iteration the reference data is updated (113) to include the detected one or more objects, such that the updated reference data represents the object support having thereon the one or more objects detected up to the iteration, and such that in the next iteration one or more additional objects placed on the object support can be detected (105).

3. The method according to any one of the preceding claims, wherein The at least one geometric property comprises a discrete object support index selected from a predetermined discrete ordinal set of object support indices to identify a position of the object along a longitudinal direction of the object support.

4. A method according to any one of the preceding claims, wherein: The at least one geometric attribute includes a detected centerline of an object associated with the at least one geometric attribute.

5. The method according to any of the preceding claims, comprising presenting (107) to an operator at least one geometric property determined (104) for one or more detected (105) objects, wherein The image and / or spatial data acquisition (101), the determination (104) of the at least one geometric attribute and the presentation (107) of the at least one geometric attribute are repeatedly (106) performed, thereby dynamically guiding the operator to locate and / or orient the detected one and / or multiple objects.

6. The method according to claim 5 comprises storing (108) the at least one geometric attribute in a data storage device and / or comprises retrieving (109) the at least one geometric attribute stored (108) in a previous process for presenting the retrieved information in the presentation (107) for comparison purposes.

7. A device (10) for assisting in positioning and / or orienting a plurality of objects including a subject and / or at least one auxiliary instrument item on a subject support during an imaging and / or treatment procedure, the device comprising: a camera system (11) for acquiring images and / or spatial data of the object support on which at least one of the objects is placed, wherein the camera system is suitable for acquiring depth and / or three-dimensional 3D information, - a processor (12), and - an output portion (13), Wherein, the processor is suitable for obtaining reference data representing the object support on which the at least one object is not placed, the reference data comprising reference depth and / or 3D information, wherein the processor is adapted to detect the at least one object by taking into account at least the reference depth and / or 3D information in the reference data and the depth and / or 3D information in the acquired image and / or spatial data, to determine at least one geometric property of the at least one object, and to output the at least one geometric property via the output unit (13), wherein the processor (12) is adapted to detect the at least one object by determining a three-dimensional point whose corresponding acquisition depth and / or 3D information is substantially different from the corresponding reference depth and / or 3D information, and associating the point and / or the determined cluster of points with the at least one object being detected, wherein the processor (12) is adapted to directly compute the at least one geometric property detection for each of the plurality of objects based on the points and / or clusters of points associated with the objects by computing one or more statistics of the points and / or subsets thereof.

8. The device according to claim 7, wherein: The processor (12) is adapted to determine the at least one geometric property, the at least one geometric property comprising a discrete object support index selected from a predetermined discrete ordinal set of object support indices to identify a position of a detected object associated with the at least one geometric property along a longitudinal direction of the object support.

9. The device according to claim 7 or 8, wherein: The processor (12) is adapted to determine the at least one geometric property, the at least one geometric property comprising a centerline of a detected object associated therewith.

10. The device according to any one of claims 7 to 9, wherein The camera system (11) comprises at least one camera arranged above the object support so as to obtain a two-dimensional and / or three-dimensional top view of the object support.

11. The device according to any one of claims 7 to 10, comprising an input (14) for receiving the current position of the object support from an automated object support execution system, and / or wherein: The processor is adapted to determine the current position based on the acquired image and / or spatial data, The processor (12) is adapted to determine the reference depth and / or 3D information based on the current position and a three-dimensional geometric model of the object support and / or based on the current position and a set of reference depth images of the object support corresponding to a plurality of different positions.

12. The apparatus according to any one of claims 7 to 11, wherein: The output unit (13) includes a display monitor (14), and wherein the processor (12) is suitable for presenting a visual representation of at least one geometric attribute determined for one or more detected objects to an operator via the display monitor, wherein the processor is suitable for repeatedly performing the image and / or spatial data acquisition, determining the at least one geometric attribute and presenting the at least one geometric attribute, thereby dynamically guiding the operator to locate and / or orient the detected one and / or more objects.

13. The device according to any one of claims 7 to 12, comprising a data storage interface (15), wherein: The processor (12) is adapted to store the at least one geometrical attribute in a data storage device via the data storage interface (15) for future reference.

14. A medical imaging and / or radiotherapy system or a workstation for such a system, wherein: The system or the workstation is adapted to perform the method according to any one of claims 1 to 6.

15. A computer program product for executing the method according to any one of claims 1 to 6 when being run on a computer.

Images