System for positioning module and computer system

Abstract

The invention relates to a system (200) for positioning a medical device relative to at least one target area (40) of a patient (70). The system comprises a support (30) for the medical module (31), and a positioning device (20) configured to move the support (30) for the medical module (31) relative to at least one target area (40). The system further comprises an imaging component (10) configured to obtain at least imaging data of the at least one landmark (50, 51, 52) arranged in a predefined relationship with the at least one target area (40). The system further includes a computing system (80). The computing system (80) includes an input interface (81) for receiving imaging data. The computing system (80) further comprises an output interface (82) for outputting data for controlling movement of the positioning device (20). The computing system (80) is configured to determine a position of the at least one landmark (50, 51, 52) based on the imaging data. The computing system (80) is further configured to control a position of the positioning device (20) relative to the at least one target area (40) based on the determined position of the at least one landmark (50, 51, 52).

Description

Technical Field

[0001] According to the preamble of the independent claim, the present invention relates to a system for positioning a medical device relative to at least one target area, a computing system for determining position data based on imaging data, a method for positioning a module relative to a target area, a computer program product for performing method steps for a positioning module, and a non-transitory computer-readable storage medium storing the computer program product.

[0002] Methods for obtaining patient position information are known. Background Art

[0003] US 9,789,338 B1 discloses a patient monitoring system for monitoring a patient undergoing radiotherapy, the system including a projector operable to project a light pattern onto the patient undergoing radiation treatment. A patient restraint device is operable to restrain the patient relative to the treatment device. An image detector is operable to obtain a patient image, and a model generation module is operable to process the patient image obtained by the image detector to generate a model of the surface of a patient part. At least a portion of the patient restraint device is colored, and the model generation module is prohibited from generating a model of the colored portion of the patient restraint device.

[0004] US 2,019,134,426 A1 discloses an image obtained by a camera system arranged to obtain an image of a patient undergoing radiotherapy, the image being processed by a modeling unit that generates a model of the monitored patient surface. Additionally, the patient monitoring system processes imaging data not used to generate the model of the monitored patient surface to determine more information about the patient's treatment. Such additional data may include data identifying the relative positions of the patient and the treatment device. This can be achieved by providing a plurality of reflective markers on the treatment device and on a mechanical bed for positioning the patient relative to the treatment device, and monitoring the presence and position of the markers in the image portion obtained by the stereo camera.

[0005] The paper titled "Automated Marker Localization in the Planning Phase of Robotic Neurosurgery", published in *IEEE Access*, Volume 5, pages 12265 - 12274, 2017, doi: 10.1109 / ACCESS.2017.271862, discloses that accurate patient registration is a key issue in medical image - guided interventions. The robotic neurosurgery system RObotic Neuro - NAvigation (RONNA) uses four reflective spheres on markers attached to the patient's skull for patient registration in physical and image spaces. In this paper, an automated localization algorithm for spherical fiducials in CT scans is proposed and clinically evaluated. The developed localization algorithm combines machine vision algorithms, biomedical image filtering methods, and mathematical estimation methods. The performance of the localization algorithm is evaluated by comparison with four skilled human operators. The developed algorithm provides machine - vision - based patient localization for the clinical application of the robotic system RONNA.

[0006] US 8,047,991 B2 discloses automatically identifying orientations in medical diagnostic ultrasound images. Area - or volume - based processes (such as region shrinking or using positions associated with flow or tissue structures) determine the direction or orientation. Based on the direction or orientation, real - time imaging in B - mode and / or flow - mode can be enhanced.

[0007] US 9,792,408 B2 discloses that the presence or absence of an object with a transponder tag can be determined in an environment where a medical procedure is performed via an interrogation and detection system, which includes a controller and a plurality of antennas placed along a patient support structure. The antennas can be placed along an operating table, bed, mattress or pad, sheet, or can be placed on a surgical drape or light - absorbing curtain. A wireless physiological condition monitor can detect the patient's physiological conditions and wirelessly transmit signals indicating these conditions.

[0008] EP 1569576 B1 discloses a method for determining the pelvic position of a surgical patient via a tracking system and inputting that position into a computer, applicable to navigated partial or total hip replacement surgery. According to this method, the patient is first aligned with an anatomical reference point regarding corresponding positioning features on a patient locator. Then the position of an index feature on the patient locator is acquired via the tracking system. Based on the position of the index feature and its known relationship with the positioning features, the position of the anatomical reference feature is calculated, and thereby the pelvic plane is defined.

[0009] EP 3254621 B1 discloses a calibrator for 3D images, a surgical positioning system and a positioning method. The calibrator for 3D images includes a calibrator plane and a calibrator handle. The calibrator plane is planar or arcuate, and at least four marker points recognized by a 3D imaging device are arranged on the calibrator plane. One end of the calibrator handle is fixedly connected to the calibrator plane, and the other end of the calibrator is provided with a connector for connecting to a surgical robotic arm.

[0010] EP 1858430 A1 discloses a method, apparatus and computer program code for automatically planning at least a part of a surgical procedure to be performed on a patient's body part. A virtual model of the body part is provided, which has data associated therewith that represents at least a part of the planned surgical procedure. Then, using data derived from the patient's actual body part, the virtual model is deformed to the body part, thereby also adjusting the part of the planned surgical procedure to reflect the anatomy of the patient's actual body part.

[0011] US 2004177449 A1 discloses an adjustable mattress and pillow system and related methods, wherein a sensing pad positioned on the top surface of the mattress influences the optimization of the microprocessor-controlled mattress and pillow contours based on the user's position. In one embodiment, the mattress and pillow system provides real-time contour optimization by using a variety of sensing techniques, which makes the system very useful in environments such as hospital intensive care facilities. Summary of the Invention

[0012] The object of the present invention is to overcome the deficiencies of the prior art, in particular to provide a more precise, more repeatable, less error-prone (especially less prone to human error) and / or more flexible positioning system for positioning a medical device on a patient. In addition, the object of the present invention is also to provide a more flexible, more adaptable and / or more robust marker and its monitoring system.

[0013] This object is achieved by the systems, methods, products and devices defined in the independent patent claims. More embodiments are provided by the dependent patent claims.

[0014] The system according to the present invention includes a support for a medical module and a positioning device configured to move the support for the medical module relative to at least one target area. The system further includes an imaging component configured to obtain at least imaging data of at least one landmark arranged in a predefined relationship with the at least one target area. The predefined relationship may refer to position and optionally also orientation. The system further includes a computing system having an input interface for receiving the imaging data and an output interface for outputting data for controlling the movement of the positioning device. The computing system is configured to determine the position of the at least one landmark based on the imaging data and to control the position of the positioning device relative to the at least one target area based on the determined position of the at least one landmark.

[0015] The at least one target area may be an area on a patient where a medical module is to be placed or should be targeted. As an example, the at least one target area may be an area or point above a blood vessel on the patient's skin. The at least one target area may include a plurality of target areas on the patient.

[0016] As another example, a patient area that may be the at least one target area may be an area or point at or above at least one limb of the patient. The limb may be at least one leg of the patient. Alternatively, the limb may be at least one arm of the patient. The target area is preferably an area or point at or above a blood vessel in the limb.

[0017] The target area may be an area or point at or above the patient's head. Thus, the target area may be the patient's oral cavity such that the medical module is positioned in the area of the patient's oral cavity (e.g., an area above the patient's oral cavity).

[0018] As an additional example, the at least one target area may be an area or point on the patient's abdomen. In particular, it is an area or point on the abdomen at or above a blood vessel.

[0019] Wherein, the at least one target area includes a plurality of target areas, and the plurality of target areas may include a plurality of areas or points selected from one or more of the patient's head, abdomen or limbs. The plurality of target areas may be a plurality of points on the same limb, or a plurality of points on different limbs. Alternatively, the plurality of target areas may be a plurality of areas or points on the patient's abdomen, or a plurality of areas or points on the patient's head.

[0020] The at least one landmark may be a reference point or a reference area. The at least one landmark may be used as an origin for locating other points of interest and in particular the at least one target area. The at least one landmark may also be used as a reference point with known coordinates and optionally also having a coded orientation, allowing a coordinate system to be determined based on the at least one landmark. The at least one landmark may be an optically recognizable symbol or an optically recognizable structure.

[0021] As an example, a landmark provided as an optically recognizable symbol or an optically recognizable structure may be included in an element of the system, such as a positioning device.

[0022] The patient may be fixed to the operating table by a body support structure, such as an open container for receiving the patient's legs and fixing the patient's legs to the operating table. The body support structure may be used as the at least one landmark.

[0023] The at least one landmark helps to determine a position or a reference system, i.e., the position of the at least one target area or the reference system for the at least one target area. The at least one landmark helps to determine the position or the reference system in a repeatable manner that is less error-prone, particularly less prone to human error.

[0024] The imaging component may for example be a digital camera, an imaging sensor with a lens system such as a charge-coupled device (CCD) or an active pixel sensor (CMOS sensor), a video camera, a stereo camera device, a digital camera combined with light detection and ranging (LIDAR), or a digital camera combined with an infrared (IR) depth camera. At least one imaging probe may for example be placed on the positioning device or placed in a defined spatial relationship with the positioning device, preferably at a distance of less than 20 meters from the positioning device, more preferably at a distance of less than 5 meters from the positioning device.

[0025] The at least one landmark is particularly arranged at a predefined distance from the at least one target area. The predefined relationship at least allows the position of the at least one target area to be determined based on the position of the at least one landmark. Preferably, the predefined relationship further allows the orientation of the at least one target area to be determined given the orientation of the at least one landmark. The predefined relationship may allow the coordinate reference system of the at least one target area to be determined based on the coordinate reference system of the at least one landmark.

[0026] The imaging component can acquire imaging data from the area where the patient lies (e.g., the area above the operating table on which the patient lies). If the imaging component can image the patient's head, for example, if the patient's head is not covered by a surgical drape, the distance between the head and the at least one target area can be measured by preoperative computed tomography (CT scan). This is supplementary to the at least one landmark, rather than an actual landmark. The computing system can use the distance between the head and the at least one target area to move the positioning device from the patient's head to the at least one target area, where the at least one landmark can be searched for to more precisely locate the at least one target area.

[0027] The computing system can evaluate the body skin using the imaging data through shape analysis. The computing system can predict the position of the at least one target area based on the shape analysis of the skin surface from the imaging data.

[0028] In a preferred embodiment, the system further includes a medical module, and the medical module includes at least one surgical instrument. The surgical instrument can be attached to or attached to a bracket for the medical module.

[0029] The at least one surgical tool can be, for example, a catheter needle, an injection needle, a dilator, a speculum, an endoscope (such as a fiber optic endoscope), surgical scissors, or a mechanical cutter (such as a scalpel).

[0030] In an advantageous embodiment, the medical module further includes at least one imaging probe, particularly an ultrasound probe, for imaging the application site of the probe.

[0031] The at least one imaging probe can be, for example, an ultrasound probe, and / or a photoacoustic imaging probe, and / or a pulsed infrared probe. Preferably, the at least one imaging probe can be used to detect changes in blood volume in the microvascular bed of the tissue, thereby obtaining a photoplethysmogram.

[0032] Using such a probe within the target area allows for creating an image of the structures (such as blood vessels) within the patient. It is particularly advantageous to use an ultrasound probe to locate blood vessels within the patient and determine their position within the patient (e.g., relative to the ultrasound probe). Knowing the in - body structures allows, for example, controlling the positioning device, such as precisely positioning the at least one surgical tool above an in - body structure such as a blood vessel.

[0033] The at least one imaging probe can be used to refine the position of the at least one target region determined by the computing system based on the at least one landmark (to improve the accuracy of the determined position), preferably also to refine the orientation of the at least one target region, or only to refine the orientation of the at least one target region. For example, this can be achieved by comparing a preoperative computed tomography (CT scan) with the images recorded by the imaging probe.

[0034] Red light or infrared light (preferably pulsed light) can be projected onto the body surface of the patient to detect changes in blood flow during the heartbeat, which are changes in the amount of light transmitted through the patient's body or the amount of light reflected by the patient's body, and in the latter case, the changes may come from the light reflected by the patient's blood vessels.

[0035] Preferably, the medical module can also include additional medical imaging device instruments or robotic devices, which can be configured to generate 3D or higher-dimensional medical imaging data.

[0036] The additional device can be, for example, a 3D ultrasound probe.

[0037] In a particularly preferred embodiment, the at least one landmark is formed by the boundary defining the at least one target region. Preferably, the computing system of the system is configured to determine the shape of the boundary defining the at least one target region.

[0038] The patient is preferably covered with a surgical drape. Preferably, the at least one landmark is, for example, an incision area on the surgical drape covering the patient. The incision forms a boundary, which preferably defines the at least one target region. The incision can have a geometric shape such as a square, rectangle, triangle, polygon, or circle, and the incision area is preferably less than 2500 square centimeters, more preferably less than 500 square centimeters.

[0039] The computing system can be used, for example, to identify the position and / or orientation of the patient on the operating table using a trained algorithm. For the same purpose, a transparent surgical drape or a thermal imager can be used, and the thermal imager can obtain thermal imaging data of the patient even when the patient is covered with a surgical drape. The computing system can use the thermal imaging data to determine the orientation of the patient covered with the surgical drape.

[0040] It is advantageous to determine the shape of the at least one landmark because it allows, for example, more information to be extracted from the at least one landmark. Preferably, the shape of the at least one landmark is such that it is asymmetric in at least one direction, thus allowing the orientation of the at least one landmark to be used to indicate the orientation of the patient. As an example, an incision in a surgical drape having the shape of an isosceles triangle with a vertical axis of symmetry can be used as the at least one landmark. The vertical axis of the isosceles triangle serves as an arrow, and the surgical drape is placed over the patient such that the arrow points towards the patient's head, thereby defining the orientation in addition to the position.

[0041] In addition to or as an alternative to the patient orientation information obtained through the at least one landmark, information regarding the patient orientation can also be indicated by a symbol. For example, the patient can wear a headband including a unique marker such as a predefined color or an infrared reflective coating. In this case, the headband is not covered by the surgical drape or can still be seen through the surgical drape. The computing system can identify the symbol based on the imaging data and associate it with a predefined position on the patient (such as the patient's head). Combining this information with the position data of the at least one landmark can allow the determination of the patient's orientation.

[0042] The patient's face can be used to detect the patient orientation because it is capable of distinguishing left from right.

[0043] To determine the orientation and / or position of the patient relative to, for example, an operating table, a deformable surface, preferably a vacuum positioning pillow, can be placed between the patient and the operating table surface. There can be components that are rigidly attached to the deformable surface when placed on the deformable surface to quantify the contact occurring between the patient and the surface, and this component is further referred to as a body contact sensor strip. These can include:

[0044] - Strain or stretch sensors, which can rely on a change in resistance, such as strain gauges or conductive rubber, and / or they can rely on a change in capacitance, such as a change in dielectric dimensions; and / or they can rely on piezoelectric charges in a strain function. Advantageously, the strain or stretch sensors can be suitable for integration into or onto textiles, such as, for example, integrated into or onto a surgical drape.

[0045] - Contact sensors for distinguishing contact with the patient. Contact can be detected, for example, through conductivity, and for this purpose, conductive wires (such as silver wires) can be integrated into, for example, the fabric on top of the operating table.

[0046] - Force or pressure sensors; which can be derived from a combination of a strain sensor and a known flexure.

[0047] - Shape sensor; it can be obtained from a discrete grid embedding strain or stretch sensors, or contact sensors, or force or pressure sensors. Alternatively or additionally, dedicated shape sensors such as tower gratings FBG or 3D tracking sensors (such as electromagnetic tracking) can also be embedded.

[0048] - Distance sensor; it can be used to measure the distance of the top surface of the pillow relative to the operating table. For example, an optical distance sensor grid can be used, which can measure the distance along the normal of the operating table surface.

[0049] The deformable surface can at least partially quantify the patient reference relative to the position frame, where the reference frame can be, for example, the reference frame of the positioning device. This can be based on, for example, one of the following:

[0050] - The orientation of the patient's body relative to the table, where assuming the patient is supine, the orientation of the up / down axis of the patient's body on the table can be determined. The up axis can be the axis towards the patient's head, and the down axis can be the axis towards the patient's feet.

[0051] - The contour of the patient's body relative to the table

[0052] - The registration of the digital anatomical model with the operating table. This can be achieved by applying a registration method that uses in-situ anatomical quantification and the digital model as inputs and provides the pose information of the digital model relative to the operating table.

[0053] In a preferred embodiment, the positioning device has a robotic arm.

[0054] The robotic arm is preferably configured to perform positioning tasks with a positioning accuracy better than 20 mm, preferably better than 5 mm, and more preferably better than 1 mm. The robotic arm preferably has a movement range less than 5 m, more preferably less than 3 m, and most preferably less than 1 m. The robotic arm can be configured to perform high-speed movements with a lower accuracy than low-speed movements (such as less than 5 cm per second), for example, more than 5 cm per second.

[0055] In a preferred embodiment, the imaging component is configured to provide depth information of an object in the field of view of the imaging component.

[0056] The field of view of the imaging component refers to the solid angle to which the imaging component (such as a camera or an infrared camera) is sensitive to electromagnetic radiation. If the imaging component includes more than one image acquisition unit, for example, if it includes a 2D camera and an IR depth camera, or if it includes a 2D camera and LIDAR, the imaging component has more than one field of view because each image acquisition unit has its own field of view.

[0057] Depth information is information that at least allows the extraction of the distance between objects, and preferably also allows the extraction of spatial information (such as the relative position between objects), for example, allows the determination of the Euclidean vector specifying the position of one object relative to another object, or the absolute position of an object in a given reference coordinate system.

[0058] Not all of the image acquisition units included in the imaging component can be used to determine depth information. The combination of two image acquisition units may be able to obtain depth information, while each of the two optical instruments used alone cannot extract depth information. For example, two cameras horizontally displaced from each other can be used to obtain two different images of a scene taken at different angles. By comparing these two images, depth information can be obtained.

[0059] Since the imaging component relies on discrete imaging sensors, such as sensors with a given number of pixels, such as charge-coupled devices (CCDs), the depth information will also be discrete.

[0060] In a particularly preferred embodiment, the imaging component at least includes a stereo camera device, a 2D camera, and a LIDAR, or a 2D camera and an IR depth camera.

[0061] The stereo camera device can be a camera with two or more lenses, each lens having an independent image sensor or film holder, or can be a device composed of two or more individual cameras. For a device composed of two or more individual cameras, the cameras must be triggered approximately simultaneously, that is, each camera acquires images approximately simultaneously. Approximately simultaneously means that the time difference is less than 1 second, preferably less than 0.1 second, and more preferably less than 0.01 second.

[0062] The computing system can further be configured to perform image registration between preoperative images, such as, for example, image registration between 2D pelvic fluoroscopy and 3D CT scans or between 3D CT body surfaces and 3D depth cameras, in order to determine the position of the at least one target area.

[0063] Particularly preferably, the imaging component can include a rotating 3D LIDAR or a fisheye RGB camera. If the imaging component includes a rotating 3D LIDAR or a fisheye RGB camera, the computing system can determine the obstacles around the patient and / or the positioning system with high precision.

[0064] In a preferred embodiment, the computing system is configured to identify the position of the at least one landmark in real time based on the imaging data.

[0065] The computing system can use the real-time position of the at least one landmark to adjust the position of the positioning device according to the movement of the patient. The movement of the at least one landmark and / or the at least one target area can be caused, for example, by the patient's breathing.

[0066] Real - time identification of the position of the at least one landmark means repeatedly (preferably continuously) identifying the position of the at least one landmark. When repeatedly (preferably continuously) identifying the position of the at least one landmark, even if there is a slight delay, such as less than 3 seconds, preferably less than 1 second, it is still considered real - time identification.

[0067] Real - time identification of the position of the at least one landmark can be accomplished by means of traditional image processing or preferably using artificial intelligence. As an example, a neural network trained on imaging data with or without depth information can be used to identify the position of the at least one landmark in real - time. The same method can be used in both of the above cases. A feasible deep neural network (DNN) model is the encoder - decoder architecture. The encoder allows extracting robust features of the imaging data, that is, features of the imaging data containing depth information, and it can be a backbone network pre - trained on a public imaging dataset (such as a MobileNet backbone network), or trained from scratch using a large amount of imaging data of the imaging component (i.e., imaging data containing the depth information of the imaging component) or similar imaging data (for example, the data may be acquired by another imaging component and / or contain different types of landmarks). The decoder projects the extracted features back to the original image size through, for example, a series of upsampling layers. In particular, the last layer performs effective semantic segmentation of the field by classifying each pixel as a landmark or background. Additional image post - processing can be applied, such as aiming to filter out false detections. The at least one landmark can be extracted by means of a connected - component algorithm.

[0068] In an advantageous embodiment, the computing system is configured to further decode a fiducial marker preferably placed near the at least one target region.

[0069] The fiducial marker can be used as a landmark or in combination with another landmark.

[0070] Near the at least one target region means, for example, less than 30 cm, preferably less than 20 cm, more preferably less than 5 cm from the at least one target region.

[0071] Information related to the position of the fiducial marker and / or the position of at least one of the at least one landmark can be encoded in the fiducial marker. For example, for a landmark with a square geometry, the position of the at least one landmark can be given by specifying the coordinates of the four corners of the landmark relative to a given fiducial marker. Preferably encoded in the fiducial marker is the identity of the at least one landmark. For example, specifying the landmark adjacent to a given fiducial marker as landmark "A", or any unique identity. Preferably, the total number of landmarks can also be encoded in the fiducial marker.

[0072] The position of the at least one fiducial can be identified in real time by decoding in real time the position encoded in the fiducial marker by a computing system.

[0073] Fiducial markers placed near the at least one target area, such as AprilTags, ArUco tags, ARTags, ARToolKit tags, or QR codes, can be fixed to, for example, a surgical drape, directly to the patient, or to any suitable object near the at least one target area. As an example, the fiducial marker can be directly woven, glued, or mechanically attached to the surgical drape, preferably around an opening in the surgical drape, and the opening can be used as a fiducial. Temporary devices can also be used to place and remove fiducial markers fixed to the patient.

[0074] Fiducial markers can also be used to store information such as patient orientation or general patient information. Fiducial markers that store only information such as patient orientation or general patient information may not need to be placed near the at least one target area.

[0075] In a particularly preferred embodiment, the computing system is configured to determine the centroid of the at least one fiducial from the imaging data, determine the 3D Cartesian coordinates of points in the imaging data, and determine the 3D surface normal at the 3D centroid of the at least one fiducial based on the 3D Cartesian coordinates of points near the centroid.

[0076] The 3D centroid (commonly also referred to as the geometric center) of the at least one fiducial is the point at which the at least one fiducial would balance when placed on a needle.

[0077] The imaging data is discrete data. For example, the imaging data of a fiducial will contain a certain number of pixels, such as 2 million pixels.

[0078] The 3D Cartesian coordinates of points within a small rectangular area (preferably 50x50 pixels, more preferably 30x30 pixels, most preferably 10x10 pixels) around the centroid of the at least one fiducial are used to robustly estimate the 3D surface normal at the 3D centroid of the at least one fiducial. For example, this can be achieved by using a least-squares optimization technique to fit a tangent plane to the surface near the 3D centroid using a set of 3D adjacent points and directly deriving the surface normal from the estimated plane equation.

[0079] Preferably, the computing system is configured to determine the centroid of the at least one fiducial based on the imaging data. Each of the at least one fiducials has its own associated centroid.

[0080] In a preferred embodiment, the computing system is configured to fit the shape of the at least one fiducial.

[0081] The shape of the at least one fiducial can be fitted in the following manner: based on the imaging data, the contour of the at least one fiducial is determined, which can have a geometric shape such as a square, rectangle, triangle, polygon, or circle; then a geometric shape is fitted that minimizes the difference between the fitted geometric shape and the determined contour of the at least one fiducial. The geometric figure does not have to lie in a two-dimensional plane; for example, it can have curvature or bend and fold in different directions at different positions of the geometric figure. Thus, even if the at least one fiducial has a non-two-dimensional shape, the shape of the at least one fiducial can be accurately fitted.

[0082] For example, if the shape of the fiducial is known, the 3D rectangular shape of the at least one fiducial can be obtained directly from the 3D coordinates of the four corner points of the 2D fitted rectangle of the at least one fiducial extracted.

[0083] In a preferred embodiment, the computing system is configured to further determine the relative orientation and / or position between the stent for the medical module and the at least one fiducial based on the imaging data.

[0084] Knowing the relative orientation between the stent for the medical module and the at least one fiducial allows, for example:

[0085] - The stent for the medical module can be placed in a given orientation relative to the at least one fiducial by a positioning device;

[0086] - Position the medical device mounted on the stent for the medical module in a relative orientation with respect to the at least one fiducial.

[0087] Since the at least one fiducial is arranged in a predefined relationship with the at least one target region, the medical device can be positioned in a predefined relative orientation with respect to the at least one target region. Knowing this relative orientation helps, for example, to place the medical device contained in the medical module in the desired orientation.

[0088] The pose (i.e., the combination of position and orientation) of the medical module can be arbitrarily selected. However, for some applications, it is important to place the medical module (preferably the medical device) at a predefined angle with respect to a certain axis (preferably the normal of the target region). As an example, it is desirable for the longitudinal probe axis (i.e., the axis along the imaging probe) to be aligned with the normal of the target region. In such cases, the orientation of the medical module can be selected to ensure an orthogonal alignment between the imaging probe and the determined surface of the at least the target region. For example, a coordinate reference system can be obtained by defining a second axis that follows, for example, the patient orientation. The third axis required to obtain a three-dimensional coordinate reference system needs to be selected to be orthogonal to the above two axes.

[0089] Patient orientation can be determined by using an ultrasound probe of at least one imaging probe and by using the Doppler effect. As an example, artificial intelligence can be trained to detect and distinguish venous blood flow from arterial blood flow based on imaging data of the ultrasound probe. By comparing the obtained results with known human anatomy, the patient's orientation can be determined.

[0090] In an advantageous embodiment, the computing system is configured to determine a 3D map of the surroundings of the at least one landmark based on depth information of objects in the field of view of the imaging component.

[0091] In a preferred embodiment, the computing system is configured to determine an obstacle-free path connecting the current position of the stent with the medical device and a target position near the at least one target area based on the 3D map of the surrounding environment. The stent can move along the path without colliding with obstacles.

[0092] According to another aspect of the present invention, a computing system for determining positioning data based on imaging data is disclosed. The computing system includes a data input for receiving imaging data and a data output for outputting positioning data. The computing system is configured to determine a position of at least one landmark relative to a reference point based on the received imaging data. The computing system is configured to provide positioning data based on the determined position of the at least one landmark, and output the positioning data.

[0093] The computing system for determining positioning data based on imaging data can be used in combination with the aforementioned system for positioning a medical device relative to at least one target area of ​​a patient. The computing system can be arranged at a location remote from the positioning device of the system for positioning the medical device.

[0094] According to another aspect of the present invention, a method for operating a system for positioning a medical device relative to a target area is provided, preferably using a system for positioning a medical device relative to a target area as described above. The system for positioning a medical device includes a bracket for a medical module, and a positioning device for moving relative to the at least one target area, as well as an imaging component and a computing system. The computing system includes an input interface for receiving the imaging data and an output interface for outputting data for controlling the movement of the positioning device. Imaging data is obtained using an imaging component. The position of the at least one landmark is determined by the computing system based on the imaging data. The position of the positioning device relative to the at least one target area is controlled by the computing system based on the determined position of the at least one landmark.

[0095] According to another aspect of the present invention, a computer program product is provided having program code instructions stored on a computer readable medium, which, when executed on a computer, execute the method steps of the method for operating a system for positioning a module relative to a target area.

[0096] According to another aspect of the present invention, a non-transitory computer-readable storage medium storing a computer program product is disclosed.

[0097] The present invention can be employed in clinical interventions such as TAVI procedures or gastrointestinal surgeries according to the above aspects.

[0098] For example, in a TAVI procedure, the surgical tool of the medical module will include at least one catheter, in particular a catheter needle, and the at least one target area will be an area or point on the patient's leg at or above the femoral artery. The position of the catheter needle (i.e., carried by the stent) will be adjusted by the positioning device such that the catheter needle is positioned or aimed at the area on the patient's leg at or above the femoral artery.

[0099] As another example, in a gastrointestinal surgery, the surgical tool of the medical module will include at least one endoscope, and the at least one target area will be an area or point at or above the patient's oral cavity. The position of the endoscope will be adjusted by the positioning device such that the endoscope is positioned or aimed at the patient's oral cavity area. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] The present invention will now be described with reference to certain embodiments and the accompanying drawings, which show:

[0101] Figure 1 : A schematic diagram of a first embodiment of a system for positioning a medical device relative to at least one target area of a patient;

[0102] Figure 2 : The calculated poses of two landmarks and their calculated centroid for an embodiment of landmarks.

[0103] Figure 3 : A representation of an embodiment of a system for patient body orientation detection.

[0104] Figure 4 : A representation of an embodiment of a system for patient body orientation detection using image registration. DETAILED DESCRIPTION

[0105] Figure 1Shows a first embodiment of system 200. System 200 includes a positioning device 20. The positioning device 20 is configured to move a stent 30 for a medical module 31 relative to a target area 40 on a patient 70 covered by a surgical drape 60. The target area 40 is an area near a blood vessel 71 of the patient 70. An imaging component 10 and its field of view 11 are configured to obtain imaging data of a landmark 50 arranged in a predefined relationship with the target area 40. The landmark is formed by an opening in the surgical drape 60. The imaging component 10 is connected (wired or wirelessly) via a connection 83 to a computing system 80. The computing system 80 includes an input interface 81 for receiving the imaging data. The computing system 80 is configured to determine the position of the landmark based on the imaging data. The computing system 80 is configured to control the position of the positioning device 20 relative to the target area 40 by outputting data via an output interface 82 included in the computing system 80. The output data is used to control the movement of the positioning device 20 connected (wired or wirelessly) to the computing system 80 via a connection 84. The positioning device 20 includes a robotic arm 21 that allows for precise positioning of the stent 30 of the medical module 31.

[0106] Figure 2 : Schematic diagram of experimental results showing the poses (i.e., positions and orientations) of two landmarks and their centroids. Both the pose and the centroid are determined by the computing system 80. The surgical drape 60 covers the patient 70. The surgical drape 60 has two openings that serve as landmarks 51 and 52. Both landmarks 51 and 52 have centroids 90 and 91 determined by the computing system 80. Based on the imaging data containing depth information, the computing system can determine:

[0107] - The normals 100, 101 of the local surface of the patient 70 at the centroids 90, 91 of the landmarks 52, 51. The normals 100, 101 extend from the centroids 90, 91 towards the inside of the patient 70.

[0108] - The first tangents 110, 112 of the surface at the centroids, which form right angles with the normals 100, 101 of the surface of the patient 70 at the centroids 90, 91 of the landmarks 52, 51.

[0109] - The second tangents 111, 113 of the surface at the centroids, which form right angles with the normals 100, 101 of the surface of the patient 70 at the centroids 90, 91 of the landmarks 52, 51 and also form right angles with the first tangents 110, 112 of the surface at the centroids.

[0110] The normal 100, the first tangent 110, and the second tangent 111 form an orthogonal basis. The normal 101, the first tangent 112, and the second tangent 113 also form an orthogonal basis.

[0111] Preferably, the normal vectors 100, 101, the first tangents 110, 112 of the surface at the centroid, and the second tangents 111, 113 of the surface at the centroid are determined by the computing system such that they are all unit length. Thus, they form two orthogonal bases.

[0112] Figure 3 An embodiment of an operating table is shown that can be used for Figure 1 the system shown in. The operating table surface 122 is covered by a vacuum positioning pillow 121. A sensor 120, such as a contact sensor, preferably a contact sensor strip, is provided on top of the vacuum positioning pillow 121. Based on information from the vacuum positioning pillow 121 and / or the sensor 120, the computing system 80 can determine the orientation of a patient lying on the vacuum positioning pillow 121 and covering at least a portion of the sensor 120. Based on information from the vacuum positioning pillow 121 and / or the sensor 120, the computing system 80 can determine the direction of the superior axis 130 of the patient and / or the direction of the inferior axis 131 of the patient.

[0113] Figure 4 An embodiment is shown in which image registration is performed between the digital model 140 and the in-situ anatomical quantification 141 by the computing system 80. Based on the image registration, pose information of the digital model 142 relative to the operating table can be obtained. The computing system 80 can determine the in-situ anatomical quantification 141 based on a deformable surface (preferably a vacuum positioning pillow) placed between the patient and the operating table surface 122. The computing system 80 can determine the digital model based on imaging data, preferably based on imaging data containing depth information, more preferably based on a preoperative 3D CT scan.

Claims

1. A system (200) for positioning a medical device relative to at least one target region (40) of a patient (70), the system comprising: - a support (30) for a medical module (31), and - a positioning device (20) configured to move the support (30) for the medical module (31) relative to the at least one target region (40), and - an imaging component (10) configured to obtain at least imaging data of at least one landmark (50, 51, 52) arranged in a predefined relationship with the at least one target region (40), and - a computing system (80) wherein the computing system (80) comprises: - an input interface (81) for receiving the imaging data, and - an output interface (82) for outputting data for controlling the movement of the positioning device (20) wherein the computing system (80) is configured to: - determine the position of the at least one landmark (50, 51, 52) based on the imaging data, and - control the position of the positioning device (20) relative to the at least one target region (40) based on the determined position of the at least one landmark (50, 51, 52).

2. The system according to claim 1, further comprising a medical module (31), the medical module (31) comprising at least one surgical instrument, the surgical instrument being attachable or attached to the support (30) for the medical module (31).

3. The system according to claim 2, wherein The medical module (31) further comprises at least one imaging probe, in particular an ultrasound probe, for imaging the application site of the probe.

4. The system according to one of claims 1 to 3, wherein, The computing system (80) is further configured to determine the shape of the landmark (50, 51, 52) formed by the boundary defining the target region (40).

5. The system according to any one of claims 1 to 4, wherein, The positioning device (20) has a robotic arm (21).

6. The system according to one of claims 1 to 5, wherein, The imaging component (10) is configured to provide depth information of an object in the field of view (11) of the imaging component (10).

7. The system according to one of claims 1 to 6, wherein The imaging component (10) comprises at least one of the following - a stereo camera device - a 2D camera and a LIDAR; and - a 2D camera and an IR depth camera.

8. The system according to any one of claims 1 to 7, wherein, The computing system (80) is configured to identify the position of the at least one landmark (50, 51, 52) in real time based on the imaging data.

9. The system according to any one of claims 1 to 8, wherein, The computing system (80) is configured to decode a fiducial marker preferably placed near the at least one target region (40).

10. The system according to one of claims 1 to 9, wherein, The computing system (80) is configured to - determine the centroid (90, 91) of the at least one landmark (50, 51, 52) from the imaging data, - determine the 3D Cartesian coordinates of points in the imaging data, and - determine the 3D surface normals (100, 101) at the 3D centroid of the at least one landmark (50, 51, 52) based on the 3D Cartesian coordinates of points near the centroid (90, 91).

11. The system according to one of claims 1 to 10, wherein, The computing system (80) is configured to fit the shape of the at least one landmark (50, 51, 52).

12. The system according to one of claims 1 to 11, wherein, The computing system (80) is configured to determine a relative orientation between the stent (30) for the medical module (31) and the at least one landmark (50, 51, 52).

13. The system according to one of claims 6 to 12, wherein The computing system (80) is configured to determine a 3D map of the surroundings of the at least one landmark (50, 51, 52) based on depth information of an object in the field of view (11) of the imaging component (10).

14. The system according to claim 13, wherein, The computing system (80) is configured to determine a collision-free path connecting the current position of the stent (30) for the medical module (31) to the target area (40) based on the 3D map of the surroundings, and the stent (30) for the medical module (31), preferably including the medical module (31), can move along the collision-free path without colliding with obstacles.

15. A computing system (80) for determining positioning data based on imaging data, the system comprising: - an input interface (81) for receiving imaging data, and - an output interface (82) for outputting positioning data, wherein the computing system (80) is configured to: - determine the position of at least one landmark (50, 51, 52) relative to a reference based on the received imaging data, - provide positioning data based on the determined position of the at least one landmark (50, 51, 52), and - output the determined position of the landmark (50, 51, 52) and / or the positioning data.

16. A method for operating a system for positioning a medical device relative to a target region (40), preferably using a system (200) for positioning a medical device relative to a target region (40) according to one of claims 1 to 14, wherein, The system comprises: - a stent (30) for the medical module (31), and - a positioning device (20) for moving the stent (30) for the medical module (31) relative to the at least one target area (40), and - an imaging component (10), and - a computing system (80) wherein the computing system (80) comprises: - an input interface (81) for receiving the imaging data, and - an output interface (82) for outputting data for controlling the movement of the positioning device (20), wherein the method comprises the following steps - obtaining, by the imaging component (10), imaging data of at least one landmark (50, 51, 52) arranged in a predefined relationship with the at least one target area (40), - determining, by the computing system (80), the position of the at least one landmark (50, 51, 52) based on the imaging data, and - controlling, by the computing system (80), the position of the positioning device (20) relative to the at least one target area (40) based on the determined position of the at least one landmark (50, 51, 52).

17. A computer program product comprising program code instructions stored on a computer-readable medium, which when executed on a computer perform the method steps according to claim 16.

18. A non-transitory computer-readable storage medium storing the computer program product according to claim 17.

Images