Positioning and orientation of manually guided modal devices

Abstract

A manual guided modal device (4) is to be positioned with less effort and / or radiation load. To this end, an object model is generated from 3D data of the object (3). Furthermore, a surface model of the object (3) is generated. And the two models are mutually registered. A starting position and a starting orientation of the manual guided modal device (4) are calculated from the target area (1) in the object model. In addition, a target image of the object (3) is calculated on the basis of the object model, the target region and the starting position and the starting orientation, and in addition, a real ultrasound image is detected from the starting position and the starting orientation. Based on a deviation between the ultrasound image and the target image, repositioning and / or reorientation of the manual guided modal device (4) is carried out.

Description

Field of the Invention

[0001] The present invention relates to a method for positioning and orienting a modal device for manual guidance. Furthermore, the present invention relates to a device having a modal device for manual guidance, which should be positioned in a suitable manner. Background Art

[0002] Microwave and radiofrequency ablation are a treatment option for early-stage liver cancer. The ablation needle must be precisely positioned in the tumor to be removed. Usually, pre-interventional 3D imaging studies (CT or MR) can be used, which show the location of the tumor. Important for the intervention is the optimal insertion and placement of the needle. For this purpose, imaging methods can assist the doctor.

[0003] There are various methods for controlling the positioning of the needle. For example, ultrasound is used as a real-time imaging system to visualize the needle during insertion. In some cases, pre-interventional MR or CT data sets are registered with real-time ultrasound signals to assist in tumor localization. The registration can be rigid or elastic. Visualization can be performed by images arranged side by side or by superimposed images. For robust registration, a tracking device is usually required, which continuously monitors the position and orientation (and / or direction) of the ultrasound probe and adjusts the visualization of the CT / MR data set accordingly. Registration is initiated by identifying position markers in the ultrasound and CT / MR (exact features in the image, such as blood vessels) or by user interaction or by automatic image analysis. Usually, at least two image planes from different directions are required to localize such position markers in 3D. Alternatively, 3D scanning can be performed by ultrasound (a recording sequence from at least two different directions), and multiple image stacks can be oriented relative to each other by manual user interaction or a suitable algorithm. Usually, due to patient position changes and respiratory artifacts, organ deformation occurs, making image registration troublesome and requiring a trained and experienced operator to be able to perform the method.

[0004] In further practice, CT or MR devices are used for continuously monitoring the positioning of the needle. The advantage of this method is that it provides a complete 3D data set, which makes the positioning of the needle easier. The disadvantages are that CT (computed tomography) is associated with a radiation load for the patient and the operator, and MR (magnetic resonance tomography) is associated with limited accessibility and high costs. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide precise imaging for the intervention as an alternative to CT and MR.

[0006] According to the present invention, this technical problem is solved by the device and method according to the independent claims. Advantageous refinements of the present invention result from the dependent claims.

[0007] The present invention thus provides a method for positioning and orienting a manually guided modality device. Throughout this text, the term "ultrasound device" is also used to represent other manually guided modality devices (e.g., SPECT device; single photon emission computed tomography). The term "manually guided" means that at least a part of the modality device (e.g., the probe) can be manually guided. The ultrasound device is applied to replace the cumbersome CT method and the expensive MR method. However, in order to achieve high image quality especially for the positioning of (medical) instruments, the ultrasound device needs to be positioned and oriented very precisely. In addition to the position, the orientation of the ultrasound device is also important for it.

[0008] In one step of the method, 3D data of the object is loaded. Loading the 3D data can be part of the acquisition process. However, the 3D data (e.g., MR or CT data) can also be provided from a storage medium and / or via an interface. In this case, it must be loaded and / or stored in the working memory of the computing device, for example. The object can be a human or animal patient, but can also be a technical object.

[0009] In a further step, an object model of the object is automatically generated from the 3D data. The object model is usually a 3D model that reproduces the object together with its internal structure. Thus, the object model can have, for example, spatially segmented regions accordingly. In the case of a patient, the object model is a patient model that preferably reproduces important internal organs, such as the liver, blood vessels, chest cavity, etc. The object model can be automatically obtained from the 3D data by means of a suitable application.

[0010] Furthermore, in the method according to the invention, a target region in the object model is predetermined. The target region can be, for example, a lesion to be observed and / or treated. The target region is marked in the object model, for example, by user interaction. If necessary, the target region can also be automatically determined or predetermined by data input.

[0011] In addition, the surface data of the object is detected. Since the ultrasound device must be positioned on the surface of the object, it is essentially necessary to detect at least part of the surface of the object. For example, if markers are present on the surface, they are detected, for example, optically or, if necessary, by other electromagnetic techniques. The detection technique should be coordinated with the surface properties of the object.

[0012] A surface model of the object is automatically generated from the surface data. Thus, a surface model of the object and / or the patient is generated independently of the object model. The object model is automatically generated from complex, preferably pre-intervention 3D data (e.g., MR or CT), while the current surface model can be simply obtained from the data obtained optically.

[0013] Optionally, additionally detect the first surface model also before the intervention during the recording of the object model. Then there is a direct association between the object and the surface model here and it can be compared with the new or current surface model during the ultrasound-assisted intervention.

[0014] This has the particular advantage that the two surface models can include larger patient segments than the object model and are thus easier to register.

[0015] In a further step, registration of the surface model relative to the object model is carried out. The object model also reproduces the surface structure which matches or is registered with the surface structure of the surface model. Thus, the internal structure of the object can be estimated in terms of position and orientation based on the current surface model. In certain cases, the above-mentioned first surface model can be used for the registration.

[0016] Furthermore, the starting position and starting orientation of the ultrasound device on the object model are automatically calculated based on the target region. The ultrasound device, in particular its probe, has to be positioned on the object. For this purpose, the starting position and starting orientation of the ultrasound device are virtually calculated based on the desired target region on the object model. Since the surface model is registered with the object model, there is to a certain extent a refreshed object model on which the ultrasound device is virtually positioned and oriented. Thereby, a first estimate of the position and orientation of the ultrasound device is obtained in order to obtain a suitable recording of the target region.

[0017] In a further step, the target image of the sub-region of the object is automatically calculated based on the object model, the target region and the starting position and starting orientation of the ultrasound device. Thus, the target image (for example, a virtual ultrasound image) which reproduces the sub-region of the object is estimated from the starting position and starting orientation of the ultrasound device. For this purpose, a layer view (target image) corresponding to the starting position and starting orientation is usually calculated from the 3D data set. It is usually the responsibility of the user to adjust the real ultrasound image to match the calculated target image. The user can start here with the comparison of the real ultrasound image with the calculated MR / CT contrast image which is calculated from the 3D or voxel data set for the respective layer. The above-mentioned sub-region of the object preferably includes the above-mentioned target region (ROI: region of interest). Since the recording geometry of the ultrasound device is usually known, the target image can be estimated based on the determined starting position and starting orientation of the ultrasound device with the aid of the object model. In this document, the term "virtual ultrasound image" is also used to represent the term "target image". In addition to the virtual ultrasound image (simulated US-contrast image), the calculated target image can also be a CT contrast image or an MR contrast image.

[0018] Furthermore, in the method according to the invention, a real ultrasound image of a sub-region of an object is detected by an ultrasound device in a starting position and a starting orientation. For this purpose, the real ultrasound device is brought into a real starting position and a real starting orientation, which corresponds to the automatically calculated virtual starting position and virtual starting orientation. If necessary, the virtual data corresponds to the real data, where the data can be identical or can be obtained by a coordinate transformation.

[0019] In a subsequent step, the (real) ultrasound device is repositioned and / or reoriented based on the deviation between the target image and the real ultrasound image. This repositioning and / or reorientation can be carried out automatically, semi-automatically or manually. The deviation between the target image and the real ultrasound image can be determined, for example, based on position markers. For example, if a blood vessel branch in the target image is present in a different image region than in the real ultrasound image, there is a corresponding deviation, which can be corrected, if necessary, by reorienting the real ultrasound device (fully or largely). Thus, by means of a target image obtained based on an object model and / or preferably 3D data detected before the intervention, the real ultrasound device can be optimally positioned and oriented, since the object model is updated by an easily obtainable surface model. Thus, the ultrasound device can be positioned and oriented in a simple manner (optimally) without an additional radiation load.

[0020] Thus, it is possible to assist the user in identifying position markers, injuries / tumors and needles with greater safety and to accelerate the corresponding workflow. In this method, information about the interior of the body (3D-MR / CT data), information about the body surface (patient virtual avatar based on sensor data from, for example, a stereoscopic camera) and information about the actual position / orientation of the ultrasound probe are used to determine the appropriate target position / orientation of the ultrasound probe in order to visualize position markers, injuries and needles (devices).

[0021] In an embodiment, it is provided that the object is a patient and that during the automatic generation of the object model, the organs of the patient are automatically segmented. Thus, the object model is a patient model, where organs, such as the liver, blood vessels, etc., are modeled separately. The modeling is based on 3D data. If necessary, the patient model can be supplemented by a deformation model in order to, for example, take into account deformations caused by the patient's position and respiratory artifacts.

[0022] According to another embodiment, the target image is a calculated ultrasound image, an MR image or a CT image. Thus, a virtual image can be established in the best possible way for the respective situation.

[0023] In another embodiment, it is provided that, in addition to the 3D data of the object, first surface data of the object obtained during the detection of the 3D data is additionally loaded, a first surface model is generated from the first surface data, and the first surface model implicitly registered with the object model is used for registering the surface model with respect to the object model. This has the advantage that the first surface model of the object can be obtained before the intervention under the same conditions as the 3D data or the object model of the object. Thus, the first surface model is automatically registered with the object model and can therefore be used for the indirect registration of the object model with the current surface model.

[0024] In another embodiment, a 3D camera is used for detecting the surface data of the object, and this 3D camera is also used for detecting the position and orientation of the manually guided modality device. Thus, the 3D camera has a dual function.

[0025] Furthermore, in one embodiment, it can be provided that the position and orientation of the instrument are detected by the 3D camera, and the instrument is optionally guided on the manually guided modality device. The 3D camera thereby obtains an additional function, namely, monitoring the instrument. For example, there is the following process: The needle (i.e., the instrument) is moved forward through a guide rail fixed to the ultrasound head, or the operator guides the ultrasound head with one hand and the needle holder with the other hand.

[0026] In another embodiment, the path of the instrument entering a sub-region of the object is planned according to the object model, and this path is used for the automatic calculation of a target image (such as a virtual ultrasound image). The instrument can be, for example, a surgical instrument, such as an ablation needle. The path of the ablation needle, etc. can be planned by processing the model, for example. Here, hidden structures, such as ribs, can also be considered. For example, a path passing through the ribs must be planned. The planned path can be used for calculating a virtual ultrasound image. The path should be clearly visible (in-plane) in the ultrasound image. Therefore, the planned path should be considered for optimizing the position and orientation of the ultrasound device.

[0027] The planning of the path can include repositioning of the object. For example, in a first planning step, a first draft path that is not optimal for the processing is planned. Thus, additional planning can include steps of repositioning the object and / or the patient. This means a change in the pose (position and orientation) of the object. Thereby, generally not only does the spatial relationship of the sub-structures of the object relative to external devices (processing instruments, ultrasound devices, etc.) change, but also, if necessary, the deformation of the internal structures and / or organs of the object / patient changes.

[0028] In a further embodiment, it can be provided that the repositioning of the object is taken into account by updating the object model with the aid of a deformation model. As described above, the object model can be supplemented by the deformation model in order to take into account the current deformation (such as that of a patient's organ) for the path planning of the instrument and / or the positioning and orientation of the ultrasound device. In particular, for example, it can be inferred therefrom the surface deformation in the 3D object model registered to the surface model.

[0029] In another embodiment, the acquisition geometry of the ultrasound device can be taken into account in the automatic calculation of virtual ultrasound images. For the acquisition geometry, in addition to the position and orientation of the ultrasound device, it can also include, for example, the viewing angle for imaging. For example, if a previously known ultrasound device is used, it is advantageous for the virtual ultrasound image to adapt the acquisition geometry to the real ultrasound device. Thereby, the similarity between the virtual and real ultrasound images can be increased. The acquisition geometry can also include a separately selected plane for ultrasound image detection relative to an instrument (such as a cannula or a needle).

[0030] In a special embodiment, during the automatic calculation of virtual ultrasound images, optimization regarding different ultrasound image planes is performed. For example, so-called in-plane ultrasound images or out-of-plane images can be detected. In the in-plane technique, for example, a cannula or a needle extends in the acoustic plane. In the out-of-plane technique, the cannula or the needle intersects the acoustic plane. In the latter case, for example, the transducer and the needle are at an angle of, for example, 90 degrees to each other. If necessary, the user can also participate in the optimization of the virtual ultrasound image with respect to the corresponding plane and determine the plane through a suitable interface.

[0031] In another embodiment, it can be provided that user interaction is considered for repositioning and / or reorienting the ultrasound device. Thus, for example, it is advantageous to provide an interface for so-called fine-tuning, which enables the user to at least partially reposition and / or reorient the ultrasound device. Thus, for example, for fine-tuning, repositioning and / or reorienting can also be performed manually or semi-automatically.

[0032] In another embodiment, the deviation between the virtual ultrasound image and the real ultrasound image is determined only by a unique position marker in two ultrasound images for repositioning and / or reorienting the ultrasound device. For example, a unique blood vessel branch is used as a position marker to identify the deviation between the real and virtual ultrasound images and register the two images accordingly. In the case of fine registration, multiple position markers can be observed.

[0033] Thus, a first rough repositioning can be performed based on a single position marker, while additional fine positioning is performed through multiple position markers.

[0034] In the process of optimizing the position or orientation of the ultrasound device, in an embodiment, it can be provided that the step of detecting the real ultrasound image on the one hand and the step of repositioning and / or reorienting the ultrasound device on the other hand are alternately repeated multiple times in order to minimize the difference between the virtual and real ultrasound images. Thus, in the optimization loop, an ultrasound recording can be performed after each repositioning and / or reorienting in order to compare it again with the virtual ultrasound image.

[0035] In a particularly preferred embodiment, the path for the instrument is mechanically determined by the positioning and orientation of the ultrasound device. For example, the ultrasound device has guiding elements (such as guide rails) on its housing. By positioning and orienting the ultrasound device, the path for the instrument is also positioned and oriented thereby. Thus, the path for the instrument can be determined simultaneously by the ultrasound device in a simple manner. Alternatively or additionally, as described above, the position and orientation of the instrument / needle can also be monitored and controlled by a 3D camera or a stereoscopic camera, and the surface data of the object is also detected by the 3D camera or the stereoscopic camera.

[0036] The above technical problem is also solved according to the invention by a device, which comprises

[0037] - a manually guided modality device,

[0038] - a first detection device for loading (and optionally detecting) 3D data (MR, CT) of the object and for pre-determining a target region (such as a lesion) in the object model,

[0039] - a second detection device for detecting the surface data of the object,

[0040] - a computing device, which is designed to

[0041] * automatically generate an object model of the object from the 3D data,

[0042] * automatically generate a surface model of the object from the surface data,

[0043] * register the surface model relative to the object model,

[0044] * automatically calculate the starting position and starting orientation of the manually guided modality device on the object model based on the target region, and

[0045] * automatically calculate a target image of a sub-region (ROI) of the object based on the object model, the target region, and the starting position and starting orientation of the manually guided modality device, where

[0046] - the true ultrasound image of the sub-region of the object can be detected by the ultrasound device in the starting position and starting orientation by the manually guided modality device, and

[0047] - the device has a display device, which is configured to visualize the deviation between the target image and the true ultrasound image.

[0048] Based on the deviation between the target image and the true ultrasound image, the manually guided modality device can be (automatically / manually) repositioned and / or reoriented. Here, the positioning and orientation of the manually guided modality device means that at least a part (such as the ultrasound probe) of the manually guided modality device is positioned and / or oriented.

[0049] An ultrasound device is an imaging device based on ultrasound. It may have an ultrasound probe, which is typically placed on the surface of an object and / or patient for ultrasound recording.

[0050] The first detection device is at least configured to load 3D data of the object. For example, these 3D data can be provided by a storage unit. In particular, pre-intervention data can be provided for the device in this way. If necessary, the first detection device may also have an MR or CT scanner, whereby the 3D data is directly recorded and provided. In addition, the first detection device is also configured to predetermine a target region in the object model. This can be done by loading the target data again. Alternatively, the target region can be predetermined by recording user interaction through a corresponding interface and converting the target region coordinates. For example, the user can mark a lesion in the three-dimensional object model, thereby determining the target region.

[0051] The second detection device for detecting surface data may include, for example, a stereo camera for optically detecting the surface of the object.

[0052] The computing device for generating the object model and the surface model, for registering the two models, for calculating the starting position and starting orientation of the ultrasound device, and for calculating the target image may have a computer and / or a processor with storage elements. In certain cases, however, the computing device may also take over the control tasks for controlling the ultrasound device and the detection device.

[0053] The positioning device of the device may have positioning means for automatic, semi-automatic or manual positioning and / or orientation. Thus, the ultrasound device or a part thereof can be positioned and aligned with respect to the target region.

[0054] The same advantages and variant possibilities as described above for the method according to the invention of course apply to the device according to the invention, and vice versa. Thus, method features can be regarded as corresponding functional features of the device.

[0055] In an embodiment, the ultrasound device has a needle guiding device for guiding an ablation needle or cannula. The needle guiding device can be designed as a mechanical guiding device, which, for example, enables degrees of freedom of movement for the ablation needle or cannula. If necessary, the corresponding instrument (such as an ablation needle or cannula) is automatically moved in the needle guiding device by a propulsion device. Preferably, the needle guiding device is firmly fixed on the ultrasound device and in particular on the housing of the ultrasound device. This has the advantage that by aligning and / or positioning the ultrasound device, the needle guiding device is also automatically positioned and aligned.

[0056] In another embodiment, the device may further have a planning device for planning a path of an instrument into a sub-region of an object according to an object model; and a navigation device for navigating the instrument along the planned path based on real ultrasonic images recorded after repositioning and / or reorienting of the ultrasonic device. The planning device and / or the navigation device may be based on a calculator and / or a computer. The planning device may have a corresponding HMI interface. For example, the desired position and the actual position of the ultrasonic probe and / or the instrument may be displayed through the HMI interface to give an indication to the user about the change of the actual position. The navigation device may have corresponding control elements for controlling a separate driving device for the ultrasonic device or the ultrasonic device itself.

[0057] For application scenarios or application situations that may occur in the method and are not explicitly described herein, it may be stipulated that an error report and / or a request for inputting user feedback and / or setting standard settings and / or a predetermined initial state are output according to the method.

[0058] Regardless of the grammatical gender of a specific term, persons with male and female gender identities are included. Description of the Drawings

[0059] The present invention will now be explained in more detail with reference to the accompanying drawings, in which:

[0060] Figure 1 A schematic diagram showing an embodiment of a device according to the present invention;

[0061] Figure 2 A schematic flow chart showing an embodiment of a method according to the present invention; and

[0062] Figure 3 Showing a specific design of method details in a schematic flow. Detailed Description of the Invention

[0063] The embodiments described in more detail below are preferred embodiments of the present invention.

[0064] Figure 1 The device shown in the figure can be understood as a system to assist the user in positioning an ultrasonic probe, so that the registration process and possible needle positioning can be easier. For example, the device is applied to a system that can be used to treat a liver tumor 1 in the liver 2 of a patient 3. Here, the patient 3 represents an object for ultrasonic examination and / or treatment. In this embodiment, the ultrasonic device 4 has an ultrasonic probe 5, an evaluation and control unit 6, and optionally a positioning device 7. Preferably, the ultrasonic probe 5 is manually guided and its position and orientation are monitored by a second detection device (such as a stereo camera) when needed. Through the optional positioning device 7, the ultrasonic probe 5 can be positioned and oriented based on the evaluation and control unit 6, especially on the surface of the object and / or the patient 3.

[0065] The ultrasound device can be, for example, a 2D ultrasound system. For example, it can provide real-time B-mode images.

[0066] The ultrasound probe 5 can have a guiding device 8 through which, for example, an ablation needle 9 can be guided externally through the chest cavity and the liver 2 of the patient 3. Here, for example, it should be noted that the path of the ablation needle 9 does not intersect the rib cage 10.

[0067] In this case, the liver tumor 1 or a part thereof is the target area, which on the one hand should be reached by the ablation needle 9 and on the other hand should be imaged by the ultrasound device 4. In particular, it is advantageous here that the ultrasound device 4 can image the target area in real time.

[0068] Figure 1 The device also has a first detection device 11 for loading 3D data of the patient 3. For example, a 3D data set obtained before the intervention of the patient 3 can be used for the device. The 3D data set can, for example, come from an MR or CT system. In particular, the data set should represent the target area. The detection device 11 can have an interface to the Internet, a CD or DVD drive, or another data interface. But if necessary, the first detection device 11 is also part of an MR or CT system, etc. In this example, the 3D data set can come from an MRI examination of the liver, in which a segmented lesion for ablation is set.

[0069] Figure 1 The device shown also has a second detection device 12 for detecting surface data of the object and / or the patient 3. In this example, it can be a stereoscopic camera, by which the surface of the patient 3 can be scanned three-dimensionally. Thus, a virtual avatar (surface model) of the patient's surface can be formed and the internal position of the liver can be estimated together with the 3D data. Optionally, the stereoscopic camera can already detect the surface model of the patient during the pre-intervention image detection and provide it together with the object model through the detection device 11. If necessary, the surface model detected during the pre-intervention image detection can be used to calculate the registration with the surface model of the intervention and the virtual surface model. Optionally, visible markers can be arranged on the patient's body during the pre-intervention image detection. If necessary, the markers can be used again during the intervention.

[0070] The ultrasound probe 5 can also have one or more markers 13 through which the second detection device 12 can identify the position and / or orientation of the ultrasound probe 5. With this, the second detection device 12 can not only provide the surface data of the patient 3, but also provide the position and orientation data of the ultrasound probe 5 for controlling the device.

[0071] In addition Figure 1The device shown in has a computing device 14. The computing device 14 can be used, for example, to automatically generate a patient model from 3D data. In addition, it can be used to generate a surface model of patient 3 from the data of the second detection device 12. In addition, it can also be used to estimate a suitable position and orientation in order to position the ultrasound device 4 or its ultrasound probe 5 such that the target region (ROI) is visible. For this purpose, the computing device 14 can, for example, automatically calculate the starting position and starting orientation of the ultrasound device 4 on the object model or the surface model such that the target region is visible. Thus, the starting position and starting orientation represent the approximate pose of the ultrasound device 4.

[0072] The computing device 14 can furthermore be used to automatically calculate a virtual ultrasound image of a sub-region of the object based on the object model, the target region, and the (starting) position and (starting) orientation of the ultrasound device 4. Thus, for example, a virtual B-mode ultrasound image is simulated based on a 3D data set, where the anatomical structure of patient 3 is taken into account. From the virtual ultrasound image or the target image, the one in which the target region is most clearly visible can be found.

[0073] Thus, when using the computing device 14 to calculate the virtual ultrasound image, a real ultrasound image of a sub-region of patient 3 can be obtained by means of the ultrasound device 4. The deviation between the virtual ultrasound image or the target image and the real ultrasound image can be used to reposition or reorient the ultrasound device 4. The computing device 14 can also determine the deviation between the two ultrasound images. For this purpose, the computing device 14 uses, for example, unique or even multiple position markers in the two ultrasound images. When the deviation is minimal and / or the two ultrasound images coincide, the optimal position and orientation of the ultrasound device 4 can be obtained.

[0074] The device can also have a feedback device to assist the user in arranging the ultrasound sample. For example, the feedback device can be a monitor ( Figure 1 not shown in ), which displays the planned and current position of the ultrasound probe 5 and, if necessary, gives an indication of where it should be moved. Alternatively, a laser guidance system indicating the optimal position and optimal orientation can also be used.

[0075] Furthermore, a monitor 15 (the same or a different monitor) can be provided, which plays the real-time ultrasound image and the virtual ultrasound image and thereby enables the user to adjust the real ultrasound image until it coincides with the virtual image (target image). Optionally, the views can be displayed side by side or superimposed on the 3D data.

[0076] Figure 1 The device shown in for positioning and orienting the ultrasound device 4 can also optionally be used for positioning and / or orienting an instrument, such as an ablation needle 9. This means that the second detection device 12, the computing device 14, and the feedback device can be used for this purpose.

[0077] The exemplary system can be based on a 3D-MR / CT imaging system for pre-intervention imaging, an optical tracking system, and an ultrasound device for real-time imaging ("Live"). For this purpose Figure 2 A preferred application process is schematically shown in FIG. In the pre-intervention phase PI, 3D data (e.g., MR or CT data) of the region of interest is detected in step S1. The region of interest covers, for example, the abdominal body region, including the liver. Optionally, a surface model can also be detected, or markers can also be arranged on the body surface. In step S1, a patient model is also established from the 3D data set. For this purpose, the patient surface, the thoracic cavity, the object of interest, such as the liver surface, the hepatic blood vessels and bile ducts, and the target lesion are segmented by means of known automatic and user-guided segmentation methods.

[0078] In step S2, for example, the user 16 marks the target region and / or the target lesion in the 3D object model. Optionally, the treatment can also be modeled. Using specialized visualization and interaction tools, the user 16 can, for example, plan the optimal path for arranging the treatment device (needle, cannula, etc.) relative to the target lesion, the risk structure, and the hidden structure (e.g., the thoracic cavity). As an additional function, the repositioning of patient 3 can be modeled by means of specialized body deformation modeling and deformation techniques for better access to the treatment location.

[0079] As step S3, the extraction of the patient surface is again highlighted separately here. In the simplest case, the patient surface corresponds to the patient surface segmented in step S1. But here, if necessary, possible deformations are also considered according to step S2.

[0080] In the optical tracking phase OT, in step S4, the navigation system for surface-based registration is initialized. In particular, multi-level registration is required to roughly align the patient and the treatment model on the ultrasound system for image-based real-time treatment guidance. Patient 3 is positioned as described above in the planning step.

[0081] According to step S5, the navigation system (the second detection device 12) "scans" the surface of patient 3 and creates a real-time surface model (virtual avatar) of the patient's body including the markers arranged on the body.

[0082] In step S6, the body surface of the real-time image and the body surface of the pre-intervention patient model obtained according to step S3 are balanced or registered with each other. If necessary, the patient model is deformed by means of specialized deformation models. These models take as input the body surface (surface model) detected by the navigation system and the patient model of step S1, and further transmit the surface deformation to deeper body parts, for example, by means of 3D extrapolation and optical flow methods.

[0083] In a further step S7, based on the registration of the two opposed surfaces, the target lesion (target area) is calculated relative to the virtual avatar (surface model of the navigation system).

[0084] In the ultrasound phase US, it is first preferred to initialize the real-time ultrasound imaging system according to step S8. In a subsequent step S9, the ultrasound probe 5 is registered in the coordinate system of the navigation system. Thus, the ultrasound system and the optical system have the same coordinates.

[0085] In a subsequent step S10, the optimal probe position and probe orientation are calculated relative to the target area (lesion). A layer view corresponding to the optimal probe position and probe orientation is calculated from the 3D data set here. This view can be based on the type of 3D data set (CT or MR), or a virtual ultrasound image can optionally be generated therefrom. As input quantities for the calculation can be the treatment path, the pre-intervention 3D data set, the patient model including the lesion location, the detection geometry of the ultrasound system to be simulated and the (tilted) ultrasound imaging plane, for example a model of the optimal imaging geometry for in-plane and out-of-plane b-mode ultrasound images. The virtual ultrasound image can be displayed to the user 16.

[0086] In a further step S11, the probe position of the real ultrasound probe is optically tracked and / or detected, for example by means of a second detection device 12.

[0087] Furthermore, in step S12, a real real-time ultrasound image is provided to the user 16. If necessary, this can more precisely position the ultrasound probe 5, which is explained in more detail below in connection with Figure 3 In ultrasound image acquisition, for example, the user is asked to record ultrasound images as planned in the virtual ultrasound imaging session. The navigation system guides the user to the corresponding probe position and orientation (starting position and starting orientation). For this purpose, the navigation system (see Figure 2 and Figure 3 ) determines the target area relative to the virtual avatar (surface model) in step S7. Then the optimal starting position and starting orientation are determined for this purpose according to step S10. This starting position and starting orientation can be provided to the user 16. Furthermore, it can be provided to the real ultrasound device, the probe position and orientation of which are tracked according to step S11. Furthermore, the starting position and starting orientation are used in step S13 to generate a virtual ultrasound image based on the pre-intervention 3D data set. In a further step S14, the virtual ultrasound image and the real ultrasound image can be displayed.

[0088] In a post - processing step, the virtual ultrasound image can be compared with the real or live ultrasound image. This can be done by the user 16 or automatically. If necessary, the user 16 and / or the corresponding algorithm match the two ultrasound images with each other according to step S15. Based on this match, the registration between the virtual avatar and the target region is updated according to step S16. Using this updated registration, the re - positioning and / or re - orientation of the ultrasound device 4 can be performed. Thus, after the starting position (the first optimal position), the second optimal position and orientation relative to the target region can be calculated according to step S17. This second or refined position and orientation can be used in steps S11 and S13 and can be provided to the user 16. Thus, through user interaction, the registration between the real - time data stream and the patient and / or the processing model can be further refined. This refinement is performed, for example, in Figure 3 the loop shown, which can run repeatedly.

[0089] In another optional step, the navigation system can be used for real - time guidance of an instrument (needle, cannula, etc.). Specifically, the user can obtain real - time guidance regarding the position of the treatment device relative to the patient and / or the treatment model and the real - time ultrasound image stream with the aid of the navigation system.

[0090] For example, this allows the following workflow:

[0091] 1. Load a 3D imaging study report that defines the target lesion in the system.

[0092] 2. Position the patient on the intervention table and create a virtual avatar by the system.

[0093] 3. The system estimates the registration of the 3D data set with the virtual avatar and determines the target position / orientation of the ultrasound probe to visualize the position markers.

[0094] 4. The user positions the probe and identifies the first orientation point (position marker).

[0095] 5. The system adjusts the registration based on the actual measurement and suggests additional positions / orientations to visualize additional orientation points / features / lesions.

[0096] 6. After the registration is completed, the system suggests the position / orientation for needle placement and displays the ideal trajectory of the needle in the real - time image and the registered image.

[0097] 7. The insertion of the needle is assisted by a physical needle holder fixed to the ultrasound probe or can be guided by the navigation / tracking system.

[0098] 8. When the needle is optimally positioned in the first ultrasound pose, the system suggests a second position / orientation to confirm the needle placement in 3D.

Claims

1. A method for positioning and orienting a manually guided modality device (4), wherein - load the 3D data of the object (3), - automatically generating an object model of the object (3) from the 3D data, - pre-determining the target area in the object model, - Surface data of the detected object, - automatically generating a surface model of the object (3) from the surface data, - registering the surface model relative to the object model, - automatically calculating the starting position and the starting orientation of the manually guided modality device (4) on the object model according to the target area, - automatically calculating a target image of a subregion of the object (3) based on the object model, the target region and the manually guided starting position and starting orientation of the modality device (4), - detecting a real ultrasound image of a subregion of the object (3) by means of a manually guided modality device (4) in a starting position and a starting orientation, - Repositioning and / or reorienting the manually guided modality device (4) based on the deviation between the virtual ultrasound image and the real ultrasound image (automatic / manual).

2. The method according to claim 1, characterized in that The object (3) is a patient, and during the automatic generation of the object model, the patient's organs are automatically segmented.

3. The method according to any one of the preceding claims, characterized in that The target image is a computed ultrasound image, an MR image, or a CT image.

4. The method according to any one of the preceding claims, characterized in that In addition to the 3D data of the object (3), first surface data of the object obtained during the detection of the 3D data are loaded, a first surface model is generated from the first surface data, and The first surface model implicitly registered with the object model is used for registration of the surface model relative to the object model.

5. The method according to any one of the preceding claims, characterized in that A 3D camera is used to detect the surface data of the object, which is also used to detect the position and orientation of the manual guidance modality.

6. The method according to claim 5, characterized in that The position and orientation of the instrument is detected by a 3D camera, the instrument can optionally be guided on a manually guided modality.

7. The method according to any one of the preceding claims, characterized in that The path of the instrument (9) entering the sub-region of the object (3) is planned according to the object model and is used for the automatic calculation of the target image.

8. The method according to claim 7, characterized in that The planning of the path includes the repositioning of the object (3).

9. The method according to claim 8, characterized in that The repositioning of the object (3) is taken into account by updating the object model with the aid of the deformation model.

10. The method according to any one of the preceding claims, characterized in that The acquisition geometry of the manually guided modality device (4) is taken into account in the automatic calculation of the target image.

11. The method according to claim 10, characterized in that When automatically calculating the target image, optimization is performed with respect to different ultrasound image planes.

12. The method according to any one of the preceding claims, characterized in that User interaction is taken into account for repositioning and / or reorienting the manual guidance modality device (4).

13. The method according to any one of the preceding claims, characterized in that The deviation between the target image and the true ultrasound image is determined solely by unique position markers in the two ultrasound images for repositioning and / or reorienting the manually guided modality device (4).

14. The method according to claim 13, characterized in that Additional repositioning and / or reorientation of the manually guided modality device (4) is performed by at least one additional position marker in the two ultrasound images.

15. The method according to any one of the preceding claims, characterized in that The steps of detecting the real ultrasound image on the one hand and repositioning and / or reorienting the manual guidance modality device (4) on the other hand are repeated alternately a plurality of times in order to minimize the difference between the virtual ultrasound image and the real ultrasound image.

16. The method according to any one of the preceding claims, characterized in that The path for the instrument is mechanically determined by positioning and orienting the manually guided modality device (4).

17. The method according to any one of claims 4 to 16, characterized in that Before loading the 3D data of the object (3), the 3D data is detected by a modality device and the first surface data of the object is detected at the same time.

18. A device comprising - a manually guided modal device (4), - a first detection device (11) for loading 3D data of the object (3) and a target area predetermined in a model of the object, - a second detection device (12) for detecting surface data of the object (3), - a computing device (14) designed to * Automatically generate object models of objects (3) from 3D data, * Automatically generate a surface model of an object (3) from surface data, * registering the surface model relative to the object model, * automatically calculating a starting position and a starting orientation of a manually guided modality device (4) on the object model based on the target region, and * Automatically calculating a target image of a sub-region of the object (3) based on the object model, the target region and the starting position and starting orientation of the manually guided modality device (4), wherein: - a real ultrasound image of a sub-region of the object (3) can be detected by means of the manually guided modality device (4) in a starting position and a starting orientation, and The device has a display device which is designed to visualize the deviation between the target image and the actual ultrasound image.

19. The device according to claim 18, characterized in that The manually guided modality device is an ultrasound device or a single photon emission computed tomography device.

20. The device according to claim 18 or 19, characterized in that The manually guided modality device (4) has a needle guide device (8) for guiding an instrument, in particular an ablation needle (9) or a cannula.

21. The device according to claim 20, characterized in that The device has a planning device for planning a path for an instrument to enter a sub-region of an object (3) based on an object model; and a navigation device for navigating the instrument along the planned path based on real ultrasound images recorded after repositioning and / or reorientation of a manually guided modality device (4).

Images