Method for finding a set of corresponding points in images to be registered, image registration method, medical image registration system

By combining manual and automatic methods with a medical image registration system, and using depth map and geometric feature descriptor matching, the problem of identifying corresponding points in abdominal organ images was solved, achieving accurate registration between laparoscopic images and CT images, and improving the reliability and efficiency of registration.

CN115965665BActive Publication Date: 2026-04-14OLYMPUS CORPORATION(JP)
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-08-26
Publication Date
2026-04-14

AI Technical Summary

Technical Problem

In medical image registration, especially in the registration of abdominal organs such as the liver, existing technologies struggle to accurately identify corresponding points, leading to inaccurate transformation matrices and consequently inaccurate registration results. This is particularly true when there are multiple possible candidate points in textureless surface images, making accurate registration difficult.

Method used

By combining image-based automatic and manual registration methods, a medical image registration system is used to calculate depth maps of the reference and search regions, generate point clouds, and match reference points with candidate points using geometric feature descriptors to ensure reliable and accurate positioning of corresponding points. Depth values ​​are calculated using stereoscopic laparoscopy or deep learning networks to improve the accuracy of corresponding points.

Benefits of technology

It enables reliable and rapid localization of corresponding points in textureless surface images, improving the accuracy and efficiency of image registration, especially the spatial alignment effect when registering laparoscopic images with CT images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115965665B_ABST
    Figure CN115965665B_ABST
Patent Text Reader

Abstract

The invention relates to a method of finding a set of corresponding points in images to be registered, an image registration method, a medical image registration system. According to the method, an input unit (32) of the system (30) receives a first user input indicating a reference point (10) in an intraoperative image. A processing unit (34) sets a reference region (12) around the reference point (10) and converts image data points (14) in the reference region (12) into an intraoperative point cloud. The input unit (32) receives a second user input indicating a candidate point (20) in a preoperative image. The processing unit (34) sets a search region (22) around the reference point (20) and converts image data points (24) in the search region (22) into a preoperative point cloud. By comparing geometric feature descriptors of the image data points (14, 24), the processing unit (34) finds a target point (28) in the preoperative image corresponding to the reference point (10) and defines these points (10, 28) as a set of corresponding points.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to a method for finding a set of corresponding points in an image to be registered, an image registration method, a medical image registration system, and a software program product. Background Technology

[0002] Image registration is a technique for transforming different image datasets to a common coordinate system. For example, in medical imaging, image registration is used to align and overlay preoperative images (such as computed tomography (CT) or magnetic resonance (MR) images) onto intraoperative images (such as intraoperative laparoscopic video images). Image registration allows for the enrichment of intraoperative images with additional information extracted from preoperative images.

[0003] A transformation matrix is ​​needed to map preoperative images onto intraoperative images. To calculate the transformation matrix, multiple sets of corresponding points in the preoperative and intraoperative images must be determined. Each set of corresponding points includes a reference point in the intraoperative image and a target point in the preoperative image, where the target point corresponds to the reference point. There are many methods to find these sets of corresponding points. For example, optical or electromagnetic tracking techniques can be used to find the corresponding points in each image.

[0004] Another option is a purely image-based registration method. In this method, the algorithm identifies salient feature points in both images and uses these salient feature points as corresponding points for image registration. Salient features include, for example, edges, corners, anatomical landmarks, and similar features visible in both images.

[0005] However, if no significant features can be found in either image, it is impossible to accurately select corresponding points in this method because mutually exclusive correspondences cannot be established between image data points in the images. In other words, there may be more than one possible candidate point in the preoperative image that can represent the target point in the intraoperative image. Inaccurate identification of corresponding points leads to inaccurate determination of the transformation matrix, resulting in inaccurate registration results.

[0006] This problem is exacerbated in images of objects with smooth, textureless surfaces, such as images of abdominal organs like the liver. These images often include more than one possible candidate point, which hinders accurate registration. Summary of the Invention

[0007] The purpose of this invention is to provide a method for finding a set of corresponding points in an image to be registered, an image registration method, a medical image registration system, and a software program product that allows for reliable and accurate positioning of a set of corresponding points for registration.

[0008] This objective is achieved by a method that uses a medical image registration system to find a set of corresponding points in the images to be registered. This set of corresponding points includes reference points in intraoperative image data and target points in preoperative image data. The medical image registration system includes an input unit, a processing unit, and a storage unit. The storage unit stores the preoperative image data and the intraoperative image data.

[0009] The input unit receives a first user input, which indicates an image data point selected by the user in the intraoperative image data.

[0010] The processing unit

[0011] The image data points in the intraoperative image data are set as reference points in the intraoperative image data.

[0012] A reference region is set, which includes multiple image data points of the intraoperative image data, including the reference points.

[0013] A reference depth map is calculated by calculating depth values ​​and assigning them to each image data point in the reference region.

[0014] Intraoperative point cloud is generated based on the reference depth map.

[0015] The input unit receives a second user input, which indicates the image data point selected by the user in the preoperative image data.

[0016] The processing unit

[0017] The image data points selected by the user in the preoperative image data are set as candidate points, and these candidate points are estimated to correspond to reference points in the intraoperative image data.

[0018] Define a search region comprising multiple image data points, including the candidate points, in the preoperative image data.

[0019] A search depth map is calculated by calculating depth values ​​and assigning them to each image data point in the search area.

[0020] Preoperative point cloud is generated based on the search depth map.

[0021] A geometric feature descriptor is calculated and assigned to each image data point in the intraoperative point cloud and each image data point in the preoperative point cloud, wherein the geometric feature descriptor of the image data point indicates at least one geometric relationship between the image data point and at least one of its neighboring image data points.

[0022] The geometric feature descriptor of the reference point is compared with the geometric feature descriptors of at least two image data points in the preoperative point cloud.

[0023] The image data point whose geometric feature descriptor in the preoperative point cloud best matches the geometric feature descriptor of the reference point is set as the target point.

[0024] The reference points in the intraoperative image data and the target points in the preoperative image data are assigned as corresponding point groups.

[0025] Advantageously, this method combines image-based automatic registration with manual registration methods. The user selects image data points from intraoperative image data and preoperative image data data, assuming they correspond to each other. This coarse estimation is used to reduce the number of candidate points from which target points are selected, since only image data points within the search area are considered possible target points. The target point can be any image data point within the search area, or even a candidate point itself.

[0026] A reference point is a portion of a plurality of image data points within a reference region. Similarly, a candidate point is a portion of a plurality of image data points within a search region. According to one embodiment, the reference region is a continuous region surrounding and including the reference point. According to another embodiment, the search region includes a continuous region surrounding and including the candidate point. The image data points within the reference region specifically include all image data points within a first radius surrounding the reference point. The image data points within the search region specifically include all image data points within a second radius surrounding the candidate point. In particular, the first radius is equal to the second radius.

[0027] The depth map includes information about the distance of each image data point to the plane. Specifically, the reference depth map includes information indicating the distance of image data points in a reference region from the viewpoint or observation plane. The search depth map may include information indicating the distance of image data points in a search region from the viewpoint or observation plane. The reference depth map and / or the search depth map may include a depth value for each image data point.

[0028] Three-dimensional coordinates (especially Cartesian coordinates) can be assigned to each image data point in the intraoperative point cloud and / or preoperative point cloud.

[0029] Image data points, especially pixels and / or voxels.

[0030] Geometric feature descriptors indicate the geometric relationship between image data points and their neighboring image data points. They allow for precise differentiation of image data points that might otherwise appear very similar. Therefore, comparing the geometric feature descriptors of image data points in a reference point and a preoperative point cloud is an accurate and fast method for finding target points and assigning them to corresponding point groups.

[0031] In particular, the processing unit compares the geometric feature descriptor of the reference point with the geometric feature descriptors of all image data points in the preoperative point cloud.

[0032] Preferably, the geometric feature descriptor of at least one image data point in the intraoperative point cloud and / or preoperative point cloud indicates the surface normal of the image data point and / or the distance to at least one of its neighboring image data points and / or the direction to at least one of its neighboring image data points. In particular, the geometric feature descriptor of all image data points in the intraoperative point cloud and / or preoperative point cloud indicates the surface normal of the image data point and / or the distance to at least one of its neighboring image data points and / or the direction to at least one of its neighboring image data points.

[0033] The surface normal of an image data point, its distance to at least one of its neighboring image data points, and its direction to at least one of its neighboring image data points are geometric relationships that are well-suited for distinguishing image data points.

[0034] Preferably, the medical image registration system includes a laparoscope, wherein the intraoperative image data is recorded by the laparoscope (especially as intraoperative laparoscopic video data) and sent to a storage unit.

[0035] Applying this method to laparoscopic image data allows for precise and rapid registration of images captured by a laparoscopy with preoperative image data from a CT scanner on a magnetic resonance imaging (MRI) device. Intraoperative image data can be intraoperative laparoscopic video data, which allows for image registration from video data from the laparoscopy during the surgical procedure.

[0036] The laparoscopy is preferably a stereoscopic laparoscopy, wherein the intraoperative image data recorded by the stereoscopic laparoscopy is stereoscopic intraoperative image data, wherein the stereoscopic intraoperative image data indicates the depth value of each image data point of the intraoperative image data, and wherein the processing unit calculates a reference depth map by using the stereoscopic intraoperative image data to calculate the depth value of each image data point in the reference region.

[0037] Stereoscopic laparoscopy is a simple way to assign depth values ​​to image data points. This allows for the rapid calculation of a reference depth map using intraoperative stereoscopic image data. The intraoperative stereoscopic image data can be intraoperative stereoscopic video data.

[0038] Alternatively or concurrently, the processing unit preferably generates a reference depth map and / or a search depth map by utilizing a deep learning network for depth estimation, wherein the deep learning network for depth estimation uses a deep learning algorithm to compute the depth value for each image data point, and in particular, the deep learning network for depth estimation is stored on a storage unit.

[0039] Deep learning networks and algorithms used for depth estimation can be found, for example, in “C. Godard and O. MacAodha and M. Firman and GJ Brostow, Digging into Self-Supervised Monocular Depth Prediction. The International Conference on Computer Vision (ICCV), October 2019”. Such deep learning networks are capable of computing depth values ​​and assigning them to image data points in a two-dimensional image after training with appropriate image data.

[0040] Preferably, the processing unit calculates a geometric feature descriptor for each image data point in the intraoperative point cloud and each image data point in the preoperative point cloud using a feature descriptor program (especially a fast point feature histogram descriptor program), wherein, in particular, the feature descriptor program is stored on a storage unit.

[0041] Geometric feature descriptors represent the geometric relationships of a point relative to its neighboring points. A feature descriptor program is a program that can quickly and efficiently compute the geometric relationships at a point. An example of a 3D point cloud descriptor program is the Fast Point Feature Histogram (FPFH) program, known from “RBRusu, N. Blodow and M. Beetz, Fast Point Feature Histograms (FPFH) for 3D registration, 2009 IEEE International Conference on Robotics and Automation”. Other types of 3D point cloud descriptors include Fast Point Feature Histograms, Oriented Histogram (SHOT) descriptors, or Rotated Image Descriptors. An overview of different types of 3D point cloud descriptor programs can be found in “XFHan, JSJin, J.Xie, MJWang, W.Jiang, A comprehensive review of 3D pointcloud descriptors, arXiv:1802.02297v1 [cs.CV] 7 Feb 2018”.

[0042] Preferably, preoperative image data is recorded using a preoperative image capture device before surgery, and intraoperative image data is recorded using an intraoperative image capture device during surgery. The preoperative image capture device transmits the preoperative image data to a storage unit, and the intraoperative image capture device transmits the intraoperative image data to a storage unit.

[0043] Intraoperative image capture devices are, for example, endoscopes or laparoscopes, and preoperative image capture devices are, for example, CT scanners or magnetic resonance imaging (MRI) devices. In particular, medical image registration systems include intraoperative image capture devices and / or preoperative image capture devices.

[0044] This objective is also achieved by an image registration method that aligns preoperative and intraoperative images via a medical image registration system, the medical image registration system comprising an input unit, a processing unit, and a storage unit, wherein the medical image registration system allocates at least three different sets of corresponding points using a method according to any of the foregoing embodiments, wherein the processing unit calculates a transformation matrix based on the set of corresponding points, wherein the transformation matrix is ​​configured to map the preoperative image data onto the intraoperative image data when applied to the preoperative image data, wherein the processing unit spatially aligns the preoperative image data and the intraoperative image data by applying the transformation matrix to the preoperative image data.

[0045] The image registration method uses a set of corresponding points to reliably and accurately find multiple sets of corresponding points. These corresponding points are used to calculate the transformation matrix and register the preoperative image data with the intraoperative image data.

[0046] This objective is also achieved by a medical image registration system for aligning preoperative and intraoperative images, the medical image registration system comprising an input unit, a processing unit, and a storage unit storing preoperative and intraoperative image data, wherein the medical image registration system is designed and configured to perform the method according to any of the foregoing embodiments.

[0047] The same or similar advantages as those of the previously mentioned methods for finding a set of corresponding points and image registration methods are applicable to the medical image registration system.

[0048] This objective is also achieved by a software program product comprising a program code medium for a medical imaging system according to a previously described embodiment, the program code medium including a control program component executed in a processing unit of the medical imaging system, characterized in that the control program component is designed to perform a method according to any of the previously described embodiments.

[0049] The same or similar advantages as those mentioned above regarding medical image registration systems, methods for finding a set of corresponding points, and image registration methods are applicable to the software program product.

[0050] Other features of the invention will become apparent from the description and claims of embodiments thereof, as well as the included drawings. Embodiments of the invention may implement a single feature or a combination of several features. Attached Figure Description

[0051] The invention is described below based on exemplary embodiments, but this does not limit the overall intent of the invention, wherein all details of the invention not described in more detail in the text are explicitly disclosed with reference to the accompanying drawings. In the drawings:

[0052] Figure 1 A simplified schematic diagram of intraoperative images of the liver is shown;

[0053] Figure 2 A simplified schematic diagram of a preoperative image of the liver is shown;

[0054] Figure 3 A simplified schematic diagram of a medical image registration system is shown.

[0055] Figure 4 A simplified schematic diagram of intraoperative images of the liver with reference points and reference regions is shown; and

[0056] Figure 5 A simplified schematic diagram of a preoperative image of the liver with candidate points, search regions, and target points is shown.

[0057] In the accompanying drawings, elements or corresponding parts of the same or similar type have the same reference numerals to prevent the need to reintroduce the item.

[0058] List of reference numerals

[0059] 2 organs

[0060] 4 Surface

[0061] 10 reference points

[0062] 12 Reference Area

[0063] 14 image data points

[0064] 20 candidate points

[0065] 22 Search Area

[0066] 24 image data points

[0067] 28 Target Points

[0068] 30 Medical Image Registration System

[0069] 31 workstations

[0070] 32 Input Units

[0071] 34 processing units

[0072] 36 storage units

[0073] 38 Intraoperative image capture devices

[0074] 39 Preoperative image capture equipment Detailed Implementation

[0075] Figure 1 A schematic diagram of intraoperative images of organ 2, in this case, is shown. Intraoperative image data, such as that captured using a laparoscopy, can be part of intraoperative laparoscopic video data. Intraoperative images can be registered with preoperative images to spatially align the two images, allowing additional information to enrich the intraoperative images. This requires multiple sets of corresponding points in the two images, each set including a reference point 10 in the intraoperative image and a target point in the preoperative image, typically detected by an algorithm. For proper image alignment, the target point must precisely correspond to the reference point 10. Figure 1 In the diagram, reference point 10 is shown as a square.

[0076] The algorithm utilizes salient features in the image, such as angles or anatomical features, as corresponding points. However, if there are no visible salient features in the image, it is difficult to find the corresponding points. This is the case with the surface of the liver, which is largely textureless, making it difficult to identify target points, such as... Figure 2 As shown. Figure 2 A schematic diagram of a preoperative image of organ 2, such as a CT image, is shown. Due to the textureless surface 4 of organ 2, it is not possible to reliably identify image data points 24 in the preoperative image data corresponding to reference point 10 in the intraoperative image data, because there are multiple image data points 24, all exhibiting similar geometric features to reference point 10. In other words, there is no mutually exclusive correspondence between reference point 10 and individual image data points 24 in the preoperative image data.

[0077] To solve this problem, the medical image registration system 30 performs a method to find a set of corresponding points, such as... Figure 3 As shown. The medical image registration system 30 includes an input unit 32, a processing unit 34, and a storage unit 36, which are implemented, for example, in a workstation 31, a computer, or a medical device. The input unit 32 is, for example, a touchscreen that displays intraoperative and / or preoperative images and allows the user to select image data points on the screen as reference points 10 or candidate points. Alternatively, the input unit 32 may be a mouse, keyboard, or similar input device. Preoperative image data and intraoperative image data are stored in the storage unit 36, for example, a computer memory. The processing unit 34 is configured and designed to perform methods for finding a set of corresponding points and image registration methods. For example, the processing unit 34 is a CPU or similar device. The medical image registration system 30 may also include, for example, an intraoperative image capture device 38 for laparoscopy and a preoperative image capture device 39 for CT scanner.

[0078] Figure 4 Intraoperative images of organ 2 are shown, utilizing a method that finds a set of corresponding points using a medical image registration system 30. Figure 5 Preoperative images of organ 2 are shown, obtained using a method that finds a set of corresponding points using a medical image registration system 30. In the following text, by utilizing... Figure 4 and Figure 5 This section describes an exemplary implementation of a method for finding a set of corresponding points.

[0079] A user (e.g., a surgeon) selects a target point 28 in the intraoperative image data by inputting a first user input into input unit 32. Reference point 10 is located at... Figure 4 The middle part is represented by a slash. Input unit 32 receives the first user input, and processing unit 34 sets it as the target point 28. Then, processing unit 34 sets a reference region 12 including a plurality of image data points 14 containing reference point 10 in the intraoperative image data. For simplicity of understanding, in Figure 4 In the diagram, although all the squares, including the reference point 10 itself, are image data points 14, only one image data point 14 shown as a square has a reference symbol. According to one embodiment, the reference region 12 includes a continuous region surrounding and encompassing the reference point 10. For example, the reference region 12 includes each image data point 14 within a specified radius around the reference point 10. Figure 4 In the diagram, reference region 12 is depicted as a circle that includes, but is not centered on, reference point 10.

[0080] Then, processing unit 34 calculates a reference depth map of reference region 12 by calculating depth values ​​and assigning them to each image data point 14 in reference region 12. The depth values ​​are obtained, for example, through stereoscopic laparoscopy or a deep learning network for depth estimation. Based on the reference depth map, processing unit 32 generates an intraoperative point cloud. The intraoperative point cloud includes the coordinate information of each image data point 14 within the intraoperative point cloud.

[0081] The surgeon also selects candidate points 20 from the preoperative image data by inputting a second user input into the input unit 32. These candidate points 20 are... Figure 5 The reference point 20 is represented by a vertical line. The surgeon considers candidate point 20 to roughly correspond to reference point 10, although the target point 28, represented by a diagonal line, is actually the correct corresponding point. The precise selection of the correct candidate point 20 may be hindered, for example, by the fact that the surgeon must select image data point 24 on a two-dimensional screen depicting the three-dimensional organ 4. The input unit 32 receives a second user input, and the processing unit 34 sets it as candidate point 20.

[0082] The processing unit 34 sets up a search region 22 including multiple image data points 24 containing candidate points 20 from the preoperative image data. To improve... Figure 5The clarity is such that only one image data point 24, shown as a square, has a reference symbol. Nevertheless, Figure 5 All squares in the search area (including candidate point 20 and target point 28) are image data points 24. According to an implementation, the search area 22 includes a continuous region surrounding and encompassing the candidate point 20. For example, the search area 22 includes image data points 24 within a specified radius around the candidate point 20. Figure 5 In the diagram, the search area 22 is depicted as a circle that includes, but is not centered on, candidate point 20.

[0083] Processing unit 34 calculates a search depth map of search region 22 by calculating the depth value of each image data point in search region 22. The depth values ​​are obtained, for example, through a deep learning network for depth estimation. Based on the search depth map, processing unit 34 generates a preoperative point cloud. The preoperative point cloud includes the coordinate information of each image data point 24 within the preoperative point cloud.

[0084] Then, processing unit 34 calculates geometric feature descriptors and assigns them to each image data point 14 in the intraoperative point cloud and each image data point 24 in the preoperative point cloud. The geometric feature descriptors represent at least one geometric relationship between the image data points 14, 24 and at least one of their neighboring image data points 14, 24. They include at least one geometric feature, such as the surface normal of the image data point 14, 24, its distance to at least one of its neighboring image data points 14, 24, or its direction to at least one of its neighboring image data points 14, 24. Therefore, the geometric feature descriptors of image data points 14, 24 distinguish them from other image data points 14, 24, even if they appear similar when viewed in an isolated state.

[0085] Then, processing unit 34 compares the geometric feature descriptor of reference point 10 with the geometric feature descriptor of image data point 24 in the preoperative point cloud. This enables reliable identification of target point 28, which is image data point 24, and whose geometric feature descriptor best matches the geometric feature descriptor of reference point 10. Figure 5 In this method, the user's selection of candidate point 20 is corrected, and instead, the target point 28 that truly corresponds to reference point 10 is selected. By restricting the selection of target point 28 to the preoperative point cloud, image data points outside the search region 22 whose geometric feature descriptors match those of reference point 10 will not be accidentally selected as target point 28. This improves the accuracy of the method.

[0086] Finally, reference point 10 in the intraoperative image data and target point 28 in the preoperative image data were assigned to corresponding point groups.

[0087] This method can be repeated to obtain different sets of corresponding points for subsequent calculation of the transformation matrix. The transformation matrix enables spatial alignment of preoperative and intraoperative images. This achieves reliable and accurate image registration.

[0088] All mentioned features (including those obtained individually from the accompanying drawings, and various features disclosed in combination with other features) are considered individually and in combination to be essential to the invention. Embodiments according to the invention can be achieved by individual features or combinations of several features. Features combined with the wording "especially" or "particularly" will be considered preferred embodiments.

Claims

1. A method for finding a set of corresponding points in an image to be registered using a medical image registration system (30), the set of corresponding points including a reference point (10) in intraoperative image data and a target point (28) in preoperative image data, the medical image registration system (30) including an input unit (32), a processing unit (34) and a storage unit (36), the storage unit (36) storing the preoperative image data and the intraoperative image data, wherein, The input unit (32) receives a first user input, which indicates the image data point (14) selected by the user in the intraoperative image data. The processing unit (34) The image data point (14) in the intraoperative image data is set as the reference point (10). A reference region (12) is set, which includes multiple image data points (14) in the intraoperative image data that contain the reference point (10). A reference depth map is calculated by calculating a depth value and assigning that depth value to each image data point (14) in the reference region (12). Intraoperative point cloud is generated based on the reference depth map. The input unit (32) receives a second user input, which indicates the image data point (24) selected by the user in the preoperative image data. The processing unit (34) The image data point (24) selected by the user in the preoperative image data is set as a candidate point (20), and the candidate point (20) is estimated to correspond to the reference point (10) in the intraoperative image data. Define a search region (22), which includes multiple image data points (24) in the preoperative image data that contain the candidate point (20). A search depth map is calculated by calculating a depth value and assigning that depth value to each image data point (24) in the search region (22). Preoperative point cloud is generated based on the search depth map. A geometric feature descriptor is calculated and assigned to each image data point (14) in the intraoperative point cloud and each image data point (24) in the preoperative point cloud, wherein the geometric feature descriptor of the image data point (14, 24) indicates at least one geometric relationship between the image data point (14, 24) and at least one of its neighboring image data points (14, 24). The geometric feature descriptor of the reference point (10) is compared with the geometric feature descriptors of at least two image data points (24) in the preoperative point cloud. The image data point (24) in which the geometric feature descriptor of the preoperative point cloud best matches the geometric feature descriptor of the reference point (10) is set as the target point (28). The reference point (10) in the intraoperative image data and the target point (28) in the preoperative image data are assigned as the corresponding point group.

2. The method according to claim 1, wherein, The geometric feature descriptor of at least one image data point (14, 24) in the intraoperative point cloud and / or the preoperative point cloud indicates the surface normal of the image data point (14, 24) and / or the distance of the image data point (14, 24) to at least one of its neighboring image data points (14, 24) and / or the direction of the image data point (14, 24) to at least one of its neighboring image data points (14, 24).

3. The method according to claim 1, wherein, The medical image registration system includes a laparoscope, wherein the intraoperative image data is recorded by the laparoscope as intraoperative laparoscopic video data and transmitted to the storage unit (36).

4. The method according to claim 3, wherein, The laparoscope is a stereoscopic laparoscope, wherein the intraoperative image data recorded by the stereoscopic laparoscope is stereoscopic intraoperative image data, wherein the stereoscopic intraoperative image data represents the depth value of each image data point (14) in the intraoperative image data, wherein the processing unit (34) calculates the reference depth map by using the stereoscopic intraoperative image data to calculate the depth value of each image data point (14) in the reference region (12).

5. The method according to claim 3, wherein, The processing unit (34) generates the reference depth map and / or the search depth map by utilizing a deep learning network for depth estimation, wherein the deep learning network for depth estimation uses a deep learning algorithm to calculate the depth value of each image data point (14, 24), and wherein the deep learning network for depth estimation is stored on the storage unit (36).

6. The method according to claim 1, wherein, The processing unit (34) calculates a geometric feature descriptor for each image data point (14) in the intraoperative point cloud and for each image data point (24) in the preoperative point cloud using a feature descriptor program, wherein the feature descriptor program is stored on the storage unit (36).

7. The method according to claim 6, wherein, The processing unit (34) calculates a geometric feature descriptor for each image data point (14) in the intraoperative point cloud and for each image data point (24) in the preoperative point cloud using a fast point feature histogram descriptor program.

8. The method according to claim 1, wherein, The preoperative image data is recorded using a preoperative image capture device (39) before the operation and the intraoperative image data is recorded using an intraoperative image capture device (38) during the operation. The preoperative image capture device (39) transmits the preoperative image data to the storage unit (36) and the intraoperative image capture device (38) transmits the intraoperative image data to the storage unit (36).

9. An image registration method for aligning preoperative images with intraoperative images using a medical image registration system (30), wherein the medical image registration system (30) includes an input unit (32), a processing unit (34), and a storage unit (36). in, The medical image registration system (30) uses the method according to claim 1 to assign at least three different sets of corresponding points. The processing unit (34) calculates a transformation matrix based on the corresponding point group, wherein the transformation matrix is ​​configured to map the preoperative image data onto the intraoperative image data when applied to the preoperative image data. The processing unit (34) spatially aligns the preoperative image data with the intraoperative image data by applying the transformation matrix to the preoperative image data.

10. A medical image registration system (30) for aligning preoperative images with intraoperative images, the medical image registration system (30) comprising an input unit (32), a processing unit (34), and a storage unit (36), wherein the storage unit (36) stores preoperative image data and intraoperative image data, wherein, The medical image registration system (30) is designed and configured to perform the method according to claim 1.

11. A software program product comprising a program code medium for a medical image registration system (30) according to claim 10, the software program product comprising a control program component executed in a processing unit (34) of the medical image registration system (30), characterized in that, The control program component is designed to execute the method according to claim 1 when it is run in the processing unit (34).

Citation Information

Patent Citations

  • Information processing apparatus, information processing method and user equipment

    US20180253909A1

  • Generating three-dimensional geo-registered maps from image data

    US20200082611A1