Multi-view medical image registration method, device and equipment

Abstract

The present application discloses a registration method, device and equipment for multi-view medical images, which relates to the field of medical image technology, and provides a method for registering multi-view medical images using densely sampled image features in the effective area of ​​the projection, reducing the loss of 3D spatial information during the perspective projection process, and improving the registration efficiency. The method includes: obtaining a feature image of a reference image and a feature image of a floating image, determining the effective area of ​​the perspective projection according to the feature image of the reference image, and using the effective area of ​​the perspective projection to project the feature image of the floating image to obtain at least one two-dimensional projection image; registering by moving the floating image, and using at least one two-dimensional projection image during the registration process to calculate the similarity measure of the registration for the current spatial position of the floating image; if the similarity measure of the registration meets the preset conditions, the rigid transformation result of the registration is output according to the current spatial position of the floating image.

Description

Technical Field

[0001] The present application relates to the field of medical image technology, and in particular to a multi-view medical image registration method, device and equipment. Background Art

[0002] Image-guided minimally invasive vascular interventional surgery (MIIS) is gradually replacing traditional invasive surgical treatments due to its small incision, fast postoperative recovery, strong patient universality, and low risk of complications. It is commonly used to treat various types of cerebrovascular diseases, such as vascular stenosis or occlusion, aneurysms, or arteriovenous malformations.

[0003] At present, according to the requirements of real-time and clinical operability, clinicians obtain the patient's two-dimensional medical images in real time during minimally invasive interventional surgery, and directly guide the surgical instrument intervention in three-dimensional space based on this. This requires clinicians not only to have the professional ability to reconstruct the three-dimensional anatomical scene in real time, but also to have the professional skills of coordinated operation of images and interventional instruments. The difficulty of its implementation restricts the safety and effectiveness of minimally invasive interventional surgery. A solution has been proposed clinically, and the specific implementation steps are as follows: 1) Collect the patient's three-dimensional medical images before minimally invasive interventional surgery to obtain the three-dimensional information and anatomical information of the lesion target area; 2) Clinicians can plan treatment plans through three-dimensional medical images; 3) With the help of computer algorithms, the preoperative three-dimensional information or treatment plan is automatically and accurately approved to the intraoperative two-dimensional medical image to compensate for the spatial information lost in the two-dimensional medical image. It can be seen that computer-based 3D-2D multimodal medical image registration is one of the core technologies of clinical solutions, which can directly affect the efficacy of minimally invasive interventional surgery. Here, 3D-2D multimodal medical image registration can be divided into two categories according to the subordinate relationship of the registered images: one is to treat the preoperative image as a floating image, reduce the dimension through perspective projection geometry, and establish a similarity measure between the preoperative 3D image and the intraoperative 2D image, which is called the forward projection strategy. The other is to treat the intraoperative 2D image as a floating image, increase the dimension through perspective projection geometry, and establish a similarity measure between the intraoperative 2D image and the previous 3D image, which is called the reverse projection. In addition, 3D-2D multimodal medical image registration can be divided into 3D-2D single-view medical image registration, that is, a 3D preoperative image is registered with a 2D intraoperative image; and 3D-2D multi-view medical image registration, that is, a 3D preoperative image is registered with at least one 2D intraoperative image, according to the number of 2D images involved in the registration.

[0004] In the related art, the methods for 3D-2D multi-view medical image registration mainly include registration based on image grayscale information and registration based on image feature information. However, the registration method based on image grayscale information has poor registration efficiency because it needs to process the global information of floating images and reference images, but the accuracy and robustness of the registration results are relatively high; the registration method based on image features performs discrete sampling of feature items of the image and only uses a small number of feature items to perform registration, which significantly reduces the amount of calculation and improves the registration efficiency, but also aggravates the loss of 3D spatial information during perspective projection, resulting in reduced accuracy and sensitivity of registration. Summary of the invention

[0005] In view of this, the present application provides a multi-view medical image registration method, device and equipment, the main purpose of which is to solve the problem that the registration based on image grayscale information and the registration based on image feature information in the prior art are insufficient in registration accuracy and registration efficiency.

[0006] According to a first aspect of the present application, a multi-view medical image registration method is provided, the method comprising:

[0007] Acquire a characteristic image of a reference image and a characteristic image of a floating image, wherein the reference image is at least one two-dimensional medical image acquired during surgery, and the floating image is a three-dimensional medical image acquired before surgery;

[0008] Determine an effective area of ​​perspective projection according to a feature image of the reference image, and project the feature image of the floating image using the effective area of ​​perspective projection to obtain at least one two-dimensional projection image;

[0009] Performing registration by moving the floating image, and calculating a registration similarity measure for a current spatial position of the floating image using the at least one two-dimensional projection image during the registration;

[0010] If the similarity measure of the registration meets a preset condition, the rigid transformation result of the registration is output according to the current spatial position of the floating image.

[0011] Furthermore, the reference image is two two-dimensional medical images acquired from different viewing directions during surgery, and the step of acquiring the characteristic image of the reference image and the characteristic image of the floating image specifically includes:

[0012] For the two two-dimensional medical images acquired from different viewing directions during the operation, a two-dimensional image matrix constructed respectively by combining a multi-scale enhancement strategy is used to determine a feature image of a reference image;

[0013] For the three-dimensional medical image acquired before the operation, a three-dimensional image matrix constructed in combination with a multi-scale enhancement strategy is used to determine a characteristic image of the floating image.

[0014] Furthermore, the two-dimensional medical images acquired from different viewing directions during the operation are combined with two-dimensional image matrices respectively constructed by multi-scale enhancement strategy to determine the characteristic image of the reference image, specifically including:

[0015] For the two two-dimensional medical images acquired from different viewing directions during the operation, two-dimensional image matrices are constructed respectively in combination with a multi-scale enhancement strategy;

[0016] Generate blood vessel enhancement responses of the two-dimensional image in each scale space using the eigenvalues ​​obtained by decomposing the two-dimensional image matrix;

[0017] Determine a binarized image in the two two-dimensional medical images according to the blood vessel enhancement response of the two-dimensional image in each scale space;

[0018] Performing distance transformation on the binarized images in the two two-dimensional medical images to obtain a feature image of the reference image.

[0019] Furthermore, determining the binarized image in the two two-dimensional medical images according to the blood vessel enhancement response of the two-dimensional image in each scale space specifically includes:

[0020] Selecting the maximum value corresponding to the blood vessel enhancement response of the two-dimensional image in each scale space to construct a blood vessel structure enhancement image of the two-dimensional image;

[0021] After the vascular structure enhanced image of the two-dimensional image is segmented using a threshold segmentation algorithm, the maximum connected domain in the vascular structure enhanced image of the segmented two-dimensional image is selected to obtain a binary image in the two two-dimensional medical images.

[0022] Furthermore, the three-dimensional medical image acquired before the operation is combined with the three-dimensional image matrix constructed by the multi-scale enhancement strategy to determine the characteristic image of the floating image, specifically including:

[0023] For the three-dimensional medical image acquired before the operation, a three-dimensional image matrix is ​​constructed by combining a multi-scale enhancement strategy;

[0024] Generate blood vessel enhancement responses of the three-dimensional image in each scale space by using the eigenvalues ​​obtained by decomposing the three-dimensional image matrix;

[0025] The maximum value corresponding to the blood vessel enhancement response of the three-dimensional image in each scale space is selected to construct a characteristic image of the floating image.

[0026] Further, determining the effective area of ​​perspective projection according to the feature image of the reference image, and using the effective area of ​​perspective projection to project the feature image of the floating image to obtain at least one two-dimensional projection image specifically includes:

[0027] Determining a valid area of ​​perspective projection by normalizing a feature image of the reference image;

[0028] By utilizing the effective area of ​​the perspective projection, perspective projection is performed on the characteristic images of the floating image at least one preset viewing angle corresponding to the acquisition of the reference image to obtain at least one two-dimensional projection image.

[0029] Furthermore, determining the effective area of ​​the perspective projection by normalizing the feature image of the reference image specifically includes:

[0030] By normalizing the feature image of the reference image, the feature image of the reference image is divided according to grayscale intensity;

[0031] An image portion whose grayscale intensity in the feature image of the reference image is less than a preset threshold is selected as a valid area for perspective projection.

[0032] Further, the effective area of ​​the perspective projection is used to perform perspective projection on the feature images of the floating image at least one preset viewing angle corresponding to the acquisition of the reference image to obtain at least one two-dimensional projection image, which specifically includes:

[0033] Determine a source point and multiple end points of a perspective projection ray in a device coordinate system, wherein the source point is the spatial coordinate of the ray source, and the multiple end points are the spatial coordinates of each voxel in a valid area of ​​the perspective projection;

[0034] Connecting the source point of the perspective projection ray to multiple end points respectively to form multiple perspective projection rays in the reference image, and traversing the projection process so that the multiple perspective projection rays pass through the voxels of the floating image;

[0035] The maximum grayscale values ​​of the voxels of the plurality of perspective projection rays passing through the floating image are selected as projection grayscale values ​​to construct at least one two-dimensional projection image.

[0036] Furthermore, the step of calculating a registration similarity measure for a current spatial position of the floating image using the at least one two-dimensional projection image specifically includes:

[0037] In a valid area of ​​the perspective projection, calculating a similarity measure for registration in different view directions using the at least one two-dimensional projection image;

[0038] The average value corresponding to the similarity measure of the registration in different view directions is selected as the similarity measure of the registration corresponding to the current spatial position of the floating image.

[0039] According to a second aspect of the present application, a multi-view medical image registration device is provided, the device comprising:

[0040] An acquisition unit, configured to acquire a feature image of a reference image and a feature image of a floating image, wherein the reference image is at least one two-dimensional medical image acquired during surgery, and the floating image is a three-dimensional medical image acquired before surgery;

[0041] A projection unit, configured to determine an effective area of ​​perspective projection according to a characteristic image of the reference image, and project the characteristic image of the floating image using the effective area of ​​perspective projection to obtain at least one two-dimensional projection image;

[0042] a calculation unit, configured to implement the registration by moving the floating image, and to calculate a similarity measure of the registration for a current spatial position of the floating image using the at least one two-dimensional projection image during the registration process;

[0043] The output unit is used for outputting the rigid transformation result of the registration according to the current spatial position of the floating image if the similarity measure of the registration meets the preset condition.

[0044] Furthermore, the reference image is two two-dimensional medical images acquired from different viewing directions during surgery, and the acquisition unit includes:

[0045] A first determination module is used to determine a feature image of a reference image by combining two-dimensional image matrices respectively constructed by a multi-scale enhancement strategy with respect to two two-dimensional medical images acquired from different viewing directions during the operation;

[0046] The second determination module is used to determine the characteristic image of the floating image based on the three-dimensional medical image acquired before the operation and the three-dimensional image matrix constructed by the multi-scale enhancement strategy.

[0047] Furthermore, the first determination module is specifically used to construct two-dimensional image matrices respectively for the two two-dimensional medical images acquired from different viewing directions during the operation in combination with a multi-scale enhancement strategy;

[0048] The first determination module is further configured to generate a blood vessel enhancement response of the two-dimensional image in each scale space by using the eigenvalues ​​obtained by decomposing the two-dimensional image matrix;

[0049] The first determination module is further configured to determine a binarized image in the two two-dimensional medical images according to a vascular enhancement response of the two-dimensional image in each scale space;

[0050] The first determination module is further configured to perform distance transformation on the binarized images in the two two-dimensional medical images to obtain a feature image of the reference image.

[0051] Furthermore, the first determination module is further configured to select a maximum value corresponding to the blood vessel enhancement response of the two-dimensional image in each scale space to construct a blood vessel structure enhancement image of the two-dimensional image;

[0052] The first determination module is specifically used to segment the vascular structure enhanced image of the two-dimensional image using a threshold segmentation algorithm, and then select the largest connected domain in the vascular structure enhanced image of the segmented two-dimensional image to obtain the binary image of the two two-dimensional medical images.

[0053] Furthermore, the second determination module is specifically used to construct a three-dimensional image matrix in combination with a multi-scale enhancement strategy for the three-dimensional medical image acquired before the operation;

[0054] The second determination module is further configured to generate a blood vessel enhancement response of the three-dimensional image in each scale space by using the eigenvalues ​​obtained by decomposing the three-dimensional image matrix;

[0055] The second determination module is further configured to select a maximum value corresponding to the blood vessel enhancement response of the three-dimensional image in each scale space to construct a characteristic image of the floating image.

[0056] Furthermore, the projection unit comprises:

[0057] A third determination module is used to determine the effective area of ​​the perspective projection by normalizing the feature image of the reference image;

[0058] The projection module is used to perform perspective projection on the characteristic images of the floating image respectively under at least one preset viewing angle corresponding to the acquisition of the reference image using the effective area of ​​the perspective projection, so as to obtain at least one two-dimensional projection image.

[0059] Furthermore, the third determination module includes:

[0060] A division submodule, used for dividing the feature image of the reference image according to grayscale intensity by normalizing the feature image of the reference image;

[0061] The selection submodule is used to select an image portion whose grayscale intensity is less than a preset threshold in the feature image of the reference image as a valid area for perspective projection.

[0062] Furthermore, the projection module comprises:

[0063] A determination submodule, used to determine a source point and multiple end points of a perspective projection ray in a device coordinate system, wherein the source point is the spatial coordinate of the ray source, and the multiple end points are the spatial coordinates of each voxel in a valid area of ​​the perspective projection;

[0064] A projection submodule, used for connecting the source point of the perspective projection ray with a plurality of end points respectively to form a plurality of perspective projection rays in the reference image, and traversing the voxels of the floating image through which the plurality of perspective projection rays pass during the projection process;

[0065] The construction submodule is used to select the maximum grayscale value of the voxels of the plurality of perspective projection rays passing through the floating image as the projection grayscale value, and construct at least one two-dimensional projection image.

[0066] Furthermore, the calculation unit comprises:

[0067] A calculation module, configured to calculate, in a valid area of ​​the perspective projection, a similarity measure of registration in different view directions using the at least one two-dimensional projection image;

[0068] The selection module is used to select the mean value corresponding to the similarity measure of the registration in different view directions as the similarity measure of the registration corresponding to the floating image in the current spatial position.

[0069] According to a third aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned multi-view medical image registration method is implemented.

[0070] According to a fourth aspect of the present application, a multi-view medical image registration device is provided, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the multi-view medical image registration method when executing the program.

[0071] By means of the above technical scheme, the present application provides a multi-view medical image registration method, device and equipment, which obtains a feature image of a reference image and a feature image of a floating image, wherein the reference image is at least one two-dimensional medical image acquired during surgery, and the floating image is a three-dimensional medical image acquired before surgery, and determines the effective area of ​​perspective projection according to the feature image of the reference image, and uses the effective area of ​​perspective projection to project the feature image of the floating image to obtain at least one two-dimensional projection image, and performs registration by moving the floating image, and uses at least one two-dimensional projection image during the registration process to calculate a similarity measure of the registration for the current spatial position of the floating image, and if the similarity measure of the registration meets a preset condition, outputs a rigid transformation result of the registration according to the current spatial position of the floating image. Compared with the existing registration methods based on image grayscale information and image feature information, the present application clarifies the effective area of ​​perspective projection by selecting the feature image of the reference image. On the one hand, only the projection image within the effective area is calculated during projection, which greatly reduces the number of voxels that need to be projected and improves the projection efficiency of the floating image. On the other hand, in the process of calculating the similarity measure using the two-dimensional projection image after the floating image is projected, it is only necessary to traverse the voxels within the effective area to obtain the feature information, which reduces the calculation amount and complexity of the similarity measure. By using densely sampled image features in the effective area of ​​the projection projection to perform multi-view medical image registration, the loss of 3D spatial information during perspective projection is reduced, ensuring that the feature items in the feature image are not seriously distorted and the pathological degree in the registration problem is not aggravated. At the same time, only features are densely collected in the effective area for iterative optimization, ensuring that the amount of calculation in the registration process meets clinical needs and improves the registration accuracy and efficiency.

[0072] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0074] Figure 1 A schematic diagram of a process of a multi-view medical image registration method provided in an embodiment of the present application is shown;

[0075] Figure 2 A schematic diagram of a process flow of another multi-view medical image registration method provided in an embodiment of the present application is shown;

[0076] Figure 3 A flowchart of a multi-view medical image registration process provided by an embodiment of the present application is shown;

[0077] Figure 4 A schematic diagram of the structure of a multi-view medical image registration device provided in an embodiment of the present application is shown;

[0078] Figure 5 A schematic structural diagram of another multi-view medical image registration device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0079] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0080] In the related art, the methods for 3D-2D multi-view medical image registration mainly include registration based on image grayscale information and registration based on image feature information. However, the registration method based on image grayscale information has poor registration efficiency because it needs to process the global information of floating images and reference images, but the accuracy and robustness of the registration results are relatively high; the registration method based on image features performs discrete sampling of feature items of the image and only uses a small number of feature items to perform registration, which significantly reduces the amount of calculation and improves the registration efficiency, but also aggravates the loss of 3D spatial information during perspective projection, resulting in reduced accuracy and sensitivity of registration.

[0081] In order to solve this problem, this embodiment provides a multi-view medical image registration method, such as Figure 1 As shown, the method can be applied to the server of the image registration platform, including the following steps:

[0082] 101. Obtain a feature image of a reference image and a feature image of a floating image.

[0083] Among them, the reference image is at least one two-dimensional medical image acquired during the operation, usually at least one two-dimensional medical image acquired using different viewing angles during the operation. In the case of using two two-dimensional medical images, the two two-dimensional medical images can be acquired at approximately orthogonal viewing angles, respectively, and the orthogonal viewing angles are the anterior-posterior view (AP) and the lateral view (LAP). The floating image is a three-dimensional medical image acquired before the operation. Here, the reference image can be a three-dimensional X-ray angiography image (3DXA), and the floating image can be a two-dimensional digital subtraction angiography image (DSA).

[0084] In the specific process of obtaining the characteristic image of the reference image, two 2D-DSA images are taken as examples for explanation. The multi-scale enhancement strategy can be combined to construct 2D image matrices respectively. The 2D image matrices can reflect the vascular characteristics in each 2D-DSA image. Each 2D image matrix is ​​further decomposed by eigenvalues, and the obtained eigenvalues ​​are used to generate vascular enhancement responses in each scale space, and the maximum value of the vascular enhancement response of the 2D image in each scale space is obtained to obtain the vascular structure enhanced images in the two 2D images. After the vascular enhancement images in the two 2D images are segmented by the segmentation algorithm, the maximum connected domain in the image is selected as the binarized vascular image in the 2D-DSA image, and the binarized vascular image in the 2D-DSA image is distance transformed to obtain the characteristic images of the two 2D-DSA images.

[0085] In the specific process of obtaining the characteristic image of the floating image, taking the 3D-DSA image as an example, the multi-scale enhancement strategy can be combined to construct a 3D image matrix, which can reflect the vascular characteristics in the 3D-DSA image. The 3D image matrix is ​​further decomposed by eigenvalues, and the obtained eigenvalues ​​are used to generate the vascular enhancement response of the 3D image in each scale space, and the maximum value of the vascular enhancement response of the 3D image in each scale space is obtained to obtain the characteristic image of the 3D-DSA image.

[0086] The executor of the embodiment of the present invention may be a multi-view medical image registration device, which may be specifically configured on the server side of the image registration platform. The registration process of the reference image and the floating image may be performed by acquiring the feature image of the reference image and the feature image of the floating image. The feature image here may represent the anatomical information and position information of the vascular structure with the help of grayscale values, so that the loss rate of the vascular spatial position information is reduced during the image registration process.

[0087] 102. Determine an effective area of ​​perspective projection according to a feature image of the reference image, and use the effective area of ​​perspective projection to project the feature image of the floating image to obtain at least one two-dimensional projection image.

[0088] Specifically, in the process of determining the effective area of ​​perspective projection based on the feature image of the reference image, the grayscale value of the image feature of the reference image can be squared and normalized to obtain the grayscale distribution in the image feature, and the image area with a grayscale value less than a preset threshold is selected as the effective area of ​​perspective projection based on the grayscale distribution.

[0089] The effective area of ​​the perspective projection is used as the projection basis of the floating image. Specifically, in the process of projecting the characteristic image of the floating image using the effective area of ​​the perspective projection, a plurality of perspective projection rays can be determined using the effective area of ​​the perspective projection, and the plurality of perspective projection rays are used to traverse the maximum value of the voxel grayscale of the characteristic image of the floating image passing through the projection process to form at least one two-dimensional projection image.

[0090] It can be understood that the feature image of the reference image is extracted using distance information, and has the anatomical information of the vascular structure required for alignment, while taking into account the position information of the vascular space required for alignment. The effective area of ​​the perspective projection determined by the feature image of the reference image has an order of magnitude smaller voxel pixels than the reference image, which greatly reduces the number of accelerations required for projection and improves projection efficiency.

[0091] 103. Perform registration by moving the floating image, and calculate a registration similarity measure for a current spatial position of the floating image using the at least one two-dimensional projection image during the registration process.

[0092] Since the spatial position of the floating image will change in real time during the projection of the characteristic image of the floating image, the at least one two-dimensional projection image obtained by the projection will also change accordingly. Here, in the real-time registration process of the mobile floating image, at least one two-dimensional projection image obtained at the current spatial position of the floating image can be used to calculate the similarity measure of the floating image at the current spatial position for registration. Here, the similarity measure is equivalent to a numerical value for judging the registration accuracy. The higher the value, the higher the registration accuracy.

[0093] Specifically, in the process of calculating the similarity measure for registration for the current spatial position of the floating image, a weighted average can be performed on the normalized grayscale image corresponding to the feature image of at least one two-dimensional projection image and the corresponding reference image within the effective area of ​​the perspective projection to obtain the similarity measure for registration. Here, each two-dimensional projection image and a grayscale image will form an image similarity measure. Taking into account the different viewing directions used by the two-dimensional projection image in the image acquisition stage, weighting coefficients can be set for different viewing directions, and then at least one similarity measure calculated in different viewing directions is weighted averaged according to the weighting coefficient. Of course, in order to save the amount of calculation, the average value of at least one similarity measure can be directly taken as the similarity measure for registration.

[0094] 104. If the similarity measure of the registration meets a preset condition, outputting a rigid transformation result of the registration according to the current spatial position of the floating image.

[0095] It is understandable that the similarity measure for registration of floating images at different spatial positions is different. In order to find the optimal image registration effect, the preset condition for judging the optimal registration effect can be set according to the accuracy required for registration, and can also be set according to the value of the similarity measure.

[0096] Specifically, an optimization algorithm can be used to find a similarity measure that meets preset conditions, and then a rigid transformation result of the registration can be output according to the current spatial position of the floating image. The output form of the rigid transformation result can be a numerical representation or a matrix representation of the mobile position information of the floating image.

[0097] The multi-view medical image registration method provided in the embodiment of the present application obtains a feature image of a reference image and a feature image of a floating image, wherein the reference image is at least one two-dimensional medical image acquired during surgery, and the floating image is a three-dimensional medical image acquired before surgery, and determines an effective area of ​​perspective projection according to the feature image of the reference image, and uses the effective area of ​​perspective projection to project the feature image of the floating image to obtain at least one two-dimensional projection image, performs registration by moving the floating image, and uses the at least one two-dimensional projection image during the registration process to calculate a similarity measure of the registration for the current spatial position of the floating image, and if the similarity measure of the registration meets a preset condition, outputs a rigid transformation result of the registration according to the current spatial position of the floating image. Compared with the existing registration methods based on image grayscale information and image feature information, the present application clarifies the effective area of ​​perspective projection by selecting the feature image of the reference image. On the one hand, only the projection image within the effective area is calculated during projection, which greatly reduces the number of voxels that need to be projected and improves the projection efficiency of the floating image. On the other hand, in the process of calculating the similarity measure using the two-dimensional projection image after the floating image is projected, it is only necessary to traverse the voxels within the effective area to obtain the feature information, which reduces the calculation amount and complexity of the similarity measure. By using densely sampled image features in the effective area of ​​the projection projection to perform multi-view medical image registration, the loss of 3D spatial information during perspective projection is reduced, ensuring that the feature items in the feature image are not seriously distorted and the pathological degree in the registration problem is not aggravated. At the same time, only features are densely collected in the effective area for iterative optimization, ensuring that the amount of calculation in the registration process meets clinical needs and improves the registration accuracy and efficiency.

[0098] Further, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, this embodiment provides another multi-view medical image registration method, such as Figure 2 As shown, the method includes:

[0099] For two two-dimensional medical images collected from different viewing directions during surgery, the characteristic image of the reference image is determined by respectively constructing two-dimensional image matrices combined with a multi-scale enhancement strategy.

[0100] In this step, specifically, for two two-dimensional medical images collected from different viewing directions during the operation, two-dimensional image matrices can be constructed respectively in combination with a multi-scale enhancement strategy, and the eigenvalues ​​obtained by decomposing the two-dimensional image matrix can be used to generate the vascular enhancement response of the two-dimensional image in each scale space, and based on the vascular enhancement response of the two-dimensional image in each scale space, the binarized images in the two two-dimensional medical images are determined, and distance transformation is performed on the binarized images in the two two-dimensional medical images to obtain the characteristic image of the reference image.

[0101] Here, in the process of determining the binary images in the two two-dimensional medical images, the maximum value corresponding to the vascular enhancement response of the two-dimensional image in each scale space can be selected to construct a vascular structure enhanced image of the two-dimensional image. After further segmenting the vascular structure enhanced image of the two-dimensional image using a threshold segmentation algorithm, the maximum connected domain in the vascular structure enhanced image of the segmented two-dimensional image is selected to obtain the binary images in the two two-dimensional medical images.

[0102] In actual application, the reference image is two 2D-DSA images I collected from two orthogonal perspectives. DSA-AP and I DSA-LAT Feature extraction is performed separately, where I DSA-AP It is a two-dimensional medical image collected from the front and back perspectives during surgery. DSA-LAT The two-dimensional medical images collected from the lateral perspective before surgery are obtained, and the characteristic images I of the two 2D-DSA images are obtained. DTF-AP and I DTF-LAT , similarly, I DTF-AP The image features of two-dimensional medical images collected from the front and back perspectives during surgery are DTF-LAT The image features of the two-dimensional medical image are collected from the preoperative lateral view. Specifically, in the process of determining the feature image of the reference image by respectively constructing the two-dimensional image matrix with the multi-scale enhancement strategy, the two-dimensional image matrix here can be a two-dimensional Hessian matrix (2D-Hessian). First, the 2D-DSA image I DSA-AP and I DSA-LAT , combined with the multi-scale enhancement strategy, construct the 2D-Hessian matrix H(x,σ),σ min ≤σ≤σ max , where σ is the smoothing factor of the multi-scale space, σ min is the minimum radius of the vascular structure that needs to be enhanced, σ max In order to correspond to the maximum radius of the vascular structure that needs to be enhanced, the 2D-Hessian matrix is ​​further decomposed to obtain the eigenvalue eigH(x,σ)→λi ,i=1,2, where the eigenvalues ​​are sorted by λ1>λ2. Then, the 2D-Hessian matrix eigenvalues ​​are used to generate the vascular enhancement response υ at each scale space. k :[eigH(x,σ)],σ min ≤σ≤σ max , and find the maximum value of vascular enhancement response in each scale space sup = sup{υ k [eigH(x,σ)]:σ min ≤σ≤σ max}, get the vascular structure enhanced image I vesselness-AP and I vesselness-LAT , and then use the Otsu segmentation algorithm to segment the blood vessels and enhance the vascular structure image I vesselness-AP and I vesselness-LAT , and select the largest connected domain in the image to obtain the binary cerebrovascular image I in the 2D-DSA image SegVessel-AP and I SegVessel-LAT , and to I SegVessel-AP and I SegVessel-LAT Perform distance transform (DTF) to obtain the feature images I of the two 2D-DSA images. DTF-AP and I DTF-LAT .

[0103] For the three-dimensional medical image acquired before surgery, the characteristic image of the floating image is determined by combining the three-dimensional image matrix constructed by the multi-scale enhancement strategy.

[0104] In this step, specifically, for the three-dimensional medical images collected before surgery, a three-dimensional image matrix can be constructed in combination with a multi-scale enhancement strategy, and the eigenvalues ​​obtained by decomposing the three-dimensional image matrix can be used to generate the vascular enhancement response of the three-dimensional image in each scale space, and the maximum value corresponding to the vascular enhancement response of the three-dimensional image in each scale space is selected to construct a feature image of the floating image.

[0105] In actual application, the floating image is the 3D-DSA image I acquired before surgery. Volume To extract features, I Volume is the three-dimensional medical image collected before surgery, and the characteristic image I of the 3D-DSA image is obtained Vesselness-Volume Specifically, in the process of determining the characteristic image of the floating image by combining the three-dimensional image matrix constructed by the multi-scale enhancement strategy, the three-dimensional image matrix here can be a three-dimensional Hessian matrix (3D-Hessian). First, for the 3D-DSA image I Volume , combined with the multi-scale enhancement strategy, construct the 3D-Hessian matrix H(x,σ),σ min ≤σ≤σ max, where σ is the smoothing factor of the multi-scale space, σ min is the minimum radius of the vascular structure that needs to be enhanced, σ max is the maximum radius of the vascular structure that needs to be enhanced. Then the 3D-Hessian matrix is ​​decomposed by eigenvalue to obtain the eigenvalue eigH(x,σ)→λ i ,i=1,2,3, where the eigenvalues ​​are sorted as λ1>λ2>λ3. And using the eigenvalues ​​of the 3D-Hessian matrix, the vascular enhancement response υ in each scale space is generated k :[eigH(x,σ)],σ min ≤σ≤σ max , find the maximum value of vascular enhancement response in each scale space, υ sup = sup{υ k [eigH(x,σ)]:σ min ≤σ≤σ max}, get the characteristic image I of 3D-DSA image Vesselness-Volume .

[0106] 203. Determine a valid area of ​​perspective projection by normalizing a feature image of the reference image.

[0107] In this step, the feature image of the reference image can be normalized, divided according to grayscale intensity, and the image portion of the feature image of the reference image with grayscale intensity less than a preset threshold is selected as the effective area of ​​the perspective projection.

[0108] In practical applications, the feature images I of the two 2D-DSA images in step 201 can be DTF-AP and I DTF-LAT The gray value is squared and normalized to obtain image I NormDTF-AP and I NormDTF-LAT According to the set empirical threshold, the part with gray value less than the empirical threshold is selected as the effective area R of perspective projection AP and R LAT .

[0109] 204. Using the effective area of ​​the perspective projection, perform perspective projection on the feature images of the floating image at least one preset viewing angle corresponding to the acquisition of the reference image to obtain at least one two-dimensional projection image.

[0110] In this step, the source point and multiple end points of the perspective projection ray can be determined in the device coordinate system, where the source point is the spatial coordinate of the ray source, and the multiple end points are the spatial coordinates of each voxel in the effective area of ​​the perspective projection. The source point of the perspective projection ray is respectively connected to the multiple end points to form multiple perspective projection rays in the reference image, and the multiple perspective projection rays pass through the voxels of the floating image during the projection process. The maximum grayscale value of the voxels of the floating image passed by the multiple perspective projection rays is selected as the projection grayscale value to construct at least one two-dimensional projection image.

[0111] In practical applications, the spatial coordinates of the ray source can be clearly defined in the device coordinate system. This is the focus of the perspective projection ray and also the source point of all perspective projection rays. The effective area R of the perspective projection determined in step 203 is AP and R LAT , respectively aligned to the device coordinate system, using the effective area R AP and R LAT The spatial coordinates of each voxel in the floating image I are used as the end points of the perspective projection ray. Vesselness-Volume Execute the projection, according to the perspective projection ray connecting the source point and multiple end points, traverse the I through which the ray passes during the projection process Vesselness-Volume Voxel, I traversed by each perspective projection ray Vesselness-Volume The maximum value of the voxel grayscale is used as the projection grayscale value in the perspective projection result to obtain two 2D projection images I DRR-AP and I DRR-LAT .

[0112] 205. Perform registration by moving the floating image, and during the registration process, calculate a similarity measure of registration in different view directions in a valid area of ​​the perspective projection using the at least one two-dimensional projection image.

[0113] 206 . Select a mean value corresponding to the similarity measure of the registration in different view directions as the similarity measure of the registration corresponding to the floating image at the current spatial position.

[0114] Here, different viewing directions may include front and rear viewing angles and side viewing angles. In the specific process of calculating the similarity measure, for the effective area R of the perspective projection determined in step 203, AP and R LAT , and the two 2D projection images I obtained in step 204 DRR-AP and I DRR-LAT .

[0115] For the front and back viewing angles, in the effective area R AP According to I DRR-AP and I NormDTF-AP Calculate the similarity measure S in the forward and backward viewing directions AP; For the direction finding angle, in the effective area R LAT According to I DRR-LAT and I NormDTF-LAT Calculate the similarity measure S of the lateral viewing direction LAT , and the mean of the similarity measures of the front and rear view angles and the side view angles is used as the similarity measure S for the final registration vessel .

[0116] 207. If the similarity measure of the registration meets a preset condition, output a rigid transformation result of the registration according to the current spatial position of the floating image.

[0117] Specifically, when judging whether the similarity measure of the registration meets the preset conditions, the Brent-Powell optimization algorithm can be used to determine whether the current floating image I Volume The similarity measure S of the registration corresponding to the current spatial position vessel Is it optimal? If S vessel When the optimal value is reached, the registration is terminated and the floating image I Volume The current spatial position, output rigid transformation result T rigid , if S vessel The optimal result is not achieved. According to the iterative process of the Brent-Powell optimization algorithm, the floating image I is adjusted. Volume The current spatial position until the similarity measure S vessel The registration is terminated after reaching the optimum, or after the Brent-Powell optimization algorithm iteration cutoff condition is triggered, and the perceptual transformation result of the registration is output.

[0118] In practical application scenarios, the registration process of multi-view medical images can be as follows: Figure 3 As shown, the specific steps include:

[0119] For the 3D-DSA floating image, first, the 3D-DSA image I Volume , combined with the multi-scale enhancement strategy, construct the 3D-Hessian matrix:

[0120] Among them, σ is the smoothing factor of the multi-scale space, σ min Corresponding to the minimum radius of the cerebrovascular structure that needs to be enhanced, σ max Corresponding to the maximum radius of the cerebrovascular structure that needs to be enhanced. Perform eigenvalue decomposition on the 3D-Hessian matrix to obtain the eigenvalue eigH(x,σ)→λ i ,i=1,2,3, where the eigenvalues ​​are sorted as λ1>λ2>λ3;

[0121] Then, the 3D-Hessian matrix eigenvalues ​​are used to generate the vascular enhancement response at each scale space:

[0122]

[0123] in,

[0124] Further obtain the maximum value of vascular enhancement response in each scale space sup = sup{υ k [eigH(x,σ)]:σ min ≤σ≤σ max}, get the characteristic image I of 3D-DSA image Vesselness-Volume .

[0125] For the 2D-DSA reference image, first, the 2D-DSA image I DSA-AP and I DSA-LAT , combined with the multi-scale enhancement strategy, construct the 2D-Hessian matrix respectively:

[0126] Among them, σ is the smoothing factor of the multi-scale space, σ min Corresponding to the minimum radius of the cerebrovascular structure that needs to be enhanced, σ max Corresponding to the maximum radius of the cerebrovascular structure that needs to be enhanced. Perform eigenvalue decomposition on the 2D-Hessian matrix to obtain the eigenvalue eigH(x,σ)→λ i ,i=1,2, here the eigenvalues ​​are sorted according to λ1>λ2.

[0127] Then, the 2D-Hessian matrix eigenvalues ​​are used to generate σ in each scale space. min ≤σ≤σ max Vascular enhancement response:

[0128]

[0129] in,

[0130] Further, the maximum value of the vascular enhancement response in each scale space is obtained, υ sup = sup{υ k [eigH(x,σ)]:σ min ≤σ≤σ max}, get the enhanced image of cerebral vascular structure I vesselness-AP and I vesselness-LAT .

[0131] Then, the Otsu segmentation algorithm is used to segment the cerebrovascular structure enhanced image, and then the largest connected domain in the image is selected to obtain the binary cerebrovascular image I in the 2D-DSA image. vesselness-AP and I SegVessel-LAT , and the binary cerebrovascular image I SegVessel-AP and I SegVessel-LAT Perform distance transform (DTF) to obtain the feature images I of the two 2D-DSA images. DTF-AP and I DTF-LAT .

[0132] Further, the vascular enhancement response is used as the characteristic image (characteristic item) of the floating image, and the distance map is used as the characteristic image (characteristic item) of the reference image as the constrained projection area. DTF-AP and I DTF-LAT The gray value is squared and normalized to obtain image I NormDTF-AP and I NormDTF-LAT , and determine the effective area R of DRR projection based on the empirical threshold AP and R LAT , using the effective area R of DRR projection AP and R LAT Perform DRR projection to obtain two 2D projection images I DRR-AP and I DRR-LAT , and according to the two 2D projection images I DRR-AP and I DRR-LAT , calculate the similarity measure of the floating image at the current spatial position for cerebrovascular structure registration, and further use the Powell algorithm to determine whether the similarity measure is optimal. If so, output the rigid transformation result, otherwise, repeat the DRR projection process until the similarity measure reaches the optimal.

[0133] In summary, the multi-view medical image registration method proposed in this application has at least the following beneficial effects compared with the registration based on image grayscale information and the registration based on image feature information in the prior art:

[0134] 1) The vesselness vascular feature enhancement response is selected as the feature item of the 3D medical image, and the DTF distance information is selected as the feature item of the 2D medical image. The selection of the feature item here ensures that the anatomical information of the cerebral vessels required for registration is available, while also taking into account the spatial position information of the cerebral vessels required for registration, so as to improve the accuracy and effectiveness of 3D-2D multi-view medical image registration for cerebral vessels.

[0135] 2) A DRR projection method based on vascular features is proposed to make the vascular structure in the floating image more complete and with higher contrast, so as to improve the accuracy of 3D-2D multi-view medical image registration for cerebral vessels.

[0136] 3) A new similarity measure calculation method is proposed. The calculation amount of this similarity measure is low, but it can guarantee the accuracy requirements of 3D-2D multi-view medical image registration for cerebral vessels.

[0137] 4) A 3D-2D multi-view medical image registration method for cerebral blood vessels is proposed, which uses densely sampled feature information in the vascular feature area, that is, it ensures that the feature items are not seriously distorted and does not aggravate the pathological degree of the 3D-2D registration problem; at the same time, features are densely collected only in the effective area for the iterative optimization process to ensure that the computational complexity of the registration process meets clinical needs.

[0138] In addition, the related art also proposes that in order to register the cerebrovascular information in the preoperative 3D image to the intraoperative 2D image, a set of 3D CBCT images can be collected during the operation. First, 3D-3D rigid registration is performed based on the skull structure in the preoperative 3D image and the intraoperative 3D CBCT image. Secondly, the cerebrovascular structure is registered based on the relative spatial position information of the skull and the cerebrovascular structure in the 3D image. Finally, the registered preoperative 3D image is perspective projected. However, adding a set of 3DCBCT during the operation will increase the radiation dose to the patient. The image registration method in this application does not require the addition of 3D images during the operation, which is patient-friendly. In addition, in order to determine the relative spatial position information of the skull and the cerebrovascular structure in the 3D image, it is necessary to accurately segment and locate the skull and cerebrovascular respectively, which increases the complexity of the preprocessing link before registration and is not conducive to the real-time performance required for intravascular interventional treatment.

[0139] Further, as Figure 1-Figure 2 The specific implementation of the method, the embodiment of the present application provides a multi-view medical image registration device, such as Figure 4 As shown, the device includes: an acquisition unit 31, a projection unit 32, a calculation unit 33, and an output unit 34.

[0140] An acquisition unit 31 may be used to acquire a feature image of a reference image and a feature image of a floating image, wherein the reference image is at least one two-dimensional medical image acquired during surgery, and the floating image is a three-dimensional medical image acquired before surgery;

[0141] The projection unit 32 may be configured to determine an effective area of ​​perspective projection according to a characteristic image of the reference image, and project the characteristic image of the floating image using the effective area of ​​perspective projection to obtain at least one two-dimensional projection image;

[0142] The calculation unit 33 may be used to implement the registration by moving the floating image, and calculate the similarity measure of the registration for the current spatial position of the floating image using the at least one two-dimensional projection image during the registration process;

[0143] The output unit 34 may be configured to output a rigid transformation result of the registration according to the current spatial position of the floating image if the similarity measure of the registration satisfies a preset condition.

[0144] The multi-view medical image registration device provided by the embodiment of the present invention obtains a feature image of a reference image and a feature image of a floating image, wherein the reference image is at least one two-dimensional medical image acquired during surgery, and the floating image is a three-dimensional medical image acquired before surgery, and determines an effective area of ​​perspective projection according to the feature image of the reference image, and uses the effective area of ​​perspective projection to project the feature image of the floating image to obtain at least one two-dimensional projection image, performs registration by moving the floating image, and uses the at least one two-dimensional projection image during the registration process to calculate a similarity measure of the registration for the current spatial position of the floating image, and outputs a rigid transformation result of the registration according to the current spatial position of the floating image if the similarity measure of the registration meets a preset condition. Compared with the existing registration methods based on image grayscale information and image feature information, the present application clarifies the effective area of ​​perspective projection by selecting the feature image of the reference image. On the one hand, only the projection image within the effective area is calculated during projection, which greatly reduces the number of voxels that need to be projected and improves the projection efficiency of the floating image. On the other hand, in the process of calculating the similarity measure using the two-dimensional projection image after the floating image is projected, it is only necessary to traverse the voxels within the effective area to obtain the feature information, which reduces the calculation amount and complexity of the similarity measure. By using densely sampled image features in the effective area of ​​the projection projection to perform multi-view medical image registration, the loss of 3D spatial information during perspective projection is reduced, ensuring that the feature items in the feature image are not seriously distorted and the pathological degree in the registration problem is not aggravated. At the same time, only features are densely collected in the effective area for iterative optimization, ensuring that the amount of calculation in the registration process meets clinical needs and improves the registration accuracy and efficiency.

[0145] In specific application scenarios, such as Figure 5 As shown, the acquisition unit 31 includes:

[0146] The first determination module 311 may be used to determine the characteristic image of the reference image by combining the two-dimensional image matrices respectively constructed by the multi-scale enhancement strategy with respect to the two two-dimensional medical images acquired from different viewing directions during the operation;

[0147] The second determination module 312 may be used to determine the characteristic image of the floating image based on the three-dimensional medical image acquired before the operation and the three-dimensional image matrix constructed by the multi-scale enhancement strategy.

[0148] In a specific application scenario, the first determination module 311 can be specifically used to construct two-dimensional image matrices respectively for two two-dimensional medical images acquired from different viewing directions during the operation in combination with a multi-scale enhancement strategy;

[0149] The first determination module 311 may be further configured to generate a blood vessel enhancement response of the two-dimensional image in each scale space by using the eigenvalues ​​obtained by decomposing the two-dimensional image matrix;

[0150] The first determination module 311 may be specifically configured to determine a binarized image in the two two-dimensional medical images according to the blood vessel enhancement response of the two-dimensional image in each scale space;

[0151] The first determination module 311 may be specifically configured to perform distance transformation on the binarized images in the two two-dimensional medical images to obtain a feature image of the reference image.

[0152] In a specific application scenario, the first determination module 311 may be further configured to select a maximum value corresponding to the blood vessel enhancement response of the two-dimensional image in each scale space to construct a blood vessel structure enhancement image of the two-dimensional image;

[0153] The first determination module 311 can also be specifically used to segment the vascular structure enhanced image of the two-dimensional image using a threshold segmentation algorithm, and then select the maximum connected domain in the vascular structure enhanced image of the segmented two-dimensional image to obtain the binary image in the two two-dimensional medical images.

[0154] In a specific application scenario, the second determination module 312 may be specifically used to construct a three-dimensional image matrix for the three-dimensional medical image acquired before surgery in combination with a multi-scale enhancement strategy;

[0155] The second determination module 312 may be specifically configured to generate a blood vessel enhancement response of the three-dimensional image in each scale space by using the eigenvalues ​​obtained by decomposing the three-dimensional image matrix;

[0156] The second determination module 312 may be specifically configured to select a maximum value corresponding to the blood vessel enhancement response of the three-dimensional image in each scale space to construct a feature image of the floating image.

[0157] In specific application scenarios, such as Figure 5 As shown, the projection unit 32 includes:

[0158] The third determination module 321 may be used to determine a valid area of ​​the perspective projection by normalizing a feature image of the reference image;

[0159] The projection module 322 may be configured to utilize the effective area of ​​the perspective projection to perform perspective projection on the feature images of the floating image at least one preset viewing angle corresponding to the acquisition of the reference image, so as to obtain at least one two-dimensional projection image.

[0160] In specific application scenarios, such as Figure 5 As shown, the third determination module 321 includes:

[0161] The division submodule 3211 may be used to divide the feature image of the reference image according to grayscale intensity by normalizing the feature image of the reference image;

[0162] The selection submodule 3212 may be used to select an image portion whose grayscale intensity is less than a preset threshold in the feature image of the reference image as a valid area for perspective projection.

[0163] In specific application scenarios, such as Figure 5 As shown, the projection module 322 includes:

[0164] The determination submodule 3221 may be used to determine a source point and multiple end points of a perspective projection ray in a device coordinate system, wherein the source point is the spatial coordinate of the ray source, and the multiple end points are the spatial coordinates of each voxel in a valid area of ​​the perspective projection;

[0165] The projection submodule 3222 may be used to connect the source point of the perspective projection ray with multiple end points respectively to form multiple perspective projection rays in the reference image, and the multiple perspective projection rays pass through the voxels of the floating image during the traversal projection process;

[0166] The construction submodule 3223 may be used to select the maximum grayscale value of the voxels of the plurality of perspective projection rays passing through the floating image as the projection grayscale value, and construct at least one two-dimensional projection image.

[0167] In specific application scenarios, such as Figure 5 As shown, the calculation unit 33 includes:

[0168] A calculation module 331 may be used to calculate a similarity measure of registration in different view directions using the at least one two-dimensional projection image in a valid area of ​​the perspective projection;

[0169] The selection module 332 may be used to select the mean value corresponding to the similarity measure of the registration in different view directions as the similarity measure of the registration corresponding to the floating image at the current spatial position.

[0170] It should be noted that for other corresponding descriptions of the functional units involved in the multi-view medical image registration device applicable to the server side provided in this embodiment, please refer to Figure 1 and Figure 2 The corresponding description in will not be repeated here.

[0171] Based on the above Figure 1-Figure 2 The method shown in the embodiment of the present application accordingly provides a storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned Figure 1-Figure 2 The registration method of multi-view medical images shown;

[0172] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0173] Based on the above Figure 1-Figure 2 The method shown, and Figure 4-Figure 5 In order to achieve the above-mentioned purpose, the embodiment of the present application further provides a server-side entity device, which can be a computer, a server, or other network devices, etc. The entity device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1-Figure 2 The registration method for multi-view medical images is shown.

[0174] Optionally, the above-mentioned physical devices may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display screen (Display), an input unit such as a keyboard (Keyboard), etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.

[0175] Those skilled in the art will appreciate that the physical device structure for multi-view medical image registration provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different component arrangements.

[0176] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the physical device for the registration of the multi-view medical images, and supports the operation of the information processing program and other software and / or programs. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the information processing physical device.

[0177] Through the description of the above implementation methods, those skilled in the art can clearly understand that the present application can be implemented by means of software plus the necessary general hardware platform, or by hardware. By applying the technical solution of the present application, compared with the current existing methods, the present application uses densely sampled image features in the effective area of ​​the projection to perform multi-view medical image registration, thereby reducing the loss of 3D spatial information during the perspective projection process, ensuring that the feature items in the feature image are not seriously distorted and do not aggravate the pathological degree of the registration problem, and at the same time, only densely collect features in the effective area for iterative optimization, ensuring that the amount of calculation in the registration process meets clinical needs, and improving the registration accuracy and efficiency.

[0178] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.

[0179] The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of this application, but this application is not limited to them, and any changes that can be thought of by technicians in this field should fall within the scope of protection of this application.

Claims

1. A multi-view medical image registration method, characterized in that: include: Acquire a characteristic image of a reference image and a characteristic image of a floating image, wherein the reference image is at least one two-dimensional medical image acquired during surgery, and the floating image is a three-dimensional medical image acquired before surgery; Determine an effective area of ​​perspective projection according to the feature image of the reference image, use the effective area of ​​perspective projection to project the feature image of the floating image to obtain at least one two-dimensional projection image, specifically square the grayscale value of the image feature of the reference image and then normalize it to obtain the grayscale distribution in the image feature, and select the image area with a grayscale value less than a preset threshold as the effective area of ​​perspective projection according to the grayscale distribution; Performing registration by moving the floating image, and calculating a registration similarity measure for a current spatial position of the floating image using the at least one two-dimensional projection image during the registration; If the similarity measure of the registration meets a preset condition, the rigid transformation result of the registration is output according to the current spatial position of the floating image.

2. The method according to claim 1, characterized in that: The reference image is two two-dimensional medical images acquired from different viewing directions during surgery, and the step of acquiring the characteristic image of the reference image and the characteristic image of the floating image specifically includes: For the two two-dimensional medical images acquired from different viewing directions during the operation, a two-dimensional image matrix constructed respectively by combining a multi-scale enhancement strategy is used to determine a feature image of a reference image; For the three-dimensional medical image acquired before the operation, a three-dimensional image matrix constructed in combination with a multi-scale enhancement strategy is used to determine a characteristic image of the floating image.

3. The method according to claim 2, characterized in that The method of determining the characteristic image of the reference image by respectively constructing two-dimensional image matrices based on the two two-dimensional medical images acquired from different viewing directions during the operation in combination with a multi-scale enhancement strategy specifically includes: For the two two-dimensional medical images acquired from different viewing directions during the operation, two-dimensional image matrices are constructed respectively in combination with a multi-scale enhancement strategy; Generate blood vessel enhancement responses of the two-dimensional image in each scale space using the eigenvalues ​​obtained by decomposing the two-dimensional image matrix; Determine a binarized image in the two two-dimensional medical images according to the blood vessel enhancement response of the two-dimensional image in each scale space; Performing distance transformation on the binarized images in the two two-dimensional medical images to obtain a feature image of the reference image.

4. The method according to claim 3, characterized in that Determining the binarized image in the two two-dimensional medical images according to the blood vessel enhancement response of the two-dimensional image in each scale space specifically includes: Selecting the maximum value corresponding to the blood vessel enhancement response of the two-dimensional image in each scale space to construct a blood vessel structure enhancement image of the two-dimensional image; After the vascular structure enhanced image of the two-dimensional image is segmented using a threshold segmentation algorithm, the maximum connected domain in the vascular structure enhanced image of the segmented two-dimensional image is selected to obtain a binary image in the two two-dimensional medical images.

5. The method according to claim 2, characterized in that: The three-dimensional medical image acquired before the operation is determined by combining a three-dimensional image matrix constructed with a multi-scale enhancement strategy with a characteristic image of the floating image, specifically including: For the three-dimensional medical image acquired before the operation, a three-dimensional image matrix is ​​constructed by combining a multi-scale enhancement strategy; Generate blood vessel enhancement responses of the three-dimensional image in each scale space by using the eigenvalues ​​obtained by decomposing the three-dimensional image matrix; The maximum value corresponding to the blood vessel enhancement response of the three-dimensional image in each scale space is selected to construct a characteristic image of the floating image.

6. The method according to claim 1, characterized in that The determining of the effective area of ​​perspective projection according to the feature image of the reference image, and using the effective area of ​​perspective projection to project the feature image of the floating image to obtain at least one two-dimensional projection image specifically includes: Determining a valid area of ​​perspective projection by normalizing a feature image of the reference image; By utilizing the effective area of ​​the perspective projection, perspective projection is performed on the characteristic images of the floating image at least one preset viewing angle corresponding to the acquisition of the reference image to obtain at least one two-dimensional projection image.

7. The method according to claim 6, characterized in that The determining the effective area of ​​the perspective projection by normalizing the feature image of the reference image specifically includes: By normalizing the feature image of the reference image, the feature image of the reference image is divided according to grayscale intensity; An image portion whose grayscale intensity in the feature image of the reference image is less than a preset threshold is selected as a valid area for perspective projection.

8. The method according to claim 6, characterized in that The method of using the effective area of ​​the perspective projection to perform perspective projection on the feature images of the floating image at least one preset viewing angle corresponding to the acquisition of the reference image to obtain at least one two-dimensional projection image specifically includes: Determine a source point and multiple end points of a perspective projection ray in a device coordinate system, wherein the source point is the spatial coordinate of the ray source, and the multiple end points are the spatial coordinates of each voxel in a valid area of ​​the perspective projection; Connecting the source point of the perspective projection ray to multiple end points respectively to form multiple perspective projection rays in the reference image, and traversing the projection process so that the multiple perspective projection rays pass through the voxels of the floating image; The maximum grayscale values ​​of the voxels of the plurality of perspective projection rays passing through the floating image are selected as projection grayscale values ​​to construct at least one two-dimensional projection image.

9. The method according to any one of claims 1 to 8, characterized in that The step of calculating the registration similarity measure for the current spatial position of the floating image using the at least one two-dimensional projection image specifically includes: In a valid area of ​​the perspective projection, calculating a similarity measure for registration in different view directions using the at least one two-dimensional projection image; The average value corresponding to the similarity measure of the registration in different view directions is selected as the similarity measure of the registration corresponding to the current spatial position of the floating image.

10. A multi-view medical image registration device, characterized in that: include: An acquisition unit, configured to acquire a feature image of a reference image and a feature image of a floating image, wherein the reference image is at least one two-dimensional medical image acquired during surgery, and the floating image is a three-dimensional medical image acquired before surgery; a projection unit, configured to determine an effective area of ​​perspective projection according to a feature image of the reference image, and use the effective area of ​​perspective projection to project the feature image of the floating image to obtain at least one two-dimensional projection image, specifically squaring and normalizing the grayscale value of the image feature of the reference image to obtain a grayscale distribution in the image feature, and selecting an image area with a grayscale value less than a preset threshold as an effective area of ​​perspective projection according to the grayscale distribution; a calculation unit, configured to implement the registration by moving the floating image, and to calculate a similarity measure of the registration for a current spatial position of the floating image using the at least one two-dimensional projection image during the registration process; The output unit is used for outputting the rigid transformation result of the registration according to the current spatial position of the floating image if the similarity measure of the registration meets the preset condition.

11. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the multi-view medical image registration method according to any one of claims 1 to 9 is implemented.

12. A multi-view medical image registration device, comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the program, the multi-view medical image registration method according to any one of claims 1 to 9 is implemented.

Images