Image processing method and DSA device

Abstract

The invention discloses an image processing method and DSA equipment, and relates to the field of image processing. According to the method, an initial transformation matrix used for representing a mapping relation between a first face image after head movement of a scanned object and a second face image before head movement can be obtained, and based on a target transformation matrix obtained by the initial transformation matrix, an angiography image obtained after head movement is transformed to obtain an angiography image. Therefore, the angiography image is corrected. Then, the method can perform registration fusion on the transformed angiography image and the three-dimensional blood vessel image. According to the method, before the three-dimensional blood vessel image is registered and fused, the target transformation matrix is adopted to correct the angiography image, so that the corrected angiography image is aligned with the angiography image before the head moves to a large extent, the influence of the head movement on registration can be effectively eliminated, and the registration accuracy is improved. The high registration accuracy of the angiography image and the three-dimensional blood vessel image is ensured.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method and a DSA device. Background Art

[0002] Digital subtraction angiography (DSA) 3D navigation technology is a 3D navigation technology that integrates intraoperative fluoroscopic images (such as angiography images) with preoperative 3D vascular images to provide an accurate and comprehensive view of vascular structures. DSA 3D navigation technology can better assist doctors in performing interventional surgery to improve the success rate of surgery, and is widely used in intracranial interventional treatment.

[0003] During intracranial interventional surgery, local anesthesia is usually used on the scanned object, which causes the scanned object to involuntarily shift its head. When the scanned object's head shifts, the registration accuracy of the fluoroscopic image and the three-dimensional vascular image is reduced, thereby affecting the accuracy of the interventional treatment. Summary of the invention

[0004] The present application provides an image processing method and a DSA device, which can solve the problem of low registration accuracy of intraoperative fluoroscopic images and three-dimensional vascular images in the related art. The technical solution is as follows:

[0005] In one aspect, an image processing method is provided, the method comprising:

[0006] Acquire an initial transformation matrix, where the initial transformation matrix is ​​used to represent a mapping relationship between a first face image and a second face image of a scanned object, where the first face image is a face image acquired after the head of the scanned object moves, and the second face image is a face image acquired before the head moves;

[0007] transforming the angiography image of the head acquired after the head moves based on a target transformation matrix obtained from the initial transformation matrix to obtain a target transformed image;

[0008] The target transformed image is registered and fused with the three-dimensional blood vessel image.

[0009] Optionally, get the initial transformation matrix, including:

[0010] Acquire the first face image and the second face image through an image acquisition device;

[0011] Based on the first face image, obtaining a first posture of the head relative to the image acquisition device after the movement;

[0012] Based on the second facial image, obtaining a second posture of the head relative to the image acquisition device before the movement;

[0013] Based on the first posture and the second posture, an initial transformation matrix is ​​obtained.

[0014] Optionally, the image acquisition device is a camera, the first posture is represented by a first transformation matrix between a world coordinate system and a first camera coordinate system, and the second posture is represented by a second transformation matrix between the world coordinate system and a second camera coordinate system; and obtaining an initial transformation matrix based on the first posture and the second posture includes:

[0015] The inverse matrix of the second transformation matrix is ​​multiplied by the first transformation matrix to obtain an initial transformation matrix.

[0016] Optionally, before multiplying the inverse matrix of the second transformation matrix by the first transformation matrix to obtain an initial transformation matrix, the method further includes:

[0017] Inputting the target face image into the transformation matrix determination model to obtain the auxiliary transformation matrix output by the transformation matrix determination model;

[0018] Wherein, the target facial image is the first facial image or the second facial image;

[0019] When the target facial image is the first facial image, the auxiliary transformation matrix is ​​the first transformation matrix; when the target facial image is the second facial image, the auxiliary transformation matrix is ​​the second transformation matrix.

[0020] Optionally, before transforming the angiography image of the head acquired after the head movement based on the target transformation matrix obtained from the initial transformation matrix to obtain the target transformed image, the method further includes:

[0021] The initial transformation matrix is ​​adjusted to obtain a target transformation matrix.

[0022] Optionally, adjusting the initial transformation matrix to obtain a target transformation matrix includes:

[0023] Based on the initial transformation matrix, transform the angiography image to obtain an initial transformed image;

[0024] Determining the similarity between each digitally reconstructed image obtained by projecting the three-dimensional vascular image and the initial transformed image;

[0025] From the plurality of digitally reconstructed images, obtaining a digitally reconstructed image having the greatest similarity to the initial transformed image;

[0026] Based on the intermediate transformation matrix between the initial transformation image and the digitally reconstructed image with the greatest similarity, the initial transformation matrix is ​​adjusted to obtain a target transformation matrix.

[0027] Optionally, adjusting the initial transformation matrix based on an intermediate transformation matrix between the initial transformation image and the digitally reconstructed image with the greatest similarity to obtain a target transformation matrix includes:

[0028] The initial transformation matrix is ​​multiplied by the intermediate transformation matrix to obtain the target transformation matrix.

[0029] In another aspect, a DSA device is provided, the DSA device comprising: a processor; the processor is configured to:

[0030] Acquire an initial transformation matrix, where the initial transformation matrix is ​​used to represent a mapping relationship between a first face image and a second face image of a scanned object, where the first face image is a face image acquired after the head of the scanned object moves, and the second face image is a face image acquired before the head moves;

[0031] transforming the angiography image of the head acquired after the head moves based on a target transformation matrix obtained from the initial transformation matrix to obtain a target transformed image;

[0032] The target transformed image is registered and fused with the three-dimensional blood vessel image.

[0033] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image processing method described in the above aspects is implemented.

[0034] On the other hand, a computer program product is provided, which includes a computer program or computer instructions, and when the computer program or the computer instructions are executed by a processor, the image processing method as described in the above aspects is implemented.

[0035] The beneficial effects brought about by the technical solution provided by this application include at least:

[0036] The present application provides an image processing method and a DSA device, which can obtain an initial transformation matrix for characterizing the mapping relationship between a first face image after head movement of a scanned object and a second face image before the head movement, and then transform the angiography image obtained after the head movement based on a target transformation matrix obtained from the initial transformation matrix to correct the angiography image. Then, the method can register and fuse the transformed angiography image with a three-dimensional vascular image. Since the method can use a target transformation matrix to correct the angiography image before registering and fusing with the three-dimensional vascular image, so that the corrected angiography image is aligned with the angiography image before the head movement to a greater extent, the influence of head movement on the registration can be effectively eliminated, thereby ensuring that the registration accuracy of the angiography image and the three-dimensional vascular image is high.

[0037] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flow chart of an image processing method provided by an embodiment of the present application;

[0039] Figure 2 is a flowchart of another image processing method provided by an embodiment of the present application;

[0040] Figure 3 is a schematic diagram of a transformation matrix determination model provided in an embodiment of the present application;

[0041] Figure 4 It is a flow chart of a method for adjusting an initial transformation matrix to obtain a target transformation matrix provided by an embodiment of the present application;

[0042] Figure 5 It is a schematic diagram of the implementation principle of a ray projection method provided in an embodiment of the present application;

[0043] Figure 6 It is a registration method based on image gradient features, which registers the vascular subtraction image obtained during surgery at the first perspective with the three-dimensional vascular image to obtain the registered image;

[0044] Figure 7 is a schematic diagram of a fused image at a first viewing angle provided in an embodiment of the present application;

[0045] Figure 8 It is a registration method based on image gradient features, which registers the vascular subtraction image obtained during surgery at the second viewing angle with the three-dimensional vascular image to obtain the registered image;

[0046] Fig. 9 is a schematic diagram of a fused image under a second viewing angle provided in an embodiment of the present application;

[0047] Fig.10 It is a structural diagram of a DSA device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0049] The present application embodiment provides an image processing method, which is applied to a DSA device. Figure 1 , the method comprising:

[0050] Step 101: Obtain an initial transformation matrix.

[0051] The initial transformation matrix is ​​used to characterize the mapping relationship between the first face image and the second face image of the scanned object. The scanned object is a human body, an animal body or a phantom. The first face image is a face image obtained after the head of the scanned object moves, and the second face image is a face image obtained before the head moves.

[0052] In an embodiment of the present application, the initial transformation matrix may be pre-stored by the DSA device. Alternatively, the DSA device may obtain a first face image and a second face image through an image acquisition device, and obtain a first posture of the head after movement relative to the image acquisition device based on the first face image, and obtain a second posture of the head before movement relative to the image acquisition device based on the second face image. Afterwards, the DSA device may obtain an initial transformation matrix based on the amount of change of the first posture relative to the second posture. Optionally, the image acquisition device may be a camera or a still camera.

[0053] Step 102: Based on the target transformation matrix obtained from the initial transformation matrix, the angiography image of the head acquired after the head movement is transformed to obtain a target transformed image.

[0054] The DSA device can transform the angiography image using a target transformation matrix to correct the angiography image. The corrected angiography image can be aligned to a large extent with the angiography image before the head movement.

[0055] Optionally, the target transformation matrix is ​​the initial transformation matrix. Alternatively, the DSA device may adjust the initial transformation matrix to obtain the target transformation matrix.

[0056] Step 103: register and fuse the target transformed image with the three-dimensional blood vessel image.

[0057] After the DSA device obtains the target transformation image, it can register the target transformation image with the three-dimensional vascular image. Then, the DSA device can fuse the registered target transformation image with the three-dimensional vascular image to obtain a fused image, and display the fused image to assist doctors in performing interventional surgery. The three-dimensional vascular image can be obtained by scanning and reconstructing before the head of the scanned object moves.

[0058] In summary, the embodiment of the present application provides an image processing method, which can obtain an initial transformation matrix for characterizing the mapping relationship between a first face image after the head movement of the scanned object and a second face image before the head movement, and then transform the angiography image obtained after the head movement based on the target transformation matrix obtained from the initial transformation matrix to correct the angiography image. Then, the method can register and fuse the transformed angiography image with the three-dimensional vascular image. Since the method can use the target transformation matrix to correct the angiography image before registering and fusing with the three-dimensional vascular image, so that the corrected angiography image is aligned with the angiography image before the head movement to a large extent, the influence of head movement on the registration can be effectively eliminated, thereby ensuring that the registration accuracy of the angiography image and the three-dimensional vascular image is high.

[0059] In the embodiment of the present application, the image acquisition device is a camera, the initial transformation matrix is ​​determined based on the first face image and the second face image, and the target transformation matrix is ​​obtained by adjusting the initial transformation matrix. The image processing method provided in the embodiment of the present application is exemplarily described, and the method can be applied to the DSA device. Figure 2 , the method may include:

[0060] Step 201: Acquire a first face image and a second face image of a scanned object through a camera.

[0061] When the head of the scanned object is deflected, the face of the scanned object will also be deflected, and the face can provide relatively abundant feature points. Therefore, in the embodiment of the present application, the DSA device can obtain the first face image and the second face image of the scanned object, and then obtain the initial transformation matrix based on the first face image and the second face image, so as to realize the correction of the angiography image obtained after the head is deflected.

[0062] The first face image of the scanned object is a face image obtained after the head of the scanned object moves. The second face image of the scanned object is a face image obtained before the head of the scanned object moves. Optionally, the camera can be a monocular camera or a multi-camera camera, for example, a monocular camera.

[0063] It should be understood that since the posture of the head of the scanning object when acquiring the first facial image is different from the posture of the head when acquiring the second facial image, and the posture of the camera remains unchanged during the process of acquiring the first facial image and the second facial image, the acquired first facial image is different from the second facial image.

[0064] Optionally, the first face image and the second face image may both be two-dimensional (dimension, D) images, and may both be color images. The color image uses color information of three channels, red (red, R), green (green, G) and blue (blue, B), to describe the color of each pixel, and thus may also be called an RGB image.

[0065] It is understandable that, in the scenario of interventional surgery, the second face image may be acquired before the surgery, and the first face image may be acquired after the scanned object is anesthetized.

[0066] Step 202: Based on the first face image, obtain the first posture of the head of the scanned object relative to the camera after the movement.

[0067] In practical applications, the camera is usually fixed. The head of the scanned object can move relative to the camera. Conversely, assuming that the head is used as a reference object during the movement of the head, it can be regarded as the movement of the camera relative to the head. Moreover, the relative transformation of the posture of the head before and after the movement of the head is equivalent to the relative transformation of the posture of the camera before and after the movement of the camera. Correspondingly, the first posture of the head of the scanned object relative to the camera after the movement is equivalent to the third posture of the camera relative to the head after the movement of the head. Therefore, the method provided in the embodiment of the present application can convert the initial transformation matrix obtained based on the posture transformation of the head into the initial transformation matrix based on the posture transformation of the camera.

[0068] Since the posture of the camera determines the position and direction of the camera coordinate system relative to the world coordinate system, the third posture can be represented by the first transformation matrix between the world coordinate system and the first camera coordinate system. In other words, the first posture can be represented by the first transformation matrix. The first camera coordinate system refers to: after the head of the scanned object moves, the camera coordinate system established based on the posture of the camera relative to the head at this time. The origin of the camera coordinate system is the optical center of the camera (i.e., the center of the lens), the Z axis is the optical axis of the camera (usually pointing to the front of the camera), and the X axis and Y axis are both perpendicular to the Z axis and conform to the right-hand rule.

[0069] In some embodiments, the DSA device pre-stores a transformation matrix determination model. The DSA device can input the acquired target face image into the transformation matrix determination model to obtain an auxiliary transformation matrix output by the transformation matrix determination model.

[0070] The target facial image may be a first facial image or a second facial image. When the target facial image is the first facial image, the auxiliary transformation matrix is ​​the first transformation matrix, and when the target facial image is the second facial image, the auxiliary transformation matrix is ​​the second transformation matrix. When it is necessary to obtain the first transformation matrix, the DSA device may input the first facial image into the transformation matrix determination model to obtain the first transformation matrix output by the transformation matrix determination model.

[0071] In the examples of this application, see Figure 3 The transformation matrix determination model includes: a feature extraction module 01, a 3D face reconstruction module 02, a correspondence learning module 03 and a matrix acquisition module 04. The output end of the feature extraction module 01 is connected to the input end of the 3D face reconstruction module 02 and the input end of the correspondence learning module 03. The output end of the 3D face reconstruction module 02 and the output end of the correspondence learning module 03 are both connected to the matrix acquisition module 04.

[0072] The feature extraction module 01 is used to extract the 3D features and 2D image features of the face from the input first face image. The 2D features include: 2D face image features and 2D position features. The 2D face image features are used to describe the visual attributes of the face image, such as edges, textures, and shapes. The 2D position features encode the positions of the key feature points of the face. Specifically, the feature extraction module 01 includes: an encoder, a first decoder, and a second decoder. The output end of the encoder is connected to the input end of the first decoder and the second decoder, respectively. The encoder and the first decoder are used to extract 3D features from the first face image, and the encoder and the second encoder are used to extract 2D features from the first face image.

[0073] The 3D face reconstruction module 02 is used to reconstruct a 3D face in a standard space. The input end of the 3D face reconstruction module 02 is connected to the output end of the first decoder, and the 3D face reconstruction module 02 can regress the UV position map based on the 3D features output by the first decoder to obtain a 3D face in the standard space. The 3D face can be represented by at least 3D face shape information. The 3D face shape information includes: the positions of multiple key points of the face in the standard space.

[0074] The correspondence learning module 03 is used to estimate the 2D-3D correspondence between the 3D face and the target face image in the standard space. The input end of the correspondence learning module 03 is connected to the first decoder and the second decoder respectively, and is used to fuse the 2D face image features, the encoded 2D position features and the 3D features to learn the correspondence between the 2D pixels in the 2D target face image and the 3D points in the 3D face.

[0075] The matrix acquisition module 04 is used to calculate the 6 degrees of freedom (DoF) posture parameters of the camera. The 6DoF posture parameters include: rotation (roll, pitch and yaw) and 3D translation of the camera relative to the world, that is, the 6DoF posture parameters can describe the position and direction of the first camera coordinate system relative to the world coordinate system. In other words, the 6DoF posture parameters can be used to characterize the first transformation matrix between the world coordinate system and the first camera coordinate system mentioned above. Specifically, the matrix acquisition module 04 can obtain 3D points corresponding to 2D pixels in the target face image from the 3D face based on the 2D-3D correspondence relationship output by the correspondence learning module 03, the 3D face output by the 3D face reconstruction module 02, and the target face image. Then, the matrix acquisition module 04 can calculate the 6DoF posture parameters of the camera based on the position of the 2D pixels in the target face image, the position of the corresponding 3D points and the internal parameters (i.e., internal parameters) of the camera through the perspective-n-point (PnP) algorithm and the random sample and consensus (RANSAC) algorithm. Among them, the camera's internal parameters may include focal length and distortion coefficient, etc.

[0076] Step 203: Based on the second face image, obtain a second posture of the head of the scanned object relative to the camera before the movement.

[0077] Assuming that the head is used as a reference object during the movement of the head, it can be regarded as the movement of the camera relative to the head. Correspondingly, the second posture of the head of the scanned object relative to the camera before the movement is equivalent to the fourth posture of the camera relative to the head before the head moves. The fourth posture can be represented by the second transformation matrix of the world coordinate system and the second camera coordinate system. In other words, the second posture can be represented by the second transformation matrix. The second camera coordinate system refers to: before the head of the scanned object moves, the camera coordinate system established based on the posture of the camera relative to the head at this time.

[0078] It can be understood that the process of the DSA device acquiring the second transformation matrix can refer to the relevant implementation process of acquiring the first transformation matrix in step 202, and the embodiments of the present application will not be repeated here.

[0079] Step 204: Obtain an initial transformation matrix based on the first pose and the second pose.

[0080] Since the first pose is represented by the first transformation matrix and the second pose is represented by the second transformation matrix, the DSA device can calculate the inverse matrix of the second transformation matrix and multiply the inverse matrix by the first transformation matrix to obtain the initial transformation matrix.

[0081] The initial transformation matrix can represent the mapping relationship between the first face image and the second face image. The mapping relationship is actually the conversion relationship between the first camera coordinate system and the second camera coordinate system mentioned above, and is also the position transformation relationship of the head before and after the head movement.

[0082] Since DSA three-dimensional navigation has extremely high requirements on accuracy, the correction accuracy of the angiography image corrected by the initial transformation matrix may not be sufficient to meet the requirements. Therefore, after obtaining the initial transformation matrix, the DSA device can also perform the following step 205.

[0083] Step 205: Adjust the initial transformation matrix to obtain the target transformation matrix.

[0084] The adjustment of the initial transformation matrix refers to adjusting the parameters in the initial transformation matrix, so that the initial transformation matrix can be optimized.

[0085] Since the DSA device can adjust the initial transformation matrix to obtain the target transformation matrix after obtaining the initial transformation matrix, on the one hand, it can ensure that the correction accuracy of the angiography image using the obtained target transformation matrix is ​​high, so that the transformed angiography image can meet the surgical requirements; on the other hand, it can improve the acquisition efficiency of the target transformation matrix while ensuring high correction accuracy.

[0086] In the examples of this application, see Figure 4 The DSA device adjusts the initial transformation matrix to obtain the target transformation matrix, which may include:

[0087] Step 2051: transform the angiography image based on the initial transformation matrix to obtain an initial transformed image.

[0088] After the DSA device obtains the initial transformation matrix, it can use the initial transformation matrix to transform the angiography image to perform a rough correction on the angiography image to obtain an initial transformed image.

[0089] Step 2052: Determine the similarity between the digitally reconstructed image obtained by projecting the three-dimensional vascular image and the initial transformed image.

[0090] In an embodiment of the present application, the DSA device can project the three-dimensional vascular image based on multiple projection angles to project the three-dimensional vascular image onto a 2D plane, thereby obtaining multiple digitally reconstructed images (digitally reconstructed radiograph, DRR). Each DRR is a 2D image. Each projection angle corresponds to a DRR. Any two projection angles are different. The three-dimensional vascular image can be obtained by scanning and reconstructing the head of the scanned object before surgery. The three-dimensional vascular image can be a computed tomography angiography (computed tomography angiography, CTA) image.

[0091] Then, for each DRR, the DSA device may process the DRR and the initial transformed image using a similarity calculation algorithm to obtain the similarity between the DRR and the initial transformed image. The similarity calculation algorithm may be one of the following algorithms: a Euclidean distance calculation algorithm, a cosine distance calculation algorithm, and a normalized cross correlation (NCC) algorithm.

[0092] For example, since there is a roughly linear dependency between the DRR and the initial transformed image, the DSA device provided in the embodiment of the present application can use the NCC algorithm to determine the similarity between the DRR and the initial transformed image.

[0093] It is understandable that the DSA device can use ray casting to project the three-dimensional blood vessel image of the head. The implementation principle of the ray casting method is as follows: Figure 5 The pixel value of each pixel in the obtained DRR may be: the line integral of the CT value of the voxel located on the beam path corresponding to the pixel.

[0094] Step 2053: Obtain the DRR with the greatest similarity to the initial transformed image from the multiple digitally reconstructed images.

[0095] After the DSA device obtains the similarity between each DRR and the initial transformed image, it can obtain the maximum similarity from multiple similarities, and then obtain the DRR with the greatest similarity to the initial transformed image.

[0096] Optionally, the DSA device may use the Powell algorithm to perform the above steps 2052 and 2053 to obtain a DRR with the greatest similarity to the initial transformed image. Specifically, first, define an objective function for the Powell algorithm, which may be a similarity measure between the DRR and the initial transformed image. Accordingly, the function value of the objective function is the similarity between the DRR and the initial transformed image. Secondly, initialize the parameters, that is, provide an initial projection angle for the Powell algorithm.

[0097] Then, the Powell algorithm can iteratively execute the similarity determination process until the stop condition is met to converge to the optimal solution, and the DRR corresponding to the optimal solution has the largest similarity with the initial transformed image. The similarity determination process includes: using the current projection angle to project the three-dimensional vascular image to generate the DRR; determining the similarity between the DRR and the initial transformed image, and determining the similarity as the function value of the objective function; updating the current projection angle according to the function value.

[0098] The stop condition may include: the number of iterations reaches a preset number, and / or the function value of the objective function is higher than a numerical threshold. The preset number and the numerical threshold may be pre-stored by the DSA device. In addition, it should be understood that each iteration of the Powell algorithm generates a new DRR.

[0099] Since the Powell algorithm can use the conjugate direction to accelerate the convergence speed, the Powell algorithm can effectively improve the efficiency of obtaining the maximum similarity, thereby improving the efficiency of obtaining the target transformation matrix.

[0100] Step 2054: Based on the intermediate transformation matrix between the initial transformation image and the DRR with the greatest similarity, the initial transformation matrix is ​​adjusted to obtain a target transformation matrix.

[0101] In an embodiment of the present application, after the DSA device obtains the DRR with the greatest similarity to the initial transformed image, it can determine the intermediate transformation matrix between the DRR and the initial transformed image, and then adjust the initial transformation matrix based on the intermediate transformation matrix to obtain the target transformation matrix.

[0102] The intermediate transformation matrix between the DRR and the initial transformation image may refer to: the transformation matrix between the image coordinate system where the DRR is located and the image coordinate system where the initial transformation image is located. The image coordinate system where each image in the DRR and the initial transformation image is located may refer to: a coordinate system established with a point on the image as the origin, the direction in which the pixel row extends as the X direction, and the direction in which the pixel column extends as the Y direction.

[0103] Optionally, the DSA device may multiply the intermediate transformation matrix with the initial transformation matrix to adjust the initial transformation matrix, and determine the product of the intermediate transformation matrix and the initial transformation matrix as the target transformation matrix.

[0104] According to the above description, the method provided in the embodiment of the present application can realize the accurate correction of the angiography image during 3D navigation surgery by combining the correction based on head posture with the image registration based on DRR, so as to eliminate the influence of the head movement of the scanned object on the registration of the angiography image acquired during the surgery and the three-dimensional vascular image acquired before the surgery to a large extent, thereby ensuring the high registration accuracy of the angiography image and the three-dimensional vascular image.

[0105] Step 206: Based on the target transformation matrix, transform the angiography image of the head acquired after the head movement to obtain a target transformed image.

[0106] After obtaining the target transformation matrix, the DSA device can use the target transformation matrix to transform the angiography image to correct the angiography image, thereby obtaining a target transformation image, which is the corrected image.

[0107] Step 207: register and fuse the target transformed image with the three-dimensional blood vessel image of the head.

[0108] In the embodiment of the present application, after the DSA device obtains the target transformation image, it can register the target transformation image with the three-dimensional vascular image. Then, the DSA device can fuse the registered target transformation image with the three-dimensional vascular image to obtain a fused image to assist doctors in performing interventional surgery.

[0109] During intracranial interventional surgery, local anesthesia is usually used for the scanned object, which will cause the scanned object to involuntarily shift its head. When the scanned object's head shifts, the registration accuracy of the perspective image and the three-dimensional vascular image will be reduced, thereby affecting the accuracy of interventional treatment. In related technologies, an image gradient feature registration method can be used to register the vascular subtraction image obtained during the operation with the three-dimensional vascular image. However, the registration accuracy of the image gradient feature registration method is low.

[0110] For example, Figure 6 The invention shows a registration method based on image gradient features, which registers a vascular subtraction image acquired during surgery at a first viewing angle with a three-dimensional vascular image to obtain a registered image. Figure 7 Schematic diagram of a fused image at a first viewing angle provided in an embodiment of the present application. The first viewing angle is approximately 0 degrees (°).

[0111] Figure 6 and Figure 7In the figure, the black blood vessels are the angiographic images obtained after correction of the angiographic images obtained after the head shift during the operation. The red blood vessels are the three-dimensional vascular images obtained before the operation. Figure 6 and Figure 7 The area indicated by the yellow arrow can be seen. Figure 6 In the area indicated by the yellow arrow, the red blood vessels are not aligned with the black blood vessels. Figure 7 In the area indicated by the yellow arrow, the red blood vessels and the black blood vessels can be precisely aligned.

[0112] Figure 8 The invention shows a registration method based on image gradient features, which is used to register a vascular subtraction image acquired during surgery at a second viewing angle with a three-dimensional vascular image to obtain a registered image. Fig. 9 Schematic diagram of a fused image at a second viewing angle provided by an embodiment of the present application. The first viewing angle is 90 degrees (°).

[0113] contrast Figure 8 and Fig. 9 The area indicated by the yellow arrow can be seen. Figure 8 In the area indicated by the yellow arrow, the red blood vessels are not aligned with the black blood vessels. Fig. 9 In the area indicated by the yellow arrow, the red blood vessels and the black blood vessels can be precisely aligned.

[0114] according to Figures 6 to 9 It can be seen that compared with the image gradient feature-based registration method, the registration accuracy of the method provided in the embodiment of the present application is higher.

[0115] It is understandable that the order of the steps of the image processing method provided in the embodiment of the present application can be appropriately adjusted, and the steps can also be increased or decreased accordingly according to the situation. For example, steps 201 to 204 can be deleted according to the situation, that is, the DSA device pre-stores the initial transformation matrix. Any technician familiar with the technical field can easily think of a method of change within the technical scope disclosed in this application, which should be included in the protection scope of this application, so it will not be repeated.

[0116] In summary, the embodiment of the present application provides an image processing method, which can obtain an initial transformation matrix for characterizing the mapping relationship between a first face image after the head movement of the scanned object and a second face image before the head movement, and then transform the angiography image obtained after the head movement based on the target transformation matrix obtained from the initial transformation matrix to correct the angiography image. Then, the method can register and fuse the transformed angiography image with the three-dimensional vascular image. Since the method can use the target transformation matrix to correct the angiography image before registering and fusing with the three-dimensional vascular image, so that the corrected angiography image is aligned with the angiography image before the head movement to a large extent, the influence of head movement on the registration can be effectively eliminated, thereby ensuring that the registration accuracy of the angiography image and the three-dimensional vascular image is high.

[0117] The present application embodiment provides a DSA device, see Fig.10 The DSA device 300 includes a processor 301; the processor 301 is used to:

[0118] Obtaining an initial transformation matrix, where the initial transformation matrix is ​​used to represent a mapping relationship between a first face image and a second face image of the scanned object, where the first face image is a face image obtained after the head of the scanned object moves, and the second face image is a face image obtained before the head moves;

[0119] Based on a target transformation matrix obtained from an initial transformation matrix, transforming a head angiography image acquired after the head moves to obtain a target transformed image;

[0120] The target transformed image is registered and fused with the three-dimensional vascular image.

[0121] Optionally, the processor 301 may be configured to:

[0122] Acquire a first face image and a second face image through an image acquisition device;

[0123] Based on the first face image, obtaining a first posture of the head relative to the image acquisition device after the movement;

[0124] Based on the second face image, obtaining a second posture of the head relative to the image acquisition device before the movement;

[0125] Based on the first pose and the second pose, get the initial transformation matrix.

[0126] Optionally, the image acquisition device is a camera, the first pose is represented by a first transformation matrix between a world coordinate system and a first camera coordinate system, and the second pose is represented by a second transformation matrix between a world coordinate system and a second camera coordinate system. The processor 301 may be used to:

[0127] The inverse matrix of the second transformation matrix is ​​multiplied by the first transformation matrix to obtain the initial transformation matrix.

[0128] Optionally, the processor 301 may also be used for:

[0129] Before multiplying the inverse matrix of the second transformation matrix by the first transformation matrix to obtain an initial transformation matrix, inputting the target face image into a transformation matrix determination model to obtain an auxiliary transformation matrix output by the transformation matrix determination model;

[0130] Wherein, the target face image is the first face image or the second face image;

[0131] When the target face image is a first face image, the auxiliary transformation matrix is ​​a first transformation matrix, and when the target face image is a second face image, the auxiliary transformation matrix is ​​a second transformation matrix.

[0132] Optionally, the processor 301 may be configured to:

[0133] Before transforming the angiography image of the head acquired after the head movement based on the target transformation matrix obtained from the initial transformation matrix to obtain the target transformed image, the initial transformation matrix is ​​adjusted to obtain the target transformation matrix.

[0134] Optionally, the processor 301 may be configured to:

[0135] Based on the initial transformation matrix, the angiography image is transformed to obtain an initial transformed image;

[0136] Determining the similarity between each digitally reconstructed image obtained by projecting the three-dimensional vascular image and the initial transformed image;

[0137] From the multiple digitally reconstructed images, a digitally reconstructed image having the greatest similarity to the initial transformed image is obtained;

[0138] Based on the intermediate transformation matrix between the initial transformation image and the digitally reconstructed image with the greatest similarity, the initial transformation matrix is ​​adjusted to obtain the target transformation matrix.

[0139] Optionally, the processor 301 may be configured to:

[0140] Multiply the initial transformation matrix by the intermediate transformation matrix to obtain the target transformation matrix.

[0141] In summary, an embodiment of the present application provides a DSA device, which can obtain an initial transformation matrix for characterizing the mapping relationship between a first facial image after head movement of a scanned object and a second facial image before the head movement, and then transform the angiography image obtained after the head movement based on a target transformation matrix obtained from the initial transformation matrix to correct the angiography image. Then, the DSA device can register and fuse the transformed angiography image with the three-dimensional vascular image. Since the DSA device can use the target transformation matrix to correct the angiography image before registering and fusing with the three-dimensional vascular image, so that the corrected angiography image is aligned with the angiography image before the head movement to a greater extent, the influence of head movement on the registration can be effectively eliminated, thereby ensuring that the registration accuracy of the angiography image and the three-dimensional vascular image is high.

[0142] like Fig.10 As shown, the DSA device 300 provided in the embodiment of the present application may also include: a memory 303. The processor 301 and the memory 303 are connected, such as through a bus 302. Optionally, the controller 300 may also include a transceiver 304. It should be noted that in actual applications, the transceiver 304 is not limited to one, and the structure of the controller 300 does not constitute a limitation on the embodiment of the present application.

[0143] Processor 301 may be a CPU (Central Processing Unit), a general purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0144] The bus 302 may include a path to transmit information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0145] The memory 303 is used to store a computer program corresponding to the image processing method of the above embodiment of the present application, and the computer program is controlled and executed by the processor 301. The processor 301 is used to execute the computer program stored in the memory 303 to implement the content shown in the above method embodiment.

[0146] The present application embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the image processing method provided in the above method embodiment is implemented. For example, Figure 1 or Figure 2 The method shown.

[0147] The present application embodiment provides a computer program product, which includes a computer program or computer instructions. When the computer program or computer instructions are executed by a processor, the image processing method provided by the above method embodiment is implemented. For example, Figure 1 or Figure 2 The method shown.

[0148] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0149] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0150] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0151] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0152] In this application, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0153] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire an initial transformation matrix, where the initial transformation matrix is ​​used to represent a mapping relationship between a first face image and a second face image of a scanned object, where the first face image is a face image acquired after the head of the scanned object moves, and the second face image is a face image acquired before the head moves; transforming the angiography image of the head acquired after the head moves based on a target transformation matrix obtained from the initial transformation matrix to obtain a target transformed image; The target transformed image is registered and fused with the three-dimensional blood vessel image.

2. The method according to claim 1, characterized in that: Get the initial transformation matrix, including: Acquire the first face image and the second face image through an image acquisition device; Based on the first face image, obtaining a first posture of the head relative to the image acquisition device after the movement; Based on the second facial image, obtaining a second posture of the head relative to the image acquisition device before the movement; Based on the first posture and the second posture, an initial transformation matrix is ​​obtained.

3. The method according to claim 2, characterized in that The image acquisition device is a camera, the first posture is represented by a first transformation matrix between a world coordinate system and a first camera coordinate system, and the second posture is represented by a second transformation matrix between the world coordinate system and a second camera coordinate system; Based on the first posture and the second posture, obtaining an initial transformation matrix includes: The inverse matrix of the second transformation matrix is ​​multiplied by the first transformation matrix to obtain an initial transformation matrix.

4. The method according to claim 3, characterized in that Before multiplying the inverse matrix of the second transformation matrix by the first transformation matrix to obtain an initial transformation matrix, the method further includes: Inputting the target face image into the transformation matrix determination model to obtain the auxiliary transformation matrix output by the transformation matrix determination model; Wherein, the target facial image is the first facial image or the second facial image; When the target facial image is the first facial image, the auxiliary transformation matrix is ​​the first transformation matrix; when the target facial image is the second facial image, the auxiliary transformation matrix is ​​the second transformation matrix.

5. The method according to any one of claims 1 to 4, characterized in that: Before transforming the angiography image of the head acquired after the head movement based on the target transformation matrix obtained from the initial transformation matrix to obtain the target transformed image, the method further includes: The initial transformation matrix is ​​adjusted to obtain a target transformation matrix.

6. The method according to claim 5, characterized in that The initial transformation matrix is ​​adjusted to obtain a target transformation matrix, including: Based on the initial transformation matrix, transform the angiography image to obtain an initial transformed image; Determining the similarity between each digitally reconstructed image obtained by projecting the three-dimensional vascular image and the initial transformed image; From the plurality of digitally reconstructed images, obtaining a digitally reconstructed image having the greatest similarity to the initial transformed image; Based on the intermediate transformation matrix between the initial transformation image and the digitally reconstructed image with the greatest similarity, the initial transformation matrix is ​​adjusted to obtain a target transformation matrix.

7. The method according to claim 6, characterized in that Based on the intermediate transformation matrix between the initial transformation image and the digitally reconstructed image with the greatest similarity, the initial transformation matrix is ​​adjusted to obtain a target transformation matrix, including: The initial transformation matrix is ​​multiplied by the intermediate transformation matrix to obtain the target transformation matrix.

8. A DSA device, characterized in that: The DSA device comprises: a processor; the processor is configured to: Acquire an initial transformation matrix, where the initial transformation matrix is ​​used to represent a mapping relationship between a first face image and a second face image of a scanned object, where the first face image is a face image acquired after the head of the scanned object moves, and the second face image is a face image acquired before the head moves; transforming the angiography image of the head acquired after the head moves based on a target transformation matrix obtained from the initial transformation matrix to obtain a target transformed image; The target transformed image is registered and fused with the three-dimensional blood vessel image.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program or a computer instruction, and when the computer program or the computer instruction is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images