A method, apparatus, and electronic device based on DSA and OCT fusion imaging

Abstract

This invention provides a method, apparatus, and electronic device based on DSA and OCT fusion imaging. The method includes: synchronously and continuously acquiring DSA and OCT sequence images; time-stamping the acquired DSA and OCT sequence images; distributing the acquired DSA sequence images to different GPU units for processing according to the time sequence of the stamped images, calculating and marking the position of the imaging ring in the DSA sequence image; and synchronously displaying the OCT sequence image and the DSA sequence image with the marked imaging ring position on the same display according to the marked time. This invention can quickly and accurately obtain a fused DSA and OCT image by matching the imaging ring position on the DSA sequence image with the OCT sequence image obtained through synchronous imaging. This allows doctors to more clearly and intuitively see the retraction of the OCT lens in the blood vessel within the DSA image, providing image reference and technical support for subsequent localization of vascular lesions.

Description

Technical Field

[0001] This invention belongs to the field of medical device technology, specifically relating to a method, apparatus, and electronic device based on DSA and OCT fusion imaging. Background Technology

[0002] Optical coherence tomography (OCT) integrates semiconductor laser technology, optical technology, and computer image processing technology to enable non-contact, non-invasive in vivo morphological examination of the human body and obtain cross-sectional images of the internal microstructure of biological tissues. This information is used to guide percutaneous coronary intervention, including immediate observation after stent placement and evaluation of long-term intravascular repair after stent placement.

[0003] Using OCT technology to image coronary arteries after stent implantation, observers can clearly observe the characteristics of each atherosclerotic plaque, assess whether the stent is well attached to the vessel wall, and whether there is stent location obstruction, tissue prolapse, tissue tearing, in-stent restenosis, plaques, or thrombi. These are of great significance for clinical diagnosis and treatment.

[0004] An imaging catheter is a catheter used in conjunction with an optical interferometry tomography system for imaging coronary arteries. It is used in medical facilities to image the coronary arteries of patients who require endovascular interventional procedures.

[0005] Digital subtraction angiography (DSA) works by taking two X-ray images before and after contrast agent injection, digitizing them, and then inputting them into a computer. Through subtraction, enhancement, and re-imaging processes, bone and soft tissue images are eliminated from the angiographic images to obtain clear vascular images. DSA offers high contrast resolution, clear images, and requires less contrast agent, making it extremely important in the diagnosis of interventional vascular procedures.

[0006] DSA (Digital Subtraction Angiography) vascular image fusion technology refers to coronary angiography using cardiac imaging catheter insertion. An OCT (Optical Characteristic Transmission) imaging catheter is inserted into the blood vessel to reach the coronary artery lesion or distal stent. Contrast agent is then injected into the coronary artery, making it visible on X-ray images. This allows doctors to assess the morphology and structure of the artery by observing the DSA and OCT cross-sectional views. However, due to the physical distance limitations of DSA and OCT equipment, manually matching DSA and OCT images is time-consuming and laborious, and on-site comparison results are not possible. Therefore, DSA image fusion technology is used to quickly match DSA and OCT images and accurately track the position of the contrast ring, facilitating doctors' determination of the imaging catheter location and vascular lesions, thereby further determining the diagnostic results and treatment plan. Summary of the Invention

[0007] To address the problems existing in the prior art, the present invention provides a method and apparatus based on the fusion development of DSA and OCT.

[0008] Specifically, the present invention relates to the following aspects:

[0009] A method for developing images based on the fusion of DSA and OCT, characterized in that the method includes:

[0010] Simultaneously and continuously acquire DSA and OCT sequence images;

[0011] Time-stamping is performed on the acquired DSA and OCT sequence images;

[0012] The acquired DSA sequence images are evenly distributed to different GPU units according to the chronological order of the marking, and the positions of the development rings in the DSA sequence images are calculated and marked.

[0013] The OCT sequence image and the DSA sequence image after marking the location of the development ring are synchronously displayed on the same monitor according to the marked time.

[0014] According to some embodiments of this application, the processing is selected from one or more of Hessian-enhanced multi-scale filtering, Gaussian-Laplace point processing, and local feature point processing.

[0015] According to some embodiments of this application, the processing includes Hessian-enhanced multi-scale filtering, Gaussian-Laplace point processing, and local feature point processing.

[0016] According to some embodiments of this application, the processing obtains vascular paths, local feature points, and Gaussian Laplacian points from the DSA sequence image, which are used to calculate and mark the position of the imaging loop in the DSA sequence image.

[0017] An apparatus for DSA and OCT fusion imaging, characterized in that the apparatus comprises:

[0018] An image acquisition unit is used to simultaneously and continuously acquire DSA sequence images and OCT sequence images;

[0019] A time-stamping unit is used to time-stamp the acquired DSA and OCT sequence images;

[0020] The image processing unit is used to distribute the acquired DSA sequence images equally to different GPU units according to the time sequence of the marking, and to calculate and mark the position of the development ring in the DSA sequence image;

[0021] The image fusion unit is used to synchronously present the OCT sequence image and the DSA sequence image after marking the position of the development ring on the same display according to the marked time.

[0022] According to some embodiments of this application, the GPU unit includes one or more of the following: a Hessian-enhanced multi-scale filtering subunit, a Gaussian-Laplace point processing subunit, and a local feature point processing subunit.

[0023] According to some embodiments of this application, the GPU unit includes multiple Hessian-enhanced multi-scale filtering processing sub-units, multiple Gaussian-Laplace point processing sub-units, and multiple local feature point processing sub-units.

[0024] According to some embodiments of this application, the Hessian-enhanced multi-scale filtering subunit is used to calculate the vascular path of the DSA sequence image.

[0025] According to some embodiments of this application, the Gaussian Laplacian point processing subunit is used to calculate the Gaussian Laplacian points of the DSA sequence image.

[0026] According to some embodiments of this application, the local feature point processing subunit is used to calculate local feature points of the DSA sequence image.

[0027] An electronic device, characterized in that the electronic device comprises:

[0028] Processor; and

[0029] A memory storing computer program instructions that, when executed by the processor, cause the processor to perform the aforementioned method.

[0030] This invention allows for the rapid and accurate generation of fused DSA and OCT images by matching the position of the imaging ring on the DSA sequence image with the OCT sequence image obtained by the synchronous lens. This enables doctors to more clearly and intuitively see the retraction of the OCT lens in the blood vessel in the DSA image, providing image reference and technical support for subsequent localization of vascular lesions. Attached Figure Description

[0031] Figure 1 This is a flowchart of a method for developing based on the fusion of DSA and OCT according to an embodiment of the present invention.

[0032] Figure 2 This is a block diagram of an apparatus for DSA and OCT fusion imaging according to an embodiment of the present invention.

[0033] 1 Image acquisition unit, 2 Time stamping unit, 3 Image processing unit, 4 Image fusion unit. Detailed Implementation

[0034] The present invention will be further described below with reference to embodiments. It should be understood that the embodiments are only used to further illustrate and explain the present invention, and are not intended to limit the present invention.

[0035] Unless otherwise defined, technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art. While similar or identical methods and materials may be applied in experimental or practical applications, materials and methods are described herein. In case of conflict, the definitions included herein shall prevail. Furthermore, materials, methods, and examples are for illustrative purposes only and are not intended to be limiting. The invention is further described below with reference to specific embodiments, but is not intended to limit the scope of the invention.

[0036] As described above, in order to solve the problems existing in the fusion of DSA and OCT images in the prior art, the present invention provides a method for developing images based on the fusion of DSA and OCT, such as... Figure 1 As shown, the method includes:

[0037] S1 simultaneously and continuously acquires DSA and OCT sequence images;

[0038] S2 timestamps the acquired DSA and OCT sequence images;

[0039] S3 distributes the acquired DSA sequence images equally among different GPU units according to the chronological order of the markings, and calculates and marks the position of the development ring in the DSA sequence images;

[0040] S4 synchronously displays the OCT sequence image and the DSA sequence image after marking the location of the development ring on the same display according to the marked time.

[0041] In step S1, DSA sequence images and OCT sequence images are acquired simultaneously, and the DSA sequence images and OCT sequence images are actually a collection of multiple DSA sequence images and OCT sequence images that are consecutive in time.

[0042] In step S2, the acquired DSA sequence image and OCT sequence image are time-stamped. This is to ensure that the subsequent DSA sequence images are calculated in chronological order, and also to ensure the final matching of the OCT sequence image and the DSA sequence image.

[0043] In step S3, the acquired DSA sequence images are evenly distributed to different GPU units for processing according to the chronological order of the labels. Specifically, GPU-CPU collaborative computing can be used.

[0044] GPU-CPU collaborative computing, as a special parallel processing method, can leverage the capabilities of different computing resources according to the characteristics of the relevant task, offering significant advantages in improving the computing performance, energy efficiency, and real-time performance of devices. After acquiring DSA sequence image signals, the sheer volume of data and the substantial time cost of single-pipeline image processing can be mitigated by GPU-CPU collaborative computing, which significantly reduces the waiting time required during the operation.

[0045] Specifically, the processing of DSA sequence images in the GPU unit can be selected from one or more of the following: Hessian-enhanced multi-scale filtering, Laplacian point processing, and local feature point processing. Among them, Hessian-enhanced multi-scale filtering can calculate the blood vessel path in the DSA sequence image, Laplacian point processing can calculate the Laplacian points in the DSA sequence image, and local feature point processing can calculate the local feature points in the DSA sequence image.

[0046] In one specific implementation, the processing includes Hessian-enhanced multi-scale filtering, Gaussian-Laplace point processing, and local feature point processing.

[0047] In one specific implementation, the processing obtains the vascular path, local feature points, and Gaussian Laplacian points of the DSA sequence image, which are used to calculate and mark the position of the imaging loop in the DSA sequence image.

[0048] Specifically, after obtaining the vascular path, local feature points, and Gaussian Laplace points of the DSA sequence image, a DSA image signal with clear contrast is selected. The position of the OCT lens (contrast ring) on ​​the blood vessel, the start point and the stop point of the observation channel in the DSA image are marked, and the path of the observation channel in the DSA image is solved.

[0049] The observation channel path of DSA sequence images is solved in batches by the equation of motion, and the estimated position of the development ring of DSA sequence images is obtained. The estimated position of the development ring and the position of the Gaussian Laplace point are matched with the cell bitmap of the development ring position of a selected DSA image signal with clear imaging, and the development ring position of DSA sequence images is obtained by self-learning template matching.

[0050] When the number of DSA sequence images is 45, the time spent on DSA processing is reduced to 1 / (45*3) of the time spent on sequence calculation before using the method described in this invention, thus greatly improving the processing efficiency.

[0051] The present invention also provides an apparatus based on the fusion development of DSA and OCT, such as Figure 2 As shown, the device includes:

[0052] Image acquisition unit 1 is used to synchronously and continuously acquire DSA sequence images and OCT sequence images;

[0053] Time stamping unit 2 is used to time stamp the acquired DSA sequence images and OCT sequence images;

[0054] Image processing unit 3 is used to distribute the acquired DSA sequence images equally to different GPU units according to the time sequence of the marking, and to calculate and mark the position of the developing ring in the DSA sequence image;

[0055] Image fusion unit 4 is used to synchronously present the OCT sequence image and the DSA sequence image after marking the position of the development ring on the same display according to the marked time.

[0056] In one specific implementation, the image acquisition unit 1 is the image acquisition card of the OCT device.

[0057] In one specific implementation, the GPU unit includes one or more of the following: a Hessian-enhanced multi-scale filtering subunit, a Gaussian-Laplacian point processing subunit, and a local feature point processing subunit. The Hessian-enhanced multi-scale filtering subunit is used to calculate the vascular path in the DSA sequence image. The Gaussian-Laplacian point processing subunit is used to calculate the Gaussian-Laplacian points in the DSA sequence image. The local feature point processing subunit is used to calculate local feature points in the DSA sequence image.

[0058] In one specific implementation, the GPU unit includes multiple multi-scale filtering processing sub-units, multiple Gaussian Laplacian point processing sub-units, and multiple local feature point processing sub-units.

[0059] In addition to the methods and devices described above, embodiments of the present invention also provide an electronic device, the electronic device comprising: a processor; and

[0060] A memory storing computer program instructions that, when executed by the processor, cause the processor to perform the method described above.

[0061] A processor can be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and can control other components in an electronic device to perform desired functions.

[0062] The memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor may execute the program instructions to implement the above-described method based on DSA and OCT fusion imaging and / or other desired functions. Various contents, such as prediction results, may also be stored in the computer-readable storage medium.

[0063] The computer program can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0064] Furthermore, the memory can employ any combination of one or more readable media. A readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0065] Example

[0066] Step 1: Signal Acquisition. Connect the integrated device and DSA equipment via HDMI cable. After preparatory operations such as catheter intervention, contrast agent injection, and connection of the PCU motion control unit, begin driving the PCU motion control unit to retract the catheter. Simultaneously, drive the acquisition card to acquire the sequence image signals from the DSA equipment. Assume the frequency of the DSA sequence image signal is 15fps (there are also 30fps, 45fps, etc.), the frequency of the OCT sequence image signal is 90fps (there are also 60fps, 120fps, etc.), and the retraction time is 3 seconds. Simultaneously acquire 45 DSA sequence image signals and 270 OCT sequence image signals. Therefore, the correspondence is 1:6, with one DSA sequence image signal corresponding to 6 OCT sequence image signals.

[0067] Step 2: Feature Extraction and GPU Parallel Acceleration. Due to the independence of the Hessian-enhanced multi-scale filtering module, the Gaussian-Laplacian point cloud module, and the local feature point cloud module, multiple working units (threads) can be allocated within each stream structure of the GPU. These threads can perform sub-computations synchronously without interference. Simultaneously, the DSA sequence image signals are divided into data blocks. A stream structure exists in the GPU, allowing computations in different stream structures to occur simultaneously, while computations within the same stream are executed sequentially. Furthermore, the 45 DSA sequence image signals are independent of each other, and the computational tasks are highly repetitive. They are assigned to grids and blocks corresponding to different stream structures of the GPU. Each grid contains NB blocks, and each block contains NT threads. By segmenting the data and assigning it to each thread, along with the computations each thread needs to perform, multi-threaded parallel computation of the N image signals is achieved.

[0068] Therefore, assuming the time for a single subtask is Tw, the traditional CPU sequential execution time would require 3*45*Tw to complete the calculation of the three subtasks for the current DSA sequence of 45 image signals, while with GPU parallel acceleration, the calculation time would only be Tw.

[0069] Step 3: Draw the observation channel path P and the position R of the contrast ring for a single image signal. Select a clearly angiographic DSA image signal from the DSA sequence image signal on the integrated device. Mark the position R of the OCT lens (contrast ring) and the positions of the observation channel start point S and stop point E on the blood vessel in this DSA image. Using the Hessian-enhanced multi-scale filtered blood vessel mask as the boundary and the local feature point cloud (the point with the largest gray value in the block matrix) as the map of the observation channel, obtain the observation channel path for a single image through Dijkstra's shortest path planning.

[0070] Compared to the centerline method of topology refinement, this method takes into account the curvature of the catheter and the fact that the catheter is not located in the center of the blood vessel, and increases the probability of the appearance of the imaging ring in the catheter, providing a reference for subsequent imaging ring identification and increasing the identification accuracy.

[0071] Step 4: Draw the observation channel path PN of the DSA sequence image signal. Due to the translational stretching and contraction of blood vessels caused by cardiac pacing, the changes between consecutive frames of the DSA sequence image are significant. Therefore, it is necessary to solve the motion equation of each image to obtain the observation channel path of the DSA sequence image signal. First, based on the characteristics that the stop point E is located at the proximal end of the main blood vessel, close to the heart and with many easily identifiable main branches, template matching is used to obtain the positions EN of all stop points in the DSA sequence image signal, and the offsets XN and YN of all stop points EN relative to the marked stop point E are obtained. Then, affine matching is performed on the marked observation channel path to obtain the angle offset A and scale C of the DSA sequence image. Combined with the current image signal offset X and Y relative to the marked signal offsets, the affine matrix D is obtained, where D is the motion equation of a single image. Next, the motion equation is solved in parallel to obtain all motion equations DN of the DSA sequence image signal. The starting point SN is solved by substituting the coordinates of the starting point S into the motion equation DN. Finally, using the starting point SN and the stop point EN, the observation channel path PN of the DSA sequence image signal is obtained according to Step 3.

[0072] Step 5: Determine the location RN of the imaging loop. RN is a point on the observation channel path PN. Several reference points are obtained by combining the estimated points on PN calculated from the data and the Gaussian Laplacian points bounded by the Hessian-enhanced multi-scale filtered vascular mask. A template for the imaging loop is obtained through iterative learning, and multi-scale template matching is used to match the similarity between the reference points of the DSA sequence image signal and the imaging loop learning template. The location RN of the imaging loop is then determined, and the similarity is verified using algorithms such as the similarity between preceding and following frames and the labeled template, and anti-backtracking algorithms for the imaging loop.

[0073] Step 6: Interpolation point estimation of DSA sequence image signals. Since one DSA sequence image signal corresponds to 6 OCT sequence image signals, it is necessary to fill in the positions of the contrast rings on the DSA sequence image signal. Therefore, it is necessary to insert the positions of the contrast rings of the remaining 5 images into the current DSA sequence image signal. The positions of the contrast rings of the remaining 5 images are obtained by evenly distributing the movement distance using the position information of the preceding and following frames. In addition, when encountering blood vessel occlusion, it is necessary to provide the estimated points of the contrast rings. This is obtained by calculating the similarity between local feature points with Hessian-enhanced multi-scale filtered blood vessel masks as boundaries and the contrast ring learning template. Finally, the obtained 6*RN positions of the contrast rings are marked on the DSA sequence image, and the corresponding OCT is played synchronously.

[0074] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0075] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0076] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0077] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0078] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for imaging based on the fusion of DSA and OCT, characterized in that, The method includes: Simultaneously and continuously acquire DSA and OCT sequence images; Time-stamping is performed on the acquired DSA and OCT sequence images; The acquired DSA sequence images are evenly distributed to different GPU units for processing according to the chronological order of the marking. The processing yields the vascular path, local feature points, and Gaussian Laplacian points of the DSA sequence images, which are used to calculate and mark the position of the imaging ring in the DSA sequence images. The OCT sequence image and the DSA sequence image after marking the position of the development ring are synchronously displayed on the same display according to the marked time. After obtaining the vascular path, local feature points, and Gaussian Laplace points of the DSA sequence image, the process further includes: drawing the observation channel path of the DSA sequence image signal, which includes the following steps: The positions of all stop points in the DSA sequence image signal are obtained by template matching, and the offset of each stop point relative to the stop point of the labeled image signal is obtained. By performing affine matching on the marked observation channel path, the angle offset and scale of the DSA sequence image signal are obtained. Combined with the offset of the current image signal relative to the stop point of the marked image signal, the affine matrix is ​​obtained as the motion equation of the current image. Parallel solution of motion equations yields all motion equations for the DSA sequence image signal. The starting point is solved by substituting the coordinates of the starting point into the motion equations. The observation channel paths of the DSA sequence image signal are derived by using the start and stop points.

2. The method according to claim 1, characterized in that, The processing method is selected from one or more of the following: Hessian-enhanced multi-scale filtering, Gaussian Laplacian point processing, and local feature point processing.

3. An apparatus based on the fusion development of DSA and OCT, characterized in that, The device includes: An image acquisition unit is used to simultaneously and continuously acquire DSA sequence images and OCT sequence images; A time-stamping unit is used to time-stamp the acquired DSA and OCT sequence images; An image processing unit is used to distribute the acquired DSA sequence images equally to different GPU units according to the time sequence of the marking. The processing obtains the vascular path, local feature points and Gaussian Laplacian points of the DSA sequence images to calculate and mark the position of the imaging ring in the DSA sequence images. An image fusion unit is used to synchronously present OCT sequence images and DSA sequence images marked with the development ring position on the same display according to the marked time. The image processing unit, after processing the blood vessel path, local feature points, and Gaussian Laplacian points of the DSA sequence image, further includes: drawing the observation channel path of the DSA sequence image signal, which includes the following steps: The positions of all stop points in the DSA sequence image signal are obtained by template matching, and the offset of each stop point relative to the stop point of the labeled image signal is obtained. The angle offset and scale of the DSA sequence image signal are obtained by performing affine matching on the marked observation channel path. Combined with the offset of the current image signal relative to the stop point of the marked image signal, the affine matrix is ​​obtained as the motion equation of the current image. Parallel solution of motion equations yields all motion equations for the DSA sequence image signal. The starting point is solved by substituting the coordinates of the starting point into the motion equations. The observation channel paths of the DSA sequence image signal are derived by using the start and stop points.

4. The apparatus according to claim 3, characterized in that, The GPU unit includes one or more of the following: Hessian-enhanced multi-scale filtering subunit, Gaussian-Laplace point processing subunit, and local feature point processing subunit.

5. The apparatus according to claim 4, characterized in that, The Hessian-enhanced multi-scale filtering subunit is used to calculate the vascular path in the DSA sequence image.

6. The apparatus according to claim 4, characterized in that, The Gaussian Laplacian point processing subunit is used to calculate the Gaussian Laplacian points of the DSA sequence image.

7. The apparatus according to claim 4, characterized in that, The local feature point processing subunit is used to calculate the local feature points of the DSA sequence image.

8. An electronic device, characterized in that, The electronic device includes: Processor; and A memory storing computer program instructions that, when executed by the processor, cause the processor to perform the method as described in any one of claims 1-2.

Images