IVUS image and angiography image registration method, device and equipment and storage medium

Abstract

The invention relates to an IVUS image and angiography image registration method and device, equipment and a storage medium. The method comprises the steps of obtaining multiple frames of IVUS images with branch blood vessels, obtaining contrast images corresponding to the multiple frames of IVUS images, calculating first diameters of the branch blood vessels in the multiple frames of IVUS images to obtain a first diameter set of all the branch blood vessels in the multiple frames of IVUS images, calculating second diameters of the branch blood vessels in the contrast images to obtain a second diameter set of all the branch blood vessels in the multiple frames of IVUS images to obtain a second diameter set of all the branch blood vessels in the contrast images; and obtaining a second diameter set of all branch blood vessels in the angiography image, traversing registration schemes of the first diameter set and the second diameter set, calculating an error value of each registration scheme, and taking the registration scheme with the minimum error value as a target registration scheme. According to the invention, the position of the IVUS image can be matched with the exact position of the IVUS image on the blood vessel route map obtained by coronary artery angiography.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a registration method, device, equipment and storage medium for IVUS images and angiography images. Background Art

[0002] Coronary angiography images can reflect the two-dimensional anatomical morphology of coronary arteries, so as to determine the specific location and degree of stenosis of coronary heart disease, but it is difficult to clearly display the abnormal location in the vascular wall and cannot provide information such as the morphology and properties of plaques. Intravascular ultrasound (IVUS) and other intravascular images can clearly display the cross-section of the blood vessels and reflect the organizational structure and fine geometric anatomical morphology of the vascular wall, but intravascular images cannot well reflect the topological structure information of the blood vessels. Based on the functional complementarity of coronary angiography images and intravascular images, the coronary angiography images and intravascular images are registered and fused, which can not only observe the spatial position of the coronary vessels, but also obtain the quantitative data of the inner and outer membranes and plaques of the vascular segments of interest.

[0003] At present, the existing technology reconstructs a three-dimensional vascular image through two angiography images in different body positions, and then manually finds the vascular segment of interest and aligns it with IVUS. Although this method can achieve better results using a double-sided angiography system, the process is complicated and the popularity rate is very low. Due to the low similarity between angiography images and IVUS images, the main limitation of matching angiography images and IVUS images is that the position of the obtained IVUS image is not related to its exact position on the vascular roadmap obtained by angiography.

[0004] Therefore, how to achieve matching of the position of the IVUS image with its exact position on the vascular roadmap obtained by angiography has become a technical problem that needs to be urgently solved by those skilled in the art. Summary of the invention

[0005] In view of the above, the present application provides a method, device, equipment and storage medium for registration of IVUS images and angiography images, the purpose of which is to solve the above technical problems.

[0006] In a first aspect, the present application provides a method for registering an IVUS image with an angiography image, the method comprising:

[0007] Acquire multiple frames of IVUS images with branched blood vessels, and acquire angiography images corresponding to the multiple frames of IVUS images;

[0008] Calculating the first diameter of the branch blood vessels in the multiple frames of IVUS images to obtain a first diameter set of all branch blood vessels in the multiple frames of IVUS images;

[0009] Calculating the second diameter of the branch blood vessels in the angiography image to obtain a second diameter set of all branch blood vessels in the angiography image;

[0010] The registration schemes of the first diameter set and the second diameter set are traversed, an error value of each registration scheme is calculated, and a registration scheme with the smallest error value is taken as a target registration scheme.

[0011] In a second aspect, the present application provides a registration device for an IVUS image and an angiography image, the device comprising:

[0012] An acquisition module: used for acquiring multiple frames of IVUS images with branched blood vessels, and acquiring angiography images corresponding to the multiple frames of IVUS images;

[0013] A first calculation module: used to calculate the first diameter of the branch blood vessels in the multiple frames of IVUS images, and obtain the first diameter set of all branch blood vessels in the multiple frames of IVUS images;

[0014] A second calculation module: used to calculate the second diameter of the branch blood vessels in the angiography image, and obtain a second diameter set of all branch blood vessels in the angiography image;

[0015] Registration module: used for traversing the registration schemes of the first diameter set and the second diameter set, calculating the error value of each registration scheme, and taking the registration scheme with the smallest error value as the target registration scheme.

[0016] In a third aspect, the present application provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0017] Memory, used to store computer programs;

[0018] The processor is used to implement the IVUS image and angiography image registration method described in any embodiment of the first aspect when executing the program stored in the memory.

[0019] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for registering an IVUS image with an angiography image as described in any one of the embodiments of the first aspect is implemented.

[0020] The above technical solution provided by the embodiment of the present application has the following advantages compared with the prior art:

[0021] The present application obtains multiple frames of IVUS images with branched blood vessels and angiography images corresponding to the multiple frames of IVUS images, calculates a first diameter set of all branched blood vessels in the multiple frames of IVUS images, and calculates a second diameter set of all branched blood vessels in the angiography images, and then traverses the registration schemes of the first diameter set and the second diameter set. Since the relative position error value of the registration scheme can be calculated by using the branch blood vessel diameter characteristics of the IVUS image and the angiography image, the registration scheme with the smallest error value is used as the target registration scheme, which can achieve matching of the position of the IVUS image with its exact position on the vascular roadmap obtained by coronary angiography. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0024] Figure 1 A schematic diagram of a flow chart of an embodiment of a registration method for IVUS images and angiography images of the present application;

[0025] Figure 2 It is a schematic diagram of an IVUS image with branched blood vessels in an embodiment of the present application;

[0026] Figure 3 It is a schematic diagram of performing registration of any number of branch vessels according to the branching order in an embodiment of the present application;

[0027] Figure 4 A schematic diagram of completing registration of an IVUS image and angiography image in an embodiment of the present application;

[0028] Figure 5 A schematic diagram of a module of a preferred embodiment of the registration device for IVUS images and angiography images of the present application;

[0029] Figure 6 A schematic diagram of a preferred embodiment of the electronic device of the present application;

[0030] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0032] It should be noted that the descriptions involving "first", "second", etc. in this application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0033] The present application provides a registration method for IVUS images and angiography images. Figure 1 , which is a schematic diagram of a method flow of an embodiment of the method for registering an IVUS image with an angiography image of the present application. The method can be performed by an electronic device, and the electronic device can be implemented by software and / or hardware. The method for registering an IVUS image with an angiography image includes:

[0034] Step S10: acquiring multiple frames of IVUS images with branched blood vessels, and acquiring angiography images corresponding to the multiple frames of IVUS images;

[0035] Step S20: calculating the first diameter of the branch blood vessels in the multiple frames of IVUS images, and obtaining a first diameter set of all branch blood vessels in the multiple frames of IVUS images;

[0036] Step S30: calculating the second diameter of the branch blood vessels in the angiography image, and obtaining a second diameter set of all branch blood vessels in the angiography image;

[0037] Step S40: traverse the registration schemes of the first diameter set and the second diameter set, calculate the error value of each registration scheme, and take the registration scheme with the smallest error value as the target registration scheme.

[0038] In this embodiment, the IVUS image is a cross-sectional image of the blood vessel developed by an electronic imaging system installed in the ultrasound catheter after a catheter equipped with a miniaturized ultrasonic transducer is sent into the blood vessel lumen. Therefore, the IVUS image can not only observe the morphology of the lumen, but also the structure or lesions of the vessel wall. The contrast image Agino refers to the angiographic image corresponding to the IVUS image. The angiographic image is obtained by introducing a contrast agent into the target blood vessel, eliminating the bone and soft tissue images on the angiographic film through a computer, and only displaying the image of the blood vessel on the film.

[0039] The working process of IVUS is as follows:

[0040] 1. Move the tip of the guiding catheter to the coronary ostium;

[0041] 2. Move the guide wire along the guiding catheter into the coronary artery (blood vessel);

[0042] 3. The IVUS catheter is moved along the guidewire into the coronary artery (blood vessel);

[0043] 4. The ultrasonic transducer in the IVUS catheter retracts and rotates to collect IVUS images.

[0044] IVUS images with branched vessels refer to images of blood vessels with branch points in IVUS images, such as Figure 2 As shown, it is a schematic diagram of an IVUS image with branched blood vessels in an embodiment of the present application. Acquiring multiple frames of IVUS images with branched blood vessels and corresponding angiography images can be acquired from a preset storage path. For example, a doctor observes the IVUS image set generated by IVUS during the withdrawal process with naked eyes, and stores the IVUS images with branched blood vessels in the preset storage path. Specifically, acquiring multiple frames of IVUS images with branched blood vessels includes:

[0045] Acquire an IVUS image set generated by the IVUS during the withdrawal process;

[0046] Using the pre-trained branch vessel recognition model, multiple frames of IVUS images with branch vessels are identified from the IVUS image set.

[0047] The branch vessel recognition model can be obtained based on convolutional neural network training. IVUS will generate an IVUS image set during the withdrawal process. Each frame of IVUS image in the IVUS image set is input into the branch vessel recognition model to obtain the recognition result of each frame of IVUS image. If the IVUS image is recognized as an IVUS image with branch vessels, it is retained. If the IVUS image is not recognized as an IVUS image with branch vessels, it is filtered. Through identification by the branch vessel recognition model, IVUS images with branch vessels can be screened out more efficiently.

[0048] For the same branch vessel, multiple consecutive IVUS images with branch inflow can be identified. Assuming that when the branch begins to inflow, the frame number of the identified IVUS image is recorded as BranchVessel start , then when the branch is completed, the frame number of the IVUS image identified is recorded as BranchVessel end , the difference between the starting position and the ending position of the branch confluence is considered to be the diameter of the branch vessel, and the diameter of each branch vessel in the multi-frame IVUS image can be calculated (recorded as the first diameter), so that the first diameter set of all branch vessels in the multi-frame IVUS image can be obtained. The first diameter set includes the diameter of each branch vessel in the multi-frame IVUS image. The first diameter set is [b 1 、b 2 ……b n ]express.

[0049] Specifically, calculating the first diameter of the branch blood vessels in the multiple frames of IVUS images includes:

[0050] Determine the initial frame number and the end frame number of the IVUS image corresponding to the branch blood vessels respectively;

[0051] The first diameter of the branch blood vessels in the multiple frames of IVUS images is calculated according to the initial frame number, the end frame number, the retraction speed and the frame rate of the IVUS.

[0052] The initial frame number of the IVUS image refers to the frame number of the IVUS image identified when the branch begins to merge, and the end frame number of the IVUS image refers to the frame number of the IVUS image identified when the branch ends to merge. Based on the initial frame number, the end frame number, the IVUS retraction speed and the frame rate, the first diameter of each branch blood vessel in multiple frames of IVUS images can be calculated.

[0053] The step of calculating the first diameter of the branch blood vessels in the multiple-frame IVUS images according to the initial frame number, the end frame number, the IVUS retraction speed and the frame rate includes:

[0054] The first diameter of the branch blood vessels in the multi-frame IVUS images is calculated using the following formula, which includes:

[0055]

[0056] Among them, IVUS_Branch_Diameter represents the first diameter of the branch vessel, BranchVessel start Indicates the initial frame number, BranchVessel endrepresents the end frame number, PullbackSpeed ​​represents the pullback speed, and FrameRate represents the frame rate.

[0057] Then calculate the diameter of each branch vessel in the angiography image (recorded as the second diameter) to obtain the second diameter set of all branch vessels in the angiography image. The second diameter set includes the diameter of each branch vessel in the angiography image. The second diameter set is expressed as [d 1 ,d 2 ……d m ] indicates that the diameter of the branch blood vessels in the angiography image can be calculated by using a picture archiving and communication system (PACS) or a video acquisition card.

[0058] Specifically, calculating the second diameter of the branch blood vessel in the angiography image includes:

[0059] Calculating a second diameter of the branch blood vessel in the angiography image using an image archiving and communication system;

[0060] Alternatively, the diameter pixel size of the branch blood vessel in the angiography image is calculated, and the second diameter of the branch blood vessel in the angiography image is calculated according to the actual physical size of the pixel and the diameter pixel size.

[0061] The actual physical diameter of the branch blood vessels can be calculated using the PACS system. 1 , in millimeters (mm), or, using a video capture card, you can usually only calculate the pixel size of the branch vessel diameter r 2 , in pixels, combined with the actual physical size of the pixel ImagePixelSpacing (in millimeters), the second diameter of each branch vessel in the angiography image can also be calculated. 1 and r 2 The conversion formula is r 1 =r 2 *ImagePixelSpacing.

[0062] After obtaining the first diameter set and the second diameter set, the registration schemes of the first diameter set and the second diameter set are traversed. In order to traverse all branch vessel matching schemes, the idea of ​​permutation and combination can be used to select multiple branch vessels from the branch vessels of the angiography image and the branch vessels of the IVUS image for matching. Specifically, the registration schemes of the first diameter set and the second diameter set are traversed, and the error value of each registration scheme is calculated, including:

[0063] Determine a minimum value K of the number of branch blood vessels in the multiple-frame IVUS image and the number of branch blood vessels in the angiography image;

[0064] The registration schemes corresponding to 2 to K branch vessels in the first diameter set and the second diameter set are traversed in sequence, and the error value of each registration scheme is calculated.

[0065] The first diameter set is denoted as [b 1 、b 2 ……b n ], the second diameter is denoted as [d 1 ,d 2 ……d m ], first determine the minimum value K (K = min[n,m]) of the number of branch vessels in the multi-frame IVUS image and the number of branch vessels in the angiography image, such as Figure 3 As shown, it is a schematic diagram of performing registration of any number of branch vessels in the order of branches in an embodiment of the present application, but the number of registration schemes does not exceed the K value, that is, the number of branch vessel registration schemes ranges from [2, K], and the registration schemes corresponding to 2 to K branch vessels in the first diameter set and the second diameter set are traversed in turn, and then the error value of each registration scheme is calculated, and the registration scheme with the smallest error value in each registration scheme is used as the target registration scheme, and the IVUS image is registered with the angiography image using the target registration scheme. That is, the IVUS image is segmented and matched according to the branch vessels, and each segment evenly distributes the IVUS image at equal intervals on the center line of the retracement path vessel, thereby completing the registration and achieving the match between the position of the IVUS image and its exact position on the vascular roadmap obtained by coronary angiography. As shown Figure 4 As shown, it is a schematic diagram of the registration of the IVUS image and the angiography image in the embodiment of the present application.

[0066] The calculating of the error value of each registration scheme includes: calculating the error value of the registration scheme corresponding to the k branch vessels using the following formula, the formula including:

[0067] diameter_error=(d 1 -b 1 ) 2 +(d 2 -b 2 ) 2 +...+(d k -b k ) 2

[0068] Among them, diameter_error represents the error value of the registration scheme corresponding to n branch vessels, b 1 、b 2 , …, b krepresents the first diameters of k branch vessels selected from the first diameter set, d 1 d 2 ,…,d k represents the second diameters of k branch vessels selected from the second diameter set.

[0069] Taking the calculation of the error value of the registration scheme corresponding to three branch vessels as an example, the formula is:

[0070] diameter_error=(d 2 -b 1 ) 2 +(d 3 -b 3 ) 2 +(d m -b n ) 2

[0071] Among them, diameter_error represents the error value of the registration scheme corresponding to the three branch vessels, b 1 、b 3 and b n represents the first diameters of the three branch vessels selected from the first diameter set, d 2 d 3 and m represents the second diameters of three branch vessels selected from the second diameter set.

[0072] Taking 3 pairs of branch vessels for registration as an example, the diameters of the corresponding branch vessels in the angiography image are [d2, d3, dm], and the diameters of the branch vessels in the IVUS image are [b1, b3, bn]. The relative error of the diameter can be calculated using a PACS system or a video acquisition card. The actual physical length of the branch vessel diameter can be directly calculated using the PACS system. The unit is unified in millimeters (mm), so the error can be directly calculated. The calculation formula is as follows:

[0073] diameter_error=(d 2 -b 1 ) 2 +(d 3 -b 3 ) 2 +(d m -b n ) 2

[0074] The use of video capture cards cannot directly obtain the Digital Imaging and Communications in Medicine (DICOM) information of the corresponding image, and cannot directly calculate the actual physical length of the branch blood vessels. Instead, it uses pixels as the unit, so the error cannot be directly calculated. The relative proportional relationship between the diameters is used for calculation. First, the selected data needs to be normalized each time. The normalization formula is as follows:

[0075]

[0076] Then the error is calculated based on the normalized branch vessel diameter, and the calculation formula is as follows:

[0077] diameter_error=(d' 2 -b 1 ') 2 +(d 3 '-b 3 ') 2 +(d' m -b' n ) 2

[0078] Among them, diameter_error represents the error value of the registration scheme corresponding to the three branch vessels, b' 1 , b' 3 and b' n represents the normalized value of the first diameters of the three branch vessels selected from the first diameter set, d' 2 , d' 3 and d' m represents the normalized value of the second diameters of three branch vessels selected from the second diameter set.

[0079] The actual physical length of the branch vessel in millimeters can also be indirectly calculated using the guiding catheter features on the angiography image using a video capture card. The angiography image is segmented into regions of interest by annotating the guiding catheter feature training set to obtain a guiding catheter feature image. Based on the guiding catheter feature image and the inherent size of the guiding catheter, the actual physical size of each pixel of the angiography image is calculated. The actual physical size of each branch vessel is then calculated based on the actual physical size of the pixels of the angiography image. Assuming that the diameter of the guiding catheter obtained by analyzing the guiding catheter feature image is R / pixel, and then combined with the actual size of the guiding catheter S / mm, the actual physical size of each pixel of the angiography image, S / R (mm), is calculated. The diameter error calculation formula is as follows:

[0080] diameter_error=(d2 *S / Rb 1 ) 2 +(d 3 *S / Rb 3 ) 2 +(d m *S / Rb n ) 2

[0081] Among them, diameter_error represents the error value of the registration scheme corresponding to the three branch vessels, b 1 、b 3 and b n represents the first diameters of the three branch vessels selected from the first diameter set, d 2 d 3 and m represents the second diameters of three branch vessels selected from the second diameter set.

[0082] Reference Figure 5 , which is a schematic diagram of the functional modules of the IVUS image and angiography image registration device 100 of the present application.

[0083] The IVUS image and angiography image registration device 100 described in the present application can be installed in an electronic device. According to the functions to be implemented, the IVUS image and angiography image registration device 100 may include an acquisition module 110, a first calculation module 120, a second calculation module 130 and a registration module 140. The module described in the present application may also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0084] In this embodiment, the functions of each module / unit are as follows:

[0085] Acquisition module 110: used to acquire multiple frames of IVUS images with branched blood vessels, and acquire angiography images corresponding to the multiple frames of IVUS images;

[0086] A first calculation module 120 is used to calculate the first diameter of the branch blood vessels in the multiple frames of IVUS images, and obtain a first diameter set of all branch blood vessels in the multiple frames of IVUS images;

[0087] The second calculation module 130 is used to calculate the second diameter of the branch blood vessels in the angiography image, and obtain the second diameter set of all the branch blood vessels in the angiography image;

[0088] The registration module 140 is used to traverse the registration schemes of the first diameter set and the second diameter set, calculate the error value of each registration scheme, and take the registration scheme with the smallest error value as the target registration scheme.

[0089] In one embodiment, acquiring multiple frames of IVUS images with branched blood vessels includes:

[0090] Acquire an IVUS image set generated by the IVUS during the withdrawal process;

[0091] Using the pre-trained branch vessel recognition model, multiple frames of IVUS images with branch vessels are identified from the IVUS image set.

[0092] In one embodiment, calculating the first diameter of the branch blood vessels in the multiple frames of IVUS images includes:

[0093] Determine the initial frame number and the end frame number of the IVUS image corresponding to the branch blood vessels respectively;

[0094] The first diameter of the branch blood vessels in the multiple frames of IVUS images is calculated according to the initial frame number, the end frame number, the retraction speed and the frame rate of the IVUS.

[0095] In one embodiment, the calculating of the first diameter of the branch blood vessel in the multi-frame IVUS image according to the initial frame number, the end frame number, the IVUS retraction speed and the frame rate includes:

[0096] The first diameter of the branch blood vessels in the multi-frame IVUS images is calculated using the following formula, which includes:

[0097]

[0098] Among them, IVUS_Branch_Diameter represents the first diameter of the branch vessel, BranchVessel start Indicates the initial frame number, BranchVessel end represents the end frame number, PullbackSpeed ​​represents the pullback speed, and FrameRate represents the frame rate.

[0099] In one embodiment, calculating the second diameter of the branch blood vessel in the angiography image includes:

[0100] Calculating a second diameter of the branch blood vessel in the angiography image using an image archiving and communication system;

[0101] Alternatively, the actual physical size of the pixel is calculated based on the characteristic image of the guiding catheter and the inherent size of the guiding catheter, and the diameter pixel size of the branch blood vessel in the angiography image is calculated; based on the actual physical size of the pixel and the diameter pixel size, the second diameter of the branch blood vessel in the angiography image is calculated.

[0102] In one embodiment, the actual physical size of the pixel is calculated based on the characteristic image of the guiding catheter and the intrinsic size of the guiding catheter, including:

[0103] Performing region of interest image segmentation on the angiography image to obtain the guiding catheter feature image;

[0104] Calculating the actual physical size of each pixel of the angiography image based on the characteristic image of the guiding catheter and the intrinsic size of the guiding catheter;

[0105] The actual physical size of the branch blood vessel is calculated based on the actual physical size of each pixel of the angiography image.

[0106] In one embodiment, traversing the registration schemes of the first diameter set and the second diameter set and calculating the error value of each registration scheme includes:

[0107] Determine a minimum value K of the number of branch blood vessels in the multiple-frame IVUS image and the number of branch blood vessels in the angiography image;

[0108] The registration schemes corresponding to 2 to K branch vessels in the first diameter set and the second diameter set are traversed in sequence, and the error value of each registration scheme is calculated.

[0109] In one embodiment, the calculating the error value of each registration scheme includes: calculating the error value of the registration scheme corresponding to the k branch vessels using the following formula, the formula including:

[0110] diameter_error=(d 1 -b 1 ) 2 +(d 2 -b 2 ) 2 +...+(d k -b k ) 2

[0111] Among them, diameter_error represents the error value of the registration scheme corresponding to n branch vessels, b 1 、b 2 , …, b k represents the first diameters of k branch vessels selected from the first diameter set, d 1 d2 ,…,d k represents the second diameters of k branch vessels selected from the second diameter set.

[0112] Reference Figure 6 , which is a schematic diagram of a preferred embodiment of the electronic device 1 of the present application.

[0113] The electronic device 1 includes but is not limited to: a memory 11, a processor 12, a display 13 and a communication interface 14. The electronic device 1 can be connected to a network through the communication interface 14. The network can be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, a call network, etc.

[0114] Among them, the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a hard disk or memory of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in hard disk equipped with the electronic device 1, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 11 can also include both the internal storage unit of the electronic device 1 and its external storage device. In this embodiment, the memory 11 is generally used to store the operating system and various computer programs installed on the electronic device 1, such as the program code of the registration program 10 of IVUS images and angiography images, etc. In addition, the memory 11 can also be used to temporarily store various types of data that have been output or are to be output.

[0115] In some embodiments, the processor 12 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 12 is generally used to control the overall operation of the electronic device 1, such as performing control and processing related to data interaction or communication. In this embodiment, the processor 12 is used to run the program code stored in the memory 11 or process data, such as running the program code of the registration program 10 of the IVUS image and the angiography image.

[0116] The display 13 may be referred to as a display screen or a display unit. In some embodiments, the display 13 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an organic light-emitting diode (OLED) touch device, etc. The display 13 is used to display information processed in the electronic device 1 and to display a visual working interface.

[0117] The communication interface 14 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), and the communication interface 14 is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0118] Figure 6 Only the electronic device 1 with the components 11 - 14 and the registration procedure 10 of the IVUS image and the angiographic image is shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0119] In the above embodiment, the processor 12 can implement the following steps when executing the IVUS image and angiography image registration program 10 stored in the memory 11:

[0120] Acquire multiple frames of IVUS images with branched blood vessels, and acquire angiography images corresponding to the multiple frames of IVUS images;

[0121] Calculating the first diameter of the branch blood vessels in the multiple frames of IVUS images to obtain a first diameter set of all branch blood vessels in the multiple frames of IVUS images;

[0122] Calculating the second diameter of the branch blood vessels in the angiography image to obtain a second diameter set of all branch blood vessels in the angiography image;

[0123] The registration schemes of the first diameter set and the second diameter set are traversed, an error value of each registration scheme is calculated, and a registration scheme with the smallest error value is taken as a target registration scheme.

[0124] The storage device may be the memory 11 of the electronic device 1 , or may be another storage device that is communicatively connected to the electronic device 1 .

[0125] For a detailed description of the above steps, please refer to the above Figure 5 Functional module diagram of an embodiment of an IVUS image and angiography image registration device 100 and Figure 1 Description of a flowchart of an embodiment of a method for registering IVUS images with contrast images.

[0126] In addition, the embodiment of the present application also proposes a computer-readable storage medium, which can be non-volatile or volatile. The computer-readable storage medium can be any one or any combination of a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, etc. The computer-readable storage medium includes a data storage area and a program storage area, and the program storage area stores a registration program 10 for IVUS images and angiography images. When the registration program 10 for IVUS images and angiography images is executed by a processor, the following operations are implemented:

[0127] Acquire multiple frames of IVUS images with branched blood vessels, and acquire angiography images corresponding to the multiple frames of IVUS images;

[0128] Calculating the first diameter of the branch blood vessels in the multiple frames of IVUS images to obtain a first diameter set of all branch blood vessels in the multiple frames of IVUS images;

[0129] Calculating the second diameter of the branch blood vessels in the angiography image to obtain a second diameter set of all branch blood vessels in the angiography image;

[0130] The registration schemes of the first diameter set and the second diameter set are traversed, an error value of each registration scheme is calculated, and a registration scheme with the smallest error value is taken as a target registration scheme.

[0131] The specific implementation of the computer-readable storage medium of the present application is roughly the same as the specific implementation of the above-mentioned IVUS image and angiography image registration method, and will not be repeated here.

[0132] It should be noted that the serial numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof herein are intended to cover non-exclusive inclusion, so that the process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, device, article or method including the element.

[0133] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware simulation platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, an electronic device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0134] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A registration method for IVUS images and angiography images, characterized in that: The method comprises: Acquire multiple frames of IVUS images with branched blood vessels, and acquire angiography images corresponding to the multiple frames of IVUS images; Calculating the first diameter of the branch blood vessels in the multiple frames of IVUS images to obtain a first diameter set of all branch blood vessels in the multiple frames of IVUS images; Calculating the second diameter of the branch blood vessels in the angiography image to obtain a second diameter set of all branch blood vessels in the angiography image; The registration schemes of the first diameter set and the second diameter set are traversed, an error value of each registration scheme is calculated, and a registration scheme with the smallest error value is taken as a target registration scheme.

2. The registration method of IVUS image and angiography image according to claim 1, characterized in that: The method of acquiring a multi-frame IVUS image with branched blood vessels comprises: Acquire an IVUS image set generated by the IVUS during the withdrawal process; Using the pre-trained branch vessel recognition model, multiple frames of IVUS images with branch vessels are identified from the IVUS image set.

3. The registration method of IVUS image and angiography image according to claim 1, characterized in that: The step of calculating the first diameter of the branch blood vessels in the multiple frames of IVUS images comprises: Determine the initial frame number and the end frame number of the IVUS image corresponding to the branch blood vessels respectively; The first diameter of the branch blood vessels in the multiple frames of IVUS images is calculated according to the initial frame number, the end frame number, the retraction speed and the frame rate of the IVUS.

4. The registration method of IVUS image and angiography image according to claim 3, characterized in that: The calculating, according to the initial frame number, the end frame number, the IVUS withdrawal speed and the frame rate, a first diameter of the branch blood vessel in the multi-frame IVUS image comprises: The first diameter of the branch blood vessels in the multi-frame IVUS images is calculated using the following formula, which includes: Among them, IVUS_Branch_Diameter represents the first diameter of the branch vessel, BranchVessel start Indicates the initial frame number, BranchVessel end represents the end frame number, PullbackSpeed ​​represents the pullback speed, and FrameRate represents the frame rate.

5. The registration method of IVUS image and angiography image according to claim 1, characterized in that: The step of calculating the second diameter of the branch blood vessel in the angiography image comprises: Calculating a second diameter of the branch blood vessel in the angiography image using an image archiving and communication system; Alternatively, the actual physical size of the pixel is calculated based on the characteristic image of the guiding catheter and the inherent size of the guiding catheter, and the diameter pixel size of the branch blood vessel in the angiography image is calculated; based on the actual physical size of the pixel and the diameter pixel size, the second diameter of the branch blood vessel in the angiography image is calculated.

6. The registration method of IVUS image and angiography image according to claim 5, characterized in that: The calculating the actual physical size of the pixel based on the characteristic image of the guiding catheter and the intrinsic size of the guiding catheter includes: Performing region of interest image segmentation on the angiography image to obtain a guiding catheter feature image; Calculating the actual physical size of each pixel of the angiography image based on the characteristic image of the guiding catheter and the intrinsic size of the guiding catheter; The actual physical size of the branch blood vessel is calculated based on the actual physical size of each pixel of the angiography image.

7. The registration method of IVUS image and angiography image according to claim 1, characterized in that: The traversing the registration schemes of the first diameter set and the second diameter set and calculating the error value of each registration scheme includes: Determine a minimum value K of the number of branch blood vessels in the multiple-frame IVUS image and the number of branch blood vessels in the angiography image; The registration schemes corresponding to 2 to K branch vessels in the first diameter set and the second diameter set are traversed in sequence, and the error value of each registration scheme is calculated.

8. The registration method of IVUS image and angiography image according to claim 7, characterized in that: The calculating the error value of each registration scheme includes: calculating the error value of the registration scheme corresponding to the k branch blood vessels using the following formula, the formula including: diameter_error=(d1-b1) 2 +(d2-b2) 2 +...+(d k -b k ) 2 Among them, diameter_error represents the error value of the registration scheme corresponding to n branch vessels, b1, b2, ..., b k represents the first diameters of k branch vessels selected from the first diameter set, d1, d2, ..., d k represents the second diameters of k branch vessels selected from the second diameter set.

9. A registration device for IVUS images and angiography images, characterized in that: The device comprises: An acquisition module: used for acquiring multiple frames of IVUS images with branched blood vessels, and acquiring angiography images corresponding to the multiple frames of IVUS images; A first calculation module: used to calculate the first diameter of the branch blood vessels in the multiple frames of IVUS images, and obtain the first diameter set of all branch blood vessels in the multiple frames of IVUS images; A second calculation module: used to calculate the second diameter of the branch blood vessels in the angiography image, and obtain a second diameter set of all branch blood vessels in the angiography image; Registration module: used for traversing the registration schemes of the first diameter set and the second diameter set, calculating the error value of each registration scheme, and taking the registration scheme with the smallest error value as the target registration scheme.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the registration method of the IVUS image and the angiography image as described in any one of claims 1 to 8 when executing the program stored in the memory.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for registering an IVUS image with an angiography image according to any one of claims 1 to 8 is implemented.

Images