Image processing method and device, equipment and storage medium

Abstract

The invention relates to an image processing method and device, equipment and a storage medium. The method comprises the steps of obtaining multiple frames of IVUS images with branch blood vessels, obtaining angiography images corresponding to the multiple frames of IVUS images, calculating first position information of the branch blood vessels in the multiple frames of IVUS images, obtaining a first position information set of all the branch blood vessels in the multiple frames of IVUS images, calculating second position information of the branch blood vessels in the angiography images, and obtaining a second position information set of all the branch blood vessels in the angiography images; and obtaining a second position information set of all branch blood vessels in the angiography image, traversing registration schemes of the first position information set and the second position information set, calculating an error value of each registration scheme, and taking the registration scheme with the minimum error value as a target registration scheme. According to the invention, the position of the IVUS image can be matched with the exact position of the IVUS image on the blood vessel route map obtained by coronary artery angiography.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image processing method, apparatus, device and storage medium. Background Art

[0002] Coronary angiography images can reflect the two-dimensional anatomical morphology of coronary arteries, so as to determine the specific location and degree of stenosis of coronary heart disease, but it is difficult to clearly display the abnormal location in the vascular wall and cannot provide information such as the morphology and properties of plaques. Intravascular ultrasound (IVUS) and other intravascular images can clearly display the cross-section of the blood vessels and reflect the organizational structure and fine geometric anatomical morphology of the vascular wall, but intravascular images cannot well reflect the topological structure information of the blood vessels. Based on the functional complementarity of coronary angiography images and intravascular images, the coronary angiography images and intravascular images are registered and fused, which can not only observe the spatial position of the coronary vessels, but also obtain the quantitative data of the inner and outer membranes and plaques of the vascular segments of interest.

[0003] At present, the existing technology reconstructs a three-dimensional vascular image through two angiography images in different body positions, and then manually finds the vascular segment of interest and aligns it with IVUS. Although this method can achieve better results using a double-sided angiography system, the process is complicated and the popularity rate is very low. Due to the low similarity between angiography images and IVUS images, the main limitation of matching angiography images and IVUS images is that the position of the obtained IVUS image is not related to its exact position on the vascular roadmap obtained by angiography.

[0004] Therefore, how to achieve matching of the position of the IVUS image with its exact position on the vascular roadmap obtained by angiography has become a technical problem that needs to be urgently solved by those skilled in the art. Summary of the invention

[0005] In view of the above, the present application provides an image processing method, device, equipment and storage medium, the purpose of which is to solve the above technical problems.

[0006] In a first aspect, the present application provides an image processing method, the method comprising:

[0007] Acquire multiple frames of IVUS images with branched blood vessels, and acquire angiography images corresponding to the multiple frames of IVUS images;

[0008] Calculating the first position information of the branch blood vessels in the multiple frames of IVUS images to obtain a first position information set of all the branch blood vessels in the multiple frames of IVUS images;

[0009] Calculating second position information of the branch blood vessels in the angiography image to obtain a second position information set of all branch blood vessels in the angiography image;

[0010] The registration schemes of the first position information set and the second position information set are traversed, an error value of each registration scheme is calculated, and a registration scheme with the smallest error value is taken as a target registration scheme.

[0011] In a second aspect, the present application provides an image processing device, the device comprising:

[0012] An acquisition module: used for acquiring multiple frames of IVUS images with branched blood vessels, and acquiring angiography images corresponding to the multiple frames of IVUS images;

[0013] A first calculation module: used to calculate the first position information of the branch blood vessels in the multi-frame IVUS image, and obtain the first position information set of all the branch blood vessels in the multi-frame IVUS image;

[0014] A second calculation module: used to calculate the second position information of the branch blood vessels in the angiography image, and obtain a second position information set of all the branch blood vessels in the angiography image;

[0015] Registration module: used for traversing the registration schemes of the first position information set and the second position information set, calculating the error value of each registration scheme, and taking the registration scheme with the smallest error value as the target registration scheme.

[0016] In a third aspect, the present application provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0017] Memory, used to store computer programs;

[0018] The processor is used to implement the image processing method described in any embodiment of the first aspect when executing the program stored in the memory.

[0019] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image processing method as described in any embodiment of the first aspect is implemented.

[0020] The above technical solution provided by the embodiment of the present application has the following advantages compared with the prior art:

[0021] The present application obtains multiple frames of IVUS images with branched blood vessels and angiography images corresponding to the multiple frames of IVUS images, calculates a first position information set of all branched blood vessels in the multiple frames of IVUS images, and calculates a second position information set of all branched blood vessels in the angiography images, and then traverses the registration schemes of the first position information set and the second position information set. Since the relative position error value of the registration scheme can be calculated by using the branch blood vessel position features of the IVUS image and the angiography image, the registration scheme with the smallest error value is used as the target registration scheme, which can achieve matching of the position of the IVUS image with its exact position on the vascular roadmap obtained by coronary angiography. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0024] Figure 1 This is a flow chart of an embodiment of the image processing method of the present application;

[0025] Figure 2 A schematic diagram of an IVUS image with branched blood vessels in an embodiment of the present application;

[0026] Figure 3 For the embodiment of the present application, a schematic diagram of the center line of a single pixel coronary vessel tree is obtained from the segmented coronary vessel tree binary image;

[0027] Figure 4 This is a schematic diagram of the nodes of the branch blood vessels in the embodiment of the present application;

[0028] Figure 5 A schematic diagram of the IVUS withdrawal path of an embodiment of the present application;

[0029] Figure 6 A schematic diagram of performing registration of any number of branch vessels in branch order according to an embodiment of the present application;

[0030] Figure 7 A schematic diagram of completing registration of an IVUS image and angiography image in an embodiment of the present application;

[0031] Figure 8 A schematic diagram of modules of a preferred embodiment of the image processing device of the present application;

[0032] Fig. 9 A schematic diagram of a preferred embodiment of the electronic device of the present application;

[0033] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0035] It should be noted that the descriptions involving "first", "second", etc. in this application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0036] The present application provides an image processing method. Figure 1 FIG. 1 is a flow chart of an embodiment of the image processing method of the present application. The method may be executed by an electronic device, and the electronic device may be implemented by software and / or hardware. The image processing method includes:

[0037] Step S10: acquiring multiple frames of IVUS images with branched blood vessels, and acquiring angiography images corresponding to the multiple frames of IVUS images;

[0038] Step S20: calculating the first position information of the branch blood vessels in the multiple frames of IVUS images, and obtaining a first position information set of all the branch blood vessels in the multiple frames of IVUS images;

[0039] Step S30: calculating the second position information of the branch blood vessels in the angiography image, and obtaining a second position information set of all the branch blood vessels in the angiography image;

[0040] Step S40: traverse the registration schemes of the first position information set and the second position information set, calculate the error value of each registration scheme, and take the registration scheme with the smallest error value as the target registration scheme.

[0041] In this embodiment, the IVUS image is a cross-sectional image of the blood vessel developed by an electronic imaging system installed in the ultrasound catheter after a catheter equipped with a miniaturized ultrasonic transducer is sent into the blood vessel lumen. Therefore, the IVUS image can not only observe the morphology of the lumen, but also the structure or lesions of the vessel wall. The contrast image Agino refers to the angiographic image corresponding to the IVUS image. The angiographic image is obtained by introducing a contrast agent into the target blood vessel, eliminating the bone and soft tissue images on the angiographic film through a computer, and only displaying the image of the blood vessel on the film.

[0042] The working process of IVUS is as follows:

[0043] 1. Move the tip of the guiding catheter to the coronary ostium;

[0044] 2. Move the guide wire along the guiding catheter into the coronary artery (blood vessel);

[0045] 3. The IVUS catheter is moved along the guidewire into the coronary artery (blood vessel);

[0046] 4. The ultrasonic transducer in the IVUS catheter retracts and rotates to collect IVUS images.

[0047] IVUS images with branched vessels refer to images of blood vessels with branch points in IVUS images, such as Figure 2 As shown, it is a schematic diagram of an IVUS image with branched blood vessels in an embodiment of the present application. Acquiring multiple frames of IVUS images with branched blood vessels and corresponding angiography images can be acquired from a preset storage path. For example, a doctor observes the IVUS image set generated by IVUS during the withdrawal process with naked eyes, and stores the IVUS images with branched blood vessels in the preset storage path. Specifically, acquiring multiple frames of IVUS images with branched blood vessels includes:

[0048] Acquire an IVUS image set generated by the IVUS during the withdrawal process;

[0049] Using the pre-trained branch vessel recognition model, multiple frames of IVUS images with branch vessels are identified from the IVUS image set.

[0050] The branch vessel recognition model can be obtained based on convolutional neural network training. IVUS will generate an IVUS image set during the withdrawal process. Each frame of the IVUS image in the IVUS image set is input into the branch vessel recognition model to obtain the recognition result of each frame of the IVUS image. If the IVUS image is recognized as an IVUS image with branch vessels, it is retained. If the IVUS image is not recognized as an IVUS image with branch vessels, it is filtered. Through the branch vessel recognition model, IVUS images with branch vessels can be screened out more efficiently.

[0051] For the same branch vessel, multiple consecutive IVUS images with branch inflow can be identified. Assuming that when the branch begins to inflow, the frame number of the identified IVUS image is recorded as BranchVessel start , then when the branch is completed, the frame number of the IVUS image identified is recorded as BranchVessel end , the position of the branch vessel is the center position from the beginning to the end of the branch, and the actual physical distance between the position of the branch vessel and the starting point of the IVUS withdrawal is calculated (recorded as the first position information of the branch vessel), so that the first position information set of all branch vessels in the multi-frame IVUS image can be obtained. The first position information set includes the position information of each branch vessel in the multi-frame IVUS image. The first position information set is [a1, a2...a n ]express.

[0052] Specifically, calculating the first position information of the branch blood vessels in the multiple frames of IVUS images includes:

[0053] Determine the initial frame number and the end frame number of the IVUS image corresponding to the branch blood vessels respectively;

[0054] The IVUS retracement starting point or the coronary artery ostium branch vessel point is used as the reference point, wherein the coronary artery ostium branch vessel point refers to the midpoint of the branch vessel closest to the coronary artery ostium;

[0055] According to the initial frame number, the end frame number, the retraction speed and frame rate of the IVUS, the actual physical distance between the branch blood vessel and the reference point in the multi-frame IVUS image is calculated, and the actual physical distance between the branch blood vessel and the reference point is used as the first position information of the branch blood vessel. The initial frame number of the IVUS image refers to the frame number of the IVUS image identified when the branch begins to merge, and the end frame number of the IVUS image refers to the frame number of the IVUS image identified when the branch merges to the end. According to the initial frame number, the end frame number, the retraction speed and frame rate of the IVUS, the actual physical distance between the branch blood vessel and the reference point in the multi-frame IVUS image can be calculated.

[0056] The step of calculating the actual physical distance between the branch blood vessels and the reference point in the multiple-frame IVUS images according to the initial frame number, the end frame number, the IVUS retraction speed and the frame rate includes:

[0057] The position information of the branch blood vessels in the multi-frame IVUS images is calculated using the following formula, which includes:

[0058]

[0059] Among them, IVUS_Branch_Position represents the actual physical distance between the branch vessel and the reference point, BranchVessel start Indicates the initial frame number, BranchVessel end represents the end frame number, PullbackSpeed ​​represents the pullback speed, FrameRate represents the frame rate, and SF represents the frame number of the reference point.

[0060] Alternatively, the actual physical distance between adjacent branch vessels in the multi-frame IVUS image is calculated based on the initial frame number, the end frame number, the retraction speed and the frame rate of the IVUS, and the actual physical distance between adjacent branch vessels is used as the first position information of the branch vessels. For example, there are branch vessels 1, branch vessels 2, branch vessels 3, and branch vessels 4 in the IVUS image, and the actual physical distance between branch vessels 1 and branch vessels 2 is used as the first position information a1 of the branch vessels, the actual physical distance between branch vessels 2 and branch vessels 3 is used as the first position information a2 of the branch vessels, and the actual physical distance between branch vessels 3 and branch vessels 4 is used as the first position information a3 of the branch vessels, and a1, a2, and a3 form a first position information set.

[0061] Alternatively, the actual physical distance between adjacent branch vessels in the multi-frame IVUS image may be calculated based on the initial frame number, the end frame number, the retraction speed and frame rate of the IVUS, and the proportional relationship between the actual physical distances between adjacent branch vessels may be used as the first position information of the branch vessels. For example, there are branch vessels 1, branch vessels 2, branch vessels 3, branch vessels 4, and branch vessels 5 in the IVUS image. The actual physical distance L1 between branch vessels 1 and 2 is calculated, the actual physical distance L2 between branch vessels 2 and 3 is calculated, the actual physical distance L3 between branch vessels 3 and 4 is calculated, and the actual physical distance L4 between branch vessels 4 and 5 is calculated. The ratio of L1 to L2 is used as the first position information a1 of the branch vessel, the ratio of L2 to L3 is used as the first position information a2 of the branch vessel, and the ratio of L3 to L4 is used as the first position information a3 of the branch vessel. a1, a2, and a3 form a first position information set.

[0062] Then calculate the position of each branch blood vessel in the contrast image (recorded as the second position information), and obtain the second position information set of all branch blood vessels in the contrast image. The second position information set includes the position of each branch blood vessel in the contrast image. The second position information set is represented by [c1, c2...c mThe second position information of the branch blood vessels in the angiography image can be calculated by using a picture archiving and communication system (PACS) or a video acquisition card.

[0063] Specifically, calculating the second position information of each branch blood vessel in the angiography image includes:

[0064] determining a centerline of a single pixel coronary vessel tree in the angiographic image;

[0065] Based on the center line, calculating the coordinate information of the branch blood vessels in the angiography image;

[0066] The IVUS retracement starting point or the coronary ostium branch vessel point is used as the reference point, wherein the coronary ostium branch vessel point is the midpoint of the branch vessel closest to the coronary ostium;

[0067] Based on the coordinate information of the branch blood vessels, the actual physical distance between the branch blood vessels in the angiography image and the reference point is calculated, and the actual physical distance between each branch blood vessel and the reference point is used as the second position information of the branch blood vessels.

[0068] For angiography images, it is necessary to extract the center line to obtain the location of branch vessels. Figure 3 As shown in the figure, a schematic diagram of a single pixel coronary vessel tree centerline is obtained from the segmented coronary vessel tree binary image in the embodiment of the present application. Based on the coronary vessel tree centerline, the coordinate information of each branch vessel in the angiography image [(x1, y1), (x2, y2)…(x m ,y m )],like Figure 4 FIG. 1 is a schematic diagram of the node of the branch blood vessels in the embodiment of the present application. Figure 4 The bifurcation point is the node of the branch vessels. Figure 5 Shown is a schematic diagram of the IVUS retraction path of an embodiment of the present application.

[0069] According to the coordinate information of each branch vessel, the actual physical distance between each branch vessel and the reference point in the angiography image is calculated. For example, the Digital Imaging and Communications in Medicine (DICOM) information of the corresponding image can be obtained using the PACS system, such as the actual physical size of the pixel ImagePixelSpacing (in millimeters). According to the coordinate information of the branch vessel and the coordinates of the retracement path, the actual physical distance between the branch vessel and the reference point can be calculated. The calculation formula is as follows:

[0070]

[0071] Wherein, i = starting point, where starting point refers to the reference point.

[0072] The actual physical distance between the branch vessel and the IVUS retracement starting point can also be calculated using a video capture card. Since the video capture card can only obtain angiography images but cannot obtain corresponding DICOM information, that is, the actual physical size of the key pixels cannot be obtained, the location of the branch vessel in the angiography image can only be expressed in pixels, and only the pixel distance between the branch node and the retracement path starting point can be calculated. Specifically, the process of using a video capture card to calculate the actual physical distance of the branch vessel includes:

[0073] Performing region of interest image segmentation on the angiography image to obtain a guiding catheter feature image;

[0074] Calculating the actual physical size of each pixel of the angiography image based on the guiding catheter characteristic image and the intrinsic size of the guiding catheter;

[0075] The actual physical distance of the branch blood vessels is calculated based on the actual physical size of each pixel of the angiography image.

[0076] In order to obtain the pixel distance of each branch node from the reference point in millimeters, the region of interest image is segmented according to the angiography image to obtain the characteristic image of the guiding catheter. Then, the diameter R / pixel of the guiding catheter is calculated based on the characteristic image of the guiding catheter. Then, combined with the actual size Size / mm of the guiding catheter, the actual physical size Size / R (mm) of each pixel of the angiography image is calculated. Then, the actual physical distance of the branch blood vessel from the reference point is calculated based on the actual physical size of the angiography image pixel. The calculation formula is as follows:

[0077]

[0078] Wherein, i=starting point, the starting point refers to a reference point, and the actual physical distance between the branch blood vessel and the reference point is used as the second position information of the branch blood vessel.

[0079] Alternatively, based on the coordinate information of the branch vessels, the actual physical distance between adjacent branch vessels in the angiography image is calculated, and the actual physical distance between adjacent branch vessels is used as the second position information of the branch vessels. For example, there are branch vessels 1, 2, 3, and 4 in the angiography image, and the actual physical distance between branch vessels 1 and 2 is used as the second position information c1 of the branch vessels, the actual physical distance between branch vessels 2 and 3 is used as the second position information c2 of the branch vessels, and the actual physical distance between branch vessels 3 and 4 is used as the second position information c3 of the branch vessels, and c1, c2, and c3 form a second position information set.

[0080] Alternatively, the actual physical distances between adjacent branch vessels in the angiography image may be calculated based on the coordinate information of the branch vessels, and the proportional relationship between the actual physical distances between adjacent branch vessels may be used as the second position information of the branch vessels. For example, there are branch vessels 1, 2, 3, 4, and 5 in the angiography image. The actual physical distance H1 between branch vessel 1 and 2 is calculated, the actual physical distance H2 between branch vessel 2 and 3 is calculated, the actual physical distance H3 between branch vessel 3 and 4 is calculated, and the actual physical distance H4 between branch vessel 4 and 5 is calculated. The ratio of H1 to H2 is used as the second position information c1 of the branch vessels, the ratio of H2 to H3 is used as the second position information c2 of the branch vessels, and the ratio of H3 to H4 is used as the second position information c3 of the branch vessels. c1, c2, and c3 form a second position information set.

[0081] After obtaining the first position information set and the second position information set, the registration schemes of the first position information set and the second position information set are traversed. In order to traverse all branch blood vessel matching schemes, the idea of ​​permutation and combination can be used to select multiple branch blood vessels in turn from the branch blood vessels of the angiography image and the branch blood vessels of the IVUS image for matching. Specifically, the registration schemes of the first position information set and the second position information set are traversed, the error value of each registration scheme is calculated, and the registration scheme with the smallest error value is used as the target registration scheme.

[0082] Specifically, the actual physical distance between the branch blood vessel and the reference point is taken as the position information as an example;

[0083] The traversing the registration schemes of the first position information set and the second position information set, and calculating the error value of each registration scheme, includes:

[0084] Determine a minimum value K of the number of branch blood vessels in the multiple-frame IVUS image and the number of branch blood vessels in the angiography image;

[0085] The registration schemes corresponding to 2 to K branch blood vessels in the first position information set and the second position information set are traversed in sequence, and the error value of each registration scheme is calculated.

[0086] The first position information set is recorded as [a1, a2...a n ], the second location information set is recorded as [c1, c2...c m ], first determine the minimum value K (K = min[n,m]) of the number of branch vessels in the multi-frame IVUS image and the number of branch vessels in the angiography image, such as Figure 6 As shown, it is a schematic diagram of performing registration of any number of branch vessels in the order of branches in an embodiment of the present application, but the number of registration schemes does not exceed the K value, that is, the number of branch vessel registration schemes ranges from [2, K], and the registration schemes corresponding to 2 branch vessels to K branch vessels in the first position information set and the second position are traversed in turn, and then the error value of each registration scheme is calculated, and the registration scheme with the smallest error value in each registration scheme is used as the target registration scheme, and the IVUS image is registered with the angiography image using the target registration scheme. That is, the IVUS image is segmented and matched according to the branch vessels, and each segment evenly distributes the IVUS image on the center line of the retracement path vessel at equal intervals, thereby completing the registration and achieving the match between the position of the IVUS image and its exact position on the vascular roadmap obtained by coronary angiography. As shown Figure 7 As shown, it is a schematic diagram of the registration of the IVUS image and the angiography image in the embodiment of the present application.

[0087] The formula for calculating the error value of the registration scheme corresponding to the k branch vessels includes:

[0088]

[0089] Among them, position_error represents the error value of the registration scheme corresponding to k branch vessels, a1, a2, ..., a K represents the first position information of K branch vessels selected from the first position information set, c1, c2, ..., c K Represents the second position information of k branch blood vessels selected from the second position information set.

[0090] Taking the calculation of the error value of the registration scheme corresponding to three branch vessels as an example, the calculation formula is:

[0091] position_error=(|c3-c2|-|a3-a1|) 2 +(|c m -c3|-|a n -a3|) 2

[0092] Among them, position_error represents the error value of the registration scheme corresponding to the three branch vessels, a1, a3 and a n represents the first position information of three branch vessels selected from the first position information set, c1, c3 and c m Represents the second position information of three branch blood vessels selected from the second position information set.

[0093] Taking the registration of three pairs of branch vessels as an example, the position information of the corresponding branch vessels in the angiography image is [c1, c3, c m ], the location information of branch vessels in IVUS images is [a1, a3, a n ], the relative error of the calculated position can be calculated using a PACS system or a video acquisition card. The PACS system can be used to obtain the actual physical coordinates and distance of the branch blood vessel position, and the units are all in millimeters (mm), so the error can be calculated directly.

[0094] The use of video capture cards cannot obtain the Digital Imaging and Communications in Medicine (DICOM) information of the corresponding image, and the actual physical length of the branch blood vessels cannot be directly calculated. Instead, the unit is pixels, so the position error cannot be directly calculated. The actual physical coordinates and distance can be indirectly calculated by the size of the guiding catheter, and then the error can be calculated. The proportional relationship between the branch blood vessels can also be used for calculation. First, the selected data needs to be normalized each time. The normalization formula is as follows:

[0095]

[0096] Then, the error is calculated based on the normalized proportional relationship of the relative positions of the branch vessels. The calculation formula is as follows:

[0097] position_error=(|c'3-c'2|-|a'3-a1'|) 2 +(|c' m -c'3|-|a' n -a'3|) 2

[0098] Among them, position_error represents the error value of the registration scheme corresponding to the three branch vessels, a'1, a'3 and a' n represents the normalized values ​​of the first position information of the three branch vessels selected from the first position information set, c'2, c'3 and c' m Represents the normalized value of the second position information of three branch vessels selected from the second position information set.

[0099] Reference Figure 8 , which is a schematic diagram of the functional modules of the image processing device 100 of the present application.

[0100] The image processing device 100 described in the present application can be installed in an electronic device. According to the functions to be implemented, the image processing device 100 may include an acquisition module 110, a first calculation module 120, a second calculation module 130 and a registration module 140. The module described in the present application may also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0101] In this embodiment, the functions of each module / unit are as follows:

[0102] Acquisition module 110: used to acquire multiple frames of IVUS images with branched blood vessels, and acquire angiography images corresponding to the multiple frames of IVUS images;

[0103] A first calculation module 120: used to calculate the first position information of the branch blood vessels in the multiple frames of IVUS images, and obtain the first position information set of all the branch blood vessels in the multiple frames of IVUS images;

[0104] The second calculation module 130 is used to calculate the second position information of the branch blood vessels in the angiography image, and obtain the second position information set of all the branch blood vessels in the angiography image;

[0105] The registration module 140 is used to traverse the registration schemes of the first position information set and the second position information set, calculate the error value of each registration scheme, and take the registration scheme with the smallest error value as the target registration scheme.

[0106] In one embodiment, acquiring multiple frames of IVUS images with branched blood vessels includes:

[0107] Acquire an IVUS image set generated by the IVUS during the withdrawal process;

[0108] Using the pre-trained branch vessel recognition model, multiple frames of IVUS images with branch vessels are identified from the IVUS image set.

[0109] In one embodiment, calculating the first position information of the branch blood vessels in the multiple frames of IVUS images includes:

[0110] Determine the initial frame number and the end frame number of the IVUS image corresponding to the branch blood vessels respectively;

[0111] The IVUS retracement starting point or the coronary ostium branch vessel point is used as the reference point, wherein the coronary ostium branch vessel point is the midpoint of the branch vessel closest to the coronary ostium;

[0112] According to the initial frame number, the end frame number, the retraction speed and frame rate of the IVUS, the actual physical distance between the branch blood vessel and the reference point in the multiple frames of IVUS images is calculated, and the actual physical distance between the branch blood vessel and the reference point is used as the first position information of the branch blood vessel.

[0113] In one embodiment, calculating the first position information of the branch blood vessels in the multiple frames of IVUS images includes:

[0114] Determine the initial frame number and the end frame number of the IVUS image corresponding to the branch blood vessels respectively;

[0115] According to the initial frame number, the end frame number, the retraction speed and frame rate of the IVUS, the actual physical distance between adjacent branch vessels in the multi-frame IVUS image is calculated, and the actual physical distance between adjacent branch vessels is used as the first position information of the branch vessels.

[0116] In one embodiment, calculating the first position information of the branch blood vessels in the multiple frames of IVUS images includes:

[0117] Determine the initial frame number and the end frame number of the IVUS image corresponding to the branch blood vessels respectively;

[0118] According to the initial frame number, the end frame number, the retraction speed and frame rate of the IVUS, the actual physical distance between adjacent branch vessels in the multi-frame IVUS image is calculated, and the proportional relationship between the actual physical distances between adjacent branch vessels is used as the first position information of the branch vessels.

[0119] In one embodiment, the calculating the actual physical distance between the branch blood vessels and the reference point in the multiple frames of IVUS images according to the initial frame number, the end frame number, the IVUS retraction speed and the frame rate includes:

[0120] The position information of the branch blood vessels in the multi-frame IVUS images is calculated using the following formula, which includes:

[0121]

[0122] Among them, IVUS_Branch_Position represents the actual physical distance between the branch vessel and the reference point, BranchVessel start Indicates the initial frame number, BranchVessel endrepresents the end frame number, PullbackSpeed ​​represents the pullback speed, FrameRate represents the frame rate, and SF represents the frame number of the reference point.

[0123] In one embodiment, calculating the second position information of the branch blood vessel in the angiography image includes:

[0124] determining a centerline of a single pixel coronary vessel tree in the angiographic image;

[0125] Based on the center line, calculating the coordinate information of the branch blood vessels in the angiography image;

[0126] The IVUS retracement starting point or the coronary ostium branch vessel point is used as the reference point, wherein the coronary ostium branch vessel point is the midpoint of the branch vessel closest to the coronary ostium;

[0127] Based on the coordinate information of the branch blood vessel, the actual physical distance between the branch blood vessel and the reference point in the angiography image is calculated, and the actual physical distance between the branch blood vessel and the reference point is used as the second position information of the branch blood vessel.

[0128] In one embodiment, calculating the second position information of the branch blood vessel in the angiography image includes:

[0129] determining a centerline of a single pixel coronary vessel tree in the angiographic image;

[0130] Based on the center line, calculating the coordinate information of the branch blood vessels in the angiography image;

[0131] Based on the coordinate information of the branch blood vessels, the actual physical distance between adjacent branch blood vessels in the angiography image is calculated, and the actual physical distance between adjacent branch blood vessels is used as the second position information of the branch blood vessels.

[0132] In one embodiment, calculating the second position information of the branch blood vessel in the angiography image includes:

[0133] determining a centerline of a single pixel coronary vessel tree in the angiographic image;

[0134] Based on the center line, calculating the coordinate information of the branch blood vessels in the angiography image;

[0135] Based on the coordinate information of the branch blood vessels, the actual physical distance between adjacent branch blood vessels in the angiography image is calculated, and the proportional relationship between the actual physical distances between adjacent branch blood vessels is used as the second position information of the branch blood vessels.

[0136] In one embodiment, the process of calculating the actual physical distance of the branch blood vessels in the angiography image includes:

[0137] Performing region of interest image segmentation on the angiography image to obtain the guiding catheter feature image;

[0138] Calculating the actual physical size of each pixel of the angiography image based on the guiding catheter characteristic image and the intrinsic size of the guiding catheter;

[0139] The actual physical distance of the branch blood vessels is calculated based on the actual physical size of each pixel of the angiography image.

[0140] In one embodiment, traversing the registration schemes of the first position information set and the second position information set and calculating the error value of each registration scheme includes:

[0141] Determine a minimum value K of the number of branch blood vessels in the multiple-frame IVUS image and the number of branch blood vessels in the angiography image;

[0142] The registration schemes corresponding to 2 to K branch blood vessels in the first position information set and the second position information set are traversed in sequence, and the error value of each registration scheme is calculated.

[0143] In one embodiment, the calculating the error value of each registration scheme includes: calculating the error value of the registration scheme corresponding to the k branch vessels using the following formula, the formula including:

[0144]

[0145] Among them, position_error represents the error value of the registration scheme corresponding to k branch vessels, a1, a2, ..., a K represents the first position information of K branch vessels selected from the first position information set, c1, c2, ..., c K Represents the second position information of k branch blood vessels selected from the second position information set.

[0146] Reference Fig. 9 , which is a schematic diagram of a preferred embodiment of the electronic device 1 of the present application.

[0147] The electronic device 1 includes but is not limited to: a memory 11, a processor 12, a display 13 and a communication interface 14. The electronic device 1 can be connected to a network through the communication interface 14. The network can be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, a call network, etc.

[0148] Among them, the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a hard disk or memory of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped with the electronic device 1. Of course, the memory 11 can also include both the internal storage unit of the electronic device 1 and its external storage device. In this embodiment, the memory 11 is generally used to store the operating system and various computer programs installed in the electronic device 1, such as the program code of the image processing program 10, etc. In addition, the memory 11 can also be used to temporarily store various types of data that have been output or are to be output.

[0149] In some embodiments, the processor 12 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 12 is generally used to control the overall operation of the electronic device 1, such as performing control and processing related to data interaction or communication. In this embodiment, the processor 12 is used to run the program code stored in the memory 11 or process data, such as running the program code of the image processing program 10.

[0150] The display 13 may be referred to as a display screen or a display unit. In some embodiments, the display 13 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an organic light-emitting diode (OLED) touch device, etc. The display 13 is used to display information processed in the electronic device 1 and to display a visual working interface.

[0151] The communication interface 14 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), and the communication interface 14 is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0152] Fig. 9 Only the electronic device 1 having components 11 - 14 and the image processing program 10 is shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0153] In the above embodiment, the processor 12 may implement the following steps when executing the image processing program 10 stored in the memory 11:

[0154] Acquire multiple frames of IVUS images with branched blood vessels, and acquire angiography images corresponding to the multiple frames of IVUS images;

[0155] Calculating the first position information of the branch blood vessels in the multiple frames of IVUS images to obtain a first position information set of all the branch blood vessels in the multiple frames of IVUS images;

[0156] Calculating second position information of the branch blood vessels in the angiography image to obtain a second position information set of all branch blood vessels in the angiography image;

[0157] The registration schemes of the first position information set and the second position information set are traversed, an error value of each registration scheme is calculated, and a registration scheme with the smallest error value is taken as a target registration scheme.

[0158] The storage device may be the memory 11 of the electronic device 1 , or may be another storage device that is communicatively connected to the electronic device 1 .

[0159] For a detailed description of the above steps, please refer to the above Figure 8 Functional module diagram of the embodiment of the image processing device 100 and Figure 1 Description of a flowchart of an embodiment of an image processing method.

[0160] In addition, the embodiment of the present application also proposes a computer-readable storage medium, which can be non-volatile or volatile. The computer-readable storage medium can be any one or any combination of a hard disk, a multimedia card, an SD card, a flash memory card, an SMC, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, etc. The computer-readable storage medium includes a data storage area and a program storage area, and the program storage area stores an image processing program 10. When the image processing program 10 is executed by the processor, the following operations are implemented:

[0161] Acquire multiple frames of IVUS images with branched blood vessels, and acquire angiography images corresponding to the multiple frames of IVUS images;

[0162] Calculating the first position information of the branch blood vessels in the multiple frames of IVUS images to obtain a first position information set of all the branch blood vessels in the multiple frames of IVUS images;

[0163] Calculating second position information of the branch blood vessels in the angiography image to obtain a second position information set of all branch blood vessels in the angiography image;

[0164] The registration schemes of the first position information set and the second position information set are traversed, an error value of each registration scheme is calculated, and a registration scheme with the smallest error value is taken as a target registration scheme.

[0165] The specific implementation of the computer-readable storage medium of the present application is substantially the same as the specific implementation of the above-mentioned image processing method, and will not be described in detail here.

[0166] It should be noted that the serial numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof herein are intended to cover non-exclusive inclusion, so that the process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, device, article or method including the element.

[0167] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware simulation platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, an electronic device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0168] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire multiple frames of IVUS images with branched blood vessels, and acquire angiography images corresponding to the multiple frames of IVUS images; Calculating the first position information of the branch blood vessels in the multiple frames of IVUS images to obtain a first position information set of all the branch blood vessels in the multiple frames of IVUS images; Calculating second position information of the branch blood vessels in the angiography image to obtain a second position information set of all branch blood vessels in the angiography image; The registration schemes of the first position information set and the second position information set are traversed, an error value of each registration scheme is calculated, and a registration scheme with the smallest error value is taken as a target registration scheme.

2. The image processing method according to claim 1, characterized in that: The method of acquiring a multi-frame IVUS image with branched blood vessels comprises: Acquire an IVUS image set generated by the IVUS during the withdrawal process; Using the pre-trained branch vessel recognition model, multiple frames of IVUS images with branch vessels are identified from the IVUS image set.

3. The image processing method according to claim 1, characterized in that: The calculating the first position information of the branch blood vessels in the multiple frames of IVUS images includes: Determine the initial frame number and the end frame number of the IVUS image corresponding to the branch blood vessels respectively; The IVUS retracement starting point or the coronary ostium branch vessel point is used as the reference point, wherein the coronary ostium branch vessel point is the midpoint of the branch vessel closest to the coronary ostium; According to the initial frame number, the end frame number, the retraction speed and frame rate of the IVUS, the actual physical distance between the branch blood vessel and the reference point in the multiple frames of IVUS images is calculated, and the actual physical distance between the branch blood vessel and the reference point is used as the first position information of the branch blood vessel.

4. The image processing method according to claim 1, wherein: The calculating the first position information of the branch blood vessels in the multiple frames of IVUS images includes: Determine the initial frame number and the end frame number of the IVUS image corresponding to the branch blood vessels respectively; According to the initial frame number, the end frame number, the retraction speed and frame rate of the IVUS, the actual physical distance between adjacent branch vessels in the multi-frame IVUS image is calculated, and the actual physical distance between adjacent branch vessels is used as the first position information of the branch vessels.

5. The image processing method according to claim 1, wherein: The calculating the first position information of the branch blood vessels in the multiple frames of IVUS images includes: Determine the initial frame number and the end frame number of the IVUS image corresponding to the branch blood vessels respectively; According to the initial frame number, the end frame number, the retraction speed and frame rate of the IVUS, the actual physical distance between adjacent branch vessels in the multi-frame IVUS image is calculated, and the proportional relationship between the actual physical distances between adjacent branch vessels is used as the first position information of the branch vessels.

6. The image processing method according to claim 3, characterized in that: The calculating, according to the initial frame number, the end frame number, the IVUS retraction speed and the frame rate, the actual physical distance between the branch blood vessels in the multi-frame IVUS images and the reference point comprises: The position information of the branch blood vessels in the multi-frame IVUS images is calculated using the following formula, which includes: Among them, IVUS_Branch_Position represents the actual physical distance between the branch vessel and the reference point, BranchVessel start Indicates the initial frame number, BranchVessel end represents the end frame number, PullbackSpeed ​​represents the pullback speed, FrameRate represents the frame rate, and SF represents the frame number of the reference point.

7. The image processing method according to claim 1, characterized in that: The calculating the second position information of the branch blood vessel in the angiography image includes: determining a centerline of a single pixel coronary vessel tree in the angiographic image; Based on the center line, calculating the coordinate information of the branch blood vessels in the angiography image; The IVUS retracement starting point or the coronary ostium branch vessel point is used as the reference point, wherein the coronary ostium branch vessel point is the midpoint of the branch vessel closest to the coronary ostium; Based on the coordinate information of the branch blood vessel, the actual physical distance between the branch blood vessel and the reference point in the angiography image is calculated, and the actual physical distance between the branch blood vessel and the reference point is used as the second position information of the branch blood vessel.

8. The image processing method according to claim 1, characterized in that: The calculating the second position information of the branch blood vessel in the angiography image includes: determining a centerline of a single pixel coronary vessel tree in the angiographic image; Based on the center line, calculating the coordinate information of the branch blood vessels in the angiography image; Based on the coordinate information of the branch blood vessels, the actual physical distance between adjacent branch blood vessels in the angiography image is calculated, and the actual physical distance between adjacent branch blood vessels is used as the second position information of the branch blood vessels.

9. The image processing method according to claim 1, characterized in that: The calculating the second position information of the branch blood vessel in the angiography image includes: determining a centerline of a single pixel coronary vessel tree in the angiographic image; Based on the center line, calculating the coordinate information of the branch blood vessels in the angiography image; Based on the coordinate information of the branch blood vessels, the actual physical distance between adjacent branch blood vessels in the angiography image is calculated, and the proportional relationship between the actual physical distances between adjacent branch blood vessels is used as the second position information of the branch blood vessels.

10. The image processing method according to any one of claims 7 to 9, characterized in that: The process of calculating the actual physical distance of the branch blood vessels in the angiography image includes: Performing region of interest image segmentation on the angiography image to obtain the guiding catheter feature image; Calculating the actual physical size of each pixel of the angiography image based on the guiding catheter characteristic image and the intrinsic size of the guiding catheter; The actual physical distance of the branch blood vessels is calculated based on the actual physical size of each pixel of the angiography image.

11. The image processing method according to claim 3, characterized in that: The traversing the registration schemes of the first position information set and the second position information set, and calculating the error value of each registration scheme, includes: Determine a minimum value K of the number of branch blood vessels in the multiple-frame IVUS image and the number of branch blood vessels in the angiography image; The registration schemes corresponding to 2 to K branch blood vessels in the first position information set and the second position information set are traversed in sequence, and the error value of each registration scheme is calculated.

12. The image processing method according to claim 11, characterized in that: The calculating the error value of each registration scheme includes: calculating the error value of the registration scheme corresponding to the k branch blood vessels using the following formula, the formula including: Among them, position_error represents the error value of the registration scheme corresponding to k branch vessels, a1, a2, ..., a K represents the first position information of K branch vessels selected from the first position information set, c1, c2, ..., c K Represents the second position information of k branch blood vessels selected from the second position information set.

13. An image processing device, characterized in that: The device comprises: An acquisition module: used for acquiring multiple frames of IVUS images with branched blood vessels, and acquiring angiography images corresponding to the multiple frames of IVUS images; A first calculation module: used to calculate the first position information of the branch blood vessels in the multi-frame IVUS image, and obtain the first position information set of all the branch blood vessels in the multi-frame IVUS image; A second calculation module: used to calculate the second position information of the branch blood vessels in the angiography image, and obtain a second position information set of all the branch blood vessels in the angiography image; Registration module: used for traversing the registration schemes of the first position information set and the second position information set, calculating the error value of each registration scheme, and taking the registration scheme with the smallest error value as the target registration scheme.

14. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, configured to implement the image processing method according to any one of claims 1 to 12 when executing a program stored in a memory.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 12 is implemented.

Images