Image processing method and device based on frame insertion method, equipment and storage medium

Abstract

The invention relates to an image processing method and device based on a frame insertion method, equipment and a storage medium. The method comprises the following steps: acquiring a contrast image sequence with a contrast agent, a perspective sequence and an IVUS image, respectively determining a target image sequence in the contrast image sequence and a plurality of target perspective sequences in the perspective sequence, and aligning each target perspective sequence with the target image sequence, fusing the aligned target perspective image sequence and the target image sequence to obtain a fused image sequence, determining a frame insertion number of the fused image sequence, inserting a frame insertion image corresponding to the frame insertion number into the fused image sequence to obtain a fused image sequence after frame insertion, and identifying a preset mark of the fused image sequence after frame insertion, and performing image registration processing on the target image sequence and the IVUS image according to a preset mark. According to the invention, the accuracy of the registration processing of the contrast image and the IVUS image can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, device, equipment and storage medium based on interpolation. Background Art

[0002] IVUS (Intravascular Ultrasound) is an examination method that performs imaging by inserting an ultrasound probe into the inside of a blood vessel. IVUS can provide high-resolution information such as the morphology of the blood vessel wall, the degree of stenosis, and the composition of plaques, which is of great significance for the evaluation of coronary artery lesions and their treatment effects. During an IVUS examination, the doctor inserts a thin catheter into the patient's blood vessels and sends the ultrasound probe through the catheter to the location that needs to be examined, thereby obtaining high-quality vascular images. IVUS can provide intraluminal views and cross-sectional images of the coronary vessels, but lacks the overview of the vascular area provided by angiography.

[0003] Contrast imaging is an important application of X-rays in medical imaging. Since contrast imaging projects three-dimensional spatial structures onto a two-dimensional plane, the contrast image will affect the doctor's observation. In contrast, IVUS embeds a tiny ultrasound probe at the top of the cardiac catheter and places it in the blood vessel. It can obtain cross-sectional images of the vascular wall and lumen, and can detect and display the morphology, structure and histological characteristics of coronary atherosclerotic plaques. Therefore, contrast imaging and IVUS imaging are complementary.

[0004] In the prior art, the method for achieving the registration of angiography images and IVUS images is usually to synchronously acquire angiography images during the IVUS withdrawal process, and then achieve registration by identifying the position of the ultrasound probe in the angiography image. The defects of this method are: since the frame rate of coronary angiography images is low and the number of acquisitions is small, while the frame rate of IVUS is high and the number of acquisitions is far greater than that of angiography images, there is a problem of continuous playback of angiography images during the registration process, that is, there is a technical problem that one angiography image corresponds to multiple IVUS images, resulting in the registration processing of angiography images and IVUS images is not accurate enough. Summary of the invention

[0005] In view of the above, the present application provides an image processing method, device, equipment and storage medium based on interpolation method, which aims to solve the above technical problems.

[0006] In a first aspect, the present application provides an image processing method based on interpolation method, the method comprising:

[0007] Acquire angiographic image sequences with contrast agent, fluoroscopic image sequences without contrast agent, and IVUS images;

[0008] Determine, according to the image features of the guiding catheter in the image, a target image sequence in the angiography image sequence and a plurality of target perspective image sequences in the perspective image sequence, and align each of the target perspective image sequences with the target image sequence;

[0009] fusing the aligned target perspective image sequence and the target image sequence to obtain a fused image sequence;

[0010] Determining the number of interpolated frames of the fused image sequence according to the number of frames of the fused image sequence and the number of frames of the IVUS image;

[0011] Inserting interpolated frame images corresponding to the number of interpolated frames into the fused image sequence to obtain an interpolated fused image sequence;

[0012] A preset mark of the inserted fused image sequence is identified, and based on the preset mark, the target image sequence and the IVUS image corresponding to the target image sequence are subjected to image processing, wherein the image processing includes image registration.

[0013] In a second aspect, the present application provides an image processing device based on an interpolation method, the device comprising:

[0014] Acquisition module: used to acquire angiographic image sequences with contrast agents, fluoroscopic image sequences without contrast agents, and IVUS images;

[0015] An alignment module: used for determining a target image sequence in the angiography image sequence and a plurality of target perspective image sequences in the perspective image sequence respectively according to image features of the guiding catheter in the image, and aligning each of the target perspective image sequences with the target image sequence;

[0016] Fusion module: used for fusing the aligned target perspective image sequence and the target image sequence to obtain a fused image sequence;

[0017] A determination module: used to determine the number of interpolated frames of the fused image sequence according to the number of frames of the fused image sequence and the number of frames of the IVUS image;

[0018] Insertion module: used for inserting the insertion frame images corresponding to the insertion frame number into the fused image sequence to obtain the inserted fused image sequence;

[0019] Processing module: used for identifying the preset mark of the inserted fusion image sequence, and performing image processing on the target image sequence and the IVUS image corresponding to the target image sequence based on the preset mark, wherein the image processing includes image registration.

[0020] In a third aspect, the present application provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0021] Memory, used to store computer programs;

[0022] The processor is used to implement the image processing method based on the interpolation method described in any embodiment of the first aspect when executing the program stored in the memory.

[0023] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image processing method based on the interpolation method as described in any embodiment of the first aspect is implemented.

[0024] The above technical solution provided by the embodiment of the present application has the following advantages compared with the prior art:

[0025] The present application obtains an angiography image sequence with contrast agent before IVUS withdrawal, obtains a perspective view sequence and IVUS images during the IVUS withdrawal process, and determines a target image sequence in the angiography image sequence and multiple target perspective view sequences in the perspective view sequence according to the image features of the guiding catheter in the image, aligns each target perspective view sequence with the target image sequence, fuses the aligned target perspective view sequence and the target image sequence to obtain a fused image sequence, determines the number of interpolated frames of the fused image sequence, inserts interpolated frame images corresponding to the number of interpolated frames into the fused image sequence to obtain a fused image sequence after interpolation, identifies preset marks of the fused image sequence after interpolation, and performs image registration between the target image sequence and the IVUS image corresponding to the target image sequence through the preset marks, so that IVUS can provide an intraluminal view and a cross-sectional image of the coronary blood vessel during the withdrawal process, and the angiography image sequence provides an overview of the IVUS withdrawal vascular area, thereby realizing real-time registration processing of the angiography image and the IVUS image, and improving the accuracy of the registration processing of the angiography image and the IVUS image. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0028] Figure 1This is a flow chart of an embodiment of an image processing method based on the interpolation method of the present application;

[0029] Figure 2 A schematic diagram of the tip of a guiding catheter according to an embodiment of the present application;

[0030] Figure 3 Schematic diagram of interior points, non-interior points and isolated points for the implementation of this application;

[0031] Figure 4 A schematic diagram of the preset mark for this application;

[0032] Figure 5 This is a module diagram of a preferred embodiment of the image processing device based on the interpolation method of the present application;

[0033] Figure 6 A schematic diagram of a preferred embodiment of the electronic device of the present application;

[0034] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0036] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0037] The present application provides an image processing method based on interpolation. Figure 1 , which is a schematic diagram of a method flow of an embodiment of an image processing method based on the interpolation method of the present application. The method can be executed by an electronic device, and the electronic device can be implemented by software and / or hardware. The image processing method based on the interpolation method includes:

[0038] Step S10: acquiring a contrast image sequence with contrast agent, a perspective image sequence without contrast agent, and an IVUS image;

[0039] Step S20: determining a target image sequence in the angiography image sequence and a plurality of target perspective image sequences in the perspective image sequence respectively according to image features of the guiding catheter in the image, and aligning each of the target perspective image sequences with the target image sequence;

[0040] Step S30: fusing the aligned target perspective image sequence and the target image sequence to obtain a fused image sequence;

[0041] Step S40: determining the number of interpolated frames of the fused image sequence according to the number of frames of the fused image sequence and the number of frames of the IVUS image;

[0042] Step S50: inserting interpolated frame images corresponding to the interpolated frame number into the fused image sequence to obtain a fused image sequence after interpolation;

[0043] Step S60: identifying a preset mark of the inserted fused image sequence, and performing image processing on the target image sequence and the IVUS image corresponding to the target image sequence based on the preset mark, wherein the image processing includes image registration.

[0044] In this embodiment, the angiographic image sequence with contrast agent refers to the angiographic image sequence of IVUS before withdrawal. The angiographic image is obtained by introducing contrast agent into the target blood vessel, eliminating the images of bones and soft tissues on the angiographic film through a computer, and only displaying the image of the blood vessel on the film. The perspective image sequence without contrast agent is also called the vascular image sequence without contrast agent. The IVUS image is a cross-sectional image of the blood vessel developed by an electronic imaging system installed in the ultrasound catheter after a catheter equipped with a miniaturized ultrasonic transducer is sent into the blood vessel cavity. The IVUS image can not only observe the morphology of the lumen, but also the structure and lesions of the vessel wall. Among them, the working process of IVUS is as follows:

[0045] 1. Move the guiding catheter to the coronary artery ostium;

[0046] 2. Move the guide wire along the guiding catheter into the coronary artery (blood vessel);

[0047] 3. The IVUS catheter is moved along the guidewire into the coronary artery (blood vessel);

[0048] 4. The ultrasonic transducer in the IVUS catheter retracts and rotates to collect IVUS images.

[0049] A rotating shaft is arranged in the IVUS catheter, an ultrasonic transducer is arranged at the distal end of the rotating shaft, and a Mark mark is arranged on the rotating shaft. The ultrasonic transducer and the Mark mark can move synchronously.

[0050] The angiography image sequence with contrast agent is acquired before the IVUS is withdrawn, and the perspective view sequence without contrast agent is acquired during the IVUS withdrawal process. The acquired angiography image sequence with contrast agent and the perspective view sequence without contrast agent are corresponding. For example, when the angiography image sequence of blood vessel A with contrast agent is acquired, the corresponding perspective view sequence of blood vessel A without contrast agent is acquired.

[0051] Since the guiding catheter of the angiography image sequence has a certain change rule, the target image sequence in the angiography image sequence can be determined according to the change rule of the image characteristics of the guiding catheter in the angiography image sequence. The target image sequence refers to the periodic sequence with the highest clarity in the angiography image sequence. The periodic sequence is a periodic sequence calculated by the tip of the guiding catheter, which contains the angiography images obtained in the complete heartbeat cycle, from the end of systole to the end of diastole. Specifically, the target image sequence in the angiography image sequence is obtained by analyzing the periodic change rule of the guiding catheter of the angiography image sequence and the image information of the angiography image sequence, including:

[0052] Segmenting a guiding catheter region of the angiography image sequence, and calculating position information of a guiding catheter tip of the angiography image sequence based on the guiding catheter region;

[0053] Acquiring the periodic variation rule of the position information of the tip of the guiding catheter of the angiography image sequence;

[0054] Acquiring image information of the angiography image sequence, wherein the image information includes grayscale information;

[0055] Based on the periodic variation rule and the image information of the angiography image sequence, a target image sequence in the angiography image sequence is analyzed and obtained, wherein the target image sequence is an image sequence of a complete heartbeat cycle with the highest definition in the angiography image sequence.

[0056] The region of interest image segmentation algorithm can be used to automatically detect the guiding catheter region in the angiography image sequence, and then segment the guiding catheter region in the angiography image sequence to obtain a segmented image. The position information (e.g., coordinate information) of the guiding catheter tip can be calculated based on the segmented image of the guiding catheter. Figure 2 The figure shows a schematic diagram of the guide catheter tip of the embodiment of the present application. The periodic detection algorithm is then used to analyze the periodic change law of the position information of the guide catheter tip, wherein the periodic change law of the position information of the guide catheter tip is consistent with the patient's heartbeat law. According to the periodic change law and the grayscale information of the angiography image sequence, the image sequence of the complete heartbeat cycle with the highest clarity in the angiography image sequence can be screened out, and the image sequence of the complete heartbeat cycle with the highest clarity can be used as the target image sequence.

[0057] Furthermore, the target image sequence can be binarized to obtain a binary image of the target image sequence. The background point in the binary image is set to 0 and the target point is set to 1. The centerline of the coronary artery tree can be calculated based on the binary image. The centerline of the blood vessel can be obtained by the distance transformation method. The minimum distance from the internal point of the blood vessel to the non-interior point is calculated, and then the distance is classified to obtain the centerline of the blood vessel. When calculating the centerline of the blood vessel, (1) the target points on the binary image are first classified (into internal points, non-interior points, and isolated points). Non-interior points are also called boundary points. Then the isolated points are deleted to obtain a binary image containing only internal points and non-interior points. (2) The distance from each internal point to the non-interior point is calculated and the distance is assigned to the image pixel value at the current position. (3) The image is binarized. The binarization process is to set the image pixel value with a distance greater than 1 to 1 and the image pixel value with a distance less than or equal to 1 to 0. Repeat steps (1), (2) and (3) until the next distance image binarization result is all 0.

[0058] In the above step (1) of finding the centerline of the blood vessel, the distinction between internal points, non-interior points, and isolated points is as follows: Figure 3 As shown, taking the four neighborhoods as an example, if the central pixel is the target pixel and its four neighbors are all target pixels, then the point is called an internal point, such as Figure 3 If the central pixel is the target pixel and its four neighbors are background pixels, then the point is called an isolated point. The points in other cases are called non-interior points (also called boundary points), such as Figure 3 As shown in the right figure.

[0059] The accuracy of fusion between the target perspective sequence and the target image sequence can be improved by aligning and matching the calculated blood vessel centerline and the guidewire;

[0060] Furthermore, the periodic variation law of the position information of the guiding catheter tip of the angiography image sequence is obtained, including:

[0061] performing a preprocessing operation on the time series data of the position information of the tip of the guiding catheter in the angiography image sequence;

[0062] The time series data after the preprocessing operation is analyzed using a periodic detection algorithm to obtain the periodic variation law of the position information of the tip of the guiding catheter in the angiography image sequence.

[0063] Due to the high complexity of time series data, the periodic detection process will be affected by non-periodic components such as noise, abnormal data and missing data in the time series data, so it is necessary to pre-process the data effectively. Specifically, for noise interference, filters or smoothing algorithms are used to reduce the interference of noise. For abnormal data interference, extreme value removal or outlier detection methods are used to process abnormal data to reduce the interference of abnormal data. For the impact of missing data, the median or mean filling method is used to fill in the case of small missing data segments, and the linear interpolation method is used to fill in the case of large missing data segments.

[0064] The periodicity detection algorithm is to determine the periodicity of the data through the autocorrelation coefficient, and to measure the correlation between the same event in two different periods through the autocorrelation coefficient. The mathematical expression of the autocorrelation coefficient includes the following forms:

[0065]

[0066] Among them, Coeff represents the autocorrelation coefficient, Y t represents the position data of the guiding catheter tip at time t, Y is the average value, and by calculating the correlation degree of the data itself at different periods, the periodic transformation law of the guiding catheter with the movement of the heart is obtained, and the period k is obtained, where k refers to the sequence number of the target image sequence.

[0067] After analyzing the contrast image sequence to obtain the target image sequence with the highest definition, the perspective image sequence is divided according to the change law of the image characteristics of the guiding catheter in the perspective image sequence to obtain multiple target perspective image sequences. Specifically, the guiding catheter region in the perspective image sequence is detected and segmented by the image region of interest segmentation algorithm, and then the periodic change law of the guiding catheter tip is analyzed, and the collected perspective image sequence is divided into multiple groups according to the periodic change law of the guiding catheter tip, and each group obtained by the division represents a period of the target perspective image sequence, thereby obtaining multiple target perspective image sequences.

[0068] After obtaining a plurality of target perspective image sequences, each target perspective image sequence is periodically aligned with a target image sequence. Specifically, aligning each target perspective image sequence with the target image sequence includes:

[0069] Taking the target image sequence as a reference, each target perspective view sequence is respectively aligned with the target image sequence by sequence translation.

[0070] Periodic alignment means taking the target image sequence R as a reference, performing sequence translation alignment on each target perspective sequence T and the target image sequence R, that is, there is a translation amount m, and T(i+m)=R(i).

[0071] After performing the periodic alignment, the aligned target perspective image sequence is fused with the target image sequence to obtain a fused image sequence. Specifically, the fused image sequence is fused with the target image sequence to obtain a fused image sequence, including:

[0072] identifying the radiopaque tip of the guidewire in the target image sequence and the radiopaque tip of the guidewire in the target perspective image sequence;

[0073] Performing a non-rigid transformation based on the opaque tip of the guidewire in the target image sequence and the opaque tip of the guidewire in the target perspective image sequence to obtain a fused image;

[0074] The fused images are arranged according to the time sequence of the fused images to obtain the fused image sequence.

[0075] The non-transmissive tip of the guidewire in the target image sequence and the non-transmissive tip of the guidewire in the target perspective sequence are identified using the pre-trained model, and a non-rigid transformation is performed on the non-transmissive tip of the guidewire in the target image sequence and the non-transmissive tip of the guidewire in the target perspective sequence to obtain a fused image. The non-rigid transformation process is as follows: Assume that the characteristic coordinates of the non-transmissive tip of the guidewire in the perspective sequence are O i (x i ,y i ), i = 1, 2, 3, ..., n, the characteristic coordinates of the Mark are P (x, y), and the characteristic coordinates of the non-transmissive tip of the guide wire in the angiographic image sequence are O i ′(x i ′,y i ′),i=1,2,3,...,n, Mark feature coordinates are P′(x′,y′), then there exists a non-rigid transformation coefficient H such that [O′,P′]=H[O,P]. According to the non-rigid transformation coefficient H, the point set of the perspective sequence is mapped to the point set of the contrast image sequence, thereby realizing the accurate fusion of the target perspective sequence and the target image sequence, and then the fused images are arranged in chronological order to obtain the fused image sequence.

[0076] After obtaining the fused image sequence, perform interpolation operation on the fused image sequence, analyze the fused sequence images, obtain key information of each frame of the fused image (for example, image content, direction of movement, speed, etc.), determine the interpolation image based on these key information, and then determine the number of interpolation frames (the number of interpolation frames refers to the total number of interpolation frames that need to be inserted in the fused image sequence). Generally, the higher the number of interpolation frames, the higher the image playback fluency and clarity, and the higher the correspondence between the IVUS sequence and the target image sequence, that is, the higher the registration accuracy of the IVUS image and the coronary angiography image. Wherein, determining the number of interpolation frames of the fused image sequence based on the number of frames of the fused image sequence and the number of frames of the IVUS image includes:

[0077] The number of interpolated frames is calculated using the following formula, which includes:

[0078] S=M / (N-1)

[0079] Wherein, S represents the number of interpolated frames, M represents the number of frames of the IVUS image, and N represents the number of frames of the fused image sequence.

[0080] After the number of interpolation frames is determined, interpolation frame images corresponding to the number of interpolation frames are inserted into the fused image sequence to obtain the interpolated fused image sequence. Specifically, inserting the interpolation frame images corresponding to the number of interpolation frames into the fused image sequence to obtain the interpolated fused image sequence includes:

[0081] Based on the motion estimation interpolation algorithm or the optical flow interpolation algorithm, one frame of the interpolation image is sequentially inserted between two adjacent frames in the fused image sequence to obtain the interpolated fused image sequence.

[0082] The uninterpolated fused image sequence is read into the memory, and interpolation processing is performed according to the number of interpolation frames (i.e., one interpolation frame is inserted between two adjacent frames in the fused image sequence) to generate an interpolated fused image sequence. The interpolated fused image sequence can be saved as a new contrast sequence file, or the interpolated fused image sequence can be directly played in the memory. The interpolation algorithm and the number of interpolation frames can also be adjusted according to the actual effect and needs to optimize the quality and performance of the interpolated contrast fusion sequence.

[0083] In the fused image sequence after interpolation, the vascular morphology and the position of the IVUS ultrasound probe can be clearly observed, thereby realizing the registration of the contrast image and the IVUS image. However, the patient's heartbeat and respiration will affect the registration result, that is, there may be errors between the perspective image sequence and the contrast image sequence after periodic alignment, and there may be some image mismatches in the fused image sequence. In order to reduce this error, further matching is required through other features. First, the preset marks of the fused image sequence are identified using machine learning technology. The preset marks include the ultrasound probe in the IVUS catheter and / or the Mark mark in the IVUS catheter that moves with the ultrasound probe. Then, the target image sequence is registered with the IVUS image corresponding to the target image sequence according to the preset marks. Specifically, the preset marks of the fused image sequence after interpolation are identified, and the target image sequence is registered with the IVUS image corresponding to the target image sequence based on the preset marks, including:

[0084] Identifying the position information of the IVUS ultrasound probe and / or the Mark in the fused image sequence after the interpolation;

[0085] A non-rigid transformation is performed based on the position information of the IVUS ultrasound probe and / or the Mark marker to obtain the registration result of the target image sequence and the IVUS image.

[0086] The preset markers include the ultrasound probe in the IVUS catheter and / or the Mark marker in the IVUS catheter that moves with the ultrasound probe. The position information of the IVUS ultrasound transducer (ultrasound probe) and / or the Mark marker on the IVUS catheter in the fused image is identified by using the pre-trained recognition model. Figure 4 As shown, it is a schematic diagram of the preset mark of the present application. Non-rigid transformation is performed according to the position information of the IVUS ultrasound probe and / or the Mark marker to obtain the registration result of the target image sequence and the IVUS image, wherein the non-rigid transformation includes translation, rotation, scaling and affine transformation. The position point set of the IVUS ultrasound probe identified on the perspective sequence is subjected to the same non-rigid transformation to obtain the position information of the IVUS ultrasound probe on the corresponding contrast image, thereby realizing the registration function of the contrast image and the IVUS image, so that the IVUS provides the intraluminal view and cross-sectional image of the coronary blood vessels during the withdrawal process, and the contrast image sequence provides an overview of the IVUS withdrawal blood vessel area, thereby realizing the real-time registration of the contrast image and the IVUS image.

[0087] Reference Figure 5 , which is a schematic diagram of functional modules of an image processing device 100 based on the interpolation method of the present application.

[0088] The image processing device 100 based on the interpolation method described in the present application can be installed in an electronic device. According to the functions implemented, the image processing device 100 based on the interpolation method may include an acquisition module 110, an alignment module 120, a fusion module 130, a determination module 140, an interpolation module 150 and a processing module 160. The module described in the present application may also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0089] In this embodiment, the functions of each module / unit are as follows:

[0090] Acquisition module 110: used to acquire angiography image sequences with contrast agents, perspective image sequences without contrast agents, and IVUS images;

[0091] Alignment module 120: used to determine a target image sequence in the angiography image sequence and a plurality of target perspective image sequences in the perspective image sequence according to image features of the guiding catheter in the image, and align each of the target perspective image sequences with the target image sequence;

[0092] Fusion module 130: used for fusing the aligned target perspective image sequence and the target image sequence to obtain a fused image sequence;

[0093] Determining module 140: used to determine the number of interpolated frames of the fused image sequence according to the number of frames of the fused image sequence and the number of frames of the IVUS image;

[0094] Insertion module 150: used for inserting the insertion frame images corresponding to the insertion frame number into the fused image sequence to obtain the inserted fused image sequence;

[0095] Processing module 160: used for identifying preset marks of the inserted fused image sequence, and performing image processing on the target image sequence and the IVUS image corresponding to the target image sequence based on the preset marks, wherein the image processing includes image registration.

[0096] In one embodiment, the step of determining the target image sequence in the angiography image sequence and the plurality of target perspective image sequences in the perspective image sequence respectively according to the image features of the guiding catheter in the image comprises:

[0097] Analyze and obtain a target image sequence in the angiography image sequence according to the periodic variation rule of the guiding catheter in the angiography image sequence and the image information of the angiography image sequence;

[0098] According to the periodic variation rule of the guiding catheter of the perspective image sequence, the perspective image sequence is grouped to obtain a plurality of target perspective image sequences.

[0099] In one embodiment, the step of analyzing and obtaining a target image sequence in the angiography image sequence based on the periodic variation rule of the guiding catheter in the angiography image sequence and the image information of the angiography image sequence includes:

[0100] Segmenting a guiding catheter region of the angiography image sequence, and calculating position information of a guiding catheter tip of the angiography image sequence based on the guiding catheter region;

[0101] Acquiring the periodic variation rule of the position information of the tip of the guiding catheter of the angiography image sequence;

[0102] Acquiring image information of the angiography image sequence, wherein the image information includes grayscale information;

[0103] Based on the periodic variation rule and the image information of the angiography image sequence, a target image sequence in the angiography image sequence is analyzed and obtained.

[0104] In one embodiment, the target image sequence is an image sequence of a complete heartbeat cycle with the highest definition in the contrast image sequence.

[0105] In one embodiment, the periodic variation rule of the position information of the tip of the guiding catheter of the angiography image sequence is obtained, including:

[0106] performing a preprocessing operation on the time series data of the position information of the tip of the guiding catheter in the angiography image sequence;

[0107] The time series data after the preprocessing operation is analyzed using a periodic detection algorithm to obtain the periodic variation law of the position information of the tip of the guiding catheter in the angiography image sequence.

[0108] In one embodiment, aligning each of the target perspective image sequences with the target image sequence comprises:

[0109] Taking the target image sequence as a reference, each target perspective view sequence is respectively aligned with the target image sequence by sequence translation.

[0110] In one embodiment, the fusing the aligned target perspective image sequence and the target image sequence to obtain a fused image sequence includes:

[0111] identifying the radiopaque tip of the guidewire in the target image sequence and the radiopaque tip of the guidewire in the target perspective image sequence;

[0112] Performing a non-rigid transformation based on the opaque tip of the guidewire in the target image sequence and the opaque tip of the guidewire in the target perspective image sequence to obtain a fused image;

[0113] The fused images are arranged according to the time sequence of the fused images to obtain the fused image sequence.

[0114] In one embodiment, determining the number of interpolated frames of the fused image sequence according to the number of frames of the fused image sequence and the number of frames of the IVUS image includes:

[0115] The number of interpolated frames is calculated using the following formula, which includes:

[0116] S=M / (N-1)

[0117] Wherein, S represents the number of interpolated frames, M represents the number of frames of the IVUS image, and N represents the number of frames of the fused image sequence.

[0118] In one embodiment, inserting the interpolated frame images corresponding to the interpolated frame number into the fused image sequence to obtain the interpolated fused image sequence includes:

[0119] Based on the motion estimation interpolation algorithm or the optical flow interpolation algorithm, one frame of the interpolation image is sequentially inserted between two adjacent frames in the fused image sequence to obtain the interpolated fused image sequence.

[0120] In one embodiment, the preset mark includes an ultrasound probe in an IVUS catheter and / or a Mark mark in the IVUS catheter that moves with the ultrasound probe, and the preset mark of the interpolated fused image sequence is identified, and the target image sequence and the IVUS image corresponding to the target image sequence are processed based on the preset mark, including:

[0121] Identifying the position information of the IVUS ultrasound probe and / or the Mark in the fused image sequence after the interpolation;

[0122] A non-rigid transformation is performed based on the position information of the IVUS ultrasound probe and / or the Mark marker to obtain the processing results of the target image sequence and the IVUS image, and the processing results include image registration results.

[0123] Reference Figure 6 , which is a schematic diagram of a preferred embodiment of the electronic device 1 of the present application.

[0124] The electronic device 1 includes but is not limited to: a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other via the communication bus 114, the memory 113 is used to store computer programs, and when the processor 111 is used to execute the programs stored in the memory 113, the following steps are implemented:

[0125] Acquire angiographic image sequences with contrast agent, fluoroscopic image sequences without contrast agent, and IVUS images;

[0126] Determine, according to the image features of the guiding catheter in the image, a target image sequence in the angiography image sequence and a plurality of target perspective image sequences in the perspective image sequence, and align each of the target perspective image sequences with the target image sequence;

[0127] fusing the aligned target perspective image sequence and the target image sequence to obtain a fused image sequence;

[0128] Determining the number of interpolated frames of the fused image sequence according to the number of frames of the fused image sequence and the number of frames of the IVUS image;

[0129] Inserting interpolated frame images corresponding to the number of interpolated frames into the fused image sequence to obtain an interpolated fused image sequence;

[0130] A preset mark of the inserted fused image sequence is identified, and based on the preset mark, the target image sequence and the IVUS image corresponding to the target image sequence are subjected to image processing, wherein the image processing includes image registration.

[0131] For a detailed description of the above steps, please refer to the above Figure 5 Functional module diagram of an embodiment of an image processing device 100 based on interpolation method and Figure 1 Description of a flowchart of an embodiment of an image processing method based on interpolation method.

[0132] In addition, the embodiment of the present application further proposes a computer-readable storage medium, which may be non-volatile or volatile. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following operations are implemented:

[0133] Acquire angiographic image sequences with contrast agent, fluoroscopic image sequences without contrast agent, and IVUS images;

[0134] Determine, according to the image features of the guiding catheter in the image, a target image sequence in the angiography image sequence and a plurality of target perspective image sequences in the perspective image sequence, and align each of the target perspective image sequences with the target image sequence;

[0135] fusing the aligned target perspective image sequence and the target image sequence to obtain a fused image sequence;

[0136] Determining the number of interpolated frames of the fused image sequence according to the number of frames of the fused image sequence and the number of frames of the IVUS image;

[0137] Inserting interpolated frame images corresponding to the number of interpolated frames into the fused image sequence to obtain an interpolated fused image sequence;

[0138] A preset mark of the inserted fused image sequence is identified, and based on the preset mark, the target image sequence and the IVUS image corresponding to the target image sequence are subjected to image processing, wherein the image processing includes image registration.

[0139] The specific implementation of the computer-readable storage medium of the present application is substantially the same as the specific implementation of the above-mentioned image processing method based on the interpolation method, and will not be described in detail here.

[0140] It should be noted that the serial numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof herein are intended to cover non-exclusive inclusion, so that the process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the presence of other identical elements in the process, device, article or method including the element.

[0141] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware simulation platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, an electronic device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0142] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image processing method based on interpolation method, characterized in that: The method comprises: Acquire angiographic image sequences with contrast agent, fluoroscopic image sequences without contrast agent, and IVUS images; Determine, according to the image features of the guiding catheter in the image, a target image sequence in the angiography image sequence and a plurality of target perspective image sequences in the perspective image sequence, and align each of the target perspective image sequences with the target image sequence; fusing the aligned target perspective image sequence and the target image sequence to obtain a fused image sequence; Determining the number of interpolated frames of the fused image sequence according to the number of frames of the fused image sequence and the number of frames of the IVUS image; Inserting interpolated frame images corresponding to the number of interpolated frames into the fused image sequence to obtain an interpolated fused image sequence; A preset mark of the inserted fused image sequence is identified, and based on the preset mark, the target image sequence and the IVUS image corresponding to the target image sequence are subjected to image processing, wherein the image processing includes image registration.

2. The image processing method based on interpolation method according to claim 1, characterized in that: The step of determining a target image sequence in the angiography image sequence and a plurality of target perspective image sequences in the perspective image sequence respectively according to the image features of the guiding catheter in the image comprises: Analyze and obtain a target image sequence in the angiography image sequence according to the periodic variation rule of the guiding catheter in the angiography image sequence and the image information of the angiography image sequence; According to the periodic variation rule of the guiding catheter of the perspective image sequence, the perspective image sequence is grouped to obtain a plurality of target perspective image sequences.

3. The image processing method based on interpolation method according to claim 2, characterized in that: The step of analyzing and obtaining a target image sequence in the angiography image sequence based on the periodic variation rule of the guiding catheter in the angiography image sequence and the image information of the angiography image sequence comprises: Segmenting a guiding catheter region of the angiography image sequence, and calculating position information of a guiding catheter tip of the angiography image sequence based on the guiding catheter region; Acquiring the periodic variation rule of the position information of the tip of the guiding catheter of the angiography image sequence; Acquiring image information of the angiography image sequence, wherein the image information includes grayscale information; Based on the periodic variation rule and the image information of the angiography image sequence, a target image sequence in the angiography image sequence is analyzed and obtained.

4. The image processing method based on interpolation method according to claim 3, characterized in that: The target image sequence is an image sequence of a complete heartbeat cycle with the highest definition in the contrast image sequence.

5. The image processing method based on interpolation method as claimed in claim 3, characterized in that: The periodic variation rule of the position information of the tip of the guiding catheter of the angiography image sequence is obtained, including: performing a preprocessing operation on the time series data of the position information of the tip of the guiding catheter in the angiography image sequence; The time series data after the preprocessing operation is analyzed using a periodic detection algorithm to obtain the periodic variation law of the position information of the tip of the guiding catheter in the angiography image sequence.

6. The image processing method based on interpolation method according to claim 1, characterized in that: The aligning each of the target perspective image sequences with the target image sequence comprises: Taking the target image sequence as a reference, each target perspective view sequence is respectively aligned with the target image sequence by sequence translation.

7. The image processing method based on interpolation method according to claim 1, characterized in that: The step of fusing the aligned target perspective image sequence and the target image sequence to obtain a fused image sequence includes: identifying the radiopaque tip of the guidewire in the target image sequence and the radiopaque tip of the guidewire in the target perspective image sequence; Performing a non-rigid transformation based on the opaque tip of the guidewire in the target image sequence and the opaque tip of the guidewire in the target perspective image sequence to obtain a fused image; The fused images are arranged according to the time sequence of the fused images to obtain the fused image sequence.

8. The image processing method based on interpolation method according to claim 1, characterized in that: The step of determining the number of interpolated frames of the fused image sequence according to the number of frames of the fused image sequence and the number of frames of the IVUS image comprises: The number of interpolated frames is calculated using the following formula, which includes: S=M / (N-1) Wherein, S represents the number of interpolated frames, M represents the number of frames of the IVUS image, and N represents the number of frames of the fused image sequence.

9. The image processing method based on interpolation method according to claim 1, characterized in that: Inserting the interpolated frame images corresponding to the number of interpolated frames into the fused image sequence to obtain the interpolated fused image sequence includes: Based on the motion estimation interpolation algorithm or the optical flow interpolation algorithm, one frame of the interpolation image is sequentially inserted between two adjacent frames in the fused image sequence to obtain the interpolated fused image sequence.

10. The image processing method based on interpolation method according to claim 1, characterized in that: The preset mark includes an ultrasound probe in an IVUS catheter and / or a Mark mark in the IVUS catheter that moves with the ultrasound probe, the preset mark of the interpolated fused image sequence is identified, and the target image sequence and the IVUS image corresponding to the target image sequence are processed based on the preset mark, including: Identifying the position information of the IVUS ultrasound probe and / or the Mark in the fused image sequence after the interpolation; A non-rigid transformation is performed based on the position information of the IVUS ultrasound probe and / or the Mark marker to obtain the processing results of the target image sequence and the IVUS image, and the processing results include image registration results.

11. An image processing device based on interpolation method, characterized in that: The device comprises: Acquisition module: used to acquire angiographic image sequences with contrast agents, fluoroscopic image sequences without contrast agents, and IVUS images; An alignment module: used for determining a target image sequence in the angiography image sequence and a plurality of target perspective image sequences in the perspective image sequence respectively according to image features of the guiding catheter in the image, and aligning each of the target perspective image sequences with the target image sequence; Fusion module: used for fusing the aligned target perspective image sequence and the target image sequence to obtain a fused image sequence; A determination module: used to determine the number of interpolated frames of the fused image sequence according to the number of frames of the fused image sequence and the number of frames of the IVUS image; Insertion module: used for inserting the insertion frame images corresponding to the insertion frame number into the fused image sequence to obtain the inserted fused image sequence; Processing module: used for identifying the preset mark of the inserted fusion image sequence, and performing image processing on the target image sequence and the IVUS image corresponding to the target image sequence based on the preset mark, wherein the image processing includes image registration.

12. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor is used to implement the image processing method based on the interpolation method described in any one of claims 1 to 10 when executing the program stored in the memory.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image processing method based on the interpolation method according to any one of claims 1 to 10 is implemented.

Images