Blood vessel registration method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a blood vessel registration method and device, electronic equipment and a storage medium. The method comprises the following steps: for a first image and a second image in a radiography sequence of a target blood vessel, extracting a first contour of the target blood vessel in the first image and a second contour of the target blood vessel in the second image; for each first contour point pair on the first contour, determining a first feature of the first contour point pair, and obtaining a first feature sequence according to the first feature corresponding to each first contour point pair; for each second contour point pair on the second contour, determining a second feature of the second contour point pair, and obtaining a second feature sequence according to the second feature corresponding to each second contour point pair; and registering the target blood vessel in the first image and the target blood vessel in the second image according to the first feature sequence and the second feature sequence. According to the technical scheme of the embodiment of the invention, the accuracy of blood vessel registration can be improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing technology, and in particular to a blood vessel registration method, device, electronic device and storage medium. Background Art

[0002] In recent years, the incidence of coronary heart disease caused by coronary artery stenosis has remained high. In order to achieve effective detection and treatment of coronary heart disease, coronary artery imaging is currently mainly based on digital subtraction angiography (DSA), and image analysis is performed on this basis.

[0003] It should be noted that the performance of the coronary arteries may vary at different times, so it is crucial to perform vascular registration on each frame of the angiography image in the angiography sequence, which helps provide accurate vascular structure information and assists doctors in diagnosis and evaluation, surgical planning, and preoperative simulation.

[0004] However, the registration accuracy of the currently used vascular registration scheme is not high, which needs to be solved urgently. Summary of the invention

[0005] Embodiments of the present invention provide a blood vessel registration method, device, electronic device and storage medium to improve the accuracy of blood vessel registration.

[0006] According to one aspect of the present invention, a blood vessel registration method is provided, which may include:

[0007] For a first image and a second image in an angiography sequence of a target blood vessel, extract a first contour of the target blood vessel in the first image and a second contour of the target blood vessel in the second image;

[0008] For each first contour point pair on the first contour, determine the first feature of the first contour point pair, and obtain a first feature sequence according to the first features corresponding to each first contour point pair;

[0009] For each second contour point pair on the second contour, determine the second feature of the second contour point pair, and obtain a second feature sequence according to the second features corresponding to each second contour point pair;

[0010] The target blood vessel in the first image is registered with the target blood vessel in the second image according to the first feature sequence and the second feature sequence.

[0011] Optionally, determining a first feature of a first contour point pair includes:

[0012] For each first contour point in the first contour point pair, determine the position feature of each first contour point respectively, and / or determine the diameter feature of the diameter segment formed based on each first contour point;

[0013] A first feature of the first contour point pair is obtained according to the position feature and / or the diameter feature.

[0014] Optionally, registering the target blood vessel in the first image with the target blood vessel in the second image according to the first feature sequence and the second feature sequence includes:

[0015] Feature matching is performed between each first feature in the first feature sequence and each second feature in the second feature sequence, and based on the obtained feature matching result, the target blood vessel in the first image is registered with the target blood vessel in the second image.

[0016] On this basis, optionally, feature matching is performed between each first feature in the first feature sequence and each second feature in the second feature sequence, and according to the obtained feature matching result, the target blood vessel in the first image and the target blood vessel in the second image are registered, including:

[0017] Get the preset dynamic time warping algorithm;

[0018] Based on the dynamic time warping algorithm, feature matching is performed between each first feature in the first feature sequence and each second feature in the second feature sequence, and based on the obtained feature matching results, the target blood vessel in the first image is aligned with the target blood vessel in the second image.

[0019] On this basis, optionally, the first feature includes at least the third feature under the first feature dimension and the fourth feature under the second feature dimension, and the second feature includes at least the fifth feature under the first feature dimension and the sixth feature under the second feature dimension;

[0020] The dynamic time warping algorithm is a multi-dimensional dynamic time warping algorithm;

[0021] The feature matching result includes a first matching result under a first feature dimension and a second matching result under a second feature dimension.

[0022] On this basis, optionally, based on a dynamic time warping algorithm, feature matching is performed between each first feature in the first feature sequence and each second feature in the second feature sequence, and according to the obtained feature matching results, the target blood vessel in the first image is registered with the target blood vessel in the second image, including:

[0023] For each third feature in the first feature sequence and each fifth feature in the second feature sequence, based on a multi-dimensional dynamic time warping algorithm, feature matching is performed between each third feature and each fifth feature to obtain a first matching result;

[0024] For each fourth feature in the second feature sequence and each sixth feature in the second feature sequence, based on a multi-dimensional dynamic time warping algorithm, feature matching is performed between each fourth feature and each sixth feature to obtain a second matching result;

[0025] The target blood vessel in the first image is registered with the target blood vessel in the second image according to the first matching result and the second matching result.

[0026] On this basis, optionally, feature matching is performed between each third feature and each fifth feature to obtain a first matching result, including:

[0027] For the current feature in each third feature, the matching direction of the previous feature of the current feature is obtained, and based on the target direction corresponding to the matching direction, feature matching is performed between each fifth feature to obtain a first matching result, wherein the target direction includes directions other than the opposite direction of the matching direction.

[0028] Alternatively, the first characteristic dimension includes a position dimension, and the second characteristic dimension includes a diameter dimension.

[0029] Optionally, before extracting the first contour of the target blood vessel in the first image and the second contour of the target blood vessel in the second image, the above-mentioned blood vessel registration method also includes: segmenting the target blood vessel in the first image to obtain a first segmentation result, and updating the first image based on the first segmentation result; segmenting the target blood vessel in the second image to obtain a second segmentation result, and updating the second image based on the second segmentation result.

[0030] According to another aspect of the present invention, a blood vessel registration device is provided, which may include:

[0031] A contour extraction module, for extracting a first contour of the target blood vessel in the first image and a second contour of the target blood vessel in the second image, with respect to the first image and the second image in the angiography sequence of the target blood vessel;

[0032] A first feature sequence obtaining module, for determining the first feature of each first contour point pair on the first contour, and obtaining a first feature sequence according to the first features corresponding to each first contour point pair;

[0033] A second feature sequence obtaining module, for determining the second feature of each second contour point pair on the second contour, and obtaining a second feature sequence according to the second features corresponding to each second contour point pair;

[0034] The blood vessel registration module is used to register the target blood vessel in the first image with the target blood vessel in the second image according to the first feature sequence and the second feature sequence.

[0035] According to another aspect of the present invention, there is provided an electronic device, which may include:

[0036] at least one processor; and

[0037] a memory communicatively connected to at least one processor; wherein,

[0038] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by at least one processor so that the at least one processor implements the blood vessel registration method provided by any embodiment of the present invention when executing the computer program.

[0039] According to another aspect of the present invention, a computer-readable storage medium is provided, on which computer instructions are stored. The computer instructions are used to enable a processor to implement the blood vessel registration method provided by any embodiment of the present invention when executed.

[0040] The technical solution of the embodiment of the present invention is to extract the first contour of the target blood vessel in the first image and the second contour in the second image for the first image and the second contour of the target blood vessel in the angiography sequence; determine the first feature of the first contour point pair for each first contour point pair on the first contour, and obtain a first feature sequence according to the first features corresponding to each first contour point pair; determine the second feature of the second contour point pair for each second contour point pair on the second contour, and obtain a second feature sequence according to the second features corresponding to each second contour point pair; and align the target blood vessel in the first image with the target blood vessel in the second image according to the first feature sequence and the second feature sequence. In the above technical solution, compared with the features of each center point on the center line of the target blood vessel in the angiography image, each contour point pair on the contour of the angiography image has richer features. Therefore, the characteristics of each contour point pair are used to perform blood vessel alignment, thereby improving the accuracy of target blood vessel alignment.

[0041] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 is a flow chart of a blood vessel registration method provided according to an embodiment of the present invention;

[0044] Figure 2 is a schematic diagram of contour extraction in a blood vessel registration method provided according to an embodiment of the present invention;

[0045] Figure 3 is based on Figure 2 A schematic diagram of the diameter segment formed by each pair of contour points on the contour extracted from ;

[0046] Figure 4 is a flow chart of another blood vessel registration method provided according to an embodiment of the present invention;

[0047] Figure 5 is a flow chart of another blood vessel registration method provided according to an embodiment of the present invention;

[0048] Figure 6 is a schematic diagram of a result of a diameter curve matching example in another blood vessel registration method provided by an embodiment of the present invention;

[0049] Figure 7 is a flowchart of an optional example of another blood vessel registration method provided according to an embodiment of the present invention;

[0050] Figure 8 is a structural block diagram of a blood vessel registration device provided according to an embodiment of the present invention;

[0051] Fig. 9 It is a schematic diagram of the structure of an electronic device for implementing the blood vessel registration method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0053] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The situations of "target", "original", etc. are similar and will not be repeated here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0054] Figure 1 is a flow chart of a blood vessel registration method provided in an embodiment of the present invention. This embodiment is applicable to the case of blood vessel registration, and is particularly applicable to the case of registering target blood vessels in two frames of angiography images. The method can be executed by a blood vessel registration device provided in an embodiment of the present invention, and the device can be implemented in software and / or hardware, and the device can be integrated in an electronic device, and the electronic device can be various user terminals or servers.

[0055] See also Figure 1 The method of the embodiment of the present invention specifically includes the following steps:

[0056] S110 . For a first image and a second image in an angiography sequence of a target blood vessel, extract a first contour of the target blood vessel in the first image and a second contour of the target blood vessel in the second image.

[0057] Among them, the target blood vessel can be understood as a blood vessel that has been imaged based on the digital subtraction angiography (DSA) imaging technology and has a registration requirement. In practical applications, optionally, the target blood vessel can be, for example, an arterial blood vessel, a venous blood vessel or a capillary blood vessel, wherein the arterial blood vessel can be, for example, a cardiac artery, a brachiocephalic artery, a coronary artery or a common carotid artery, etc., which can be set according to actual needs and is not specifically limited here. Optionally, the blood vessel that has been imaged by DSA is referred to as an imaging blood vessel, and the target blood vessel may be the imaging blood vessel as a whole; it may also be a part of the blood vessel in the imaging blood vessel. Here, the imaging blood vessel obtained by DSA imaging of the coronary artery is taken as an example. The target blood vessel is usually the narrow part of the imaging blood vessel. The registration result of such a target blood vessel can better assist the detection and treatment process of coronary heart disease; etc., which are not limited here.

[0058] The angiography sequence can be understood as a sequence obtained after DSA imaging of the target blood vessel, and the angiography sequence may include at least two frames of angiography images. The first image and the second image can be understood as two frames of angiography images in the angiography sequence. In practical applications, they can be two frames of angiography images acquired at intervals (i.e., non-adjacent frames); or they can be two frames of angiography images acquired continuously (i.e., adjacent frames). Compared with the former, this can better ensure the accuracy of the registration of the target blood vessels in the two frames of angiography images.

[0059] The first contour can be understood as the contour of the target blood vessel in the first image. On this basis, it can be understood that since the target blood vessel is a pipe for blood circulation, that is, it is in the shape of a pipe, the first contour of the target blood vessel in the first image can be represented by two contour lines. For example, see Figure 2 , where the white filled area is the target blood vessel, and the target blood vessel has two corresponding contour lines. Extract the first contour.

[0060] The second contour of the target blood vessel in the second image is similar and will not be described in detail here.

[0061] S120. For each first contour point pair on the first contour, determine the first feature of the first contour point pair, and obtain a first feature sequence according to the first features corresponding to each first contour point pair.

[0062] Among them, as mentioned above, the first contour can be represented by two contour lines, and on this basis, each contour line is composed of multiple contour points, which makes there a corresponding relationship between each contour point on one contour line and each contour point on another contour line. Here, two or more contour points that have a corresponding relationship on the first contour are called a first contour point pair, and there are multiple first contour point pairs on the first contour.

[0063] Determine the first features corresponding to each pair of first contour points on the first contour. In practical applications, optionally, the first feature may be at least one of a position feature, a diameter feature, and a quantity feature, etc. This can be set according to actual needs and is not specifically limited here.

[0064] Further, a first feature sequence is obtained based on the first features corresponding to each first contour point pair. In practical applications, optionally, a first feature sequence can be formed based on the first features corresponding to each first contour point pair (i.e., all first features); key first features can also be selected from all first features, and then a first feature sequence can be formed based on the key first features; and so on, which are not specifically limited here.

[0065] S130. For each second contour point pair on the second contour, determine the second feature of the second contour point pair, and obtain a second feature sequence according to the second features corresponding to each second contour point pair.

[0066] The second feature is similar to the first feature, and will not be described in detail here. The second feature sequence is similar to the first feature sequence, and will not be described in detail here.

[0067] S140. Register the target blood vessel in the first image with the target blood vessel in the second image according to the first feature sequence and the second feature sequence.

[0068] Among them, since the first feature sequence can characterize a series of features of the target blood vessel in the first image, and the second feature sequence can characterize a series of features of the target blood vessel in the second image, the target blood vessel in the first image and the target blood vessel in the second image can be registered based on the first feature sequence and the second feature sequence, thereby achieving the registration of the target blood vessels in the two frames of angiography images.

[0069] On this basis, it should be noted that, in combination with the application scenarios that may be involved in the embodiments of the present invention, the significance of coronary vessel registration in coronary angiography images is:

[0070] 1. Diagnosis and evaluation: By aligning multiple frames of angiography images, doctors can more intuitively compare the changes in coronary arteries at different time points, such as stenosis, blockage, or dilation. This can help doctors diagnose and evaluate the progression of coronary heart disease more accurately.

[0071] 2. Surgical planning: By aligning multiple frames of angiography images, doctors can more intuitively compare angiography images at different time points or locations. This helps assist doctors in surgical path and decision planning, thereby determining the best surgical plan and improving the success rate and safety of the operation.

[0072] 3. Preoperative simulation: By aligning multiple frames of angiography images, doctors can understand the vascular structure and dynamic characteristics of the coronary arteries. This helps doctors simulate the coronary arteries on the computer, predict the effects of the operation and possible complications, and thus guide surgical decision-making and preparation.

[0073] The technical solution of the embodiment of the present invention is to extract the first contour of the target blood vessel in the first image and the second contour in the second image for the first image and the second contour of the target blood vessel in the second image for the first image and the second image; determine the first feature of the first contour point pair for each first contour point pair on the first contour, and obtain a first feature sequence according to the first feature corresponding to each first contour point pair; determine the second feature of the second contour point pair for each second contour point pair on the second contour, and obtain a second feature sequence according to the second feature corresponding to each second contour point pair; and align the target blood vessel in the first image with the target blood vessel in the second image according to the first feature sequence and the second feature sequence. In the above technical solution, compared with the features of each center point on the center line of the target blood vessel in the contrast image, each contour point pair on the contour of the contrast image has richer features, so here the blood vessel is aligned by using the features of each contour point pair, thereby improving the accuracy of the target blood vessel alignment.

[0074] An optional technical solution for determining a first feature of a first contour point pair includes:

[0075] For each first contour point in the first contour point pair, determine the position feature of each first contour point respectively, and / or determine the diameter feature of the diameter segment formed based on each first contour point;

[0076] A first feature of the first contour point pair is obtained according to the position feature and / or the diameter feature.

[0077] Wherein, for each first contour point pair among the plurality of first contour point pairs on the first contour, the first contour point pair may include at least two first contour points, on this basis, the position feature of each first contour point among the at least two first contour points is determined respectively, and / or the diameter feature (i.e., length feature) of the diameter segment formed by the at least two first contour points is determined. Exemplarily, Figure 3 The diameter segment formed by each first contour point pair in the plurality of first contour point pairs is illustrated. Further, the first feature of the first contour point pair is obtained according to the position feature and / or the diameter feature.

[0078] Exemplarily, here the kth first contour point pair on the first contour includes two first contour points (i.e., A k and B k ,For example Figure 2 Take the two black solid circles in the upper left part of the figure as an example, A k The position characteristics can be obtained by (X ka ,Y ka ) is used to represent B k The position characteristics can be obtained by (X kb ,Y kb ) is represented by Ak and B k The diameter segment D k (Right now Figure 2 The diameter feature of the line connecting the two black solid circles in the figure can be based on D k The first feature of the kth first contour point pair can be expressed by V k (X ka ,Y ka ,X kb ,Y kb ,D k ) is used to represent it. In practical applications, A k and B k Can be called diameter segment D k The two endpoints on .

[0079] On this basis, assuming that the first image is the mth frame of the imaging sequence, and has n first contour point pairs, the first feature sequence can be expressed as FeatureFrame m =[v 1 ,v 2 ,v 3 ...v n ].

[0080] The above technical solution, through feature representation based on position features and / or diameter features, combined with subsequent steps, can find the matching relationship between contour point pairs (i.e., blood vessel contours) between different frames and / or the matching relationship between blood vessel diameters between different frames, which is the key to subsequent blood vessel alignment.

[0081] Another optional technical solution is that before extracting the first contour of the target blood vessel in the first image and the second contour of the target blood vessel in the second image, the blood vessel registration method further includes:

[0082] Segmenting the target blood vessel in the first image to obtain a first segmentation result, and updating the first image based on the first segmentation result;

[0083] The target blood vessel in the second image is segmented to obtain a second segmentation result, and the second image is updated based on the second segmentation result.

[0084] In order to improve the accuracy of the first contour extraction, the target blood vessel in the first image may be segmented, so that contour extraction is performed based on the obtained first segmentation result (i.e., the segmentation result of the target blood vessel in the first image). In practical applications, the target blood vessel segmentation process may be optionally implemented based on a threshold segmentation algorithm, an edge segmentation algorithm, a region segmentation algorithm, a deep learning neural network, or a genetic algorithm, etc. This may be set according to actual conditions and is not specifically limited here. Similarly, in order to improve the accuracy of the second contour extraction, the processing process of the second image is similar and will not be repeated here.

[0085] The above technical solution reduces interference information in the angiography image by segmenting the target blood vessel in the angiography image, thereby ensuring the accuracy of the extraction of the first contour and the second contour.

[0086] Figure 4 : is a flow chart of another blood vessel registration method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, according to the first feature sequence and the second feature sequence, the target blood vessel in the first image is registered with the target blood vessel in the second image, including: performing feature matching between each first feature in the first feature sequence and each second feature in the second feature sequence, and according to the obtained feature matching results, registering the target blood vessel in the first image with the target blood vessel in the second image. Among them, the explanations of the terms that are the same as or corresponding to the above-mentioned embodiments are not repeated here.

[0087] See also Figure 4 The method of this embodiment may specifically include the following steps:

[0088] S210. For a first image and a second image in an angiography sequence of a target blood vessel, extract a first contour of the target blood vessel in the first image and a second contour of the target blood vessel in the second image.

[0089] S220. For each first contour point pair on the first contour, determine the first feature of the first contour point pair, and obtain a first feature sequence according to the first features corresponding to each first contour point pair.

[0090] S230. For each second contour point pair on the second contour, determine the second feature of the second contour point pair, and obtain a second feature sequence according to the second features corresponding to each second contour point pair.

[0091] S240. Perform feature matching between each first feature in the first feature sequence and each second feature in the second feature sequence, and align the target blood vessel in the first image with the target blood vessel in the second image according to the obtained feature matching result.

[0092] Among them, according to the above description, the first feature sequence includes the first features corresponding to each first contour point pair on the first contour, and the second feature sequence includes the second features corresponding to each second contour point pair on the second contour. Therefore, feature matching can be performed between each first feature and each second feature to obtain a feature matching result, or in other words, contour point pair matching results between each first contour point pair and each second contour point pair can be obtained by performing feature matching, and then blood vessel alignment can be achieved based on the obtained matching results.

[0093] In practical applications, optionally, the above-mentioned feature matching process can be based on a dynamic time warping (DTW) algorithm, a scale-invariant feature transform (SIFT) algorithm, or a local feature matching with Transformer (LoFTR) algorithm, etc., which is not specifically limited here.

[0094] The technical solution of the embodiment of the present invention realizes the matching of contour points in the two frames of angiography images by matching the features in the two frames of angiography images, and further realizes the blood vessel registration in the two frames of angiography images on this basis, thereby realizing the effective registration of the target blood vessels.

[0095] An optional technical solution is to perform feature matching between each first feature in the first feature sequence and each second feature in the second feature sequence, and to register the target blood vessel in the first image with the target blood vessel in the second image according to the obtained feature matching result, including:

[0096] Get the preset dynamic time warping algorithm;

[0097] Based on the dynamic time warping algorithm, feature matching is performed between each first feature in the first feature sequence and each second feature in the second feature sequence, and based on the obtained feature matching results, the target blood vessel in the first image is aligned with the target blood vessel in the second image.

[0098] Among them, since the first image and the second image are contrast images in the contrast sequence, that is, time series data, and the DTW algorithm is the simplest and most effective algorithm among the various similarity or distance functions existing in time series data, the technical solution realizes feature matching between each first feature and each second feature based on the DTW algorithm, thereby ensuring the efficiency and accuracy of feature matching.

[0099] Figure 5It is a flow chart of another blood vessel registration method provided in an embodiment of the present invention. This embodiment is optimized based on the above-mentioned technical solutions. In this embodiment, optionally, the first feature includes at least the third feature under the first feature dimension and the fourth feature under the second feature dimension, and the second feature includes at least the fifth feature under the first feature dimension and the sixth feature under the second feature dimension; the dynamic time warping algorithm is a multi-dimensional dynamic time warping algorithm; the feature matching result includes the first matching result under the first feature dimension and the second matching result under the second feature dimension. On this basis, further optionally, based on the dynamic time warping algorithm, feature matching is performed between each first feature in the first feature sequence and each second feature in the second feature sequence, and according to the obtained feature matching results, the target blood vessel in the first image is aligned with the target blood vessel in the second image, including: for each third feature in the first feature sequence and each fifth feature in the second feature sequence, based on the multidimensional dynamic time warping algorithm, feature matching is performed between each third feature and each fifth feature to obtain a first matching result; for each fourth feature in the second feature sequence and each sixth feature in the second feature sequence, based on the multidimensional dynamic time warping algorithm, feature matching is performed between each fourth feature and each sixth feature to obtain a second matching result; according to the first matching result and the second matching result, the target blood vessel in the first image is aligned with the target blood vessel in the second image. The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here.

[0100] See also Figure 5 The method of this embodiment may specifically include the following steps:

[0101] S310. For a first image and a second image in an angiography sequence of a target blood vessel, extract a first contour of the target blood vessel in the first image and a second contour of the target blood vessel in the second image.

[0102] S320. For each first contour point pair on the first contour, determine the first feature of the first contour point pair, and obtain a first feature sequence according to the first features corresponding to each first contour point pair, wherein the first feature includes at least the third feature under the first feature dimension and the fourth feature under the second feature dimension.

[0103] Among them, the first characteristic dimension and the second characteristic dimension are different characteristic dimensions. Here, the position feature and diameter feature in the above example are taken as an example. Optionally, the first characteristic dimension can be the position dimension, and the second characteristic dimension can be the diameter dimension. Of course, vice versa is also possible, and no specific limitation is made here.

[0104] It should be noted that, in the embodiment of the present invention, the first feature at least includes the third feature under the first feature dimension and the fourth feature under the second feature dimension.

[0105] S330. For each second contour point pair on the second contour, determine the second feature of the second contour point pair, and obtain a second feature sequence according to the second features corresponding to each second contour point pair, wherein the second features include at least the fifth feature under the first feature dimension and the sixth feature under the second feature dimension.

[0106] Among them, similar to the first feature, the second feature at least includes the fifth feature under the first feature dimension and the sixth feature under the second feature dimension.

[0107] S340. Obtain a preset multi-dimensional dynamic time warping algorithm.

[0108] According to the above description, the first feature and the second feature are both features under at least two feature dimensions. Therefore, compared with the 1-dimensional DTW algorithm, the multidimensional dynamic time warping (MD-DTW) algorithm with penalty weights is more suitable for the feature matching process between the first feature and the second feature. Get the MD-DTW algorithm.

[0109] S350. For each third feature in the first feature sequence and each fifth feature in the second feature sequence, based on a multi-dimensional dynamic time warping algorithm, feature matching is performed between each third feature and each fifth feature to obtain a first matching result under the first feature dimension.

[0110] Based on the MD-DTW algorithm, feature matching is performed on each third feature and each fifth feature under the first feature dimension, so as to obtain a first matching result under the first feature dimension.

[0111] S360. For each fourth feature in the second feature sequence and each sixth feature in the second feature sequence, based on the multi-dimensional dynamic time warping algorithm, feature matching is performed between each fourth feature and each sixth feature to obtain a second matching result under the second feature dimension.

[0112] Based on the MD-DTW algorithm, feature matching is performed on each fourth feature and each sixth feature in the second feature dimension, so as to obtain a second matching result in the second feature dimension.

[0113] For example, Figure 6 The feature matching results in the diameter dimension are shown. The ordinate in the figure is the diameter feature of each part of the target blood vessel. According to the transformation of the diameter curve, the MD-DTW algorithm is used to match the diameter curve, so that Figure 6 The feature matching results are shown.

[0114] S370. According to the first matching result and the second matching result, register the target blood vessel in the first image with the target blood vessel in the second image.

[0115] Wherein, according to the first matching result and the second matching result, that is, the feature matching results under different feature dimensions, the blood vessel registration in the two frames of angiography images is achieved.

[0116] The technical solution of the embodiment of the present invention performs feature matching of high-dimensional curve features (i.e., features in at least two feature dimensions) by using the MD-DTW algorithm, thereby ensuring the accuracy of feature matching.

[0117] On this basis, an optional technical solution is to perform feature matching between each third feature and each fifth feature to obtain a first matching result, including:

[0118] For the current feature in each third feature, the matching direction of the previous feature of the current feature is obtained, and based on the target direction corresponding to the matching direction, feature matching is performed between each fifth feature to obtain a first matching result, wherein the target direction includes directions other than the opposite direction of the matching direction.

[0119] Here, taking the third feature and the fifth feature as an example, in the process of feature matching between each third feature and each fifth feature, it is possible that one third feature matches multiple fifth features, which will affect the accuracy of subsequent vascular registration.

[0120] Therefore, to avoid the above situation, for the current feature currently being matched in each third feature, the matching direction of the previous feature of the current feature can be obtained. The matching direction can be understood as the direction in which the previous feature is applied when matching the fifth feature; then, the target direction is determined according to the matching direction. It should be noted that the target direction can be a direction other than the opposite direction of the matching direction, that is, the target direction cannot be the opposite direction of the matching direction. This is to avoid the situation where the previous feature and the current feature (that is, the next feature of the previous feature) are matched to the same fifth feature. For example, assuming that the matching direction is up, the target direction can be a direction other than down, for example, up, left or right, etc.; then, based on the target direction, the fifth feature corresponding to the current feature is matched in each fifth feature.

[0121] The above technical solution, by designing the matching logic of the MD-DTW algorithm, can avoid the one-to-many situation in the feature matching process, thereby ensuring the accuracy of subsequent vascular registration.

[0122] In practical applications, in addition to designing the matching logic, one-to-many situations can be avoided by other means, such as by pre-processing the number of contour points in two frames of contrast images to be approximately the same, or by post-processing interpolating or merging the corresponding contour points of the matched one-to-many features, etc., which are not specifically limited here.

[0123] In order to better understand the above-mentioned technical solutions as a whole, the following is an exemplary description of them in combination with specific examples. Figure 7 , obtain an angiography sequence of coronary vessels, and for any adjacent frames in the angiography sequence, here, take the adjacent frames as the first image and the second image as an example, use a convolutional neural network (CNN) to segment the coronary vessels in the first image to obtain a first segmentation result, and segment the coronary vessels in the second image to obtain a second segmentation result; extract the vessel contour based on the first segmentation result to obtain a first contour, and extract the vessel contour based on the second segmentation result to obtain a second contour; obtain the position feature (i.e., contour feature) of each first contour point pair on the first contour, and then calculate the diameter feature of the corresponding first contour point pair according to the position feature of each first contour point pair; obtain a first feature sequence according to the position feature and diameter feature corresponding to each first contour point pair; the process of obtaining the second feature sequence is similar and will not be repeated here; then, use the MD-DTW algorithm to perform feature matching between the first feature sequence and the second feature sequence, thereby realizing the registration of the coronary vessels in the first image with the coronary vessels in the second image.

[0124] The above technical solution uses the MD-DTW algorithm to perform a feature matching process of high-dimensional curve features (ie, position features and diameter features), thereby ensuring the accuracy of blood vessel registration.

[0125] Figure 8 This is a structural block diagram of a vascular registration device provided in an embodiment of the present invention, and the device is used to execute the vascular registration method provided in any of the above embodiments. The device and the vascular registration method of the above embodiments belong to the same inventive concept, and the details not described in detail in the embodiment of the vascular registration device can refer to the embodiment of the above vascular registration method. Figure 8 The device may specifically include: a contour extraction module 410, a first feature sequence acquisition module 420, a second feature sequence acquisition module 430 and a blood vessel registration module 440.

[0126] A contour extraction module 410 is used to extract a first contour of the target blood vessel in the first image and a second contour of the target blood vessel in the second image with respect to the first image and the second image in the angiography sequence of the target blood vessel;

[0127] A first feature sequence obtaining module 420 is used to determine the first feature of each first contour point pair on the first contour, and obtain a first feature sequence according to the first features corresponding to each first contour point pair;

[0128] A second feature sequence obtaining module 430 is used to determine the second feature of each second contour point pair on the second contour, and obtain a second feature sequence according to the second features corresponding to each second contour point pair;

[0129] The blood vessel registration module 440 is used to register the target blood vessel in the first image with the target blood vessel in the second image according to the first feature sequence and the second feature sequence.

[0130] Optionally, the first feature sequence obtaining module 420 may include: a diameter feature determining unit, configured to determine, for each first contour point in the first contour point pair, a position feature of each first contour point, and / or determine a diameter feature of a diameter segment formed by each first contour point;

[0131] The first feature obtaining unit is used to obtain the first feature of the first contour point pair according to the position feature and / or the diameter feature.

[0132] Optionally, the blood vessel registration module 440 may include:

[0133] The blood vessel registration submodule is used to perform feature matching between each first feature in the first feature sequence and each second feature in the second feature sequence, and to register the target blood vessel in the first image with the target blood vessel in the second image based on the obtained feature matching results.

[0134] On this basis, the optional vascular registration submodule may include:

[0135] A dynamic time warping algorithm acquisition unit, used to acquire a preset dynamic time warping algorithm;

[0136] The vascular registration unit is used to perform feature matching between each first feature in the first feature sequence and each second feature in the second feature sequence based on a dynamic time warping algorithm, and to register a target blood vessel in the first image with a target blood vessel in the second image according to the obtained feature matching results.

[0137] On this basis, optionally, the first feature includes at least the third feature under the first feature dimension and the fourth feature under the second feature dimension, and the second feature includes at least the fifth feature under the first feature dimension and the sixth feature under the second feature dimension;

[0138] The dynamic time warping algorithm is a multi-dimensional dynamic time warping algorithm;

[0139] The feature matching result includes a first matching result under a first feature dimension and a second matching result under a second feature dimension.

[0140] On this basis, an optional vessel registration unit may include:

[0141] A first matching result obtaining subunit is used for performing feature matching between each third feature in the first feature sequence and each fifth feature in the second feature sequence based on a multi-dimensional dynamic time warping algorithm to obtain a first matching result;

[0142] A second matching result obtaining subunit is used for performing feature matching between each fourth feature and each sixth feature in the second feature sequence based on a multi-dimensional dynamic time warping algorithm to obtain a second matching result;

[0143] The blood vessel registration subunit is used to register the target blood vessel in the first image with the target blood vessel in the second image according to the first matching result and the second matching result.

[0144] On this basis, optionally, the first matching result obtains a subunit, which can be specifically used for:

[0145] For the current feature in each third feature, the matching direction of the previous feature of the current feature is obtained, and based on the target direction corresponding to the matching direction, feature matching is performed between each fifth feature to obtain a first matching result, wherein the target direction includes directions other than the opposite direction of the matching direction.

[0146] Alternatively, the first characteristic dimension includes a position dimension, and the second characteristic dimension includes a diameter dimension.

[0147] Optionally, the above-mentioned blood vessel registration device may further include:

[0148] A first image updating module, configured to segment the target blood vessel in the first image before extracting a first contour of the target blood vessel in the first image and a second contour of the target blood vessel in the second image to obtain a first segmentation result, and update the first image based on the first segmentation result;

[0149] The second image updating module is used to segment the target blood vessel in the second image to obtain a second segmentation result, and update the second image based on the second segmentation result.

[0150] The blood vessel registration device provided by the embodiment of the present invention extracts the first contour of the target blood vessel in the first image and the second contour in the second image for the first image and the second image in the angiography sequence of the target blood vessel through the contour extraction module; determines the first feature of the first contour point pair for each first contour point pair on the first contour through the first feature sequence acquisition module, and obtains the first feature sequence according to the first feature corresponding to each first contour point pair; determines the second feature of the second contour point pair for each second contour point pair on the second contour through the second feature sequence acquisition module, and obtains the second feature sequence according to the second feature corresponding to each second contour point pair; and registers the target blood vessel in the first image with the target blood vessel in the second image according to the first feature sequence and the second feature sequence through the blood vessel registration module. Compared with the features of each center point on the center line of the target blood vessel in the angiography image, each contour point pair on the contour of the angiography image has richer features. Therefore, the accuracy of the target blood vessel registration is improved by using the features of each contour point pair for blood vessel registration.

[0151] The blood vessel registration device provided in the embodiment of the present invention can execute the blood vessel registration method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0152] It is worth noting that in the embodiment of the above-mentioned vascular alignment device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0153] Fig. 9 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0154] like Fig. 9As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0155] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0156] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a vascular registration method.

[0157] In some embodiments, the vascular registration method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the vascular registration method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the vascular registration method in any other appropriate manner (e.g., by means of firmware).

[0158] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0159] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0160] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0161] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0162] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0163] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0164] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0165] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A blood vessel registration method, It is characterized in that include: For a first image and a second image in an angiography sequence of a target blood vessel, extracting a first contour of the target blood vessel in the first image and a second contour of the target blood vessel in the second image; For each first contour point pair on the first contour, determine a first feature of the first contour point pair, and obtain a first feature sequence according to the first features respectively corresponding to each first contour point pair; For each second contour point pair on the second contour, determine a second feature of the second contour point pair, and obtain a second feature sequence according to the second features corresponding to each second contour point pair; The target blood vessel in the first image is registered with the target blood vessel in the second image according to the first feature sequence and the second feature sequence.

2. The method according to claim 1, It is characterized in that The determining of the first feature of the first contour point pair comprises: For each first contour point in the first contour point pair, respectively determine a position feature of each first contour point, and / or determine a diameter feature of a diameter segment formed based on each first contour point; A first feature of the first contour point pair is obtained according to the position feature and / or the diameter feature.

3. The method according to claim 1, It is characterized in that The registering the target blood vessel in the first image with the target blood vessel in the second image according to the first feature sequence and the second feature sequence includes: Feature matching is performed between each first feature in the first feature sequence and each second feature in the second feature sequence, and the target blood vessel in the first image is registered with the target blood vessel in the second image according to the obtained feature matching result.

4. The method according to claim 3, It is characterized in that The performing feature matching between each first feature in the first feature sequence and each second feature in the second feature sequence, and registering the target blood vessel in the first image with the target blood vessel in the second image according to the obtained feature matching result, comprises: Get the preset dynamic time warping algorithm; Based on the dynamic time warping algorithm, feature matching is performed between each first feature in the first feature sequence and each second feature in the second feature sequence, and based on the obtained feature matching results, the target blood vessel in the first image and the target blood vessel in the second image are aligned.

5. The method according to claim 4, It is characterized in that The first feature includes at least a third feature in the first feature dimension and a fourth feature in the second feature dimension, and the second feature includes at least a fifth feature in the first feature dimension and a sixth feature in the second feature dimension; The dynamic time warping algorithm is a multi-dimensional dynamic time warping algorithm; The feature matching result includes a first matching result under the first feature dimension and a second matching result under the second feature dimension.

6. The method according to claim 5, It is characterized in that The step of performing feature matching between each first feature in the first feature sequence and each second feature in the second feature sequence based on the dynamic time warping algorithm, and registering the target blood vessel in the first image with the target blood vessel in the second image according to the obtained feature matching result, includes: For each third feature in the first feature sequence and each fifth feature in the second feature sequence, based on the multidimensional dynamic time warping algorithm, feature matching is performed between the third features and the fifth features to obtain the first matching result; For each fourth feature in the second feature sequence and each sixth feature in the second feature sequence, based on the multidimensional dynamic time warping algorithm, feature matching is performed between each fourth feature and each sixth feature to obtain the second matching result; The target blood vessel in the first image is registered with the target blood vessel in the second image according to the first matching result and the second matching result.

7. The method according to claim 6, It is characterized in that Performing feature matching between the third features and the fifth features to obtain the first matching result includes: For the current feature among the third features, the matching direction of the previous feature of the current feature is obtained, and based on the target direction corresponding to the matching direction, feature matching is performed between the fifth features to obtain the first matching result, wherein the target direction includes directions other than the opposite direction of the matching direction.

8. The method according to claim 5, It is characterized in that The first characteristic dimension comprises a position dimension, and the second characteristic dimension comprises a diameter dimension.

9. The method according to claim 1, It is characterized in that Before extracting the first contour of the target blood vessel in the first image and the second contour of the target blood vessel in the second image, the method further includes: Segmenting the target blood vessel in the first image to obtain a first segmentation result, and updating the first image based on the first segmentation result; The target blood vessel in the second image is segmented to obtain a second segmentation result, and the second image is updated based on the second segmentation result.

10. A blood vessel registration device, It is characterized in that include: A contour extraction module, for extracting, from a first image and a second image in an angiography sequence of a target blood vessel, a first contour of the target blood vessel in the first image and a second contour of the target blood vessel in the second image; The first feature sequence obtaining module is used to determine the first feature of each first contour point pair on the first contour, and to obtain the first feature of each first contour point pair according to the first feature sequence of each first contour point pair. The first feature, obtains the first feature sequence; A second feature sequence obtaining module, used for determining, for each second contour point pair on the second contour, a second feature of the second contour point pair, and obtaining a second feature sequence according to the second features respectively corresponding to each second contour point pair; A blood vessel registration module is used to register the target blood vessel in the first image with the target blood vessel in the second image according to the first feature sequence and the second feature sequence.

11. An electronic device, It is characterized in that include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the blood vessel registration method according to any one of claims 1 to 9.

12. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the blood vessel registration method according to any one of claims 1 to 9 when executed.

Images