Method, device, equipment and medium for generating dynamic coronary artery roadmap

Abstract

The application relates to a dynamic coronary artery road map generation method, device, equipment and medium, wherein the method comprises the following steps: acquiring a first contrast image sequence of a target object, and acquiring a lead segmentation map corresponding to a real-time perspective view of the target object, the first contrast image sequence being generated according to original contrast images of the target object in an offline stage; taking the lead segmentation map and the first contrast image sequence as inputs, and outputting a target contrast image and a target simulated guide wire image through a target generation network, the target generation network being obtained by training blood vessel segmentation map data and simulated guide wire image data generated in the offline stage; registering the target contrast image by using the target simulated guide wire image and the lead segmentation map, to obtain a registration image; and fusing the registration image to the lead segmentation map, to obtain a dynamic coronary artery road map of the target object. The problem that the generated coronary artery road map has low precision due to the lack of diversity or easy breakage of the simulated guide wire is solved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, device and medium for generating dynamic coronary artery roadmaps. Background Technology

[0002] PCI (Percutaneous Coronary Intervention) is commonly used to treat diseases such as coronary artery stenosis and occlusion. It can improve myocardial blood flow, relieve angina, and reduce the risk of myocardial infarction. DCR (Dynamic Coronary Roadmap) generation technology is mainly used in PCI procedures for diseases such as coronary artery stenosis. This process is generally divided into two phases: an offline phase and an online phase.

[0003] While electrocardiography (ECG) is crucial for diagnosing heart disease, it doesn't directly participate in angiography or interventional procedures. Adding ECG functionality would also increase equipment manufacturing costs. Existing technologies offer solutions that don't rely on ECG signals. These solutions use offline coronary artery centerlines to generate simulated guidewires, compensating for cardiac motion artifacts by leveraging the structural similarity to online guidewires, and further compensating for respiratory motion artifacts through data augmentation. However, this approach results in a fixed number of pixels in the generated simulated guidewire, insufficient guidewire length information, a lack of diversity, and the guidewire obtained online is prone to breakage. This leads to low accuracy in the generated coronary artery path maps due to the lack of diversity or breakage of the simulated guidewire.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This application provides a method, apparatus, device, and medium for generating dynamic coronary artery pathway maps to solve the aforementioned technical problem of "low accuracy of generated coronary artery pathway maps due to the lack of diversity or easy breakage of simulated guidewires".

[0006] According to one aspect of the embodiments of this application, this application provides a method for generating a dynamic coronary artery path map, comprising: acquiring a first angiographic image sequence of a target object, and acquiring a lead wire segmentation map corresponding to a real-time perspective view of the target object, wherein the first angiographic image sequence is generated based on the original angiographic image of the target object in an offline stage; taking the lead wire segmentation map and the first angiographic image sequence as input, and outputting a target angiographic image and a target simulated guidewire map through a target generation network, wherein the target generation network is obtained by training on vascular segmentation map data and simulated guidewire map data generated in the offline stage; registering the target angiographic image using the target simulated guidewire map and the lead wire segmentation map to obtain a registered image; and fusing the registered image into the lead wire segmentation map to obtain a dynamic coronary artery path map of the target object.

[0007] Optionally, obtaining the first contrast image sequence of the target object includes: in an offline stage, obtaining the original contrast image of the target object, wherein the original contrast image includes multiple image frames; generating intermediate frames of the multiple image frames through frame interpolation; inserting the intermediate frames between the image frames of the original contrast image in chronological order to obtain the first contrast image sequence.

[0008] Optionally, before outputting the target angiogram and the target simulated guidewire image through the target generation network, the method further includes obtaining the target generation network in the following manner: constructing an initial generation network; obtaining a second angiogram sequence, and generating simulated guidewire image data and vessel segmentation image data based on the second angiogram sequence, wherein the second angiogram sequence is obtained through frame interpolation, and the simulated guidewire image data and vessel segmentation image data have a corresponding relationship; training the initial generation network using the simulated guidewire image data, vessel segmentation image data, and the second angiogram sequence to obtain the target generation network.

[0009] Optionally, the initial generative network is trained using simulated guidewire image data, vessel segmentation image data, and a second angiography image sequence to obtain the target generative network. This includes: extracting any corresponding first simulated guidewire image and first vessel segmentation image; using the first simulated guidewire image, first vessel segmentation image, and second angiography image sequence as input, and training for a preset number of times to obtain the target generative network. The first vessel segmentation image, starting from the first training iteration, undergoes weight adjustment according to the following formula: A = (1-α)((1-α)B + αN(0,1)), where A represents the network input after weight adjustment of the first vessel segmentation image, B represents the first vessel segmentation image, α represents the adjustment ratio, which increases by a preset step size, and N represents noise.

[0010] Optionally, generating simulated guidewire map data and vessel segmentation map data based on the second angiography map sequence includes: extracting any initial angiography map from the second angiography map sequence; segmenting the vessels and the first centerline in the initial angiography map; randomly truncating a second centerline of a preset length from the first centerline and using the second centerline as the initial simulated guidewire; generating a second simulated guidewire map corresponding to the initial simulated guidewire; generating a second vessel segmentation map corresponding to the second simulated guidewire map; performing an affine transformation operation on the second simulated guidewire map to obtain a third simulated guidewire map corresponding to the initial angiography map; determining the third simulated guidewire map corresponding to each initial angiography map in the second angiography map sequence as simulated guidewire map data; and determining the second vessel segmentation map corresponding to each initial angiography map in the second angiography map sequence as vessel segmentation map data.

[0011] Optionally, the target angiography image is registered using the target simulated guidewire image and the lead wire segmentation image to obtain a registered image, including: registering the target simulated guidewire image to the lead wire segmentation image to obtain an affine deformation field; and applying the affine deformation field to the target angiography image to obtain a registered image.

[0012] Optionally, the registered image is fused to the lead segmentation image to obtain a dynamic coronary artery path map of the target object, including: using the registered image as a background; converting the leads of the lead segmentation image to the target color; and fusing the leads of the target color with the registered image to obtain a dynamic coronary artery path map.

[0013] According to another aspect of the embodiments of this application, this application provides a dynamic coronary artery path diagram generation device, comprising: an acquisition module, configured to acquire a first angiography sequence of a target object and acquire a lead wire segmentation map corresponding to a real-time perspective view of the target object, wherein the first angiography sequence is generated based on the original angiography image of the target object in an offline stage; a generation module, configured to take the lead wire segmentation map and the first angiography sequence as inputs, and output a target angiography image and a target simulated guidewire map through a target generation network, wherein the target generation network is obtained by training on vascular segmentation map data and simulated guidewire map data generated in the offline stage; a registration module, configured to register the target angiography image using the target simulated guidewire map and the lead wire segmentation map to obtain a registered image; and a fusion module, configured to fuse the registered image to the lead wire segmentation map to obtain a dynamic coronary artery path diagram of the target object.

[0014] According to another aspect of the embodiments of this application, this application provides an electronic device, including a memory, a processor, a communication interface and a communication bus. The memory stores a computer program that can run on the processor. The memory and the processor communicate with each other through the communication bus and the communication interface. When the processor executes the computer program, it implements the steps of the above method.

[0015] According to another aspect of the embodiments of this application, this application also provides a computer-readable medium having processor-executable non-volatile program code that causes the processor to perform the above-described method.

[0016] Compared with related technologies, the technical solutions provided in this application have the following advantages:

[0017] This application discloses a method for generating dynamic coronary artery path maps, comprising: acquiring a first angiographic image sequence of a target object and acquiring a lead wire segmentation map corresponding to a real-time perspective view of the target object, wherein the first angiographic image sequence is generated based on the original angiographic images of the target object in an offline phase; using the lead wire segmentation map and the first angiographic image sequence as input, outputting a target angiographic image and a target simulated guidewire map through a target generation network, wherein the target generation network is trained on vascular segmentation map data and simulated guidewire map data generated in the offline phase; registering the target angiographic image using the target simulated guidewire map and the lead wire segmentation map to obtain a registered image; and fusing the registered image into the lead wire segmentation map to obtain a dynamic coronary artery path map of the target object. Since the target generation network of this application is trained on vascular segmentation map data and simulated guidewire map data from the offline phase, the target simulated guidewire map obtained by specifically using the target generation network has high accuracy, thus not affecting the accuracy of the subsequently generated coronary artery path map, solving the problem of low accuracy of the generated coronary artery path map due to the lack of diversity or easy breakage of simulated guidewires. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart of an optional method for generating a dynamic coronary artery path map according to an embodiment of this application;

[0021] Figure 2 This is a schematic diagram illustrating the training of an optional target generation network according to an embodiment of this application;

[0022] Figure 3 This is a block diagram of an optional dynamic coronary artery path generation device according to an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of an optional electronic device structure provided in an embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustration and has no specific meaning in itself. Therefore, "module" and "part" may be used interchangeably.

[0026] PCI (Percutaneous Coronary Intervention) is commonly used to treat diseases such as coronary artery stenosis and occlusion. It can improve myocardial blood flow, relieve angina, and reduce the risk of myocardial infarction. DCR (Dynamic Coronary Roadmap) generation technology is mainly used in PCI procedures for diseases such as coronary artery stenosis. This process is generally divided into two phases: an offline phase and an online phase.

[0027] While electrocardiography (ECG) is crucial for diagnosing heart disease, it doesn't directly participate in angiography or interventional procedures. Adding ECG functionality would also increase equipment manufacturing costs. Existing technologies offer solutions that don't rely on ECG signals. These solutions use offline coronary artery centerlines to generate simulated guidewires, compensating for cardiac motion artifacts by leveraging the structural similarity to online guidewires, and further compensating for respiratory motion artifacts through data augmentation. However, this approach results in a fixed number of pixels in the generated simulated guidewire, insufficient guidewire length information, a lack of diversity, and the guidewire obtained online is prone to breakage. This leads to low accuracy in the generated coronary artery path maps due to the lack of diversity or breakage of the simulated guidewire.

[0028] To address the problems mentioned in the background art, according to one aspect of an embodiment of this application, a method for generating dynamic coronary artery pathway maps is provided, such as... Figure 1 As shown, it includes:

[0029] Step 101: Obtain the first angiography sequence of the target object and obtain the lead segmentation map corresponding to the real-time perspective view of the target object, wherein the first angiography sequence is generated based on the original angiography image of the target object in the offline stage;

[0030] Step 103: The lead wire segmentation map and the first angiography map sequence are used as inputs, and the target angiography map and the target simulated guidewire map are output through the target generation network. The target generation network is trained on the vascular segmentation map data and simulated guidewire map data generated in the offline stage.

[0031] Step 105: Register the target angiography image using the target simulated guidewire map and the lead wire segmentation map to obtain the registered image;

[0032] Step 107: The registered image is fused into the lead segmentation map to obtain a dynamic coronary artery path map of the target object.

[0033] Since inaccurate simulated guidewire maps can lead to insufficient accuracy in the generated dynamic coronary artery pathway maps, this application proposes a method for generating dynamic coronary artery pathway maps, which improves the accuracy of dynamic coronary artery pathway map generation by improving the accuracy of simulated guidewire generation.

[0034] Specifically, to obtain the lead segmentation map corresponding to the real-time perspective view of the target object (i.e., the patient), it is necessary to first use appropriate medical imaging equipment to obtain the real-time perspective view of the target object, and then use an image segmentation algorithm to segment the real-time perspective view and extract the lead portion to obtain the lead segmentation map.

[0035] It should be noted that the real-time perspective view obtained is the same as the vascular enhancement image (i.e., angiography image) of the offline phase.

[0036] This application inputs the patient's guidewire segmentation map and the first angiographic image sequence into a pre-trained target generation network to obtain a target simulated guidewire map and a guidewire segmentation map predicted and output by the target generation network. Then, the target simulated guidewire map and the guidewire segmentation map are used for registration and fusion operations to obtain the patient's dynamic coronary artery path map. The solution provided in this application combines deep learning, image segmentation, registration, and fusion technologies to generate high-quality dynamic coronary artery path maps to assist clinical decision-making.

[0037] The embodiments provided in this application do not rely on electrocardiogram signals. Instead, they utilize the premise that the vascular centerline and guidewire structure are similar in Offline phase angiography data to train a simulated guidewire generation network, which is used to simulate the generation of guidewires with diverse morphological structures. This effectively alleviates cardiac motion artifacts and respiratory motion artifacts, and improves the generation accuracy of coronary artery roadmaps.

[0038] As an optional embodiment, obtaining the first contrast image sequence of the target object includes: in an offline stage, obtaining the original contrast image of the target object, wherein the original contrast image includes multiple image frames; generating intermediate frames of the multiple image frames through frame interpolation; and inserting the intermediate frames between the image frames of the original contrast image in chronological order to obtain the first contrast image sequence.

[0039] The original contrast images are usually a sequence of multiple image frames, each capturing the contrast of the target object at a certain moment.

[0040] To obtain smoother and more continuous image sequences, frame interpolation can be used to generate intermediate frames between the original image frames. Frame interpolation is an image processing technique that predicts and generates new image frames based on existing image frame information. Commonly used frame interpolation algorithms include linear interpolation, bilinear interpolation, and cubic spline interpolation. These algorithms can calculate the pixel values ​​of intermediate frames based on the pixel information of adjacent image frames, thereby generating realistic intermediate frames.

[0041] After generating the intermediate frames, they need to be inserted chronologically between the frames of the original contrast-enhanced images. This results in a first contrast-enhanced image sequence that contains more frames, is smoother, and is more continuous, thus more accurately reflecting the changes in the target object during the contrast-enhanced process.

[0042] This application employs frame interpolation to simulate and increase the sampling frequency of angiography images in the online phase. By generating more frames between two acquired frames, more detailed information can be presented, increasing the diversity of guidewire morphology and structure. This achieves the effect of improving the accuracy of cardiac cycle matching while correcting respiratory motion artifacts.

[0043] As an optional embodiment, before outputting the target angiogram and the target simulated guidewire map through the target generation network, the method further includes obtaining the target generation network in the following manner: constructing an initial generation network; obtaining a second angiogram sequence, and generating simulated guidewire map data and vessel segmentation map data based on the second angiogram sequence, wherein the second angiogram sequence is obtained through frame interpolation, and the simulated guidewire map data and vessel segmentation map data have a corresponding relationship; training the initial generation network using the simulated guidewire map data, vessel segmentation map data, and the second angiogram sequence to obtain the target generation network.

[0044] Figure 2 The training flowchart of the target generation network provided for this application is shown in the figure, including:

[0045] Step 201: Collect raw angiography images from multiple patients;

[0046] Step 203: Perform frame interpolation on each original angiography image to obtain an angiography image sequence;

[0047] Step 205: Construct a simulated guidewire diagram generation network and train the simulated guidewire diagram generation network until convergence using angiogram sequences.

[0048] In step 201, each patient's data includes at least one complete cardiac cycle, and the angiography shows clear visualization of the blood vessels.

[0049] In step 203, frame interpolation is performed on each data set (i.e., the original angiography image) to generate a larger number of different vascular structures and morphologies under different cardiac conditions. This scheme uses a U-shaped generative network to implement the frame interpolation operation.

[0050] The method of obtaining the second contrast image sequence by frame interpolation is similar to the method of obtaining the first contrast image sequence by frame interpolation described above, and will not be repeated here.

[0051] Both the simulated guidewire map data and the vessel segmentation map data were obtained through the second angiography sequence. The method for obtaining the simulated guidewire map data and the vessel segmentation map data will be described in detail below.

[0052] This application trains the initial generator network using simulated guidewire map data, vascular segmentation map data, and a second angiography sequence to obtain the target generator network. The specific training steps are described below.

[0053] As an optional embodiment, the initial generative network is trained using simulated guidewire image data, vessel segmentation image data, and a second angiography sequence to obtain the target generative network. This includes: extracting any first simulated guidewire image and first vessel segmentation image that have a corresponding relationship; using the first simulated guidewire image, first vessel segmentation image, and second angiography sequence as input, and training for a preset number of times to obtain the target generative network. The first vessel segmentation image is weighted according to the following formula starting from the first training: A=(1-α)((1-α)B+αN(0,1)), where A represents the network input after weighting the first vessel segmentation image, B represents the first vessel segmentation image, α represents the adjustment ratio, the adjustment ratio increases according to a preset step size, and N represents noise.

[0054] This application requires training the initial generated network a preset number of times using the same data. During each training session, the weights of the blood vessel segmentation map are adjusted according to a preset step size. This mainly involves adding noise and reducing signal amplitude in the blood vessel segmentation map. The adjustment ratios include the noise weighting ratio and the signal amplitude reduction ratio. The preset step size can be set according to the actual situation; in this application, it is set to 0.2.

[0055] The preset number of times depends on the preset step size and the range of the adjustment ratio. For example, if the adjustment ratio range is [0, 1] and the preset step size is 0.2, then the preset number of times is 5. The calculation formula is: (1-0) / 0.2=5.

[0056] For example, the second contrast image sequence is represented by {AI1,AI2,...,AI...} n The sequence indicates that it includes both the actual acquired angiographic images and the angiographic images obtained through frame interpolation. The i-th angiographic image in the sequence is represented as AI. i The corresponding simulated guidewire diagram data is The corresponding segment of the blood vessel is represented as follows: M represents AI i Number of generated simulated guidewire sequence maps.

[0057] The initial input to the generator network during training is The output is the predicted angiogram. Corresponding simulated guidewire diagram In one training session, the gold standard for the network was...

[0058] For example, the training process of the simulated guidewire generation network uses the vessel segmentation result image as auxiliary input to improve the continuity and accuracy of the predicted simulated guidewire. The specific training strategy is as follows (let B represent the vessel segmentation image):

[0059] A represents the network input to the blood vessel segmentation map B, where A = (1-α)((1-α)B+αN(0,1)), (0≤α≤1). Compared to B, A incorporates Gaussian noise (N(0,1)) and reduces the signal amplitude. This adjustment is controlled by α, which starts at 0 and increases in preset steps of 0.2 until it reaches 1. The training process is divided into multiple stages, each corresponding to a different α value. In each stage, the network uses the weights obtained from the previous stage as initial values ​​and continues training until convergence. The specific training steps are as follows:

[0060] (1) Initial training: α = 0, train the network until convergence. In this stage, A is actually B, and no noise or signal amplitude is added during training until convergence.

[0061] (2) In the second training, α is increased to 0.2, and the network is trained until convergence using the weights corresponding to the training result when α = 0 as the initial values. In this stage, A is 80% of B (i.e., (1-0.2)) plus 20% Gaussian noise. The generator network uses the weights obtained from training when α = 0 as the initial values ​​and continues to train until convergence.

[0062] (3) In the third training, α is increased to 0.4, and the network is trained until convergence using the training results weights corresponding to α = 0.2 as the initial values. In this stage, A is 60% of B (1-0.4) plus 40% Gaussian noise. Similarly, the network uses the weights obtained from the previous stage (α = 0.2) as the initial values ​​and continues to train.

[0063] (4) Repeat the execution steps as in (2) and (3) until α = 1, A is composed entirely of Gaussian noise (because (1-1)B+1*N(0,1)=N(0,1)), and does not contain any information about the blood vessel segmentation results.

[0064] Since A = 0 when α = 1, the network has already undergone multiple training phases and should possess robustness to noise, enabling prediction even without vessel segmentation results. The network parameters at this point are used for the prediction task. Since vessel segmentation results are not required during training, this approach is suitable for the online prediction scenario described in this application.

[0065] This application enables the network to gradually adapt to noisier and more uncertain input environments by progressively introducing noise and reducing signal amplitude. This strategy helps improve the network's robustness to noise, making it more stable and reliable in practical applications.

[0066] This application uses a target generation network to predict simulated guidewires, employing a U-shaped network architecture. The vessel segmentation result image is used as an auxiliary input, participating only in training and not in prediction. Compared to angiographic images, it has a stronger correlation with the predicted simulated guidewire, effectively improving the generation accuracy and reducing breakage.

[0067] As an optional embodiment, generating simulated guidewire map data and vessel segmentation map data based on the second angiography map sequence includes: extracting any initial angiography map from the second angiography map sequence; segmenting the vessels and the first centerline in the initial angiography map; randomly truncating a second centerline of a preset length from the first centerline and using the second centerline as the initial simulated guidewire; generating a second simulated guidewire map corresponding to the initial simulated guidewire; generating a second vessel segmentation map corresponding to the second simulated guidewire map; performing an affine transformation operation on the second simulated guidewire map to obtain a third simulated guidewire map corresponding to the initial angiography map; determining the third simulated guidewire map corresponding to each initial angiography map in the second angiography map sequence as simulated guidewire map data; and determining the second vessel segmentation map corresponding to each initial angiography map in the second angiography map sequence as vessel segmentation map data.

[0068] The blood vessels and the first central line in the initial angiographic image can be segmented using annotation tools.

[0069] A second centerline of a preset length is randomly selected from the first centerline. The preset length can be random, and it can be less than or equal to the length of the first centerline.

[0070] Based on the extracted second centerline, calculate its bounding box in the original blood vessel segmentation image, and crop the original blood vessel segmentation image according to the bounding box to obtain the second blood vessel segmentation image corresponding to the second simulated guidewire image.

[0071] Affine transformations include operations such as translation, rotation, scaling, and shearing. In practical applications, the appropriate transformation type can be selected according to the requirements.

[0072] When the number of simulated guidewire maps corresponding to the i-th angiogram is M (M is a random number within a certain range), then repeat the above steps M times to complete the generation of the simulated guidewire map and vessel segmentation map corresponding to the i-th angiogram. Then perform the same operation on other initial angiograms except for any of the above initial angiograms to obtain simulated guidewire map data and vessel segmentation map data.

[0073] This application uses centerline simulation to generate guidewires. The number of simulated guidewire images generated for any angiographic image and the length of the simulated guidewire during the generation process are all randomized. The purpose is to make the generated simulated guidewire images and vascular images conform to the characteristics of anatomical structures while being more diverse. The application of affine transformation can also increase the diversity of guidewire structures, so as to eliminate the influence of respiratory artifacts and improve the matching accuracy of cardiac cycles.

[0074] After obtaining the target generation network, the online phase can directly use the network to generate predicted simulated guidewire and lead wire segmentation maps. Because training occurs in the offline phase, which only contains vascular information, the centerline is extracted based on the vascular segmentation results to construct the simulated guidewire. Based on the prior knowledge of the morphological and structural similarity between the original angiography image and the perspective image in the online phase, the simulated guidewire generation strategy during training fully considers randomness, and its accuracy is almost unaffected after network training.

[0075] Because the blood vessels in the online phase are not clearly visualized in the fluoroscopy image, the model's input for prediction is the guidewire lead segmentation image (denoted as GI) in the current real-time fluoroscopy image and the angiography sequence {AI1,AI2,...,AI} in the offline phase. n The output is the predicted target imaging map (denoted as AI). p ) and the corresponding predicted target simulated guidewire diagram (denoted as SGI) p ).

[0076] As an optional embodiment, the target angiography image is registered using the target simulated guidewire image and the lead wire segmentation image to obtain a registered image, including: registering the target simulated guidewire image to the lead wire segmentation image to obtain an affine deformation field; and applying the affine deformation field to the target angiography image to obtain a registered image.

[0077] First, ensure that both the target simulation guidewire map and the lead wire segmentation map are in digital image format, and that their resolutions and pixel sizes are as similar as possible. If the image sizes do not match, preprocessing may be necessary to ensure they are aligned.

[0078] An affine registration network can be used to register the target simulated guidewire pattern to the lead wire segmentation pattern. An affine registration network is a deep learning model that can learn the affine transformation parameters between the target simulated guidewire pattern and the lead wire segmentation pattern. These parameters can describe how to align the target simulated guidewire pattern to the lead wire segmentation pattern through operations such as translation, rotation, scaling and / or shearing.

[0079] The affine registration network outputs a predicted affine transformation matrix, which defines the affine deformation field from the target simulated guidewire pattern to the lead segmentation pattern. Once the affine registration network outputs the affine transformation matrix, the affine deformation field can be generated from this matrix. The affine deformation field is a vector field describing how each pixel moves, used to align the target simulated guidewire pattern with the lead segmentation pattern.

[0080] Applying an affine deformation field to the target angiogram to obtain a registered image includes: applying an affine transformation formula to each pixel of the target angiogram, calculating their position in the new coordinate system, and using an interpolation method to obtain the corresponding pixel value, thereby obtaining a registered image corresponding to the target angiogram.

[0081] Affine registration networks can achieve high-precision image alignment by learning the affine transformation parameters between images. This alignment method considers not only image translation and rotation, but also geometric transformations such as scaling and shearing, thus ensuring accurate matching between images.

[0082] As an optional embodiment, fusing the registered image to the lead segmentation map to obtain a dynamic coronary artery path map of the target object includes: using the registered image as a background; converting the leads of the lead segmentation map to the target color; and fusing the leads of the target color with the registered image to obtain a dynamic coronary artery path map.

[0083] The preferred target color for this application is semi-transparent red.

[0084] For example, the registration image and the lead segmentation map are loaded using an image processing library (e.g., OpenCV, PIL / Pillow, Scikit-image, etc.); a mask is created to convert the leads in the lead segmentation map to semi-transparent red, which is semi-transparent red in the lead area and fully transparent in other areas; the mask image containing the semi-transparent red leads is fused with the registration image to obtain a dynamic coronary artery path map.

[0085] This application discloses a method for generating dynamic coronary artery path maps, comprising: acquiring a first angiographic image sequence of a target object and acquiring a lead wire segmentation map corresponding to a real-time perspective view of the target object, wherein the first angiographic image sequence is generated based on the original angiographic images of the target object in an offline phase; using the lead wire segmentation map and the first angiographic image sequence as input, outputting a target angiographic image and a target simulated guidewire map through a target generation network, wherein the target generation network is trained on vascular segmentation map data and simulated guidewire map data generated in the offline phase; registering the target angiographic image using the target simulated guidewire map and the lead wire segmentation map to obtain a registered image; and fusing the registered image into the lead wire segmentation map to obtain a dynamic coronary artery path map of the target object. Since the target generation network of this application is trained on vascular segmentation map data and simulated guidewire map data from the offline phase, the target simulated guidewire map obtained by specifically using the target generation network has high accuracy, thus not affecting the accuracy of the subsequently generated coronary artery path map, solving the problem of low accuracy of the generated coronary artery path map due to the lack of diversity or easy breakage of simulated guidewires.

[0086] According to another aspect of the embodiments of this application, this application provides an apparatus for generating dynamic coronary artery pathway maps, such as... Figure 3 As shown, it includes:

[0087] The acquisition module 302 is used to acquire the first contrast image sequence of the target object and acquire the lead segmentation map corresponding to the real-time perspective view of the target object, wherein the first contrast image sequence is generated based on the original contrast image of the target object in the offline stage;

[0088] The generation module 304 is used to take the lead wire segmentation map and the first angiography map sequence as input, and output the target angiography map and the target simulated guidewire map through the target generation network. The target generation network is obtained by training on the vascular segmentation map data and simulated guidewire map data generated in the offline stage.

[0089] The registration module 306 is used to register the target angiography image with the target simulated guide wire image and the lead wire segmentation image to obtain a registered image;

[0090] The fusion module 308 is used to fuse the registered image into the lead segmentation map to obtain a dynamic coronary artery path map of the target object.

[0091] It should be noted that the acquisition module 302 in this embodiment can be used to execute step 101 in this application embodiment, the generation module 304 in this embodiment can be used to execute step 103 in this application embodiment, the registration module 306 in this embodiment can be used to execute step 105 in this application embodiment, and the fusion module 308 in this embodiment can be used to execute step 107 in this application embodiment.

[0092] Optionally, the acquisition module 302 is further configured to acquire the original contrast image of the target object in the offline stage, wherein the original contrast image includes multiple image frames; generate intermediate frames of multiple image frames through frame interpolation operation; and insert the intermediate frames between the image frames of the original contrast image in chronological order to obtain the first contrast image sequence.

[0093] Optionally, the device further includes a training module for obtaining the target generation network in the following manner before outputting the target angiogram and the target simulated guidewire image through the target generation network: constructing an initial generation network; acquiring a second angiogram sequence and generating simulated guidewire image data and vessel segmentation image data based on the second angiogram sequence, wherein the second angiogram sequence is obtained through frame interpolation, and the simulated guidewire image data and vessel segmentation image data have a corresponding relationship; training the initial generation network using the simulated guidewire image data, vessel segmentation image data, and the second angiogram sequence to obtain the target generation network.

[0094] Optionally, the training module is also used to extract any corresponding first simulated guidewire image and first blood vessel segmentation image; the first simulated guidewire image, first blood vessel segmentation image and second angiography image sequence are used as inputs and trained for a preset number of times to obtain the target generation network. The first blood vessel segmentation image is weighted according to the following formula from the first training: A=(1-α)((1-α)B+αN(0,1)), where A represents the network input after weight adjustment of the first blood vessel segmentation image, B represents the first blood vessel segmentation image, α represents the adjustment ratio, the adjustment ratio increases according to a preset step size, and N represents noise.

[0095] Optionally, the training module is further configured to extract any initial angiogram from the second angiogram sequence; segment the blood vessels and the first centerline in the initial angiogram; randomly extract a second centerline of a preset length from the first centerline and use the second centerline as the initial simulated guidewire; generate a second simulated guidewire image corresponding to the initial simulated guidewire; generate a second blood vessel segmentation image corresponding to the second simulated guidewire image; perform an affine transformation operation on the second simulated guidewire image to obtain a third simulated guidewire image corresponding to the initial angiogram; determine the third simulated guidewire image corresponding to each initial angiogram in the second angiogram sequence as simulated guidewire image data; and determine the second blood vessel segmentation image corresponding to each initial angiogram in the second angiogram sequence as blood vessel segmentation image data.

[0096] Optionally, the registration module 306 is also used to register the target simulated guide wire image to the lead wire segmentation image to obtain an affine deformation field; and to apply the affine deformation field to the target contrast image to obtain a registered image.

[0097] Optionally, the fusion module 308 is also used to use the registered image as a background; convert the leads of the lead segmentation map to the target color; and fuse the leads of the target color with the registered image to obtain a dynamic coronary artery path map.

[0098] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments.

[0099] According to another aspect of the embodiments of this application, this application provides an electronic device, such as... Figure 4 As shown, the system includes a memory 401, a processor 403, a communication interface 405, and a communication bus 407. The memory 401 stores a computer program that can run on the processor 403. The memory 401 and the processor 403 communicate through the communication interface 405 and the communication bus 407. When the processor 403 executes the computer program, it implements the steps of the above method.

[0100] The memory and processor in the aforementioned electronic devices communicate with each other via a communication bus and a communication interface. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc.

[0101] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0102] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0103] According to another aspect of the embodiments of this application, a computer-readable medium having processor-executable non-volatile program code is also provided.

[0104] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0105] In specific implementation, the embodiments of this application can be referred to the above embodiments and have corresponding technical effects.

[0106] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0107] For software implementation, the techniques described herein can be implemented by units that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0108] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0109] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0110] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0111] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0112] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0113] If the aforementioned function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks. It should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0114] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for generating a dynamic coronary artery pathway map, characterized in that, include: Obtain a first angiographic image sequence of the target object, and obtain a lead segmentation map corresponding to the real-time perspective view of the target object, wherein the first angiographic image sequence is generated based on the original angiographic image of the target object in the offline stage; The lead wire segmentation map and the first angiography map sequence are used as inputs, and the target angiography map and the target simulated guidewire map are output through the target generation network. The target generation network is obtained by training on the vascular segmentation map data and simulated guidewire map data generated in the offline stage. The target angiography image is registered using the target simulated guidewire image and the lead wire segmentation image to obtain a registered image; The registered image is fused to the lead segmentation map to obtain a dynamic coronary artery path map of the target object; Before outputting the target angiogram and the target simulated guidewire map through the target generation network, the method further includes obtaining the target generation network in the following manner: constructing an initial generation network; obtaining a second angiogram sequence, and generating the simulated guidewire map data and the vessel segmentation map data based on the second angiogram sequence, wherein the second angiogram sequence is obtained through frame interpolation, and the simulated guidewire map data and the vessel segmentation map data have a corresponding relationship; training the initial generation network using the simulated guidewire map data, the vessel segmentation map data, and the second angiogram sequence to obtain the target generation network; The step of generating the simulated guidewire map data and the vessel segmentation map data based on the second angiography map sequence includes: extracting any initial angiography map from the second angiography map sequence; segmenting the vessels and the first centerline in the initial angiography map; randomly extracting a second centerline of a preset length from the first centerline and using the second centerline as the initial simulated guidewire, and generating a second simulated guidewire map corresponding to the initial simulated guidewire; generating a second vessel segmentation map corresponding to the second simulated guidewire map; performing an affine transformation operation on the second simulated guidewire map to obtain a third simulated guidewire map corresponding to the initial angiography map; determining the third simulated guidewire map corresponding to each of the initial angiography maps in the second angiography map sequence as the simulated guidewire map data, and determining the second vessel segmentation map corresponding to each of the initial angiography maps in the second angiography map sequence as the vessel segmentation map data.

2. The method according to claim 1, characterized in that, The acquisition of the first contrast image sequence of the target object includes: In the offline phase, the original contrast-enhanced image of the target object is acquired, wherein the original contrast-enhanced image comprises multiple image frames; Intermediate frames of multiple image frames are generated through frame interpolation. The intermediate frames are inserted between the image frames of the original contrast image in chronological order to obtain the first contrast image sequence.

3. The method according to claim 1, characterized in that, The process of training the initial generation network using the simulated guidewire map data, the vessel segmentation map data, and the second angiography sequence to obtain the target generation network includes: Extract any first simulated guidewire image and first blood vessel segmentation image that have the aforementioned correspondence; The first simulated guidewire image, the first vessel segmentation image, and the second angiography image sequence are used as inputs and trained a preset number of times to obtain the target generation network. The first vessel segmentation image is weighted according to the following formula starting from the first training iteration: A=(1-α)((1-α)B+αN(0,1)), Where A represents the network input after weight adjustment of the first blood vessel segmentation image, B represents the first blood vessel segmentation image, α represents the adjustment ratio, which increases according to a preset step size, and N represents noise.

4. The method according to claim 1, characterized in that, The step of registering the target angiography image with the target simulated guidewire image and the lead wire segmentation image to obtain a registered image includes: The target simulated guidewire pattern is registered to the lead wire segmentation pattern to obtain the affine deformation field; The affine deformation field is applied to the target imaging image to obtain a registered image.

5. The method according to claim 1, characterized in that, The step of fusing the registered image into the lead segmentation map to obtain a dynamic coronary artery path map of the target object includes: Use the registered image as the background; Convert the leads in the lead segmentation diagram to the target color; The target color leader is fused with the registered image to obtain the dynamic coronary artery path map.

6. A device for generating dynamic coronary artery pathway maps, characterized in that, include: The acquisition module is used to acquire a first angiography sequence of the target object and acquire a lead segmentation map corresponding to the real-time perspective view of the target object, wherein the first angiography sequence is generated based on the original angiography image of the target object in the offline stage; The generation module is used to take the lead wire segmentation map and the first angiography map sequence as input, and output the target angiography map and the target simulated guidewire map through the target generation network. The target generation network is obtained by training on the vascular segmentation map data and simulated guidewire map data generated in the offline stage. The registration module is used to register the target angiography image with the target simulated guidewire image and the lead wire segmentation image to obtain a registered image; The fusion module is used to fuse the registered image into the lead segmentation map to obtain a dynamic coronary artery path map of the target object; The training module is used to obtain the target generation network in the following manner before outputting the target angiogram and the target simulated guidewire image through the target generation network: constructing an initial generation network; obtaining a second angiogram sequence, and generating the simulated guidewire image data and the vessel segmentation image data based on the second angiogram sequence, wherein the second angiogram sequence is obtained through frame interpolation, and the simulated guidewire image data and the vessel segmentation image data have a corresponding relationship; training the initial generation network using the simulated guidewire image data, the vessel segmentation image data, and the second angiogram sequence to obtain the target generation network; The training module is further configured to extract any initial angiogram from the second angiogram sequence; segment the blood vessels and the first centerline in the initial angiogram; randomly extract a second centerline of a preset length from the first centerline, and use the second centerline as the initial simulated guidewire, and generate a second simulated guidewire image corresponding to the initial simulated guidewire; generate a second blood vessel segmentation image corresponding to the second simulated guidewire image; perform an affine transformation operation on the second simulated guidewire image to obtain a third simulated guidewire image corresponding to the initial angiogram; determine the third simulated guidewire image corresponding to each of the initial angiograms in the second angiogram sequence as the simulated guidewire image data, and determine the second blood vessel segmentation image corresponding to each of the initial angiograms in the second angiogram sequence as the blood vessel segmentation image data.

7. An electronic device comprising a memory, a processor, a communication interface, and a communication bus, wherein the memory stores a computer program executable on the processor, and the memory and the processor communicate via the communication bus and the communication interface, characterized in that... When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable medium having processor-executable non-volatile program code, characterized in that, The program code causes the processor to execute the method of any one of claims 1 to 5.

Images