Angiogram image segmentation method and device, electronic equipment and readable storage medium

Abstract

The application provides an angiography image segmentation method and device, electronic equipment and readable storage medium. The method comprises the following steps: acquiring an angiography image, and inputting the angiography image into a pre-trained neural network model; after each layer of the neural network model performs convolution operation and activation operation on the input of the layer, the output of the layer is obtained through a blood vessel edge information enhancement operation of an edge and connected domain enhancement module; and the neural network model outputs a blood vessel contour segmentation result of the angiography image. By using the proposed edge and connected domain enhancement module and adding it to the neural network model layer by layer, the blood vessel edge information of different feature scales is fused, the segmentation result can better retain the edge information of the blood vessel contour and the connectivity between the contour regions, which is conducive to realizing an accurate and complete blood vessel contour segmentation result and helps to improve the accuracy and time efficiency of evaluating the blood vessel stenosis degree.

Description

Technical Field

[0001] This invention relates to the fields of neural networks and image segmentation technology, and in particular to a method, apparatus, electronic device, and readable storage medium for segmenting angiography images. Background Technology

[0002] In X-ray angiography, accurate and complete automatic segmentation of vascular contours is beneficial for objectively and accurately quantifying vascular morphological information, and is one of the prerequisites for computer-aided precise diagnosis of vascular diseases and the formulation of treatment strategies. However, due to the inherent low signal-to-noise ratio of X-ray angiography and the presence of complex background structures in the images (such as overlapping images of catheters, the patient's diaphragm, spine, and ribs), and the highly uneven and unbalanced distribution and number of edge and non-edge pixels in vascular image features (the number of non-edge pixels is far greater than the number of edge pixels), obtaining accurate and complete vascular contours, especially at the edges and in stenotic regions, in angiography is very challenging.

[0003] Angiography segmentation methods can be broadly categorized into two types: traditional image segmentation methods and deep learning-based segmentation methods. Traditional image segmentation methods include model-driven methods and filtering methods. Model-driven methods have been proposed using active contour models to segment cerebral vessels from computed tomography angiography images. In filtering methods, researchers have proposed the Frangi vessel enhancement filter based on the Hessian matrix and tensor-based filters, aiming to alleviate or remove uneven brightness variations in the image and suppress background structure and image noise. However, these traditional segmentation methods typically require careful design of their applicable conditions and heavily rely on domain knowledge. When faced with different image conditions (such as varying image brightness, contrast, background structure, and noise), their carefully designed parameters often fail to yield optimal results, producing erroneous or even incorrect vessel contour segmentation results, which is detrimental to the reliable diagnosis of vascular diseases.

[0004] Deep learning-based segmentation methods have demonstrated superior segmentation performance. One scholar proposed a U-shaped deep convolutional neural network (also known as UNet) as an image segmentation method. In benchmark tests comparing it with traditional segmentation methods, the UNet-based segmentation method achieves high-accuracy segmentation results and can adapt to different image conditions. Building upon the UNet framework, another scholar proposed a triple-attention UNet combination (also known as 3AUNet) incorporating spatial attention, channel attention, and contextual attention.

[0005] In angiography segmentation, the aforementioned deep learning-based segmentation methods can achieve relatively accurate segmentation of the main body of the blood vessel contour, but they struggle to achieve precise segmentation of the vessel contour edges. Furthermore, when segmenting narrow sections of the vessel, they easily break previously connected vascular regions, resulting in fragmented vessel contours and making it difficult to obtain a complete vessel contour. This is because the edge and non-edge pixels of vascular structures are usually unevenly distributed, with most pixels being non-edge pixels. This causes deep learning methods to tend to learn information from non-edge pixels during training, making it difficult to learn sufficient edge pixel information. Therefore, when automatically segmenting an angiography image, the connectivity of the segmented vessel contour edges and narrow sections is usually poor, making it difficult to obtain accurate and complete vessel contour segmentation results. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a method, apparatus, electronic device and readable storage medium for segmenting angiographic images, so as to better preserve the edge information of the vascular contour and the connectivity between contour regions, which is conducive to achieving accurate and complete vascular contour segmentation results and helps to improve the accuracy and time efficiency of assessing the degree of vascular stenosis.

[0007] In a first aspect, embodiments of the present invention provide a method for segmenting angiographic images, the method comprising: acquiring an angiographic image; inputting the angiographic image into a pre-trained neural network model; performing convolution and activation operations on the input of each layer of the neural network model, and then performing vascular edge information enhancement operations through an edge and connected component enhancement module to obtain the output of the layer; and outputting the vascular contour segmentation result of the angiographic image from the neural network model.

[0008] In optional embodiments of this application, each layer of the above-described neural network model performs a convolution operation through two convolutions; each layer of the neural network model performs an activation operation through a modified linear unit activation function.

[0009] In an optional embodiment of this application, after the step of obtaining the output of the layer by performing blood vessel edge information enhancement operation through the edge and connected component enhancement module, the method further includes: if the layer belongs to the downsampling path of the neural network model, performing max pooling operation on the output of the layer to obtain the input of the next layer; if the layer belongs to the upsampling path of the neural network model, performing deconvolution operation on the output of the layer, and concatenating the result of the deconvolution operation with the feature map of the corresponding downsampling path to obtain the input of the next layer.

[0010] In an optional embodiment of this application, the step of obtaining the output of the layer by performing vascular edge information enhancement operation through the edge and connected component enhancement module includes: obtaining the feature map obtained by performing convolution and activation operations on the input of each layer of the neural network model; performing activation operation, subtraction operation with the natural number 1, deconvolution operation and max pooling operation on the feature map in sequence to obtain the result of max pooling operation; multiplying the result of max pooling operation and the feature map to obtain the edge feature map; and summing the edge feature map and the feature map to obtain the output of the layer.

[0011] In an optional embodiment of this application, the feature map is activated using a hyperbolic tangent activation function.

[0012] In optional embodiments of this application, the above method further includes: acquiring angiography training images, labeling the angiography training images with vascular contour tags; and training a neural network model based on the angiography training images and the corresponding vascular contour tags.

[0013] In an optional embodiment of this application, the step of obtaining angiography training images includes: obtaining angiography images for training; performing image preprocessing and image sample enhancement processing on the angiography images for training in sequence to obtain angiography training images.

[0014] Secondly, embodiments of the present invention also provide a vascular angiography image segmentation device, the device comprising: a vascular angiography image acquisition module, used to acquire vascular angiography images and input the vascular angiography images into a pre-trained neural network model; a vascular edge information enhancement module, used for each layer of the neural network model to perform convolution and activation operations on the input of the layer, and then perform vascular edge information enhancement operations through the edge and connected component enhancement module to obtain the output of the layer; and a vascular contour segmentation result output module, used for the neural network model to output the vascular contour segmentation result of the vascular angiography image.

[0015] Thirdly, embodiments of the present invention also provide an electronic device, including a processor and a memory, the memory storing computer-executable instructions that can be executed by the processor, the processor executing the computer-executable instructions to implement the above-described angiography image segmentation method.

[0016] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions, which, when called and executed by a processor, cause the processor to implement the above-described angiography image segmentation method.

[0017] The embodiments of the present invention bring the following beneficial effects:

[0018] This invention provides a method, apparatus, electronic device, and readable storage medium for segmenting angiographic images. The method involves acquiring angiographic images and inputting them into a pre-trained neural network model. Each layer of the neural network model performs convolution and activation operations on its input, followed by an edge and connected component enhancement module to enhance vascular edge information, resulting in the output of that layer. The neural network model outputs the vascular contour segmentation result of the angiographic image. By using the proposed edge and connected component enhancement module and incorporating it layer by layer into the neural network model, vascular edge information at different feature scales is fused. This allows the segmentation result to better preserve the edge information of the vascular contour and the connectivity between contour regions, facilitating accurate and complete vascular contour segmentation and improving the accuracy and time efficiency of assessing the degree of vascular stenosis.

[0019] Other features and advantages of this disclosure will be set forth in the following description, or some features and advantages may be inferred from the description or determined without doubt, or may be learned by practicing the techniques described above.

[0020] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0021] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 A flowchart of a method for segmenting angiography images provided in an embodiment of the present invention;

[0023] Figure 2 A flowchart of another angiography image segmentation method provided in an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of a neural network model of an edge and connected component enhancement UNet architecture provided in an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of an edge and connected component enhancement module provided in an embodiment of the present invention;

[0026] Figure 5 A schematic diagram of a neural network training stage provided in an embodiment of the present invention;

[0027] Figure 6 A schematic diagram of a neural network prediction stage provided in an embodiment of the present invention;

[0028] Figure 7 This is a schematic diagram of the structure of a blood vessel angiography image segmentation device provided in an embodiment of the present invention;

[0029] Figure 8 This is a schematic diagram of another angiography image segmentation device provided in an embodiment of the present invention;

[0030] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Currently, in angiography segmentation tasks, the aforementioned deep learning-based segmentation methods can achieve relatively accurate segmentation of the main body of the blood vessel contour, but they struggle to achieve precise segmentation of the vessel contour edges. Furthermore, when segmenting narrow sections of the vessel, they easily cause breaks in originally connected vascular regions, resulting in fragmented vessel contours and making it difficult to obtain complete vessel contours. This is because the edge and non-edge pixels of vascular structures are usually unevenly distributed, with most pixels being non-edge pixels. This leads deep learning methods to tend to learn information from non-edge pixels during training, making it difficult to learn sufficient edge pixel information. Therefore, when automatically segmenting an angiography image, the connectivity of the segmented vessel contour edges and narrow sections is usually poor, making it difficult to obtain accurate and complete vessel contour segmentation results.

[0033] Based on this, embodiments of the present invention provide a method, apparatus, electronic device, and readable storage medium for segmenting angiographic images. Specifically, a novel edge and connected component enhancement method based on the UNet framework is proposed to enhance the edge features of the vascular contour and the connectivity between vascular contour regions. By using the proposed edge and connected component enhancement modules and incorporating them layer by layer into the existing UNet framework, vascular edge information at different feature scales is fused. This allows the segmentation results to better preserve the edge information of the vascular contour and the connectivity between contour regions, which is beneficial for achieving accurate and complete vascular contour segmentation results and helps improve the accuracy and time efficiency of assessing the degree of vascular stenosis.

[0034] To facilitate understanding of this embodiment, a detailed description of an angiography image segmentation method disclosed in this embodiment of the invention will be provided first.

[0035] Example 1:

[0036] This invention provides a method for segmenting angiographic images, see [link to relevant documentation]. Figure 1 The flowchart shown illustrates a method for segmenting angiographic images, which includes the following steps:

[0037] Step S102: Obtain angiography images and input the angiography images into a pre-trained neural network model.

[0038] Existing angiography segmentation methods struggle to obtain accurate vessel contour edges and complete contour connectivity after segmentation, which is detrimental to the measurement and calculation of vascular morphological information (such as lumen diameter and vessel length). Therefore, this embodiment proposes an edge and connected component enhancement UNet method based on the UNet framework for automatic segmentation of vessel contours in angiography images.

[0039] In step S104, after each layer of the neural network model performs convolution and activation operations on the input of that layer, the output of that layer is obtained by performing blood vessel edge information enhancement operations through the edge and connected component enhancement module.

[0040] In terms of neural network architecture, the UNet architecture includes a shrinking path (also known as the downsampling path) consisting of convolution, ReLU (Rectified Linear Unit) activation, and max pooling operations, and a symmetrical expanding path (also known as the upsampling path) where deconvolution replaces max pooling operations. Skip connections are made at the outputs of modules with the same resolution in the upsampling and downsampling paths to serve as the inputs to the next upsampling module, thus realizing a symmetrical U-shaped structure with cross-layer connections.

[0041] In this embodiment, based on the aforementioned UNet architecture, an edge and connected component enhancement module is added after the double convolution and activation operations in each layer during the upsampling and downsampling stages. Therefore, this embodiment can obtain the output of a layer by performing vascular edge information enhancement operations through the edge and connected component enhancement module after performing convolution and activation operations on the input of each layer in the neural network model.

[0042] The edge and connected component enhancement modules maintain gradient updates and backpropagation in the UNet network while enabling cross-layer connections using their outputs. This achieves the effect of enhancing vessel edge features and maximizing connectivity between vessel contour regions while preserving UNet's segmentation performance. Furthermore, by integrating the edge and connected component enhancement modules into each layer of the upsampling and downsampling stages, the proposed edge and connected component enhancement UNet method can adapt to angiographic images with varying scales, brightness, and contrast.

[0043] Step S106: The neural network model outputs the vascular contour segmentation result of the angiography image.

[0044] Regarding the edge and connected component enhancement modules, the neural network model with the UNet structure proposed in this embodiment, which includes edge and connected component enhancement modules, can better learn the edge information of the blood vessel contour and ultimately output the blood vessel contour segmentation result of the angiography image.

[0045] In the edge and connected component enhancement module, this embodiment applies the Tanh activation function to the input feature map, then subtracts it from the natural number 1. Subsequently, deconvolution and max pooling operations are used to enhance the vascular edge feature information (max pooling is beneficial for capturing the most active neuron information within the pooling window). The vascular edge feature map is then obtained by intersecting the foreground and background (i.e., multiplying it with the original feature map to obtain the boundary intersection), thereby enhancing the edge information of the vascular contour and the connectivity between contours.

[0046] This invention provides a method for segmenting angiographic images. The method involves acquiring an angiographic image and inputting it into a pre-trained neural network model. Each layer of the neural network model performs convolution and activation operations on its input, followed by an edge and connected component enhancement module to enhance vascular edge information, resulting in the output of that layer. The neural network model outputs the vascular contour segmentation result from the angiographic image. By using the proposed edge and connected component enhancement module and incorporating it layer by layer into the neural network model, vascular edge information at different feature scales is fused. This allows the segmentation result to better preserve the edge information of the vascular contour and the connectivity between contour regions, leading to accurate and complete vascular contour segmentation results and improving the accuracy and time efficiency of assessing the degree of vascular stenosis.

[0047] Example 2:

[0048] This embodiment provides another method for segmenting angiography images, which is implemented based on the above embodiment. See [link to previous embodiment]. Figure 2The flowchart of another angiography image segmentation method is shown. The angiography image segmentation method in this embodiment includes the following steps:

[0049] Step S202: Obtain angiography images and input the angiography images into a pre-trained neural network model.

[0050] In step S204, after each layer of the neural network model performs two convolution operations on the input of the layer and a modified linear unit activation function, the output of the layer is obtained by performing a blood vessel edge information enhancement operation through the edge and connected component enhancement module.

[0051] In some embodiments, if the layer belongs to the downsampling path of the neural network model, the output of the layer is subjected to max pooling to obtain the input of the next layer; if the layer belongs to the upsampling path of the neural network model, the output of the layer is subjected to deconvolution, and the result of the deconvolution is concatenated with the feature map of the corresponding downsampling path to obtain the input of the next layer.

[0052] See Figure 3 The diagram shows a neural network model with edge and connected component enhancement UNet architecture. Figure 3 The UNet architecture shown includes a downsampling shrinkage path ( Figure 3 (Left side) and a symmetrical upsampling expansion path ( Figure 3 (Right side), where the downsampling path follows the typical architecture of a convolutional neural network, consisting of repeated applications of two convolutions (with 3x3 kernels) (i.e., each layer of the neural network model performs a convolution operation through two convolutions), followed by an activation unit (ReLU activation function, i.e., each layer of the neural network model performs an activation operation through the ReLU activation function) and a max pooling operation (pooling window of 2, stride of 2).

[0053] In the upsampling path, the input is the output of the previous upsampling layer, which is then deconvolved (with a 2x2 kernel). This deconvolution is then concatenated and fused with the output of the corresponding resolution from the symmetrical downsampling path. This is followed by two convolutions (with a 3x3 kernel) and repeated application of the ReLU activation function. In the final layer of the upsampling path, a 1x1 convolution is used to map each feature vector to the desired classification.

[0054] like Figure 3As shown, based on the aforementioned UNet architecture, this embodiment proposes adding an edge and connected component enhancement module after each layer's double convolution and activation operations. This method combines the cross-layer connectivity features of UNet, enabling the learning of enhanced vessel edge information at different feature map resolutions. This helps improve the learning of vessel edge information during gradient updates and backpropagation training, while preserving as much connectivity as possible in the vessel contour. This facilitates accurate and complete segmentation of the target vessel contour in angiographic images.

[0055] Figure 3 The specific processing steps of the neural network model shown are as follows:

[0056] Step 1: Input X-ray angiography images for deep learning training ( Figure 3 (The black solid line at number 1 points to the right with an arrow).

[0057] Step 2: Use two convolutions (with a 3x3 kernel) and ReLU activation operation ( Figure 3 (White right arrow at number 1), and input the result into the edge and connectivity enhancement module ( Figure 3 (White swallowtail-shaped right arrow at number 1).

[0058] Step 3: Use max pooling (pooling window size is 2, stride size is 2) Figure 3 (White downward arrow between numbers 1 and 2).

[0059] Step 4: Repeat steps 2 and 3 three times. Figure 3 (Serial numbers 2, 3, and 4).

[0060] Step 5: Repeat step 2 ( Figure 3 Serial number 5).

[0061] Step 6: Perform deconvolution (with a 2x2 kernel) Figure 3 The white upward arrow between numbers 5 and 6) connects the feature maps corresponding to the downsampling layer to achieve cross-layer edge information transfer. Figure 3 (The black dotted line between numbers 4 and 6 points to the right with an arrow), and repeat step 2 ( Figure 3 Serial number 6).

[0062] Step 7: Repeat step 6 three times. Figure 3 (Serial numbers 7, 8, and 9).

[0063] Step 8: Use convolution operation (convolution kernel is 1x1) Figure 3 (The white dotted arrow at number 9 points points to the right).

[0064] Step 9: Output the blood vessel contour segmentation results ( Figure 3 (The black solid line at number 9 points to the right with an arrow).

[0065] Figure 3 The edge and connected component enhancement UNet architecture shown in the figure retains the U-shaped structure of UNet and adds an edge and connected component enhancement module after each layer, which enhances the blood vessel contour edge information of each layer's feature map. This allows the neural network model to learn blood vessel edge information from different feature map scales, thereby obtaining a more accurate and complete blood vessel contour after segmentation.

[0066] also, Figure 3 The edge and connected component enhancement UNet architecture shown is based on cross-layer connections during downsampling and upsampling. The output of the newly added edge and connected component enhancement modules in each layer can achieve fusion enhancement of blood vessel contour edges and connected components at different depths. By fusing high-resolution feature maps with low-resolution feature maps for edge enhancement, edge information can be avoided during training iterations, improving the segmentation ability of blood vessel contour edges. Furthermore, the neural network model can adapt to different image conditions (such as angiography images with different brightness and contrast), making the blood vessel contour segmentation results more reliable.

[0067] For the specific processing steps of the edge and connected component enhancement module, in some embodiments, the feature map obtained by performing convolution and activation operations on the input of each layer of the neural network model can be obtained; the feature map can be sequentially subjected to activation operation, subtraction operation with the natural number 1, deconvolution operation and max pooling operation to obtain the result of max pooling operation; the result of max pooling operation and feature map can be multiplied to obtain edge feature map; the edge feature map and feature map can be summed to obtain the output of the layer.

[0068] See Figure 4 The diagram illustrates an edge and connected component enhancement module. The specific processing steps of the edge and connected component enhancement module may include:

[0069] Step 1: Input the feature map X obtained after double convolution and ReLU activation operations. Figure 4 Serial number 1).

[0070] Step 2: Apply an activation function and subtract it from the natural number 1. Figure 4 Serial number 2).

[0071] Step 3: Use deconvolution operation, with a 2×2 kernel. Figure 4 Serial number 3).

[0072] Step 4: Use max pooling with a pooling window of 2×2 and a stride of 2. Figure 4 Serial number 4).

[0073] Step 5: Multiply the result of max pooling with the original feature map X. Figure 4 Serial number 5).

[0074] Step 6: Obtain the intersecting edge feature map Y( Figure 4 Serial number 6).

[0075] Step 7: Summate the original feature map X with the edge feature map Y, and use this sum as the module output. Figure 4 Serial number 7).

[0076] In some embodiments, the feature map can be activated using the Tanh (Hyperbolic Tangent) activation function.

[0077] Figure 4 The Tanh activation function, number 2, is a commonly used activation function, and its calculation formula is shown in formula (1):

[0078]

[0079] Where x is the input value and e is the natural constant. The output value of the Tanh activation function ranges from (-1, 1) and is 0 at the origin (x = 0). The graph of this function is centrally symmetric at the origin.

[0080] Compared to the Sigmoid activation function with an output value range of (0,1), the Tanh activation function has an output value range of (-1,1) and is symmetrical about the origin. This symmetrical output value range can alleviate the offset and oscillation that occur during gradient descent due to the asymmetric output value range (such as the Sigmoid activation function), and helps to improve the stability and convergence speed of the edge and connected component enhancement modules during feature extraction.

[0081] Figure 4 The deconvolution operation (number 3) is used to upsample the feature map, resulting in a higher-resolution feature map. Deconvolution is the inverse operation of convolution. Let the input feature map be... The output feature map is The deconvolution kernel is H and W represent the height and width of the input feature map. and K represents the height and width of the output feature map. h and K w The height and width of the deconvolution kernel are represented by F, the number of deconvolution kernels is represented by F, and C is the number of channels in the input feature map. The calculation formula for the deconvolution operation is shown in formula (2):

[0082] Y(i,j,k)=∑ m,n,l X(m,n,l)W(im,jn,l,k) (2)

[0083] Where i and j represent the pixel coordinates of the output feature map, m and n represent the pixel coordinates of the input feature map, and l and k represent the indices of the input and output channels of the deconvolution operation, respectively.

[0084] Figure 4 In the middle, number 4 is the max pooling operation, a commonly used downsampling operation, usually used to reduce the spatial resolution of image feature maps. Its calculation formula is shown in formula (3):

[0085]

[0086] Where X represents the input feature map, Y represents the output feature map after pooling, i and j represent the pixel coordinates of the output feature map, k represents the number of channels, p and q represent the height and width of the pooling kernel, s represents the pooling stride, and max indicates the maximum value operation. Within each region, the max pooling operation selects the maximum value within that region as the pooling result for that region. In this way, max pooling can reduce the spatial resolution of the feature map while preserving important feature information.

[0087] The edge and connected component enhancement module provided in this embodiment of the invention uses the Tanh activation function, which improves the problem of offset and oscillation during convergence caused by using the traditional Sigmoid activation function, and at the same time improves the stability and convergence of feature extraction.

[0088] The edge and connected component enhancement module provided in this embodiment of the invention employs two methods to preserve and enhance the edge information and connectivity of the blood vessel contour as much as possible. The first method uses the idea of ​​intersection to extract contour edge information. Specifically, in step 2, an edge feature map of the blood vessel background is obtained by subtracting from the natural number 1. Then, in step 5, this is multiplied with the original feature map to obtain the edge feature map in step 6, effectively preserving edge features and connectivity. The second method uses max pooling and summation operations to enhance edge information. Specifically, after step 2 in the first method, deconvolution and max pooling operations are added, so that the maximum edge value within the pooling window represents the local edge, restoring and enhancing weak or even lost connected component information. Finally, in step 7, the edge feature map is summed with the original feature map as the module output, enhancing the edge information in the original feature map. These two points enable the edge and connected component enhancement module to effectively preserve and enhance the edge features and connectivity of the blood vessel contour.

[0089] Step S206: The neural network model outputs the vascular contour segmentation result of the angiography image.

[0090] The segmentation process for vascular contours in X-ray angiography proposed in this embodiment includes two stages: a training stage and a prediction stage. The prediction stage comprises steps S102-S106 and S202-S206 in the aforementioned embodiment.

[0091] During the training phase, in some embodiments, angiography training images can be acquired, and vascular contour labels can be annotated on the angiography training images; a neural network model can be trained based on the angiography training images and the corresponding vascular contour labels.

[0092] This process involves acquiring angiographic images for training; then performing image preprocessing and image sample enhancement on these training angiographic images to obtain training angiographic images. (See also...) Figure 5 The diagram shown illustrates a neural network training phase, including the following steps:

[0093] Step 1: Input X-ray angiography images for deep learning training ( Figure 5 Sequence number 1), which is the angiography image used for training.

[0094] Step 2: Preprocess the contrast image (e.g., grayscale normalization, image resizing, etc.) Figure 5 Serial number 2).

[0095] Step 3: Perform data augmentation on image samples (such as rotation, blurring, and brightness adjustment). Figure 5 Serial number 3).

[0096] Step 4: Obtain training images for deep learning ( Figure 5 Serial number 4), namely, angiography training images.

[0097] Step 5: Add the blood vessel pixel label map corresponding to the training image ( Figure 5 Serial number 5).

[0098] Step 6: Train the UNet for edge and connected component enhancement using the training images and corresponding image labels. Figure 5 Serial number 6).

[0099] Step 7: Obtain the trained edge and connected component augmentation UNet ( Figure 5 Serial number 7).

[0100] For the prediction phase, please refer to Figure 6 The diagram shown illustrates a neural network prediction stage, which includes the following steps:

[0101] Step 1: Input the X-ray angiography image of the blood vessel contour to be segmented ( Figure 6 Serial number 1).

[0102] Step 2: Input the angiography image to be segmented into the trained edge and connected component enhancement UNet for segmentation prediction. Figure 6 Serial number 2).

[0103] Step 3: Output the blood vessel contour segmentation results ( Figure 6 Serial number 3).

[0104] The method provided in this invention specifically proposes a novel UNet-based edge and connected component enhancement segmentation method to enhance the edge features of the blood vessel contour and the connectivity between blood vessel contour regions. By using the newly proposed edge and connected component enhancement modules and incorporating them layer by layer into the existing UNet framework, blood vessel edge information at different feature scales is fused. This allows the segmentation results to better preserve the edge information of the blood vessel contour and the connectivity between contour regions, which is beneficial for achieving accurate and complete blood vessel contour segmentation results and helps improve the accuracy and time efficiency of assessing the degree of blood vessel stenosis.

[0105] Example 3:

[0106] Corresponding to the above method embodiments, this invention provides a vascular angiography image segmentation device, see [link to relevant documentation]. Figure 7 The diagram shown illustrates the structure of a vascular angiography image segmentation device, which includes:

[0107] The angiography image acquisition module 71 is used to acquire angiography images and input the angiography images into a pre-trained neural network model.

[0108] The blood vessel edge information enhancement module 72 is used to perform blood vessel edge information enhancement operations on the blood vessel edge information enhancement operation through the edge and connected component enhancement module after each layer of the neural network model performs convolution and activation operations on the input of the layer to obtain the output of the layer.

[0109] The blood vessel contour segmentation result output module 73 is used by the neural network model to output the blood vessel contour segmentation result of the angiography image.

[0110] This invention provides a device for segmenting angiographic images. The device acquires angiographic images and inputs them into a pre-trained neural network model. Each layer of the neural network model performs convolution and activation operations on its input, and then enhances the vascular edge information through an edge and connected component enhancement module to obtain the output of that layer. The neural network model outputs the vascular contour segmentation result of the angiographic image. By using the proposed edge and connected component enhancement module and incorporating it layer by layer into the neural network model, vascular edge information at different feature scales is fused. This allows the segmentation result to better preserve the edge information of the vascular contour and the connectivity between contour regions, which is beneficial for achieving accurate and complete vascular contour segmentation results and improving the accuracy and time efficiency of assessing the degree of vascular stenosis.

[0111] Each layer of the above neural network model performs a convolution operation through two convolutions; each layer of the above neural network model performs an activation operation through a modified linear unit activation function.

[0112] See Figure 8 The diagram shows another angiography image segmentation device. This angiography image segmentation device also includes a neural network model processing module 74, a blood vessel edge information enhancement module 72, a neural network model processing module 74, and a blood vessel contour segmentation result output module 73 connected in sequence. If the layer belongs to the downsampling path of the neural network model, the neural network model processing module 74 performs a max pooling operation on the output of the layer to obtain the input of the next layer.

[0113] If the layer belongs to the upsampling path of the neural network model, the neural network model processing module 74 performs a deconvolution operation on the output of the layer, and connects the result of the deconvolution operation with the feature map of the corresponding downsampling path to obtain the input of the next layer.

[0114] The aforementioned blood vessel edge information enhancement module 72 is used to obtain the feature map obtained by performing convolution and activation operations on the input of each layer of the neural network model; to perform activation operation, subtraction operation with the natural number 1, deconvolution operation and max pooling operation on the feature map in sequence to obtain the result of max pooling operation; to perform multiplication operation on the result of max pooling operation and feature map to obtain edge feature map; and to perform summation operation on edge feature map and feature map to obtain the output of the layer.

[0115] The aforementioned blood vessel edge information enhancement module is used to activate the feature map using the hyperbolic tangent activation function.

[0116] like Figure 8As shown, the angiography image segmentation device further includes: a neural network model training module 75, which is connected to the angiography image acquisition module 71; the neural network model training module 75 is used to acquire angiography training images, label the angiography training images with vascular contour labels; and train a neural network model based on the angiography training images and the corresponding vascular contour labels.

[0117] The aforementioned neural network model training module is used to acquire angiographic images for training; the angiographic images used for training are then subjected to image preprocessing and image sample enhancement processing to obtain angiographic training images.

[0118] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the angiography image processing device described above can be referred to the corresponding process in the embodiments of the aforementioned angiography image processing method, and will not be repeated here.

[0119] Example 4:

[0120] This invention also provides an electronic device for running the above-described angiography image processing method; see [link to previous document]. Figure 9 The diagram shows the structure of an electronic device, which includes a memory 100 and a processor 101. The memory 100 is used to store one or more computer instructions, which are executed by the processor 101 to implement the above-mentioned angiography image processing method.

[0121] Furthermore, Figure 9 The electronic device shown also includes a bus 102 and a communication interface 103, with the processor 101, the communication interface 103 and the memory 100 connected via the bus 102.

[0122] The memory 100 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 103 (which can be wired or wireless). The communication connection can use the Internet, wide area network, local area network, metropolitan area network, etc. The bus 102 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0123] Processor 101 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 101 or by instructions in software form. Processor 101 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a readily available storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 100, and processor 101 reads information from memory 100 and, in conjunction with its hardware, completes the steps of the method described in the foregoing embodiments.

[0124] This invention also provides a computer-readable storage medium storing computer-executable instructions. When these computer-executable instructions are called and executed by a processor, they cause the processor to implement the above-described angiography image processing method. For specific implementation details, please refer to the method embodiments, which will not be repeated here.

[0125] The computer program products of the angiography image processing method, apparatus, electronic device and readable storage medium provided in the embodiments of the present invention include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods in the preceding method embodiments. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0126] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and / or device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0127] Furthermore, in the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances.

[0128] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0129] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0130] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method of segmenting an angiogram image, characterized by, The method comprises: acquiring an angiography image and inputting the angiography image into a pre-trained neural network model; each layer of the neural network model performs a convolution operation and an activation operation on the input of the layer, and then performs a blood vessel edge information enhancement operation through an edge and connected domain enhancement module to obtain the output of the layer; wherein the edge and connected domain enhancement module is arranged after each double convolution and activation operation in each layer of the up-sampling and down-sampling stages of the neural network model; in the edge and connected domain enhancement module, after using a Tanh activation function on the input feature map, a subtraction operation from a natural number 1 is performed on the input feature map, then a deconvolution and a maximum pooling operation are used to enhance the blood vessel edge feature information, and then a foreground and background intersection method is used to multiply the result of the maximum pooling operation and the input feature map to obtain an edge feature map; the edge feature map and the input feature map are summed to obtain the output of the layer; the neural network model outputs a blood vessel contour segmentation result of the angiography image.

2. The method of claim 1, wherein, each layer of the neural network model performs an activation operation through a rectified linear unit activation function.

3. The method of claim 1, wherein, After the step of obtaining the output of the layer through the edge and connected domain enhancement module, the method further comprises: if the layer belongs to the down-sampling path of the neural network model, performing a maximum pooling operation on the output of the layer to obtain the input of the next layer; if the layer belongs to the up-sampling path of the neural network model, performing a deconvolution operation on the output of the layer, connecting the result of the deconvolution operation with the feature map of the corresponding down-sampling path of the layer to obtain the input of the next layer.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: acquiring an angiography training image and labeling a blood vessel contour label for the angiography training image; training a neural network model based on the angiography training image and the corresponding blood vessel contour label.

5. The method of claim 4, wherein, The step of acquiring an angiography training image comprises: acquiring an angiography image for training; performing image preprocessing and image sample enhancement processing on the angiography image for training in sequence to obtain an angiography training image.

6. A blood vessel image segmentation apparatus characterized by comprising: The device comprises: an angiography image acquisition module configured to acquire an angiography image and input the angiography image into a pre-trained neural network model; The blood vessel edge information enhancement module is configured to, after each layer of the neural network model performs convolution operation and activation operation on the input of the layer, perform blood vessel edge information enhancement operation on the input through the edge and connected domain enhancement module to obtain the output of the layer; wherein the edge and connected domain enhancement module is arranged after each layer of the neural network model performs double convolution and activation operation in the up-sampling and down-sampling stages; in the edge and connected domain enhancement module, after the input feature map is subjected to a Tanh activation function, the input feature map is subjected to a subtraction operation with a natural number 1, then the blood vessel edge feature information is enhanced by using deconvolution and maximum pooling operation, and then the result of the maximum pooling operation and the input feature map are multiplied in a foreground and background intersection manner to obtain an edge feature map; the edge feature map and the input feature map are summed to obtain the output of the layer; The blood vessel contour segmentation result output module is configured to output the blood vessel contour segmentation result of the angiogram image by the neural network model.

7. An electronic device, comprising: The computer readable storage medium stores computer executable instructions, and the computer executable instructions, when called and executed by the processor, cause the processor to implement the angiogram image segmentation method in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer executable instructions, and the computer executable instructions, when called and executed by the processor, cause the processor to implement the angiogram image segmentation method in any one of claims 1 to 5.

Images