Angiogram image processing and segmentation method and apparatus

Abstract

The application provides a kind of angiogram image processing and segmentation method, device, comprising: obtaining angiogram, input angiogram in a plurality of filters of pre-parameter optimization;Based on the multiple super parameter values corresponding to each filter, the angiogram is filtered to obtain a plurality of filtered images, and the multiple filtered images of each filter are compounded to obtain the compound filter image of each filter;Multiple compound filter images are input into the neural network joint segmentation model trained in advance, and the compound vessel segmentation result of angiogram is output. It is beneficial to obtain the best possible vessel feature enhancement effect in the image preprocessing stage, which helps the deep neural network to learn the best possible vessel feature information;It is beneficial to obtain high confidence vessel image pixel point classification when segmenting vessels, reducing the misclassification of vessel pixels;Thus, it is helpful to obtain accurate and reliable angiography segmentation results.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of neural networks and image segmentation, and particularly relates to a method and device for processing and segmenting angiography images. BACKGROUND

[0002] In X-ray angiography, fully automatic vessel contour segmentation is a key component of computer-aided diagnosis of vascular diseases and treatment strategy. It can be used to provide objective quantitative morphological information of blood vessels and serve as a calculation of clinical diagnostic indicators (such as providing information of vessel diameter or cross-sectional area reduction percentage), which helps to systematically evaluate the severity of vascular stenosis and eliminate the subjective differences caused by different doctors using visual inspection method to evaluate the degree of vascular stenosis. However, due to the inherent low signal-to-noise ratio of X-ray angiography images and the mixed background structures (such as overlapping structures of catheters, patient spines and ribs) in the images, the features of the blood vessels in the angiography images are not easy to be distinguished from the features of the body bones or interventional devices, and even overlap with each other, which is not conducive to accurate segmentation of the vessel contour and is easy to cause differences between the automatic segmentation results and the manual segmentation results, sometimes even large differences or segmentation errors, which is not conducive to the automatic and accurate diagnosis of vascular diseases. Therefore, accurate and reliable automatic angiography segmentation has become a key and challenging task.

[0003] At present, there are several methods for automatic vessel contour segmentation of X-ray angiography images, which can be summarized as traditional segmentation methods based on image processing and deep learning segmentation methods based on convolutional neural networks. In the traditional segmentation method, it is representative to use methods such as thresholding methods, active contour models, region-growing methods and filter-based methods to segment the vessel contour in X-ray angiography. However, the segmentation performance of the above traditional segmentation methods is easily affected by the quality of the angiography images, such as uneven brightness, low contrast, containing noise and structural overlap, etc. Therefore, the segmentation robustness is poor when facing different angiography image qualities, and it is difficult to obtain accurate and reliable vessel segmentation results.

[0004] The segmentation method based on deep learning technology learns the X-ray angiography image and the corresponding blood vessel pixel label, updates the weight value of the neural network in the training process, constructs the fitting relationship between the angiography image and the blood vessel pixel label, so that it can predict the image pixel point representing the blood vessel in the given angiography according to the previously learned fitting relationship, and realize the automatic segmentation of the blood vessel contour. In the segmentation method based on deep learning technology, the angiography segmentation method based on UNet framework is representative. Compared with the traditional segmentation method which is difficult to obtain accurate and reliable segmentation results when facing different angiography image quality, the segmentation method based on deep learning can make the deep neural network learn rich blood vessel feature information, so as to have better segmentation robustness when facing different angiography image quality, and is beneficial to realize accurate and reliable blood vessel segmentation results.

[0005] Although there are several methods to improve the angiography segmentation result based on deep learning technology at present, most of the existing methods do not consider the most suitable enhancement of the blood vessel features in the image preprocessing stage, and most of the existing methods only train and use a single deep neural network model to segment the blood vessels of the angiography image. Therefore, the best blood vessel feature enhancement effect cannot be obtained in the image preprocessing stage, so that the deep neural network cannot learn the best blood vessel feature information in the training process, and it is difficult to achieve the best blood vessel segmentation result. At the same time, using a single deep neural network model for blood vessel segmentation is not enough to obtain high confidence blood vessel image pixel point classification, which is easy to cause error classification of blood vessel pixel points, that is, the pixel points that should be blood vessels are wrongly classified as non-blood vessel pixel points, and the pixel points that should be non-blood vessels are wrongly classified as blood vessel pixel points, which is also difficult to obtain the best blood vessel segmentation result. SUMMARY

[0006] Therefore, the purpose of the present application is to provide an angiography image processing and segmentation method and device, which is beneficial to obtain the best blood vessel feature enhancement effect in the image preprocessing stage, helps the deep neural network to learn the best blood vessel feature information, is beneficial to obtain high confidence blood vessel image pixel point classification in blood vessel segmentation, reduces the error classification of blood vessel pixel points, and thus helps to obtain accurate and reliable angiography segmentation results.

[0007] In a first aspect, an embodiment of the present application provides a method for processing and segmenting an angiogram image, the method comprising: obtaining the angiogram image; inputting the angiogram image into a plurality of filters with pre-selected parameters; wherein each filter corresponds to a plurality of hyperparameter values; performing filtering processing on the angiogram image based on the plurality of hyperparameter values corresponding to each filter to obtain a plurality of filtered images; performing complex processing on the plurality of filtered images of each filter to obtain a complex filtered image of each filter; inputting the plurality of complex filtered images into a pre-trained neural network joint segmentation model to output a complex vessel segmentation result of the angiogram image.

[0008] In an optional embodiment of the present application, the step of performing complex processing on the plurality of filtered images of each filter to obtain a complex filtered image of each filter comprises: performing image normalization processing on each filtered image to obtain a plurality of normalized images; and performing complex processing on the plurality of normalized images to obtain a complex filtered image of each filter.

[0009] In an optional embodiment of the present application, the step of performing complex processing on the plurality of normalized images to obtain a complex filtered image of each filter comprises: performing complex processing on the plurality of normalized images to obtain a complex filtered image of each filter by the following formula: wherein I i,j is a pixel value of the i-th row and j-th column of the complex filtered image, N is the number of hyperparameter values, is a pixel value of the i-th row and j-th column of the normalized image corresponding to the k-th hyperparameter value, and max is a maximum value operation.

[0010] In an optional embodiment of the present application, the neural network joint segmentation model comprises a plurality of deep neural network models; the step of inputting the plurality of complex filtered images into the pre-trained neural network joint segmentation model to output the complex vessel segmentation result of the angiogram image comprises: inputting the plurality of complex filtered images into the plurality of deep neural network models respectively, each deep neural network model determining a vessel segmentation result of the input complex filtered image; and the neural network joint segmentation model performing complex processing on the plurality of vessel segmentation results to obtain the complex vessel segmentation result of the angiogram image.

[0011] In an optional embodiment of the present application, the step of performing complex processing on the plurality of vessel segmentation results to obtain the complex vessel segmentation result of the angiogram image by the neural network joint segmentation model comprises: performing complex processing on the plurality of vessel segmentation results to obtain the complex vessel segmentation result of the angiogram image by the following formula: wherein C i,j is a pixel classification of the i-th row and j-th column in the complex vessel segmentation result, the pixel classification comprises a vessel classification and a non-vessel classification, P is the vessel classification, and Q is the non-vessel classification. is the number of times the pixel in the i-th row and j-th column is classified as a blood vessel in the N blood vessel segmentation results, and f is a preset number threshold.

[0012] In an optional embodiment of the present application, the number threshold is determined by the following formula: f=ceil((N+1) / 2); wherein f is the number threshold, ceil is the rounding-up operation, and N is the number of blood vessel segmentation results.

[0013] In an optional embodiment of the present application, the above method also includes: obtaining an angiography training image, inputting the angiography training image into multiple filters for parameter selection; each filter outputs a filtered training image, and normalizing each filtered training image to obtain multiple normalized training images; training a deep neural network model based on the multiple normalized training images; inputting the angiography training image into the trained deep neural network model, and outputting a blood vessel segmentation result; adjusting the hyperparameter values ​​corresponding to the filter based on the blood vessel segmentation result to obtain multiple hyperparameter values ​​corresponding to the filter.

[0014] In an optional embodiment of the present application, the above method also includes: obtaining an angiography training image, inputting the angiography training image into multiple filters with optimized parameters, each filter outputting a composite filtered training image of the angiography training image; and training the neural network joint segmentation model based on the multiple composite filtered training images as inputs of the neural network joint segmentation model to be trained.

[0015] In an optional embodiment of the present application, the filter includes at least one of the following: a Frangi filter, a Gabor filter, a Sato filter, and an Unsharp Mask filter.

[0016] In a second aspect, an embodiment of the present invention further provides an angiography image processing and segmentation device, the device comprising: an angiography image acquisition module, for acquiring an angiography image, and inputting the angiography image into a plurality of filters with pre-selected parameters; wherein each filter corresponds to a plurality of hyperparameter values; a filter processing module, for filtering the angiography image based on the plurality of hyperparameter values ​​corresponding to each filter to obtain a plurality of filtered images, and composite processing the plurality of filtered images of each filter to obtain a composite filtered image of each filter; and a neural network model segmentation module, for inputting the plurality of composite filtered images into a pre-trained neural network joint segmentation model, and outputting a composite blood vessel segmentation result of the angiography image.

[0017] The embodiments of the present invention bring the following beneficial effects:

[0018] The embodiment of the present application provides a kind of angiogram image processing and segmentation method, device, obtains angiogram image, and input angiogram image in multiple filters of pre-parameter optimization;Wherein, each filter corresponds multiple super parameter values;Multiple filtered images are obtained by filtering processing angiogram image based on multiple super parameter values corresponding to each filter, and multiple filtered images of each filter are compounded to obtain the compound filter image of each filter;Multiple compound filter images are input into pre-trained neural network joint segmentation model, and the compound blood vessel segmentation result of angiogram image is output.It is beneficial to obtain the best possible blood vessel feature enhancement effect in the image preprocessing stage, which helps the deep neural network to learn the best possible blood vessel feature information;It is beneficial to obtain high confidence blood vessel image pixel point classification when blood vessel segmentation, to reduce the error classification of blood vessel pixel points;Thus, it is helpful to obtain accurate and reliable angiography segmentation result.

[0019] Other features and advantages of the present disclosure will be described in the following description, or some features and advantages can be inferred from the description or determined without doubt, or can be known by implementing the above-mentioned technologies of the present disclosure.

[0020] In order to make the above-mentioned purposes, features and advantages of the present disclosure more obvious and easy to understand, the following preferred embodiments are described in detail below, and the accompanying drawings are described as follows. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0022] Figure 1 The flow chart of a kind of angiogram image processing and segmentation method provided by the embodiment of the present application is shown in the figure;

[0023] Figure 2 The flow chart of another angiogram image processing and segmentation method provided by the embodiment of the present application is shown in the figure;

[0024] Figure 3 The flow chart of a kind of method based on deep learning provided by the embodiment of the present application is used to determine the flow of filter super parameter value for realizing best segmentation performance;

[0025] Figure 4 The flow chart of a kind of angiogram image processing and segmentation method provided by the embodiment of the present application is shown in the figure;

[0026] Figure 5 A schematic diagram of a process for training multiple deep neural network models based on different filters and respective composite filtered images is provided for embodiments of the present application;

[0027] Figure 6 A schematic diagram of a blood vessel segmentation process based on multiple trained deep neural network models for segmentation prediction is provided for embodiments of the present application;

[0028] Figure 7 A structural schematic diagram of a blood vessel angiography image processing and segmentation device is provided for embodiments of the present application;

[0029] Figure 8 A structural schematic diagram of another blood vessel angiography image processing and segmentation device is provided for embodiments of the present application;

[0030] Figure 9 A structural schematic diagram of an electronic device is provided for embodiments of the present application. DETAILED DESCRIPTION

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions of the present application will be described below in conjunction with the accompanying drawings, obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0032] At present, although there are several methods to improve the angiography segmentation result based on deep learning technology, most of the existing methods do not consider the most suitable enhancement of the blood vessel features in the image preprocessing stage, and most of the existing methods only train and use a single deep neural network model for blood vessel segmentation of angiography images. Therefore, it is not possible to obtain the best possible blood vessel feature enhancement effect in the image preprocessing stage, so that the deep neural network is not enough to learn the best possible blood vessel feature information in the training process, and it is difficult to achieve the best blood vessel segmentation result. At the same time, using a single deep neural network model for blood vessel segmentation is not enough to obtain high confidence blood vessel image pixel point classification, which is easy to cause error classification of blood vessel pixel points, i.e. the pixel points that should be blood vessels are incorrectly classified as non-blood vessel pixel points, and the pixel points that should be non-blood vessels are incorrectly classified as blood vessel pixel points, which is also difficult to obtain the best blood vessel segmentation result.

[0033] Based on this, the embodiment of the present application provides a blood vessel angiography image processing and segmentation method and device, and specifically proposes a method for enhancing blood vessel features in an image preprocessing stage and a method for blood vessel segmentation by using a neural network joint segmentation model. The proposed blood vessel feature enhancement method is beneficial to obtaining the best possible blood vessel feature enhancement effect in the image preprocessing stage and helps the deep neural network to learn the best possible blood vessel feature information. The proposed segmentation method based on the neural network joint segmentation model is beneficial to obtaining high-confidence blood vessel image pixel point classification during blood vessel segmentation and reducing the false classification of blood vessel pixel points. The two methods proposed above are helpful to obtain accurate and reliable blood vessel angiography segmentation results.

[0034] To facilitate the understanding of the present embodiment, first, a blood vessel angiography image processing and segmentation method disclosed by the present embodiment is described in detail.

[0035] Embodiment one:

[0036] The present embodiment provides a blood vessel angiography image processing and segmentation method, referring to the flowchart of a blood vessel angiography image processing and segmentation method shown in Figure 1 The blood vessel angiography image processing and segmentation method includes the following steps:

[0037] Step S102, acquiring a blood vessel angiography image, and inputting the blood vessel angiography image into a plurality of filters with pre-optimized parameters; wherein each filter corresponds to a plurality of super parameter values.

[0038] In the present embodiment, an X-ray blood vessel angiography image can be acquired, and the blood vessel angiography image to be processed is input into a plurality of filters with pre-optimized parameters. Each filter can correspond to a plurality of super parameter values, that is, each filter can have different super parameter value modes, and each different super parameter value mode can output different composite filter images.

[0039] The process of parameter optimization can be: adjusting the super parameter values of the filter, recording the segmentation performance after each adjustment, and selecting a number of optimal super parameter values.

[0040] Step S104, filtering the blood vessel angiography image based on the plurality of super parameter values corresponding to each filter to obtain a plurality of filtered images, and performing composite processing on the plurality of filtered images of each filter to obtain a composite filter image of each filter.

[0041] This embodiment provides a method for enhancing vascular features. Each filter can filter an input angiographic image. Each filter can filter the angiographic image using each hyperparameter value corresponding to the filter. Each hyperparameter value for each filter can produce a filtered image.

[0042] The composite filtered image of each filter can be obtained by compounding the multiple filtered images of each filter. This image compounding is conducive to obtaining the best possible blood vessel feature enhancement effect suitable for deep learning.

[0043] Step S106 , inputting the multiple composite filtered images into a pre-trained neural network joint segmentation model, and outputting a composite blood vessel segmentation result of the angiography image.

[0044] This embodiment provides a joint vessel segmentation method. Each filter in the aforementioned steps can output a composite filtered image. These composite filtered images are then input into a pre-trained neural network joint segmentation model, which then composites these composite filtered images and outputs a composite vessel segmentation result for the angiographic image. This composite vessel segmentation result effectively improves the confidence level of vessel pixels, enhances the correct classification of vessel pixels, and thereby enhances the accuracy and reliability of automatic vessel segmentation.

[0045] An embodiment of the present invention provides an angiographic image processing and segmentation method, which includes acquiring an angiographic image and inputting the angiographic image into multiple filters with pre-selected parameters; wherein each filter corresponds to multiple hyperparameter values; filtering the angiographic image based on the multiple hyperparameter values ​​corresponding to each filter to obtain multiple filtered images; composite processing of the multiple filtered images of each filter to obtain a composite filtered image of each filter; inputting the multiple composite filtered images into a pre-trained neural network joint segmentation model, and outputting a composite vascular segmentation result for the angiographic image. This method is advantageous for obtaining the best possible vascular feature enhancement effect during the image preprocessing stage, and for enabling the deep neural network to learn the best possible vascular feature information; and is advantageous for obtaining high-confidence classification of vascular image pixels during vascular segmentation, reducing misclassification of vascular pixels, thereby facilitating accurate and reliable angiographic segmentation results.

[0046] Example 2:

[0047] This embodiment provides another angiography image processing and segmentation method, which is implemented on the basis of the above embodiment. Figure 2 FIG. 1 is a flow chart of another angiography image processing and segmentation method shown in FIG. 1 . The angiography image processing and segmentation method in this embodiment includes the following steps:

[0048] In step S202, the angiography image is obtained, and the angiography image is input into a plurality of filters with pre-optimized parameters; each filter corresponds to a plurality of super parameter values; the filters include at least one of the following: a Frangi filter, a Gabor filter, a Sato filter, and an Unsharp Mask filter.

[0049] The blood vessel feature enhancement method provided in the embodiment is based on the blood vessel enhancement filter to enhance the blood vessel features in the angiography image in the image preprocessing stage. The image filter, such as the Frangi filter, the Gabor filter, the Sato filter, and the Unsharp Mask filter, can enhance the image features of the tubular object region (such as the blood vessel region) in the image and enhance the image contrast between the edges and the surrounding body tissues, and at the same time, suppress the image noise and the background structure. However, directly using the above filters cannot obtain the best blood vessel feature enhancement effect suitable for deep learning, because (1) the used filter and the super parameter values thereof are not optimized for the segmentation result based on deep learning, and (2) a set of optimized super parameter values can only reflect the blood vessel feature enhancement effect under the condition of the parameter values, and cannot cover different blood vessel feature enhancement effects under the condition of other optimized super parameter values.

[0050] In view of this, the embodiment uses (1) the method of deep learning to evaluate the blood vessel segmentation performance, to determine the super parameter values of the filter that can achieve the best segmentation performance, and uses (2) the images filtered by a plurality of optimal super parameter values of the same filter to be compounded, to obtain different blood vessel feature enhancement effects using a plurality of optimal filter super parameter values and to be compounded, so as to obtain the best blood vessel feature enhancement effect suitable for deep learning in the image preprocessing stage.

[0051] In step S204, the angiography image is filtered based on the plurality of super parameter values corresponding to each filter to obtain a plurality of filtered images, and the plurality of filtered images of each filter are compounded to obtain a compound filtered image of each filter.

[0052] In some embodiments, the parameters of the filter can be optimized by the following steps: obtaining an angiography training image, inputting the angiography training image into a plurality of filters to be parameter-optimized; each filter outputs a filtered training image, and each filtered training image is normalized to obtain a plurality of normalized training images; a deep neural network model is trained based on the plurality of normalized training images; the angiography training image is input into the trained deep neural network model, and a blood vessel segmentation result is output; the super parameter values corresponding to the filter are adjusted based on the blood vessel segmentation result, to obtain a plurality of super parameter values corresponding to the filter.

[0053] See also Figure 3 The schematic diagram of a process for determining the filter hyperparameter values ​​for achieving the best segmentation performance based on a deep learning method shown in FIG. 4 can be performed by the following steps:

[0054] Step 1: Acquire X-ray angiography training images.

[0055] Step 2: Apply filters (such as Frangi filters, Gabor filters, Sato filters, and Unsharp Mask filters) to the X-ray angiography training images to enhance the image features of the vascular regions and the image contrast between the vascular edges and surrounding tissues, while suppressing image noise and background structures. The initial hyperparameters of the filters are randomly selected.

[0056] Step 3: Obtain a filtered training image with enhanced vascular features.

[0057] Step 4: Normalize the grayscale value of the filtered training image to obtain a normalized training image.

[0058] Step 5: Use the normalized training images for deep neural network model training.

[0059] Step 6: Use the trained deep neural network model to perform vessel segmentation on the angiography training images, evaluate and record the vessel segmentation results (the evaluation indicators that can be used are accuracy, precision, recall rate, F1 score, etc.).

[0060] Step 7: Randomly change the hyperparameter values ​​of the filter to obtain a new filtered training image, repeat steps 2 to 7 until the number of cycles set by the user is completed (in this embodiment, it can be set to 20 times). The process ends and multiple hyperparameter values ​​corresponding to the filter are obtained.

[0061] like Figure 3 As shown, the corresponding deep learning vessel segmentation performance obtained by the randomized filter hyperparameter values ​​after each cycle can be recorded. All recorded segmentation performances after the cycle are ranked from high to low, and the filter hyperparameter values ​​corresponding to the best vessel segmentation performance are sequentially selected (in this embodiment, the filter hyperparameter values ​​corresponding to the first three groups of best vessel segmentation performance can be used). The selected optimal filter hyperparameter values ​​are used to filter the same angiography image during the image preprocessing stage to obtain different vessel feature-enhanced images. Since each selected hyperparameter value corresponds to a deep learning vessel segmentation performance ranked from high to low, performing image composite on the resulting vessel feature-enhanced images facilitates obtaining the best possible vessel feature enhancement effect suitable for deep learning.

[0062] In some embodiments, each filtered image may be subjected to image normalization processing to obtain a plurality of normalized images; and the plurality of normalized images may be composited to obtain a composite filtered image of each filter.

[0063] See also Figure 4 The diagram shows a process of performing filtered image composite based on multiple optimal hyperparameter values ​​of the same filter. Figure 4 The image composite is shown after filtering using multiple optimal hyperparameter values ​​of the same filter, in order to obtain the best possible vascular feature enhancement effect suitable for deep learning during the image preprocessing stage. This can be performed by the following steps:

[0064] Step 1: Acquire an X-ray angiography image to be preprocessed.

[0065] Step 2: Apply filters (such as Frangi filter, Gabor filter, Sato filter, and Unsharp Mask filter) to the X-ray angiography image.

[0066] Step 3: Use the N best hyperparameter values ​​obtained in advance (i.e. Figure 4 The best hyperparameter value (1), the best hyperparameter value (2) to the best hyperparameter value (N) are filtered.

[0067] Step 4: Obtain N filtered images (i.e. Figure 4 Filtered image (a), filtered image (b) to filtered image (N) in .

[0068] Step 5: Normalize the grayscale value of the filtered image to obtain multiple normalized images.

[0069] Step 6: Compound multiple normalized images to obtain a composite filtered image.

[0070] This embodiment uses Figure 4 The demonstrated process generates composite filtered images from a preprocessed X-ray angiography image during the image preprocessing phase for use in deep neural network training. Because optimal hyperparameter values ​​for the same filter correspond to optimal deep learning vessel segmentation performance, composite images enhanced by filtering with these optimal hyperparameter values ​​are beneficial for achieving the best possible vessel enhancement results suitable for deep learning.

[0071] In some embodiments, multiple normalized images may be composited to obtain a composite filtered image of each filter using the following formula: Among them, I i,j is the pixel value of the i-th row and j-th column of the composite filtered image, N is the number of hyperparameter values, is the pixel value of the i-th row j-th column of the normalized image corresponding to the k-th hyperparameter value, and max is the maximum value operation.

[0072] As shown in Figure 4 , is the pixel value of the i-th row j-th column of the filtered normalized image obtained by the optimal hyperparameter value (one), M i k ,j is the pixel value of the i-th row j-th column of the filtered normalized image obtained by the optimal hyperparameter value (two) to the optimal hyperparameter value (N).

[0073] For example, the pixel values of the i-th row j-th column of the filtered normalized images obtained by the optimal hyperparameter values (one), (two) and (three) are and Then, according to the above formula, I i,j = 0.7. When and , then according to the above formula, I i,j = 0.8.

[0074] Step S206, inputting the plurality of composite filtered images into the pre-trained neural network joint segmentation model to output a composite blood vessel segmentation result of the angiogram image.

[0075] The joint blood vessel segmentation method proposed in this embodiment is based on the segmentation of the angiogram image by the deep neural network model trained by a plurality of different blood vessel enhancement filters, and the composite of the segmentation result. The deep learning training based on a single filter and its composite filtered image can only learn the best possible blood vessel image features enhanced by the filter to obtain the blood vessel segmentation result. However, since different filters will produce different blood vessel feature enhancement effects, using deep learning models based on different filters for prediction will produce different blood vessel segmentation results. The same pixel position in different blood vessel segmentation results is often classified as a blood vessel pixel point with high confidence, which corresponds to the correct blood vessel pixel classification result, and conversely, the same pixel position is less likely to be classified as a blood vessel pixel point, which is a low confidence blood vessel image pixel point (or a high confidence non-blood vessel image pixel point).

[0076] By training a plurality of deep neural network models based on different filters to jointly classify and determine the blood vessel image pixel points, only when more than a set number of models predict that the image pixel point is a blood vessel pixel point, the pixel point is determined to be a blood vessel pixel point, otherwise it is a non-blood vessel pixel point. This method can effectively improve the confidence of the blood vessel pixel point, improve the correct classification of the blood vessel pixel point, and further improve the accuracy and reliability of automatic blood vessel segmentation.

[0077] In some embodiments, the deep neural network model can be trained by the following steps: obtaining an angiography training image, inputting the angiography training image into the plurality of filters after parameter optimization, each filter outputting a composite filtered training image of the angiography training image; training the deep neural network model based on the plurality of composite filtered training images as the input of the deep neural network model to be trained.

[0078] Referring to Figure 5 Fig. 1 shows a flowchart of training a plurality of deep neural network models based on different filters and respective composite filtered images. Figure 5 Fig. 1 shows a flowchart of training a plurality of deep neural network models based on different filters and respective composite filtered images.

[0079] Step 1: Obtain an X-ray angiography image for deep learning training.

[0080] Step 2: Use several different filters (such as Frangi filter, Gabor filter, Sato filter, and Unsharp Mask filter, etc.) on the X-ray angiography image.

[0081] Step 3: Based on the output of each filter after parameter optimization, output the corresponding composite filtered image (i.e. composite filtered image (one), composite filtered image (two), and composite filtered image (N) in Fig. 1). Figure 5 Step 4: Train the deep neural network model using the obtained composite filtered image as the input image (i.e. deep neural network training (one), deep neural network training (two), and deep neural network training (N) in Fig. 1), obtain the trained deep neural network model corresponding to the filter, which will be used for blood vessel segmentation of the angiography image in the prediction stage.

[0082] Figure 5 In some embodiments, the neural network joint segmentation model includes a plurality of deep neural network models; the plurality of composite filtered images can be input into the plurality of deep neural network models respectively, each deep neural network model determines the blood vessel segmentation result of the input composite filtered image; the neural network joint segmentation model performs composite processing on the plurality of blood vessel segmentation results to obtain the composite blood vessel segmentation result of the angiography image.

[0083] Referring to Fig. 2 shows a blood vessel segmentation flowchart based on the segmentation prediction of the plurality of trained deep neural network models,

[0084] Figure 6 Fig. 2 shows a blood vessel segmentation flowchart based on the segmentation prediction of the plurality of trained deep neural network models, Figure 6 ​A process of using multiple deep neural network models for joint vessel segmentation in the deep learning prediction stage is demonstrated. The process uses multiple deep neural network models to segment X-ray angiograms, and makes a joint determination based on the vessel segmentation results. Only when more than a set number of models predict that the image pixel point is a vessel pixel point, the pixel point is determined as a vessel pixel point, otherwise it is a non-vessel pixel point. This approach can effectively improve the confidence of vessel pixel points, improve the correct classification of vessel pixel points, and thus improve the accuracy and reliability of automatic vessel segmentation. The following steps can be performed:

[0085] Step 1: Obtain an X-ray angiogram image for vessel segmentation.

[0086] Step 2: Use the process of Figure 4 to make composite filter image (one), composite filter image (two), and composite filter image (N).

[0087] Step 3: Perform vessel contour segmentation using the obtained composite filter image as the input image of the trained deep neural network model (i.e., the trained deep neural network (one), trained deep neural network (two), and trained deep neural network (N) in Figure 6 .

[0088] Step 4: Each deep neural network predicts a corresponding vessel segmentation result (i.e., vessel segmentation result (one), vessel segmentation result (two), and vessel segmentation result (N) in Figure 6 ).

[0089] Step 5: To improve the confidence of vessel image pixel points, composite the obtained multiple vessel segmentation results to obtain a composite vessel segmentation result.

[0090] Only when more than a set number of models predict that the image pixel point is a vessel pixel point, the pixel point is determined as a vessel pixel point, otherwise it is a non-vessel pixel point, which is beneficial to improve the correct classification of vessel pixel points.

[0091] In some embodiments, the multiple vessel segmentation results can be processed to obtain a composite vessel segmentation result of the angiogram image by the following formula: where C i,j is the pixel classification of the i-th row and j-th column in the composite vessel segmentation result, the pixel classification includes vessel classification and non-vessel classification, P is the vessel classification, Q is the non-vessel classification, is the number of times that the i-th row and j-th column pixel is classified as a vessel classification in N vessel segmentation results, and f is a pre-set number threshold.

[0092] For example, three deep neural network models trained based on different filters are used for blood vessel segmentation of an angiogram image, at the pixel position of the i-th row and j-th column, the trained deep neural network (one) classifies the pixel point as a blood vessel P, the trained deep neural network (two) classifies the pixel point as a non-blood vessel Q, and the trained deep neural network (three) classifies the pixel point as a blood vessel P, then When f is set to 1 by the user, since the pixel point is determined to be a blood vessel pixel point. When f is set to 2 by the user, since the pixel point is determined to be a blood vessel pixel point. When f is set to 3 by the user, since the pixel point is determined to be a non-blood vessel pixel point.

[0093] In some embodiments, the number threshold value can be determined by the following formula: f = ceil((N+1) / 2); where f is the number threshold value, ceil is the rounding up operation, and N is the number of blood vessel segmentation results. N is an integer greater than 1.

[0094] The above method provided by the embodiments of the present application proposes a method of enhancing blood vessel features in an image preprocessing stage and a method of blood vessel segmentation by a neural network joint segmentation model. The proposed blood vessel feature enhancement method is beneficial to obtaining the best possible blood vessel feature enhancement effect in the image preprocessing stage, and helps the deep neural network to learn the best possible blood vessel feature information. The proposed joint blood vessel segmentation method is beneficial to obtaining high-confidence blood vessel image pixel point classification during blood vessel segmentation, and reducing the false classification of blood vessel pixel points. The two methods proposed above help to obtain accurate and reliable angiogram segmentation results.

[0095] Embodiment three:

[0096] Corresponding to the above method embodiments, the embodiments of the present application provide an angiogram image processing and segmentation device, referring to Figure 7 the structural schematic diagram of an angiogram image processing and segmentation device shown in the figure, the angiogram image processing and segmentation device comprises:

[0097] The angiogram image acquisition module 71 is used to acquire an angiogram image, and input the angiogram image into a plurality of filters with pre-optimized parameters; wherein each filter corresponds to a plurality of super parameter values;

[0098] The filter processing module 72 is used to filter the angiogram image based on the plurality of super parameter values corresponding to each filter to obtain a plurality of filtered images, and to composite process the plurality of filtered images of each filter to obtain a composite filtered image of each filter;

[0099] The neural network model segmentation module 73 is configured to input the plurality of composite filtered images into a pre-trained neural network joint segmentation model, and output a composite blood vessel segmentation result of the angiogram image.

[0100] The angiogram image processing and segmentation device provided in the embodiments of the present application comprises the following steps: an angiogram image is acquired, and the angiogram image is input into a plurality of filters with pre-optimized parameters; each filter corresponds to a plurality of super parameter values; the angiogram image is filtered based on the plurality of super parameter values corresponding to each filter to obtain a plurality of filtered images; the plurality of filtered images of each filter are subjected to composite processing to obtain a composite filtered image of each filter; the plurality of composite filtered images are input into a pre-trained neural network joint segmentation model, and a composite blood vessel segmentation result of the angiogram image is output. This is beneficial to obtaining the best possible blood vessel feature enhancement effect in the image preprocessing stage, and is helpful to enabling the deep neural network to learn the best possible blood vessel feature information; this is beneficial to obtaining a high-confidence blood vessel image pixel point classification in the blood vessel segmentation, and is helpful to reducing the false classification of blood vessel pixel points; and thus, an accurate and reliable angiogram segmentation result can be obtained.

[0101] The filter processing module is configured to perform image normalization processing on each filtered image to obtain a plurality of normalized images; and perform composite processing on the plurality of normalized images to obtain a composite filtered image of each filter.

[0102] The filter processing module is configured to perform composite processing on the plurality of normalized images to obtain a composite filtered image of each filter by using the following formula: wherein, I i,j is a pixel value of an i-th row and a j-th column of the composite filtered image, N is a number of super parameter values, is a pixel value of an i-th row and a j-th column of a normalized image corresponding to a k-th super parameter value, and max is a maximum value operation.

[0103] The neural network joint segmentation model comprises a plurality of deep neural network models; the neural network model segmentation module is configured to input the plurality of composite filtered images into the plurality of deep neural network models respectively, each deep neural network model determines a blood vessel segmentation result of the input composite filtered image; and the neural network joint segmentation model performs composite processing on the plurality of blood vessel segmentation results to obtain a composite blood vessel segmentation result of the angiogram image.

[0104] The neural network model segmentation module is configured to perform composite processing on the plurality of blood vessel segmentation results to obtain a composite blood vessel segmentation result of the angiogram image by using the following formula: wherein, C i,j is a pixel classification of an i-th row and a j-th column in the composite blood vessel segmentation result, the pixel classification comprises a blood vessel classification and a non-blood vessel classification, P is the blood vessel classification, and Q is the non-blood vessel classification. is the number of times that the pixel in the i-th row and j-th column in the N vascular segmentation results is classified as a blood vessel classification, and f is a preset number threshold.

[0105] The neural network model segmentation module is configured to determine the number threshold f according to the following formula: f = ceil((N+1) / 2); where f is the number threshold, ceil is a rounding-up operation, and N is the number of vascular segmentation results.

[0106] Referring to Figure 8 Another angiogram image processing and segmentation device is shown in the structural schematic diagram. The angiogram image processing and segmentation device further comprises a filter parameter optimization module 74 connected to the filter processing module 72; the filter parameter optimization module 74 is configured to obtain an angiogram training image, input the angiogram training image into a plurality of filters to be optimized in parameter, output a filtered training image by each filter, and obtain a plurality of normalized training images by performing normalization processing on each filtered training image; train a deep neural network model based on the plurality of normalized training images; input the angiogram training image into the trained deep neural network model to output a vascular segmentation result; and adjust the value of the hyperparameter corresponding to the filter based on the vascular segmentation result to obtain a plurality of values of the hyperparameter corresponding to the filter.

[0107] As shown in Figure 8 The angiogram image processing and segmentation device further comprises a neural network model training module 75 connected to the neural network model segmentation module 73; the neural network model training module 75 is configured to obtain an angiogram training image, input the angiogram training image into a plurality of filters optimized in parameter, and output a composite filtered training image of the angiogram training image by each filter; and train a neural network joint segmentation model based on a plurality of composite filtered training images as input of the neural network joint segmentation model to be trained.

[0108] The filter comprises at least one of a Frangi filter, a Gabor filter, a Sato filter, and an Unsharp Mask filter.

[0109] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the angiogram image processing and segmentation device described above can refer to the corresponding process in the foregoing embodiments of the angiogram image processing and segmentation method, and will not be described here.

[0110] Embodiment four:

[0111] The embodiment of the present application also provides an electronic device for running the above angiogram image processing and segmentation method; referring to Figure 9A structural schematic diagram of an electronic device is shown, the electronic device includes a memory 100 and a processor 101, wherein the memory 100 is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor 101 to realize the above-mentioned angiogram image processing and segmentation method.

[0112] Further, Figure 9 The electronic device shown further includes a bus 102 and a communication interface 103, and the processor 101, the communication interface 103 and the memory 100 are connected through the bus 102.

[0113] Wherein the memory 100 can contain a high-speed random access memory (RAM, Random Access Memory), and can also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 103 (which can be wired or wireless), and the Internet, a wide area network, a local area network, a metropolitan area network, etc. can be used. The bus 102 can be an ISA bus, a PCI bus or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9 Only one bidirectional arrow is used in the middle, but it does not mean that there is only one bus or one type of bus.

[0114] The processor 101 can be an integrated circuit chip having a processing capability of signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor 101 or the instruction in the form of software. The processor 101 described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. Each method, step and logic block diagram disclosed in the embodiment of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiment of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the storage 100, and the processor 101 reads the information in the storage 100, and combines the hardware to complete the steps of the method of the above embodiment.

[0115] The embodiment of the present application further provides a computer readable storage medium, the computer readable storage medium stores computer executable instructions, when the computer executable instructions are called and executed by a processor, the computer executable instructions cause the processor to implement the above angiographic image processing and segmentation method, and specific implementation can be referred to the method embodiment, and will not be repeated here.

[0116] The computer program product of the angiographic image processing and segmentation method and device provided by the embodiment of the present application includes a computer readable storage medium storing program codes, the instructions included in the program codes can be used to execute the method in the foregoing method embodiment, and specific implementation can be referred to the method embodiment, and will not be repeated here.

[0117] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system and / or device described above can refer to the corresponding process in the foregoing method embodiment, and will not be repeated here.

[0118] In addition, in the description of the embodiments of the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connecting" should be understood in a broad sense, for example, can be fixed connection, can also be detachable connection, or integral connection; can be mechanical connection, can also be electrical connection; can be direct connection, can also be indirect connection through intervening medium, can be internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0119] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the part of the prior art or the part of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and various program code storage media.

[0120] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", "third" are only for the purpose of description, and cannot be understood as indicating or implying relative importance.

[0121] Finally, it should be noted that the above-described embodiments are merely specific embodiments of the present application, which are used to illustrate the technical solutions of the present application, but not to limit the present application, and the protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features therein, within the technical scope disclosed by the present application. Such modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for processing and segmenting angiographic images, characterized in that: The method comprises: Acquire an angiographic image, and input the angiographic image into a plurality of filters with pre-selected parameters; wherein each of the filters corresponds to a plurality of hyperparameter values; performing filtering processing on the angiography image based on multiple hyperparameter values ​​corresponding to each of the filters to obtain a plurality of filtered images, and performing composite processing on the plurality of filtered images of each of the filters to obtain a composite filtered image of each of the filters; inputting the plurality of composite filtered images into a pre-trained neural network joint segmentation model, and outputting a composite blood vessel segmentation result of the angiography image; The neural network joint segmentation model includes multiple deep neural network models; the step of inputting the multiple composite filtered images into the pre-trained neural network joint segmentation model and outputting a composite blood vessel segmentation result of the angiography image includes: inputting the multiple composite filtered images into the multiple deep neural network models respectively, each of the deep neural network models determining a blood vessel segmentation result of the input composite filtered image; and the neural network joint segmentation model performing composite processing on the multiple blood vessel segmentation results to obtain a composite blood vessel segmentation result of the angiography image; The step of composite processing the plurality of blood vessel segmentation results by the neural network joint segmentation model to obtain a composite blood vessel segmentation result of the angiography image includes: composite processing the plurality of blood vessel segmentation results by the following formula to obtain a composite blood vessel segmentation result of the angiography image: ;in, The composite vessel segmentation result OK Pixel classification of the column, the pixel classification includes blood vessel classification and non-blood vessel classification, classify the blood vessels, For the non-vascular classification, for The blood vessel segmentation results are OK The number of times the pixel in the column is classified as the vessel class, The number of times is preset.

2. The method according to claim 1, characterized in that The step of composite processing the multiple filtered images of each filter to obtain a composite filtered image of each filter comprises: Performing image normalization processing on each of the filtered images to obtain a plurality of normalized images; The plurality of normalized images are composited to obtain a composite filtered image of each filter.

3. The method according to claim 2, characterized in that The step of composite processing the plurality of normalized images to obtain a composite filtered image of each filter comprises: The composite filtered image of each filter is obtained by composite processing of the multiple normalized images using the following formula: ; in, is the first OK The pixel value of the column, N is the number of values ​​that the hyperparameter can take, For the k The first value of the normalized image corresponding to the hyperparameter value OK The pixel value of the column, To obtain the maximum value operation.

4. The method according to claim 1, wherein The number threshold is determined by the following formula: ;in, is the number threshold, For the rounding operation, is the number of blood vessel segmentation results.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Acquiring an angiography training image, and inputting the angiography training image into a plurality of filters for which parameters are to be optimized; Each of the filters outputs a filtered training image, and each of the filtered training images is normalized to obtain a plurality of normalized training images; Training a deep neural network model based on the plurality of normalized training images; Inputting the angiography training image into the trained deep neural network model and outputting a blood vessel segmentation result; The hyperparameter values ​​corresponding to the filter are adjusted based on the blood vessel segmentation result to obtain multiple hyperparameter values ​​corresponding to the filter.

6. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Acquiring an angiography training image, inputting the angiography training image into a plurality of filters after parameter optimization, each of the filters outputting a composite filtered training image of the angiography training image; The neural network joint segmentation model is trained based on a plurality of the composite filtered training images as inputs of the neural network joint segmentation model to be trained.

7. The method according to any one of claims 1 to 4, characterized in that The filter includes at least one of the following: a Frangi filter, a Gabor filter, a Sato filter, and an Unsharp Mask filter.

8. An angiography image processing and segmentation device, characterized in that: The device comprises: An angiographic image acquisition module, configured to acquire an angiographic image and input the angiographic image into a plurality of filters with pre-selected parameters; wherein each of the filters corresponds to a plurality of hyperparameter values; a filter processing module, configured to filter the angiography image based on multiple hyperparameter values ​​corresponding to each filter to obtain a plurality of filtered images, and composite the plurality of filtered images of each filter to obtain a composite filtered image of each filter; a neural network model segmentation module, configured to input the plurality of composite filtered images into a pre-trained neural network joint segmentation model and output a composite blood vessel segmentation result of the angiography image; The neural network joint segmentation model includes multiple deep neural network models; the neural network model segmentation module is used to input the multiple composite filtered images into the multiple deep neural network models respectively, and each deep neural network model determines a blood vessel segmentation result of the input composite filtered image; the neural network joint segmentation model performs composite processing on the multiple blood vessel segmentation results to obtain a composite blood vessel segmentation result of the angiography image; The neural network model segmentation module is used to perform composite processing on the multiple blood vessel segmentation results using the following formula to obtain a composite blood vessel segmentation result of the angiography image: ;in, The composite vessel segmentation result OK Pixel classification of the column, the pixel classification includes blood vessel classification and non-blood vessel classification, classify the blood vessels, For the non-vascular classification, for The blood vessel segmentation results are OK The number of times the pixel in the column is classified as the vessel class, The number of times is preset.

Images