Blood vessel segmentation model training method, blood vessel segmentation method and electronic equipment

Abstract

The invention discloses a blood vessel segmentation model training method, a blood vessel segmentation method and electronic equipment. The model training method comprises the following steps: inputting obtained DSA training data into a neural network model for training to obtain a blood vessel segmentation model; the neural network model comprises an encoder comprising at least one layer of encoding units, an attention module comprising attention units in one-to-one correspondence with the encoding units, and a decoder; during training, for each coding unit except the last layer of coding unit, inputting an image feature output by the coding unit into an attention unit corresponding to the coding unit, and inputting a product of an attention weight map output by the coding unit and the image feature into the next layer of coding unit; and inputting the image feature output by the last layer of coding unit into the attention unit corresponding to the coding unit, and inputting the product of the output attention weight map and the image feature into a decoder for identification and segmentation, thereby realizing automatic blood vessel segmentation.

Description

Technical Field

[0001] This disclosure generally relates to the field of artificial intelligence. More specifically, this disclosure relates to a method for training a vascular segmentation model, a vascular segmentation method, and an electronic device. Background Art

[0002] Medical image segmentation technology is very important in medical image analysis and can be used for human organ segmentation, vascular segmentation, vascular tumor segmentation, etc. For the vascular segmentation task, the current vascular segmentation methods mainly rely on manual segmentation, that is, relying on doctors' observation and manual marking. This not only results in low efficiency but also requires a high level of doctors, making it impossible to be popularized. At the same time, because the vascular structure is complex and there are many tiny blood vessels, doctors need to spend a lot of time and energy, often delaying the treatment time of patients. In view of this, there is an urgent need to provide a method for training a vascular segmentation model, a vascular segmentation method, and an electronic device to achieve automatic vascular segmentation. Summary of the Invention

[0003] In order to solve at least one or more of the above-mentioned technical problems, this disclosure proposes a method for training a vascular segmentation model, a vascular segmentation method, and an electronic device in multiple aspects.

[0004] In a first aspect, this disclosure provides a method for training a vascular segmentation model, the method including: obtaining DSA training pairs; the DSA training pairs include corresponding DSA training images and DSA training annotation images one by one, and the DSA training annotation images are obtained by performing vascular segmentation annotation on the DSA training images; inputting the DSA training data pairs into a neural network model for training to obtain a vascular segmentation model; wherein, the neural network model at least includes: an encoder for extracting image features, an attention module, and a decoder for recognizing and segmenting image features; the encoder includes at least one layer of encoding units, the attention module includes attention units corresponding to the encoding units one by one, and the attention units are connected to their corresponding encoding units; during the training process, for each encoding unit except the last layer of encoding units, input the image features output by this encoding unit into the attention unit corresponding to this encoding unit to obtain an attention weight map, and input the product of the attention weight map and the image features into the next layer of encoding units; for the last layer of encoding units, input the image features output by this encoding unit into the attention unit corresponding to this encoding unit to obtain an attention weight map, and input the product of the attention weight map and the image features into the decoder for the decoder to perform operations of recognizing and segmenting image features.

[0005] In some embodiments, the method further includes: obtaining a DSA verification data pair; the DSA verification data pair includes a corresponding DSA verification image and a DSA verification annotation image, and the DSA verification annotation image is obtained by performing vascular segmentation annotation on the DSA verification image; during the process of inputting the DSA training data pair into a neural network model for training to obtain a vascular segmentation model, the vascular segmentation model is verified based on the DSA verification data pair.

[0006] In some embodiments, before inputting the DSA training data pair into a neural network model for training to obtain a vascular segmentation model, the method further includes: respectively preprocessing the DSA training image and the DSA training annotation image in the DSA training data pair to obtain a preprocessed DSA training image and a preprocessed DSA training annotation image; the preprocessing includes at least one of resampling, normalization, image cropping, image padding, and image enhancement; inputting the DSA training data pair into a neural network model for training to obtain a vascular segmentation model includes: inputting the preprocessed DSA training image and the preprocessed DSA training annotation image into the neural network model for training to obtain the vascular segmentation model.

[0007] In some embodiments, inputting the DSA training data pair into a neural network model for training to obtain a vascular segmentation model includes: performing vascular segmentation on the DSA training image by the neural network model to obtain a vascular segmentation prediction result; calculating at least two candidate loss values based on the vascular segmentation prediction result and the DSA training annotation image; obtaining a target loss value based on the at least two candidate loss values, and adjusting the model parameters of the neural network model based on the target loss value until a set iteration condition is met to obtain the vascular segmentation model.

[0008] In some embodiments, the candidate loss value at least includes a centerline dice loss value; the centerline dice loss value is calculated by the following formula: where cl-DiceLoss represents the centerline dice loss value; T prec represents the intersection ratio between the vascular segmentation prediction result and the DSA training annotation image; T scns represents the intersection ratio between the DSA training annotation image and the vascular segmentation prediction result.

[0009] In some embodiments, the vascular segmentation model is an nnUNet model.

[0010] In a second aspect, the present disclosure provides a blood vessel segmentation method, the method comprising: obtaining a DSA image to be segmented; inputting the DSA image to be segmented into a blood vessel segmentation model for blood vessel segmentation to obtain a blood vessel segmentation result; wherein, the blood vessel segmentation model is trained based on the blood vessel segmentation model training method according to any one of claims 1-6.

[0011] In some embodiments, after obtaining the blood vessel segmentation result of the DSA image to be segmented, the method further comprises: post-processing the blood vessel segmentation result to obtain a target blood vessel segmentation result; the post-processing includes at least one of denoising and smoothing.

[0012] In a third aspect, the present disclosure provides an electronic device, comprising: a processor configured to execute program instructions; and a memory configured to store the program instructions, when the program instructions are loaded and executed by the processor, causing the processor to execute the steps of the blood vessel segmentation model training method according to the first aspect or any optional embodiment of the first aspect or execute the steps of the blood vessel segmentation method according to the second aspect or any optional embodiment of the second aspect.

[0013] In a fourth aspect, the present disclosure provides a computer-readable storage medium, in which program instructions are stored, when the program instructions are loaded and executed by a processor, causing the processor to execute the steps of the blood vessel segmentation model training method according to the first aspect or any optional embodiment of the first aspect or execute the steps of the blood vessel segmentation method according to the second aspect or any optional embodiment of the second aspect.

[0014] Through the blood vessel segmentation model training method, blood vessel segmentation method and electronic device provided as above, the embodiments of the present disclosure obtain DSA training pairs and train a blood vessel segmentation model by using the DSA training pairs, and then can use the trained blood vessel segmentation model for blood vessel segmentation, realizing automatic blood vessel segmentation and improving the blood vessel segmentation efficiency. Further, by adding an attention module to the neural network model, the model assigns different weights to each pixel in the DSA training image during the training process to increase the semantic representation of the pixel or a certain region, improving the segmentation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the drawings, several embodiments of the present disclosure are shown in an exemplary rather than restrictive manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0016] Figure 1 An exemplary flowchart of the blood vessel segmentation model training method according to some embodiments of the present disclosure is shown;

[0017] Figure 2 Shows a specific example diagram of a neural network model according to some embodiments of the present disclosure;

[0018] Figure 3 Shows an exemplary flowchart of a blood vessel segmentation method according to some embodiments of the present disclosure;

[0019] Figure 4 Shows an overall schematic diagram of model training and model inference according to some embodiments of the present disclosure;

[0020] Figure 5 Shows an exemplary structural block diagram of a blood vessel segmentation model training device according to some embodiments of the present disclosure;

[0021] Figure 6 Shows an exemplary structural block diagram of a blood vessel segmentation device according to some embodiments of the present disclosure;

[0022] Figure 7 Shows an exemplary structural block diagram of an electronic device according to some embodiments of the present disclosure. Detailed implementation manners

[0023] Next, the technical solutions in the embodiments of the present disclosure will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are some, but not all, of the embodiments of the present disclosure. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present disclosure.

[0024] It should be understood that the terms "including" and "comprising" used in the specification and claims of the present disclosure indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0025] It should also be understood that the terms used in the specification of the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure. As used in the specification and claims of the present disclosure, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms. It should also be further understood that the term " / and / " used in the specification and claims of the present disclosure refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0026] As used in this specification and the claims, the term "if" can be interpreted as "when" or "once" or "in response to determining" or "in response to detecting" depending on the context. Similarly, the phrase "if determined" or "if [the described condition or event] is detected" can be interpreted as meaning "once determined" or "in response to determining" or "once [the described condition or event] is detected" or "in response to detecting [the described condition or event]" depending on the context.

[0027] The specific embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0028] Figure 1 An exemplary flowchart of a method 100 for training a vascular segmentation model according to some embodiments of the present disclosure is shown.

[0029] As Figure 1 shown, the method 100 for training a vascular segmentation model provided by the present disclosure includes: Step S110: Obtain DSA training pairs; Step S120: Input the DSA training data pairs into a neural network model for training to obtain a vascular segmentation model; wherein, the neural network model at least includes: an encoder for extracting image features, an attention module, and a decoder for recognizing and segmenting image features; the encoder includes at least one layer of encoding units, the attention module includes attention units corresponding one-to-one to the encoding units, and the attention units are connected to their corresponding encoding units; during the training process, for each encoding unit except the last layer of encoding units, the image features output by the encoding unit are input into the attention unit corresponding to the encoding unit to obtain an attention weight map, and the product of the attention weight map and the image features is input into the next layer of encoding units; for the last layer of encoding units, the image features output by the encoding unit are input into the attention unit corresponding to the encoding unit to obtain an attention weight map, and the product of the attention weight map and the image features is input into the decoder for the decoder to perform operations of recognizing and segmenting image features.

[0030] Exemplarily, the DSA training pairs in the above step S110 include corresponding DSA training images and DSA training annotation images. It can be understood that the above DSA (Digital Subtraction Angiography) is a high-end medical imaging technology that combines traditional angiography and computer image processing technology. Its principle is to take two frames of X-ray images before and after injecting a contrast agent at the same part of the human body, subtract them after digital processing to eliminate the images of bones and soft tissues, and only retain the images of blood vessels. DSA images are of great value for vascular variations, vascular diseases, and showing the relationship between lesions and blood vessels. In the embodiments of the present disclosure, the above DSA training images can be 3D DSA grayscale images or 3D DSA color images. The embodiments of the present disclosure do not make specific limitations in this regard. The embodiments of the present disclosure may describe the case where the DSA training images are grayscale images as an example.

[0031] In the embodiments of the present disclosure, the DSA training images can be the collected brain DSA images, heart DSA images, abdominal DSA images, neck DSA images, etc., and can realize the vascular segmentation of DSA images of multiple parts.

[0032] In the embodiments of the present disclosure, the DSA training annotation images are obtained by performing vascular segmentation annotation on the DSA training images. It can be understood that there are many data annotation methods. For example, manual annotation, that is, manually performing vascular segmentation annotation on the DSA training images. Another example is automatic annotation, that is, inputting the DSA training images into a pre-trained annotation model, and the annotation model performs vascular segmentation annotation on the DSA training images. Another example is the combination of automatic annotation and manual annotation, that is, inputting the DSA training images into a pre-trained annotation model, the annotation model performs vascular segmentation annotation on the DSA training images, and then manually corrects the vascular segmentation annotation results of the model. The DSA training annotation images in the embodiments of the present disclosure are obtained by manually performing vascular segmentation annotation on the DSA training images.

[0033] In the embodiments of the present disclosure, when performing vascular segmentation annotation on DSA training images, each pixel point of the DSA training image is annotated as to whether it is a vascular pixel point, so as to form vascular annotation information. That is, the vascular pixel points (denoted as foreground pixel points) and non-vascular pixel points (denoted as background pixel points) in the DSA training image are annotated to form the foreground region of the blood vessels and the background region of non-blood vessels. Specifically, binary classification annotation can be performed on each pixel point on the DSA training image. For example, if the pixel point belongs to a blood vessel, the pixel point is annotated with a first label, and if the pixel point does not belong to a blood vessel, the pixel point is annotated with a second label. Here, the first label and the second label can have many forms of representation, such as different numbers, different colors, different letters, etc. As a specific embodiment, the first label is "1" and the second label is "0", and the embodiments of the present disclosure do not make specific limitations on the first label and the second label.

[0034] Exemplarily, in the embodiments of the present disclosure, the neural network model in the above step S120 is an nnUNet model. Thus, the trained vascular segmentation model is also an nnUNet model. In the embodiments of the present disclosure, the above neural network model at least includes: an encoder for extracting image features, an attention module, and a decoder for recognizing and segmenting image features. Among them, the encoder includes at least one layer of encoding units. For example, 3 layers, 5 layers, and the embodiments of the present disclosure do not make specific limitations on the number of encoding units included in the encoder.

[0035] In the embodiments of the present disclosure, the above attention module is a spatial attention module, which can enable the neural network model to focus on the key spatial regions of the feature map, help the model better focus on the target region, and improve the model segmentation accuracy. In the embodiments of the present disclosure, the above attention module includes attention units corresponding one-to-one to the encoding units, and the attention units are connected to their corresponding encoding units.

[0036] Figure 2 Shows a specific example diagram of the neural network model of some embodiments of the present disclosure. As Figure 2 shown, the neural network model includes an input layer (data) for inputting DSA training images, an encoder for extracting image features, an attention module, an intermediate layer, a decoder for recognizing and segmenting image features, and an output layer for outputting vascular segmentation results. Among them, Figure 2 the encoder in Figure 2 includes 3 layers of encoding units, and each layer of encoding units includes a convolution operation (doubleconv) and a downsampling operation (1*1conv, max pool). Here, as

[0037] As also shown Figure 2 in Figure 2 , the attention module includes three layers of attention units, which are respectively arranged behind each encoding unit of the encoder, that is, the output of each encoding unit is connected to an attention unit. Here, the attention unit includes an average pooling operation. During the training process, for each encoding unit except the last layer of encoding units, the image features output by the encoding unit are input into the attention unit corresponding to the encoding unit to obtain an attention weight map, and the product of the attention weight map and the image features output by the encoding unit is input into the next layer of encoding units; for the last layer of encoding units, the image features output by the encoding unit are input into the attention unit corresponding to the encoding unit to obtain an attention weight map, and the product of the attention weight map and the image features output by the encoding unit is input into the decoder for the decoder to perform operations of identifying and segmenting the image features. Here, the product of the attention weight map output by the attention unit corresponding to the last layer of encoding units and the image features output by the encoding unit can be directly input into the decoder, or can be input into the decoder after performing a convolution operation through an intermediate layer ( Figure 2 shown in Figure 2 ). The present disclosure embodiment does not make specific limitations on this.

[0038] In the present disclosure embodiment, as also Figure 2 shown in Figure 2 , the decoder includes three layers of decoding units, and each layer of decoding unit includes an upsampling operation (Conv Transpose) and a convolution operation (double conv). In the present disclosure embodiment, the output of the decoder is input into the output layer of the neural network model for 1*1 convolution and normalized through the activation function sigmoid to obtain the blood vessel segmentation result (i.e., Segs Res).

[0039] It should be noted that the number of the above-mentioned encoding units, the number of convolutions, and the downsampling operation are all examples and are not used to limit the present disclosure. All implementation manners applicable to the present disclosure are within the protection scope of the present disclosure.

[0040] Exemplarily, in the present disclosure embodiment, when training the model using DSA training data pairs, the initial learning rate can be set to 0.01, the optimizer adopts the method of stochastic gradient descent, and the batch size and training block size, etc. can be determined according to the size of the graphics card. The present disclosure embodiment does not make specific limitations on this.

[0041] As for the specific training process, the following embodiments give examples for description and will not be elaborated here for the time being.

[0042] In the embodiments of the present disclosure, by obtaining DSA training pairs and using the DSA training pairs to train a vascular segmentation model, the vascular segmentation model can be further used for vascular segmentation, achieving automated vascular segmentation and improving the efficiency of vascular segmentation. Further, by adding an attention module to the neural network model, the model assigns different weights to each pixel in the DSA training image during training to increase the semantic representation of the pixel or a certain region, thereby improving the segmentation accuracy.

[0043] As an optional embodiment of the present disclosure, the above-mentioned vascular segmentation model training method 100 further includes: obtaining DSA verification data pairs; during the process of inputting the DSA training data pairs into the neural network model for training to obtain the vascular segmentation model, verifying the vascular segmentation model based on the DSA verification data pairs.

[0044] Exemplarily, in the embodiments of the present disclosure, the above-mentioned DSA verification data pairs include a DSA verification image and a DSA verification annotation image that correspond one by one. Among them, the DSA verification annotation image is obtained by performing vascular segmentation annotation on the DSA verification image. Here, the DSA verification image can be a 3D DSA grayscale image or a 3D DSA color image, and the embodiments of the present disclosure do not make specific limitations on this, as long as it is consistent with the DSA training image.

[0045] In the embodiments of the present disclosure, the above-mentioned DSA verification image can be, for example, any one of a brain DSA image, a neck DSA image, a heart DSA image, and an abdominal DSA image, as long as it is consistent with the DSA training image. The DSA verification image includes vascular details of the corresponding part.

[0046] In the embodiments of the present disclosure, the method for obtaining the DSA verification annotation image by performing vascular segmentation annotation on the DSA verification image is the same as the method for obtaining the DSA training annotation image by performing vascular segmentation annotation on the DSA training image, and it is also obtained through manual annotation.

[0047] In the embodiments of the present disclosure, the number of DSA verification data pairs can be any number. For example, the ratio to the number of DSA training data pairs can be 8:2, 7:3, etc., and the embodiments of the present disclosure do not make specific limitations on this.

[0048] As a specific embodiment, DSA sample data pairs can be obtained first, and then the DSA sample data pairs are divided into a DSA training data pair set and a DSA validation data pair set according to a set ratio (for example, 8:2). That is, 80% of the DSA data pairs in the DSA sample data pairs are divided into the DSA training data pair set; the remaining 20% of the DSA data pairs are divided into the DSA validation data pair set. Then, the vascular segmentation model is trained based on the DSA training data pairs in the DSA training data pair set, and the model is verified based on the DSA validation data pairs in the DSA validation data pair set.

[0049] Of course, in the embodiments of this disclosure, a k-fold cross-validation strategy can also be used to obtain the vascular segmentation model. The method of obtaining the vascular segmentation model using the k-fold cross-validation strategy is a conventional method, which can be referred to the description in relevant materials and will not be elaborated here.

[0050] In the embodiments of this disclosure, during the process of training the vascular segmentation model above, the vascular segmentation model can be verified based on the DSA validation data. Specifically, during the process of training the candidate vascular segmentation model, the vascular segmentation model can be verified based on the DSA validation data once per round of training; or the vascular segmentation model can be verified based on the DSA validation data once every 5 rounds of training. Of course, the vascular segmentation model can also be verified based on the DSA validation data after the training is completed. The embodiments of this disclosure do not make specific limitations on this.

[0051] In the embodiments of this disclosure, verifying the vascular segmentation model based on the DSA validation data can specifically be to calculate various metrics of the vascular segmentation model for verification. The metrics here can be, for example, the Dice correlation coefficient, precision, sensitivity, etc. The embodiments of this disclosure do not make specific limitations on this.

[0052] In the embodiments of this disclosure, by verifying the vascular segmentation model based on the DSA validation data during the training process of the vascular segmentation model to evaluate the generalization ability of the vascular segmentation model, it can better help train the vascular segmentation model and improve the model generalization ability.

[0053] As an optional embodiment of this disclosure, before inputting the DSA training data pairs into the neural network model for training to obtain the vascular segmentation model, the above-mentioned vascular segmentation model training method 100 further includes: preprocessing the DSA training images and the DSA training annotation images in the DSA training data pairs respectively to obtain the preprocessed DSA training images and the preprocessed DSA training annotation images; inputting the DSA training data pairs into the neural network model for training to obtain the vascular segmentation model, including: inputting the preprocessed DSA training images and the preprocessed DSA training annotation images into the neural network model for training to obtain the vascular segmentation model.

[0054] Exemplarily, in the embodiments of the present disclosure, the preprocessing includes at least one of resampling, normalization, image cropping, image padding, and image enhancement; the image enhancement here includes, but is not limited to, one or more of rotation, scaling, flipping, blurring, gamma enhancement, or contrast enhancement.

[0055] In the embodiments of the present disclosure, after preprocessing the DSA training image and the DSA training annotation image in the DSA training data pair, a candidate blood vessel segmentation model is trained using the preprocessed DSA training image and the preprocessed DSA training annotation image.

[0056] The specific process of model training is described below. Specifically, the DSA training data pair is input into a neural network model for training to obtain a blood vessel segmentation model, including: the neural network model performs blood vessel segmentation on the DSA training image to obtain a blood vessel segmentation prediction result; at least two candidate loss values are calculated based on the blood vessel segmentation prediction result and the DSA training annotation image; a target loss value is obtained based on the at least two candidate loss values, and the model parameters of the neural network model are adjusted based on the target loss value until a set iteration condition is met to obtain the blood vessel segmentation model.

[0057] Exemplarily, in the embodiments of the present disclosure, the blood vessel segmentation prediction result is obtained by the neural network model performing blood vessel segmentation on the DSA training image. Here, the blood vessel segmentation prediction result is the probability value of each pixel point on the DSA training image being a blood vessel pixel point and the probability value of a non-blood vessel pixel point, and the prediction result of each pixel point can be a 1*2 matrix. As a specific embodiment, for each pixel point on the DSA training image, its prediction result can be, for example: the probability that the pixel point is a blood vessel pixel point is 0.8, and the probability that it is a non-blood vessel pixel point is 0.2, then the pixel point is a blood vessel pixel point.

[0058] In the embodiments of the present disclosure, after obtaining the blood vessel segmentation prediction result, at least two candidate loss values are calculated based on the blood vessel segmentation prediction result and the DSA training annotation image. The candidate loss values here can be at least two of cross-entropy loss value, centerline dice loss value, dice loss value, mean square error loss value, etc., and the embodiments of the present disclosure do not make specific limitations in this regard.

[0059] The above candidate loss values can all be calculated through corresponding loss functions. The embodiments of the present disclosure only take the calculation of the centerline dice loss value as an example for illustration. As for the cross-entropy loss function, dice loss function, and mean square error loss function, they are all conventional loss functions, and their specific calculation formulas can be referred to the descriptions in relevant materials and will not be elaborated here.

[0060] The centerline dice loss value (cl-DiceLoss) here is a novel topology-preserving loss function specifically for the segmentation tasks of tubular structures such as blood vessels, neurons, and roads. Specifically, it can be calculated by the following formula:

[0061]

[0062] where cl-DiceLoss represents the centerline dice loss value; T prec represents the intersection ratio of the blood vessel segmentation prediction result and the DSA training annotation image, which is used to measure the accuracy of the blood vessel segmentation prediction result. Specifically, it represents how much of the blood vessel segmentation prediction result truly belongs; T scns represents the intersection ratio of the DSA training annotation image and the blood vessel segmentation prediction result, which is used for the coverage ability of the blood vessel segmentation prediction result to the DSA training annotation image. Specifically, it represents how much of the DSA training annotation image is covered by the blood vessel segmentation prediction result.

[0063] In the embodiments of the present disclosure, after obtaining at least two candidate loss values, the target loss value can be obtained based on the at least two candidate loss values. Specifically, the above target loss value can be obtained by performing weighted average calculation on the at least two candidate loss values. When performing the weighted average calculation, the coefficients of different candidate loss values can be set arbitrarily. For example, the coefficients of each candidate loss value are the same. For another example, different coefficients can be assigned to different candidate loss values according to actual needs. Taking the number of candidate loss values as 2 for example, for the convenience of description, the two candidate loss values are calculated as the first loss value and the second loss value, and the coefficients of the first loss value and the second loss value are both 0.5. For another example, the first coefficient of the first loss value is 0.3 and the coefficient of the second loss value is 0.7. The embodiments of the present disclosure do not make specific limitations on this and can be determined according to actual situations.

[0064] In the embodiments of the present disclosure, after obtaining the target loss value, the model parameters of the neural network model can be adjusted reversely based on the target loss value. Here, the model parameters can include, for example, the weights and biases of the model, etc., to realize the training of the neural network model.

[0065] In the embodiments of the present disclosure, the set iteration condition can be the convergence of the target loss value, or it can be reaching the set number of iterations (for example, 1000 times), etc. The embodiments of the present disclosure do not make specific limitations on this. When the model training reaches the set iteration condition, the trained blood vessel segmentation model can be obtained.

[0066] The embodiments of the present disclosure significantly improve the situation of blood vessel disconnection by adding cl-DiceLoss to the training of the blood vessel segmentation model.

[0067] Figure 3An exemplary flowchart of the vascular segmentation method 300 according to some embodiments of the present disclosure is shown.

[0068] As Figure 3 shown, the vascular segmentation method 300 provided by the present disclosure includes: Step S310: Obtain the DSA image to be segmented; Step S320: Input the DSA image to be segmented into the vascular segmentation model for vascular segmentation to obtain a vascular segmentation result; wherein, the vascular segmentation model is trained based on the vascular segmentation model training method described in the above embodiments.

[0069] Exemplarily, the DSA image to be segmented in the above step S310 may be a 3D DSA grayscale image or a 3D DSA color image. The embodiments of the present disclosure do not make specific limitations on this, as long as it is consistent with the DSA training image.

[0070] In the embodiments of the present disclosure, the DSA image to be segmented may be, for example, one of a brain DSA image, a chest DSA image, an abdominal DSA image, etc., which includes the vascular details of the corresponding part.

[0071] Exemplarily, in the embodiments of the present disclosure, the DSA image to be segmented is input into the vascular segmentation model for vascular segmentation to obtain a vascular segmentation result including the vascular contour. Here, the vascular segmentation model is trained based on the embodiments of the above vascular segmentation model training method. Therefore, by segmenting the DSA image to be segmented based on the vascular segmentation model, a more accurate vascular segmentation result can be obtained.

[0072] As an optional embodiment of the present disclosure, after obtaining the vascular segmentation result of the DSA image to be segmented, the above vascular segmentation method 300 further includes: post-processing the vascular segmentation result to obtain a target vascular segmentation result.

[0073] Exemplarily, in the embodiments of the present disclosure, the above post-processing may include at least one of denoising and smoothing. By post-processing the vascular segmentation result in the embodiments of the present disclosure, the noise in the vascular segmentation result can be removed, the vascular segmentation result can be optimized, and a more accurate target vascular segmentation result can be obtained.

[0074] Figure 4 An overall schematic diagram of model training and model inference according to some embodiments of the present disclosure is shown.

[0075] As Figure 4As shown, first, prepare a dataset pair, which includes a 3D DSA grayscale image (i.e., the above DSA training image) and a 3D vascular mask (i.e., the above DSA training annotation image) that correspond one by one. Then, perform at least one preprocessing operation such as resampling, normalization, image cropping, image padding, and image enhancement on the 3D DSA grayscale image and the 3D vascular mask to obtain the preprocessed 3D DSA grayscale image and the preprocessed 3D vascular mask. Input the preprocessed 3D DSA grayscale image and the preprocessed 3D vascular mask into the nnUNet model for training to obtain a model (i.e., the above vascular segmentation model). Then, input the 3D DSA grayscale image (i.e., the above DSA image to be annotated) into the vascular segmentation model, and the vascular segmentation model infers the 3D DSA vascular segmentation result of the 3D DSA grayscale image (i.e., the above vascular segmentation result).

[0076] Figure 5 FIG. shows an exemplary structural block diagram of a vascular segmentation model training device 500 according to some embodiments of the present disclosure.

[0077] As Figure 5 shown, a vascular segmentation model training device 500 provided by an embodiment of the present disclosure includes: a DSA training data pair acquisition module 510, configured to acquire a DSA training pair; the DSA training pair includes a DSA training image and a DSA training annotation image that correspond one by one, and the DSA training annotation image is obtained by performing vascular segmentation annotation on the DSA training image; a model training module 520, configured to input the DSA training data pair into a neural network model for training to obtain a vascular segmentation model; wherein, the neural network model at least includes: an encoder for extracting image features, an attention module, and a decoder for recognizing and segmenting image features; the encoder includes at least one layer of encoding units, the attention module includes attention units corresponding one by one to the encoding units, and the attention units are connected to their corresponding encoding units; during training, for each encoding unit except the last layer of encoding units, input the image features output by the encoding unit into the attention unit corresponding to the encoding unit to obtain an attention weight map, and input the product of the attention weight map and the image features into the next layer of encoding units; for the last layer of encoding units, input the image features output by the encoding unit into the attention unit corresponding to the encoding unit to obtain an attention weight map, and input the product of the attention weight map and the image features into the decoder for the decoder to perform operations of recognizing and segmenting image features.

[0078] As an optional embodiment of the present disclosure, the above-mentioned vascular segmentation model training device 500 further includes: a DSA verification data pair acquisition module, configured to acquire DSA verification data pairs; the DSA verification data pairs include DSA verification images and DSA verification annotation images that correspond one by one, and the DSA verification annotation images are obtained by performing vascular segmentation annotation on the DSA verification images; a verification module, configured to verify the vascular segmentation model based on the DSA verification data pairs during the process of inputting the DSA training data pairs into the neural network model for training to obtain the vascular segmentation model.

[0079] As an optional embodiment of the present disclosure, the above-mentioned vascular segmentation model training device 500 further includes: a preprocessing module, configured to preprocess the DSA training images and DSA training annotation images in the DSA training data pairs respectively to obtain preprocessed DSA training images and preprocessed DSA training annotation images; the preprocessing includes at least one of resampling, normalization, image cropping, image padding, and image enhancement; the above-mentioned model training module 520 is specifically configured to: input the preprocessed DSA training images and preprocessed DSA training annotation images into the neural network model for training to obtain the vascular segmentation model.

[0080] As an optional embodiment of the present disclosure, the above-mentioned model training module 520 is specifically configured to: perform vascular segmentation on the DSA training images by the neural network model to obtain a vascular segmentation prediction result; calculate at least two candidate loss values based on the vascular segmentation prediction result and the DSA training annotation images; obtain a target loss value based on the at least two candidate loss values, and adjust the model parameters of the neural network model based on the target loss value until the set iteration condition is satisfied to obtain the vascular segmentation model.

[0081] As an optional embodiment of the present disclosure, the candidate loss values at least include a centerline dice loss value; the centerline dice loss value is calculated by the following formula: where cl-DiceLoss represents the centerline dice loss value; T prec represents the intersection ratio of the vascular segmentation prediction result and the DSA training annotation image; T scns represents the intersection ratio of the DSA training annotation image and the vascular segmentation prediction result.

[0082] As an optional embodiment of the present disclosure, the vascular segmentation model is an nnUNet model.

[0083] Figure 6 The exemplary structural block diagram of the vascular segmentation device 600 according to some embodiments of the present disclosure is shown.

[0084] As Figure 6As shown in the figure, a vascular segmentation device 600 provided by an embodiment of the present disclosure includes: a DSA image to be segmented acquisition module 610, configured to acquire a DSA image to be segmented; a vascular segmentation module 620, configured to input the DSA image to be segmented into a vascular segmentation model for vascular segmentation to obtain a vascular segmentation result; wherein, the vascular segmentation model is trained based on the vascular segmentation model training method of the above embodiment.

[0085] As an optional embodiment of the present disclosure, the above-mentioned vascular segmentation device 600 further includes: a post-processing module, configured to perform post-processing on the vascular segmentation result to obtain a target vascular segmentation result; the post-processing includes at least one of denoising and smoothing.

[0086] Correspondingly, an embodiment of the present disclosure also provides Figure 5 or Figure 6 a hardware structure diagram of the device shown, specifically as Figure 7 shown, the electronic device 700 may be a device for implementing the above method 100 or method 300. As Figure 7 shown, the electronic device 700 includes: a processor 710 and a memory 720. Among them, the memory 720 is configured to store program instructions; the processor 710 is configured to load and execute the program instructions stored in the memory 720 to implement the corresponding embodiment of the vascular segmentation model training method or the corresponding embodiment of the vascular segmentation method as shown above.

[0087] As an embodiment, the memory 720 may be any electronic, magnetic, optical or other physical storage device, and may contain or store information such as program instructions, data, and the like. For example, the memory 720 may be: a volatile memory, a non-volatile memory or a similar storage medium. Specifically, the memory 720 may be a RAM (Random Access Memory), a flash memory, a storage drive (such as a hard disk drive), a solid-state drive, any type of storage disk (such as an optical disk, a DVD, etc.), or a similar storage medium, or a combination thereof.

[0088] Although multiple embodiments of the present disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art can envision many changes, alterations, and alternative ways without departing from the spirit and scope of the present disclosure. It should be understood that various alternatives to the embodiments of the present disclosure described herein may be employed in practicing the present disclosure. The appended claims are intended to define the scope of protection of the present disclosure and thus cover equivalents or alternatives within the scope of these claims.

Claims

1. A blood vessel segmentation model training method, characterized in that: The method comprises: Acquire a DSA training pair; the DSA training pair includes a one-to-one corresponding DSA training image and a DSA training annotated image, wherein the DSA training annotated image is obtained by performing blood vessel segmentation and annotation on the DSA training image; The DSA training data pair is input into a neural network model for training to obtain a blood vessel segmentation model; wherein the neural network model at least includes: an encoder for extracting image features, an attention module, and a decoder for identifying and segmenting image features; the encoder includes at least one layer of encoding units, the attention module includes attention units corresponding to the encoding units one by one, and the attention units are connected to the corresponding encoding units; During the training process, for each coding unit except the last layer of coding units, the image features output by the coding unit are input into the attention unit corresponding to the coding unit to obtain an attention weight map, and the product of the attention weight map and the image features is input into the next layer of coding units; for the last layer of coding units, the image features output by the coding unit are input into the attention unit corresponding to the coding unit to obtain an attention weight map, and the product of the attention weight map and the image features is input into the decoder so that the decoder can perform operations of identifying and segmenting image features.

2. The method according to claim 1, characterized in that: The method further comprises: Acquire a DSA verification data pair; the DSA verification data pair includes a one-to-one corresponding DSA verification image and a DSA verification annotated image, wherein the DSA verification annotated image is obtained by performing blood vessel segmentation and annotation on the DSA verification image; In the process of inputting the DSA training data pair into the neural network model for training to obtain the blood vessel segmentation model, the blood vessel segmentation model is verified based on the DSA verification data.

3. The method according to claim 1, characterized in that Before inputting the DSA training data pair into the neural network model for training to obtain the blood vessel segmentation model, the method further includes: Preprocessing the DSA training image and the DSA training annotated image in the DSA training data pair respectively to obtain a preprocessed DSA training image and a preprocessed DSA training annotated image; the preprocessing includes: at least one of resampling, normalization, image cropping, image filling, and image enhancement; The step of inputting the DSA training data pair into a neural network model for training to obtain a blood vessel segmentation model comprises: The preprocessed DSA training image and the preprocessed DSA training annotated image are input into the neural network model for training to obtain the blood vessel segmentation model.

4. The method according to claim 1, characterized in that: The step of inputting the DSA training data pair into a neural network model for training to obtain a blood vessel segmentation model comprises: Performing blood vessel segmentation on the DSA training image by using the neural network model to obtain a blood vessel segmentation prediction result; Calculate at least two candidate loss values ​​based on the blood vessel segmentation prediction result and the DSA training annotated image; A target loss value is obtained based on the at least two candidate loss values, and model parameters of the neural network model are adjusted based on the target loss value until a set iteration condition is met, thereby obtaining the blood vessel segmentation model.

5. The method according to claim 4, characterized in that The candidate loss values ​​at least include a centerline dice loss value; the centerline dice loss value is calculated by the following formula: Among them, cl-DiceLoss represents the centerline dice loss value; T prec represents the intersection ratio between the vascular segmentation prediction result and the DSA training annotated image; T scns Represents the intersection ratio of the DSA training annotation image and the vascular segmentation prediction result.

6. The method according to claim 1, characterized in that The blood vessel segmentation model is a nnUNet model.

7. A blood vessel segmentation method, characterized in that: The method comprises: Obtaining a DSA image to be segmented; The DSA image to be segmented is input into a vascular segmentation model to perform vascular segmentation to obtain a vascular segmentation result; wherein the vascular segmentation model is trained based on the vascular segmentation model training method according to any one of claims 1 to 6.

8. The method according to claim 7, characterized in that After obtaining the blood vessel segmentation result of the DSA image to be segmented, the method further includes: The blood vessel segmentation result is post-processed to obtain a target blood vessel segmentation result; the post-processing includes: at least one of denoising and smoothing.

9. An electronic device, characterized in that: include: a processor configured to execute program instructions; as well as A memory configured to store the program instructions, and when the program instructions are loaded and executed by the processor, the processor executes the steps of the blood vessel segmentation model training method as described in any one of claims 1 to 6 or executes the steps of the blood vessel segmentation method as described in any one of claims 7 to 8.

10. A computer-readable storage medium having program instructions stored therein, characterized in that: When the program instructions are loaded and executed by the processor, the processor executes the steps of the blood vessel segmentation model training method according to any one of claims 1 to 6 or the steps of the blood vessel segmentation method according to any one of claims 7 to 8.

Images