Human brain image blood vessel segmentation method based on semantic-level context screening mechanism

Abstract

The invention discloses a human brain image blood vessel segmentation method based on a semantic-level context screening mechanism, and the method comprises the steps: giving a medical image, obtaining a pixel-level context feature based on the medical image, inputting the pixel-level context feature into a pseudo tag generation module, and obtaining a pseudo tag; semantic-level context modeling is carried out on the pixel-level features of the pixel-level context based on pseudo tags, semantic-level context features are obtained, and the semantic-level context features are optimized and improved; constructing a semantic-level relation matrix, respectively interacting the semantic-level context features and the pixel-level context features through the semantic-level relation matrix, and obtaining a pixel relation between the interior of the semantic context and the image feature representation; and segmenting the medical image based on a pixel relationship between the interior of the semantic context and the image feature representation to obtain a segmentation result. According to the invention, the accuracy of brain medical image blood vessel segmentation is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a human brain image blood vessel segmentation method based on a semantic level context screening mechanism. Background Art

[0002] Blood vessel segmentation in images such as human brain SNAP requires pixel-by-pixel classification of blood vessel pixels in medical images to accurately identify the blood vessel areas in medical images. Traditional image segmentation algorithms often rely on manually designed feature extraction and classifiers, which have problems such as low recognition accuracy and poor adaptability to complex images. Unlike traditional image segmentation technology, semantic segmentation technology based on deep learning can more accurately identify objects in images. By learning a large amount of data, it automatically learns the features and patterns in the image, thereby achieving more accurate object recognition and segmentation.

[0003] In the field of semantic segmentation based on deep learning, related research work can be roughly divided into two types. One is to design a new backbone network to obtain more reliable deep features. Considering the gradient disappearance and model degradation problems of deep networks, a modular architecture ResNet is proposed in the prior art to capture cross-feature interactions and learn different representations by utilizing channel attention on different network branches. So far, it is still the most widely used backbone network. At the same time, some improvements have emerged, such as ResNeXt and ResNeSt. The other is to aggregate the context information of the image to achieve remote pixel dependency modeling to alleviate the problem of limited receptive field in FCN. In order to obtain effective multi-scale context information, PSPNet introduces SPP to capture hierarchical global information containing different scales and different sub-region relationships. The Deeplab series of work introduces dilated convolution and spatial pyramid pooling (ASPP) to capture multi-scale context information; OCRNet groups pixels into category regions based on relevance, and splices the object context feature representation (OCR) with the network deep feature representation to obtain enhanced context information; Segmenter is based on Transformer (ViT) and extends it to semantic segmentation tasks for global context aggregation. There are some subsequent works, such as ISNet, which proposes to mine more valuable contextual information for each pixel from a semantic perspective by calculating context similarity in addition to the classic pixel-level aggregation scheme to further improve pixel representation.

[0004] However, due to the special imaging conditions of human brain SNAP and other imaging methods and the complex distribution of blood vessels, the above-mentioned semantic segmentation technical solutions based on deep learning all have insufficient or invalid context information aggregation, because the pixel relationship is modeled using a large number of predetermined priors. Therefore, in actual application scenarios, fixed semantic context priors will inevitably introduce noise while enhancing feature representation, resulting in the inability to achieve accurate medical image vascular segmentation. Summary of the invention

[0005] Existing semantic segmentation methods based on deep learning cannot achieve accurate medical image vascular segmentation. To this end, based on existing semantic segmentation methods based on deep learning, the present invention aims to solve the problem that fixed semantic-level context priors introduce noise, designs a screening mechanism to guide the construction of pixel relationships between semantic context and image feature representation, and improves the accuracy of semantic segmentation, thereby achieving accurate medical image vascular segmentation such as SANP.

[0006] To achieve the above object, the present invention provides a human brain image blood vessel segmentation method based on a semantic level context screening mechanism, comprising:

[0007] Given a medical image, obtaining pixel-level context features based on the medical image, and generating pseudo labels using the pixel-level context features;

[0008] Performing semantic-level context modeling on the pixel-level context features based on the pseudo-labels to obtain semantic-level context features, and optimizing and improving the semantic-level context features;

[0009] Constructing a semantic-level relationship matrix, and interacting the semantic-level context features and the pixel-level context features respectively through the semantic-level relationship matrix to obtain pixel relationships within the semantic context and between image feature representations;

[0010] The medical image is segmented based on the pixel relationship between the semantic context and the image feature representation to obtain a segmentation result.

[0011] Preferably, acquiring pixel-level context features based on the medical image includes:

[0012] The nonlinearity in the medical image is mapped to the feature space using a backbone network, and is adjusted to a matching dimension through a bottleneck layer to obtain the pixel-level context feature; wherein the bottleneck layer is a convolutional layer.

[0013] Preferably, the method for obtaining the pixel-level context feature is:

[0014]

[0015] Among them, R pis the pixel-level context feature; is the backbone network, used for preliminary feature extraction; Conv() is a convolutional layer, used to adjust the features to the appropriate dimension.

[0016] Preferably, the method for generating the pseudo-label is:

[0017]

[0018] In the formula, is the predicted probability of K categories; It is a pseudo label; is a preliminary prediction head; It consists of convolutional layers and activation layers, which are used to map features to high-dimensional space; argmax() is the maximum independent variable function.

[0019] Preferably, obtaining the semantic level context feature includes:

[0020]

[0021] In the formula, R sl is the semantic level context feature, N e is an image The number of categories present in ; is the weighted sum operation; R p,[*,i,j] is the feature of the (i, j)th pixel; Y c,[i,j] is the pseudo label prediction category of the (i, j)th pixel, k is the category, and K is the total number of categories.

[0022] Preferably, the method for optimizing and improving the semantic level context features is:

[0023] R ie,[k,*] =R ie,[k,*] ×(1-m)+R gt ×m

[0024] Where m is the update momentum, Represents the discriminant vector of each category in the training data set, R gt is the discriminant vector of each category obtained based on the true label, R ie,[k,*] is the discriminant vector of the kth class.

[0025] Preferably, the improved semantic level context feature is:

[0026]

[0027] In the formula, ⊕ represents the connection operation.

[0028] Preferably, the method for interacting the semantic level context features based on the semantic relationship matrix is:

[0029]

[0030] In the formula, represents matrix multiplication, is the semantic-level context representation after interaction enhancement, For the Convert to R sl The form of the shape, is the semantic level relationship matrix, R sl is the semantic level context feature, and dim is the normalized dimension.

[0031] Preferably, the method for interacting the pixel-level context features based on the semantic relationship matrix is:

[0032]

[0033] In the formula, k idx YesR sl Medium category k idx The corresponding matrix index, R sl is the semantic level context feature; is the feature representation initialized to a zero matrix; c,[i,j] is the pseudo label prediction category of the (i, j)th pixel; R a,[*,i,j] is a feature representation initialized to a zero matrix, and R is used according to the pseudo label of each pixel position. esl To fill in the context representation in; is the part of the semantic-level contextual features belonging to class k after interactive enhancement; k is the category; class ids in R sl For R sl All categories involved.

[0034] Preferably, obtaining the segmentation result includes:

[0035] The medical image is segmented based on the pixel relationship between the semantic context and the image feature representation to obtain a segmentation result, wherein the medical image is segmented using To enhance Make the final prediction, specifically:

[0036]

[0037] In the formula, is a convolutional layer, is a bilinear interpolation operation, P o is the final output class probability matrix for segmentation prediction, R pis the obtained pixel-level context feature, R a is the semantic-level context feature obtained after interaction enhancement, is matrix addition.

[0038] Compared with the prior art, the present invention has the following advantages and technical effects:

[0039] The present invention aims to solve the problem of inaccurate vascular segmentation of medical images such as human brain SNAP by using existing semantic segmentation methods based on deep learning. A context interaction module based on a screening mechanism is designed to guide the construction of pixel relationships within the semantic context and between the image feature representation, thereby reducing the impact of irrelevant noise on vascular segmentation and improving the accuracy of vascular segmentation in brain medical images. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0041] Figure 1 This is a flow chart of a method for segmenting blood vessels in human brain images based on a semantic-level context screening mechanism according to an embodiment of the present invention. DETAILED DESCRIPTION

[0042] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0043] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0044] The present invention proposes a human brain image blood vessel segmentation method based on a semantic level context filtering mechanism, such as Figure 1 Given a medical image, this method groups image-level context features into semantic-level context features according to the predicted pseudo-labels. Then, a relationship matrix is ​​constructed to make the semantic-level context and pixel-level context interact with each other, where the relationship matrix is ​​updated through a filtering mechanism. Finally, the semantic-level context features are used to guide the enhancement of the original pixel representation to achieve pixel-level accurate segmentation of medical image objects.

[0045] Specifically:

[0046] Given a medical image, pixel-level context features are obtained based on the medical image, and pseudo labels are generated using the pixel-level context features;

[0047] Specifically, in this embodiment, more than 20 medical images are taken, and the target area is manually classified and labeled pixel by pixel to form a data set for the subsequent semantic segmentation model. Among them, manual classification labeling refers to the data labeler determining the target category to which the pixel belongs based on the image content and labeling the corresponding label. For example, if the pixel belongs to category 1, it is labeled with label 1; if the pixel does not belong to any type of target, it is usually labeled as 0 (background).

[0048] Given an input image First, the backbone network is used to nonlinearly map I to the feature space and adjust it to the appropriate dimension through a bottleneck layer to obtain the pixel-level semantic feature representation R p :

[0049]

[0050] Among them, R p is the pixel-level context feature; is the backbone network, used for preliminary feature extraction; Conv() is a convolutional layer, used to adjust the features to the appropriate dimension.

[0051] R p The size is The number of channels is C. The bottleneck layer is implemented by a convolutional layer.

[0052] Input the obtained pixel-level features into the pseudo-label generation module:

[0053]

[0054] is the predicted probability of K categories; It is a pseudo label; is a preliminary prediction head; It consists of convolutional layers and activation layers, which are used to map features to high-dimensional space; argmax() is the maximum independent variable function. It projects pixel representation into probability space.

[0055] Based on pseudo labels, semantic-level context modeling is performed on pixel-level context features to obtain semantic-level context features, and the semantic-level context features are optimized and improved;

[0056] Specifically, since the screening mechanism of directly modeling pixel-level relationships is computationally infeasible, this embodiment proposes to obtain semantic context simplified relationship modeling by grouping pixel representations with the same pseudo-label. The obtained pseudo-labels are used to group pixel-level representations and model them as semantic context features:

[0057]

[0058] Among them, R sl is the semantic level context feature, N e is an image The number of categories present in ; is the weighted sum operation; R p，[*,i,j] is the feature of the (i, j)th pixel; Y c，[i，j] is the pseudo label prediction category of the (i, j)th pixel, k is the category, and K is the total number of categories.

[0059] Due to the inaccuracy of pseudo-labels, this method is designed to strengthen the aggregation module to assist the modeling process of semantic-level context, that is, to optimize the semantic-level context features. The improved semantic-level context modeling process is expressed as:

[0060]

[0061] Among them, ⊕ represents the connection operation, and Represents the discriminant vector of each category in the training dataset. This method is designed to build R by learning the data-level representation of each category. ie .

[0062] R ie The update strategy is:

[0063] R ie，[k,*] =R ie，[k,*] ×(1-m)+R gt ×m

[0064] Where m is an update momentum, which is set to 0.1 by default. gt is the discriminant vector of each category obtained based on the true label.

[0065] Before training, R i e is initialized to a zero matrix, and R i e is calculated as follows:

[0066] R gt ={Mean({R p[*，i，] |GT [i，j] =k}), k∈[0,K)}

[0067] Among them, GT represents the true segmentation mask and Mean(·) is used to average the extracted pixel representations with the same true label.

[0068] Constructing a semantic-level relationship matrix, and interacting the semantic-level context features and the pixel-level context features respectively through the semantic-level relationship matrix to obtain the pixel relationship between the semantic context and the image feature representation;

[0069] Specifically, after the improved semantic-level context modeling module, this method is designed to use a semantic-level relationship matrix make Interacting with the original feature representation to further filter and calibrate the context modeling, enhance the relationship between related contexts, and filter the obtained semantic level context features R sl The process can be expressed as:

[0070]

[0071] represents matrix multiplication, is the semantic-level context representation after interaction enhancement, Used to Convert to R sl The form of the shape is expressed as:

[0072]

[0073] Introduction To ensure There is no interaction between the two semantic-level representations with weaker relations in . t is used to identify the semantic-level representations with weaker relations. In practice, if is less than t, then in the calculation Set to negative infinity.

[0074] Finally rearrange R esl To the size of the original pixel representation, the obtained pixel-level context feature representation R p To enhance:

[0075]

[0076] Among them, k idx YesR sl Medium category k idx The corresponding matrix index, R sl is the semantic level context feature; is the feature representation initialized to a zero matrix; c,[i,j] is the pseudo label prediction category of the (i, j)th pixel; R a,[*,i,j] is a feature representation initialized to a zero matrix, and R is used according to the pseudo label of each pixel position. esl To fill in the context representation in; is the part of the semantic-level contextual features belonging to class k after interactive enhancement; k is the category; class ids in R sl For R sl All categories involved.

[0077] Segmenting the medical image based on the pixel relationship between the semantic context and the image feature representation to obtain a segmentation result, including:

[0078] use To enhance Make the final prediction, specifically:

[0079]

[0080] in, is a convolutional layer, is a bilinear interpolation operation, P o is the final output class probability matrix for segmentation prediction, R p is the obtained pixel-level context feature, R a is the semantic-level context feature obtained after interaction enhancement, is matrix addition.

[0081] This method aims to solve the problem of inaccurate vascular segmentation in medical images such as human brain SNAP by using existing semantic segmentation methods based on deep learning. It designs a context interaction module based on a screening mechanism to guide the construction of pixel relationships within the semantic context and between the image feature representation, reduce the impact of irrelevant noise on vascular segmentation, and improve the accuracy of vascular segmentation in brain medical images.

[0082] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A human brain image blood vessel segmentation method based on a semantic level context filtering mechanism, characterized in that: include: Given a medical image, obtaining pixel-level context features based on the medical image, and generating pseudo labels using the pixel-level context features; Performing semantic-level context modeling on the pixel-level context features based on the pseudo-labels to obtain semantic-level context features, and optimizing and improving the semantic-level context features; Constructing a semantic-level relationship matrix, and interacting the semantic-level context features and the pixel-level context features respectively through the semantic-level relationship matrix to obtain pixel relationships within the semantic context and between image feature representations; The medical image is segmented based on the pixel relationship between the semantic context and the image feature representation to obtain a segmentation result.

2. The method for human brain image blood vessel segmentation based on semantic level context screening mechanism according to claim 1, characterized in that: Acquiring pixel-level context features based on the medical image includes: The nonlinearity in the medical image is mapped to the feature space using a backbone network, and is adjusted to a matching dimension through a bottleneck layer to obtain the pixel-level context feature; wherein the bottleneck layer is a convolutional layer.

3. The method for human brain image blood vessel segmentation based on semantic level context screening mechanism according to claim 2, characterized in that: The method for obtaining the pixel-level context feature is: Among them, R p is the pixel-level context feature; is the backbone network, used for preliminary feature extraction; Conv() is a convolutional layer, used to adjust the features to the appropriate dimension.

4. The method for human brain image blood vessel segmentation based on semantic level context screening mechanism according to claim 1, characterized in that: The method for generating the pseudo label is: In the formula, is the predicted probability of K categories; It is a pseudo label; is a preliminary prediction head; It consists of convolutional layers and activation layers, which are used to map features to high-dimensional space; argmax() is the maximum independent variable function.

5. The method for human brain image blood vessel segmentation based on semantic level context screening mechanism according to claim 1, characterized in that: Obtaining the semantic level context feature includes: In the formula, R sl is the semantic level context feature, N e is an image The number of categories present in ; is the weighted sum operation; R p,[*,i,j] is the feature of the (i, j)th pixel; Y c,[i,j] is the pseudo label prediction category of the (i, j)th pixel, k is the category, and K is the total number of categories.

6. The method for human brain image blood vessel segmentation based on semantic level context screening mechanism according to claim 5, characterized in that: The method for optimizing and improving the semantic level context features is: R ie,[k,*] =R ie,[k,*] ×(1-m)+R gt ×m Where m is the update momentum, Represents the discriminant vector of each category in the training data set, R gt is the discriminant vector of each category obtained based on the true label, R ie,[k,*] is the discriminant vector of the kth class.

7. The method for human brain image blood vessel segmentation based on semantic level context screening mechanism according to claim 6, characterized in that: The improved semantic-level context features are: In the formula, Represents a join operation.

8. The method for human brain image blood vessel segmentation based on semantic level context screening mechanism according to claim 1, characterized in that: The method for interacting the semantic level context features based on the semantic relationship matrix is: In the formula, represents matrix multiplication, is the semantic-level context representation after interaction enhancement, For the Convert to R sl The form of shape, is the semantic level relationship matrix, R sl is the semantic level context feature, and dim is the normalized dimension.

9. The method for human brain image blood vessel segmentation based on semantic level context screening mechanism according to claim 8, characterized in that: The method for interacting the pixel-level context features based on the semantic relationship matrix is: In the formula, k idx YesR sl Medium category k idx The corresponding matrix index of R sl is the semantic level context feature; is the feature representation initialized to a zero matrix; Y c,[i,j] is the pseudo label prediction category of the (i, j)th pixel; R a,[*,i,j] is a feature representation initialized to a zero matrix, and R is used according to the pseudo label of each pixel position. esl To fill in the context representation in; is the part of the semantic-level contextual features belonging to class k after interactive enhancement; k is the category; class ids in R sl For R sl All categories involved.

10. The method for human brain image blood vessel segmentation based on semantic level context screening mechanism according to claim 9, characterized in that: Obtaining the segmentation result includes: The medical image is segmented based on the pixel relationship between the semantic context and the image feature representation to obtain a segmentation result, wherein the medical image is segmented using To enhance Make the final prediction, specifically: In the formula, is a convolutional layer, is a bilinear interpolation operation, P o is the final output class probability matrix for segmentation prediction, R p is the obtained pixel-level context feature, R a is the semantic-level context feature obtained after interaction enhancement, is matrix addition.

Images