Optical coherence tomography blood vessel segmentation method based on priori guidance local geometry modeling

Abstract

The present disclosure provides an optical coherence tomography blood vessel segmentation method based on prior guidance local geometry modeling. The method uses a residual quantization variational autoencoder (RQ-VAE) to model the local geometry of the blood vessels, and the residual quantized discrete vector is stored in the codebook as the blood vessel mask prior feature; then a segmentation network is trained, including coarse alignment and fine alignment; first, the mask feature and the image feature are projected into the principal component space, and the coarse feature is aligned by minimizing the principal component alignment loss, so as to alleviate the overfitting problem caused by the domain difference between the mask and the original image. The fine alignment operation aligns the local geometric structure of the mask feature and the replaced latent feature through a local linear mapping operation. Using the present application can reduce the anatomical error of the blood vessels.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image analysis, and particularly relates to an optical coherence tomography blood vessel segmentation method based on priori guided local geometry modeling. BACKGROUND

[0002] Optical coherence tomography (OCT) is a fast, non-invasive imaging technique that can provide high-resolution imaging of the anterior segment, retina and choroid. At present, the OCT-based ophthalmic disease diagnosis and treatment method has important clinical value, and most of these methods rely on the quantification of eye anatomical structures by OCT / OCTA images. Among them, as one of the main anatomical structures of the eye, the blood vessel structure has been identified as an important indicator for evaluating the severity of ophthalmic diseases and providing clues for understanding the progress of related diseases. Optical coherence tomography angiography (OCTA) imaging can be used for various clinical applications, from the diagnosis of corneal pathology to the monitoring of treatment, and the quantification of retinal microvessels can evaluate pathologies such as diabetic retinopathy, glaucoma and retinal artery obstruction; choroidal vessel index is the main marker of choroid, which is helpful for the diagnosis and treatment of age-related macular degeneration and choroidal neovascularization. Therefore, automatic blood vessel segmentation in OCT / OCTA images has become a key to disease diagnosis and treatment planning.

[0003] In the past decade, some deep learning based methods have been proposed for OCT / OCTA image vessel segmentation. Zhang et al. proposed a knowledge infused global-local network for choroid layer and choroid vessel segmentation (see H. Zhang, J. Yang, K. Zhou, F. Li, Y. Hu, Y. Zhao, C. Zheng, X. Zhang, and J. Liu, “Automatic segmentation and visualization of choroid in oct with knowledge infused deep learning,” IEEE Journal of Biomedical and Health Informatics, vol. 24, no. 12, pp. 3408-3420, 2020.) and studied the correlation between glaucoma and choroid vessel changes. Zhu et al. proposed a choroid U-net for layer and vessel segmentation (see L. Zhu, J. Li, R. Zhu, X. Meng, P. Rong, Y. Zhang, Z. Jiang, M. Geng, B. Qiu, X. Rong et al., “Synergistically segmenting choroidal layer and vessel using deep learning for choroid structure analysis,” Physics in Medicine & Biology, vol. 67, no. 8, p. 085001, 2022.). The proposed network first extracts the choroid layer, and then segments the choroid vessels according to the relevant features of the layer structure and vessels. For OCTA vessel segmentation, Mou et al. proposed a channel and spatial attention network for curvilinear structure segmentation (see L. Mou, Y. Zhao, L. Chen, J. Cheng, Z. Gu, H. Hao et al., “Csnet: Channel and spatial attention network for curvilinear structure segmentation,” in Proc. Int. Conf. Med. Image Comput. Comput. Assist. Intervent. (MICCAI), 2019, pp. 721-730.), which achieved good results in the task of vessel structure extraction on OCTA images; Li et al. proposed a 3D-2D image projection network (see Y. Ma, H. Hao, J. Xie, H. Fu, J. Zhang, J.Yang, Z. Wang, J. Liu, Y. Zheng, and Y. Zhao, “Rose: A retinal oct-angiography vessel segmentation dataset and new model,” IEEE transactions on medical imaging, vol. 40, no. 3, pp. 928-939, 2021.), which can extract 2D vessel projection maps from 3D OCTA volume data; Ma et al. proposed a coarse-to-fine network based on split attention mechanism (see M. Li, Y. Chen, Z. Ji, K. Xie, S. Yuan, Q. Chen, and S. Li, “Image projection network: 3d to 2d image segmentation in octa images,” IEEE Transactions on Medical Imaging, vol. 39, no. 11, pp. 3343-3354, 2020.), for retinal vessel segmentation in OCTA images; Hao et al. proposed a voting mechanism-based key structure extraction algorithm for OCTA images (see J. Hao, T. Shen, X. Zhu, Y. Liu, A. Behera, D. Zhang, B. Chen, J. Liu, J. Zhang, and Y. Zhao, “Retinal structure detection in octa image via voting-based multitask learning,” IEEE Transactions on Medical Imaging, vol. 41, no. 12, pp. 3969-3980, 2022.), which realizes the detection of blood vessels, fovea, and key points. Chen et al. proposed a multi-task joint optimization framework for choroidal neovascularization segmentation (see T. Chen, Y. Zhao, L. Mou, D. Zhang, X. Xu, M. Liu, H. Fu, and J. Zhang, “Rbgnet: Reliable boundary-guided segmentation of choroidal neovascularization,” in International Conference on Medical Image Computing.), which combines a dual-stream encoding structure and an uncertainty estimation strategy to enhance the network’s ability to handle ambiguous boundaries.

[0004] Although learning-based vessel segmentation methods have achieved good performance, these models are pixel-level classification, ignoring the inherent morphological features (geometry and shape) of blood vessels. This paradigm can lead to unreasonable vessel mask structures for the network, which brings great challenges to the quantification of related parameters and is not suitable for clinical applications. Previous medical image segmentation tasks partially utilized the prior knowledge of segmentation masks. For example, Zotti et al. first designed a quality regression module to capture the mask shape prior, and then combined the average prior with the CNN model to obtain more accurate anatomical mask results (see C. Zotti, Z. Luo, O. Humbert, A. Lalande, and P.-M. Jodoin, “Gridnet with automatic shape prior registration for automatic mri cardiac segmentation,” in Statistical Atlases and Computational Models of the Heart. ACDC and MMWHS Challenges: 8th International Workshop, STACOM 2017, Held in Conjunction with MICCAI 2017, Quebec City, Canada, September 10-14, 2017, Revised Selected Papers 8. Springer, 2018, pp. 73-81.). This method only encodes the mask shape simply, but it may be difficult to represent complex mask shapes. Therefore, some recent researchers have proposed deep networks for encoding mask prior features. Oktay et al. proposed a new general training strategy for cardiac image segmentation (see O. Oktay, E. Ferrante, K. Kamnitsas, M. Heinrich, W. Bai, J. Caballero, S. A. Cook, A. De Marvao, T. Dawes, D. P. O’Regan et al., “Anatomically constrained neural networks (ACNNs): application to cardiac image enhancement and segmentation,” IEEE transactions on medical imaging, vol. 37, no. 2, pp. 384-395, 2017.).The model aligns the input image latent features with the anatomical prior latent features using a regularization model, prompting the model to learn global anatomical properties. Painchaud et al. introduced a VAE-based anatomically guaranteed cardiac image segmentation network (see N. Painchaud, Y. Skandarani, T. Judge, O. Bernard, A. Lalande, and P.-M. Jodoin, “Cardiac segmentation with strong anatomical guarantees,” IEEE transactions on medical imaging, vol. 39, no. 11, pp. 3703-3713, 2020.) by introducing a mask shape distribution into the post-processing network to correct unreliable predicted masks. On this basis, Judge et al. proposed a medical image segmentation framework for uncertainty prediction, which alleviates the problem of mismatch between predicted mask features and prior features by modeling the joint latent representation of mask shape and input image (see T. Judge, O. Bernard, M. Porumb, A. Chartsias, A. Beqiri, and P.-M. Jodoin, “Crisp-reliable uncertainty estimation for medical image segmentation,” in International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2022, pp. 492-502.).

[0005] Previous medical image segmentation tasks partially utilize the prior knowledge of segmentation masks. These methods can effectively encode mask priors, regularize the network, and thus improve the accuracy and reliability of the network's predicted masks. However, few works use mask modeling methods for blood vessel segmentation. There are mainly two challenges:

[0006] First, the above works use autoencoders (AE) and variational autoencoders (VAE) to model mask priors and output latent continuous vectors. These frameworks tend to encode the overall prior distribution of the mask. In this case, they are difficult to represent the local complex structure of microvessels, which will lead to the failure of prior regularization constraints for local blood vessel segmentation results. Therefore, how to realize the local efficient modeling of microvessel masks is the first challenge to be solved.

[0007] Second, the low signal-to-noise ratio in blood vessels leads to domain discrepancies and local errors between vessel mask features and image features. Strong distance constraints between features can cause network overfitting. Therefore, how to fully utilize the prior features of the vessel mask and effectively interact with image features is the second challenge. Summary of the Invention

[0008] In view of this, the present invention provides a method for segmenting blood vessels by optical coherence tomography based on prior-guided local geometric modeling, which can reduce anatomical errors of blood vessels.

[0009] To solve the above-mentioned technical problems, the present invention is implemented as follows.

[0010] A method for optical coherence tomography vessel segmentation based on guided local geometry modeling includes:

[0011] Step 1: Local modeling of blood vessels: The local geometry of blood vessels is modeled using a residual quantization variational autoencoder (RQ-VAE); the discrete vectors after residual quantization (RQ) are stored in the codebook as prior features of the blood vessel mask; wherein, the RQ-VAE includes a first encoder, a first decoder and a codebook, and the first encoder and the second encoder are trained through the local geometry modeling process;

[0012] Step 2: Training the blood vessel segmentation network:

[0013] The blood vessel segmentation network includes an alignment network and a segmentation network; the alignment network uses the first encoder and first decoder trained in step 1; the segmentation network includes a segmentation encoder and a segmentation decoder.

[0014] During network training, a standard mask is input to the first encoder to obtain mask features, and the image to be segmented is input to the segmentation encoder to obtain image features. The mask features and image features are projected onto the principal component space, and coarse feature alignment is performed with the goal of minimizing the principal component alignment loss.

[0015] The quantized mask features are obtained by performing residual quantization (RQ) on both the mask features and the image features. and quantized image features Projection is performed in a low-dimensional space constructed based on the codebook to obtain discrete mask feature projection P. (n) and discrete image feature projection Q (n) Projecting discrete mask features onto P (n) Each vector (i,j) is represented by a local linear combination of its adjacent vectors, denoted as And determine the combination coefficients that minimize the local linear expression error, and use these combination coefficients. Constructing image feature projection Q (n) Local linear representation of each vector Project the mask features P(n) Local linear representation of image feature projection The error is used as the local linear alignment loss, and fine feature alignment is performed with the goal of minimizing the local linear alignment loss.

[0016] Step 3: Use the trained segmentation network to perform optical coherence tomography (OCT) segmentation of blood vessels.

[0017] Preferably, a contrastive loss is added to the loss function of RQ-VAE training, and the contrastive loss is used to perform contrastive learning to improve the separation of different discrete vectors;

[0018] Construct a cosine similarity matrix A representing the similarity of discrete vectors:

[0019]

[0020] in, This represents matrix multiplication; b represents the codebook, consisting of K C-dimensional discrete vectors b1 to b2. K After normalization, the cosine similarity matrix A consists of any off-diagonal element A. ij This represents the cosine similarity between the i-th and j-th discrete vectors;

[0021] During the training process in step 1, the cosine similarity matrix A is made close to an identity matrix to minimize the similarity between different discrete vectors.

[0022] Preferably, the loss function for training RQ-VAE further includes a hybrid loss function. for:

[0023]

[0024] In equation (2) above, M is the encoder E rec The input standard mask, For decoder D rec The output reconstructed mask; β = 0.25 is the multiplication factor coefficient; F rec For encoder E rec The mask feature map output by encoding the standard mask; D is the mask feature map F. rec The stacking depth represents the encoder E. rec Output mask feature map F rec It is composed of D discrete vectors stacked from the codebook; sg[·] represents the stopping gradient; Represents the composition of the mask feature map F rec The i-th of the D discrete eigenvectors.

[0025] Preferably, when modeling with RQ-VAE in step 1, a U-Net without skip connections is used as the backbone network, and the encoder is replaced with ResNet.

[0026] Preferably, the step of projecting mask features and image features into the principal component space, with the goal of minimizing principal component alignment loss, and performing coarse feature alignment is as follows:

[0027] The K C-dimensional discrete vectors b1~b in the codebook K Arranged into a matrix, represented as X∈R C×K The projection matrix is ​​defined as W∈R N×C ,N≤C; Through matrix factorization, XX T ∈R C×C Represented as ΣΛΣ T , where Λ∈R C×C It is a diagonal matrix whose diagonal elements are arranged in descending order of eigenvalues; Σ∈R C×C It is an orthogonal matrix; each column vector of Σ is XX. T An orthogonal eigenvector;

[0028] Select Σ T The first N rows are used as the projection matrix W to project the mask features and image features into the principal component space;

[0029] Principal component alignment loss Represented as:

[0030]

[0031] In equation (3) above, F rec For the mask features output by the first encoder, F seg To segment the image features output by the encoder;

[0032] Principal component alignment loss With minimization as the objective, coarse feature alignment is performed through network training.

[0033] Preferably, the specific process of fine feature alignment is as follows:

[0034] Σ T The rows are divided into S submatrices W1, ..., W2. S And each submatrix is ​​a C / S×C projection matrix; then the quantization mask features and quantized image features The projection in the low-dimensional space is P. (n) and Q (n) , represented as:

[0035]

[0036] For P (n) Any 3x3 region centered at (i,j) in P (n) Each vector in P is expressed by a linear combination of 8 neighboring vectors; the 8 neighboring vectors are arranged in a matrix The weight coefficients are defined as Then The local linear expression of P

[0037] Solve the weight coefficients that minimize the local linear expression error; the solving objective is:

[0038]

[0039] For each projection Q of the quantized image feature (n) , define as the neighborhood matrix of any vector; the local linear expression of Q (n) is defined as

[0040] The local linear alignment loss L LLA is defined as:

[0041]

[0042] Where h x w is the dimension of each vector in P (n) and Q (n) ;

[0043] Through network training, realize fine alignment of features by minimizing the local linear alignment loss L LLA .

[0044] Advantages:

[0045] (1) The application proposes a discrete prior guided local linear alignment framework, and effective discrete vectors are introduced in the pre-trained codebook to replace the defective latent feature map extracted by the segmentation encoder, which can improve the effectiveness of the local shape representation of blood vessels.

[0046] (2) The application uses the RQ-VAE model to quantify and store the prior features of the blood vessel mask, and realizes efficient modeling of the local structure of the mask.

[0047] (3) During the training of the segmentation network, a new image latent feature and prior feature interaction framework is constructed, and a combination of coarse alignment and local fine alignment is adopted to ensure that the mask prior and the latent feature are locally accurately aligned, and the network's ability to capture microvessels is enhanced.

[0048] (4) The coarse alignment stage, by performing coarse matching on the mask features and the principal component features of the image features, the principal component alignment loss can make the network directly focus on the high-dimensional representation of the main features of the blood vessels, and ignore some noise and details, preventing overfitting caused by forced matching of image features and prior features.

[0049] (5) The local fine alignment stage, considering that the geometric structure of the blood vessels exhibits strong local continuity, the application proposes a fine designed fine alignment loss, which performs local geometric structure alignment on the mask features and the replaced latent features through local linear mapping.

[0050] (6) In a preferred embodiment, a contrast loss is added during the RQ-VAE training process to ensure the sparsity of the discrete vectors in the codebook. Vector density may cause the wrong application of similar vectors, and the addition of the contrast loss can reduce the influence of vector density on the segmentation accuracy.

[0051] The proposed framework provides an effective tool for evaluating and quantifying structural changes in ophthalmic blood vessels, assisting clinicians in early screening and treatment of blood vessel-related diseases. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 The first stage blood vessel mask modeling framework.

[0053] Figure 2 The second stage image latent feature and mask feature interaction framework. DETAILED DESCRIPTION

[0054] The application will be described in detail below with reference to the accompanying drawings and examples.

[0055] The application provides an optical coherence tomography blood vessel segmentation scheme based on prior guided local geometric modeling, which includes two stages. In the first stage, the local modeling of the blood vessels is performed, and the residual quantization variational autoencoder (RQ-VAE) is used to model the local geometric shape of the blood vessels. The discrete vector after residual quantization RQ is stored in the codebook as the blood vessel mask prior feature. In the second stage, the blood vessel segmentation network is trained, and the application proposes a new OCT / OCTA blood vessel segmentation framework. The framework can include the discrete vector stored in the first stage into any U-shaped network, and reduce the anatomical error of the blood vessels by using the local shape prior of the blood vessels.

[0056] In the second stage, this invention proposes two novel distance-constrained losses based on the characteristics of blood vessel segmentation. First, although there are subtle differences between image features and mask features, the local blood vessel shapes they represent should have roughly similar structures. Therefore, this invention proposes Principal Component Alignment (PCA) loss, which maps image features and mask features to the principal component space for coarse alignment. On the other hand, this invention aims to ensure that the new feature map after residual quantization accurately represents the local geometry of the blood vessel. Simultaneously, the high-dimensional image features projected onto a specific low-dimensional space should preserve local geometric correlations. Therefore, this invention proposes a local linear registration loss for fine registration of the residual-quantized image features and mask features.

[0057] The specific implementation process of the present invention will be described in detail below.

[0058] Step 1: Local geometric modeling of blood vessels.

[0059] In the first stage, this invention uses RQ-VAE to model the local geometry of blood vessels. The RQ-VAE network consists of encoder E rec Decoder D rec It consists of a codebook B. The codebook contains K C-dimensional discrete vectors b. i ∈R C , i∈[K]. For example... Figure 1 As shown, similar to an autoencoder, the gold standard for input, i.e., the standard mask definition M∈R, is... H×W×1 After passing through encoder E rec After processing, it is encoded into a continuous feature map, called the mask feature F. rec ∈R h×w×C Each C-dimensional vector From the D discrete vectors in the codebook Stacked representation.

[0060] The selection of each discrete vector is as follows:

[0061]

[0062] The above operation is called residual quantization (RQ). Let D be the zero vector and D be the depth.

[0063] When residual quantization RQ is used in F rec When all h×w vectors are in the vector, this invention can obtain D discrete feature maps. F rec The discrete representation is as follows

[0064]

[0065] Ultimately, the decoder will As input to obtain the reconstruction result

[0066] The training of the conventional RQ-VAE relies on a hybrid loss function:

[0067]

[0068] Where sg[·] represents the stop gradient, and β is the multiplication factor coefficient. Currently, vector quantization has been widely applied to image generation.

[0069] Traditional methods usually impose no restrictions on the discrete vectors in the codebook. However, in medical image segmentation, images usually come with a unique mask. Therefore, the density of vectors in the codebook can lead to the erroneous application of similar vectors, thereby affecting the accuracy of segmentation. Therefore, the present application utilizes contrastive learning to improve the separation between different discrete vectors and ensure the sparsity of the code set. As shown in Figure 1 The distance between any two discrete vectors is measured by cosine similarity. The calculation formula of the cosine similarity matrix A is:

[0070]

[0071] Where, represents matrix multiplication, and b represents the codebook, which is a CxK matrix. Each column of b is a normalized discrete vector. The diagonal elements of matrix A are 1, and any non-diagonal element A ij represents the cosine similarity between the ith and jth discrete vectors.

[0072] The goal of the present application is to minimize the similarity between different discrete vectors, so that matrix A approaches a unit matrix. The present application uses cross-entropy to achieve this contrastive loss:

[0073]

[0074] Finally, the reconstruction of the mask is trained by combining the hybrid loss and the contrastive loss.

[0075] The present application uses a pass-through estimator to solve the backpropagation problem in the RQ operation.

[0076] Step 2: Vessel segmentation network training

[0077] Most segmentation methods use U-shaped networks for medical image segmentation, and the feature extraction capability of the encoder has a great influence on the performance of the network. However, OCTA / OCT images may have problems such as unclear blood vessels, artifacts, noise and horizontal stripes. This makes it difficult for existing methods to extract effective latent feature maps to accurately represent the local shape of blood vessels. To this end, the present application introduces effective discrete vectors from a pre-trained codebook to replace the defective latent feature maps extracted by the segmentation encoder.

[0078] However, there is a domain shift between the image and the manual annotation, so when the pre-trained code set is applied to the image features, it may lead to incorrect interactions. In previous studies on mask preposition, researchers have used Euclidean distance and cosine similarity to make image features closer to mask features. However, due to the domain gap and local structure error between mask features and image features, forcibly aligning the two different features may lead to overfitting. leading to overfitting. In addition, the introduction of discrete vectors may cause the network to ignore the local geometric relationship of blood vessels.

[0079] To solve the above problems, as shown in Figure 2 , the present application proposes two new distance constraint losses according to the characteristics of blood vessel segmentation.

[0080] First, see Figure 2 for an introduction to the structure of the blood vessel segmentation network. The blood vessel segmentation network includes an alignment network and a segmentation network. The alignment network uses the first encoder and the first decoder trained in step 1, see Figure 2 the upper half of the blue encoder and decoder in the middle; the segmentation network includes a segmentation encoder and a segmentation decoder, see Figure 2 the lower half of the green encoder and decoder in the middle. The alignment network is used for training, and the segmentation network is used for blood vessel segmentation.

[0081] During network training, the features output by the first encoder and the segmentation encoder are used for coarse alignment, and the features after residual quantization RQ are used for fine alignment.

[0082] Among them, for the coarse alignment part: the standard mask is input into the first encoder to obtain the mask feature F rec , and the image to be segmented is input into the segmentation encoder to obtain the image feature F seg . Although there are differences in details between image features and mask features, the local blood vessel shapes they represent should have roughly similar structures. Therefore, the present application proposes a principal component alignment (PCA) loss to map image features and mask features to a principal component space for coarse alignment.

[0083] For the fine alignment part: after the mask feature and the image feature are subjected to residual quantization RQ respectively, the quantized mask feature and the quantized image feature The new feature map of residual quantization can accurately represent the local geometry of the blood vessels. At the same time, the high-dimensional image features projected into a specific low-dimensional space should retain the local geometric correlation. Therefore, the present application proposes a local linear registration loss for fine registration of the residual quantized image features and the mask features.

[0084] (1) Coarse alignment based on principal component alignment loss

[0085] Through mask reconstruction, the latent space can effectively encode the intrinsic data structure of the local geometry of the blood vessels. In order to capture the principal components of the local blood vessel structure, it is crucial to analyze each latent vector Z rec in the mask feature map F i,j . Principal component analysis is a solution. Specifically, the present application defines the latent space as In each batch of training, the pre-trained mask encoder is fixed, so the latent vector Z i,j can be understood as a sample of X. Therefore, the most direct method to obtain the principal component space is to perform principal component analysis on X. However, the samples in each batch are insufficient to represent the entire latent space. Each vector in the latent space can be linearly represented by D discrete vectors in the codebook. Therefore, the cardinality of the discrete vectors spans the entire latent space and can replace the latent space as the object of analysis. The present application arranges the K discrete vectors in the codebook into a matrix (x1, x2, …, x K ), denoted as X∈R C×K . The projection matrix is defined as W∈R N×C , N≤C, and the projection Z∈R N×K can be represented as WX. When the present application performs zero-centering on the original data X, the covariance matrix of the projection is as follows:

[0086]

[0087] C Z The diagonal elements represent the variance, indicating the importance of different dimensions. The elements outside the diagonal represent the covariance, indicating the correlation between different dimensions. The present application hopes that the projection in the low-dimensional space can capture the principal components of the original samples while ensuring the orthogonality between the components of different dimensions. Therefore, the goal of the present application is to find a projection matrix that makes C Z a diagonal matrix while maximizing its diagonal elements. Through matrix decomposition, XX T ∈R C×C can be represented as ΣΛΣ T , where Λ∈R C×C is a diagonal matrix whose diagonal elements are arranged in descending order of eigenvalues. Σ∈R C×Cis an orthogonal matrix. Each column vector of Σ is an orthogonal eigenvector of XX T . Selecting different column vectors of Σ as row vectors of W can make C Z a diagonal matrix. The selection of orthogonal vectors determines the size of C Z and the diagonal elements. Therefore, the present application selects the first N rows of Σ T (corresponding to the first N largest eigenvalues of XX T ) as the projection matrix W. The principal component alignment loss can be expressed as:

[0088]

[0089] W T represents the feature reconstruction matrix in the high-dimensional space, which can use a new basis to represent the features. The principal component alignment loss can make the network focus on the high-dimensional representation of the main features of the blood vessels, while ignoring some noise and details.

[0090] With the objective of minimizing the principal component alignment loss , the feature coarse alignment is performed through network training.

[0091] (2) Fine alignment based on local linear alignment loss

[0092] After residual quantization, the goal of the present application is to introduce discrete vectors to make up for the lack of image feature details. Therefore, it is necessary to consider both the principal components and the detail components. On the other hand, the geometric structure of blood vessels exhibits strong local continuity. This makes it difficult for existing methods to capture the local geometric relationship of blood vessels. The introduction of discrete vectors may further destroy the local geometric correlation in the latent space. Local linear embedding can reflect the local geometric relationship of the manifold in the embedding space according to the estimation of adjacent weights. However, the local vectors in the latent space are usually non-dense, so it is challenging to establish a linear correlation between these vectors in the original dimension. Some high-dimensional data may exhibit meaningful local structure in low-dimensional space. For this purpose, the present application proposes to establish the local geometric correlation of blood vessels between the projections of different low-dimensional spaces.

[0093] The present application divides the rows of Σ T into S sub-matrices W1,..., WS S , and each sub-matrix is a C / S x C projection matrix. Since the diagonal elements of Λ are arranged in descending order of eigenvalues, the former sub-matrices can extract principal components, while the latter sub-matrices focus on detail components. After residual quantization, the projections P(n), Q(n) ∈ R h×w×C / S of the mask feature and the image feature in the low-dimensional space can be expressed as:

[0094]

[0095] As shown in Figure 2 , for any 3x3 region centered at (i,j), each vector of the projected mask feature can be approximated by a linear combination of its neighboring vectors. The 8 neighboring vectors are arranged into a matrix When the combination coefficients are defined as the local linear representation is denoted as The goal of the present invention is to find the coefficients that minimize the error:

[0096]

[0097] The solution of the coefficients can be translated into:

[0098]

[0099] where 1 in the above equation represents the identity matrix;

[0100] is defined as:

[0101]

[0102] By the Lagrange method, the present invention obtains the optimal solution:

[0103]

[0104] The present invention hopes that this local linear relationship can be preserved in the image features. For each projection Q (n) of the image feature , the present invention defines as the neighborhood matrix of any vector. The local linear representation is defined as The local linear alignment loss is defined as:

[0105]

[0106] Therefore, through different subspaces, L LLA can gradually establish the local geometric connection of the blood vessels from coarse to fine. With the goal of minimizing the local linear alignment loss L LLA , the feature fine alignment is achieved through network training.

[0107] The present invention performs joint network training based on the principal component alignment loss and the local linear alignment loss L LLA , achieves coarse alignment and fine alignment, and thus reduces the anatomical error of the blood vessels.

[0108] In the network framework design, for the first stage, the present application uses RQVAE to reconstruct the blood vessel mask. In a preferred embodiment, a U-Net without skip connection is used as the backbone network, in which the encoder E rec is replaced by ResNet. In order to standardize the framework, the resolution (h, w) of the latent space is designed as (H / 16, W / 16), and the dimension C of the latent space is set as 512. The codebook is configured as follows, the depth D is set as 4, and the number K of discrete vectors is set as 2048. The present application uses a combination of hybrid loss and contrast loss to train the network. The multiplication factor β in the hybrid is set as 0.25.

[0109] In the second stage, the pre-trained mask encoder and codebook in the first stage will be fixed to guide the training of the segmentation network. Any segmentation network can be embedded into the segmentation framework of the present application. As long as the resolution of the latent space meets the following standard (H / 16, W / 16). The present application uses the same encoder and decoder as in the first stage to compose the initial segmentation network. As shown in Figure 2 the present application replaces the original image features with residual quantization representation as the input of the segmentation decoder. The present application uses a pass-through estimator to ensure the gradient backpropagation in the above process.

[0110] Step 3: Use the trained segmentation network to perform optical coherence tomography blood vessel segmentation.

[0111] The above specific embodiments only describe the design principles of the present application, and the shapes and names of the components in the description can be different and are not limited. Therefore, those skilled in the art can modify or equivalently replace the technical solutions described in the foregoing embodiments; and these modifications and replacements do not deviate from the purpose and technical solutions of the present application, and should all belong to the protection scope of the present application.

Claims

1. A method for optical coherence tomography vessel segmentation based on prior-guided local geometric modeling, characterized in that, include: Step 1: Local modeling of blood vessels: The local geometry of blood vessels is modeled using a residual quantization variational autoencoder (RQ-VAE); the discrete vectors after residual quantization (RQ) are stored in the codebook as prior features of the blood vessel mask; wherein, the RQ-VAE includes a first encoder, a first decoder and a codebook, and the first encoder and the second encoder are trained through the local geometry modeling process; Step 2: Training the blood vessel segmentation network: The blood vessel segmentation network includes an alignment network and a segmentation network; the alignment network uses the first encoder and first decoder trained in step 1; the segmentation network includes a segmentation encoder and a segmentation decoder. During network training, a standard mask is input to the first encoder to obtain mask features, and the image to be segmented is input to the segmentation encoder to obtain image features. The mask features and image features are projected onto the principal component space, and coarse feature alignment is performed with the goal of minimizing the principal component alignment loss. The coarse alignment process is as follows: K C-dimensional discrete vectors in the codebook b 1~ b K Arranged into a matrix, represented as The projection matrix is ​​defined as follows: N≤C; through matrix decomposition, Represented as ,in It is a diagonal matrix whose diagonal elements are arranged in descending order of eigenvalues; It is an orthogonal matrix; each column vector of Σ is An orthogonal eigenvector; choose The first N rows are used as the projection matrix W to project the mask features and image features into the principal component space; Principal component alignment loss Represented as: （3） In the above formula (3), The mask features are the output of the first encoder. To segment the image features output by the encoder; Principal component alignment loss With minimization as the objective, coarse feature alignment is performed through network training; The quantized mask features are obtained by performing residual quantization (RQ) on both the mask features and the image features. and quantized image features Projection is performed in a low-dimensional space constructed based on the codebook to obtain discrete mask feature projections. and discrete image feature projection Projecting discrete mask features Each vector ( i , j It can be expressed as a local linear combination of adjacent vectors, denoted as And determine the combination coefficients that minimize the local linear expression error, and use these combination coefficients. Constructing image feature projection Local linear representation of each vector Projecting mask features Local linear representation of image feature projection The error is used as the local linear alignment loss, and fine feature alignment is performed with the goal of minimizing the local linear alignment loss. Step 3: Use the trained segmentation network to perform optical coherence tomography (OCT) segmentation of blood vessels.

2. The method as described in claim 1, characterized in that, Add contrastive loss to the loss function of RQ-VAE training, and use contrastive loss to perform contrastive learning to improve the separation of different discrete vectors; Construct a cosine similarity matrix A representing the similarity of discrete vectors: （1） in, Represents matrix multiplication; b The codebook is represented by... K C-dimensional discrete vectors b 1~ b K After normalization, it is composed of any off-diagonal elements of the cosine similarity matrix A. Indicates the first i The and the first j Cosine similarity between discrete vectors; During the training process in step 1, the cosine similarity matrix A is made close to an identity matrix to minimize the similarity between different discrete vectors.

3. The method as described in claim 2, characterized in that, The loss function for training RQ-VAE further includes a hybrid loss function. for: （2） In the above formula (2), M For encoder The input standard mask, For decoder The output reconstructed mask; These are the multiplication factor coefficients; For encoder The mask feature map output by encoding the standard mask; D Mask feature map The stacking depth represents the encoder. Output mask feature map From the codebook D It is composed of stacked discrete vectors; sg[ [] indicates stopping the gradient; Represents the composition of the mask feature map of D The th discrete eigenvector i indivual.

4. The method as described in claim 1, characterized in that, When modeling using RQ-VAE in step 1, a U-Net without skip connections is used as the backbone network, and the encoder is replaced by a ResNet.

5. The method as described in claim 1, characterized in that, The specific process of fine feature alignment is as follows: Σ T The rows are divided into S submatrices W1, ..., W2. S And each submatrix is ​​a C / S × C projection matrix; then the quantization mask features and quantized image features The projection in low-dimensional space is and , represented as: for China and Israel ( i , j Any 3 x 3 area centered on ) will Each vector in All are expressed through linear combinations of eight adjacent vectors; these eight adjacent vectors are arranged column-wise into a matrix. ; The weighting coefficient is defined as follows: = ( ω 1 ,…,ω 8) T ,but The local linear expression is denoted as = T ; The objective is to find the weighting coefficients that minimize the local linear expression error. For quantizing image features Each projection ,Will Defined as the neighborhood matrix of any vector; The local linear expression is defined as = T ; Local linear alignment loss LLA Defined as: in, h × w for and The dimension of each vector in the vector; Local linear alignment loss LLA With minimization as the objective, fine-grained feature alignment is achieved through network training.

Images