Optical coherence tomography blood vessel segmentation method based on prior guidance local geometric modeling

Abstract

The invention provides an optical coherence tomography blood vessel segmentation method based on prior guidance local geometric modeling. According to the method, a residual quantization variational auto-encoder RQ-VAE is used for modeling a local geometrical shape of a blood vessel, and discrete vectors after residual quantization serve as prior features of a blood vessel mask and are stored in a codebook; training the segmentation network, wherein the training comprises coarse alignment and fine alignment; firstly, mask features and image features are projected to a principal component space, feature coarsening is carried out with principal component alignment loss minimization as a target, and the overfitting problem caused by mask and original image middle domain difference is relieved. In the fine alignment operation, local geometric structure alignment is carried out on the mask features and the potential features after replacement through local linear mapping operation. The anatomy error of the blood vessel can be reduced by using the method.

Description

Technical Field

[0001] The invention belongs to the technical field of medical image analysis, and in particular relates to an optical coherence tomography blood vessel segmentation method based on prior-guided local geometric modeling. Background Art

[0002] Optical coherence tomography (OCT) is a fast, non-invasive imaging technique that can perform high-resolution imaging of structures such as the anterior segment of the eye, retina, and choroid. Currently, OCT-based ophthalmic disease diagnosis and treatment methods have important clinical value, and most of these methods rely on OCT / OCTA images to quantify the anatomical structure of the eye. Among them, as one of the main anatomical structures of the eye, vascular structure has been identified as an important indicator for assessing the severity of ophthalmic diseases and provides clues for understanding the progression of related diseases. Optical coherence tomography angiography (OCTA) imaging can be used for a variety of clinical applications, from the diagnosis of corneal pathology to monitoring treatment. The quantification of retinal microvessels can assess pathologies such as diabetic retinopathy, glaucoma, and retinal artery occlusion; the choroidal vascular index is a major marker of the choroid, which is helpful for the diagnosis and treatment of age-related macular degeneration and choroidal neovascularization. Therefore, automatic vessel segmentation in OCT / OCTA images has become the key to disease diagnosis and treatment planning.

[0003] In the past decade, researchers have proposed some deep learning-based methods for vascular segmentation in OCT / OCTA images. Zhang et al. proposed a biomarker-infused global-local network for choroidal layer and choroidal vascular segmentation (see H. Zhang, J. Yang, K. Zhou, F. Li, Y. Hu, Y. Zhao, C. Zheng, X. Zhang, and J. Liu, "Automatic segmentation and visualization of choroid in oct with knowledge infused deep learning," IEEE Journal of Biomedical and Health Informatics, vol. 24, no. 12, pp. 3408–3420, 2020.), and studied the correlation between glaucoma and choroidal vascular changes. Zhu et al. proposed a choroidal U-network for layer and vessel segmentation (see L. Zhu, J. Li, R. Zhu, X. Meng, P. Rong, Y. Zhang, Z. Jiang, M. Geng, B. Qiu, X. Rong et al., “Synergistically segmenting choroidal layer and vessel using deep learning for choroid structure analysis,” Physics in Medicine & Biology, vol. 67, no. 8, p. 085001, 2022.). The proposed network first extracts the choroidal layer and then performs choroidal vessel segmentation based on the relevant features of the layer structure and vessels. For OCTA vascular segmentation, Mou et al. proposed a channel and spatial attention network for linear structure segmentation (see L. Mou, Y. Zhao, L. Chen, J. Cheng, Z. Gu, H. Hao et al., "Csnet: Channeland spatial attention network for curvilinear structure segmentation," in Proc. Int. Conf. Med. Image Comput. Comput. Assist. Intervent. (MICCAI), 2019, pp. 721–730.), which achieved good results in the task of vascular structure extraction from OCTA images; Li et al. proposed a projection network for 3D-2D images (see Y. Ma, H. Hao, J. Xie, H. Fu, J. Zhang, J.Yang, Z. Wang, J. Liu, Y. Zheng, and Y. Zhao, "Rose: Aretinal oct-angiography vessel segmentation dataset and new model," IEEE transactions on medical imaging, vol. 40, no. 3, pp. 928–939, 2021.), can extract 2D vascular projection maps from 3D OCTA volume data; Ma et al. proposed a coarse-to-fine network based on split attention mechanism (see M. Li, Y. Chen, Z. Ji, K. Xie, S. Yuan, Q. Chen, and S. Li, "Image projection network: 3D to 2D image segmentation in octa images," IEEE Transactions on Medical Imaging, vol.39, no.11, pp.3343–3354, 2020.), which is used for retinal vessel segmentation in OCTA images; Hao et al. proposed an OCTA image key structure extraction algorithm based on a voting mechanism (see J.Hao, T.Shen, X.Zhu, Y.Liu, A.Behera, D.Zhang, B.Chen, J.Liu, J.Zhang, and Y.Zhao, “Retinal structure detection in octa image via voting-based multitasklearning,” IEEE Transactions on Medical Imaging, vol.41, no.12, pp.3969–3980, 2022.), which realizes the detection of blood vessels, fovea and key points. Chen et al. proposed a multi-task joint optimization framework for choroidal neovascularization segmentation (see T.Chen, Y.Zhao, L.Mou, D.Zhang, X.Xu, M.Liu, H.Fu, and J.Zhang, “Rbgnet: Reliable boundary-guided segmentation of choroidalneovascularization,” in International Conference on Medical Image Computing.), combining a dual-stream encoding structure and an uncertain estimation strategy to enhance the network's ability to handle fuzzy boundaries. .

[0004] Although learning-based vascular segmentation methods have achieved good performance, these models are pixel-level classifications that ignore the inherent morphological features (geometry and shape) of blood vessels. This paradigm will cause the network to produce unreasonable vascular mask structures, which will bring great challenges to the quantification of related parameters and is not suitable for clinical applications. Previous medical image segmentation tasks partially utilized prior knowledge of segmentation masks. For example, Zotti et al. first designed a quality regression module to capture the mask shape prior, and then combined the average prior with the CNN model to obtain a more accurate anatomical mask result (see C. Zotti, Z. Luo, O. Humbert, A. Lalande, and P.-M. Jodoin, "Gridnet with automatic shape prior registration for automatic mri cardiac segmentation," in Statistical Atlases and Computational Models of the Heart. ACDC and MMWHS Challenges: 8th International Workshop, STACOM 2017, Held in Conjunction with MICCAI 2017, Quebec City, Canada, September 10-14, 2017, Revised Selected Papers 8. Springer, 2018, pp. 73–81.). This type of method only encodes the mask shape simply, but may have difficulty representing complex mask shapes. Therefore, some researchers have recently proposed deep networks for encoding mask prior features. Oktay et al. proposed a new general training strategy for cardiac image segmentation (see O. Oktay, E. Ferrante, K. Kamnitsas, M. Heinrich, W. Bai, J. Caballero, S. A. Cook, A. De Marvao, T. Dawes, D. P. O'Regan et al., "Anatomically constrained neural networks (acnns): application to cardiac image enhancement and segmentation," IEEE transactions on medical imaging, vol. 37, no. 2, pp. 384–395, 2017.).The model uses a regularization model to align the input image latent features with the anatomical prior latent features, prompting the model to learn global anatomical properties. Painchaud et al. introduced a VAE-based anatomically guaranteed cardiac image segmentation network (see N. Painchaud, Y. Skandarani, T. Judge, O. Bernard, A. Lalande, and P.-M. Jodoin, "Cardiac segmentation with strong anatomical guarantees," IEEE transactions on medical imaging, vol. 39, no. 11, pp. 3703–3713, 2020.), which corrects unreliable predicted masks by introducing mask shape distribution into the post-processing network. On this basis, Judge et al. proposed an uncertainty prediction medical image segmentation framework to alleviate the mismatch problem between predicted mask features and prior features by modeling the joint potential representation of mask shape and input image (see T.Judge, O.Bernard, M.Porumb, A.Chartsias, A.Beqiri, and P.-M.Jodoin, "Crisp-reliable uncertainty estimation for medical image segmentation," in International Conference on Medical Image Computing and Computer-Assisted Intervention.Springer, 2022, pp.492–502.).

[0005] Previous medical image segmentation tasks have partially utilized the prior knowledge of segmentation masks. These methods can effectively encode the mask prior and regularize the network, thereby improving the accuracy and reliability of the network's predicted mask. However, there are few works that use mask modeling methods for vascular segmentation. There are two main challenges:

[0006] First, the above works use autoencoders (AE) and variational autoencoders (VAE) to model mask priors and output potential continuous vectors. These frameworks prefer to encode the overall prior distribution of the mask. In this case, they have difficulty in characterizing the local complex structure of microvessels, which leads to the failure of the prior regularization constraints of local vascular segmentation results. Therefore, how to achieve local and efficient modeling of microvascular masks is the first challenge to be solved.

[0007] Second, the low signal-to-noise ratio in blood vessels will lead to domain gaps and local errors between the blood vessel mask features and the image features. Strong distance constraints between features may lead to network overfitting. Therefore, how to make full use of the blood vessel mask prior features and effectively interact with the image features is the second challenge. Summary of the invention

[0008] In view of this, the present invention provides an optical coherence tomography blood vessel segmentation method based on prior-guided local geometric modeling, which can reduce the anatomical error of the blood vessel.

[0009] In order to solve the above technical problems, the present invention is implemented as follows.

[0010] An optical coherence tomography blood vessel segmentation method based on guided local geometric modeling, comprising:

[0011] Step 1: Local blood vessel modeling: Use residual quantization variational autoencoder RQ-VAE to model the local geometric shape of the blood vessel; the discrete vector after residual quantization RQ is stored in the codebook as the blood vessel mask prior feature; wherein RQ-VAE includes a first encoder, a first decoder and a codebook, and the first encoder and the second encoder are trained through the local geometric shape modeling process;

[0012] Step 2: Vessel segmentation network training:

[0013] The blood vessel segmentation network includes an alignment network and a segmentation network; the alignment network uses the first encoder and the first decoder trained in step 1; the segmentation network includes a segmentation encoder and a segmentation decoder;

[0014] During network training, the standard mask is input into the first encoder to obtain mask features, and the image to be segmented is input into the segmentation encoder to obtain image features. The mask features and image features are projected into the principal component space, and the feature rough alignment is performed with the goal of minimizing the principal component alignment loss.

[0015] After the mask feature and image feature are subjected to residual quantization RQ, the quantized mask feature is obtained. and quantify image features Projection is performed in a low-dimensional space constructed based on the codebook to obtain the discrete mask feature projection P (n) and discrete image feature projection Q (n) , project the discrete mask features P (n) Each vector (i, j) is expressed as a local linear combination of adjacent vectors, denoted by And determine the combination coefficient that minimizes the local linear expression error, and use this combination coefficient Construct image feature projection Q (n) The local linear expression of each vector of Project the mask feature P(n) Local linear expression with image feature projection The error is used as the local linear alignment loss, and the feature fine alignment is performed with the goal of minimizing the local linear alignment loss;

[0016] Step 3: Use the trained segmentation network to perform optical coherence tomography blood vessel segmentation.

[0017] Preferably, contrast loss is added to the loss function of RQ-VAE training, and contrast loss is used for contrast learning to improve the separation of different discrete vectors;

[0018] Construct a cosine similarity matrix A that represents the similarity of discrete vectors:

[0019]

[0020] in, represents matrix multiplication; b represents the codebook, which consists of K C-dimensional discrete vectors b 1 ~b K After normalization, any non-diagonal element of the cosine similarity matrix A is composed of ij Represents the cosine similarity between the i-th and j-th discrete vectors;

[0021] During the training process in step 1, the cosine similarity matrix A is made close to a unit matrix to minimize the similarity between different discrete vectors.

[0022] Preferably, the loss function for RQ-VAE training further includes a mixed loss, the mixed loss function for:

[0023]

[0024] In the above formula (2), M is the encoder E rec Standard mask for input, Decoder D rec Output reconstruction mask; β = 0.25 is the multiplication factor coefficient; F rec For encoder E rec The mask feature map output by encoding the standard mask; D is the mask feature map F rec The stacking depth of encoder E rec Output mask feature map F rec It is composed of D discrete vectors of the codebook stacked together; sg[·] indicates the stop gradient; Represents the composition of the mask feature map F rec The i-th of the D discrete eigenvectors of .

[0025] Preferably, when RQ-VAE is used for modeling in step 1, U-Net without jump connections is used as the backbone network, and the encoder is replaced by ResNet.

[0026] Preferably, the mask features and the image features are projected into the principal component space, and the coarse alignment of the features is performed with the goal of minimizing the principal component alignment loss as follows:

[0027] The K C-dimensional discrete vectors b in the codebook are 1 ~b K Arranged into a matrix, represented as X∈R C×K ; The projection matrix is ​​defined as W∈R N×C ,N≤C; through matrix decomposition, XX T ∈R C×C Expressed as ΣΛΣ T , where Λ∈R C×C It is a diagonal matrix, whose diagonal elements are arranged in descending order of eigenvalue; Σ∈R C×C is an orthogonal matrix; each column vector of Σ is XX T an orthogonal eigenvector of ;

[0028] Select Σ T The first N rows of are used as the projection matrix W to project the mask features and image features into the principal component space;

[0029] Principal component alignment loss It is expressed as:

[0030]

[0031] In the above formula (3), F rec is the mask feature output by the first encoder, F seg Image features output by the segmentation encoder;

[0032] Principal component alignment loss With minimization as the goal, rough feature alignment is performed through network training.

[0033] Preferably, the specific process of feature fine alignment is:

[0034] Σ T The row direction is divided into S sub-matrices W 1 , ..., W S , and each sub-matrix is ​​a C / S×C projection matrix; then the quantized mask features and quantify image features The projection in the low-dimensional space is P (n) and Q (n) , expressed as:

[0035]

[0036] For P (n) In any 3×3 region centered at (i,j), P (n) Each vector in are expressed by linear combinations of 8 adjacent vectors; the 8 adjacent vectors are arranged in columns into a matrix The weighting coefficient is defined as but The local linear expression of

[0037] Solve the weighting coefficient that minimizes the local linear expression error. The solution goal is:

[0038]

[0039] For quantifying image features Each projection Q (n) ,Will Defined as the neighborhood matrix of any vector; Q (n) The local linear expression of

[0040] Local linear alignment loss L LLA Defined as:

[0041]

[0042] Among them, h×w is P (n) and Q (n) The dimension of each vector in ;

[0043] With local linear alignment loss L LLA With minimization as the goal, fine feature alignment is achieved through network training.

[0044] Beneficial effects:

[0045] (1) The present invention proposes a discrete prior-guided local linear alignment framework, which introduces effective discrete vectors into the pre-trained codebook to replace the defective latent feature map extracted by the segmentation encoder, thereby improving the effectiveness of the local shape representation of blood vessels.

[0046] (2) The present invention utilizes the RQ-VAE model to quantify and store the prior features of the vascular mask, thereby achieving efficient modeling of the local structure of the mask.

[0047] (3) During the training of the segmentation network, a new interactive framework between the image latent features and the prior features was constructed. A combination of coarse alignment and local fine alignment was used to ensure that the mask prior and the latent features were locally and accurately aligned, thereby enhancing the network's ability to capture microvessels.

[0048] (4) In the coarse alignment stage, by coarsely matching the principal component features of the mask features with the image features, the principal component alignment loss can enable the network to directly focus on the high-dimensional representation of the main features of the blood vessels while ignoring some noise and details, thereby preventing overfitting caused by forced matching of image features with prior features.

[0049] (5) Local fine alignment stage. Considering that the geometric structure of blood vessels exhibits strong local continuity, the present invention proposes a carefully designed fine alignment loss to perform local geometric structure alignment between mask features and replaced latent features through local linear mapping.

[0050] (6) In a preferred embodiment, contrast loss is added during RQ-VAE training to ensure the sparsity of discrete vectors in the codebook. Vector density may lead to incorrect application of similar vectors. The addition of contrast loss can reduce the impact of vector density on segmentation accuracy.

[0051] The proposed framework provides an effective tool for evaluating and quantifying structural changes in ophthalmic blood vessels, assisting clinicians in early screening and treatment of vascular-related diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 Modeling framework for the first stage vessel mask.

[0053] Figure 2 It is the interactive framework between the latent features of the image and the mask features in the second stage. DETAILED DESCRIPTION

[0054] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0055] The present invention provides an optical coherence tomography vascular segmentation scheme based on prior-guided local geometric modeling, which includes two stages. In the first stage, local vascular modeling is performed, and the local geometric shape of the blood vessel is modeled using residual quantization variational autoencoders (RQ-VAE). The discrete vectors after residual quantization RQ are stored in the codebook as vascular mask prior features. In the second stage, vascular segmentation network training is performed, and the present invention proposes a new OCT / OCTA vascular segmentation framework. The framework can incorporate the discrete vectors stored in the first stage into any U-shaped network, and reduce the anatomical error of the blood vessel by utilizing the local shape prior of the blood vessel.

[0056] In the second stage, the present invention proposes two new distance constraint losses according to the characteristics of blood vessel segmentation. First, although there are differences in details between image features and mask features, the local blood vessel shapes they represent should have roughly similar structures. Therefore, the present invention proposes a principal component alignment (PCA) loss to map image features and mask features to the principal component space for coarse alignment. On the other hand, the present invention hopes that the new feature map of residual quantization can accurately represent the local geometric shape of the blood vessel. At the same time, the high-dimensional image features projected to a specific low-dimensional space should retain local geometric correlations. Therefore, the present invention proposes a local linear registration loss for fine registration of image features and mask features after residual quantization.

[0057] The specific implementation process of the present invention is described in detail below.

[0058] Step 1: Local geometry modeling of blood vessels.

[0059] In the first stage, the present invention uses RQ-VAE to model the local geometry of the blood vessels. The RQ-VAE network consists of an encoder E rec , decoder D rec and codebook B. The codebook contains K C-dimensional discrete vectors b i ∈R C , i∈[K]. Figure 1 As shown, similar to the autoencoder, the gold standard of the input, that is, the standard mask definition M∈R H×W×1 , after encoder E rec After processing, it is encoded into a continuous feature map, called mask feature F rec ∈R h×w×C Each C-dimensional vector The D discrete vectors in the codebook Stacked representation.

[0060] Each discrete vector is chosen as follows:

[0061]

[0062] The above operation is called residual quantization (RQ). Set to zero vector and D to be the depth.

[0063] When residual quantization RQ is used for F rec When all h×w vectors are in the matrix, the present invention can obtain D discrete feature maps. F rec The discrete representation of

[0064]

[0065] Finally, the decoder will As input, to obtain the reconstruction result

[0066] Conventional RQ-VAE training relies on a hybrid loss function:

[0067]

[0068] Where sg[·] represents the stop gradient and β is the multiplication factor coefficient. Currently, vector quantization has been widely used in image generation.

[0069] Traditional methods usually do not impose any restrictions on the discrete vectors in the codebook. However, in medical image segmentation, the image usually has a unique mask. Therefore, the density of vectors in the codebook may lead to the incorrect application of similar vectors, thereby affecting the accuracy of segmentation. Therefore, the present invention uses contrastive learning to improve the separation between different discrete vectors and ensure the sparsity of the code set. Figure 1 As shown, the distance between any two discrete vectors is measured by cosine similarity. The calculation formula of the cosine similarity matrix A is:

[0070]

[0071] in, represents matrix multiplication, b represents the codebook, which is a C×K matrix. Each column of b is a normalized discrete vector. The diagonal elements of the matrix A are 1, and any non-diagonal elements A ij represents the cosine similarity between the i-th and j-th discrete vectors.

[0072] The goal of the present invention is to minimize the similarity between different discrete vectors so that the matrix A is close to an identity matrix. The present invention uses cross entropy to achieve this contrast loss:

[0073]

[0074] Finally, the mask reconstruction is trained by a combination of hybrid loss and contrastive loss.

[0075] The present invention adopts a through-estimator to solve the back-propagation problem in the RQ operation.

[0076] Step 2: Vessel segmentation network training

[0077] Most segmentation methods use U-shaped networks for medical image segmentation, and the feature extraction ability of the encoder has a great impact on the performance of the network. However, OCTA / OCT images may have problems such as unclear blood vessels, artifacts, noise, and horizontal stripes. This makes it difficult for existing methods to extract effective latent feature maps to accurately represent the local shape of blood vessels. To this end, the present invention introduces effective discrete vectors from a pre-trained codebook to replace the defective latent feature maps extracted by the segmentation encoder.

[0078] However, there is a domain shift between images and manual annotations, so when the pre-trained code set is applied to image features, it may lead to incorrect interactions. In previous studies on mask pre-positioning, researchers have used Euclidean distance and cosine similarity to make image features closer to mask features. However, due to the domain gap and local structure errors between mask features and image features, forcibly aligning two different features may lead to overfitting. In addition, the introduction of discrete vectors may cause the network to ignore the local geometric relationship of blood vessels.

[0079] To solve the above problems, Figure 2 As shown, the present invention proposes two new distance constraint losses according to the characteristics of blood vessel segmentation.

[0080] See first Figure 2 The structure of the blood vessel segmentation network is introduced. The blood vessel segmentation network includes an alignment network and a segmentation network. The alignment network uses the first encoder and the first decoder trained in step 1, see Figure 2 The blue encoder and decoder in the upper middle part; the segmentation network includes the segmentation encoder and segmentation decoder, see Figure 2 The green encoder and decoder in the lower middle part. The alignment network is used for training, and the segmentation network is used for vessel segmentation.

[0081] During network training, the features output by the first encoder and the segmentation encoder are used for coarse alignment, and the features after residual quantization (RQ) are used for fine alignment.

[0082] Among them, for the coarse alignment part: the standard mask is input into the first encoder to obtain the mask feature F rec , the image to be segmented is input into the segmentation encoder to obtain the image feature F seg Although there are differences in details between image features and mask features, the local blood vessel shapes they represent should have roughly similar structures. Therefore, the present invention proposes a principal component alignment (PCA) loss to map image features and mask features into the principal component space for rough alignment.

[0083] For the fine alignment part: the mask features and image features are respectively subjected to residual quantization RQ to obtain the quantized mask features and quantify image features The present invention hopes that the new feature map of residual quantization can accurately represent the local geometric shape of the blood vessel. At the same time, the high-dimensional image features projected into a specific low-dimensional space should retain local geometric correlation. Therefore, the present invention proposes a local linear registration loss for fine registration of image features and mask features after residual quantization.

[0084] (1) Coarse alignment based on principal component alignment loss

[0085] Through mask reconstruction, the latent space can effectively encode the intrinsic data structure of the local geometric shape of the blood vessel. In order to capture the main components of the local vascular structure, the mask feature map F is analyzed. rec Each latent vector Z in i,j is the key. Principal component analysis is a solution. Specifically, the present invention defines the latent space as In each training batch, the pre-trained mask encoder is fixed, so the latent vector Z i,j It can be understood as a sample of X. Therefore, the most direct way to obtain the principal component space is to perform principal component analysis on X. However, the samples in each batch are not enough to represent the entire latent space. Each vector in the latent space can be linearly represented by D discrete vectors in the codebook. Therefore, the cardinality of the discrete vector spans the entire latent space and can replace the latent space as the analysis object. The present invention arranges the K discrete vectors in the codebook into a matrix (x 1 ,x 2 ,…,x K ), denoted as X∈R C×K The projection matrix is ​​defined as W∈R N×C ,N≤C,projection Z∈R N×K It can be expressed as WX. When the present invention performs zero-centering processing on the original data X, the covariance matrix of its projection is as follows:

[0086]

[0087] C Z The diagonal elements in represent variance and indicate the importance of different dimensions. The elements outside the diagonal represent covariance and indicate the correlation between different dimensions. The present invention hopes that the projection in the low-dimensional space can capture the main components of the original sample while ensuring the orthogonality between the components of different dimensions. Therefore, the goal of the present invention is to find a projection matrix such that C Z Become a diagonal matrix while maximizing its diagonal elements. By matrix decomposition, XX T ∈R C×C It can be expressed as ΣΛΣ T , where Λ∈R C×C Is a diagonal matrix whose diagonal elements are arranged in descending order of eigenvalue.C×C is an orthogonal matrix. Each column vector of Σ is XX T An orthogonal eigenvector of . By selecting different column vectors from Σ as the row vectors of W, C Z becomes a diagonal matrix. The choice of orthogonal vectors determines C Z The size and diagonal elements of . Therefore, the present invention selects Σ T The first N rows (corresponding to XX T The first N largest eigenvalues ​​of are used as the projection matrix W. The principal component alignment loss can be expressed as:

[0088]

[0089] W T Represents the feature reconstruction matrix in high-dimensional space, which can use a new basis to represent the features. Represents the feature reconstruction matrix in high-dimensional space, which can use a new basis to represent the main components of the features. The principal component alignment loss can make the network directly focus on the high-dimensional representation of the main features of the blood vessels, while ignoring some noise and details.

[0090] Principal component alignment loss With minimization as the goal, rough feature alignment is performed through network training.

[0091] (2) Fine alignment based on local linear alignment loss

[0092] After residual quantization, the goal of the present invention is to introduce discrete vectors to make up for the lack of image feature details. Therefore, it is necessary to consider both its principal component and detail component. On the other hand, the geometric structure of blood vessels exhibits strong local continuity. This makes it difficult for existing methods to capture the local geometric relationship of blood vessels. The introduction of discrete vectors may further destroy the local geometric correlation in the latent space. Local linear embedding can reflect the local geometric relationship of the manifold in the embedded space based on the estimation of adjacent weights. However, the local vectors in the latent space are usually non-dense, so it is challenging to establish linear correlations between these vectors in the original dimension. Some high-dimensional data may exhibit meaningful local structures in low-dimensional space. To this end, the present invention proposes to establish local geometric correlations of blood vessels between projections of different low-dimensional spaces.

[0093] The present invention will T The row direction is divided into S sub-matrices W 1 , ..., W S , and each submatrix is ​​a C / S×C projection matrix. Since the diagonal elements of Λ are arranged in descending order of eigenvalue, the front submatrix can extract the principal component, while the back submatrix focuses on the detail component. After residual quantization, the projections of mask features and image features in low-dimensional space P(n), Q(n)∈Rh×w×C / S It can be expressed as:

[0094]

[0095] like Figure 2 As shown, for any 3×3 region centered at (i, j), each vector of the projected mask feature All can be approximated by the linear combination of their adjacent vectors. The present invention arranges the eight adjacent vectors into a matrix by column: When the combination coefficient is defined as The local linear representation of The goal of the present invention is to find the coefficients that minimize the error:

[0096]

[0097] The solution of the coefficient can be transformed into:

[0098]

[0099] Among them, 1 in the above formula represents the unit matrix;

[0100] is defined as:

[0101]

[0102] Through the Lagrangian method, the present invention obtains the optimal solution:

[0103]

[0104] The present invention hopes that this local linear relationship can be preserved in the image features. Each projection Q (n) , the present invention will is defined as the neighborhood matrix of any vector. The local linear representation is defined as The local linear alignment loss is defined as:

[0105]

[0106] Therefore, through different subspaces, L LLA The local geometric connection of the blood vessels can be gradually established from coarse to fine. LLA With minimization as the goal, fine feature alignment is achieved through network training.

[0107] The present invention is based on principal component alignment loss and the local linear alignment loss L LLAJoint network training is performed to achieve coarse and fine alignment, thereby reducing the anatomical error of the blood vessels.

[0108] In terms of network framework design, for the first stage, the present invention uses RQVAE to reconstruct the vascular mask. In a preferred embodiment, a U-Net without jump connections is used as the backbone network, in which the encoder E rec Replaced by ResNet. For the standardization of the framework, the present invention designs the resolution (h, w) of the latent space to (H / 16, W / 16), and sets the dimension C of the latent space to 512. The codebook is configured as follows, with its depth D set to 4 and the number of discrete vectors K set to 2048. The present invention uses a combination of hybrid loss and contrastive loss to train the network. The multiplication factor β in the hybrid is set to 0.25.

[0109] In the second stage, the mask encoder and codebook pre-trained in the first stage will be fixed to guide the training of the segmentation network. Any segmentation network can be embedded in the segmentation framework of the present invention. As long as the resolution of the latent space meets the following standards (H / 16, W / 16). The present invention uses the same encoder and decoder as in the first stage to form the initial segmentation network. Figure 2 As shown, the present invention replaces the original image features with residual quantization representation as the input of the segmentation decoder. The present invention uses a straight-through estimator to ensure the gradient back propagation in the above process.

[0110] Step 3: Use the trained segmentation network to perform optical coherence tomography blood vessel segmentation.

[0111] The above specific embodiments only describe the design principle of the present invention. The shapes and names of the components in the description may be different and are not limited. Therefore, those skilled in the art in the field of the present invention may modify or replace the technical solutions recorded in the above embodiments; and these modifications and replacements do not deviate from the creative purpose and technical solutions of the present invention and should all fall within the protection scope of the present invention.

Claims

1. An optical coherence tomography blood vessel segmentation method based on prior-guided local geometric modeling, characterized in that: include: Step 1: Local blood vessel modeling: Use residual quantization variational autoencoder RQ-VAE to model the local geometric shape of the blood vessel; the discrete vector after residual quantization RQ is stored in the codebook as the blood vessel mask prior feature; wherein RQ-VAE includes a first encoder, a first decoder and a codebook, and the first encoder and the second encoder are trained through the local geometric shape modeling process; Step 2: Vessel segmentation network training: The blood vessel segmentation network includes an alignment network and a segmentation network; the alignment network uses the first encoder and the first decoder trained in step 1; the segmentation network includes a segmentation encoder and a segmentation decoder; During network training, the standard mask is input into the first encoder to obtain mask features, and the image to be segmented is input into the segmentation encoder to obtain image features. The mask features and image features are projected into the principal component space, and the feature rough alignment is performed with the goal of minimizing the principal component alignment loss. After the mask feature and image feature are subjected to residual quantization RQ, the quantized mask feature is obtained. and quantify image features Projection is performed in a low-dimensional space constructed based on the codebook to obtain the discrete mask feature projection P (n) and discrete image feature projection Q (n) , project the discrete mask features P (n) Each vector (i, j) is expressed as a local linear combination of adjacent vectors, denoted by And determine the combination coefficient that minimizes the local linear expression error, and use this combination coefficient Construct image feature projection Q (n) The local linear expression of each vector of Project the mask feature P (n) Local linear expression with image feature projection The error is used as the local linear alignment loss, and the feature fine alignment is performed with the goal of minimizing the local linear alignment loss; Step 3: Use the trained segmentation network to perform optical coherence tomography blood vessel segmentation.

2. The method according to claim 1, characterized in that Add contrast loss to the loss function of RQ-VAE training, and use contrast loss for contrast learning to improve the separation of different discrete vectors; Construct a cosine similarity matrix A that represents the similarity of discrete vectors: in, represents matrix multiplication; b represents the codebook, which consists of K C-dimensional discrete vectors b1~b K After normalization, any non-diagonal element of the cosine similarity matrix A is composed of ij Represents the cosine similarity between the i-th and j-th discrete vectors; During the training process in step 1, the cosine similarity matrix A is made close to a unit matrix to minimize the similarity between different discrete vectors.

3. The method according to claim 2, characterized in that The loss function for RQ-VAE training further includes mixed loss, mixed loss function for: In the above formula (2), M is the encoder E rec Standard mask for input, Decoder D rec Output reconstruction mask; β = 0.25 is the multiplication factor coefficient; F rec For encoder E rec A mask feature map output by encoding the standard mask; D is the mask feature map F rec The stacking depth of encoder E rec Output mask feature map F rec It is composed of D discrete vectors of the codebook stacked together; sg[·] indicates the stop gradient; Represents the composition of the mask feature map F rec The i-th of the D discrete eigenvectors of .

4. The method according to claim 1, characterized in that In step 1, when RQ-VAE is used for modeling, U-Net without skip connections is used as the backbone network, and the encoder is replaced by ResNet.

5. The method according to claim 1, characterized in that The mask features and image features are projected into the principal component space, and the principal component alignment loss is minimized to perform rough feature alignment as follows: The K C-dimensional discrete vectors b1~b in the codebook are K Arranged into a matrix, represented as X∈R C×K ; The projection matrix is ​​defined as W∈R N×C ,N≤C; through matrix decomposition, XX T ∈R C×C Expressed as where Λ∈R C×C It is a diagonal matrix, whose diagonal elements are arranged in descending order of eigenvalue; Σ∈R C×C is an orthogonal matrix; each column vector of Σ is an orthogonal eigenvector of ; choose The first N rows of are used as the projection matrix W to project the mask features and image features into the principal component space; Principal component alignment loss It is expressed as: In the above formula (3), F rec is the mask feature output by the first encoder, F seg Image features output by the segmentation encoder; Principal component alignment loss With minimization as the goal, rough feature alignment is performed through network training.

6. The method according to claim 5, characterized in that The specific process of feature fine alignment is as follows: Σ T The row direction is divided into S sub-matrices W1, ..., W S , and each sub-matrix is ​​a C / S×C projection matrix; then the quantized mask features and quantify image features The projection in the low-dimensional space is P (n) and Q (n) , expressed as: For P (n) In any 3×3 region centered at (i,j), P (n) Each vector in are expressed by linear combinations of 8 adjacent vectors; the 8 adjacent vectors are arranged in columns into a matrix The weighting coefficient is defined as but The local linear expression of Solve the weighting coefficient that minimizes the local linear expression error. The solution goal is: For quantifying image features Each projection Q (n) ,Will Defined as the neighborhood matrix of any vector; Q (n) The local linear expression of Local linear alignment loss L LLA Defined as: Among them, h×w is P (n) and Q (n) The dimension of each vector in ; With local linear alignment loss L LLA With minimization as the goal, fine feature alignment is achieved through network training.

Images