Blood vessel segmentation labeling method for optical coherence tomography angiography

Abstract

The invention provides an optical coherence tomography angiography blood vessel segmentation labeling method which comprises the following steps: a model training and preparation stage: pre-training an MONAI segmentation model by using an OCTA-500 data set, and finely adjusting an SAM large model through an LoRA technology; in the image input and preprocessing stage, format analysis and standardization are carried out on the uploaded OCTA image; a segmentation reasoning stage: calling a global or local model to generate a preliminary mask according to user selection; in the interactive labeling optimization stage, the user corrects the mask in real time through a paintbrush and an eraser tool; in the result exporting stage, the optimized mask is stored in standard formats such as JSON, TIFF and NIFTI, and a labeling data set for model retraining is synchronously exported. The method has high precision and high efficiency, is flexible in deployment, and is suitable for clinical and scientific research scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and medical image processing, and in particular to a method for segmenting and labeling blood vessels in optical coherence tomography angiography. Background Art

[0002] OCTA has become an important tool for high-resolution imaging of fundus microvascular networks and is widely used in the diagnosis of diseases such as diabetic retinopathy and macular degeneration. Existing automatic segmentation methods, mostly based on end-to-end deep learning models, can only achieve global vessel extraction and struggle to accurately capture small local vascular branches. The lack of effective interactive correction mechanisms makes annotation efficiency and accuracy difficult to meet clinical needs. Summary of the Invention

[0003] The present invention aims to at least solve the technical problems existing in the prior art, and in particular innovatively proposes a method for segmenting and labeling blood vessels in optical coherence tomography angiography, comprising:

[0004] S1, model training and preparation

[0005] S1-1, obtain the OCTA training dataset containing retinal vascular annotation information;

[0006] S1-2, training the first segmentation model using an end-to-end deep learning framework to obtain a first model weight for global vessel segmentation;

[0007] S1-3, based on the pre-trained visual Transformer encoder, the low-rank adaptation matrix is ​​inserted into its weight matrix and only the low-rank parameters are updated to complete the fine-tuning of the large model and obtain the second model weights for local blood vessel segmentation;

[0008] S2, image input and preprocessing

[0009] S2-1, receiving the OCTA image or image folder to be processed through a visual interactive interface;

[0010] S2-2, performs format parsing, size normalization, and grayscale normalization on the received image to generate a standardized image tensor for inference;

[0011] S3, segmentation reasoning

[0012] S3-1, determining whether the segmentation mode is a global segmentation mode or a local segmentation mode according to the user's selection in the interactive interface;

[0013] S3-2: If the global segmentation mode is selected, calling the first segmentation model corresponding to the first model weight to infer the normalized image tensor and output a first segmentation mask;

[0014] S3-3, if it is a local segmentation mode, an interest region and prompt information indicated by a user on an image are obtained, a large model corresponding to the second model weight is called to perform inference on the interest region, and a second segmentation mask is output;

[0015] S4, interactive labeling optimization

[0016] S4-1, the segmentation mask is superimposed and displayed on an original image in the interactive interface;

[0017] S4-2, an editing instruction issued by a user through drawing, erasing or fragment erasing tools is received, and the segmentation mask is updated in real time based on the editing instruction;

[0018] S5, result export

[0019] S5-1, the segmentation mask optimized through interaction is saved in a medical image compatible file format;

[0020] S5-2, a labeling data set corresponding to the segmentation mask is synchronously exported for subsequent model retraining or clinical analysis;

[0021] In a preferred embodiment of the present application, the image input and preprocessing include:

[0022] The OCTA-500 data set is subjected to standardization processing, including image size adjustment, normalization and image enhancement, and the standardized image is used for input into an existing segmentation model or a self-defined segmentation model;

[0023] The normalization formula is

[0024] wherein I norm is a normalized image gray value;

[0025] I is an original image gray value;

[0026] μ is a mean value of the image;

[0027] σ is a standard deviation of the image;

[0028] Since the OCTA data set sample size is limited, data enhancement operation for enriching image representation is necessary, and the enhancement process can be divided into three aspects, specifically including:

[0029] Brightness and contrast adjustment: the brightness and contrast of the OCTA image can change due to acquisition conditions, and random adjustment of brightness and contrast helps to simulate image quality under different devices and environments, and the brightness and contrast of the image will randomly change with a probability of 20%, and the formula is:

[0030] I' = max(0, min(1, (1 ± 0.15)I ± 0.25);

[0031] where I' is the image intensity value after brightness and contrast adjustment;

[0032] max(0,*) is the lower bound truncation function, which ensures that the final result is not lower than 0;

[0033] min(1,*) is the upper bound truncation function, which ensures that the intermediate calculation result does not exceed 1;

[0034] By randomly rotating within a range of ±15° or vertically and horizontally flipping with a probability of 25%, different blood vessel distribution directions are simulated to enhance the adaptability of the model to geometric changes. The rotation formula is as follows:

[0035]

[0036] where the pixel coordinates of the image are (x, y), and the new coordinates (x', y') after rotation transformation;

[0037] where θ ~ U(-15°, 15°) is the rotation angle, which is subject to uniform distribution;

[0038] Pixels outside the image boundary can be mirror filled along the boundary line;

[0039] The formula for horizontal flipping is x' = W - x, y' = y, and the formula for vertical flipping is x' = x, y' = H - y;

[0040] where H and W are the height and width of the image, respectively;

[0041] Gaussian convolution blur, local geometric disturbance, and random occlusion are used as three enhancement methods, with a triggering probability of 0.1;

[0042] The formula expressions are as follows:

[0043]

[0044] where I(x+i, y+j) is the gray value of the original image at (x+i, y+j), and i and j represent the horizontal and vertical offsets, respectively;

[0045] I''(x, y) is the output gray value at (x, y) after filtering;

[0046] represents the Gaussian blur scenario, and K(i, j) is the two-dimensional filter kernel weight, which satisfies the normalization condition ∑K(i, j) = 1;

[0047] Local geometric disturbance:

[0048] affine matrix parameter a 11 , a 22 is a scaling parameter, which determines the scaling ratio of the image in the horizontal and vertical directions, and takes a 11 , a 22 ~ U (0.9, 1.1);

[0049] parameter a 12 , a 21 is a shear parameter, which allows the pixels to be slightly offset along the tilt direction determined by the two parameters, and takes a 12 , a 21 ~ U (-0.1, 0.1), offset t x and t y are random values in [-7, 7];

[0050] Random occlusion: 20 to 40 random occlusion regions are generated in the image, and the definition of the occlusion region R i is as follows:

[0051] R i = {(x, y) | x l ≤ x ≤ x l + w, y l ≤ y ≤ y l + h};

[0052] where: (x l , y l ) is the coordinate of the occlusion rectangle, w and h are the width and height of the rectangle, and satisfy w, h ~ U (8, 12);

[0053] Model structure-weight double file mechanism:

[0054] Structure definition file;

[0055] The network topology is described using YAML syntax, including layer type, layer order, hyperparameter, and topology verification hash;

[0056] Loading process: the system first parses YAML and calculates the topology hash, and then reads the weight file; only when the two are consistent can it be executed, if they are not consistent, an exception is thrown and the user is prompted to match again;

[0057] In the model training process, the preprocessed OCTA image data is input, the training hyperparameter learning rate η is set to 0.001, the batch size is 16, a total of 50 training rounds are sufficient to make the performance of the model converge, and the Dice loss function is used for optimization;

[0058] Where Y is the predicted segmentation area, Y' is the manually annotated area, the optimizer is responsible for updating the model parameters to make the loss function gradually decrease, and it adjusts the model weights according to the gradient information to continuously improve the model performance, and the method uses the Adam optimizer, and the update formula is θ t+1 is the current model weight parameter, is the current gradient;

[0059] In each round of training process, the Dice coefficient and the intersection over union are used as evaluation indexes, and the IoU calculation formula is

[0060] In particular, since the input of SAM-OCTA involves two parts of image and prompt point, SAM-OCTA uses a pre-trained Vision Transformer as an image encoder, and uses a low-rank adaptation technology for fine-tuning, specifically, an additional linear layer is inserted in each Transformer block of ViT as a multi-layer perception, which is in the form of:

[0061] W'=W+ΔW,ΔW=B LoRA ·A LoRA ;

[0062] Where, W is the original weight matrix; B LoRA and A LoRA are low-rank matrices;

[0063] ΔW is a low-rank update, and the original weight W is frozen during training, and only B LoRA and A LoRA are updated, which significantly reduces the number of trainable parameters.

[0064] In a preferred embodiment of the present application, the segmentation inference comprises:

[0065] The image data is transferred from the memory-CPU-GPU, which is to copy the tensor from the host memory to the video memory, and the GPU is used to accelerate the calculation;

[0066] After loading the model weight, a layer of Sigmoid activation function needs to be added to control the output form;

[0067] The loaded model is used to predict the image tensor, and the probability value tensor is output, and the probability value output by the model is thresholded to generate a binary segmentation result;

[0068] The segmentation result is converted from the tensor form to the NumPy format, and is enlarged to the [0, 255] interval for subsequent processing, and the operation adopts the method of directly multiplying the enlargement coefficient: P npy =P binary ×255;

[0069] P npy is the image matrix obtained by converting the binary mask into a NumPy array and then mapping it to 0-255 grayscale;

[0070] Connected region analysis:

[0071] The binary segmentation result is subjected to connected region analysis, different segmentation regions are marked, and the segmentation result is superimposed on the original image in a specific color to generate a visual preview.

[0072] In a preferred embodiment of the present application, the interactive annotation optimization comprises:

[0073] Real-time update of connected regions:

[0074] During modification, the modified region is updated in real time and superimposed into the segmentation result, and as the strokes increase or decrease, the connected region analysis is also updated in real time;

[0075] Processing non-connected regions:

[0076] The connectedness analysis algorithm is used to accurately identify independent connected components in the image, the geometric area of each connected component is calculated, and redundant branches with small area or abnormal shape are gradually removed according to the set area threshold, which is expressed as:

[0077] Connected domain identification: C i ={p|connectedness(p)=i,i=1,2,...,N};

[0078] Where C i represents the i-th connected component, p is a pixel point, and N is the total number of total connected components;

[0079] The area of each connected component is calculated:

[0080] Where A(C i ) represents the geometric area of the connected component C i ;

[0081] Area-based hierarchical filtering: set the area threshold T and the morphological screening condition to filter out small-area fragment connected components:

[0082]

[0083] Where represents the screened component.

[0084] The present application also discloses a computer system, comprising:

[0085] A processor;

[0086] a memory for storing processor-executable instructions;

[0087] The processor is configured to implement the optical coherence tomography angiography blood vessel segmentation annotation method when executing the executable instructions.

[0088] The application further discloses a computer readable storage medium comprising:

[0089] a memory having a computer program stored thereon;

[0090] a processor configured to execute the program in the memory to implement the optical coherence tomography angiography blood vessel segmentation annotation method.

[0091] In summary, due to the adoption of the above technical solutions, the application has the following beneficial effects:

[0092] On one hand, the MONAI end-to-end segmentation model is combined to realize global blood vessel extraction, and on the other hand, the SAM large model is fine-tuned through the low rank adaptation (LoRA) technology to realize local fine segmentation; and through the Gradio front end and real-time interaction tools such as a brush and an eraser, the user can modify the preliminary result in real time. The method significantly improves the blood vessel segmentation accuracy and annotation efficiency, supports export of various standard formats, and can be deployed in a home-level GPU environment, and has low resource consumption.

[0093] Additional aspects and advantages of the application will be described in part below, some will become apparent from the following description, or will be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0094] The above and / or additional aspects and advantages of the application will become apparent and be readily understood from the following description, taken in conjunction with the accompanying drawings, in which:

[0095] Figure 1 is a method flowchart of the application.

[0096] Figure 2 is an image enhancement strategy effect schematic diagram of training / fine-tuning.

[0097] Figure 3 is a SAM-OCTA model structure schematic diagram after fine-tuning.

[0098] Figure 4 is a Gradio front end operation interface schematic diagram.

[0099] Figure 5 is a method flowchart of connected region analysis.

[0100] Figure 6 is a pre-annotation result optimization editing tool schematic diagram.

[0101] Figure 7 is a schematic diagram of editing connected blood vessel branches with a brush tool and updating the connected domain statistics in real time.

[0102] Figure 8 is a schematic diagram of implementing local blood vessel segmentation editing based on SAM-OCTA prompt points.

[0103] Figure 9 is a schematic diagram of blood vessel branch hierarchical screening based on a connected domain area threshold.

[0104] Figure 10 is a schematic diagram of a blood vessel annotation process interface based on Vimium keyboard shortcuts. DETAILED DESCRIPTION

[0105] Embodiments of the present application will be described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numbers represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation of the present application.

[0106] The present application aims to at least solve the problem of insufficient interactivity in OCTA image segmentation tools, and particularly innovatively proposes a blood vessel segmentation annotation tool based on an end-to-end deep learning model and a fine-tuned large model, providing an interactive management method for blood vessel segmentation prediction and annotation tasks.

[0107] In order to achieve the above-mentioned purpose of the present application, the present application provides a blood vessel segmentation annotation tool based on an end-to-end deep learning model and a fine-tuned large model, which includes the following steps and can be divided into two stages of model weight pre-training and data annotation, and the flow chart is as shown in Figures 1-10

[0108] 1. Model training and preparation

[0109] 1.1 Standardization processing

[0110] The OCTA-500 dataset is standardized, including image size adjustment, normalization and image enhancement. The standardized image is used to input into an existing segmentation model or a self-defined segmentation model.

[0111] The normalization formula is

[0112] wherein, I norm is the normalized image gray value;

[0113] I is the original image gray value;

[0114] μ is the mean value of the image; ​

[0115] σ is the standard deviation of the image.

[0116] Due to the limited sample size of OCTA dataset, data augmentation operation to enrich image representation is necessary. The augmentation process can be divided into three aspects, including:

[0117] Brightness and contrast adjustment: The brightness and contrast of OCTA images may vary due to acquisition conditions. Randomly adjusting brightness and contrast helps simulate image quality under different devices and environments. The brightness and contrast of the image will randomly change with a probability of 20%, and the formula is:

[0118] I' = max(0, min(1, (1 ± 0.15)I ± 0.25);

[0119] where I' is the image gray value after brightness and contrast adjustment;

[0120] max(0, *) is the lower limit truncation function, which ensures that the final result is not less than 0;

[0121] min(1, *) is the upper limit truncation function, which ensures that the intermediate calculation result does not exceed 1;

[0122] Geometric transformation: The distribution of blood vessel network varies in different patients, for example, the blood vessels in some areas may extend horizontally or vertically. By randomly rotating or vertically flipping within a range of ±15° with a probability of 25%, different blood vessel distribution directions are simulated to enhance the model's adaptability to geometric changes. The rotation formula is as follows:

[0123]

[0124] where the pixel coordinates of the image are (x, y), and the new coordinates (x', y') after rotation transformation;

[0125] where θ ~ U(-15°, 15°) is the rotation angle, which is subject to uniform distribution;

[0126] Pixels beyond the image boundary can be mirror filled along the boundary line.

[0127] The formula for horizontal flipping is x' = W - x, y' = y, and the formula for vertical flipping is x' = x, y' = H - y;

[0128] where H and W are the height and width of the image, respectively.

[0129] Blur and local distortion: OCTA images may be blurred due to device or acquisition process, which affects the identification of details, or locally deformed due to patient motion, light source changes or other disturbances, and there may be invalid data in some areas during the acquisition process (e.g. occlusion, artifacts or signal loss). For this reason, the method uses three kinds of enhancement methods: Gaussian convolution blur, local geometric distortion, and random occlusion, with a trigger probability of 0.1;

[0130] The formula is as follows:

[0131]

[0132] Where I(x+i, y+j) is the gray value of the original image at (x+i, y+j), and i and j represent the horizontal and vertical offsets, respectively;

[0133] I''(x, y) is the output gray value at (x, y) after filtering;

[0134] represents the Gaussian blur scenario, K(i, j) is the two-dimensional filter kernel weight, and satisfies the normalization condition ∑K(i, j) = 1;

[0135] Local geometric distortion:

[0136] Affine matrix parameters a 11 , a 22 are scaling parameters that determine the scaling ratio of the image in the horizontal and vertical directions, and take values a 11 , a 22 ~ U(0.9, 1.1);

[0137] Parameters a 12 , a 21 are shear parameters that allow pixels to be slightly misaligned along the tilt direction determined by these two parameters, and take values a 12 , a 21 ~ U(-0.1, 0.1), and the offsets t x and t y are random values in [-7, 7].

[0138] Random occlusion: 20 to 40 random occlusion regions are generated in the image, and the definition of the occlusion region R i is as follows:

[0139] R i = {(x, y) | x l ≤ x ≤ x l + w, y l ≤ y ≤ y l + h};

[0140] Where: (x l ,y l ) are the coordinates of the occluding rectangle, w and h are the width and height of the rectangle, satisfying w,h~U(8,12).

[0141] The image enhancement effect is as follows Figure 2 shown.

[0142] 1.2 Segmentation Model Architecture

[0143] Model structure-weight dual file mechanism:

[0144] Structure definition file (*.yaml)

[0145] The network topology is described using YAML syntax, including layer type, layer order, hyperparameters (convolution kernel size, step size, number of channels, activation function, etc.) and topology check hash.

[0146] Parsing process: The system calls yaml.safe_load() to read the file → generates an ordered layer configuration → dynamically builds the torch.nn.Module network based on the configuration.

[0147] Weight files (.pth / .pt)

[0148] Only the model parameter tensors and a checksum hash are stored.

[0149] Loading process: The system first parses the YAML and calculates the topology hash, then reads the weight file; execution is performed only when the two are consistent. If they are inconsistent, an exception is thrown and the user is prompted to re-match.

[0150] This method selects the following classic and commonly used segmentation model structures:

[0151] UNet: A classic segmentation model with a symmetrical encoder-decoder structure suitable for medical image processing.

[0152] DynUNet: Dynamically adjusts network depth and width to adapt to input images of different sizes.

[0153] AttentionUNet: An improved version of UNet that adds an attention mechanism to enhance feature extraction performance and is suitable for segmentation tasks with complex backgrounds.

[0154] SegResNet: Combined with residual network design, it enhances the learning ability of deep features and is suitable for high-resolution image segmentation.

[0155] SwinUNETR: A Transformer-based segmentation model that utilizes the local attention mechanism of Swin Transformer and is suitable for processing medical images with global contextual relationships.

[0156] 1.3 Model Training

[0157] During the model training process, the preprocessed OCTA image data was input, the training hyperparameter learning rate η was set to 0.001, the batch size was 16, and a total of 50 rounds of training were sufficient to make the model performance converge, and the Dice loss function was used. Optimize

[0158] Where Y is the predicted segmentation area and Y' is the manually labeled area. The optimizer is responsible for updating the model parameters to gradually reduce the loss function. It adjusts the model weights based on the gradient information to continuously improve the model performance. This method uses the Adam optimizer, and the update formula is θ t+1 is the current model weight parameter, is the current gradient.

[0159] In each round of training, the Dice coefficient and Intersection over Union (IoU) are used as evaluation indicators. The IoU calculation formula is:

[0160] After evaluating performance, save the best-performing model weight file for use in the subsequent inference phase. The evaluation process uses the Dice coefficient and IoU calculations as metrics on a small number of reserved validation set images (20 samples) after each training round. The metric values ​​are recorded to monitor changes in model performance. Based on the sum of the IoU and Dice coefficient, the best-performing model is selected and its weight file is saved or replaced.

[0161] In particular, since the input of SAM-OCTA involves two parts: image and cue points, SAM-OCTA uses the pre-trained Vision Transformer (ViT) as the image encoder and uses the Low-Rank Adaptation (LoRA) technique for fine-tuning. Specifically, an additional linear layer is inserted into each Transformer block of ViT as a multi-layer perceptron (similar to the ResNet block), in the form of:

[0162] W'=W+ΔW,ΔW=B LoRA ·A LoRA ;

[0163] Among them, W is the original weight matrix; B LoRA and A LoRA is a low-rank matrix;

[0164] (rank r<<min(d in ,dout) takes the value of 32 in this method,

[0165] AW is a low-rank update, the original weight W is frozen during training, and only B is updated LoRA and A LoRA , significantly reducing the number of trainable parameters. The process is shown in Figure 3

[0166] Fine-tuning of SAM-OCTA requires the generation of prompt inputs in conjunction with prompt points. Such prompt inputs are divided into global and local modes. The global mode, like a conventional segmentation model, generates virtual prompt coordinate encodings that avoid mapping to the image, thereby achieving global segmentation. The local mode uses eight-connection criteria and identifies blood vessel connected regions as positive regions, while using inflation methods to identify surrounding regions as negative regions. Each region randomly selects 1 to 6 positive points (target blood vessels) and 0 to 5 negative points (adjacent background). Positive points directly tell the model which pixels belong to the target to be segmented, helping the model quickly focus on the correct area, for example, marking on a certain blood vessel area, allowing the model to accurately identify the distribution of blood vessels. In OCTA images, blood vessels usually account for a small proportion. Positive points can ensure that the model will not ignore small blood vessels due to background dominance. Negative points explicitly indicate which regions should not be segmented, reducing false positives.

[0167] 2. Image input and preprocessing

[0168] 2.1 Upload method

[0169] Select image source - users can choose two ways to upload images. (1) Hard disk path upload: In the "image path" input box, directly fill in the folder path, the system will automatically check and read the image files under this path, and the supported formats include JPEG, PNG, and TIFF, etc. By clicking the previous / next, you can click to switch between different images. (2) Single image upload: Click the upload button (usually a file selection icon) in the interface, manually select the single image file to upload.

[0170] 2.2 Image viewing

[0171] After successful upload, the image will be automatically loaded and displayed in the thumbnail area on the left side of the interface. The loaded images will be arranged in order, making it easy for users to preview. Users can click on the thumbnail to view the specific image content. During this process, the background will automatically adjust the image to an input size of 512x512.

[0172] 3. Segmentation inference

[0173] ​For each OCTA image, due to the topological complexity of blood vessels, it is very complex and time-consuming to directly use binary brush annotation, so each image needs to be pre-annotated using a deep model before annotation to get the preliminary annotation result. This process is automatically completed when selecting the model through the interface, which includes image normalization and data format conversion from CPU to GPU required by the specific model. Users can choose between overall segmentation and local segmentation according to their needs.

[0174] 3.1 Normalize the data

[0175] Image data is transferred from memory-CPU-GPU, which is a process of copying tensors from host memory to video memory, aiming to accelerate computation using GPU. Specifically, first load the image using open-cv or Pillow library and convert it to a NumPy array, then use PyTorch to convert the NumPy array to a tensor. In this process, the shape of the image data needs to be adjusted from HWC ([height, width, channel]) to CHW format. After completing image prediction, the prediction results are transmitted back to the CPU as a NumPy array to be stored on the hard disk or displayed on the front end.

[0176] 3.2 Image segmentation

[0177] For overall segmentation, it is suitable for processing complete images without the need for users to manually specify the segmentation area, mainly for large-scale automated processing. According to the model name input by the user, select the appropriate MONAI segmentation model (such as SwinUNETR, UNet, or SegResNet) or custom model (and the corresponding model structure configuration file). At this time, you need to store the corresponding weights in the specified folder. If there are no available weights, use the identity mapping without substantive effect to avoid running errors.

[0178] 3.3 Activation function

[0179] After loading the model weights, you also need to add a layer of Sigmoid activation function to control the output form:

[0180] M'(x) = Sigmoid(M(x));

[0181] Where M(x) and M'(x) are the segmentation model before and after adding the activation layer;

[0182] Sigmoid() is the Sigmoid activation function.

[0183] The loaded model weights can be regarded as the model's understanding of the image, and adding a Sigmoid activation function to the model output layer can facilitate the calculation of the loss function during model training and the conversion of data form in prediction.

[0184] 3.4 Tensor Prediction

[0185] Use the loaded model to predict the image tensor, output the probability value tensor, and threshold the probability value output by the model to generate a binary segmentation result. The process can be expressed as:

[0186]

[0187] I tensor is the raw output tensor of the neural network segmentation model;

[0188] P binary is the segmentation mask matrix after binarization;

[0189] Convert the segmentation result from tensor form to NumPy format and amplify it to the [0,255] interval for subsequent processing. This operation is done by directly multiplying the amplification factor: P npy =P binary ×255.

[0190] 3.5 Connected Region Analysis

[0191] like Figure 5 As shown in the figure, the connected region analysis is performed on the binary segmentation result, different segmentation regions are marked, and the segmentation result is superimposed on the original image with a specific color to generate a visual preview. The connected region analysis process is as follows:

[0192] Step Z1 (initialization): Create an integer matrix C_labeled with the same size as the binary image P_binary, with an initial value of 0; set the label counter ID=1.

[0193] Step Z2 (pixel scanning): traverse P_binary in row and column order. When it is detected that P_binary[x, y] = 1 and C_labeled[x, y] = 0, execute step Z3.

[0194] Step Z3 (connected domain expansion):

[0195] Z3-1 pushes the current coordinate into the first-in-first-out queue Q;

[0196] Loop Z3-2: Get the first coordinate (x1, y1) and set C_labeled[x1, y1] = ID;

[0197] Z3-3 judges the eight neighboring pixels of (x1, y1) one by one: if they are within the image range and correspond to P_binary = 1 and C_labeled = 0, then push them into Q;

[0198] Z3-4 When Q is empty, the connected domain marking is completed, ID is incremented by 1, and step Z2 is returned.

[0199] Step Z4 (visual coloration): color mapping is performed on P_color[x, y] according to the label value, for example, color_index = (ID mod N), where N is the preset number of high-contrast colors.

[0200] Step Z5 (result superposition): the colored image P_color is overlaid on the original OCTA image with alpha transparency, and is updated to the interactive interface in real time for the user to review or further edit.

[0201] The above eight-neighborhood search is only a preferred example, and a person skilled in the art can use a 4-neighborhood or depth-first search to achieve the same function; color mapping is a visualization step and has no effect on segmentation accuracy.

[0202] The following color array with high contrast between adjacent elements is redefined for segmented assignment coloring:

[0203] [[0, 0, 255], [255, 0, 0], [0, 255, 255], [0, 255, 0], [255, 0, 255], [255, 255, 0], [0, 165, 255], [128, 0, 0], [100, 200, 100], [0, 0, 128], [200, 100, 200], [50, 205, 50], [203, 192, 255], [135, 206, 235], [42, 42, 128]]

[0204] The coloring method is to color in the color array according to the connected component value as the index, and when the index exceeds the data length, the index value is taken modulo operation, so that the topological relationship correctness of the segmentation result can be observed and checked.

[0205] 4. Interactive labeling optimization

[0206] The method provides a set of editing and repairing channels for pre-labeling results for labeling optimization, which is completed interactively through gradio front-end controls, and the interface provides zooming and dragging functions for users to conveniently view details.

[0207] 4.1 Visual optimization

[0208] To facilitate observation, the blood vessel foreground is represented in yellow and the background is represented in purple. Selection needs to be made from the following interactive tools, such as Figure 6The tools are shown: (1) brush tool: used to fill in the missing blood vessel area; (2) eraser tool: used to erase the misdetected area. (3) patch erasing tool: quickly filters and eliminates small connected components. Tool parameters (such as brush, eraser size) can be adjusted by sliders, or by the canvas buttons for undo and redo.

[0209] 4.2 Real-time update of connected regions

[0210] As shown in Figure 7 , during the modification, the modified area is updated in real time and superimposed into the segmentation result, and as the strokes increase or decrease, the connected region analysis is also updated in real time. As shown in Figure 8 , for SAM-OCTA, the hint points can be used for local segmentation of blood vessels during pre-labeling.

[0211] 4.3 Processing non-connected regions

[0212] As shown in Figure 9 , for the non-connected redundant branches existing in the image, a dynamic component screening method based on connected domain area characteristics and multi-level filtering strategy is used. First, the connectedness analysis algorithm is used to accurately identify the independent connected components in the image, and the geometric area of each connected component is calculated, and then according to the set area threshold, the redundant branches with small area or abnormal morphology are gradually removed, and the formula is expressed as:

[0213] Connected domain identification: C i = {p | connectedness (p) = i, i = 1, 2,..., N};

[0214] Where C i represents the ith connected component, p is a pixel point, and N is the total number of total connected components.

[0215] The area (pixel number) of each connected component is calculated:

[0216] Where A(C i ) represents the geometric area of the connected component C i .

[0217] Area-based hierarchical filtering: set the area threshold T and the morphological screening condition, and filter out the small fragment connected components:

[0218]

[0219] Where represents the component screened out.

[0220] 4.4 Save format

[0221] After optimization, users can save the final segmentation results in various standardized formats such as JSON, TIFF, or NIFTI to meet the needs of different application scenarios. These formats not only ensure the structuring and standardization of the segmentation results, but also facilitate direct use in subsequent analysis, data sharing, and other processing steps. For example, the JSON format is suitable for storing key points, boundary information, or metadata of segmentation, the TIFF format is suitable for high-quality image storage, and the NIFTI format is particularly common in the medical imaging field and can be used for three-dimensional data display and processing (layer sequence sample labeling).

[0222] 5. Plug-in extension

[0223] This tool focuses on professionalism and efficiency, and supports deep integration with the Vimium plug-in to achieve seamless experience of quick operations. Vimium is a browser plug-in designed to improve web browsing efficiency. It provides users with keyboard shortcuts similar to the Vim editor, allowing users to quickly navigate, open links, scroll pages, or perform other operations through the keyboard, reducing dependence on the mouse. Vimium supports browsers such as Chrome and Firefox, making it suitable for users who prefer keyboard operations. Through keyboard shortcuts, users can easily complete functions such as saving images, quickly switching between browsing images, and undoing editing operations, greatly improving user interaction efficiency. Combined with the keyboard navigation capabilities of Vimium, users can smoothly operate without relying on the mouse, demonstrating the outstanding performance of modern tools in productivity improvement. This design not only reflects the integration of high-end technology and user experience, but also provides professional users with accurate and efficient solutions. The binding of keyboard shortcuts is shown in Figure 9 .

[0224] 6. Deployment tool requirements

[0225] The deployment environment of this tool focuses on cost-effectiveness, fully meeting the model running requirements while maintaining the affordability of home-level configurations. The actual hardware resources required are as follows:

[0226] Graphics card memory: Monai models only require 4GB of video memory, and the highest demand for SAM-OCTA models is 12GB of video memory. Mainstream home graphics cards such as NVIDIA RTX 3060 can meet the requirements, with stable performance and reasonable prices.

[0227] Processor: The tool has relatively low requirements for CPU performance, and mainstream home-level processors such as Intel Core i5 or AMD Ryzen 5 series can meet the requirements.

[0228] Memory: The running environment has relatively relaxed memory requirements, and 16GB or 32GB of memory can run smoothly.

[0229] Operating System: Support 64-bit system, based on x64 architecture, mainstream home Windows system can meet the deployment requirements.

[0230] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to the embodiments without departing from the principles and spirit of the present application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for segmenting and labeling blood vessels in optical coherence tomography angiography, characterized in that: include: S1, model training and preparation S1-1, obtain the OCTA training dataset containing retinal vascular annotation information; S1-2, training the first segmentation model using an end-to-end deep learning framework to obtain a first model weight for global vessel segmentation; S1-3, based on the pre-trained visual Transformer encoder, the low-rank adaptation matrix is ​​inserted into its weight matrix and only the low-rank parameters are updated to complete the fine-tuning of the large model and obtain the second model weights for local blood vessel segmentation; S2, image input and preprocessing S2-1, receiving the OCTA image or image folder to be processed through a visual interactive interface; S2-2, performs format parsing, size normalization, and grayscale normalization on the received image to generate a standardized image tensor for inference; S3, segmentation reasoning S3-1, determining whether the segmentation mode is a global segmentation mode or a local segmentation mode according to the user's selection in the interactive interface; S3-2: If the global segmentation mode is selected, calling the first segmentation model corresponding to the first model weight to infer the normalized image tensor and output a first segmentation mask; S3-3, if the local segmentation mode is selected, obtaining the region of interest and prompt information indicated by the user on the image, calling the large model corresponding to the second model weight to infer the region of interest, and outputting a second segmentation mask; S4, interactive annotation optimization S4-1, displaying the segmentation mask superimposed on the original image in the interactive interface; S4-2, receiving an editing instruction issued by a user through a drawing, erasing or debris erasing tool, and updating the segmentation mask in real time based on the editing instruction; S5, result export S5-1, saving the interactively optimized segmentation mask into a file format compatible with medical imaging; S5-2, synchronously exporting the annotated dataset corresponding to the segmentation mask for subsequent model retraining or clinical analysis.

2. The method for segmenting and labeling blood vessels in optical coherence tomography angiography according to claim 1, characterized in that: The image input and preprocessing include: Standardize the OCTA-500 dataset, including image resizing, normalization, and image enhancement. The standardized images are then fed into existing or custom segmentation models. The normalization formula is Among them, I norm is the normalized image grayscale value; I is the grayscale value of the original image; μ is the mean of the image; σ is the standard deviation of the image; Due to the limited sample size of the OCTA dataset, data augmentation operations to enrich image representation are necessary. The enhancement process can be divided into three aspects, including: Brightness and contrast adjustment: The brightness and contrast of OCTA images may vary depending on the acquisition conditions. Randomly adjusting the brightness and contrast helps simulate the image quality under different devices and environments. The brightness and contrast of the image will change randomly with a probability of 20%. The formula is: I′=max(0,min(1,(1±0.15)I norm ±0.25); Where I′ is the grayscale value of the image after brightness and contrast adjustment; max(0,*) is the lower limit truncation function to ensure that the final result is not lower than 0; min(1,*) is the upper bound truncation function, ensuring that the intermediate calculation result does not exceed 1; By randomly rotating or flipping vertically or horizontally within the range of ±15° with a probability of 25%, different blood vessel distribution directions are simulated to enhance the model's adaptability to geometric changes. The rotation formula is as follows: Among them, the pixel coordinates of the image are (x, y), and the new coordinates after rotation transformation are (x′, y′); Among them, θ~U(-15°,15°) is the rotation angle and obeys uniform distribution; Pixels beyond the image boundary can be mirror-filled along the boundary line; The formula for horizontal flip is x'=Wx,y'=y, while the formula for vertical flip is x'=x,y'=Hy; Where H and W are the height and width of the image respectively; Use three enhancement methods: Gaussian convolution blur, local geometric perturbation, and random occlusion, with a trigger probability of 0.1; The formulas are as follows: Where I″(x,y) is the grayscale value output at (x,y) after filtering; I(x+i,y+j) is the grayscale value of the offset image at (x+i,y+j), where i and j represent the offset in the horizontal and vertical directions, respectively; Represents a Gaussian blurred scene, K(i,j) is the two-dimensional filter kernel weight, satisfying the normalization condition ∑K(i,j)=1; Local geometric perturbations: Affine matrix parameter a 11 , a 22 is the scaling parameter, which determines the scaling ratio of the image in the horizontal and vertical directions, and its value is a 11 ,a 22 ~U(0.9,1.1); Parameter a 12 , a 21 is the shearing parameter, which allows pixels to be slightly misaligned along the tilt direction determined by these two parameters, and its value is a 12 ,a 21 ~U(-0.1,0.1), offset t x and t y is a random integer value in [-7,7]; Random occlusion: Generate 20 to 40 random occlusion regions in the image. i Definition: R i ={(x,y)|x l ≤x≤x l +w,y l ≤y≤y l +h}; Among them, R i Indicates the occluded area; (x l ,y l ) is the occlusion area R i The coordinates of the rectangle are w and h, respectively, satisfying w,h~U(8,12); Model structure-weight dual file mechanism: Structure definition file; Use YAML syntax to describe the network topology, including layer type, layer order, hyperparameters, and topology check hash; Loading process: The system first parses the YAML and calculates the topology hash, then reads the weight file. Execution is only possible when the two are consistent. If they are inconsistent, an exception is thrown and the user is prompted to re-match. During the model training process, the preprocessed OCTA image data was input, the training hyperparameter learning rate η was set to 0.001, the batch size was 16, and a total of 50 rounds of training were sufficient to make the model performance converge, and the Dice loss function was used. Optimize; Among them, L Dice is the Dice loss function; Y is the predicted segmentation area; Y' is the manually marked area; |*| is a technical symbol for sets, used to indicate the total number of elements; ∩ is the intersection, which represents the overlapping part of two regions; The optimizer is responsible for updating the model parameters to gradually reduce the loss function. It adjusts the model weights based on the gradient information to continuously improve the model performance. The update formula is: θ t+1 is the model weight parameter after the next round of iteration; θ t is the current model weight parameter; η is the learning rate; is the current gradient; In each round of training, the Dice coefficient and intersection over union ratio are used as evaluation indicators, and the IoU calculation formula is: Among them, IoU is an indicator to measure the degree of overlap between two regions, and its value range is [0,1]. A is the pixel set corresponding to the predicted segmentation result; B is the set of pixels corresponding to the manually labeled true segmentation area; ∪ and ∩ represent the union and intersection respectively; In particular, since the input of SAM-OCTA involves two parts: image and cue points, SAM-OCTA uses the pre-trained Vision Transformer as the image encoder and fine-tunes it using low-rank adaptation technology. Specifically, an additional linear layer is inserted into each Transformer block of ViT as a multi-layer perceptron, in the form of: W′=W+ΔW,ΔW=B LoRA ·AND LoRA ; Among them, W′ is the new weight matrix obtained after low-rank fine-tuning; W is the original weight matrix; B LoRA and A LoRA is a low-rank matrix; ΔW is a low-rank update, freezing the original weight matrix W during training and only updating B LoRA and A LoRA , significantly reducing the amount of trainable parameters.

3. The method for segmenting and labeling blood vessels in optical coherence tomography angiography according to claim 1, characterized in that: The segmentation reasoning includes: Image data is transferred from memory to CPU and then to GPU. The process is to copy tensors from host memory to video memory and use GPU to accelerate calculations. After loading the model weights, you also need to add a layer of Sigmoid activation function to control the output form; Use the loaded model to predict the image tensor, output the probability value tensor, and threshold the probability value output by the model to generate a binary segmentation result; Convert the segmentation result from tensor form to NumPy format and amplify it to the [0,255] interval for subsequent processing. This operation is done by directly multiplying the amplification factor: P npy =P binary ×255; P npy It is the image matrix obtained by converting the binary mask into a NumPy array and mapping it to 0-255 grayscale; Connected region analysis: Perform connected region analysis on the binary segmentation results, mark different segmentation areas, and overlay the segmentation results on the original image with specific colors to generate a visual preview.

4. The method for segmenting and labeling blood vessels in optical coherence tomography angiography according to claim 1, wherein: The interactive annotation optimization includes: Update connected areas in real time: During the modification process, the modified area is updated in real time and re-superimposed on the segmentation result. As the number of strokes increases or decreases, the connected area analysis is also updated in real time. Dealing with disconnected areas: The connectivity analysis algorithm is used to accurately identify independent connected components in the image, and the geometric area of ​​each connected component is calculated hierarchically. Then, redundant branches with small areas or abnormal shapes are gradually eliminated according to the set area threshold. The formula is expressed as: Connected domain identification: C q ={p|connectivity(p)=q,q=1,2,...,N}; Among them, C i represents the i-th connected component, and p is a pixel point; q is the index of the connected component, N is the total number of connected components; Calculate the area of ​​each connected component: Among them, A(C q ) represents the connected component C q The geometric area of It represents the sum of the number of pixels of connected components, and the geometric area is expressed in terms of the number of pixels; Area-based hierarchical filtering: Set the area threshold T and morphological filtering conditions to filter out the fragmented connected components with smaller areas: in Indicates the components that passed the screening.

5. A computer system, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the optical coherence tomography angiography blood vessel segmentation and labeling method according to any one of claims 1 to 4 when executing the executable instructions.

6. A computer-readable storage medium, characterized in that include: a memory having a computer program stored thereon; A processor is configured to execute the program in the memory to implement the optical coherence tomography angiography blood vessel segmentation and labeling method according to any one of claims 1 to 4.

Images