Lightweight choroidal neovascularization OCT image segmentation method and system

Through the improved UNeXt network structure, the excellent convolution module and the deep feature extraction module are used, combined with multiple attention mechanisms, the problems of computational complexity and large parameters in the OCT image segmentation of choroidal neovascularization are solved, achieving efficient and accurate Image segmentation.

CN119006491AActive Publication Date: 2024-11-22SUZHOU UNIV

Patent Information

Application Number
CN202410929142.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-11
Publication Date
2024-11-22
Estimated Expiration
2044-07-11

AI Technical Summary

Technical Problem

The prior art has a large calculation complexity and large amount of parameters in the OCT image segmentation of choroidal neovascularization, resulting in low efficiency and difficulty in achieving accuracy in automatic segmentation.

Method used

Adopting the improved UNeXt network structure, replacing the depth separable convolution block as an excellent convolution block, and introducing a deep feature extraction module and a Tokenized MLP block, combining multiple attention mechanisms, the loss function is optimized to improve the performance of the segmentation model.

Benefits of technology

While reducing the calculation amount and parameter amount, the segmentation accuracy and efficiency of choroidal neovascular OCT images are significantly improved, and higher feature extraction ability and model generalization ability are achieved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006491A_ABST
    Figure CN119006491A_ABST
Patent Text Reader

Abstract

The invention discloses a lightweight choroidal neovascularization OCT image segmentation method and system, and relates to the technical field of image segmentation. The method comprises the following steps: receiving and preprocessing a choroidal neovascularization OCT image, and dividing image data into a training set and a test set; an improved segmentation model based on a UNeXt network is constructed, the improved segmentation model comprises an encoder, a decoder and a jump connection, and depth separable convolution blocks in the encoder and the decoder are replaced by excellent convolution modules. According to the method, the excellent convolution module is added behind the first three layers of encoders, features of different levels and different scales can be captured, useful information in a feature map can be fully mined, the introduced depth feature extraction module is fused with multiple attention mechanisms, the feature extraction capability and the model generalization capability can be remarkably improved, and the accuracy of feature extraction is improved. And meanwhile, an introduced excellent convolution module performs efficient and lightweight convolution operation on only part of channels, so that important features can be concerned, and computing resources can be saved at the same time.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image segmentation, and in particular to a lightweight choroidal neovascularization OCT image segmentation method and system. Background Art

[0002] Age-related macular degeneration (AMD) is one of the leading causes of visual impairment and blindness in the elderly. According to statistics, the number of AMD patients worldwide in 2020 is approximately 196 million, and it is expected to increase to 288 million by 2040. Choroidal neovascularization (CNV) is a hallmark of exudative AMD, and its development can lead to retinal damage and vision loss. Therefore, early detection and treatment of CNV is crucial to prevent vision loss. Optical coherence tomography (OCT) is a non-invasive imaging technology widely used in ophthalmology for the detection and diagnosis of CNV. The interpretation of OCT images usually relies on the subjective experience of doctors. In addition, manual CNV annotation is also a highly labor-intensive and time-consuming task. Therefore, there is a significant demand for automatic segmentation of choroidal neovascularization. However, the automatic segmentation of CNV still faces challenges due to the fuzzy boundaries of CNV, large morphological differences, speckle noise, and interference from other similar diseases.

[0003] To solve the above problems, the application of deep learning in medical image processing has gradually increased in recent years, especially in the classification of disease images and non-disease images. The high efficiency of convolutional neural networks (CNNs) in target detection, image recognition and segmentation has been widely recognized, proving its importance in OCT image analysis, especially the U-shaped structure network using convolutional neural networks, with its unique encoder-decoder structure and jump connection structure, densely integrating shallow features with deep features, and the amount of medical imaging data matches the volume of the UNet model. It has been widely used in various tasks. In addition, many Transformer-based networks have also been proposed for medical image segmentation because they learn a global understanding of the image, which helps segmentation. For example, TransUNet has become a leading medical image segmentation method.

[0004] However, traditional segmentation methods focus on improving the performance of the network, but the overall computational complexity, inference time or number of parameters are large. To this end, we propose a lightweight choroidal neovascularization OCT image segmentation method and system. Summary of the invention

[0005] The object of the present invention is to provide a lightweight choroidal neovascularization OCT image segmentation method and system, which can improve the accuracy and efficiency of segmentation while reducing the amount of calculation and the amount of parameters.

[0006] Embodiment 1:

[0007] To achieve the above object, the present invention provides the following technical solution: a lightweight choroidal neovascularization OCT image segmentation method, comprising the following steps:

[0008] Receiving choroidal neovascularization OCT images for preprocessing, and dividing the image data into a training set and a test set;

[0009] An improved segmentation model based on the UNeXt network is constructed. The improved segmentation model includes an encoder, a decoder, and skip connections. The depthwise separable convolutional blocks in the encoder and decoder are replaced with superior convolutional modules. A deep feature extraction module is introduced at the output of the first three layers of the encoder, and Tokenized MLP blocks are introduced in both the encoder and the decoder.

[0010] The improved segmentation model is trained using the training set, and the objective function of the improved segmentation model is optimized by minimizing the overall loss function. The parameters of the improved segmentation model are adjusted according to the validation set, and a new round of training is performed until an improved segmentation model that meets the standards is obtained.

[0011] The optimized improved segmentation model is used to identify and segment the ultrasound image to be identified.

[0012] Further, the choroidal neovascularization OCT images are received for preprocessing, and the image data are divided into a training set and a test set, as follows:

[0013] (1) Receive choroidal neovascularization OCT image data and manually annotate the diseased area of ​​the patient's OCT image;

[0014] (2) Randomly select some of the annotated images as training set samples and the remaining images as test set samples, perform data augmentation on the training set samples, and expand the training samples.

[0015] Furthermore, the encoder includes 5 layers, the first 3 layers are the same convolution blocks, the last two layers use Tokenized MLP blocks, and convolution downsampling is used between each layer to reduce the feature map.

[0016] Furthermore, each of the convolution blocks comprises a superior convolution module, a 2D convolution layer, a batch normalization layer and a ReLU activation function.

[0017] Furthermore, the superior convolution module includes partial convolution and phantom convolution. The superior convolution module divides the number of channels of the input feature map into two parts, one part is subjected to phantom convolution operation, and the other part remains unchanged, so as to concentrate computing resources on more important feature extraction on the basis of retaining part of the original information;

[0018] The phantom convolution generates more feature maps through cheap operations, which complement the original features at a lower computational cost. The whole process is specifically expressed as follows:

[0019] Y = Concat(X keep ,GhostConv(X ghost ))

[0020] Where X is the input feature map, X keep represents the channel that remains unchanged, X ghost Represents the Ghost channel, GhostConv(X ghost ) represents the phantom convolution operation on the phantom convolution channel, and Concat means concatenating the unchanged channel and the feature map after the phantom convolution to finally obtain the output feature map Y.

[0021] Furthermore, the deep feature extraction module includes a CBAM module and a parallel processing module. The parallel processing module is located at the output end of the CBAM module. The CBAM module applies a channel attention mechanism to calculate a channel attention map through global pooling and a fully connected layer, and then applies a spatial attention mechanism to calculate a spatial attention map through pooling and convolution to enhance the expressiveness of the feature map.

[0022] Furthermore, the parallel processing module includes an AG module and a SE module. The AG module enhances the features of the target area and suppresses background noise by calculating the attention coefficient. The SE module compresses the information of the spatial dimension into a global description through global average pooling and global maximum pooling, and then calculates the weight of the channel through the fully connected layer, and finally redistributes the weight to each channel of the feature map. The specific process is shown as follows:

[0023] F refined =(F CBAM ·σ(FC(GAP(F CBAM ))))+(F CBAM σ(ReLU(Conv 1×1 (F CBAM ))))

[0024] In the formula, F CBAM represents the feature map after processing by the CBAM module, σ(FC(GAP(F CBAM ))) represents the global average pooling and fully connected layer processing of the SE module, σ(ReLU(Conv 1×1 (F CBAM ))) represents the convolution and ReLU processing of the AG module, and the final F refined It is a fine feature map fused from both.

[0025] Furthermore, the decoder uses the decoding part of UNeXt to reconstruct the high-resolution feature map, which includes 5 layers, the first 2 layers are Tokenized MLP blocks for gradually restoring the feature resolution, and the last 3 layers are convolution blocks. The decoder uses bilinear interpolation for upsampling.

[0026] Furthermore, the BCEDiceLoss is used as the overall loss function, which is specifically the sum of the BCE Loss loss function and the Dice loss function. The BCE loss function (Binary Cross-Entropy Loss) is used for the binary classification problem, and its formula is as follows:

[0027]

[0028] The Dice loss function is mainly used to evaluate the overlap, and its formula is as follows:

[0029]

[0030] The BCEDiceLoss overall loss function is the weighted sum of the above two loss functions, and its formula is as follows:

[0031] BCEDiceLoss=BCE Loss+Dice Loss

[0032] In the formula, N represents the number of samples, y i represents the true label of the i-th sample, Represents the predicted value of the i-th sample.

[0033] According to another aspect of the present invention, the present invention provides a lightweight choroidal neovascularization OCT image segmentation system, comprising:

[0034] A receiving module, used for receiving choroidal neovascularization OCT images for preprocessing, and dividing the image data into a training set and a test set;

[0035] The model building module is used to build an improved segmentation model based on the UNeXt network. The improved segmentation model includes an encoder, a decoder, and skip connections. The depthwise separable convolutional blocks in the encoder and decoder are replaced with superior convolutional modules. A deep feature extraction module is introduced at the output of the first three layers of the encoder, and Tokenized MLP blocks are introduced in both the encoder and the decoder.

[0036] A training module is used to train the improved segmentation model using the training set, optimize the objective function of the improved segmentation model by minimizing the overall loss function, and adjust the parameters of the improved segmentation model according to the validation set, and conduct a new round of training until an improved segmentation model that meets the standards is obtained;

[0037] The segmentation output module is used to recognize and segment the ultrasound image to be recognized by using the optimized improved segmentation model.

[0038] The present invention has at least the following beneficial effects:

[0039] 1. The present invention can capture features of different levels and scales by adding an excellent convolution module after the first three layers of encoders, fully tap the useful information in the feature map, and the introduced deep feature extraction module integrates multiple attention mechanisms, which can significantly improve the feature extraction capability and the generalization ability of the model;

[0040] 2. The superior convolution module introduced in the present invention can save computing resources while focusing on important features by performing efficient and lightweight convolution operations on only some channels.

[0041] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a structural schematic diagram of the improved segmentation model in the present invention;

[0043] Figure 2 It is a structural schematic diagram of the deep feature extraction module in the present invention;

[0044] Figure 3 Schematic diagram of the structure of the excellent convolution module in the present invention;

[0045] Figure 4 This is a comparison chart of the segmentation results of the method proposed in the present invention and other methods. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0047] See also Figure 1-4 The present invention provides a technical solution: a lightweight choroidal neovascularization OCT image segmentation method, comprising the following steps:

[0048] S1. Receive choroidal neovascularization OCT images for preprocessing and divide the image data into a training set and a test set, as follows:

[0049] (S1.1) receiving choroidal neovascularization OCT image data and manually annotating the diseased area of ​​the patient's OCT image;

[0050] (S2.2) Randomly select some of the annotated images as training set samples and the remaining images as test set samples, perform data enhancement processing on the training set samples, and expand the training samples;

[0051] It should be noted that the CNV optical coherence tomography data used in this embodiment comes from Shanghai First People's Hospital. The OCT data includes 1070 OCT B-ultrasound images of 63 CNV patients. The images are manually annotated by experienced retinal experts to ensure the accuracy and reliability of data annotation.

[0052] Furthermore, data augmentation processing is performed on the training set samples, including horizontal flipping, vertical flipping, and random rotation, to alleviate the overfitting phenomenon of the deep neural network;

[0053] S2. Build an improved segmentation model based on the UNeXt network. The improved segmentation model includes an encoder, a decoder, and skip connections. The depthwise separable convolutional blocks in the encoder and decoder are replaced with superior convolutional modules. A deep feature extraction module is introduced at the output of the first three layers of the encoder, and Tokenized MLP blocks are introduced in both the encoder and the decoder.

[0054] For the technical solution of this embodiment, the encoder uses the encoding part of UNeXt as the feature extractor, which includes 5 stages. The first 3 stages are composed of the same convolutional blocks, and the last two stages use Tokenized MLP blocks. This module converts convolutional features into tags and processes them through MLP to achieve efficient feature representation and extraction. Convolution downsampling is used between each stage to reduce the feature map, thereby increasing the receptive field of convolution;

[0055] Furthermore, each convolutional block contains a superior convolutional module, a convolutional layer, a batch normalization layer, and a ReLU activation function;

[0056] The superior convolution module (SConv) includes partial convolution and ghost convolution. Based on the idea of ​​partial convolution, the superior convolution module (SConv) divides the number of channels of the input feature map into two parts: one part is subjected to ghost convolution operation, and the other part remains unchanged. This division allows computing resources to be concentrated on more important feature extraction while retaining some original information. Ghost Convolution generates more feature maps through cheap operations (such as point-by-point convolution). These feature maps supplement the original features at a lower computing cost. The whole process can be described by the following formula:

[0057] Y = Concat(X keep ,GhostConv(X ghost ))

[0058] Where X is the input feature map, X keep represents the channel that remains unchanged, X ghost Represents the Ghost channel, GhostConv(X ghost ) represents the Ghost convolution operation on the Ghost channel, and Concat means concatenating the unchanged channel and the feature map after Ghost convolution to finally obtain the output feature map Y;

[0059] In order to improve the flexibility and adaptability of the model, the improved model provides two forward propagation methods (forward_slicing and forward_split_cat), which are suitable for different application scenarios and needs. Forward_slicing is used during inference and only performs Ghost Convolution operations on the segmented channels; forward_split_ca can be used during training and inference to rearrange the input features to adapt to the Ghost Convolution operation. Through these design methods, SConv reduces the demand for computing resources while maintaining or even improving model performance;

[0060] According to the technical solution of this embodiment, the deep feature extraction module includes a CBAM module and a parallel processing module. The parallel processing module includes an AG module and an SE module. The feature map extracted by the convolution layer is processed by the CBAM module and then processed by the parallel structure of the AG and SE modules. Different types of attention mechanisms are used to enhance the features of the target area, adjust the channel weights, and consider the attention of the channel and spatial dimensions at the same time, so as to fully mine the useful information in the feature map. Specifically,

[0061] (1) The CBAM module first applies the channel attention mechanism to calculate the channel attention map through global pooling and fully connected layers, and then applies the spatial attention mechanism to calculate the spatial attention map through pooling and convolution to further enhance the expressiveness of the feature map;

[0062] (2) The feature map processed by CBAM is then processed by the parallel structure of the AG and SE modules. The AG module enhances the features of the target area and suppresses background noise by calculating the attention coefficient, while the SE module compresses the information of the spatial dimension into a global description through global average pooling and global maximum pooling. The weight of the channel is then calculated through the fully connected layer, and finally the weight is redistributed to each channel of the feature map. This can enhance the features of important channels and suppress unimportant channels.

[0063] The entire processing flow is described by the following formula:

[0064] F refined =(F CBAM ·σ(FC(GAP(F CBAM ))))+(F CBAM σ(ReLU(Conv 1×1 (F CBAM ))))

[0065] In the formula, F CBAM represents the feature map after processing by the CBAM module, σ(FC(GAP(F CBAM ))) represents the global average pooling and fully connected layer processing of the SE module, σ(ReLU(Conv 1×1 (F CBAM ))) represents the convolution and ReLU processing of the AG module, and the final F refined It is a fine feature map fused from both;

[0066] Specifically, by combining CBAM, AG and SE modules, the deep feature extraction module (DEM) module can comprehensively utilize three different attention mechanisms to process and enhance feature maps from multiple perspectives. In addition, the combination of various attention mechanisms helps to improve the generalization ability of the model; the CBAM module considers the attention of channel and spatial dimensions at the same time, enhancing the overall expression ability of the feature map; the AG module enhances the characteristics of the target area, so that the model focuses more on the key area when processing different images; the SE module adjusts the channel weights to make the model better adapt to different channel characteristics;

[0067] Although multiple attention mechanisms are introduced, the DEM block is designed to minimize computational complexity through parallel structure and effective weight calculation method. After the preliminary processing of the CBAM module, the results are fused into the AG and SE modules. This design not only improves the effect of feature extraction, but also maintains high computational efficiency;

[0068] According to the technical solution of this embodiment, the decoder uses the decoding part of UNeXt to reconstruct the high-resolution feature map, which includes 5 stages. The first two stages are Tokenized MLP blocks, which are used to gradually restore the feature resolution. The last three stages are convolution blocks, which are the same as the convolution blocks in the encoder. Each convolution block contains a superior convolution module, a convolution layer, a batch normalization layer and a ReLU activation function. The decoder uses bilinear interpolation for upsampling to increase the size of the feature map.

[0069] S3. Use the training set to train the improved segmentation model, minimize the overall loss function as the optimization objective function of the improved segmentation model, and adjust the parameters of the improved segmentation model according to the validation set, and conduct a new round of training until an improved segmentation model that meets the standards is obtained;

[0070] For the technical solution of this embodiment, BCEDiceLoss is used as the overall loss function, which is specifically the sum of the BCELoss loss function and the Dice loss function. The BCE loss function (Binary Cross-Entropy Loss) is used for the binary classification problem, and its formula is as follows:

[0071]

[0072] The Dice loss function is mainly used to evaluate the overlap, and its formula is as follows:

[0073]

[0074] BCEDiceLoss is the weighted sum of the above two loss functions, and its formula is as follows:

[0075] BCEDiceLoss=BCE Loss+Dice Loss

[0076] In the formula, N represents the number of samples, y i represents the true label of the i-th sample, Represents the predicted value of the i-th sample. The BCEDiceLoss loss function combines the binary cross entropy loss and the Dice loss. By adding these two loss functions, it can better handle the problem of class imbalance and improve the accuracy of the segmentation task.

[0077] It should be noted that the loss function may also adopt the standard BCEWithLogitsLoss, which is not limited in this embodiment. According to the configuration, the corresponding loss function is selected and CUDA is used for acceleration;

[0078] Furthermore, the improved segmentation model of this embodiment is based on the PyTorch deep learning framework, the GPU configuration is NVIDIA RTX4090 with 24G memory, a total of 300 cycles are trained, a multivariate learning rate strategy is used, the initial learning rate is 1e-4, and the batch_size value during training is set to 16; during training, the Adam optimizer is used to optimize the model parameters, and the learning rate is gradually reduced through the CosineAnnealingLR scheduler;

[0079] S4. Use the optimized improved segmentation model to identify and segment the ultrasound image to be identified.

[0080] In order to quantitatively evaluate the performance of the method proposed in this embodiment, two commonly used evaluation indicators in medical image segmentation, Dice coefficient (DSC) and intersection over union (IoU), are used as the measurement criteria for the experimental results. DSC is a function that evaluates the similarity between the predicted results and the manual annotations, and is defined as:

[0081]

[0082] TP (True Positive) is the number of true positive pixels, FP (False Positive) is the number of false positive pixels, and FN (False Negative) is the number of false negative pixels.

[0083] The intersection-over-union ratio is the overlap of two regions divided by the combined region of the two regions, and is defined as:

[0084]

[0085] Where A is the predicted area and B is the true area. |A∩B| represents the intersection area of ​​the predicted segmented area and the true segmented area (i.e. the number of overlapping pixels), and |A∪B| represents the union area of ​​the predicted segmented area and the true segmented area (i.e. the total number of pixels, including all pixels in the predicted and true areas).

[0086] In order to prove the effectiveness of the module, the UNeXt model is selected as the basic network. The corresponding ablation experiment results are shown in Table 1. The results show that the DEM and SConv modules proposed in this embodiment are effective. As shown in Table 1, starting from the original model, and then adding the DEM module, the segmentation accuracy has increased, which shows that the DEM module enhances the feature extraction capability. Then, the Funnel module is added to the model, and the Dice coefficient and IoU are further improved, which shows the advantage of the Funnel module in further refining features. Finally, the SConv module is added to the improved model, and the Dice coefficient and IoU reach 0.8790 and 0.7846 respectively, indicating that the SConv module significantly improves the accuracy and robustness of feature extraction; the number of channels is a major hyperparameter, which affects the number of parameters, complexity and performance of the model. C1 to C5 represent the number of channels of different blocks. Usually, we set C1 = 32, C2 = 64, C3 = 128, C4 = 160 and C5 = 256.

[0087] In addition, by setting the number of channels in each block, the number and complexity of model parameters can be flexibly controlled to adapt to different tasks and datasets.

[0088] This example compares the proposed method with TransUnet and UNet++. The segmentation results of different methods are shown in Figure 4 As shown in Figure 2, the method of this embodiment achieves more accurate results than other methods, while also reducing the number of parameters and the amount of calculation. Table 2 shows the quantitative segmentation results of different methods. The results show that the method of this embodiment has the highest Dice coefficient and IoU coefficient for all CNV segmentations.

[0089] Table 1 Comparison of segmentation results of ablation experiments

[0090]

[0091] Table 2 Performance comparison with SOTA medical image segmentation model

[0092]

[0093] In summary, the lightweight choroidal neovascularization OCT image segmentation method based on the improved UNeXt structure proposed in this embodiment can improve the accuracy and efficiency of segmentation while reducing the amount of calculation and the amount of parameters.

[0094] Embodiment 2:

[0095] The present invention proposes a lightweight choroidal neovascularization OCT image segmentation system, comprising:

[0096] A receiving module, used for receiving choroidal neovascularization OCT images for preprocessing, and dividing the image data into a training set and a test set;

[0097] The model building module is used to build an improved segmentation model based on the UNeXt network. The improved segmentation model includes an encoder, a decoder, and skip connections. The depthwise separable convolutional blocks in the encoder and decoder are replaced with superior convolutional modules. A deep feature extraction module is introduced at the output of the first three layers of the encoder, and Tokenized MLP blocks are introduced in both the encoder and the decoder.

[0098] A training module is used to train the improved segmentation model using the training set, optimize the objective function of the improved segmentation model by minimizing the overall loss function, and adjust the parameters of the improved segmentation model according to the validation set, and conduct a new round of training until an improved segmentation model that meets the standards is obtained;

[0099] The segmentation output module is used to recognize and segment the ultrasound image to be recognized by using the optimized improved segmentation model.

[0100] Specifically, the above-mentioned receiving module, model building module, training module and segmentation output module can be embedded in a computer processing system. The computer calls the above-mentioned modules to complete the task of equivalent modeling based on the above-mentioned lightweight choroidal neovascularization OCT image segmentation method; the receiving module, model building module, training module and segmentation output module can perform operations according to the specific steps given in the above-mentioned lightweight choroidal neovascularization OCT image segmentation method.

[0101] It should be noted that it should be understood that the division of the various modules of the above system is only the division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated, and these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. For example, the receiving module can be a separately established processing element, or it can be integrated in a certain chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and a certain processing element of the above device calls and executes the functions of the above signal processing module. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by the hardware integrated logic circuit in the processor element or the instructions in the form of software.

[0102] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital singnal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module above is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0103] It should be noted that, in this embodiment, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0104] For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. When an element is referred to as "assembled on", "installed on", "fixed on" or "arranged on" another element, it can be directly on another element or there can also be a centered element. When an element is considered to be "connected" to another element, it can be directly connected to another element or there may be a centered element at the same time. The terms "vertical", "horizontal", "up", "down", "left", "right" and similar expressions used in the present embodiment are only for illustrative purposes and are not intended to be the only implementation method.

[0105] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0106] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

Claims

1. A lightweight choroidal neovascularization OCT image segmentation method, characterized in that: The following steps are involved: Receiving choroidal neovascularization OCT images for preprocessing, and dividing the image data into a training set and a test set; An improved segmentation model based on the UNeXt network is constructed. The improved segmentation model includes an encoder, a decoder, and skip connections. The depthwise separable convolutional blocks in the encoder and decoder are replaced with superior convolutional modules. A deep feature extraction module is introduced at the output of the first three layers of the encoder, and Tokenized MLP blocks are introduced in both the encoder and the decoder. The improved segmentation model is trained using the training set, and the objective function of the improved segmentation model is optimized by minimizing the overall loss function. The parameters of the improved segmentation model are adjusted according to the validation set, and a new round of training is performed until an improved segmentation model that meets the standards is obtained. The optimized improved segmentation model is used to identify and segment the ultrasound image to be identified.

2. The lightweight choroidal neovascularization OCT image segmentation method according to claim 1, characterized in that: The choroidal neovascularization OCT images are received for preprocessing, and the image data is divided into a training set and a test set, as follows: (1) Receive choroidal neovascularization OCT image data and manually annotate the diseased area of ​​the patient's OCT image; (2) Randomly select some of the annotated images as training set samples and the remaining images as test set samples, perform data augmentation on the training set samples, and expand the training samples.

3. The lightweight choroidal neovascularization OCT image segmentation method according to claim 2, characterized in that: The encoder includes 5 layers, the first 3 layers are the same convolutional blocks, the last two layers use Tokenized MLP blocks, and convolution downsampling is used between each layer to reduce the feature map.

4. The lightweight choroidal neovascularization OCT image segmentation method according to claim 3, characterized in that: Each of the convolution blocks includes a superior convolution module, a 2D convolution layer, a batch normalization layer, and a ReLU activation function.

5. The lightweight choroidal neovascularization OCT image segmentation method according to claim 4, characterized in that: The superior convolution module includes partial convolution and phantom convolution. The superior convolution module divides the number of channels of the input feature map into two parts, one part is subjected to phantom convolution operation, and the other part remains unchanged, so as to concentrate computing resources on more important feature extraction on the basis of retaining part of the original information; The phantom convolution generates more feature maps through cheap operations, which complement the original features at a lower computational cost. The whole process is specifically expressed as follows: Y=Concat(X keep ,GhostConv(X ghost )) Where X is the input feature map, X keep represents the channel that remains unchanged, X ghost Represents the Ghost channel, GhostConv(X ghost ) represents the phantom convolution operation on the phantom convolution channel, and Concat means concatenating the unchanged channel and the feature map after the phantom convolution to finally obtain the output feature map Y.

6. The lightweight choroidal neovascularization OCT image segmentation method according to claim 5, characterized in that: The deep feature extraction module includes a CBAM module and a parallel processing module. The parallel processing module is located at the output end of the CBAM module. The CBAM module applies a channel attention mechanism to calculate a channel attention map through global pooling and a fully connected layer, and then applies a spatial attention mechanism to calculate a spatial attention map through pooling and convolution to enhance the expressiveness of the feature map.

7. The lightweight choroidal neovascularization OCT image segmentation method according to claim 6, characterized in that: The parallel processing module includes an AG module and a SE module. The AG module enhances the features of the target area and suppresses background noise by calculating the attention coefficient. The SE module compresses the information of the spatial dimension into a global description through global average pooling and global maximum pooling, and then calculates the weight of the channel through the fully connected layer. Finally, the weight is redistributed to each channel of the feature map. The specific process is shown as follows: F refined =(F CBAM ·σ(FC(GAP(F CBAM ))))+(F CBAM ·σ(ReLU(Conv 1×1 (F CBAM )))) In the formula, F CBAM represents the feature map after processing by the CBAM module, σ(FC(GAP(F CBAM ))) represents the global average pooling and fully connected layer processing of the SE module, σ(ReLU(Conv 1×1 (F CBAM ))) represents the convolution and ReLU processing of the AG module, and the final F refined It is a fine feature map fused from both.

8. The lightweight choroidal neovascularization OCT image segmentation method according to claim 5, characterized in that: The decoder uses the decoding part of UNeXt to reconstruct the high-resolution feature map, which contains 5 layers. The first 2 layers are TokenizedMLP blocks for gradually restoring the feature resolution, and the last 3 layers are convolution blocks. The decoder uses bilinear interpolation for upsampling.

9. The lightweight choroidal neovascularization OCT image segmentation method according to claim 1, characterized in that: BCEDiceLoss is used as the overall loss function, which is the sum of the BCE Loss loss function and the Dice loss function. The BCE loss function (Binary Cross-EntropyLoss) is used for binary classification problems, and its formula is as follows: The Dice loss function is mainly used to evaluate the overlap, and its formula is as follows: The BCEDiceLoss overall loss function is the weighted sum of the above two loss functions, and its formula is as follows: BCEDiceLoss=BCE Loss+Dice Loss In the formula, N represents the number of samples, y i represents the true label of the i-th sample, Represents the predicted value of the i-th sample.

10. A lightweight choroidal neovascularization OCT image segmentation system, characterized in that: include: A receiving module, used for receiving choroidal neovascularization OCT images for preprocessing, and dividing the image data into a training set and a test set; The model building module is used to build an improved segmentation model based on the UNeXt network. The improved segmentation model includes an encoder, a decoder, and skip connections. The depthwise separable convolutional blocks in the encoder and decoder are replaced with superior convolutional modules. A deep feature extraction module is introduced at the output of the first three layers of the encoder, and Tokenized MLP blocks are introduced in both the encoder and the decoder. A training module is used to train the improved segmentation model using the training set, optimize the objective function of the improved segmentation model by minimizing the overall loss function, and adjust the parameters of the improved segmentation model according to the validation set, and conduct a new round of training until an improved segmentation model that meets the standards is obtained; The segmentation output module is used to recognize and segment the ultrasound image to be recognized by using the optimized improved segmentation model.

Citation Information

Patent Citations

  • Method and system for segmenting choroidal neovascularization from fundus OCT image

    CN111292338A

  • Eye fundus image blood vessel segmentation method based on improved U-Net model

    CN116740080A

Cited By

  • Coal mine low-illumination image enhancement method and system based on multi-scale feature interaction and application

    CN120876344A