OCT image choroid segmentation network system and method based on feature integration method

Abstract

The embodiment of the application provides an OCT image choroid segmentation network system and method based on a feature integration method, a network model comprises a preprocessing module, a feature extraction module, a feature fusion module and a secondary feature fusion module, firstly, an anisotropic diffusion filtering algorithm is used for filtering in the preprocessing module, and an exponential and linear enhancement method is used for enhancement; then, a residual Inception structure is used in the feature extraction module, different size receptive fields are fused to extract rich feature expressions of the image, a pyramid pooling is used in the feature fusion process to further acquire context information, a channel attention module pays more attention to the relationship between channels, extracts a channel weighting vector of a feature map, acquires the importance of each channel, and a secondary feature fusion process fuses the last layer of the feature encoder with the last layer result of the feature encoder, and loss is calculated. The application effectively improves the performance of choroid segmentation, and shows superiority compared with other existing methods.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image segmentation, in particular to an OCT image choroid segmentation network system and method based on a feature integration method. BACKGROUND

[0002] Myopia and high myopia are global public health problems, and high myopia patients account for 0.5%-5.0% of the global population. The incidence of myopia in China is high, and the prevalence of high myopia in children is 6.5%-38.4%, and there is a trend of younger onset. With the increase of myopic refractive number and axial length, the thickness of choroid decreases, and this relationship is particularly evident in patients with high myopia. Therefore, quantitative measurement and analysis of choroid is essential and can be beneficial to the understanding of many diseases.

[0003] The use of high-resolution optical coherence tomography (OCT) images provides the possibility for clinicians to analyze 3D retinal and choroidal tissue. Although most commercial OCT instruments provide tools for automatic layered analysis, commercial software may not be able to segment all layers of interest, and clinicians repeatedly observe and diagnose, which is time-consuming and subjective. It is necessary to develop an automatic segmentation algorithm to segment the choroid layer to assist doctors in making more accurate judgments and provide reliable basis for clinical diagnosis and pathological research.

[0004] At present, the method of automatic segmentation of choroid layer mainly includes the following two types:

[0005] I. Based on traditional algorithm. Traditional segmentation methods include threshold-based segmentation method, region-based segmentation method, edge-based segmentation method and clustering or graph-based segmentation method, etc., which mainly rely on manual parameter setting and are usually low in efficiency, introduce noise in the segmentation process, and are prone to under-segmentation or over-segmentation results, and cannot cope with the segmentation of pathological eyes.

[0006] II. Based on deep learning method. Compared with surface learning relying on mathematical model, deep learning reduces human interference, can learn more abstract and high-dimensional features, and has excellent feature learning ability. From the beginning of using deep neural network to detect the upper and lower boundaries of the choroid to segment the choroid, to the end-to-end network based on encoder-decoder type, many strategies and methods have been developed to integrate and aggregate multi-level deep features for better segmentation. However, most networks focus on the relationship between features of the same level and ignore the relationship between layers; various segmentation models based on U-Net as the backbone network will also lose spatial information due to continuous convolution and pooling operations, resulting in discontinuous segmentation boundaries.

[0007] Therefore, there is an urgent need for a network model and method capable of accurately segmenting choroid in an OCT image. SUMMARY

[0008] The application aims to provide an OCT image choroid segmentation network system and method based on a feature integration method, effectively improving the performance of choroid segmentation and showing superiority compared with other existing methods.

[0009] To achieve the above-mentioned purpose, the application provides an OCT image choroid segmentation network model and method based on a feature integration method, comprising a preprocessing module, a feature extraction module, a feature fusion module, and a secondary feature fusion module.

[0010] The preprocessing module is used for preprocessing training and test images; the feature extraction module is used for extracting multi-scale features of the choroid layer of the OCT image from multiple depth levels, and the feature extraction module comprises a feature encoder and a feature decoder; the feature fusion module is embedded in the feature extraction module, used for better aggregating multi-level features, and outputting multi-level prediction probability maps; and the secondary feature fusion module fuses the last layer of the feature encoder and the result of the last layer of the feature encoder, and counts the loss.

[0011] The feature encoder is composed of residual Inception modules, and there are five layers in total, the convolution kernel sizes of the first three layers are 3x3, 5x5, and 7x7, the convolution kernel sizes of the last two layers are 1x1, 3x3, and 5x5, and then a 1x1 convolution is performed, and the channel number is doubled and the image size is reduced by one after each layer.

[0012] The feature decoder is composed of residual Inception modules, and the convolution is the same as the feature encoder, and the channel number is reduced by one and the image size is increased by one after each layer, and finally the original image size is restored.

[0013] The feature fusion module is composed of a channel attention module and a pyramid pooling module and is embedded in the feature extraction module; the channel attention module adopts a pooling operation to extract a channel weighting vector of a feature map and obtain the importance of each channel; and the pyramid pooling module integrates different sizes of pooling operations as global features.

[0014] The secondary feature fusion module fuses the last layer of the feature encoder module and the result of each layer of the feature decoder, and finally outputs a prediction map of the entire network model.

[0015] A deep supervision loss function similar to HED is adopted, wherein the training loss is defined as:

[0016]

[0017] where M = 6 means the last layer of the feature encoder shares six layers with the feature decoder, and m means the number of layers of the feature encoder and the feature decoder, is the loss of the last layer of the feature encoder and each layer of the feature decoder, and l fuse is the loss of the final fusion output prediction map, and w fuse is the weight of each loss term, and for each term l, the loss is calculated using the standard binary cross-entropy:

[0018]

[0019] where (r, c) is the pixel coordinate, and (H, W) is the image size: height and width. P G(r,c) and P S(r,c) represent the pixel values of the GT and the predicted saliency probability map, respectively, and the training process tries to minimize the total loss L of formula (1), and in the testing process, the fusion output l fuse is selected as the final prediction map.

[0020] Preferably, the pre-processing module is used for pre-processing the training and test set images, specifically comprising:

[0021] The training and test set images are flattened and cropped to the effective area of the choroid, filtered using an anisotropic diffusion filtering algorithm, enhanced using an exponential and linear enhancement method, scaled to an appropriate size and then randomly flipped and mirrored to obtain the processed images.

[0022] Preferably, the method comprises the following steps:

[0023] S1, a certain number of OCT images are collected to construct a data set, and the data set is divided into a training set and a test set;

[0024] S2, an OCT image choroid segmentation network model based on a feature integration method is constructed;

[0025] S3, the training set and the test set data are input into the network model for training, and a trained OCT image choroid segmentation network model based on the feature integration method is obtained;

[0026] S4, the segmented image is input into the trained OCT image choroid segmentation model based on the feature integration method, and a choroid segmentation map is obtained.

[0027] Preferably, the construction method of the OCT image choroid segmentation network model based on the feature integration method comprises:

[0028] The residual Inception module is built, an encoder-decoder structure network is constructed, a channel attention module is added to a skip connection part of the network, a pyramid pooling module is added to an up-sampling part of the network, and a secondary feature fusion module is added after a decoder of the network.

[0029] A computer storage medium has a computer software product, which includes a plurality of instructions to make a computer device execute the OCT image choroid segmentation method of the base feature integration method in claim 3.

[0030] Therefore, the OCT image choroid segmentation network system and method based on the feature integration method have the following technical effects:

[0031] (1) Firstly, an anisotropic diffusion filtering algorithm is used for filtering in the preprocessing module, and an exponential and linear enhancement method is used for enhancement;

[0032] (2) Then, the residual Inception structure is used in the feature extraction module, different sizes of receptive fields are fused to extract rich feature expressions of the image, the pyramid pooling is used in the feature fusion process to further obtain context information, the channel attention module pays more attention to the relationship between channels, extracts a channel weighting vector of a feature map, and obtains the importance of each channel, and the secondary feature fusion process fuses the last layer of the feature encoder and the result of the last layer of the feature encoder, and counts the loss.

[0033] (3) The application effectively improves the performance of choroid segmentation and shows superiority compared with other existing methods.

[0034] The technical solutions of the application will be further described in detail below with the help of the drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 It is a structure schematic diagram of the OCT image choroid segmentation network model based on the feature integration method;

[0036] Figure 2 It is a residual Inception structure schematic diagram in the feature extraction module;

[0037] Figure 3 It is a pyramid pooling module structure schematic diagram in the feature fusion module;

[0038] Figure 4 It is a channel attention module structure schematic diagram in the feature fusion module;

[0039] Figure 5 It is an OCT choroid image;

[0040] Figure 6 a segmentation result image of the OCT image choroid segmentation network based on the feature integration method. DETAILED DESCRIPTION

[0041] The technical solutions of the present application are further described below by means of the accompanying drawings and examples.

[0042] Unless otherwise defined, technical or scientific terms used in the present application shall have the same meaning as those commonly understood by one of ordinary skill in the art to which the present application belongs.

[0043] It will be apparent to those skilled in the art that the present application is not limited to the details of the above-exemplified embodiments, but that the present application can be implemented in other concrete forms without departing from the spirit or essential characteristics of the present application. Thus, the embodiments are to be considered in all respects as illustrative and not restrictive, the scope of the present application being indicated by the appended claims rather than by the above description, and it is intended that all changes and modifications which come within the meaning and range of equivalency of the claims are reserving to the present application. Therefore, no matter from which point of view, the embodiments should be considered as exemplary and non-limiting, and the scope of the present application is defined by the appended claims, not by the above description, and it is intended that all changes and modifications which come within the meaning and range of equivalency of the claims are reserving to the present application. Therefore, no matter from which point of view, the embodiments should be considered as exemplary and non-limiting, and the scope of the present application is defined by the appended claims, not by the above description, and it is intended that all changes and modifications which come within the meaning and range of equivalency of the claims are reserving to the present application. Therefore, no matter from which point of view, the embodiments should be considered as exemplary and non-limiting, and the scope of the present application is defined by the appended claims, not by the above description, and it is intended that all changes and modifications which come within the meaning and range of equivalency of the claims are reserving to the present application.

[0044] In addition, it should be understood that although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the description manner of the specification is only for the sake of clarity, and those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be properly combined to form other embodiments which can be understood by those skilled in the art. These other embodiments are also covered by the protection scope of the present application.

[0045] It should also be understood that the above-described specific embodiments are only used to explain the present application, and the protection scope of the present application

[0046] is not limited thereto, and any skilled person in the art can make equivalent replacements or changes according to the technical solutions and inventive concepts of the present application within the technical scope disclosed by the present application, which should be covered by the protection scope of the present application / invention.

[0047] Techniques, methods, and equipment known to those skilled in the relevant art can not be discussed in detail, but in appropriate cases, the techniques, methods, and equipment should be considered as part of the specification.

[0048] The disclosures of the prior art cited in the specification of the present application are all incorporated by reference into the present application, and thus are part of the disclosure of the present application.

[0049] Example One

[0050] As Figure 1As shown, the present application provides an OCT image choroid segmentation network system based on a feature integration method, which includes a preprocessing module, a feature extraction module, a feature fusion module and a secondary feature fusion module.

[0051] The preprocessing module is used for preprocessing the training and test images; the feature extraction module is used for extracting multi-scale features of the OCT image choroid layer from multiple depth levels, and the feature extraction module includes a feature encoder and a feature decoder; the feature fusion module is embedded in the feature extraction module, used for better aggregation of multi-level features, and outputs a multi-level prediction probability map; and the secondary feature fusion module is used for retraining the feature fusion module based on the multi-level prediction probability map to obtain an updated segmentation network model.

[0052] Firstly, an anisotropic diffusion filtering algorithm is used in the preprocessing module for filtering, and an exponential and linear enhancement method is used for enhancement; then a residual Inception structure is used in the feature extraction module, and different size receptive fields are fused to extract rich feature expressions of the image; pyramid pooling is used in the feature fusion process to further obtain context information; a channel attention module pays more attention to the relationship between channels, extracts a channel weighting vector of the feature map, and obtains the importance of each channel; and the secondary feature fusion process fuses the last layer of the feature encoder and the result of the last layer of the feature encoder, and counts the loss.

[0053] Specifically, the training set and test set images are flattened and cropped to the effective area of the choroid in the preprocessing module; an anisotropic diffusion filtering algorithm is used for filtering to reduce image noise while effectively retaining image boundary lines and other information; an exponential and linear enhancement method is used for enhancement to selectively highlight the features of interest in the image while attenuating unnecessary features; the image is scaled to an appropriate size (320x320) and then randomly flipped to obtain the processed image.

[0054] The feature extraction module uses a classic encoder-decoder network structure. The encoder has 5 layers, each of which is composed of a residual Inception structure, and the residual Inception structure is as follows Figure 2As shown in the figure, the convolution kernel size of the first three layers is 3x3, 5x5, and 7x7, the convolution kernel size of the last two layers is 1x1, 3x3, and 5x5, and then a 1x1 convolution is performed, followed by batch normalization and ReLU activation function after each convolution operation, and down-sampling is performed once after each layer, the number of channels is doubled, and the image size is reduced by one. The decoder is composed of residual Inception modules, and the convolution is the same as the feature encoder, followed by batch normalization and ReLU activation function after each convolution operation, and up-sampling is performed once after each layer, the number of channels is reduced by one, the image size is increased by one, and finally the original image size is restored.

[0055] The feature fusion module is composed of a channel attention module and a pyramid pooling module embedded in the feature extraction module. The pyramid pooling module is as shown in the figure Figure 3 , which includes 1x1, 2x2, 3x3, and 6x6 pooling operations of different sizes, and then a 1x1 convolution is performed on the pooled feature map to reduce the feature map, and then bilinear interpolation is used for up-sampling, that is, the same size feature map before the pyramid pooling module is obtained through up-sampling, and the channel is spliced.

[0056] Further, the channel attention module is as shown in the figure Figure 4 , which first performs a squeeze operation using a global average pooling operation to convert each two-dimensional feature channel to a real number; second, an excitation operation is performed using two fully connected operations, the first of which reduces the number of channels to reduce the computational load, and the second restores the original channel, followed by a Sigmoid function to generate weights for each feature channel; finally, the weights are multiplied to the previous features channel by channel to complete the re-rating of the original features in the channel dimension.

[0057] Further, in the feature decoder, the up-sampled feature map is spliced with the shallow feature map connected by the channel attention module to serve as the input of the feature decoder.

[0058] The secondary feature fusion module up-samples the last layer of the feature encoder and the results of each layer of the feature decoder to the input image size, and uses a 1x1 convolution layer and a Sigmoid function combined with a connection operation to fuse them to generate the final feature map. And the result of each layer is included in the loss function to optimize the segmentation network model.

[0059] A deep supervision loss function similar to HED is used, where the training loss is defined as:

[0060]

[0061] where M=6 represents the last layer of the feature encoder and the six layers of the feature decoder, m represents the number of layers of the feature encoder and the feature decoder, is the loss of the last layer of the feature encoder and the feature decoder of each layer, and l fuse is the loss of the final fusion output prediction map, and w fuse is the weight of each loss term, and for each term l, the loss is calculated using the standard binary cross-entropy:

[0062]

[0063] where (r, c) is the pixel coordinate, and (H, W) is the image size: height and width. P G(r,c) and P S(r,c) represent the pixel values of the GT and predicted saliency probability map, respectively. The training process tries to minimize the total loss L of formula (4). In the testing process, the fusion output l fuse is selected as the final prediction map.

[0064] The embodiment of the present application also provides an OCT image choroid segmentation method based on a feature integration method, and the method comprises the following steps:

[0065] S1, collecting a certain number of OCT images to construct a data set, and dividing the data set into a training set and a test set;

[0066] S2, constructing an OCT image choroid segmentation network model based on a feature integration method;

[0067] The construction method of the segmentation network model comprises the following steps:

[0068] A residual Inception module is built to construct an encoder-decoder structure network;

[0069] A channel attention module is added to the skip connection part of the network;

[0070] A pyramid pooling module is added to the up-sampling part of the network;

[0071] A secondary feature fusion module is added after the decoder of the network.

[0072] S3, inputting the training set and test set data into the network model for training to obtain a trained OCT image choroid segmentation network model based on a feature integration method;

[0073] S4, inputting the segmented image into the trained OCT image choroid segmentation model based on the feature integration method to obtain a choroid segmentation map.

[0074] Therefore, the present application adopts the OCT image choroid segmentation network system and method based on the feature integration method, which effectively improves the performance of choroid segmentation and shows superiority compared with other existing methods.

[0075] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application but not to limit them, although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that: it can still modify or equivalently replace the technical solutions of the present application, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present application.

Claims

1. An OCT image choroid segmentation network system based on a feature integration method, characterized in that, The method comprises a preprocessing module, a feature extraction module, a feature fusion module, and a secondary feature fusion module. The preprocessing module is used for preprocessing training and test images; the feature extraction module is used for extracting multi-scale features of a choroid layer of an OCT image from multiple depth levels, and the feature extraction module comprises a feature encoder and a feature decoder; the feature fusion module is embedded in the feature extraction module, and is used for better aggregating multi-level features and outputting multi-level prediction probability maps; and the secondary feature fusion module fuses the last layer of the feature encoder and a result of the last layer of the feature encoder, and is used for calculating a loss. The feature encoder is composed of residual Inception modules, and has five layers in total. The feature decoder is composed of residual Inception modules, and has the same convolution as the feature encoder. The feature fusion module is composed of a channel attention module and a pyramid pooling module, and is embedded in the feature extraction module. The secondary feature fusion module fuses the last layer of the feature encoder and a result of each layer of the feature decoder, and finally outputs a prediction map of the entire network model. A deep supervision loss function similar to HED is used, wherein a training loss is defined as follows: （1）； where M = 6 represents that the last layer of the feature encoder shares six layers with the feature decoder, and m represents the number of layers of the feature encoder and the feature decoder, is the loss of the last layer of the feature encoder and each layer of the feature decoder, and is the loss of the final fusion output prediction map, is the loss of the final fusion output prediction map, and is the weight of each loss term, and for each term the loss is calculated using the standard binary cross-entropy: （2）； where, are the pixel coordinates, and are the image dimensions: height and width, and denote the pixel values of the GT and predicted saliency probability maps, respectively, and the training process tries to minimize the total loss of equation (1) L At the testing process, the fused output is chosen as the final predicted map. 2.The OCT image choroid segmentation network system based on the feature integration method of claim 1, wherein, The preprocessing module is used for preprocessing training and test images, and specifically comprises the following steps. The training set and the test set are flattened and cropped to the effective area of the choroid, filtered by using an anisotropic diffusion filtering algorithm, enhanced by using an exponential and linear enhancement method, scaled to an appropriate size, and then randomly flipped to obtain processed images.

3. The OCT image choroid segmentation method based on the feature integration method, characterized in that, The OCT image choroid segmentation system based on the feature integration method of claim 1 is executed, and comprises the following steps: S1, a certain number of OCT images are collected to construct a data set, and the data set is divided into a training set and a test set; S2, an OCT image choroid segmentation network model based on the feature integration method is constructed; S3, the training set and the test set data are input into the network model for training, and a trained OCT image choroid segmentation network model based on the feature integration method is obtained; S4, a segmented image is input into the trained OCT image choroid segmentation model based on the feature integration method, and a choroid segmentation image is obtained.

4. The OCT image choroid segmentation method based on the feature integration method according to claim 3, wherein, The construction method of the OCT image choroid segmentation network model based on the feature integration method comprises: The residual Inception module is built, an encoder-decoder structure network is constructed, a channel attention module is added in a skip connection part of the network, a pyramid pooling module is added in an up-sampling part of the network, and a secondary feature fusion module is added after a decoder of the network.

5. A computer storage medium, characterized in that, The computer storage medium has a computer software product, and the computer software product includes a plurality of instructions to enable a computer device to execute the OCT image choroid segmentation method based on the feature integration method in claim 3.

Images