OCT image classification method

Abstract

The invention provides an OCT image classification method, and the method comprises the steps: obtaining a to-be-classified image and a classification network, the to-be-classified image being an OCT image, and the classification network comprising a processing layer; the processing layer comprises a first branch and at least two second branches; inputting the to-be-classified image to the first branch to output an optimized feature map, the first branch being used for extracting features; and inputting the optimized feature map to the second branch, and outputting a classification result after the optimized feature map is processed by a multi-attention unit of the second branch, the second branch being used for enhancing features. According to the method, through double-branch separation feature extraction and path enhancement, a first branch maintains an original channel as a substrate feature; and the second branch segments the remaining channels into four groups of processing sub-channels to form collaborative enhancement of pathological features and structural features so as to solve the problem of low classification accuracy of OCT images.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to an OCT image classification method. BACKGROUND

[0002] Diagnosis of diabetic retinopathy relies on optical coherence tomography equipment to obtain retinal cross-sectional images, which generates micron-level resolution biological tissue structure images through near-infrared light interference technology. The clinical requirement is to realize the synchronous identification of microaneurysms and neovascularization, and the algorithm is required to have spatial positioning ability and feature discrimination ability of the lesion area at the same time, and to be compatible with the morphological differences of different sizes of drusen.

[0003] The convolutional neural network scheme adopts a single-branch structure to perform an end-to-end classification task, wherein an improved model based on the ResNet50 architecture uses multiple convolution kernels to extract multi-scale features, some schemes embed a channel attention module in the last two layers to optimize feature selection, or add a global spatial pooling operation before the fully connected layer to enhance the position perception ability, and another method applies Fourier filtering in the preprocessing stage to suppress frequency domain noise interference.

[0004] The single-branch structure forces pathological features and basic structural features to be mixed and conducted in the same channel, resulting in the loss of retinal internal limiting membrane reflection features during channel compression; the channel attention mechanism weakens the local lesion response when calculating the weight globally, and cannot accurately locate the small hemorrhage points in the macular fovea; the fixed receptive field of the convolution kernel is difficult to adapt to the size change of the drusen; the single-type attention mechanism causes frequency domain feature confusion when processing images with coexisting exudate low-frequency features and neovascularization high-frequency features, and in summary, the OCT image classification accuracy is low. SUMMARY

[0005] The present application provides an OCT image classification method to solve the problem of low OCT image classification accuracy.

[0006] The present application provides an OCT image classification method, comprising:

[0007] An image to be classified and a classification network are obtained, the image to be classified is an OCT image, and the classification network comprises a processing layer; the processing layer comprises a first branch and at least two second branches;

[0008] The image to be classified is input into the first branch to output an optimized feature map, and the first branch is used for feature extraction;

[0009] The optimized feature map is input into the second branch, and after processing by the multi-attention unit of the second branch, a classification result is output, and the second branch is used for feature enhancement.

[0010] In some possible embodiments, the first branch includes a feature splitting unit and a first feature optimization unit, the first feature optimization unit includes a first processing unit, a second processing unit and a third processing unit; the convolution kernel size of the first processing unit and the third processing unit is a first size, and the convolution kernel size of the second processing unit is a second size;

[0011] The inputting the image to be classified into the first branch to output an optimized feature map includes:

[0012] The image to be classified is input into the feature splitting unit to perform feature channel splitting processing through the feature splitting unit, and output first and second sub-features;

[0013] The first sub-feature is sequentially input into the first processing unit, the second processing unit and the third processing unit to perform first convolution processing through the first processing unit, second convolution processing through the second processing unit and third convolution processing through the third processing unit, and output first optimized sub-features;

[0014] The second sub-feature is input into a fourth convolution kernel processing unit to perform fourth convolution processing through the fourth convolution kernel processing unit, and output second optimized sub-features;

[0015] The first optimized sub-feature and the second optimized sub-feature are input into an activation function unit to perform feature fusion activation processing through the activation function unit, and output the optimized feature map.

[0016] In some possible embodiments, the second branch includes a feature reorganization unit, a channel splitting unit, a multi-attention unit and a channel splicing unit.

[0017] The inputting the optimized feature map into the second branch further includes:

[0018] The optimized feature map is input into the feature reorganization unit to perform feature channel reorganization processing through the feature reorganization unit, and output main channel features and auxiliary channel features;

[0019] The main channel features are input into the channel splitting unit to perform multi-channel splitting processing through the channel splitting unit, and output four groups of feature sub-channels;

[0020] The feature sub-channels are respectively input into the multi-attention unit to perform multi-dimensional feature enhancement processing through the multi-attention unit, and output optimized sub-channel features;

[0021] The four groups of optimized sub-channel features are input into the channel splicing unit to perform feature splicing processing through the channel splicing unit, and output optimized main channel features;

[0022] The optimized main channel feature and the auxiliary channel feature are input into a feature fusion unit to perform channel fusion processing through the feature fusion unit, and a classification feature map is output.

[0023] In some possible embodiments, the multi-attention unit includes a channel attention module, a filter attention module, a spatial attention module, and a convolution kernel attention module.

[0024] The optimized sub-channel feature includes a channel weighting feature, a frequency domain optimization feature, a spatial weighting feature, and a dynamic convolution feature.

[0025] The method further includes:

[0026] The feature sub-channel is input into the channel attention module to perform channel dimension weight calculation processing through the channel attention module, and a channel weighting feature is output.

[0027] The feature sub-channel is input into the filter attention module to perform frequency domain feature optimization processing through the filter attention module, and a frequency domain optimization feature is output.

[0028] The feature sub-channel is input into the spatial attention module to perform spatial position weight calculation processing through the spatial attention module, and a spatial weighting feature is output.

[0029] The feature sub-channel is input into the convolution kernel attention module to perform convolution kernel parameter dynamic adjustment processing through the convolution kernel attention module, and a dynamic convolution feature is output.

[0030] In some possible embodiments, the multi-attention unit further includes a second feature optimization unit, which is connected to the channel attention module, the filter attention module, the spatial attention module, and the convolution kernel attention module, respectively.

[0031] The second feature optimization unit includes a fourth processing unit, a fifth processing unit, and a sixth processing unit; the convolution kernel size of the fourth processing unit and the sixth processing unit is a first size, and the convolution kernel size of the fifth processing unit is a second size.

[0032] The optimized sub-channel feature includes a third optimization sub-feature, a fourth optimization sub-feature, a fifth optimization sub-feature, and a sixth optimization sub-feature.

[0033] The input of the four groups of the optimized sub-channel feature into the channel splicing unit to perform feature splicing processing through the channel splicing unit, and output of the optimized main channel feature, includes:

[0034] The channel weighting features are sequentially input into the fourth processing unit, the fifth processing unit and the sixth processing unit to output third optimized sub-features;

[0035] The frequency domain optimized features are sequentially input into the fourth processing unit, the fifth processing unit and the sixth processing unit to output fourth optimized sub-features;

[0036] The spatial weighting features are sequentially input into the fourth processing unit, the fifth processing unit and the sixth processing unit to output fifth optimized sub-features;

[0037] The dynamic convolution features are sequentially input into the fourth processing unit, the fifth processing unit and the sixth processing unit to output sixth optimized sub-features;

[0038] The third optimized sub-features, the fourth optimized sub-features, the fifth optimized sub-features and the sixth optimized sub-features are input into the channel splicing unit to output optimized main channel features, wherein the third optimized sub-features, the fourth optimized sub-features, the fifth optimized sub-features and the sixth optimized sub-features are features after transformation processing.

[0039] In some possible embodiments, the processing layer includes a first feature processing stage, a second feature processing stage, a third feature processing stage and a fourth feature processing stage.

[0040] The output classification result includes:

[0041] The image to be classified is input into the first feature processing stage to perform preliminary feature extraction processing through the first feature processing stage, and output a first stage feature map;

[0042] The first stage feature map is input into the second feature processing stage to perform intermediate feature extraction processing through the second feature processing stage, and output a second stage feature map;

[0043] The second stage feature map is input into the third feature processing stage to perform deep feature extraction processing through the third feature processing stage, and output a third stage feature map;

[0044] The third stage feature map is input into the fourth feature processing stage to perform high-level feature integration processing through the fourth feature processing stage, and output a classification result;

[0045] The first feature processing stage includes two second branches, the second feature processing stage includes three second branches, the third feature processing stage includes five second branches, and the fourth feature processing stage includes two second branches.

[0046] In some possible embodiments, the classification network further comprises a global feature aggregation unit, a linear mapping unit, and a probability conversion unit.

[0047] The third stage feature map is input into a fourth feature processing stage to perform high-level feature integration processing through the fourth feature processing stage, and a classification result is output.

[0048] The third stage feature map is input into a fourth feature processing stage to perform multi-feature fusion processing through the fourth feature processing stage, and a fusion feature is output.

[0049] The fusion feature is input into a global feature aggregation unit to perform spatial dimension compression processing through the global feature aggregation unit, and an aggregated feature vector is output.

[0050] The aggregated feature vector is input into a linear mapping unit to perform feature dimension transformation processing through the linear mapping unit, and an initial classification result is output.

[0051] The initial classification result is input into a probability conversion unit to output a category probability distribution, and the category probability distribution represents the classification result.

[0052] In some possible embodiments, the classification network further comprises an input layer, and the input layer comprises a seventh processing unit and a spatial compression unit.

[0053] Before the image to be classified is input into the first branch to output an optimized feature map, the method further comprises:

[0054] The image to be classified is input into a seventh processing unit to perform initial convolution processing through the seventh processing unit, and an initial feature map is output.

[0055] The initial feature map is input into a spatial compression unit to perform spatial dimension reduction processing through the spatial compression unit, and a size-reduced feature map is output.

[0056] The size-reduced feature map is input into the first branch to perform feature optimization processing through the first branch, and an optimized feature map is output.

[0057] After the classification result is output, the method further comprises:

[0058] A balance factor weight value is determined through sample category identification.

[0059] A sample attention weight value is calculated according to the classification result and a focus factor.

[0060] The balance factor weight value and the sample attention weight value are multiplied to generate an adjustment coefficient.

[0061] The base loss value is multiplied by the adjustment coefficient to output a final loss value.

[0062] In some possible embodiments, the determining the balance factor weight value through the sample category identification comprises:

[0063] If the sample category identification is the target category, the balance factor weight value is determined as a first weight value;

[0064] If the sample category identification is the non-target category, the balance factor weight value is determined as a second weight value;

[0065] Wherein, the sum of the first weight value and the second weight value is 1.

[0066] From the above technical solutions, the present application provides an OCT image classification method, which comprises obtaining a to-be-classified image and a classification network, the to-be-classified image is an OCT image, and the classification network comprises a processing layer; the processing layer comprises a first branch and at least two second branches; the to-be-classified image is input into the first branch to output an optimized feature map, and the first branch is used for extracting features; the optimized feature map is input into the second branch, and after processing by a multi-attention unit of the second branch, a classification result is output, and the second branch is used for enhancing features. The method separates the feature extraction and enhancement paths through a double-branch, the first branch maintains the original channel as a basic feature; the second branch divides the remaining channels into four groups of processing sub-channels, forming a synergistic enhancement of pathological features and structural features, so as to solve the problem of low accuracy of OCT image classification. BRIEF DESCRIPTION OF DRAWINGS

[0067] In order to more clearly illustrate the technical solutions of the present application, the drawings needed in the embodiments will be briefly introduced below, and obviously, other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0068] Figure 1 The flowchart of the OCT image classification method provided by the embodiments of the present application;

[0069] Figure 2 The classification network structure diagram provided by the embodiments of the present application;

[0070] Figure 3 The first branch structure diagram provided by the embodiments of the present application;

[0071] Figure 4 The second branch structure diagram provided by the embodiments of the present application;

[0072] Figure 5Schematic diagram of the training loss and training accuracy of the resnet50 original classification network provided in the embodiment of the present application;

[0073] Figure 6 Schematic diagram of the training loss and training accuracy of the resnet50 original classification network after FocalLoss adjustment provided in an embodiment of the present application;

[0074] Figure 7 Schematic diagram of the training loss and training accuracy of the classification network provided in this application after adjustment by FocalLoss provided in an embodiment of this application. DETAILED DESCRIPTION

[0075] The following embodiments are described in detail, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numbers in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following embodiments are not intended to represent all possible implementations consistent with the present application. They are merely examples of systems and methods consistent with certain aspects of the present application, as detailed in the claims.

[0076] like Figure 1 As shown, the present application provides an OCT image classification method, comprising:

[0077] S110: Obtain an image to be classified and a classification network.

[0078] The image to be classified refers to a cross-sectional image of the fundus structure acquired using an optical coherence tomography device, i.e., an OCT image. More specifically, in this embodiment, the image to be classified refers to a cross-sectional image of the macular region acquired using an optical coherence tomography device, which includes typical pathological features of age-related macular degeneration (AMD), such as drusen, geographic atrophy, and choroidal neovascularization. The image displays the layered structure of the retina at a resolution of 512×512 pixels.

[0079] The classification network is a deep convolutional neural network architecture consisting of multiple cascaded processing layers that receives image data and outputs a probability distribution of pathology categories. The network utilizes a hierarchical feature processing framework, with initial layers capturing basic texture features and deeper layers extracting abstract semantic features.

[0080] The classification network provided in this embodiment is an improved neural network architecture based on resnet50. Figure 2 As shown, the classification network includes an input layer and a processing layer, wherein the processing layer includes multiple feature processing stages, and each feature processing stage includes a first branch and at least two second branches.

[0081] The first branch is used for extracting features, and the second branch is used for enhancing features, wherein the first branch extracts strong reflection features of the glass membrane wart, and the second branch forms an optimized feature map of lesion enhancement.

[0082] S120: inputting the image to be classified into the first branch to output an optimized feature map.

[0083] After the image to be classified is input into the classification network, it passes through the input layer and enters the first branch of the feature processing stage, and the first branch performs lesion feature extraction.

[0084] For example, after obtaining the macular OCT image of an AMD patient, the image is input into the classification network processing layer, and the first branch performs lesion feature extraction.

[0085] As Figure 3 shown, in some embodiments, the first branch includes two paths, a first path for feature splitting and a second path for feature optimization. For example, the first path extracts strong reflection features of the glass membrane wart, and the second path maintains the integrity of the outer retinal structure.

[0086] Specifically, the first branch includes a feature splitting unit and a first feature optimization unit. The feature splitting unit is a channel segmentation module that equally divides the input feature map along the channel dimension into two independent sub-feature groups through a tensor slicing operation.

[0087] The first feature optimization unit includes a first processing unit, a second processing unit and a third processing unit. The convolution kernel size of the first processing unit and the third processing unit is a first size, and the convolution kernel size of the second processing unit is a second size, wherein the first size is 1x1 and the second size is 3x3. The first processing unit uses a 1x1 size convolution kernel to perform feature channel dimension compression processing. The second processing unit uses a 3x3 size convolution kernel to perform feature spatial structure strengthening processing. The third processing unit uses a 1x1 size convolution kernel to perform feature channel dimension recovery processing. The three units are connected in series in order to form a feature refining chain, and the input and output spatial dimensions remain unchanged.

[0088] The image to be classified is input into the feature splitting unit to perform feature channel splitting processing through the feature splitting unit to output first and second sub-features. After the image to be classified is input into the feature splitting unit, the channel dimension is divided according to a predetermined ratio.

[0089] For example, with a 512-channel input, the first 256 channels are output as first sub-features, and the last 256 channels are output as second sub-features. The two features maintain the same spatial size.

[0090] The first sub-feature is sequentially input into the first processing unit, the second processing unit and the third processing unit to perform first convolution processing by the first processing unit, second convolution processing by the second processing unit and third convolution processing by the third processing unit, and output a first optimized sub-feature.

[0091] The first sub-feature is input into the first processing unit to perform channel dimension reduction processing, and a 1x1 convolution kernel compresses 512 channels to 256 channels. The obtained feature is input into the second processing unit to perform spatial relationship modeling, and a 3x3 convolution kernel enhances the local correlation of the feature while keeping the number of channels unchanged. The processing result is input into the third processing unit to perform channel expansion processing, and a 1x1 convolution kernel restores 256 channels to 512 channels, thereby generating the first optimized sub-feature.

[0092] The second sub-feature is input into the fourth convolution kernel processing unit to perform fourth convolution processing by the fourth convolution kernel processing unit, and output a second optimized sub-feature. The fourth convolution kernel processing unit is a 1x1 size convolution kernel, performs feature information preservation processing, performs linear transformation on the input feature and preserves the original feature distribution characteristics, and outputs a feature map with the same number of input channels.

[0093] The first optimized sub-feature and the second optimized sub-feature are input into an activation function unit to perform feature fusion activation processing by the activation function unit, and output the optimized feature map.

[0094] The activation function unit is composed of an adder and a nonlinear activation function. The adder performs double-path feature tensor element stacking operation, and the output result is input into the ReLU nonlinear activation function to generate the optimized feature map with feature zero value suppression. The first optimized sub-feature and the second optimized sub-feature complete element-level stacking operation in the adder of the activation function unit. The stacking result is input into the ReLU activation function to set the negative value in the feature matrix to zero and preserve the positive value feature response, and finally output the optimized feature map with nonlinear expression characteristics.

[0095] S130: input the optimized feature map into the second branch to output a classification result after processing by the multi-attention unit of the second branch.

[0096] As shown in Figure 4 The second branch includes a feature reorganization unit and a multi-attention unit. The feature reorganization unit splits the optimized feature map into a main channel and an auxiliary channel, and the main channel is divided into four groups of feature sub-channels. Each group of sub-channels is input into a channel attention module, a filtering attention module, a spatial attention module and a convolution kernel attention module for parallel processing. The outputs of each module are reconstructed through a feature fusion unit to generate a discriminative classification feature map.

[0097] The purpose of the second branch channel segmentation is to leave half of the original channel as the baseline feature, ensuring that the basic information is not lost, and the other half of the channel is used to learn various attention features, forming a complement, and the channel attention enhances the sensitivity of the retina to various lesions. The spatial attention accurately locates the abnormalities in the macular area, the filtering attention is to strengthen the frequency components specific to the lesions, including drusen and geographic atrophy, and the convolution kernel attention can adjust the receptive field and shape because the size of the lesions in the fundus OCT imaging of different people is different.

[0098] The multi-attention unit includes a channel attention module, a filtering attention module, a spatial attention module, and a convolution kernel attention module; the channel attention module calculates a channel weight matrix to strengthen the response of key frequency bands; the filtering attention module optimizes the frequency feature distribution; the spatial attention module generates a region weight map; and the convolution kernel attention module dynamically adjusts the convolution parameters.

[0099] For the typical pathological features of age-related macular degeneration (AMD), the channel attention strengthens the high-frequency reflection signals of lesions such as drusen deposits; the spatial attention focuses on the subretinal fluid area; the filtering attention suppresses the RPE layer fracture artifact; and the convolution kernel attention adapts to the irregular shape of the neovascularization. The multi-mechanism output generates a pathological feature map after fusion.

[0100] The method simultaneously captures micron-sized drusen and atrophy areas through the first branch, and separates and processes the drusen deposits, leakage areas, and blood vessel shapes through the four types of attention in the second branch. The auxiliary channel completely retains the photoreceptor layer to avoid misjudgment of the ellipsoid band fracture, and the output features can represent the CNV leakage area and atrophy lesion.

[0101] For the improved resnet50 classification network of the present application, in some embodiments, the classification network further includes an input layer including a seventh processing unit and a spatial compression unit, wherein the seventh processing unit performs initial convolution processing, adopts a preset convolution kernel to perform basic feature extraction on the input image, the convolution kernel size is set to cover the retinal feature scale, the layered structure information of the OCT image is extracted through a multi-channel filtering operation, the spatial resolution of the output feature map is maintained at the original input size, and the number of channels is expanded to a preset dimension.

[0102] The spatial compression unit performs spatial dimension reduction processing through MaxPooling, reduces the spatial resolution through local region feature sampling operation, adopts a fixed step downsampling algorithm, compresses the spatial dimension under the premise of maintaining the integrity of the feature distribution, and synchronously reduces the height and width of the output feature map while keeping the channel dimension unchanged.

[0103] The image to be classified is input into the seventh processing unit to perform initial convolution processing by the seventh processing unit to output an initial feature map; the initial feature map is input into a spatial compression unit to perform spatial dimension reduction processing by the spatial compression unit to output a size-reduced feature map; and the size-reduced feature map is input into the first branch to perform feature optimization processing by the first branch to output an optimized feature map.

[0104] In the embodiment, the size of the image to be classified is 224x224x3, the initial feature map generated after the processing of the seventh processing unit is 112x112x64, and the size-reduced feature map obtained through maximum pooling is 56x56x64.

[0105] In some embodiments, the processing layer includes a first feature processing stage, a second feature processing stage, a third feature processing stage, and a fourth feature processing stage, and each of the multiple processing stages includes a first branch and a second branch. The output of the last second branch after the fourth feature processing stage is the classification result.

[0106] The size-reduced feature map obtained by the input layer is input into the first feature processing stage, i.e., processed by the first branch.

[0107] In the embodiment, the first feature processing stage includes one first branch and two second branches, the second feature processing stage includes one first branch and three second branches, the third feature processing stage includes one first branch and five second branches, and the fourth feature processing stage includes one first branch and two second branches.

[0108] In the first feature processing stage, the first feature processing stage performs preliminary feature extraction processing. After the first branch in the first feature processing stage, the size of the output feature map is 56x56x256, and then two second branches are output in sequence. The output of the first second branch is input into the second second branch, and the feature map output by the second branch has the same size as the output of the first branch.

[0109] In some embodiments, the second branch includes a feature reorganization unit, a channel segmentation unit, a multi-attention unit, and a channel splicing unit. The feature reorganization unit performs channel dimension feature reorganization operation, and splits the input feature map into a main channel and an auxiliary channel by a preset ratio. In OCT image processing, the main channel carries lesion feature information, and the auxiliary channel retains tissue structure features. After splitting, the two channel features maintain the original spatial size unchanged.

[0110] The channel segmentation unit equally divides the main channel features along the channel dimension into four groups of independent feature sub-channels, each group of sub-channels is allocated a fixed proportion of channel resources and receives feature information at the same spatial position. The four groups of feature sub-channels process feature information of different dimensions in parallel.

[0111] The channel splicing unit performs sub-channel feature integration operation, splices the four groups of optimized sub-channel features along the channel dimension in sequence, and reconstructs the complete channel number in the input order during the splicing process. The output feature map has the same spatial resolution as the input.

[0112] The feature fusion unit splices the optimized main channel features and the auxiliary channel features along the channel dimension, integrates the two types of feature information by increasing the channel dimension, and outputs a feature map with twice the number of channels of the original input feature map.

[0113] In some embodiments, the optimized feature map is input into the feature reorganization unit to perform feature channel reorganization processing by the feature reorganization unit, outputting main channel features and auxiliary channel features;

[0114] The main channel features are input into the channel segmentation unit to perform multi-channel segmentation processing by the channel segmentation unit, outputting four groups of feature sub-channels;

[0115] The feature sub-channels are respectively input into the multi-attention unit to perform multi-dimensional feature enhancement processing by the multi-attention unit, outputting optimized sub-channel features;

[0116] The four groups of optimized sub-channel features are input into the channel splicing unit to perform feature splicing processing by the channel splicing unit, outputting optimized main channel features;

[0117] The optimized main channel features and the auxiliary channel features are input into the feature fusion unit to perform channel fusion processing by the feature fusion unit, outputting classification feature maps.

[0118] After the optimized feature map is input into the feature reorganization unit, a segmentation operation is performed according to a preset channel proportion. For example, 70% of the channels constitute the main channel, and 30% of the channels constitute the auxiliary channel. The main channel features input into the channel segmentation unit are equally divided into four groups of feature sub-channels.

[0119] Each group of feature sub-channels is input into the multi-attention unit for independent processing. The first group performs channel dimension correlation optimization; the second group performs frequency domain feature enhancement; the third group strengthens spatial position weight; and the fourth group adapts to local feature morphology. Each group of processing maintains the original spatial resolution.

[0120] The processed four groups of optimization sub-channel feature input channel splicing units reorganize the main channel according to the original segmentation order. The reorganized optimization main channel features and auxiliary channel features are jointly input into a feature fusion unit and spliced into a complete classification feature map along the channel dimension.

[0121] When the AMD lesion OCT image is input, the feature reorganization unit assigns the macular edema features to the main channel and retains the retinal layer structure in the auxiliary channel. The first group of sub-channels after channel segmentation focuses on the lesion areas such as drusen and geographic atrophy, and the multi-attention unit strengthens the lesion discrimination features; the auxiliary channel completely retains the outer retinal fracture information. The final fused feature map includes both lesion enhanced features and tissue structure features.

[0122] The multi-attention unit includes a channel attention module, a filtering attention module, a spatial attention module, and a convolution kernel attention module.

[0123] The channel attention module performs channel dimension weight calculation processing and generates a channel importance coefficient matrix through global feature analysis, including a channel compression layer and a weight generation layer. The compression layer calculates the feature statistics of the spatial dimension, and the weight generation layer assigns the weighting coefficients of each channel accordingly. The response intensity of each channel in the feature map is scaled by the weight value.

[0124] The feature sub-channels are input into the channel attention module to perform channel dimension weight calculation processing through the channel attention module, outputting channel weighted features. After the feature sub-channels are input into the channel attention module, the global feature pooling layer calculates the mean value of the spatial dimension features to generate a channel statistics vector. The weight generation layer maps the statistics vector to a channel weight sequence. Each channel of the original feature map is multiplied by the corresponding weight coefficient, and the channel weighted features are output for enhancing the feature response of the discriminative channel.

[0125] The filtering attention module performs frequency domain feature optimization processing to enhance key frequency components through frequency domain transformation and bandpass filtering, including a Fourier transform layer, a frequency response matrix, and an inverse Fourier transform layer. The frequency response matrix dynamically adjusts the passband range according to the feature spectral characteristics, and the output signal retains the key frequency domain pattern features.

[0126] The feature sub-channels are input into the filtering attention module to perform frequency domain feature optimization processing through the filtering attention module, outputting frequency domain optimized features. The feature sub-channels are input into the filtering attention module to perform frequency domain conversion. The Fourier transform layer converts the spatial features to the frequency domain to generate a complex spectrum graph. The frequency response matrix adjusts the frequency spectrum amplitude according to the preset rule, suppresses the noise frequency band, and amplifies the lesion feature frequency band. The inverse Fourier transform reconstructs the spatial features to output the frequency domain optimized feature map.

[0127] The spatial attention module performs spatial position weight calculation processing to generate a two-dimensional attention map with the same size as the input feature map. The local area statistical features are extracted by the spatial feature pooling layer, and the position-related importance coefficients are generated by the weight mapping layer. The feature value of each spatial position is multiplied by the corresponding weight coefficient to strengthen the response of the key area.

[0128] The feature sub-channels are input into the spatial attention module to perform spatial position weight calculation processing by the spatial attention module, and spatial weighted features are output. The feature map is extracted by the spatial feature pooling layer to generate local maximum value features, and the weight mapping layer expands the dimension-reduced features into a two-dimensional weight matrix with the original size. The position value of the input feature is multiplied by the corresponding matrix coefficient to generate a spatial weighted feature map.

[0129] The convolution kernel attention module performs convolution kernel parameter dynamic adjustment processing to generate optimized convolution kernel parameters according to the characteristics of the input features, including a kernel weight generation network that learns the local structure pattern of the input features to output adaptive convolution kernel weight matrices. The output feature map includes the results of dynamic kernel convolution operations.

[0130] The feature sub-channels are input into the spatial attention module to perform spatial position weight calculation processing by the spatial attention module, and spatial weighted features are output. The feature map is extracted by the spatial feature pooling layer to generate local maximum value features, and the weight mapping layer expands the dimension-reduced features into a two-dimensional weight matrix with the original size. The position value of the input feature is multiplied by the corresponding matrix coefficient to generate a spatial weighted feature map.

[0131] The channel attention processing solves the response imbalance problem of different lesion areas and ensures that the features of small hemorrhagic points are not overwhelmed by background noise. The filter attention processing enhances the frequency domain feature extraction capability and improves the recognition of lesions such as drusen and geographic atrophy. The spatial attention processing breaks through the receptive field limitation of traditional convolution and accurately locates the local lesion area. The convolution kernel attention processing dynamically adjusts the kernel parameters to adapt to the morphological feature differences of distorted blood vessels.

[0132] In the analysis of macular hole lesions, the spatial attention locates the edge area of the hole, the channel attention strengthens the response of the full-layer break features, the convolution kernel attention adapts to the elliptical band break shape, and the filter attention improves the frequency domain signal-to-noise ratio of the foveal reflex loss, which can significantly improve the analysis accuracy of layered structure abnormalities.

[0133] In some embodiments, the multi-attention unit further includes a second feature optimization unit, the second feature optimization unit includes a fourth processing unit, a fifth processing unit and a sixth processing unit, and the fourth processing unit, the fifth processing unit and the sixth processing unit are sequentially connected in order.

[0134] The convolution kernel size of the fourth processing unit and the sixth processing unit is a first size, i.e., 1x1, the fourth processing unit performs feature dimension reduction integration processing, the input feature is compressed in the channel dimension through the convolution kernel, and the key feature response mode is reserved. The number of output feature map channels is reduced to 1 / N (N is a preset compression coefficient) of the input.

[0135] The convolution kernel size of the fifth processing unit is a second size, i.e., 3x3, and the 3x3 size convolution kernel performs feature space correlation enhancement processing, captures the local structure correlation of the feature by expanding the convolution kernel perception field, optimizes the feature space distribution mode, and the spatial size and the number of channels of the feature map remain unchanged before and after processing.

[0136] The sixth processing unit performs feature channel dimension recovery processing, expands the compressed channel number to the original dimension through the convolution kernel, and reconstructs the integrity of the feature channel. The number of output feature map channels is equal to the original value before the fourth processing unit.

[0137] In the embodiment, the second feature optimization unit is connected to different attention modules, that is, the outputs of different attention modules are processed through the second feature optimization unit, that is, processed through the fourth processing unit, the fifth processing unit and the sixth processing unit. The different attention modules will be described below.

[0138] For the channel attention module, the channel weighting feature is sequentially input into the fourth processing unit, the fifth processing unit and the sixth processing unit to output a third optimized sub-feature. Specifically, the 1x1 convolution kernel performs feature channel dimension reduction from 256 channels to 128 channels, the reduced dimension feature is input into the fifth processing unit, the 3x3 convolution kernel performs spatial feature structure optimization to enhance the local correlation of the lesion area, the processing result is input into the sixth processing unit, the 1x1 convolution kernel restores the channel to 256 channels, and the third optimized sub-feature is output.

[0139] For the filter attention module, the spatial attention module and the convolution kernel attention module, the same as the channel attention module, they are sequentially input into the fourth processing unit, the fifth processing unit and the sixth processing unit to obtain the fourth optimized sub-feature, the fifth optimized sub-feature and the sixth optimized sub-feature, which will not be described here.

[0140] The four groups of optimized sub-features are input into the channel splicing unit and spliced and integrated along the channel dimension in a preset order: the third optimized sub-feature (256 channels), the fourth optimized sub-feature (256 channels), the fifth optimized sub-feature (256 channels), and the sixth optimized sub-feature (256 channels) are spliced into 1024 channel optimized main channel features, and the optimized main channel features optimize the nerve fiber layer reflection features.

[0141] For the second second branch of the first feature processing stage, the processing process of the second second branch is the same as that of the first second branch, but the input is the output of the first second branch, which is used to strengthen the recognition of RPE layer continuity, that is, the output first stage feature map, and the size of the first stage feature map is 56x56x256.

[0142] The first stage feature map is input into the second feature processing stage to perform intermediate feature extraction processing through the second feature processing stage, output the second stage feature map, and the internal processing process of the first branch and the second branch of the second feature processing stage is the same as that of the first feature processing stage. Further, the first branch of the second feature processing stage performs 2 times down-sampling, the first second branch performs glass membrane wart cluster distribution pattern extraction, the second second branch enhances the boundary of early atrophy area, the third second branch detects subretinal fluid dark area, and the size of the second stage feature map is 28x28x512, the channel is expanded to 512 dimensions, and the spatial resolution is compressed by 50%.

[0143] The second stage feature map is input into the third feature processing stage to perform deep feature extraction processing through the third feature processing stage, output the third stage feature map, and the internal processing process of the first branch and the second branch of the third feature processing stage is the same as that of the first feature processing stage. Further, the first branch of the third feature processing stage is connected to the entrance, the first second branch and the second second branch cooperatively analyze the morphology of choroidal neovascularization (CNV), the third second branch reconstructs the three-dimensional structure of pigment epithelial detachment (PED), the fourth second branch and the fifth second branch analyze the microstructure of the geographic atrophy area, and the size of the third stage feature map is 14x14x1024.

[0144] Finally, the third stage feature map is input into the fourth feature processing stage to perform high-level feature integration processing through the fourth feature processing stage, output the classification result, and the internal processing process of the first branch and the second branch of the fourth feature processing stage is the same as that of the first feature processing stage. Further, the first branch of the fourth feature processing stage is a multi-scale feature fusion layer, the first second branch is a high-level semantic feature refining layer, the second second branch performs spatial compression, and the classification result channel compression and spatial aggregation are cooperated, and the output size of the second branch is 7x7x2048.

[0145] The classification result can be a probability vector, for example, a probability vector P=[P1, P2, P3], wherein p1 represents the probability of no macular lesion, p2 represents the probability of dry AMD (glass membrane wart deposition type), and p3 represents the probability of wet AMD (choroidal neovascularization type).

[0146] The classification result can also be visualized as a heat map, and a lesion positioning map is generated by a gradient weighted class activation mapping (Grad-CAM), wherein a red region represents a lesion region and a blue region represents a non-lesion region.

[0147] To obtain the classification result, in some embodiments, the classification network further comprises a global feature aggregation unit, a linear mapping unit, and a probability conversion unit. The global feature aggregation unit performs spatial dimension compression processing, converts a two-dimensional feature map into a one-dimensional feature vector through a pooling operation, and calculates the mean statistics of each channel spatial dimension using a global mean pooling algorithm. The output vector dimension is consistent with the input feature map channel number, realizing spatial information integration. After passing through the global feature aggregation unit, the feature map size is converted from 7x7x2048 to 1x1x2048.

[0148] The linear mapping unit performs feature dimension transformation processing, realizes high-dimensional feature space mapping through a fully connected neural network layer, and outputs a numerical sequence corresponding to the preset classification category number through linear weight matrix operation of the input feature vector. The weight matrix parameters are optimized and learned in the training process.

[0149] The probability conversion unit performs probability distribution calculation processing, converts the numerical sequence into a probability distribution through a normalized exponential function, calculates the ratio of each element in the input numerical sequence after exponential operation to the sum of all elements to generate category probability values, and the sum of the output probability value sequence is a fixed constant. The linear mapping unit outputs a 1000-dimensional classification probability.

[0150] The third stage feature map is input into a fourth feature processing stage to perform multi-feature fusion processing through the fourth feature processing stage, output a fusion feature, and input the fusion feature into a global feature aggregation unit to perform spatial dimension compression processing through the global feature aggregation unit, output an aggregated feature vector.

[0151] For an input feature map (channel number Cxheight Hxwidth W), the pixel average value of each channel is calculated in the height and width dimensions, and the output vector dimension is Cx1, forming an aggregated feature vector representing the overall feature.

[0152] The aggregated feature vector is input into a linear mapping unit to perform feature dimension transformation processing through the linear mapping unit, output an initial classification result; and the initial classification result is input into a probability conversion unit to output a category probability distribution, which represents the classification result.

[0153] The weight matrix (dimension KxC) with preset dimension K is multiplied with the input vector (dimension Cxl) to generate an initial classification result vector with dimension Kxl, each element value of which corresponds to the original output score of the classification network. Exponential operation is performed on each element in the vector to enhance the advantage of positive value score, and the ratio of the exponential value of each element to the sum of the exponential values of all elements is calculated to output a probability distribution vector that satisfies each element value ∈ [0, 1] and the total sum is 1.

[0154] For the classification result, the prior art processes it through a traditional cross-entropy loss, but the cross-entropy loss is not optimized in cooperation with the feature enhancement module, and the traditional cross-entropy loss is easily dominated by simple negative samples in the case of extremely unbalanced samples. The formula of the cross-entropy loss is as follows:

[0155] BCE(p t )=-log(p t );

[0156] Where p t is the predicted probability of the correct class by the classification model.

[0157] In some embodiments, the balance factor weight value is determined by the sample class identifier, the sample attention weight value is calculated according to the classification result and the focus factor, the adjustment coefficient is generated by multiplying the balance factor weight value and the sample attention weight value, and the final loss value is output by multiplying the basic loss value and the adjustment coefficient.

[0158] Where the sample class identifier represents a discrete encoding signal of the image pathology type, which is obtained through data labeling. In the OCT classification task, the identifier value corresponds to predefined categories such as normal retina and macular edema, serving as reference data for supervised learning.

[0159] In some embodiments, if the sample class identifier is the target class, the balance factor weight value is determined as the first weight value; if the sample class identifier is the non-target class, the balance factor weight value is determined as the second weight value.

[0160] Where the sum of the first weight value and the second weight value is 1, and the balance factor weight value α t is used to balance the influence of positive and negative samples and prevent the excessive contribution of negative samples to the loss: for positive samples α t = α, for negative samples α t = 1- α, and the value of α is usually between [0, 1], representing the weight ratio of positive and negative samples.

[0161] For example, when the sample class label points to a target pathological class (e.g., choroidal neovascularization), a first weight value (a) is assigned; when it points to a non-target class (e.g., normal macular structure), a second weight value (1-a) is assigned. Both strictly follow the weight and constant constraint relationship.

[0162] Focus factor g: By introducing the focus factor, FocalLoss adjusts the attention of the model to easy and difficult samples. When the prediction probability p t γ Part is the key, see formula:

[0163]

[0164] When the prediction probability p t is close to 1 (i.e., the sample is easy to classify), (1-p t ) γ will be very small, reducing the contribution of the loss. When the prediction probability p t is close to 0 (i.e., the sample is difficult to classify), (1-p t ) γ will be very large, increasing the weight of the loss, so that the model pays more attention to difficult samples.

[0165] As shown in Figure 5 , the classification task is performed by the resnet50 original classification network, where the left 5a chart represents the training loss (Training Loss), which measures the deviation between the model's prediction results and the true label. The lower the loss value, the better the model fitting effect. In the initial stage (0-25 epoch), the loss drops from 0.4731 to 0.0763, and in the subsequent stage (25-175 epoch), it continues to maintain a stable platform period of 0.3-0.7.

[0166] The right 5b chart represents the training accuracy (Training Accuracy), which is the proportion of samples that the model predicts correctly. The higher the accuracy, the stronger the classification ability. In the first 25 epochs, the accuracy improved from 62% to 93%, and in the 50-175 epochs, it fluctuated between 90% and 97%, with a final accuracy of 98.05%.

[0167] As shown in Figure 6 , 6a and 6b are the same as 5a and 5b, respectively. In the initial optimization, the loss decreases from 0.5 to 0.07 in the first 25 epochs, and continues to converge in the 25-175 epochs. The loss is stable and approaches 0. The accuracy curve reaches 90% at 25 epochs, and the learning rate is dynamically adjusted to accelerate convergence. The final accuracy is 98.54%.

[0168] As shown in Figure 7 ​As shown, 7a and 7b are the same as 5a and 5b, the classification network training loss provided by the application is continuously convergent, the whole process is smooth, and the final accuracy is 98.93%.

[0169] It can be seen that the classification network provided by the application separates structural features and pathological features through a double-branch architecture, improves microlesion response through channel attention, accurately locates maculopathy through spatial attention, resolves frequency domain feature confusion through filtering attention, and adapts to lesion size span through convolution kernel attention, so as to improve the classification accuracy.

[0170] The similar parts between the embodiments provided by the application can be referred to each other, the specific embodiments provided above are only a few examples under the general concept of the application, and do not limit the protection scope of the application. Any other embodiments extended by the person skilled in the art without creative labor according to the application scheme belong to the protection scope of the application.

Claims

1. An OCT image classification method, characterized in that: include: Acquire an image to be classified and a classification network, wherein the image to be classified is an OCT image, and the classification network includes a processing layer; the processing layer includes a first branch and at least two second branches; Inputting the image to be classified into the first branch to output an optimized feature map, wherein the first branch is used to extract features; The optimized feature map is input into the second branch to be processed by the multi-attention unit of the second branch to output the classification result, and the second branch is used to enhance the features.

2. The OCT image classification method according to claim 1, characterized in that: The first branch includes a feature splitting unit, a first feature optimization unit, and a fourth convolution kernel processing unit, the first feature optimization unit includes a first processing unit, a second processing unit, and a third processing unit; the convolution kernel size of the first processing unit and the third processing unit is a first size, and the convolution kernel size of the second processing unit is a second size; Inputting the image to be classified into the first branch to output an optimized feature map includes: Inputting the image to be classified into the feature splitting unit, so that the feature splitting unit performs feature channel splitting processing and outputs a first sub-feature and a second sub-feature; Inputting the first sub-feature into the first processing unit, the second processing unit, and the third processing unit in sequence, so that the first processing unit performs a first convolution process, the second processing unit performs a second convolution process, and the third processing unit performs a third convolution process, and outputs a first optimized sub-feature; Inputting the second sub-feature into a fourth convolution kernel processing unit to perform a fourth convolution process through the fourth convolution kernel processing unit, and outputting a second optimized sub-feature; The first optimized sub-feature and the second optimized sub-feature are input into an activation function unit, so as to perform feature fusion activation processing through the activation function unit and output the optimized feature map.

3. The OCT image classification method according to claim 1, characterized in that: The second branch includes a feature recombination unit, a channel segmentation unit, a multi-attention unit, and a channel splicing unit; The step of inputting the optimized feature map into the second branch further includes: Inputting the optimized feature map into the feature recombination unit so as to perform feature channel recombination processing through the feature recombination unit and outputting main channel features and auxiliary channel features; Inputting the main channel features into the channel segmentation unit so as to perform multi-channel segmentation processing through the channel segmentation unit and output four groups of feature sub-channels; Inputting the feature sub-channels into the multi-attention unit respectively, so as to perform multi-dimensional feature enhancement processing through the multi-attention unit and output optimized sub-channel features; Inputting the four sets of optimized sub-channel features into the channel stitching unit, so as to perform feature stitching processing through the channel stitching unit and output optimized main channel features; The optimized main channel features and the auxiliary channel features are input into a feature fusion unit, so that channel fusion processing is performed by the feature fusion unit to output a classification feature map.

4. The OCT image classification method according to claim 3, characterized in that: The multi-attention unit includes a channel attention module, a filter attention module, a spatial attention module and a convolution kernel attention module; The optimized sub-channel features include channel weighted features, frequency domain optimization features, spatial weighted features and dynamic convolution features; The method further comprises: Inputting the feature sub-channel into the channel attention module to perform channel dimension weight calculation processing through the channel attention module and output channel weighted features; Inputting the feature sub-channel into the filter attention module to perform frequency domain feature optimization processing through the filter attention module and output frequency domain optimized features; Inputting the feature subchannel into the spatial attention module to perform spatial position weight calculation processing through the spatial attention module and outputting spatial weighted features; The feature sub-channel is input into the convolution kernel attention module so as to perform dynamic adjustment processing of the convolution kernel parameters through the convolution kernel attention module and output dynamic convolution features.

5. The OCT image classification method according to claim 4, characterized in that: The multi-attention unit further includes a second feature optimization unit, which is connected to the channel attention module, the filter attention module, the spatial attention module, and the convolution kernel attention module respectively; The second feature optimization unit includes a fourth processing unit, a fifth processing unit, and a sixth processing unit; the convolution kernel sizes of the fourth processing unit and the sixth processing unit are the first size, and the convolution kernel size of the fifth processing unit is the second size; The optimized sub-channel features include a third optimized sub-feature, a fourth optimized sub-feature, a fifth optimized sub-feature, and a sixth optimized sub-feature; Inputting the four sets of optimized sub-channel features into the channel stitching unit to perform feature stitching processing through the channel stitching unit and outputting optimized main channel features includes: inputting the channel weighted features into the fourth processing unit, the fifth processing unit, and the sixth processing unit in sequence to output a third optimized sub-feature; inputting the frequency domain optimization feature into the fourth processing unit, the fifth processing unit, and the sixth processing unit in sequence to output a fourth optimization sub-feature; inputting the spatial weighted feature into the fourth processing unit, the fifth processing unit, and the sixth processing unit in sequence to output a fifth optimized sub-feature; Inputting the dynamic convolution feature into the fourth processing unit, the fifth processing unit and the sixth processing unit in sequence to output a sixth optimized sub-feature; The third optimized sub-feature, the fourth optimized sub-feature, the fifth optimized sub-feature and the sixth optimized sub-feature are input into the channel stitching unit to output the optimized main channel feature, wherein the third optimized sub-feature, the fourth optimized sub-feature, the fifth optimized sub-feature and the sixth optimized sub-feature are features after transformation processing.

6. The OCT image classification method according to claim 1, characterized in that: The processing layer includes a first feature processing stage, a second feature processing stage, a third feature processing stage and a fourth feature processing stage; The output classification result includes: Inputting the image to be classified into a first feature processing stage to perform preliminary feature extraction processing through the first feature processing stage and output a first stage feature map; Inputting the first stage feature map into the second feature processing stage, performing intermediate feature extraction processing through the second feature processing stage, and outputting the second stage feature map; Inputting the second stage feature map into the third feature processing stage to perform deep feature extraction processing through the third feature processing stage and outputting the third stage feature map; Inputting the third stage feature map into the fourth feature processing stage, so as to perform high-level feature integration processing through the fourth feature processing stage and output a classification result; The first feature processing stage includes two second branches, the second feature processing stage includes three second branches, the third feature processing stage includes five second branches, and the fourth feature processing stage includes two second branches.

7. The OCT image classification method according to claim 6, characterized in that: The classification network also includes a global feature aggregation unit, a linear mapping unit and a probability conversion unit; The step of inputting the third stage feature map into the fourth feature processing stage to perform high-level feature integration processing through the fourth feature processing stage and outputting a classification result includes: Inputting the third stage feature map into the fourth feature processing stage, so as to perform multi-feature fusion processing through the fourth feature processing stage and output fusion features; Inputting the fused features into a global feature aggregation unit to perform spatial dimension compression processing through the global feature aggregation unit and output an aggregated feature vector; Inputting the aggregated feature vector into a linear mapping unit to perform feature dimension transformation processing through the linear mapping unit and output an initial classification result; The initial classification result is input into a probability conversion unit to output a category probability distribution, where the category probability distribution represents the classification result.

8. The OCT image classification method according to claim 1, characterized in that: The classification network further includes an input layer, wherein the input layer includes a seventh processing unit and a spatial compression unit; Before inputting the image to be classified into the first branch to output the optimized feature map, the method further includes: Inputting the image to be classified into a seventh processing unit, so as to perform initial convolution processing through the seventh processing unit and output an initial feature map; Inputting the initial feature map into a spatial compression unit to perform spatial dimension reduction processing through the spatial compression unit and output a size-reduced feature map; The size-reduced feature map is input to the first branch, so that feature optimization processing is performed by the first branch and an optimized feature map is output.

9. The OCT image classification method according to claim 1, characterized in that: After outputting the classification results, the following steps are included: Determine the balance factor weight value through sample category identification; Calculate the sample attention weight value according to the classification result and the focus factor; Multiplying the balance factor weight value by the sample attention weight value to generate an adjustment coefficient; Multiply the base loss value by the adjustment coefficient to output the final loss value.

10. The OCT image classification method according to claim 9, characterized in that: Determining the balance factor weight value according to the sample category identifier includes: If the sample category is identified as the target category, determining the weight value of the balancing factor to be a first weight value; If the sample category is identified as a non-target category, determining the weight value of the balancing factor to be a second weight value; The sum of the first weight value and the second weight value is 1.

Images