Ai-based image processing device for segmenting optic nerve structure to diagnose visual abnormality

Abstract

The present invention relates to an AI-based image processing device for segmenting an optic nerve structure to diagnose a visual abnormality. The AI-based image processing device for segmenting an optic nerve structure to diagnose a visual abnormality according to an aspect of the present invention comprises: a first encoder including a convolution block and multiple layer blocks and for calculating multi-level features from an RGB image obtained by photographing eyes; a second encoder for performing a multi-head self-attention operation on a top-level feature output from the first encoder, and applying same to a multi-layer perceptron to calculate a feature obtained by capturing spatial dimension dependency; and a decoder for: obtaining an image in which spatial features are emphasized, by repeating a process of connecting, to a result obtained by performing an attention operation on each feature remaining after excluding the top-level feature from among the multi-level features output from the first encoder and up-sampling the feature output from the second encoder, a result obtained by performing an attention operation on each feature from the upper level, performing a convolution operation, up-sampling same, and then connecting a result obtained by performing an attention operation on the next lower-level feature; and segmenting regions corresponding to an optic nerve disk and an optic nerve cup, respectively.

Description

AI-based image processing device that segments optic nerve structures to diagnose visual abnormalities

[0001] The present invention relates to a technology for detecting and segmenting optic nerve regions in an image of an eye.

[0002] The optic nerve is a bundle of nerves connecting the eyeball and the cerebrum that can be observed through the pupil. Visual abnormalities such as glaucoma and optic neuritis can be diagnosed based on the shape of the optic nerve identified during an examination; however, conventional diagnostic methods for visual abnormalities are performed by experts, which poses a problem of reduced reliability depending on the expert's experience.

[0003] Although structural changes in the ratio of the optic disc (OD) and optic cup (OC) have been identified as a result of the progression of eye disease, technology to automate the diagnosis of visual abnormalities using these structural changes has not been sufficiently developed.

[0004] Recently, technologies for diagnosing visual abnormalities using deep learning have been introduced, but these existing technologies have limitations in that they do not provide systematic judgment criteria in that they simply supervise learning the presence of visual abnormalities using eye images without considering the ratio of the optic nerve disc to the optic nerve cup.

[0005] In addition, due to the indistinct boundary between the optic nerve disc and the optic nerve cup in RGB images of the eye, it is difficult to accurately segment the regions corresponding to the optic nerve disc and the optic nerve cup using existing image segmentation techniques.

[0006] The objective of the present invention is to solve the above problem by providing an AI-based image processing device that segments the optic nerve structure for diagnosing visual abnormalities by dividing the regions corresponding to the optic nerve disc and optic nerve cup in an RGB image of the eye.

[0007] The objectives of the present invention are not limited to those mentioned above, and other unmentioned objectives will be clearly understood from the description below.

[0008] An AI-based image processing device for segmenting optic nerve structures to diagnose visual abnormalities according to one aspect of the present invention for achieving the aforementioned purpose comprises: a first encoder composed of serially connected convolution blocks and a plurality of layer blocks to calculate multi-level features from an RGB image of an eye; a second encoder that receives the top-level features output from the first encoder, performs multi-head self-attention operations, and applies them to a multi-layer perceptron to calculate features that capture spatial dimension dependencies from the top-level features; and a decoder that calculates an image with spatially emphasized features by repeating a process of performing attention operations on each feature excluding the top-level feature among the multi-level features output from the first encoder, upsampling the features output from the second encoder, connecting the results of attention operations on each feature starting from the upper level, performing convolution operations, upsampling, and connecting the results of attention operations on the next lower-level features, and then dividing regions corresponding to the optic nerve disc and the optic nerve cup, respectively, from the image with spatially emphasized features.

[0009] According to the present invention, by performing attention operations on features output from each layer of the encoder in an RGB image of the eye and connecting them with features calculated from the output of another layer, there is an effect of accurately segmenting the area corresponding to the optic nerve disc and the optic nerve cup.

[0010] According to the present invention, the effect of automatically diagnosing visual abnormalities using the region corresponding to the divided optic nerve disc and optic nerve cup can be expected.

[0011] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description in the claims.

[0012] FIG. 1 is a block diagram of an AI-based image processing device for segmenting optic nerve structures to diagnose visual abnormalities according to an embodiment of the present invention.

[0013] FIG. 2 is an internal block diagram of a convolution block attention block in an AI-based image processing device for segmenting an optic nerve structure to diagnose visual abnormalities according to an embodiment of the present invention.

[0014] FIG. 3 is an internal block diagram of a transformer encoder in an AI-based image processing device for segmenting an optic nerve structure to diagnose visual abnormalities according to an embodiment of the present invention.

[0015] An AI-based image processing device for segmenting optic nerve structures to diagnose visual abnormalities according to one aspect of the present invention for achieving the aforementioned purpose comprises: a first encoder composed of serially connected convolution blocks and a plurality of layer blocks to calculate multi-level features from an RGB image of an eye; a second encoder that receives the top-level features output from the first encoder, performs multi-head self-attention operations, and applies them to a multi-layer perceptron to calculate features that capture spatial dimension dependencies from the top-level features; and a decoder that calculates an image with spatially emphasized features by repeating a process of performing attention operations on each feature excluding the top-level features among the multi-level features output from the first encoder, connecting the results of attention operations on each feature starting from the upper level to the result of upsampling the features output from the second encoder, performing convolution operations, upsampling, and connecting the results of attention operations on the next lower-level features, and then dividing regions corresponding to the optic nerve disc and the optic nerve cup, respectively, from the image with spatially emphasized features.

[0016] The first encoder includes a convolution block that receives an RGB image of an eye and calculates features, a first layer block that receives the output of the convolution block and calculates features, a second layer block that receives the output of the first layer block and calculates features, a third layer block that receives the output of the second layer block and calculates features, and a fourth layer block that receives the output of the third layer block and calculates features.

[0017] The decoder upsamples the features output from the second encoder, and calculates the attention features by applying spatial attention and channel attention to the features output from the convolution block, the first layer block, the second layer block, and the third layer block of the first encoder, respectively; calculates a first connected feature by connecting the attention feature calculated from the features output from the third layer block to the result of upsampling the features output from the second encoder; calculates a second connected feature by connecting the attention feature calculated from the features output from the second layer block to the result of upsampling the first connected feature through convolution; calculates a third connected feature by connecting the attention feature calculated from the features output from the first layer block to the result of upsampling the second connected feature through convolution; and calculates a fourth connected feature by connecting the attention feature calculated from the features output from the convolution block to the result of upsampling the third connected feature through convolution. Calculate, and upsample the above-mentioned fourth connection feature to produce an image in which spatial features are emphasized.

[0018] The decoder calculates the attentioned features by sequentially performing channel attention operations and spatial attention operations on the features output from the convolution block, the first layer block, and the second layer block, respectively, and calculates the attentioned features by using the query, key, and value obtained from the features output from the third layer block to perform attention operations on the query and key, and by taking the inner product of the transpose matrix of the result of the attention operations on the query and key with the value.

[0019] An AI-based image processing device for segmenting optic nerve structures to diagnose visual abnormalities according to one aspect of the present invention further includes a visual abnormality detection unit that calculates ratio values ​​according to the structures of the optic nerve disc and the optic nerve cup based on regions corresponding to the optic nerve disc and the optic nerve cup, respectively, segmented from an image in which spatial features are emphasized in the decoder, and applies features including the calculated ratio values ​​to a preset classification model to detect whether there is a visual abnormality.

[0020] The above-mentioned visual abnormality detection unit calculates a pre-set directional segment length ratio value based on the distance from the center of the region corresponding to the optic nerve cup to the edge of the region corresponding to the optic nerve disc along a pre-set direction, and detects whether there is a visual abnormality by applying a feature including the directional segment length ratio value to a pre-set classification model.

[0021] The above-described visual abnormality detection unit calculates the area ratio of the area corresponding to the optic nerve cup and the area corresponding to the optic nerve disc in the four regions of the top, bottom, left, and right, which are distinguished by diagonally connecting the four corners of an image in which spatial features are emphasized, calculates a rim ratio value by dividing the sum of the area ratios calculated in the top and bottom regions by the sum of the area ratios calculated in the left and right regions, and detects whether there is a visual abnormality by applying the features including the rim ratio value to a pre-set classification model.

[0022] The advantages and features of the present invention, and the methods for achieving them, will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms; these embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the description of the claims. Meanwhile, the terms used in this specification are for describing the embodiments and are not intended to limit the present invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text.

[0023] The present invention is characterized by the technical feature of accurately segmenting the regions corresponding to the optic disc and optic cup in an RGB image of an eye.

[0024] These technical features can be achieved by a configuration that generates an image with emphasized detailed spatial features by refining and combining multi-level features extracted from an RGB image of the eye using attention operations, segments the optic nerve disc and optic nerve cup in the image with emphasized spatial features, calculates ratio values ​​representing structural characteristics from the segmented optic nerve disc and optic nerve cup, and applies them to a classification model.

[0025] Referring to FIG. 1, an AI-based image processing device (10) for segmenting an optic nerve structure to diagnose visual abnormalities according to one embodiment of the present invention may be configured to include a preprocessing unit (101), a first encoder (110), a second encoder (120), a decoder (200), a visual abnormality detection unit (300), and a loss function (400).

[0026] The preprocessing unit (101) can receive an RGB image of an eye taken from a user and generate an image that divides the area including the optic nerve disc and the optic nerve cup.

[0027] The preprocessing unit (101) can detect an optic nerve disc in an image of the retina of the eye taken from a user using a preset image processing algorithm and generate an image in which a box area containing the detected optic nerve disc is segmented.

[0028] Here, the detection of a box region including the optic nerve disc using a preset image processing algorithm may utilize existing object region detection algorithms and is not a core aspect of the present invention, so a detailed explanation is omitted.

[0029] The preprocessing unit (101) can improve computational efficiency in subsequent processes by splitting and outputting an image containing the optic nerve disc and optic nerve cup from an RGB image of the eye.

[0030] The first encoder (110) may be composed of serially connected convolution blocks (111) and a plurality of layer blocks (112, 113, 114, 115) to produce multiple levels of features from an RGB image containing an optic nerve disc and an optic nerve cup output from a preprocessing unit (101).

[0031] The first encoder (110) may be composed of a convolution block (111), a first layer block (112), a second layer block (113), a third layer block (114), and a fourth layer block (115).

[0032] The convolution block (111) can receive an RGB image containing an optic nerve disc and an optic nerve cup output from the preprocessing unit (101) and perform a convolution operation to produce features.

[0033] The first layer block (112) can receive the features output from the convolution block (111), perform a convolution operation, and produce features by adding the features prior to the convolution operation.

[0034] The second layer block (113) can receive the features output from the first layer block (112), perform a convolution operation, and produce features by adding the features prior to the convolution operation.

[0035] The third layer block (114) can receive the features output from the second layer block (113), perform a convolution operation, and produce features by adding the features prior to the convolution operation.

[0036] The fourth layer block (115) can receive the features output from the third layer block (114), perform a convolution operation, and produce features by adding the features prior to the convolution operation.

[0037] Here, the first encoder (110) can be implemented to consist of a convolution block (111), a first layer block (112), a second layer block (113), a third layer block (114), and a fourth layer block (115) by applying the existing ResNet18 structure excluding the last fully connected block.

[0038] The second encoder (120) may receive the top-level features output from the first encoder (110), perform multi-head self-attention operations, and apply them to a multi-layer perceptron to produce features that capture spatial dimension dependencies from the top-level features.

[0039] The second encoder (120) may be configured to include a first layer normalization block (121), an MSHA block (122), a first connector (123), a second layer normalization block (124), a multilayer perceptron block (125), and a second connector (126).

[0040] The first layer normalization block (121) may receive the features output from the fourth layer block (115), flatten the transpose matrix of the features output from the fourth layer block (115), connect it with a preset class token, and then add a preset positional bias to produce the output.

[0041] Features output from the first layer normalization block (121) ) can be expressed as shown in the following mathematical formula.

[0042]

[0043] Here, ( , The number of channels, is width, The height) is a feature output from the fourth layer block (115), and is a pre-configured class token, is a pre-set positional bias.

[0044] The MSHA block (122) is a feature output from the first layer normalization block (121) Projecting ) to obtain a query like the mathematical formula below (query, ), Key(Key, ), Value(Value, ) can be produced.

[0045]

[0046] Here, , , are each ( , , h is a learned weight matrix of the dimension of the pre-set number of heads.

[0047] And, the MSHA block (122) queries for each head ( ), key( ), Value( By performing attention operations in parallel and concatenating them, the result of a multi-head self-attention operation as shown in the mathematical formula below ( Can output ).

[0048]

[0049] Here, is the attention score for the i-th head and can be expressed as the mathematical formula below, and is the learned weight matrix.

[0050]

[0051] The first connector (123) is a feature input to the MSHA block (122) ) and input features( Features output from MSHA block (122) for ) By performing a residual connection of ) the characteristics as shown in the mathematical equation below ( It could be printing ).

[0052]

[0053] These residual connections can mitigate the problem of gradients disappearing during backpropagation and improve information flow within the layers.

[0054] The second layer normalization block (124) may receive the output of the first connector (123), normalize it layer, and output it.

[0055] The multi-layer perceptron block (125) may receive the output of the second layer normalization block (124) and pass it through a multi-layer perceptron (MLP) to output a feature of a preset size.

[0056] The second connector (126) may output the output of the first connector (123) and the output of the multilayer perceptron block (125) by residual connection.

[0057] The second encoder (120) may have a structure in which the first layer normalization block (121), MSHA block (122), first connector (123), second layer normalization block (124), multilayer perceptron block (125), and second connector (126) of FIG. 2 are repeated four times in sequence.

[0058] Accordingly, the second encoder (120) can capture dependencies across the spatial dimension of the input features and progressively improve the feature representation based on attention.

[0059] The decoder (200) may produce an image with spatially emphasized features by repeating the process of performing attention operations on each feature excluding the top-level feature among the multi-level features output from the first encoder (110), connecting the result of the attention operations on each feature starting from the top level to the result of upsampling the feature output from the second encoder (120), performing convolution operations, upsampling, and connecting the result of the attention operations on the next lower-level feature, and then producing an image with spatially emphasized features, and dividing regions corresponding to the optic nerve disk and the optic nerve cup, respectively, from the image with spatially emphasized features.

[0060] The decoder (200) may include a first CBAM block (201), a second CBAM block (202), a third CBAM block (203), an LSCA block (204), a first upsampling block (205), a first connection block (206), a first convolution upsampling block (207), a second connection block (208), a second convolution upsampling block (209), a third connection block (210), a third convolution upsampling block (211), a fourth connection block (212), a second upsampling block (213), a first segmentation block (214), a second segmentation block (215), and a third segmentation block (216).

[0061] The first CBAM block (201) may use a CBAM (Convolutional Block Attention Module) to perform spatial attention and channel attention on the features output from the convolution block (111) to produce the attentioned features.

[0062] The second CBAM block (202) may use CBAM to perform spatial attention and channel attention on the features output from the first layer block (112) to produce the attentioned features.

[0063] The third CBAM block (203) may use CBAM to perform spatial attention and channel attention on the features output from the second layer block (113) to produce the attentioned features.

[0064] Referring to FIG. 3, the first CBAM block (201), the second CBAM block (202), and the third CBAM block (203) may each perform a channel attention operation on an input feature input from the outside, combine the input feature with the feature produced by the channel attention operation to produce a first combined feature, perform a spatial attention operation on the first combined feature, and combine the first combined feature with the feature produced by the spatial attention operation to produce an attentioned feature by sequentially performing the channel attention operation and the spatial attention operation.

[0065] The LSCA block (204) may receive features output from the third layer block (114) and, based on Lightweight Spectral Convolution Attention (LSCA) operations, use the query, key, and value obtained from the features output from the third layer block (114) to perform attention operations on the query and key, and perform attention operations on the transpose matrix and value of the result of the attention operations on the query and key to produce the attentioned features.

[0066] The LSCA block (204) receives features output from the third layer block (114) and features input from the outside ( , is the batch size, is the channel size, A query like the following mathematical formula from (where is the spatial dimension of the input feature) ), key( ), Value( ) can be produced.

[0067]

[0068]

[0069]

[0070] Here, is a preset reduction ratio to increase computational efficiency by reducing the number of channels.

[0071] The LSCA block (204) is the generated query ( ) and Key( By performing attention operations on ) to obtain an attention matrix as shown in the mathematical formula below ( ) can be produced.

[0072]

[0073] And, the LSCA block (204) is a value ( The transpose of the attention matrix in ) By taking the inner product of ), the refined output as shown in the mathematical formula below ( ) can be produced.

[0074]

[0075] The LSCA block (204) is a refined output ( Pass ) through a 1×1 convolution layer to obtain input features as shown in the mathematical formula below ( It can be restored to the original number of channels, which is the number of channels of ).

[0076]

[0077] The LSCA block (204) is a feature restored to the original number of channels ( ) and input features( By forming residual connections through element-wise addition of ), characteristics such as those shown in the mathematical formula below ( It may be the final output of ).

[0078]

[0079] The first upsampling block (205) may receive a feature output from the second encoder (120) and use linear interpolation to upsample the feature so that its dimension increases by a preset multiple.

[0080] The first connection block (206) may concatenate the output of the first upsampling block (205) and the output of the LSCA block (204) to produce a first connection feature.

[0081] The first convolution upsampling block (207) may perform a convolution operation on the first connection feature output from the first connection block (206) and upsample it so that the dimension increases by a preset multiple.

[0082] The second connection block (208) may generate a second connection feature by connecting the output of the first convolution upsampling block (207) and the output of the third CBAM block (203).

[0083] The second convolution upsampling block (209) may perform a convolution operation on the second connection feature output from the second connection block (208) and upsample it so that the dimension increases by a preset multiple.

[0084] The third connection block (210) may generate a third connection feature by connecting the output of the second convolution upsampling block (209) and the output of the second CBAM block (202).

[0085] The third convolution upsampling block (211) may perform a convolution operation on the third connection feature output from the third connection block (210) and upsample it so that the dimension increases by a preset multiple.

[0086] The fourth connection block (212) may generate a fourth connection feature by connecting the output of the third convolution upsampling block (211) and the output of the first CBAM block (201).

[0087] The second upsampling block (213) may produce an image in which spatial features are emphasized by upsampling the fourth connection feature output from the fourth connection block (212).

[0088] The first segmentation block (214) may receive the output of the second upsampling block (213) and divide the region corresponding to the optic nerve disc and the region corresponding to the optic nerve cup.

[0089] The second segmentation block (215) may receive the fourth connection feature output from the fourth connection block (212) and divide the region corresponding to the optic nerve disc and the region corresponding to the optic nerve cup.

[0090] The third segmentation block (216) may receive the third connection feature output from the third connection block (210) and divide the region corresponding to the optic nerve disc and the region corresponding to the optic nerve cup.

[0091] Here, the first segmentation block (214), the second segmentation block (215), and the third segmentation block (216) are each composed of a pre-configured neural network structure for segmentation, and may be trained to segment the area corresponding to the optic nerve disc and the area corresponding to the optic nerve cup in an externally input image by receiving a training image in which the eye is captured and a ground truth in which the area corresponding to the optic nerve disc and the area corresponding to the optic nerve cup in the training image are labeled.

[0092] As a pre-configured neural network structure for segmentation, a previously disclosed segmentation neural network structure may be applied, and since this is not a core aspect of the present invention, a detailed description will be omitted.

[0093] The visual abnormality detection unit (300) may calculate ratio values ​​according to the structure of the optic nerve disc and optic nerve cup based on regions corresponding to the optic nerve disc and optic nerve cup, respectively, from the image in which spatial features are emphasized in the decoder (200), and detect the visual abnormality by applying features including the calculated ratio values ​​to a preset classification model.

[0094] Here, the visual abnormality may be glaucoma.

[0095] The visual anomaly detection unit (300) may be configured to include a numerical calculation unit (310), a GAP block (320), a first complete connection block (330), a connection block (340), a second complete connection block (350), and a third complete connection block (360).

[0096] The numerical calculation unit (310) may calculate ratio values ​​according to the structure of the optic nerve disc and the optic nerve cup from the region corresponding to the optic nerve disc and the region corresponding to the optic nerve cup divided in the decoder (200).

[0097] The numerical calculation unit (310) can calculate a predetermined directional segment length ratio value according to the distance from the center of the region corresponding to the optic nerve cup to the edge of the region corresponding to the optic nerve disc along a predetermined direction.

[0098] Referring to FIG. 4, the numerical calculation unit (310) calculates the distance (DL) to the edge of the area corresponding to the optic nerve disc and the distance (CL) to the edge of the area corresponding to the optic nerve cup along each preset direction of up, down, left, and right from the center of the area corresponding to the optic nerve cup, and can calculate the segment length ratio value (Segment Length Ratio) in each direction according to the following mathematical formula.

[0099]

[0100] Referring to FIG. 5, the numerical calculation unit (310) calculates the area ratio of the area (1002) corresponding to the optic nerve cup and the area (1001) corresponding to the optic nerve disc in the upper first region (Superior, S), lower second region (Inferior, I), left third region (Temporal, T), and right fourth region (Nasal, N), respectively, which are distinguished by diagonally connecting the four corners of the image in which spatial features are emphasized. , , , Calculate ) and the sum of the area ratios calculated from the top and bottom ( + ) is the sum of the area ratios calculated from the left and right ( + The rim ratio value can be calculated by dividing by ).

[0101] The numerical calculation unit (310) can calculate the rim ratio value (RAR) according to the following mathematical formula.

[0102]

[0103] The numerical calculation unit (310) can calculate a vertical ratio value by dividing the value of the vertical lengths of the area corresponding to the optic nerve cup by the value of the vertical lengths of the area corresponding to the optic nerve disc.

[0104] The numerical calculation unit (310) can calculate a horizontal ratio value by dividing the value of the horizontal lengths of the area corresponding to the optic nerve cup by the value of the horizontal lengths of the area corresponding to the optic nerve disc.

[0105] The numerical calculation unit (310) can calculate the rim-to-disk ratio value by dividing the value obtained by doubling the area of ​​the region corresponding to the optic nerve cup by the sum of the area of ​​the region corresponding to the optic nerve cup and the area corresponding to the optic nerve disc.

[0106] The numerical calculation unit (310) can calculate the Rim to Disc Area Ratio value according to the following mathematical formula.

[0107]

[0108] Here, is the area of ​​the region corresponding to the optic nerve cup, is the area of ​​the region corresponding to the optic nerve disc.

[0109] The numerical calculation unit (310) may generate and output a column vector containing four segment length ratio values, rim ratio values, vertical ratio values, horizontal ratio values, and rim-to-disk ratio values ​​for each of the calculated up, down, left, and right directions as components.

[0110] The GAP block (320) may output the features output from the second encoder (120) by global average pooling.

[0111] The first fully connected block (330) may fully connect the features of the GAP block (320) to output features in the form of a column vector.

[0112] The connecting block (340) may output a column vector containing four segment length ratio values, rim ratio values, vertical ratio values, horizontal ratio values, and rim-to-disk ratio values ​​for each direction of up, down, left, and right output from the numerical calculation unit (310), and a column vector output from the first complete connecting block (330) by concanating them.

[0113] The second complete connection block (350) may completely connect the output of the connection block (340) and convert it into a preset dimension for output.

[0114] The third complete connection block (360) may detect whether there is a visual abnormality by completely connecting the output of the second complete connection block (350) and outputting either a first classification value corresponding to the detection of a visual abnormality or a second classification value corresponding to the non-detection of a visual abnormality.

[0115] The loss function (400) may be calculated based on the visual abnormality detected by the visual abnormality detection unit (300), the region corresponding to the optic nerve disc and the region corresponding to the optic nerve cup divided by the decoder (200), and may be used to train the visual abnormality detection unit (300) and the decoder (200) through backpropagation.

[0116] The loss function (400) may consist of a classification loss based on the difference between the classification value of visual abnormality detected by the visual abnormality detection unit (300) and the actual value thereof, and a division loss based on the difference between the area corresponding to the optic nerve disc and the area corresponding to the optic nerve cup divided by the decoder (200) and the actual value thereof.

[0117] Loss function(400, ) can be expressed by the following mathematical formula.

[0118]

[0119] Here, is a split loss, and is the classification loss, and is the total loss function The classification loss in It is a hyperparameter to match the contribution of.

[0120] Partitioned loss ( ) can be expressed as shown in the mathematical formula below.

[0121]

[0122] Here, =0.2, =0.3, =0.5 and, It can be expressed as the splitting loss at scale i as shown in the mathematical formula below.

[0123]

[0124] Here, , , are the die loss, focal loss, and cross-entropy loss at scale i, respectively, and =1.0, =10.0, =10.0.

[0125] Die loss ( ) can be expressed as shown in the mathematical formula below.

[0126]

[0127] Here, The number of pixels, is pixels It is the probability value predicted from, and is the actual label value for it.

[0128] Focal loss ( ) can be expressed as shown in the mathematical formula below.

[0129]

[0130] Here, = It can be 2 days.

[0131] Cross-entropy loss ( ) can be expressed as shown in the mathematical formula below.

[0132]

[0133] And, classification loss ( ) can be expressed as shown in the mathematical formula below.

[0134]

[0135] Here, is a class It is the actual classification label value for, is the predicted probability value.

[0136] According to the above configuration, the present invention can accurately divide the optic nerve disc and the optic nerve cup from an image of the eye and diagnose whether there is a visual abnormality based on the structure of the divided optic nerve disc and the optic nerve cup.

[0137] Experiments were performed below to confirm that the segmentation performance of the optic nerve disc and optic nerve cup is improved according to the present invention.

[0138] For the experimental data, we used three datasets—REFUGE, RIM-ONE, and DRISHTI—which include images of normal and glaucomatous eyes and contain actual values ​​of the optic nerve disc and optic nerve cup segments, respectively.

[0139] ResNet34, ResNet50, and MobileNet were used as the existing technologies for comparison, and the experimental results are shown in the table below.

[0140] ModelDatasetOptic Disc DiceOD IoUOptic Cup DiceOC IoUResNet34REFUGE0.9580.9190.8870.797Drishti-GS0.9720.9450.9180.848RIM-ONE0.9720.9460.8930.807ResNet50REFUGE0.9530.9110.8860.795Drishti-GS0 .9720.9460.9150.843RIM-ONE0.9570.9170.8910.804MobileNetREFUGE0.9470.8 990.8830.791Drishti-GS0.9640.9310.9050.826RIM-ONE0.9690.9400.8820.789 copies invention REFUGE0.9690.9410.8900.806Drishti-GS0.9760.9480.9200.851RIM-ONE0.9700.9500.8930.808

[0141] Referring to Table 1, it can be seen that the present invention demonstrates higher performance compared to existing classification models across all datasets. In particular, compared to existing models, where the OC IoU score dropped below 0.8 depending on the type of dataset, the present invention shows a score exceeding 0.8, indicating that it consistently demonstrates superior performance in optic nerve cup segmentation.

[0142] Meanwhile, the blocks of the attached block diagram may be implemented as computer instructions that perform designated functions by being loaded into the processor or memory of an electronic device capable of data processing (e.g., a general-purpose computer, a specialized computer, a portable notebook computer, or a network computer). Since these computer program instructions can be stored in computer-readable memory, the functions described in the blocks of the block diagram may be produced as manufactured products containing means of instruction to perform them.

[0143] A person skilled in the art to which the present invention pertains will understand that the present invention may be implemented in other specific forms without altering its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the claims and their equivalents should be interpreted as being included within the scope of the present invention.

[0144] The present invention relates to a technology for detecting and segmenting optic nerve regions in an image of an eye.

Claims

1. A first encoder composed of serially connected convolution blocks and multiple layer blocks to calculate multi-level features from an RGB image of an eye; A second encoder that receives the top-level feature output from the first encoder, performs a multi-head self-attention operation, and applies it to a multi-layer perceptron to produce a feature that captures spatial dimensional dependencies in the top-level feature; and An AI-based image processing device for segmenting optic nerve structures to diagnose visual abnormalities, comprising: a decoder that generates an image with spatially emphasized features by repeating the process of performing an attention operation on each feature excluding the highest level feature among the multi-level features output from the first encoder, upsampling the feature output from the second encoder, connecting the result of the attention operation on each feature starting from the upper level, performing a convolution operation, upsampling, and then connecting the result of the attention operation on the next lower level feature, and segmenting regions corresponding to the optic nerve disc and optic nerve cup, respectively, from the image with spatially emphasized features.

2. In Paragraph 1, The above-mentioned first encoder is A convolution block that receives an RGB image of an eye and calculates features, a first layer block that receives the output of the convolution block and calculates features, a second layer block that receives the output of the first layer block and calculates features, a third layer block that receives the output of the second layer block and calculates features, and a fourth layer block that receives the output of the third layer block and calculates features. AI-based image processing device for segmenting optic nerve structures to diagnose visual abnormalities.

3. In Paragraph 2, The above decoder is Upsampling the features output from the second encoder above, and Attention features are produced by applying spatial attention and channel attention to the features output from the convolution block, the first layer block, the second layer block, and the third layer block of the first encoder, respectively. A first connected feature is calculated by connecting the attention feature derived from the feature output from the third layer block to the result of upsampling the feature output from the second encoder, and A second connected feature is calculated by convolutionally upsampling the result of the first connected feature and connecting the attention feature calculated from the feature output from the second layer block to the result, and A third connected feature is calculated by convolutionally upsampling the result of the second connected feature and connecting the attention feature calculated from the feature output from the first layer block, and A fourth connected feature is calculated by concatenating the attention feature derived from the feature output from the convolution block to the result of upsampling the third connected feature by convolution operation, and Upsampling the above-mentioned fourth connection feature to produce an image with emphasized spatial features AI-based image processing device for segmenting optic nerve structures to diagnose visual abnormalities.

4. In Paragraph 3, The above decoder is Attention features are produced by sequentially performing channel attention operations and spatial attention operations on the features output from the convolution block, the first layer block, and the second layer block, respectively, and Using the query, key, and value obtained from the feature output from the third layer block above, performing an attention operation on the query and key, and producing an attention feature by taking the inner product of the value and the transpose matrix of the result of the attention operation on the query and key. AI-based image processing device for segmenting optic nerve structures to diagnose visual abnormalities.

5. In Paragraph 1, An AI-based image processing device for segmenting optic nerve structures to diagnose visual abnormalities, further comprising: a visual abnormality detection unit that calculates ratio values ​​according to the structure of the optic nerve disc and optic nerve cup based on regions corresponding to the optic nerve disc and optic nerve cup, respectively, segmented from an image in which spatial features are emphasized in the above decoder, and detects whether there is a visual abnormality by applying features including the calculated ratio values ​​to a preset classification model.

6. In Paragraph 5, The above-mentioned visual abnormality detection unit Calculating a pre-set directional segment length ratio value based on the distance from the center of the region corresponding to the optic nerve cup along a pre-set direction to the edge of the region corresponding to the optic nerve disc, and detecting the presence of visual abnormalities by applying a feature including the directional segment length ratio value to a pre-set classification model. AI-based image processing device for segmenting optic nerve structures to diagnose visual abnormalities.

7. In Paragraph 5, The above-mentioned visual abnormality detection unit Calculating the area ratios of the region corresponding to the optic nerve cup and the region corresponding to the optic nerve disc in the four regions (top, bottom, left, and right) distinguished by diagonally connecting the four corners of an image with emphasized spatial features, calculating a rim ratio value by dividing the sum of the area ratios calculated in the top and bottom regions by the sum of the area ratios calculated in the left and right regions, and detecting the presence of visual abnormalities by applying features including the rim ratio value to a pre-configured classification model. AI-based image processing device for segmenting optic nerve structures to diagnose visual abnormalities.

Images