A light-weight cup segmentation method based on kink points

Abstract

The application discloses a light-weight optic cup segmentation method based on kink points and belongs to the field of fundus retinal image processing. The light-weight optic cup segmentation network learns consistent deep optic cup features from the fundus retinal image without eliminating the optic cup boundary and the fundus retinal image eliminating the optic cup boundary, further uses the vascular kink point constraint loss and the adversarial learning loss to make the light-weight optic cup segmentation network pay more attention to the vascular kink points in the fundus retinal image, realizes more accurate prediction of the optic cup boundary by using the vascular kink points, and further constructs the local information calculation module Local-FFN and the global information sparse calculation module Global-FFN, so that the light-weight optic cup segmentation network has the advantages of small parameter quantity and the ability to directly model global feature information. The application has the advantages of high optic cup segmentation prediction accuracy and small calculation amount, and is easy to deploy on mobile devices such as smart phones and tablets with limited computing resources.

Description

Technical Field

[0001] This invention belongs to the field of fundus retinal image processing and relates to a lightweight cup segmentation method based on knots. Background Technology

[0002] The optic cup is the most critical anatomical structure in fundus retinal images. Changes in its shape, size, and boundaries directly reflect the occurrence and progression of various eye diseases. Manual segmentation of the optic cup has always been an important part of ophthalmological clinical work. However, traditional manual segmentation methods are inefficient and prone to significant subjective errors, becoming a major bottleneck in assisting clinical ophthalmological diagnosis.

[0003] In recent years, deep learning technology, which has emerged in the field of medical image analysis, has provided new ideas for automated optic cup segmentation. By constructing a convolutional neural network, the segmentation results of the optic cup can be output end-to-end after inputting a fundus retinal image. This not only greatly improves segmentation efficiency but also enables computer vision to play an important role in clinical ophthalmology.

[0004] However, existing deep learning-based optic disc and optic cup segmentation methods still face some challenges in practical applications: 1) The existing segmentation network structures have a large number of parameters, resulting in a high computational load and excessively long inference time, which is not conducive to clinical needs; the large number of model parameters also makes these segmentation models bulky and difficult to deploy on mobile devices such as smartphones and tablets. However, mobile healthcare has advantages such as portability, convenience, and wide coverage, making the development of mobile optic disc and optic cup segmentation models of great significance. These problems limit the clinical application of current optic disc and optic cup segmentation technology and also hinder computer vision from playing a greater role in assisting ophthalmological clinical diagnosis. Therefore, designing an optic disc and optic cup segmentation network that balances inference speed, segmentation accuracy, generalization, and focus on key areas is of great significance for improving the level of ophthalmological clinical diagnosis. 2) Network optimization relies too much on dataset annotation and lacks a deep understanding of medical images, resulting in large segmentation biases; how to combine deep learning-based methods with the expert knowledge of professional doctors is a challenge in assisting medical feature extraction.

[0005] In clinical practice, when determining the optic cup boundary in fundus retinal images, physicians primarily combine chromatic aberration information around the optic cup boundary with expert knowledge of "vascular tortuosity." This is because vascular tortuosity has a strong correlation with the optic cup in real three-dimensional space; it serves as a crucial indicator of the optic cup boundary. Optic nerve fibers bend at vascular tortuosity before passing through the eyeball and connecting to nerve fibers in the brain. How to effectively integrate vascular tortuosity, as a significant piece of expert knowledge, with deep learning methods has become a hot topic in the current technological field. Summary of the Invention

[0006] To address the issues of large parameter counts and lack of physician expert knowledge in existing methods, this invention aims to provide a lightweight optic cup segmentation method based on kinks. This method enables a lightweight optic cup segmentation network to mine the relationship between vascular kinks and the optic cup boundary, introducing prior knowledge into the lightweight segmentation network to achieve more accurate prediction of the optic cup boundary using vascular kinks. Furthermore, by constructing a Local-FFN module for local information computation and a Global-FFN module for global information sparse computation, the lightweight optic cup segmentation network possesses the ability to have a small parameter count and directly model global feature information. This invention offers the advantages of high optic cup segmentation prediction accuracy and low computational cost, and is easily deployed on mobile devices such as smartphones and tablets with limited computing resources.

[0007] The objective of this invention is achieved through the following technical solution:

[0008] This invention discloses a lightweight optic cup segmentation method based on knots. During training, the lightweight optic cup segmentation network learns consistent deep optic cup features from both fundus retinal images with and without optic cup boundaries. Furthermore, it uses vascular knot constraint loss and adversarial learning loss to increase the network's emphasis on expert knowledge of vascular knots in fundus retinal images, thereby achieving more accurate prediction of the optic cup boundary using vascular knots. In addition, this invention constructs two basic computational modules for the lightweight optic cup segmentation network: a local information computation module and a global information sparse computation module. The local information computation module models feature information only within a local area and further feeds the initially extracted local feature information into a spatially and channel-separated FFN computation module. The spatially and channel-separated FFN computation module is a key module that can both extract feature information and reduce the number of parameters. The global information sparse computation module extracts information from the global information at intervals to obtain sparse information. This interval extraction achieves both data sparsity and coverage of global information sampling. The extracted sparse information is further fed into the FFN computation module based on spatial and channel separation for feature extraction. After training, the lightweight optic cup segmentation network uses the fundus retinal image X as input to predict the segmentation result of the optic cup, i.e., lightweight optic cup segmentation is achieved based on kinks.

[0009] This invention discloses a lightweight view cup segmentation method based on kinks, comprising the following steps:

[0010] Step 1: Preprocess the fundus retinal images for the training and test sets. This preprocessing includes cropping the optic disc region from the fundus retinal images and standardizing the resolution of the cropped images to obtain standardized training and test sets. These sets are used for training and testing the optic cup segmentation method, respectively. Both the standardized training and test sets contain fundus retinal images X and consistent optic cup segmentation labels Y.

[0011] Step 1.1: Use an optic disc detector to detect the optic disc in the fundus retinal image. Then, crop the fundus retinal image containing the optic disc region using the center of the optic disc as the cropping center. The cropped fundus retinal image is a square, and the side length of the square is N times the diameter of the optic disc. OC times.

[0012] In step 1.1, the optic disc detector selects either the U-Net segmentation network or the optic disc center detection method based on the YOLO detection network to initially locate the position of the optic disc in the fundus retinal image; based on the initially located position, a square area of ​​the fundus retinal image is cropped with the initially located position of the optic disc as the center.

[0013] The optic disc diameter obtained in step 1.1 is as follows: calculate the length of the optic disc in the horizontal and vertical directions in the fundus retinal image, and select the longest value in the horizontal and vertical directions as the diameter of the optic disc.

[0014] Step 1.2: Compress the resolution of the cropped image to a predetermined resolution (H, W) to obtain a resolution-normalized image, thus achieving resolution normalization. Preferably, H ranges from 224 to 512, W ranges from 224 to 512, and H = W.

[0015] Step 2: Construct a preprocessing module focusing on vascular tortuosities, and train a lightweight optic cup segmentation network based on this preprocessing module. The preprocessing module focusing on vascular tortuosities performs local blurring of the optic cup boundary on the standardized fundus retinal images from Step 1. This blurring blurs the chromatic aberration of the optic cup boundary in the fundus retinal images, keeping only the vascular tortuosity region near the optic cup boundary clear. Because the vascular tortuosity and the chromatic aberration of the optic cup boundary are the two most important factors determining the optic cup boundary, and deep learning methods are based on automatic feature extraction, they typically use the simpler chromatic aberration of the optic cup boundary as the feature for extraction, thus neglecting vascular tortuosities. To ensure that the lightweight optic cup segmentation network of this invention focuses on vascular tortuosities, the preprocessing module forcibly removes the chromatic aberration information of the optic cup boundary. During training, the lightweight optic cup segmentation network extracts optic cup information only by mining the information from vascular tortuosities, allowing it to fully utilize both simple chromatic aberrations and the information from vascular tortuosities.

[0016] Step two focuses on training a lightweight optic cup segmentation network using a preprocessing module for vascular knots, specifically implemented as follows:

[0017] Step 2.1: Determine the extent of the blurred smooth area in the fundus retinal image. Extend the coverage area to both sides of the boundary line marked Y on the optic disc in the training set as the midline. The width of the extension is equal to the diameter of the optic disc described in Step 1. The obtained extended coverage area is an approximately circular strip, and the process of extending the coverage is denoted as band(·). This approximately circular strip is used as the initial blur smoothing range mask1. The calculation process of obtaining the blur smoothing range mask1 in the fundus retinal image is denoted as formula (1).

[0018] mask1 = band(Y) (1)

[0019] Step 2.2: Narrow the initial blur smoothing range described in Step 2.1. Calculate the intersecting region of the vessel (Vessel) in the retinal fundus photograph as the vessel intersection region. Exclude the vessel intersection region from the initial blur smoothing range to obtain the blur smoothing range mask2. The calculation process of the blur smoothing range mask2 is denoted by formula (2).

[0020] mask2=mask1∩(mask1∩Vessel) c (2)

[0021] Step 2.3: Perform blur smoothing on the fundus image X according to the blur smoothing range to blur the chromatic aberration at the optic cup boundary, obtaining a blurred smoothed image. The blur smoothing process is denoted as blur(·). The chromatic aberration information at the optic cup boundary is removed from the blurred smoothed image, which only contains information about the vascular knot points related to the optic cup boundary. Therefore, the obtained blurred smoothed image is denoted as the vascular knot point fundus retinal image X. kink The blur smoothing method used is Gaussian blur. The calculation process for blur smoothing the fundus X-ray based on the blur smoothing range mask2 is denoted as formula (3).

[0022] X kink =blur(X|mask2) (3)

[0023] Step 2.4: The output of the preprocessing module focusing on vascular kinetic points is the fundus retinal image X of the vascular kinetic points from Step 2.3. kink The input X to the preprocessing module for focusing on vascular kinks is also used as the output of the preprocessing module for focusing on vascular kinks.

[0024] Step 3: Construct a lightweight view cup segmentation network for segmenting view cup features. This lightweight view cup segmentation network consists of an encoder E and a decoder D. The encoder E comprises two local information computation modules (Local-FFN) and two global information sparse computation modules (Global-FFN). The Local-FFNs are denoted as L-FFN and the Global-FFNs as G-FFN. The two local information computation modules (L-FFN) of the encoder E have the same structure but different feature sizes; the feature size of the first L-FFN is twice that of the second L-FFN. Similarly, the two global information sparse computation modules (G-FFN) of the encoder E have the same structure but different feature sizes; the feature size of the first G-FFN is twice that of the second G-FFN. The decoder D contains the same number of global information sparse computation modules (G-FFN) and local information computation modules (L-FFN) as the encoder E, meaning the decoder D consists of two global information sparse computation modules and two local information computation modules. The two global information sparse computation modules (G-FFN) of the decoder D have the same structure but different feature sizes; the feature size of the latter G-FFN is twice that of the former. Similarly, the two local information computation modules (L-FFN) of the decoder D have the same structure but different feature sizes; the feature size of the latter L-FFN is twice that of the former.

[0025] The encoder E of the lightweight optic cup segmentation network described in step three is used in the fundus retinal image X of the vascular knot point in step 2.4. kink Taking the fundus retinal image X as input, and processing X... kink Feature extraction is performed on X to obtain the encoded image features. The encoder E contains two local information calculation modules L-FFN and two global information sparse calculation modules G-FFN. These calculation modules are denoted as encoder local calculation modules. Encoder Local Calculation Module Encoder Global Information Sparse Computation Module and encoder global information sparse computation module in and Each of these has a dimensionality reduction module, Stack, which are denoted as follows: and The dimensionality reduction module is used to reduce the height and width of features and increase the number of channels in features. The dimensionality reduction module can be seen as having a 2×2 computational kernel in the height and width dimensions, with a step size of 2, adjusting the information of the 2×2 range to the channel dimension, thus reducing the height and width by a factor of 2 and increasing the channel dimension by a factor of 4.

[0026] Dimensionality reduction module Using the fundus retinal image X∈R from step two H×W×C or X kink As input, taking X as an example, we perform dimensionality reduction on X to obtain the dimensionality-reduced features. Features after dimensionality reduction Scale is Encoder Local Calculation Module With the dimensionality-reduced features Features of input and output encoders Its feature dimension is H represents height, W represents width, and C represents the number of channels; dimensionality reduction module The calculation process and encoder local calculation module The calculation process is denoted as formulas (4) and (5) respectively.

[0027]

[0028]

[0029] Dimensionality reduction module Based on features As input, for features Perform dimensionality reduction to obtain the reduced features. Features after dimensionality reduction Scale is Encoder Local Calculation Module Based on encoder features Features of input and output encoders Its feature dimension is Dimensionality reduction module The calculation process and encoder local calculation module The calculation process is denoted as formulas (6) and (7) respectively.

[0030]

[0031]

[0032] Dimensionality reduction module Based on features As input, for features Perform dimensionality reduction to obtain the reduced features. Features after dimensionality reduction Scale is Encoder Global Information Sparse Computation Module Based on encoder features Features of input and output encoders Its feature dimension is Dimensionality reduction module The calculation process and encoder global information sparse calculation module The calculation process is denoted as formulas (8) and (9).

[0033]

[0034]

[0035] Dimensionality reduction module Based on features As input, for features Perform dimensionality reduction to obtain the reduced features. Features after dimensionality reduction Scale is Encoder Global Information Sparse Computation Module Based on encoder features Features of input and output encoders Its feature dimension is Dimensionality reduction module The calculation process and encoder global information sparse calculation module The calculation process is denoted as formulas 10 and 11, respectively.

[0036]

[0037]

[0038] The decoder D of the lightweight view cup segmentation network described in step three uses encoder features. As input, for features Decoding yields the decoded image cup features. The decoder D comprises two global information sparse computation modules G-FFN and two local information computation modules L-FFN. These computation modules are denoted as follows: Global Information Sparse Computation Module (G-FFN) Decoder Global Information Sparse Computation Module Decoder Local Computation Module and decoder local computation module in, and Each of these is followed by an upscaling module, Unstack, which increases the height and width scales, and is denoted as follows: and The dimensionality-upgrading module is used to increase the height and width of features while reducing the number of channels in the features. The dimensionality-upgrading module transfers channel information to the height and width dimensions, that is, information transfer can be achieved through deformation, so that the feature increases in height and width by 2 times, while the amount of information in the channel dimension decreases by 4 times.

[0039] Decoder Global Information Sparse Computation Module Based on encoder features Features of input and output decoders Its feature dimension is Upgrade Module Based on features As input, for features Perform dimensionality increase to obtain the features after dimensionality increase. Features after dimensional upgrade Scale is Decoder Global Information Sparse Computation Module Calculation process and dimension-upgrading module The calculation process is denoted as formulas 12 and 13.

[0040]

[0041]

[0042] Decoder Global Information Sparse Computation Module Based on encoder features Features of input and output decoders Its feature dimension is Upgrade Module Based on features As input, for features Perform dimensionality increase to obtain the features after dimensionality increase. Features after dimensional upgrade Scale is Decoder Global Information Sparse Computation Module The calculation process and the dimension-upgrading module The calculation process is denoted as formulas ⒁ and ⒂.

[0043]

[0044]

[0045] Decoder Local Computation Module Based on encoder features Features of input and output decoders Its feature dimension is Upgrade Module Based on features As input, for features Perform dimensionality increase to obtain the features after dimensionality increase. Features after dimensional upgrade Scale is Decoder Local Computation Module and Upgrade Module The calculation process is denoted as formulas ⒃ and ⒄.

[0046]

[0047]

[0048] Decoder Local Computation Module Based on encoder features Features of input and output decoders Its feature dimension is Upgrade Module Based on features As input, for features Perform dimensionality increase to obtain the features after dimensionality increase. Features after dimensional upgrade The scale is R H×W×C Decoder Local Computation Module The calculation process and the dimension-upgrading module The calculation process is denoted as formulas ⒅ and ⒆.

[0049]

[0050]

[0051] Step 4: Define the structure of the Local Information Calculation Module (L-FFN). The L-FFN module consists of a local data sampling layer and an FFN calculation layer. The local data sampling layer defines the local range of data sampling, allowing the L-FFN module to focus only on the relationships between local data points without concern for relationships between distant information. Because it focuses only on the relationships between local data points, the L-FFN module is better at modeling relationships between local feature information. Furthermore, the FFN calculation layer is a spatially and channel-separated FFN calculation module. Separating spatial and channel calculations effectively reduces the computational load and the number of parameters, mitigating the problem of large parameter counts inherent in fully connected layers.

[0052] The local information calculation module L-FFN described in step four consists of a local data sampling layer and an FFN calculation layer. The specific implementation method includes steps 4.1 to 4.3:

[0053] Step 4.1: Define the input of the Local Information Calculation Module (L-FFN) as features. Where H i W i and C iThese represent the height, width, and number of channels of the input feature, respectively.

[0054] Step 4.2: Construct a local data sampling layer to process the features f input to the local information calculation module. input1 First, local blocks are divided, which is used to determine the scope of local information.

[0055] The local block division described in step 4.2 is as follows: the input feature f input1 According to P local1 ×P local1 The local blocks of different sizes are divided into non-overlapping sections according to their height and width, resulting in non-overlapping feature blocks. Its dimensions are Then features Further transform the dimensions into feature blocks Its dimensions are The local data sampling layer is represented by formula ⒇ as follows.

[0056]

[0057] Step 4.3: Construct the FFN computation layer. The FFN computation layer uses features... The input is used for feature extraction. The FFN computational layer consists of two sets of computational sublayers, denoted as FFN1 and FFN2. FFN1 is used to compute spatial information relationships, while FFN2 is used to model inter-channel information relationships. Both FFN1 and FFN2 include two fully connected layers, each followed by an activation function. The FFN computational layer has low computational cost and few parameters because it separates the computation of spatial information and the computation between channels, which significantly reduces the computational cost and the number of parameters.

[0058] In step 4.3, the first group of calculation sub-layers FFN1 pairs of features The calculation method is as follows:

[0059] The first computational sublayer FFN1 consists of two fully connected layers, with P being the number of neurons in the two consecutive fully connected layers. local1 ·P locla1 The input to the fully connected layer is Depend on It consists of vectors, each vector having a dimension of . Each vector is passed through the two fully connected layers, and the vector generated by the two fully connected layers is... The calculation process of the first sublayer FFN1 of L-FFN is denoted by formula (21).

[0060]

[0061] The second computational sublayer FFN2 outputs features based on the first computational sublayer FFN1. Feature extraction is performed on the input. The second computational sublayer FFN2 consists of two fully connected layers, with C neurons in the two consecutive fully connected layers. i The second group calculates sublayer FFN2 with... For input, Transform dimensionality into feature vectors Depend on It consists of vectors, each vector having a dimension of C. i . The input is fed into two fully connected layers, which process the features. Perform feature extraction and output features And further deform the shape of the feature, f local1 The dimension of the feature becomes The calculation process of the second sublayer L-FFN2 of L-FFN is denoted by formula (22).

[0062]

[0063] Step 5: Define the structure of the Global Information Sparse Computation Module (G-FFN). The G-FFN consists of a global data sparse sampling layer and an FFN computation layer. The global data sparse sampling layer samples features from the entire image, covering information from the entire fundus retina image. The sampling points are regularly spaced across the entire image. This global data sparse sampling layer achieves global information sampling and reduces the number of sampling points through sparse sampling. Therefore, the global data sparse sampling layer enables the G-FFN to model global information feature relationships and reduces the cost of the G-FFN.

[0064] Step 5.1: Define the input of the Global Information Sparse Computation Module G-FFN as features Where H j W j and C j These are the feature's height, width, and number of channels, respectively.

[0065] Step 5.2: Construct a global data sparse sampling layer for the features f input to the global information sparse computation module.input2 The data is divided into blocks, which are used for global sparse sampling. Preferably, the global sparse sampling method is interval sampling.

[0066] The input features f input2 According to P g ×P g The local blocks of different sizes are divided into non-overlapping feature blocks according to their height and width directions. Its dimensions are Then the feature block Further transform the dimension into feature blocks Its dimensions are Equivalent to having Each dimension is Feature blocks In P g ·P g Select dimension i from the dimensions in which it is located. The dimension is eigenvectors, indivual Feature vectors are aggregated into new feature blocks. Its dimensions are Similarly, index i traverses P g ·P g The dimension of P is obtained g ·P g indivual Obtain global feature blocks Its dimensions are Merging global feature blocks The second and third dimensions are transformed into features. Its dimensions are The global sparse sampling layer is represented by formula (23).

[0067]

[0068] Step 5.3: Construct the FFN computation layer, which uses features... The input is used for feature extraction. The FFN computation layer consists of two sets of FFN computation sub-layers, denoted as the first set of computation sub-layers FFN1 and the second set of computation sub-layers FFN2. The first set of computation sub-layers FFN1 is used to compute the relationship between spatial information, and the second set of computation sub-layers FFN2 is used to model the relationship between information between channels. Both the first set of computation sub-layers FFN1 and the second set of computation sub-layers FFN2 include two fully connected layers, each followed by an activation function.

[0069] In step 5.3, the first group of calculation sub-layers FFN1 is used to calculate the features. The calculation method is as follows:

[0070] The first computational sublayer FFN1 consists of two fully connected layers, and the number of neurons in the two consecutive fully connected layers is... The input to the fully connected layer is Equivalent to C j ·P g ·P g There are n vectors, each with dimensions of 1. C j ·P g ·P g Each vector is passed through the two fully connected layers, and the vectors generated by the two fully connected layers are... The calculation process of the first calculation sublayer G-FFN1 of G-FFN is denoted by formula (24).

[0071]

[0072] The second computational sublayer FFN2 outputs features based on the first computational sublayer FFN1. Feature extraction is performed on the input. The second computational sublayer FFN2 consists of two fully connected layers, with C neurons in the two consecutive fully connected layers. j The second group calculates sublayer FFN2 with... For input, Transform dimensions into feature vectors Equivalent to The dimension of each vector is C. j .Then The input is fed into two fully connected layers, which process the features. Perform feature extraction and output features And further deform the shape of the feature, f global1 The dimension of the feature becomes The calculation process of the second sublayer G-FFN2 of G-FFN is denoted by formula (25).

[0073]

[0074] Step Six: Based on the input dimensions of different features in Step Three, the local information computation module structure constructed in Step Four is sequentially embedded into the encoder and decoder in Step Three; and the global information sparse computation module structure constructed in Step Five is sequentially embedded into the encoder and decoder in Step Three. Based on the encoder and decoder with embedded local information computation module and global information sparse computation module structures, a view cup segmentation network with embedded local information computation module and global information sparse computation module is constructed, resulting in a structurally complete lightweight view cup segmentation network. This lightweight view cup segmentation network not only has the advantages of small parameter count and low computational cost, but also the ability to model both local and global features. The small parameter count and low computational cost of the lightweight segmentation network are due to the FFN computation layer calculating spatial and channel features independently, and the limitation on the number of sampling points by the local data sampling layer of the local information computation module and the global data sparse sampling layer of the global information sparse computation module. Compared to the convolution operators in convolutional neural networks, the global information sparse computation module has the ability to model global information because it performs sparse sampling of global information and then feeds the sampled global information into the FFN computation layer to mine the relationships between global features.

[0075] Substitute formulas (20)-(22) into formula (5) as steps to be executed sequentially, and similarly substitute them into formulas (7), (16), and (18).

[0076] Substitute formulas (23)-(25) into formula (9) as steps to be executed sequentially, and similarly substitute them into formulas (11), (12), and (14).

[0077] Step 7: Construct the total loss function L Total The encoder E and decoder D are used to train the lightweight view cup segmentation network. The network parameters obtained after training on the training set in step one are used as the final parameters of the lightweight view cup segmentation network. The loss function of this invention consists of three parts, namely the knot constraint loss L proposed in this invention. kink Combating learning loss L adv And Dice lost L Dice Among them, the torsional node constraint loss L kink This refers to the fundus image X generated in step two after eliminating the optic cup boundary. kink The deep features generated by encoder E are consistent with those of fundus image X (without removing the optic cup boundary). By ensuring consistency of these deep features, encoder E can learn deep features of the optic cup information from fundus images with chromatic aberration information removed from the optic cup boundary. These deep features can be obtained solely from vascular knots in fundus images with chromatic aberration information removed from the optic cup boundary. This is achieved through knot-constrained loss L. kinkThe constraints enable lightweight optic cup segmentation networks to better utilize prior knowledge to assist in optic cup segmentation, namely, vascular knots to assist in optic cup boundary segmentation.

[0078] The knot constraint loss L mentioned in step seven kink The following calculation is performed: the fundus image X generated in step two after eliminating the optic cup boundary. kink The fundus image X with and without the optic cup boundary removed are input into encoder E, and features are generated according to formulas (4) to (11). Will be X kink Features generated for the input encoder E Recorded as The features generated by encoder E with X as input It is denoted as F.

[0079] Feature F and feature The comparison is performed point-by-point along both length and width dimensions, using a combination of Euclidean distance and cosine similarity (cosine distance). Features F and features... The distance loss function is calculated according to formula (26).

[0080]

[0081] in, F represents the value of the feature at position i in the high dimension, position j in the wide dimension, and position c in the channel dimension. ij This represents the vector whose feature is located at position i in the height of dimension i and position j in the width of dimension j. Similarly... and They have similar meanings.

[0082] Step 7: Adversarial learning loss L adv The goal is to improve the cup segmentation results predicted by the lightweight cup segmentation network. In terms of the overall part, it resembles the real view cup segmentation result Y more closely because adversarial learning focuses on global information and can therefore grasp the overall distribution of the segmentation result. Compared with the pixel-by-pixel comparison Dice loss, the adversarial learning loss L adv It can eliminate the predicted cup segmentation results Discontinuous phenomena. Adversarial learning loss L adv The specific calculation method is as follows:

[0083] Construct a binary classification network Dis to analyze the predicted view cup segmentation results. The predicted view cup segmentation result is classified with the actual view cup segmentation result Y, thus enabling the classification of the predicted view cup segmentation result. The image is classified into class 0, representing a predicted segmentation result. Simultaneously, the binary classification network classifies the actual view cup segmentation result Y into class 1, representing a true view cup segmentation label. The lightweight view cup segmentation network aims to predict the view cup segmentation result. The result is very similar to the actual view cup segmentation Y, making it difficult for the binary classification network to accurately classify the view cup. Classify Y and Y. Adversarial learning loss L adv Calculate according to formula (27).

[0084]

[0085] in denoted by , and Log represents the expected value.

[0086] In step seven, Dice loss L Dice The goal is to compare the predicted cup segmentation results pixel by pixel. The difference between the predicted and actual cup segmentation results Y is used to optimize the predicted cup segmentation results. Dice loss is calculated according to formula (28).

[0087]

[0088] Where Y ij This represents the value of Y at position i in height dimension and j in width dimension; express The values ​​at positions i in the height dimension and j in the width dimension.

[0089] In step seven, the total loss function is the loss L. kink Loss L adv and loss L Dice The weighted sum of the values. The final total loss is calculated according to formula (29).

[0090] L Total =αL kink +βL adv +γL Dice (29)

[0091] Where α, α, and γ represent weights.

[0092] Step 8: Use the lightweight optic cup segmentation network trained in Step 7 for testing until a well-trained lightweight optic cup segmentation network is obtained. Using the fundus retinal image X from the test set in Step 1 as input, use the trained lightweight optic cup segmentation network to predict the segmentation result of the optic cup. This means that lightweight visual cup segmentation is achieved based on kink points; during the testing process, the preprocessing module focusing on vascular kink points in step two is not required.

[0093] Beneficial effects:

[0094] 1. This invention discloses a lightweight optic cup segmentation method based on kink points. It introduces a preprocessing module focusing on vascular kink points for training a lightweight optic cup segmentation network. This preprocessing module removes simple chromatic aberration points at the optic cup boundary from fundus retinal images and retains only vascular kink points used by professional physicians in the fundus retinal images. This allows the lightweight optic cup segmentation network to mine the relationship between vascular kink points and the optic cup boundary as expert knowledge, thereby introducing the physician's prior knowledge into the lightweight segmentation network and achieving more accurate optic cup segmentation prediction.

[0095] 2. To better enable lightweight optic cup segmentation networks to focus on expert knowledge of vascular tortuosities, this invention discloses a lightweight optic cup segmentation method based on tortuosities. This method uses tortuosity constraint loss to ensure that the deep features of the fundus retinal image after processing by a preprocessing module that focuses on vascular tortuosities and eliminates chromatic aberration at the optic cup boundary are consistent with those of the corresponding unprocessed fundus retinal image. This allows the lightweight optic cup segmentation network to better learn the relationship between vascular tortuosities and the optic cup boundary.

[0096] 3. To reduce the number of parameters and computational cost of the segmentation network, this invention discloses a lightweight view cup segmentation method based on kinks, constructing a Local-FFN local information computation module and a Global-FFN global information sparse computation module. The Local-FFN module limits the number of parameters and computational cost by restricting the data sampling range to local data. Furthermore, the FFN computation layer in Local-FFN calculates the spatial and channel dimensions independently, further reducing the number of parameters and computational cost. The Global-FFN module samples the global data at regular intervals, limiting the number of samples while obtaining global data information. This makes the Global-FFN module capable of efficiently modeling global data information with a small number of parameters. Compared with convolutional neural networks based on convolution operators, the lightweight view cup segmentation network of this invention has a small number of parameters and the ability to directly model global feature information. Therefore, the lightweight view cup segmentation network of this invention has stronger global modeling capabilities. Attached Figure Description

[0097] Figure 1 This invention discloses a lightweight view cup segmentation method based on kinks.

[0098] Figure 2 This is a schematic diagram of the local information calculation module of the present invention.

[0099] Figure 3This is a schematic diagram of the global information sparse computing module of the present invention. Detailed Implementation

[0100] To better illustrate the purpose and advantages of the present invention, the invention will be further described below in conjunction with the accompanying drawings and examples.

[0101] Example 1:

[0102] like Figure 1 As shown, the present invention discloses a lightweight view cup segmentation method based on kinks, comprising the following steps:

[0103] Step 1: Preprocess the fundus retinal images for the training and test sets. This preprocessing includes cropping the optic disc region from the fundus retinal images and standardizing the resolution of the cropped images to obtain standardized training and test sets. These sets are used for training and testing the optic cup segmentation method, respectively. Both the standardized training and test sets contain fundus retinal images X and consistent optic cup segmentation labels Y.

[0104] Step 1.1: Use an optic disc detector to detect the optic disc in the fundus retinal image, and then crop the fundus retinal image containing the optic disc region with the center of the optic disc as the cropping center. The cropped fundus retinal image is a square with a side length of 1.5 times the diameter of the optic disc.

[0105] In step 1.1, the optic disc detector uses either a standard U-Net segmentation network or an optic disc center detection method based on a standard YOLO detection network to initially locate the position of the optic disc in the fundus retinal image. Based on the initially located position, a square area of ​​the fundus retinal image is cropped with the initially located position of the optic disc as the center.

[0106] The optic disc diameter obtained in step 1.1 is as follows: calculate the length of the optic disc in the horizontal and vertical directions in the fundus retinal image, and select the longest value in the horizontal and vertical directions as the diameter of the optic disc.

[0107] Step 1.2: Compress the resolution of the cropped image to a predetermined resolution (256, 256) to obtain a resolution-standardized image, thus achieving resolution standardization.

[0108] Step Two: As Figure 1As shown in the preprocessing module focusing on vascular tortuosities, a lightweight optic cup segmentation network is trained based on this preprocessing module. This preprocessing module performs local blurring of the optic cup boundary on the standardized fundus retinal images from step one. This blurring reduces the chromatic aberration of the optic cup boundary in the fundus retinal images, keeping only the vascular tortuosity region near the optic cup boundary clear. Since the vascular tortuosity and the chromatic aberration of the optic cup boundary are the two most important factors determining the optic cup boundary, and deep learning methods are based on automatic feature extraction, they typically use the simpler chromatic aberration of the optic cup boundary as the feature for extraction, thus causing the vascular tortuosity to be ignored. To enable the lightweight optic cup segmentation network of this invention to focus on vascular tortuosities, the preprocessing module for focusing on vascular tortuosities forcibly removes chromatic aberration information from the optic cup boundary. During the training of the lightweight optic cup segmentation network, the network is trained to extract optic cup information solely by mining information from vascular tortuosities. This allows the lightweight optic cup segmentation network to fully utilize not only simple chromatic aberrations but also the information from vascular tortuosities.

[0109] Step two focuses on training a lightweight optic cup segmentation network using a preprocessing module for vascular knots, specifically implemented as follows:

[0110] Step 2.1: Determine the extent of the blurred smooth area in the fundus retinal image. Extend the coverage to both sides of the boundary line marked Y by the optic disc in the training set as the midline. The width of the extension is equal to the diameter of the optic disc described in Step 1. The obtained extended coverage area is an approximately circular band, and the process of extending the coverage is denoted as band(·). This approximately circular band is used as the initial blur smoothing range mask1. The calculation process of obtaining the blur smoothing range mask1 in the fundus retinal image is denoted as formula (1).

[0111] mask1 = band(Y) ⑴

[0112] Step 2.2: Narrow the initial blur smoothing range described in Step 2.1. Calculate the intersecting region of the vessel (Vessel) in the fundus photograph with the initial blur smoothing range. Exclude the vessel intersecting region from the initial blur smoothing range to obtain the blur smoothing range mask2. The calculation process of the blur smoothing range mask2 is denoted by formula (2).

[0113] mask2=mask1∩(mask1∩Vessel) c (2)

[0114] Step 2.3: Perform blur smoothing on the fundus image X according to the blur smoothing range to blur the chromatic aberration at the optic cup boundary, obtaining a blurred smoothed image. The blur smoothing process is denoted as blur(·). The chromatic aberration information at the optic cup boundary is removed from the blurred smoothed image, which only contains information about the vascular knot points related to the optic cup boundary. Therefore, the obtained blurred smoothed image is denoted as the vascular knot point fundus retinal image X. kink The blur smoothing method used is Gaussian blur. The calculation process for blur smoothing the fundus X-ray based on the blur smoothing range mask2 is denoted as formula (3).

[0115] X kink =blur(X|mask2) (3)

[0116] Step 2.4: The output of the preprocessing module focusing on vascular kinetic points is the fundus retinal image X of the vascular kinetic points from Step 2.3. kink The input X to the preprocessing module for focusing on vascular kinks is also used as the output of the preprocessing module for focusing on vascular kinks.

[0117] Step 3: As Figure 1As shown, a lightweight view cup segmentation network is constructed for segmenting view cup features. This lightweight view cup segmentation network consists of an encoder E and a decoder D. The encoder E comprises two local information computation modules (Local-FFN) and two global information sparse computation modules (Global-FFN), denoted as L-FFN and G-FFN respectively. The two local information computation modules (L-FFN) of the encoder E have the same structure but different feature sizes; the feature size of the first L-FFN is twice that of the second L-FFN. Similarly, the two global information sparse computation modules (G-FFN) of the encoder E have the same structure but different feature sizes; the feature size of the first G-FFN is twice that of the second G-FFN. The decoder D contains the same number of global information sparse computation modules (G-FFN) and local information computation modules (L-FFN) as the encoder E, meaning the decoder D consists of two global information sparse computation modules and two local information computation modules. The two global information sparse computation modules (G-FFN) of the decoder D have the same structure but different feature sizes; the feature size of the latter G-FFN is twice that of the former. Similarly, the two local information computation modules (L-FFN) of the decoder D have the same structure but different feature sizes; the feature size of the latter L-FFN is twice that of the former.

[0118] The encoder E of the lightweight vision cup segmentation network described in step three is as follows: Figure 1 As shown, the fundus retinal image X of the vascular kink point in step 2.4 is used. kink Taking the fundus retinal image X as input, and processing X... kink Feature extraction is performed on X to obtain the encoded image features. The encoder E contains two local information calculation modules L-FFN and two global information sparse calculation modules G-FFN. These calculation modules are denoted as encoder local calculation modules. Encoder Local Calculation Module Encoder Global Information Sparse Computation Module and encoder global information sparse computation module in and Each of these has a dimensionality reduction module, Stack, which are denoted as follows: and The dimensionality reduction module is used to reduce the height and width of features and increase the number of channels in features. The dimensionality reduction module can be seen as having a 2×2 computational kernel in the height and width dimensions, with a step size of 2, adjusting the information of the 2×2 range to the channel dimension, thus reducing the height and width by a factor of 2 and increasing the channel dimension by a factor of 4.

[0119] Dimensionality reduction module Using the fundus retinal image X∈R from step two 256×256×3 or X kink Let X be the input. Taking X as an example, we perform dimensionality reduction on X to obtain the dimensionality-reduced features. Features after dimensionality reduction The scale is R 128×128×12 Encoder Local Calculation Module With the dimensionality-reduced features Features of input and output encoders Its feature dimension is R 128×128×12 H represents height, W represents width, and C represents the number of channels; dimensionality reduction module The calculation process and encoder local calculation module The calculation process is denoted as formulas (4) and (5) respectively.

[0120]

[0121]

[0122] Dimensionality reduction module Based on features As input, for features Perform dimensionality reduction to obtain the reduced features. Features after dimensionality reduction The scale is R 64×64×48 Encoder Local Calculation Module Based on encoder features Features of input and output encoders Its feature dimension is R 64×64×48 Dimensionality reduction module The calculation process and encoder local calculation module The calculation process is denoted as formulas (6) and (7) respectively.

[0123]

[0124]

[0125] Dimensionality reduction module Based on features As input, for features Perform dimensionality reduction to obtain the reduced features. Features after dimensionality reduction The scale is R 32×32×192 Encoder global information sparse computation module Based on encoder features Features of input and output encoders Its feature dimension is R 32×32×192 Dimensionality reduction module The calculation process and encoder global information sparse calculation module The calculation process is denoted as formulas (8) and (9).

[0126]

[0127]

[0128] Dimensionality reduction module Based on features As input, for features Perform dimensionality reduction to obtain the reduced features. Features after dimensionality reduction The scale is R 16×16×768 Encoder global information sparse computation module Based on encoder features Features of input and output encoders Its feature dimension is R 16×16×768 Dimensionality reduction module The calculation process and encoder global information sparse calculation module The calculation process is denoted as formulas 10 and 11, respectively.

[0129]

[0130]

[0131] The decoder D of the lightweight view cup segmentation network described in step three uses encoder features. As input, for features Decoding yields the decoded image cup features. The decoder D comprises two global information sparse computation modules G-FFN and two local information computation modules L-FFN. These computation modules are denoted as follows: Global Information Sparse Computation Module (G-FFN) Decoder Global Information Sparse Computation Module Decoder Local Computation Module and decoder local computation module in, and Each of these is followed by an upscaling module, Unstack, which increases the height and width scales, and is denoted as follows: and The dimensionality-upgrading module is used to increase the height and width of the feature while reducing the number of channels. The module transfers channel information to the height and width dimensions; that is, information transfer is achieved through deformation, resulting in a doubling of the feature's height and width, while reducing the amount of information in the channel dimension by a factor of four.

[0132] Decoder Global Information Sparse Computation Module Based on encoder features Features of input and output decoders Its feature dimension is R 16×16×768 Upgrade Module Based on features As input, for features Perform dimensionality increase to obtain the features after dimensionality increase. Features after dimensional upgrade The scale is R 32×32×192 Decoder global information sparse computation module Calculation process and dimension-upgrading module The calculation process is denoted as formulas 12 and 13.

[0133]

[0134]

[0135] Decoder Global Information Sparse Computation Module Based on encoder features Features of input and output decoders Its feature dimension is R 32×32×192 Upgrade Module Based on features As input, for features Perform dimensionality increase to obtain the features after dimensionality increase. Features after dimensional upgrade The scale is R 64×64×48 Decoder global information sparse computation module The calculation process and the dimension-upgrading module The calculation process is denoted as formulas ⒁ and ⒂.

[0136]

[0137]

[0138] Decoder Local Computation Module Based on encoder features Features of input and output decoders Its feature dimension is R 64×64×48 Upgrade Module Based on features As input, for features Perform dimensionality increase to obtain the features after dimensionality increase. Features after dimensional upgrade The scale is R 128×128×12 Decoder Local Computation Module and Upgrade Module The calculation process is denoted as formulas ⒃ and ⒄.

[0139]

[0140]

[0141] Decoder Local Computation Module Based on encoder features Features of input and output decoders Its feature dimension is R 128×128×12 Upgrade Module Based on features As input, for features Perform dimensionality increase to obtain the features after dimensionality increase. Features after dimensional upgrade The scale is R 256×256×3 Decoder Local Computation Module The calculation process and the dimension-upgrading module The calculation process is denoted as formulas ⒅ and ⒆.

[0142]

[0143]

[0144] Step Four: As Figure 2 As shown, the structure of the Local Information Calculation Module (L-FFN) is defined. This module consists of a local data sampling layer and an FFN calculation layer. The local data sampling layer defines the local range of data sampling, allowing the L-FFN to focus solely on the relationships between local data points without concern for relationships between distant information. Because it focuses only on the relationships between local data points, the L-FFN is better at modeling relationships between local feature information. Furthermore, the FFN calculation layer is a spatially and channel-separated FFN calculation module. Separating spatial and channel calculations effectively reduces the computational load and the number of parameters, mitigating the problem of large parameter counts inherent in fully connected layers.

[0145] The local information calculation module L-FFN described in step four consists of a local data sampling layer and an FFN calculation layer. The specific implementation method includes steps 4.1 to 4.3:

[0146] Step 4.1: Define the input of the Local Information Calculation Module (L-FFN) as features. Where H iW i and C i These represent the height, width, and number of channels of the input feature, respectively. Assume H... i =128, W i =128 and C i =12, meaning the dimension of the input features is f. input1 ∈R 128×128×12 .

[0147] Step 4.2: As Figure 2 As shown, a local data sampling layer is constructed to process the features f input to the local information calculation module. input1 First, local blocks are divided, which is used to determine the scope of local information.

[0148] The local block division described in step 4.2 is as follows: the input feature f input1 According to P local1 ×P local1 The local blocks of different sizes are divided into non-overlapping sections according to their height and width, resulting in non-overlapping feature blocks. Its dimensions are When f input1 The dimension is R 256×256×3 or R 128×128×12 At that time, P local1 The optimal value is 8, i.e., P. local1 ×P local1 =8×8; Assume f input1 ∈R 128×128×12 According to preferred P local1 =8, Its dimensions are Then features Further transform the dimensions into feature blocks Its dimensions are The local data sampling layer is represented by formula ⒇ as follows.

[0149]

[0150] Step 4.3: As Figure 2 As shown, an FFN computation layer is constructed, which uses features... The input is used for feature extraction. The FFN computational layer consists of two sets of computational sublayers, denoted as FFN1 and FFN2. FFN1 is used to compute spatial information relationships, while FFN2 is used to model inter-channel information relationships. Both FFN1 and FFN2 include two fully connected layers, each followed by an activation function. The FFN computational layer has low computational cost and few parameters because it separates the computation of spatial information and the computation between channels, which significantly reduces the computational cost and the number of parameters.

[0151] In step 4.3, the first group of calculation sub-layers FFN1 pairs of features The calculation method is as follows:

[0152] The first computational sublayer FFN1 consists of two fully connected layers, with 8 neurons in each of the two consecutive fully connected layers. The input to the fully connected layer is... Depend on It consists of vectors, each vector having a dimension of R. (8·8) . Each vector is passed through the two fully connected layers, and the vector generated by the two fully connected layers is... The calculation process of the first sublayer FFN1 of L-FFN is denoted by formula (21).

[0153]

[0154] The second computational sublayer FFN2 outputs features based on the first computational sublayer FFN1. Feature extraction is performed on the input. The second computational sublayer FFN2 consists of two fully connected layers, with C neurons in the two consecutive fully connected layers. i The second group calculates sublayer FFN2 with... For input, Transform dimensionality into feature vectors Depend on It consists of 12 vectors, each with a dimension of 12. The input is fed into two fully connected layers, which process the features. Perform feature extraction and output feature f local1 ∈R (128·128)×12 Furthermore, the shape of the feature is deformed, f local1 The dimension of the feature becomes R 128×128×12 The calculation process of the second sublayer L-FFN2 of L-FFN is denoted by formula (22).

[0155]

[0156] Step 5: As Figure 3 As shown, the structure of the Global Information Sparse Computation Module (G-FFN) is defined. The G-FFN consists of a global data sparse sampling layer and an FFN computation layer. The global data sparse sampling layer samples from the features of the entire image, covering the information of the entire fundus retina image. The sampling points are sampled at regular intervals throughout the entire image. The global data sparse sampling layer achieves sampling of global information and reduces the number of sampling points through sparse sampling. Therefore, the global data sparse sampling layer enables the G-FFN to model the relationships between global information features and reduces the cost of the G-FFN.

[0157] Step 5.1: Define the input of the Global Information Sparse Computation Module G-FFN as features Where H j W j and C j These represent the feature's height, width, and number of channels, respectively. Assume H... j =64, W j =64 and C j =48, i.e., f input2 The dimension is R 64×64×48 .

[0158] Step 5.2: As Figure 3 As shown, a global data sparse sampling layer is constructed to process the features f input to the global information sparse computation module. input2 The data is divided into blocks, which are used for global sparse sampling. Preferably, the global sparse sampling method is interval sampling.

[0159] When feature f input2 The dimension is R 64×64×48 or R 32×32×192 At that time, P g The preferred size is 4, i.e., P g ×P g = 4 × 4. The input features f input2 According to P g ×P g A local block of size (4×4) is divided into non-overlapping feature blocks according to the height and width directions. Its dimensions are Right now Then the feature block Further transform the dimension into feature blocks Its dimensions are Equivalent to having Each dimension is R (4·4)×48 Feature blocks In P g ·P g Select dimension i from the dimensions in which it is located. The dimension is R 1×48 eigenvectors, indivual Feature vectors are aggregated into new feature blocks. Its dimensions are Similarly, index i is traversed. The first dimension has 4.4 Obtain global feature blocks Its dimensions are Merging global feature blocks The second and third dimensions are transformed into features. Its dimensions are The global sparse sampling layer is represented by formula (23).

[0160]

[0161] Step 5.3: As Figure 3 As shown, an FFN computation layer is constructed, which uses features... The input is used for feature extraction. The FFN computation layer consists of two sets of FFN computation sub-layers, denoted as the first set of computation sub-layers FFN1 and the second set of computation sub-layers FFN2. The first set of computation sub-layers FFN1 is used to compute the relationship between spatial information, and the second set of computation sub-layers FFN2 is used to model the relationship between information between channels. Both the first set of computation sub-layers FFN1 and the second set of computation sub-layers FFN2 include two fully connected layers, each followed by an activation function.

[0162] In step 5.3, the first group of calculation sub-layers FFN1 is used to calculate the features. The calculation method is as follows:

[0163] The first computational sublayer FFN1 consists of two fully connected layers, and the number of neurons in the two consecutive fully connected layers is... The input to the fully connected layer is This is equivalent to 48.4.4 vectors, each with a dimension of . 48.4.4 vectors are passed through the two fully connected layers respectively, and the vectors generated by the two fully connected layers are... The calculation process of the first calculation sublayer G-FFN1 of G-FFN is denoted by formula (24).

[0164]

[0165] The second computational sublayer FFN2 outputs features based on the first computational sublayer FFN1. Feature extraction is performed on the input. The second computational sublayer FFN2 consists of two fully connected layers, with 48 neurons in each of the two consecutive fully connected layers. The second computational sublayer FFN2 uses... For input, Transform dimensions into feature vectors Equivalent to There are 3 vectors, each with a dimension of 48. Then... The input is fed into two fully connected layers, which process the features. Perform feature extraction and output feature f global1 ∈R (64·64)×48 Furthermore, the shape of the feature is deformed, f global1 The dimension of the feature becomes R 64×64×48 The calculation process of the second sublayer G-FFN2 of G-FFN is denoted by formula (25).

[0166]

[0167] Step Six: Based on the input dimensions of different features in Step Three, the local information computation module structure constructed in Step Four is sequentially embedded into the encoder and decoder in Step Three; and the global information sparse computation module structure constructed in Step Five is sequentially embedded into the encoder and decoder in Step Three. Based on the encoder and decoder with embedded local information computation module and global information sparse computation module structures, a view cup segmentation network with embedded local information computation module and global information sparse computation module is constructed, resulting in a structurally complete lightweight view cup segmentation network. This lightweight view cup segmentation network not only has the advantages of small parameter count and low computational cost, but also the ability to model both local and global features. The small parameter count and low computational cost of the lightweight segmentation network are due to the FFN computation layer calculating spatial and channel features independently, and the limitation on the number of sampling points by the local data sampling layer of the local information computation module and the global data sparse sampling layer of the global information sparse computation module. Compared to the convolution operators in convolutional neural networks, the global information sparse computation module has the ability to model global information because it performs sparse sampling of global information and then feeds the sampled global information into the FFN computation layer to mine the relationships between global features.

[0168] Substitute formulas (20)-(22) into formula (5) as steps to be executed sequentially, and similarly substitute them into formulas (7), (16), and (18).

[0169] Substitute formulas (23)-(25) into formula (9) as steps to be executed sequentially, and similarly substitute them into formulas (11), (12), and (14).

[0170] Step 7: Construct the total loss function L Total The encoder E and decoder D are used to train the lightweight view cup segmentation network. The network parameters obtained after training on the training set in step one are used as the final parameters of the lightweight view cup segmentation network. For example... Figure 1 As shown, the loss function of this invention includes three parts, namely the knot constraint loss L proposed in this invention. kink Combating learning loss L adv And Dice lost L Dice Among them, the torsional node constraint loss L kink This refers to the fundus image X generated in step two after eliminating the optic cup boundary. kink The deep features generated by encoder E are consistent with those of fundus image X (without removing the optic cup boundary). By ensuring consistency of these deep features, encoder E can learn deep features of the optic cup information from fundus images with chromatic aberration information removed from the optic cup boundary. These deep features can be obtained solely from vascular knots in fundus images with chromatic aberration information removed from the optic cup boundary. This is achieved through knot-constrained loss L. kink The constraints enable lightweight optic cup segmentation networks to better utilize prior knowledge to assist in optic cup segmentation, namely, vascular knots to assist in optic cup boundary segmentation.

[0171] The knot constraint loss L mentioned in step seven kink The following calculation is performed: the fundus image X generated in step two after eliminating the optic cup boundary. kink The fundus image X with and without the optic cup boundary removed are input into encoder E, and features are generated according to formulas (4) to (11). Will be X kink Features generated for the input encoder E Recorded as The features generated by encoder E with X as input It is denoted as F.

[0172] Feature F and feature The comparison is performed point-by-point along both length and width dimensions, using a combination of Euclidean distance and cosine similarity (cosine distance). Features F and features... The distance loss function is calculated according to formula (26).

[0173]

[0174] in, F represents the value of the feature at position i in the high dimension, position j in the wide dimension, and position c in the channel dimension.ij This represents the vector whose feature is located at position i in the height of dimension i and position j in the width of dimension j. Similarly... and They have similar meanings.

[0175] Step 7: Adversarial learning loss L adv The goal is to improve the cup segmentation results predicted by the lightweight cup segmentation network. In terms of the overall part, it resembles the real view cup segmentation result Y more closely because adversarial learning focuses on global information and can therefore grasp the overall distribution of the segmentation result. Compared with the pixel-by-pixel comparison Dice loss, the adversarial learning loss L adv It can eliminate the predicted cup segmentation results Discontinuous phenomena. Adversarial learning loss L adv The specific calculation method is as follows:

[0176] Construct a binary classification network Dis to analyze the predicted view cup segmentation results. The predicted view cup segmentation result is classified with the actual view cup segmentation result Y, thus enabling the classification of the predicted view cup segmentation result. The image is classified into class 0, representing a predicted segmentation result. Simultaneously, the binary classification network classifies the actual view cup segmentation result Y into class 1, representing a true view cup segmentation label. The lightweight view cup segmentation network aims to predict the view cup segmentation result. The result is very similar to the actual view cup segmentation Y, making it difficult for the binary classification network to accurately classify the view cup. Classify Y and Y. Adversarial learning loss L adv Calculate according to formula (27).

[0177]

[0178] in denoted by , and Log represents the expected value.

[0179] In step seven, Dice loss L Dice The goal is to compare the predicted cup segmentation results pixel by pixel. The difference between the predicted and actual cup segmentation results Y is used to optimize the predicted cup segmentation results. Dice loss is calculated according to formula (28).

[0180]

[0181] Where Y ij This represents the value of Y at position i in height dimension and j in width dimension; express The values ​​at positions i in the height dimension and j in the width dimension.

[0182] In step seven, the total loss function is the loss L. kink Loss L adv and loss L Dice The weighted sum of the values. The final total loss is calculated according to formula (29).

[0183] L Total =L kink +L adv +L Dice (29)

[0184] Step 8: Use the lightweight optic cup segmentation network trained in Step 7 for testing until a well-trained lightweight optic cup segmentation network is obtained. Using the fundus retinal image X from the test set in Step 1 as input, use the trained lightweight optic cup segmentation network to predict the segmentation result of the optic cup. This means that lightweight visual cup segmentation is achieved based on kink points; during the testing process, the preprocessing module focusing on vascular kink points in step two is not required.

[0185] The above detailed description further illustrates the purpose, technical solution, and beneficial effects of the invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A lightweight view cup segmentation method based on knots, characterized in that: Includes the following steps, Step 1: Preprocess the fundus retinal images of the training and test sets; the preprocessing includes cropping the optic disc region from the fundus retinal images and standardizing the resolution of the cropped fundus retinal images to obtain standardized training and test sets, which are used for training and testing the cup segmentation method, respectively; wherein both the standardized training and test sets contain fundus retinal images. Consistent view cup segmentation annotation ; Step 2: Construct a preprocessing module focusing on vascular tortuosities, and train a lightweight optic cup segmentation network based on this preprocessing module. The preprocessing module focusing on vascular tortuosities performs local blurring of the optic cup boundary on the fundus retinal images in the standardized training set from Step 1. This blurring process reduces the chromatic aberration at the optic cup boundary in the fundus retinal images, keeping only the vascular tortuosity region near the optic cup boundary clear. By forcibly removing the chromatic aberration information at the optic cup boundary, the preprocessing module focuses on vascular tortuosities. During the training of the lightweight optic cup segmentation network, this allows the network to extract optic cup information solely by mining the information from vascular tortuosities, enabling it to fully utilize both simple chromatic aberrations and the information from vascular tortuosities. Step 3: Construct a lightweight view cup segmentation network for segmenting view cup features. This lightweight view cup segmentation network consists of an encoder. and decoder Composition; the encoder The encoder consists of two local information calculation modules (Local-FFN) and two global information sparse calculation modules (Global-FFN), denoted as L-FFN and G-FFN respectively. The encoder consists of two local information calculation modules (L-FFN) and two global information sparse calculation modules (G-FFN); The two local information calculation modules L-FFN have the same structure but different feature sizes; the feature size of the first local information calculation module L-FFN is twice that of the second local information calculation module L-FFN. The encoder The two global information sparse computation modules G-FFN have the same structure but different feature sizes. The feature size of the first global information sparse computation module G-FFN is twice that of the second global information sparse computation module G-FFN; the decoder The number of global information sparse computation modules (G-FFN) and local information computation modules (L-FFN) in the encoder is related to the number of encoder modules. Same, that is, the decoder It consists of two global information sparse computation modules and two local information computation modules; the decoder The two global information sparse computation modules G-FFN have the same structure but different feature sizes; the feature size of the latter G-FFN is twice that of the former. The decoder... The two local information calculation modules L-FFN have the same structure but different feature sizes. The feature size of the latter local information calculation module L-FFN is twice that of the former local information calculation module L-FFN. Step 4: Define the structure of the Local Information Calculation Module (L-FFN); the Local Information Calculation Module consists of a local data sampling layer and an FFN calculation layer; the local data sampling layer defines the local range of data sampling, so that the Local Information Calculation Module only focuses on the relationships between local data and not on the relationships between long-distance information; because it only focuses on the relationships between local data, the Local Information Calculation Module (L-FFN) is better at modeling the relationships between local feature information; in addition, the FFN calculation layer is a spatial and channel-separated FFN calculation module, which can effectively reduce the amount of computation and the number of parameters. Step 5: Define the structure of the Global Information Sparse Computation Module G-FFN. The Global Information Sparse Computation Module G-FFN consists of a global data sparse sampling layer and an FFN computation layer. The global data sparse sampling layer samples from the features of the entire image, and the sampling range covers the information of the entire fundus retinal image. The sampling points are sampled at regular intervals in the entire image. The global data sparse sampling layer enables the sampling of global information and also reduces the number of sampling points through sparse sampling. Therefore, the global data sparse sampling layer enables the global information sparse computation module G-FFN to model the relationship between global information features and reduces the cost of the global information sparse computation module G-FFN. Step Six: Based on the input sizes of different features in Step Three, the local information computation module structure constructed in Step Four is sequentially embedded into the encoder and decoder in Step Three; and the global information sparse computation module structure constructed in Step Five is sequentially embedded into the encoder and decoder in Step Three; based on the encoder and decoder with embedded local information computation module structure and global information sparse computation module structure, a view cup segmentation network with embedded local information computation module and global information sparse computation module is constructed to obtain a structurally complete lightweight view cup segmentation network; the global information sparse computation module performs sparse sampling of global information, and then sends the sampled global information into the FFN computation layer to mine the relationship between global features; Step 7: Construct the total loss function Encoder for training a lightweight sight cup segmentation network and decoder The network parameters obtained after training on the training set in step one are used as the parameters of the final lightweight view cup segmentation network; the loss function consists of three parts, namely the knot constraint loss. Combating learning loss and Dice loss Among them, the constraint loss of twisted nodes This refers to the fundus image generated in step two after eliminating the optic cup boundary. Fundus images without removal of the optic cup margin Simultaneous transmission through encoder The resulting deep features are consistent, and this consistency is achieved by making the encoder consistent. Deep features of the optic cup information can be learned from fundus images with chromatic aberration information removed from the optic cup edge. These deep features can be obtained solely from vascular kinks in fundus images with chromatic aberration information removed from the optic cup edge; loss is constrained by kinks. The constraints enable lightweight optic cup segmentation networks to better utilize prior knowledge to assist in optic cup segmentation, namely, vascular knots to assist in optic cup boundary segmentation. Step 8: Use the lightweight optic cup segmentation network trained in Step 7 for testing until a well-trained lightweight optic cup segmentation network is obtained, using the fundus retinal images from the test set in Step 1. Using the pre-trained lightweight view cup segmentation network as input, the segmentation result of the view cup is predicted. .

2. The lightweight view cup segmentation method based on knots as described in claim 1, characterized in that: The implementation method for step one is as follows: Step 1.1: Use an optic disc detector to detect the optic disc in the fundus retinal image. Then, crop the fundus retinal image containing the optic disc region using the center of the optic disc as the cropping center. The cropped fundus retinal image is square, and the side length of the square is equal to the diameter of the optic disc. times; In step 1.1, the optic disc detector uses either a U-Net segmentation network or a YOLO-based optic disc center detection method to initially locate the position of the optic disc in the fundus retinal image. Based on the initially located position, a square area of ​​the fundus retinal image is cropped with the initially located position of the optic disc as the center. The optic disc diameter obtained in step 1.1 is as follows: calculate the length of the optic disc in the horizontal and vertical directions in the fundus retinal image, and select the longest value in the horizontal and vertical directions as the diameter of the optic disc; Step 1.2: Compress the resolution of the cropped image to the predetermined resolution. This process obtains an image with normalized resolution, thus achieving resolution normalization. The range is , The range is and .

3. The lightweight view cup segmentation method based on knots as described in claim 2, characterized in that: Step two focuses on training a lightweight optic cup segmentation network using a preprocessing module for vascular knots. The specific implementation steps are as follows. Step 2.1: Determine the extent of blurred smoothness in the fundus retinal image; use training focus cup annotation. The boundary line extends from the center line to both sides, covering the area, with the width of the extension being equal to the diameter of the viewing disc described in step one. The resulting extended coverage area is an approximately circular strip, and the process of extending coverage is denoted as... ; This approximately circular strip serves as the initial range for fuzzy smoothing. ; Obtain the range of blur smoothing in fundus retinal images The calculation process is denoted as the formula. ; Step 2.2: Narrow the scope of the initial blur smoothing described in Step 2.1; based on the blood vessels in the retinal fundus photograph. The portion intersecting with the initial blurred and smoothed range is calculated as the blood vessel intersection region; The regions where blood vessels intersect are excluded from the initial fuzzy smoothing range, resulting in the fuzzy smoothing range. ; Range of fuzzy smoothing The calculation process is denoted as the formula. ; Step 2.3: Take a retinal fundus photograph based on the range of blurring and smoothing. Blur smoothing is performed to blur the chromatic aberration at the viewfinder boundary, resulting in a smoothed image; the blur smoothing process is denoted as... In the blurred and smoothed image, the chromatic aberration information of the optic cup boundary is removed. The blurred and smoothed image only contains information about the vascular kink points related to the optic cup boundary. Therefore, the resulting blurred and smoothed image is denoted as the vascular kink point fundus retinal image. Among them, the fuzzy smoothing method is Gaussian blur; according to the range of fuzzy smoothing Retinal fundus photography The calculation process for fuzzy smoothing is denoted by the formula. ; Step 2.4: The output of the preprocessing module focusing on vascular kinetic points is the fundus retinal image of the vascular kinetic points from Step 2.

3. and the input to the preprocessing module that focuses on vascular kinks. It is also used as the output of the preprocessing module that focuses on vascular kinks.

4. The lightweight view cup segmentation method based on knots as described in claim 3, characterized in that: In step three, The encoder for constructing the lightweight vision cup segmentation network described in step three The fundus retinal image of the vascular kink point in step 2.4 and fundus retinal images For input, and Feature extraction is performed to obtain the encoded image features; encoder It includes two local information calculation modules (L-FFN) and two global information sparse calculation modules (G-FFN); the calculation modules are denoted as encoder local calculation modules in sequence. Encoder Local Calculation Module Encoder global information sparse computation module Encoder global information sparse computation module ;in , , and Each of these has a dimensionality reduction module, Stack, which are denoted as follows: , , and The dimensionality reduction module is used to reduce the height and width of features and increase the number of channels in the features. Dimensionality reduction modules can be viewed as having [dimensionality] in the dimensions of height and width. The kernel for calculating the size, with a step size of 2, will The range information is adjusted to the channel dimension, thus reducing the height and width by a factor of 2, while increasing the channel dimension by a factor of 4. Dimensionality reduction module The fundus retinal image from step two or As input; taking X as an example, perform dimensionality reduction on X to obtain the dimensionality-reduced features. Features after dimensionality reduction Scale is Encoder Local Calculation Module With the dimensionality-reduced features Features of input and output encoders Its feature dimension is , W represents height, and W represents width and height. Indicates the number of channels; dimensionality reduction module The calculation process and encoder local calculation module The calculation process is denoted as the formula. and ; Dimensionality reduction module Based on features As input, for features Perform dimensionality reduction to obtain the reduced features. Features after dimensionality reduction Scale is Encoder Local Calculation Module Based on encoder features Features of input and output encoders Its feature dimension is Dimensionality reduction module The calculation process and encoder local calculation module The calculation process is denoted as the formula. and ; Dimensionality reduction module Based on features As input, for features Perform dimensionality reduction to obtain the reduced features. Features after dimensionality reduction Scale is Encoder global information sparse computation module Based on encoder features Features of input and output encoders Its feature dimension is Dimensionality reduction module The calculation process and encoder global information sparse calculation module The calculation process is denoted as the formula. and ; Dimensionality reduction module Based on features As input, for features Perform dimensionality reduction to obtain the reduced features. Features after dimensionality reduction Scale is ; Encoder Global Information Sparse Computation Module Based on encoder features Features of input and output encoders Its feature dimension is Dimensionality reduction module The calculation process and encoder global information sparse calculation module The calculation process is denoted as the formula. and ; The decoder for the lightweight visual cup segmentation network described in step three encoder features As input, for features Decode to obtain the decoded visual cup features; decoder It includes two global information sparse computation modules (G-FFN) and two local information computation modules (L-FFN); the computation modules are denoted as follows: decoder global information sparse computation module. Decoder global information sparse computation module Decoder Local Computation Module and decoder local computation module ;in, , , and Each of these is followed by an upscaling module, Unstack, which increases the height and width scales, denoted as follows: , , and The dimensionality-upgrading module is used to increase the height and width of the feature and reduce the number of channels of the feature. The dimensionality-upgrading module transfers the information of the channels to the dimensions of height and width. That is, the information transfer can be achieved through deformation, so that the feature increases in height and width by 2 times, while the amount of information in the channel dimension decreases by 4 times. Decoder Global Information Sparse Computation Module Based on encoder features Features of the input and output decoders Its feature dimension is Upgrade Module Based on features As input, for features Perform dimensionality increase to obtain the features after dimensionality increase. Features after dimensional upgrade Scale is Decoder global information sparse computation module Calculation process and dimension-upgrading module The calculation process is denoted as a formula. and ; Decoder Global Information Sparse Computation Module Based on encoder features Features of the input and output decoders Its feature dimension is Upgrade Module Based on features As input, for features Perform dimensionality increase to obtain the features after dimensionality increase. Features after dimensional upgrade Scale is Decoder global information sparse computation module The calculation process and the dimension-upgrading module The calculation process is denoted as the formula. and ; Decoder Local Computation Module Based on encoder features Features of the input and output decoders Its feature dimension is Upgrade Module Based on features As input, for features Perform dimensionality increase to obtain the features after dimensionality increase. Features after dimensional upgrade Scale is Decoder Local Computation Module and Upgrade Module The calculation process is denoted as the formula. and ; Decoder Local Computation Module Based on encoder features Features of the input and output decoders Its feature dimension is Upgrade Module Based on features As input, for features Perform dimensionality increase to obtain the features after dimensionality increase. Features after dimensional upgrade Scale is Decoder Local Computation Module The calculation process and the dimension-upgrading module The calculation process is denoted as the formula. and ; 。 5. The lightweight view cup segmentation method based on knots as described in claim 4, characterized in that: The local information calculation module L-FFN mentioned in step four consists of a local data sampling layer and an FFN calculation layer. The specific implementation method of step four includes steps 4.1 to 4.3: Step 4.1: Define the input of the Local Information Calculation Module (L-FFN) as features. ,in , and These represent the height, width, and number of channels of the input feature, respectively. Step 4.2: Construct a local data sampling layer to process the features input to the local information calculation module. First, local blocks are divided, which is used to determine the scope of local information; The local block division described in step 4.2 is as follows: dividing the input features... according to The local blocks of different sizes are divided into non-overlapping sections according to their height and width, resulting in non-overlapping feature blocks. Its dimensions are ; Then features Further transform the dimensions into feature blocks Its dimensions are The local data sampling layer is described by the formula. The following is an example; Step 4.3: Construct the FFN computation layer. The FFN computation layer uses features... For input, feature extraction is performed; the FFN computation layer consists of two sets of computational sublayers, denoted as the first set of computational sublayers. Second group of computational sublayers First group of computational sublayers The second set of computational sublayers is used to calculate spatial information relationships. Used to model the relationships between information in channels; First set of computational sublayers Second group of computational sublayers Each includes two fully connected layers, with each fully connected layer followed by an activation function; In step 4.3, the first group of computation sub-layers Features The calculation method is as follows: First group of computational sublayers It consists of two fully connected layers, and the number of neurons in the two consecutive fully connected layers is... The input to the fully connected layer is , Depend on It consists of vectors, each vector having a dimension of . Each vector is passed through the two fully connected layers, and the vector generated by the two fully connected layers is... The first set of computational sublayers of L-FFN The calculation process is denoted as formula (21); (21) The second group of computational sublayers The first group of computational sub-layers Output features Feature extraction is performed on the input; the second set of computational sublayers It consists of two fully connected layers, and the number of neurons in the two consecutive fully connected layers is... The second group of computational sublayers by For input, Transform dimensionality into feature vectors , Depend on It consists of vectors, each vector having a dimension of . The input is fed into two fully connected layers, which process the features. Perform feature extraction and output features Furthermore, the shape of the features is deformed. The dimension of the feature becomes The second set of computational sublayers of L-FFN The calculation process is denoted as formula (22); (22)。 6. The lightweight view cup segmentation method based on knots as described in claim 5, characterized in that: Step five is implemented as follows: Step 5.1: Define the input of the Global Information Sparse Computation Module G-FFN as features ,in , and These are the feature's height, width, and number of channels, respectively. Step 5.2: Construct a global data sparse sampling layer to process the features input to the global information sparse computation module. The data is divided into blocks, which are used for global sparse sampling of the data. The global sparse sampling method is interval sampling. Input features according to The local blocks of different sizes are divided into non-overlapping feature blocks according to their height and width directions. Its dimensions are Then for the feature block Further transform the dimension into feature blocks Its dimensions are , equivalent to having Each dimension is Feature blocks ;exist Select dimension in the given dimension , The dimension is eigenvectors, indivual Feature vectors are aggregated into new feature blocks. Its dimensions are Similarly, indicators Traversal Dimensional acquisition indivual Obtain global feature blocks Its dimensions are Merge global feature blocks The second and third dimensions are transformed into features. Its dimensions are The global sparse sampling layer is represented by formula (23); (23) Step 5.3: Construct the FFN computation layer. The FFN computation layer uses features... For input, feature extraction is performed; the FFN computation layer consists of two sets of FFN computation sub-layers, denoted as the first set of computation sub-layers. Second group of computational sublayers First group of computational sublayers The second set of computational sublayers is used to calculate spatial information relationships. Used to model the relationships between information in channels; First set of computational sublayers Second group of computational sublayers Each includes two fully connected layers, with each fully connected layer followed by an activation function; In step 5.3, the first group of computation sub-layers Features The calculation method is as follows: The first group of computing sub-layers It consists of two fully connected layers, and the number of neurons in the two consecutive fully connected layers is... The input to the fully connected layer is , Equivalent to There are n vectors, each with dimensions of 1. , Each vector is passed through the two fully connected layers, and the vectors generated by the two fully connected layers are... The first set of computational sublayers of G-FFN The calculation process is denoted as formula (24); (24) The second group of computational sublayers The first group of computational sub-layers Output features Feature extraction is performed on the input; the second set of computational sublayers It consists of two fully connected layers, and the number of neurons in the two consecutive fully connected layers is... The second group of computational sublayers by For input, Transform dimensions into feature vectors , Equivalent to There are vectors, each vector having a dimension of . ;Then The input is fed into two fully connected layers, which process the features. Perform feature extraction and output features Furthermore, the shape of the features is deformed. The dimension of the feature becomes The second set of computational sublayers of G-FFN The calculation process is denoted as formula (25); (25)。 7. The lightweight view cup segmentation method based on knots as described in claim 6, characterized in that: In step six, Substitute formulas (20)-(22) into formula (5) as the steps to be executed sequentially, and similarly substitute them into formulas (7), (16), and (18); Substitute formulas (23)-(25) into formula (9) as steps to be executed sequentially, and similarly substitute them into formulas (11), (12), and (14); Based on the above formula, a view cup segmentation network is constructed by substituting the transformation and embedding a local information calculation module and a global information sparse calculation module; a structurally complete lightweight view cup segmentation network is obtained; the global information sparse calculation module performs sparse sampling of global information, and then sends the sampled global information into the FFN calculation layer to mine the relationship between global features.

8. The lightweight view cup segmentation method based on knots as described in claim 7, characterized in that: In step seven, The knot constraint loss described in step seven The following calculation is performed on the fundus image generated in step two after eliminating the optic cup boundary. Fundus images without removal of the optic cup margin Input encoders respectively The generated features are calculated according to formulas (4) to (11). ; will be with Encoder for input The generated features Recorded as ; will be Encoder for input The generated features Recorded as ; feature and characteristics The comparison is performed point-by-point along both length and width dimensions, using Euclidean distance and cosine similarity. Joint comparison by distance; features and characteristics The distance loss function is calculated according to formula (26); (26) in, The values ​​of the representative feature at position i in the high dimension, position j in the wide dimension, and position c in the channel of the dimension; The vector representing the feature at position i in the height of dimension and position j in the width of dimension; similarly... and They have similar meanings; Step 7: Adversarial Learning Loss The goal is to improve the cup segmentation results predicted by the lightweight cup segmentation network. Overall, it resembles a realistic visual cup segmentation result. Because adversarial learning focuses on global information, it can represent the overall distribution of segmentation results. Compared with Dice loss, which compares pixels one by one, adversarial learning loss... It can eliminate the predicted cup segmentation results Discontinuous phenomena; counteracting learning loss The specific calculation method is as follows: Construct a binary classification network Used for predicting the cup segmentation results And the actual visual cup segmentation results Classification is performed to achieve the segmentation of predicted view cups. The image is classified into class 0, which represents the predicted segmentation result; simultaneously, the binary classification network will classify the actual image segmentation result. The image is classified into Class 1, which represents a true view cup segmentation annotation. The lightweight view cup segmentation network aims to predict the view cup segmentation result. Compared with the actual visual cup segmentation results The similarity makes the binary classification network unable to accurately classify... and Classify; counteract learning loss Calculate according to formula (27); (27) in Represents mathematical expectation, Represents a logarithm; Dice loss in step seven The goal is to compare the predicted cup segmentation results pixel by pixel. And the actual visual cup segmentation results The difference is used to optimize the predicted view cup segmentation results. Dice loss is calculated according to formula (28); (28) in This represents the value of Y at position i in height dimension and j in width dimension; express The values ​​at positions i in the height dimension and j in the width dimension; In step seven, the total loss function is the loss. ,loss and loss The weighted sum; the final total loss is calculated according to formula (29); (29) in , and Indicates the weight.

Images