An eye Placido ring image tear film breakage feature recognition method

Abstract

The application belongs to the technical field of computer vision feature recognition, and particularly relates to an eye Placido ring image tear film breakage feature recognition method, a network for eye Placido ring image tear film breakage feature recognition is constructed, the network for eye Placido ring image tear film breakage feature recognition is trained, and a tear film breakage image feature probability mask is output through the network after training. The application is fully adapted to the circular ring geometric characteristics, sensitive to multi-scale breakage, and can effectively utilize the time dimension causality evolution law to remove dynamic artifacts.

Description

Technical Field

[0001] This invention belongs to the field of computer vision feature recognition technology, specifically relating to a method for recognizing tear film breakage features in Placido ring images of the eye. Background Technology

[0002] Eye image analysis related to tear film stability typically relies on Placido ring reflection images. Corneal topography based on the Placido ring projection principle projects a series of concentric rings onto the corneal surface; when the tear film breaks down or becomes locally unstable, the acquired concentric ring images will show morphological changes such as local distortion, narrowing, or breakage in the corresponding areas. These morphologically altered regions in the images can serve as target objects for subsequent image feature recognition and quantitative analysis.

[0003] Currently, tear film morphology analysis based on Placido ring images still mainly relies on manual observation. This requires manually observing the ring morphology changes of a large number of pixels frame by frame, which is not only time-consuming and labor-intensive, but also easily affected by observation experience, fatigue levels, and differences in image quality, leading to insufficient repeatability and consistency.

[0004] While some automated auxiliary methods based on traditional digital image processing emerged in the early stages, such as using Canny edge detection, Gabor filters, or thresholding to extract ring features, these methods have extremely high requirements for image quality. In actual acquisition scenarios, eye images are often accompanied by complex noise such as uneven lighting, eyelash occlusion, eyelid shadows, and tear reflection. Traditional algorithms lack sufficient robustness and are prone to false positives or false negatives.

[0005] In recent years, deep learning technologies, represented by convolutional neural networks (CNNs), have made significant progress in image segmentation and target feature recognition. Models such as U-Net and DeepLab have been used for fine-grained image target region segmentation in various classes. However, when directly applying general semantic segmentation models to the task of tear film rupture feature recognition in Placido ring images, problems such as geometric mismatch, easy loss of small target feature regions, and insufficient utilization of temporal information still exist.

[0006] Existing convolutional neural networks (such as ResNet and VGG) are all built based on the Cartesian coordinate system, and their convolutional kernels are usually designed to be square (e.g., However, Placido images exhibit a significant concentric circle geometry, with tear film rupture manifesting as tiny fractures along the circumference. In Cartesian coordinates, the curvature of the rings varies at different locations in the image, making it difficult for the convolutional kernel to learn rotationally invariant features. This results in the model consuming a large number of parameters to fit this geometric deformation, leading to inefficiency.

[0007] Tear film breakup areas (i.e., "cracks") are typically extremely subtle and occupy a very small percentage of the overall image (sparse foreground). General segmentation networks (such as U-Net) often include multiple max-pooling or large-stride convolution downsampling, which, while expanding the receptive field, also severely smooth out high-frequency details. This makes it difficult to reconstruct subtle crack features during decoding and resolution restoration, often resulting in blurred edges or complete missed detections.

[0008] This is one of the significant limitations of existing 2D models. Placido ring image acquisition devices typically acquire dynamic video sequences, which are often accompanied by minute eye tremors, blinking, and instantaneous tear reflections during the acquisition process. Single-frame 2D networks struggle to distinguish between "transient specular artifacts" and "real structural rupture morphology features," easily producing false positive results and failing to meet the specificity and stability requirements of highly reliable image analysis applications.

[0009] In summary, existing technologies struggle to achieve high-precision, flicker-resistant, automated identification of the morphological features of microcracks in Placido rings under complex eye noise and dynamic interference. Therefore, there is an urgent need for a dedicated image feature recognition method that can adapt to the geometric characteristics of the ring, is sensitive to multi-scale fractures, and can utilize temporal evolution patterns to eliminate dynamic artifacts. Summary of the Invention

[0010] To address the aforementioned technical problems, this invention provides a method for identifying tear film breakage features in Placido ring images of the eye, comprising:

[0011] A network for tear film breakage feature recognition in Placido ring images of the eye is constructed. The network is trained and outputs a tear film breakage image feature probability mask through the trained network.

[0012] The network used for tear film breakage feature recognition in Placido ring images of the eye includes: a multi-scale pooling Siamese encoder, a spatiotemporal fusion module ST_Fusion, a spatial transformation network STN, and a nested dense skip connection decoder U-Net++Decoder.

[0013] The training of the network for tear film breakage feature recognition of Placido ring images of the eye includes: S1, inputting a sequence of original dynamic continuous video frames of the eye containing Placido rings, obtaining the location information of the region of interest (ROI), and using polar coordinate transformation to expand it into a rectangular image sequence;

[0014] S2. Input the rectangular image sequence into the multi-scale pooling SiameseEncoder for spatial feature extraction, and embed the hollow spatial pyramid pooling ASPP module in the bottleneck layer of the SiameseEncoder to output the multi-scale spatial feature sequence.

[0015] S3. Input the multi-scale spatial feature sequence into the spatiotemporal fusion module ST_Fusion, and use the spatial transformation network STN to spatially align the features of adjacent frames.

[0016] S4. In the spatiotemporal fusion module, the aligned feature sequence is modeled for temporal evolution to eliminate transient artifacts and obtain the target frame features with eliminated transient artifacts.

[0017] S5. Input the target frame features that eliminate transient artifacts into the nested densely skip-connected decoder U-Net++Decoder, and output the lesion probability mask at each level in combination with the deep supervision mechanism.

[0018] S6. Construct a global hybrid optimization objective that includes labeled smooth cross-entropy loss, regional Dice loss, and temporal area monotonicity physical constraints, and perform end-to-end training parameter optimization on the network.

[0019] The beneficial effects of this invention are:

[0020] This invention reduces the difficulty for convolutional networks to learn circumferential geometric deformations and improves the efficiency of recognizing minute fracture features in the circumferential direction by unfolding the Placido ring from a circular ring structure into a rectangular stripe structure.

[0021] This invention enhances the ability to express tear film breakup morphology features at different scales by combining ASPP multi-scale feature aggregation with U-Net++ dense skip connections.

[0022] This invention reduces false recognition caused by micro-movements of the eye and transient reflections by using STN spatial alignment and Bi-ConvLSTM temporal modeling.

[0023] This invention introduces image temporal priors into the training process by using a tolerance-based temporal area monotonicity constraint, thereby improving the stability and continuity of the output probability mask. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the overall process of an automated method for recognizing tear film breakage features in Placido ring images of the eye according to the present invention;

[0025] Figure 2 This is a schematic diagram of the structure of the Region of Interest (ROI) localization network (RoiNet) in an embodiment of the present invention;

[0026] Figure 3 This is a diagram showing the overall network architecture of the Siamese Segmentation Network (SST-UNet++) in this embodiment of the invention.

[0027] Figure 4 This is an architecture diagram of the spatial transformation alignment feature extractor (STNAligner) built into the spatiotemporal fusion module (ST_Fusion) in this embodiment of the invention;

[0028] Figure 5 This is a complete data flow graph of the bidirectional convolutional long short-term memory network (Bi-ConvLSTM) built into the spatiotemporal fusion module (ST_Fusion) in this embodiment of the invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] An automated method for identifying tear film breakage features in ocular Placido ring images based on polar coordinate transformation and multi-scale deep learning (SST-UNet++), such as Figure 1 As shown, the specific steps are as follows:

[0031] Step 1: Input a sequence of continuous video frames containing the eye movement and Placido loops into the Region of Interest (ROI) localization network and perform polar coordinate unrolling;

[0032] Step 2: Input the polar coordinate sequence into the Siamese Encoder with shared weights for layer-by-layer spatial feature extraction, and embed the Spatial Pyramid Pooling (ASPP) module in the bottleneck layer;

[0033] Step 3: Input the features into the spatiotemporal fusion module (ST_Fusion) and use the spatial transformation network (STN) to align the spatial features of adjacent frames;

[0034] Step 4: In the spatiotemporal fusion module, a bidirectional convolutional long short-term memory network (Bi-ConvLSTM) is used to extract spatiotemporal context features along the forward and reverse temporal directions and aggregate the target frames;

[0035] Step 5: Input the nested dense skip connection decoder (U-Net++Decoder), and output a multi-scale probability mask by combining a deep supervision mechanism;

[0036] Step 6: Calculate the regional temporal monotonicity constraints with tolerance based on the prediction mask. ), construct a global hybrid optimization objective and update network parameters.

[0037] The input in step S1 is a sequence of raw, continuous video frames of eye movement containing a Placido loop. The region of interest (ROI) localization information is obtained and polar coordinate transformation is performed. The specific method is as follows:

[0038] Input data: A dynamic sequence containing T consecutive frames of images.

[0039] Output data: A sequence of rectangular images after polar coordinate expansion.

[0040] Step 1: Input the original image into a pre-trained lightweight Region of Interest Regression Network (RoiNet) to predict the center coordinates (c) of the Placido ring. x c y ) and the maximum detection radius r from the center to the outermost effective Placido ring boundary. max ; where c x c represents the x-coordinate of the circle's center in the Cartesian coordinate system of the original image. y This represents the ordinate of the circle's center in the Cartesian coordinate system of the original image;

[0041] Step 2: Based on the coordinates of the center of the circle (c x c y ) and maximum detection radius r max Establish a polar coordinate system with the center of the Placido ring as the origin, and define the coordinates of the unfolded rectangular image as (u, v), where u corresponds to the angular direction and v corresponds to the radial direction. u and v together determine the position of the ring to be sampled in the original image. Each (u, v) point is back-mapped to a sampling point in the Cartesian coordinate system of the original image through the polar angle θ and the radial distance ρ.

[0042] Step 3: Calculate the sampling points on the Cartesian coordinate system of the original image. , The conversion formula is expressed as:

[0043]

[0044]

[0045]

[0046]

[0047] Where θ represents the polar angle obtained by normalizing the horizontal coordinate u of the unfolded graph, and ρ represents the radial distance obtained by converting the vertical coordinate v of the unfolded graph; and These represent the width and height of the rectangular image after polar coordinate expansion, respectively; and These represent the inner and outer diameters of the region of interest, respectively. This represents the x-coordinate of the circle's center in the Cartesian coordinate system of the original image. This represents the ordinate of the circle's center in the Cartesian coordinate system of the original image; This represents the x-coordinate of the sampling point in the Cartesian coordinate system. This represents the ordinate of the sampling point in the Cartesian coordinate system. The polar coordinate system is used to describe the angle and radius position on the ring, while the Cartesian coordinate system is used for actual sampling on the original image. The two are established in a one-to-one correspondence through the mapping formula mentioned above. The value of the corresponding pixel is obtained through bilinear interpolation, thus completing the geometric transformation from the ring to the horizontal stripes.

[0048] Step S2, which involves inputting the polar coordinate sequence image into a multi-scale pooling twin encoder for spatial feature extraction, is described in the following specific method:

[0049] Input data: A sequence of polar coordinate images with a frame length of T.

[0050] Output data: contains multi-scale spatial feature sequences at 5 levels.

[0051] Step 1: Flatten and fold the image sequence of length T along the batch dimension, so that its shape is from... Transform into And input a shared weighted twin encoder, where B represents the batch size, T represents the number of input sequence frames, C represents the number of image channels, H represents the image height, and W represents the image width;

[0052] Step 2: Extract the feature map of layer l at time t through successive convolution and pooling operations. (l=0,1,2,3,4), where l=0,1,2,3,4 represents different downsampling levels of the encoder, and t represents the time index in the input sequence.

[0053] Step 3: In the deepest bottleneck layer (l=4), the features are input into the Spatial Pyramid Pooling (ASPP) module, and the receptive field is expanded through parallel dilated convolutions with different dilation rates.

[0054] The feature aggregation calculation of the ASPP layer is represented as follows:

[0055]

[0056] in, This represents the feature map after aggregation of the hollow space pyramid pooling ASPP module. This represents a dilated convolution operation with an inflation rate of k, where k = (1, 6, 12, 18), and P GAPThis indicates global adaptive average pooling, while Concat indicates concatenation along the channel dimension. This represents the input feature map of the 4th layer at time t.

[0057] Step S3 involves inputting features into the spatiotemporal fusion module, and spatially aligning features from adjacent frames using a spatial transformation network. The specific method is as follows:

[0058] Input data: Single-layer sequence features output by the twin encoder .

[0059] Output data: Spatially aligned feature sequences .

[0060] Step 1: Select the intermediate frame in the time dimension of the sequence as the target reference frame (denoted by its time index as ). ), the corresponding feature is denoted as .

[0061] Step 2: For any moving frame in the sequence ), and compare it with the reference frame After concatenation along the channel dimension, the data is input into the Localization Network, which outputs data through multiple convolutional layers and global pooling. affine transformation matrix .

[0062] Step 3: Using the affine transformation matrix Generate a sampling grid, perform bilinear interpolation resampling on the moving frame, and obtain spatially aligned features. It is represented as:

[0063]

[0064] in, This represents the feature map of the input moving frame based on the sampling coordinate grid. A grid sampling operation is performed for bilinear interpolation resampling. Represents any moving frame in the sequence. Indicates based on the affine transformation matrix The operation of generating a normalized sampling coordinate grid.

[0065] For the reference frame, keeping the features unchanged, the final aligned feature sequence that eliminates micro-movements of the eyeball is obtained. .

[0066] Step S4 describes the use of a bidirectional convolutional long short-term memory network to model the temporal evolution of the aligned feature sequence. The specific method is as follows:

[0067] Input data: Aligned feature sequences .

[0068] Output data: Target frame features incorporating bidirectional spatiotemporal causality. .

[0069] Step 1: Construct forward and backward convolutional long short-term memory networks (ConvLSTMCell) for... Input features at time step Calculate its internal input gate Forgotten Gate Output gate and cellular memory state and hidden state Input gate The forgetting gate controls the proportion of new information in the current frame written into the cell's memory. The output gate controls the proportion of memory retention or decay from the previous time step. Controlling the output strength of the current cellular memory to the hidden state, cellular memory state This is used to accumulate stable changes in the ring rupture morphology over time. The calculation is as follows:

[0070]

[0071]

[0072]

[0073]

[0074]

[0075] Here, * represents the convolution operation, and ⊙ represents the Hadamard product (element-by-element multiplication). Output in hidden state This represents the cellular memory state. The aforementioned gating variables will continue to propagate in both forward and reverse temporal recursion: Determine whether to write the valid features of the current frame. The decision on whether or not historical memory is preserved The temporal feature strength of the final output to the next network layer is determined, so that subsequent steps can utilize it. and Extract continuous structural breakage features and suppress transient noise that occurs only in a single frame.

[0076] Step 2: The forward network processes the sequence from past to future along the timeline to obtain a list of forward hidden states. The reverse network processes the sequence backward from the future to the past along the timeline to obtain a list of reverse hidden states. .

[0077] Step 3: Extract the target time. The positive and negative hidden states are spliced ​​together and then passed through. Convolutional processing performs channel reduction and fusion to output target frame features that eliminate transient artifacts. :

[0078]

[0079] in, This represents the list of forward hidden states for the target frame. This represents the list of reverse hidden states for the target frame.

[0080] The specific method for extracting target frame features and inputting them into the nested dense skip connection decoder in step S5 is as follows:

[0081] Input data: spatiotemporal features of each level of fusion output by the ST_Fusion module.

[0082] Output data: main network segmentation mask and auxiliary prediction masks at each level.

[0083] Step 1: Construct a nested dense skip connection topology for U-Net++. For decoder nodes... (in Indicates the downsampling level. (representing the decoding depth), its feature fusion calculation process is expressed as follows:

[0084]

[0085] in, This represents the features from the i-th level ST_Fusion module after eliminating transient artifacts. This indicates a bilinear interpolation upsampling operation. The symbol represents a feature fusion module consisting of convolutional layers, normalization layers, and nonlinear activation functions. Indicates the downsampling level. Indicates the decoding depth;

[0086] Step 2: In the shallow nodes of the decoder (such as...) This leads to a deep supervision branch, through parallel... Convolutional layers independently output auxiliary probability masks of different scales to accelerate gradient backpropagation at the lower layers of the network.

[0087] Step S6 involves constructing a global hybrid optimization objective that includes temporal area monotonicity physical constraints to train the network. The specific method is as follows:

[0088] Input data: The main prediction probability mask, the auxiliary prediction probability mask set, and the ground truth labels output by the network.

[0089] Output data: Network weight parameters after backpropagation optimization.

[0090] Step 1: Calculate the master-slave supervision loss of spatial features. Define the combined spatial loss function. The binary cross-entropy loss (LS-BCE) with smoothed positive sample weights and the region-level Dice loss are used to... Proportional weighting is used to alleviate the problem of extremely sparse lesion prospects.

[0091] Step 2: Construct the temporal monotonicity physical constraint penalty term ( On the feature time series, two adjacent time points are extracted using a time series auxiliary head. Frames and Predicted probability map of frames and Calculate the predicted total tear film breakup area at each time point. and :

[0092]

[0093]

[0094] Step 3: Based on the temporal prior of images in the same acquisition sequence, which typically show continuous expansion or maintenance of tear film rupture morphology over time, and combined with the allowable image micro-jump tolerance threshold. Calculate the one-way penalty value that violates the rule that "the area of ​​the target feature increases monotonically with time". :

[0095]

[0096] Step 4: Combine and construct a global hybrid optimization objective function :

[0097]

[0098] in, The spatial loss of the main network output node, The mean of the spatial auxiliary loss for each depth-monitoring node. These are the weighting coefficients for the temporal-physical constraint term. They are minimized using the backpropagation algorithm. This enables physical priors to effectively guide the gradients of the underlying layers of neural networks.

[0099] Example 1:

[0100] In this embodiment, it is assumed that the Placido ring image acquisition device inputs a dynamic video sequence of the eye's Placido ring, which contains a total of 5 frames (i.e., a sliding window). The original single-frame image has a resolution of pixels. This embodiment performs tear film rupture image feature recognition and model optimization on this 5-frame video sequence, specifically including the following steps:

[0101] (1) Constructing a polar coordinate geometric transformation sequence:

[0102] In this embodiment, as Figure 2 As shown, the first step is to extract the 1st... The center and radius of the original image frame.

[0103] Step 1: Assume the center coordinates of the circles decoded from the network heatmap are... , And the maximum outer diameter obtained by adaptive pooling regression is Pixels, the lower limit of the set inner diameter is Pixel.

[0104] Step 2: Set the size of the target rectangular image after polar coordinate unfolding to... , The definition of polar coordinate expansion is:

[0105]

[0106] For example: for a rectangular image with coordinates... The pixels, their sampling points in the Cartesian coordinate system of the original image. The calculation is as follows:

[0107]

[0108]

[0109]

[0110]

[0111] After bilinear interpolation sampling, the mapping of all pixels is completed. After converting all 5 frames of images, the resulting shape is... A sequence of tensors.

[0112] (2) Twin encoder and ASPP multi-scale feature extraction:

[0113] Step 1: To ensure consistency in sequence feature extraction and improve hardware parallel computing efficiency, the polar coordinate tensor is folded in both the batch and time dimensions, changing its shape from... Transform into Then, the weighted Siamese encoder is input, and the overall network architecture of the Siamese segmentation network (SST-UNet++) is shown in the figure below. Figure 3As shown.

[0114] Step 2: The tensor is sequentially passed through 5 layers of convolution (Level 0 to Level 4) and... Downsampling operation. In the deepest bottleneck layer (Level 4), the tensor size is downsampled to half the size of the original image. The resulting shape is deep features .

[0115] Step 3: Put Input the ASPP module. The ASPP module utilizes the expansion rate... The dilated convolutions (to obtain receptive fields of different scales) and the globally adaptive average pooling layers are processed in parallel, and their computation is represented as follows:

[0116]

[0117] The feature map size output by this operation is maintained. The physical landscape remains unchanged, but the channel encompasses a multi-scale physical view ranging from extremely small cracks to large-area rupture patches.

[0118] (3) Spatial alignment and temporal modeling in the spatiotemporal fusion module (ST_Fusion):

[0119] In this embodiment, the deep features output by the encoder need to be unfolded to restore them to their original state. The five-dimensional tensor format. Let the time index of the 5-frame sequence be . .

[0120] Step 1: The architecture diagram of the spatial transformation alignment feature extractor (STNAligner) built into the spatiotemporal fusion module (ST_Fusion) is as follows. Figure 4 As shown, Spatial Alignment (STN): Selecting the center frame. As the target reference frame For moving frames in the sequence (such as...) frames Spatial offset features are extracted through a localization network and regressed. affine transformation matrix .

[0121] Step 2: Using F. affine grid based on A two-dimensional relative coordinate grid is generated, and then the bilinear interpolation function F.grid is used. sample right Perform pixel-level spatial resampling to obtain spatially aligned feature sequences. This step eliminates spatial errors caused by the patient's subtle eye movements.

[0122] Step 3: Bidirectional temporal modeling (Bi-ConvLSTM) Figure 5 As shown. The pure sequence Feed into the bidirectional ConvLSTM. The forward network is then... Processing, reverse network according to Processing. Using the positive cell state at time t=2 of the target frame. and hidden state For example, the calculation formula is as follows:

[0123]

[0124] at this time, Includes arrive Historical evolution information, and corresponding Then it includes Backtracking This information confirms the future.

[0125] Step 4: State Aggregation. Extraction The forward and reverse hidden states at each moment (both have shapes) After being stitched together along the channel dimension, through Convolution reduces its dimensionality to the original number of channels. :

[0126] This feature It possesses strict spatiotemporal causal logic, which is then input into the U-Net++ dense decoder and processed through dense jump connections. Output multi-scale backbone probability masks and auxiliary probability masks.

[0127] (4) Calculate the regional-level temporal monotonicity constraint and hybrid loss:

[0128] During the network training phase, a global hybrid optimization objective is constructed to address the problem of extreme sample imbalance and incorporate principles of fluid dynamics.

[0129] Step 1: Spatial blending loss calculation. Define the spatial loss of a single-layer prediction mask. Smoothing with labels (smoothing coefficient) Cross-entropy loss With regional level Weighted sum of losses:

[0130]

[0131] Step 2: Construction of the temporal monotonicity penalty function ( Based on the physical law that the tear film breakup area monotonically increases over time, a temporal classification header is derived from the bottom layer of the decoder. Extraction Time and Predicted probability plot at time (tensor element values ​​are in) (between), its spatial dimensions Perform a full-map integration to calculate the total fracture area:

[0132]

[0133] Set an area tolerance threshold that allows for physiological fluctuations. The deviation (penalty) is defined as follows:

[0134] Step 3: Specific Penalty Value Calculation Example. Assume that in a certain training batch, a tolerance threshold is set. Pixels. Calculated by integration. The total tear film breakup area at time twitch is Pixel.

[0135] Scenario A: If the network accurately captures the pathological evolution and predicts... The area at time step A_t increases to A_t = 2800 pixels. The penalty value is then calculated as follows:

[0136]

[0137] Scenario B: If the network is affected by extremely strong transient specular artifacts, it may make incorrect predictions. The fracture area actually shrank to Pixels, which severely violates the laws of physics. The penalty value is calculated as follows:

[0138] Step 4: Global Parameter Update and Gradient Backpropagation. The spatial loss of the main branch, the spatial losses of the three deep supervised auxiliary branches, and the temporal penalty term are added together to obtain the global mixed loss function. :

[0139]

[0140] in, For the set physical regularization weights (e.g.) In this embodiment, the high amount generated by situation B... This will generate a huge gradient. This gradient will bypass the static spatial convolutional layer and directly and precisely act on the gated weight matrix of the Bi-ConvLSTM in the ST_Fusion module (e.g., ...) through the backpropagation algorithm. After multiple iterations of the AdamW optimizer, the network's response activation values ​​to transient reflective features are forcibly corrected, enabling the model to ultimately possess a high-specificity diagnostic capability of "ignoring transient artifacts and pinpointing real physical fractures."

[0141] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for identifying tear film breakage features in Placido ring images of the eye, characterized in that, include: A tear film breakage feature recognition network for Placido ring images of the eye was constructed. The network was trained and output a tear film breakage image feature probability mask. The network for identifying tear film breakage features in Placido ring images of the eye includes: a multi-scale pooling Siamese encoder, a spatiotemporal fusion module ST_Fusion, a spatial transformation network STN, and a nested dense skip connection decoder U-Net++Decoder. The training of the network for tear film breakage feature recognition in Placido ring images of the eye includes: S1. Input a sequence of raw dynamic video frames of the eye containing the Placido ring, obtain the location information of the region of interest (ROI), and use polar coordinate transformation to expand it into a rectangular image sequence. S2. Input the rectangular image sequence into the multi-scale pooling SiameseEncoder for spatial feature extraction, and embed the hollow spatial pyramid pooling ASPP module in the bottleneck layer of the SiameseEncoder to output the multi-scale spatial feature sequence. S3. Input the multi-scale spatial feature sequence into the spatiotemporal fusion module ST_Fusion, and use the spatial transformation network STN to spatially align the features of adjacent frames. S4. In the spatiotemporal fusion module, the aligned feature sequence is modeled for temporal evolution to eliminate transient artifacts and obtain the target frame features with eliminated transient artifacts. S5. Input the target frame features that eliminate transient artifacts into the nested densely skip-connected decoder U-Net++Decoder, and output the lesion probability mask at each level in combination with the deep supervision mechanism. S6. Construct a global hybrid optimization objective that includes labeled smooth cross-entropy loss, regional Dice loss, and temporal area monotonicity physical constraints, and perform end-to-end training parameter optimization on the network.

2. The method for identifying tear film breakage features in Placido ring images of the eye according to claim 1, characterized in that, Input a sequence of raw, continuous video frames containing the Placido rings of the eye, obtain the region of interest (ROI) localization information, and use polar coordinate transformation to unfold it into a rectangular image sequence, including: S11: Input the original dynamic continuous video frame sequence of the eye containing the Placido ring into the pre-trained lightweight region of interest regression network RoiNet, as shown in Figure 2, to predict the center coordinates (c) of the Placido ring. x c y ) and the maximum detection radius r from the center to the outermost effective Placido ring boundary. max ; where c x c represents the x-coordinate of the circle's center in the Cartesian coordinate system of the original image. y This represents the ordinate of the circle's center in the Cartesian coordinate system of the original image; S12: Based on the center coordinates (c x c y ) and maximum detection radius r max Establish a polar coordinate system with the center of the Placido ring as the origin, and define the coordinates of the unfolded rectangular image as (u, v), where u corresponds to the angular direction and v corresponds to the radial direction. u and v together determine the position of the ring to be sampled in the original image. S13: Based on the polar angle θ and radial distance ρ corresponding to each pixel (u, v) in the polar coordinate unfolded image, calculate the sampling point (x, y) of the coordinate point (u, v) in the polar coordinate unfolded image on the Cartesian coordinate system of the original image. src y src The corresponding pixel values ​​are obtained through bilinear interpolation to complete the geometric transformation from a circular ring to horizontal stripes. Here, x... src The x-coordinate of the sampling point in the Cartesian coordinate system is represented by y. src This represents the ordinate of the sampling point in the Cartesian coordinate system.

3. The method for identifying tear film breakage features in Placido ring images of the eye according to claim 2, characterized in that, Based on the polar angle θ and radial distance ρ corresponding to each pixel (u, v) in the polar coordinate unfolded image, the sampling point (x, y) of the coordinate point (u, v) in the polar coordinate unfolded image on the Cartesian coordinate system of the original image is calculated in reverse. src y src ),include: Where θ represents the polar angle obtained by normalizing the horizontal coordinate u of the unfolded diagram, and ρ represents the radial distance obtained by converting the vertical coordinate v of the unfolded diagram; W polar and H polar Let r represent the width and height of the rectangular image after polar coordinate expansion, respectively; min and r max c represents the inner and outer diameters of the region of interest, respectively; x c represents the x-coordinate of the circle's center in the Cartesian coordinate system of the original image. y The x-coordinate represents the center of the circle in the Cartesian coordinate system of the original image; src The x-coordinate of the sampling point in the Cartesian coordinate system is represented by y. src This represents the ordinate of the sampling point in the Cartesian coordinate system.

4. The method for identifying tear film breakage features in Placido ring images of the eye according to claim 1, characterized in that, The rectangular image sequence is input into the multi-scale pooling Siamese Encoder for spatial feature extraction, including: S12: Flatten and fold the image sequence with a sequence number of frames of length T in the batch dimension, so that its shape is transformed from (B,T,C,H,W) to (B*T,C,H,W), and input it into the Siamese Encoder with shared weights; where B represents the batch size, T represents the number of input sequence frames, C represents the number of image channels, H represents the image height, and W represents the image width; S13: Extract the feature map F corresponding to the l-th layer at time t through successive convolution and pooling operations. t l , where l=0,1,2,3,4 represent different downsampling levels of the encoder, and t represents the time index in the input sequence; S14: In the deepest bottleneck layer l=4, the 4th layer feature map F t 4 The input is a dilated spatial pyramid pooling ASPP module, which expands the receptive field through parallel dilated convolutions with different dilation rates: in, P represents the feature map after aggregation of the hollow space pyramid pooling ASPP module. k This represents a dilated convolution operation with an inflation rate of k, where k = (1, 6, 12, 18), and P GAP This indicates global adaptive average pooling, while Concat indicates concatenation along the channel dimension. This represents the input feature map of the 4th layer at time t.

5. The method for identifying tear film breakage features in Placido ring images of the eye according to claim 1, characterized in that, The multi-scale spatial feature sequence is input into the spatiotemporal fusion module ST_Fusion, and the spatial transformation network STN performs spatial alignment of features from adjacent frames, including: S31: Select the intermediate frame in the sequence's time dimension as the target reference frame; its time index is... The corresponding spatial features are denoted as ,in, The time index of the target reference frame is represented, and T represents the number of frames in the input sequence. S32: For any moving frame in the sequence , ), and compare it with the reference frame After concatenation along the channel dimension, the data is input into the LocalizationNetwork, which outputs data through multiple convolutional layers and global pooling. affine transformation matrix ; S33: Using affine transformation matrices Generate a sampling grid, perform bilinear interpolation resampling on the moving frame, and obtain spatially aligned features. By preserving the features of the reference frame, an aligned feature sequence that eliminates micro-movements of the eyeball is finally obtained. ,in, This indicates that the length of the input sequence frames is... The spatially aligned features of the last frame; Using affine transformation matrix Generate a sampling grid, perform bilinear interpolation resampling on the moving frame, and obtain spatially aligned features. ,include: in, This represents the feature map of the input moving frame based on the sampling coordinate grid. A grid sampling operation is performed for bilinear interpolation resampling. Represents any moving frame in the sequence. Indicates based on the affine transformation matrix The operation of generating a normalized sampling coordinate grid.

6. The method for identifying tear film breakage features in Placido ring images of the eye according to claim 1, characterized in that, In the spatiotemporal fusion module, the aligned feature sequences are modeled for temporal evolution, and the spatiotemporal context features of the target frame are extracted and fused, including: S41: Construct forward and backward convolutional long short-term memory networks (ConvLSTMCell) for... Input features at time step Calculate its internal input gate Forgotten Gate Output gate and cellular memory state and hidden state ; S42: The forward network processes the sequence from past to future along the timeline to obtain a list of forward hidden states. The reverse network processes the sequence backward from the future to the past along the timeline to obtain a list of reverse hidden states. ; S43: Extract target time The positive and negative hidden states are spliced ​​together and then passed through. Convolutional processing performs channel reduction and fusion to output target frame features that eliminate transient artifacts. : in, This represents the list of forward hidden states for the target frame. This represents the list of reverse hidden states for the target frame.

7. The method for identifying tear film breakage features in Placido ring images of the eye according to claim 1, characterized in that, The target frame features, which eliminate transient artifacts, are input into a nested, densely skip-connected decoder, U-Net++Decoder. Combined with a deep supervision mechanism, this outputs probabilistic masks for tear film breakage features at each level, including: S51: Input the target frame features, which eliminate transient artifacts, into the nested densely skip-connected decoder U-Net++Decoder. For decoder nodes... The feature fusion process is represented as follows: in, This represents the features from the i-th level ST_Fusion module after eliminating transient artifacts. This indicates a bilinear interpolation upsampling operation. The symbol represents a feature fusion module consisting of convolutional layers, normalization layers, and nonlinear activation functions. Indicates the downsampling level. Indicates the decoding depth; S52: A deep supervision branch is introduced from the shallow nodes of the decoder, through parallel... Convolutional layers independently output auxiliary probability masks of different scales.

8. The method for identifying tear film breakage features in Placido ring images of the eye according to claim 1, characterized in that, A global hybrid optimization objective is constructed, incorporating labeled smooth cross-entropy loss, region-level Dice loss, and temporal area monotonicity physical constraints. End-to-end training parameter optimization is then performed on the network, including: S61: Calculate the master-slave supervision loss of spatial features and define the combined spatial loss function. The binary cross-entropy loss LS-BCE with smoothed positive sample weights and the region-level Dice loss are used to... Proportional weighting is used to alleviate the problem of extremely sparse lesion prospects; S62: Constructing a temporal monotonicity physical constraint penalty term On the feature time series, a time series auxiliary head is used to extract two adjacent time points. Frames and Predicted probability map of frames and Calculate the predicted total tear film breakup area at each time point. and : S63: Based on the temporal prior of images where tear film breakup morphology features typically expand or remain continuously over time within the same acquisition sequence, and combined with the allowable image micro-jitter tolerance threshold. Calculate the one-way penalty value that violates the physical law that "the area of ​​the target feature increases monotonically with time". : S64: Combining and constructing a global hybrid optimization objective function : in, This represents the spatial loss of the main network output node. This represents the mean spatial auxiliary loss for each depth-monitoring node. Represents the weighting coefficients of the temporal physical constraint terms; S65: Minimize using backpropagation algorithm This enables physical priors to effectively guide the gradients of the underlying layers of neural networks.

Images