A method and apparatus for real-time continuous tracking of the optic disc

Abstract

The application provides a kind of optic disc real-time continuous tracking method and device, it is related to medical image processing technical field.The application obtains the video sequence of ophthalmic surgery and pre-processes, then the video sequence after pre-processing is carried out multi-scale feature extraction and optic disc target detection, obtains optic disc detection target;The optic disc detection target is based on Kalman filtering prediction and appearance feature matching, introduces adaptive gate matching mechanism to carry out interframe target trajectory association, and combines appearance feature cache update mechanism and ReID cosine similarity retrieval to maintain target trajectory, outputs trajectory tracking result;Wherein, the trajectory tracking result includes optic disc position, confidence, trajectory identification and continuous tracking result of time stamp.The application improves small scale optic disc detection precision, enhances the occlusion recovery and trajectory continuity, in strong light reflection, low contrast and local occlusion and other complex surgical scene, realizes high-precision, high robustness, real-time optic disc continuous tracking.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and computer vision technology, and more specifically, to a method and apparatus for real-time continuous tracking of the visual disc. Background Technology

[0002] The optic disc, as a crucial landmark structure in fundus imaging, reflects pathological features of various ocular diseases through changes in its position and morphology. Optic disc detection and continuous tracking are critical for intraoperative navigation, surgical precision control, and postoperative evaluation. This technology aims to accurately identify and continuously track the optic disc region from ophthalmic surgical videos, assisting surgeons in achieving precise positioning and stable monitoring of key anatomical structures under dynamic microscopic conditions. With continuous advancements in deep learning and target detection technologies, the accuracy and robustness of optic disc detection and tracking have been continuously improved, providing core technological support for visual assistance systems in microsurgery.

[0003] Current research on optic disc detection and tracking can be mainly divided into three technical approaches. The first approach is based on traditional image processing and handcrafted features. These methods often utilize edge detection, Hough transform, grayscale gradient analysis, and morphological operations to achieve optic disc localization and segmentation. The second approach is based on deep convolutional neural networks (CNNs), which achieve automatic optic disc recognition and localization through end-to-end feature extraction and classification mechanisms. These models enhance feature representation capabilities through multi-scale convolution and skip connections, achieving high detection accuracy on static fundus images. The third approach is a comprehensive method based on deep spatiotemporal modeling and multimodal fusion, aiming to combine spatial features and temporal information to achieve stable correlations between consecutive frames. This approach improves robustness to occlusion, illumination changes, and non-rigid deformations by introducing spatiotemporal attention, adaptive feature fusion, and re-recognition mechanisms.

[0004] However, existing technologies all have significant drawbacks. The first type of method relies on fixed geometric assumptions and low-level pixel features, making it difficult to adapt to complex factors in surgical scenarios such as uneven lighting, tissue reflection, and target deformation, easily leading to false positives and false negatives. While the second type of method possesses strong local feature extraction capabilities, it is limited by fixed receptive fields and spatial invariance assumptions, making it difficult to capture long-range temporal dependencies and cross-frame dynamic information, resulting in trajectory interruptions and detection instability under conditions of continuous occlusion or strong reflection. Although the third type of method can achieve continuous tracking by combining spatial and temporal features, its network structure is complex and computationally expensive, and it is prone to fluctuations in detection confidence and target drift in microscopic videos. Furthermore, existing algorithms generally lack adaptive processing mechanisms for non-rigid motion and multi-source reflection in microsurgical environments, making it difficult to maintain real-time performance while simultaneously ensuring detection accuracy and trajectory stability.

[0005] In view of the above, this application is hereby submitted. Summary of the Invention

[0006] This invention aims to provide a real-time continuous optic disc tracking method and apparatus to solve the problems of detection instability, trajectory interruption, and identity drift caused by complex factors such as strong reflection, partial occlusion, low contrast, and small-scale targets in existing optic disc detection and tracking technologies in microscopic ophthalmic surgical videos. By constructing a deeply coupled integrated closed-loop visual tracking framework of "detection-tracking-re-identification", the detection accuracy, temporal continuity, and occlusion recovery ability of the optic disc area are significantly improved while maintaining real-time processing capabilities.

[0007] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0008] A method for real-time continuous tracking of a video disc includes:

[0009] S1, acquire video sequences of ophthalmic surgeries;

[0010] S2, preprocessed video sequence;

[0011] S3, perform multi-scale feature extraction and visual disk target detection on the preprocessed video sequence to obtain the visual disk detection target;

[0012] S4, the target detection of the visual disc is based on Kalman filter prediction and appearance feature matching. An adaptive gating matching mechanism is introduced to associate the target trajectory between frames. The target trajectory is maintained by combining the appearance feature cache update mechanism and ReID cosine similarity retrieval, and the trajectory tracking result is output. The adaptive gating matching mechanism is based on a Mahalanobis distance-based gating strategy, and introduces the detection confidence c and velocity residual. As a dynamic adjustment factor, the trajectory tracking result includes continuous tracking results of the disc position, confidence level, trajectory identifier, and timestamp.

[0013] Preferably, the preprocessing includes: pixel normalization, brightness enhancement, noise suppression, region cropping, and data normalization;

[0014] The pixel normalization process involves processing the images extracted frame by frame from the video sequence using linear normalization, and its expression is as follows:

[0015] ;

[0016] in, Represents the pixel-normalized image in coordinates Pixel value at; Represents the original video frame; and These represent the minimum and maximum pixel intensities in the current frame, respectively.

[0017] The brightness enhancement employs a contrast adjustment function based on adaptive histogram equalization to process the pixel-normalized video frames, as shown in the formula:

[0018] ;

[0019] in, For the brightness-enhanced image in coordinates Pixel value at; Indicates the contrast limit threshold; This indicates the grid size into which the image is divided into multiple local blocks; This represents a contrast-limited adaptive histogram equalization algorithm.

[0020] The noise suppression is achieved through Gaussian filtering to smooth and reduce noise, thereby suppressing high-frequency reflections and instrument interference signals in the image. The formula is as follows:

[0021] ;

[0022] in, This indicates the Gaussian filtered image in coordinates Pixel value at; The standard deviation of the Gaussian kernel is represented by k; k is the radius of the convolution window. This represents the row offset within the convolution window; This represents the column offset within the convolution window;

[0023] The region clipping is based on the coordinates of the field of view center. A fixed region of interest (ROI) is defined for clipping and used as the input range for subsequent detection. The formula is as follows:

[0024] ;

[0025] in, Indicates the coordinates of the center of the field of view; , Define the width and height of the ROI; The image after region cropping;

[0026] The cropped and enhanced images are scaled to the target size and then subjected to batch tensor quantization to obtain the preprocessed video sequence.

[0027] Preferably, when performing multi-scale feature extraction and visual disc target detection on the preprocessed video sequence, the CAST-YOLO detection model is used for processing;

[0028] The CAST-YOLO detection model is composed of a feature extraction backbone network, a feature fusion neck, and a multi-scale detection head. It embeds a Swing Transformer module and integrates a context aggregation module into the YOLO backbone network.

[0029] The feature extraction backbone network includes several Conv convolutional modules, several C2f modules, several SCDown modules, SPPF modules, and PSA modules, and introduces a Swing Transformer module and a context aggregation module.

[0030] The Conv convolution module is used to perform step-by-step downsampling and feature extraction on the input video frames;

[0031] The C2f module is used for residual connections to enhance channel interaction and gradient flow of local features;

[0032] The SCDown module is used to combine convolutional downsampling and channel splitting fusion mechanisms to compress the feature map scale while maintaining the integrity of channel information.

[0033] The SPPF module is used to aggregate global context information through multi-scale pooling to enhance the adaptability of deep features to view disks of different sizes while maintaining computational efficiency.

[0034] The PSA module is used to enhance features in key regions through a pixel-level attention mechanism, suppress background noise interference, and output globally enhanced deep features.

[0035] The Swing Transformer module is used to achieve efficient modeling of global structure and long-range dependencies in deep feature layers;

[0036] The context aggregation module is used to extract the local structure and global semantics of the optic disc under different receptive fields, so as to enhance the multi-scale perception and context-consistent expression of the optic disc region.

[0037] The input video frames are encoded step by step through the feature extraction backbone network combined with windowing and shift self-attention mechanism, and the deep features are fused with multi-scale context weighting to obtain multi-scale core feature maps from low-level texture to high-level semantics.

[0038] At the neck of the feature fusion, a feature pyramid structure based on upsampling and cross-layer stitching is adopted to perform cross-scale feature fusion and enhancement on the multi-scale core feature map to generate a multi-scale detection feature map, so as to improve the detection capability of small targets and boundary regions of the visual disk.

[0039] In the multi-scale detection head section, the visual disc region is predicted for each of the multi-scale detection feature maps, and the final effective visual disc detection target is output.

[0040] Preferably, the Swing Transformer module consists of layer normalization, multi-head self-attention, and multi-layer perception.

[0041] The layer normalization is used to normalize each channel of the input features;

[0042] The multi-head self-attention is used to capture the inter-pixel correlation of the layer-normalized features within a local window, and to establish long-range dependencies across windows through a shift window mechanism.

[0043] In the multilayer perceptron, the output features of multi-head self-attention are nonlinearly transformed through two fully connected layers to enhance the semantic expressive power of the features;

[0044] A convolutional layer is then connected in series after the Swin Transformer module to enhance the feature preservation capability of the view disk boundary region;

[0045] The specific calculation process in the Swin Transformer module is as follows:

[0046] The input feature map is divided into several non-overlapping local windows according to a preset window size, so that self-attention calculation can be performed independently in each window;

[0047] Within each window, the window features are flattened into a feature sequence and then layer normalization is performed.

[0048] The normalized feature sequence is mapped to a query matrix, a key matrix, and a value matrix through three independent linear projection layers, respectively.

[0049] The correlation between pixels within the window is calculated based on the query matrix and the key matrix to obtain the attention weight matrix;

[0050] The attention weight matrix and the value matrix are weighted and summed to obtain the feature sequence after self-attention enhancement within the window;

[0051] Subsequently, based on the feature sequence enhanced by self-attention within the window, a shift window mechanism is adopted to periodically shift the window boundary of the current stage so that the windows of adjacent stages partially overlap with the original window.

[0052] Self-attention is calculated for the offset window so that pixels from different windows can enter the same offset window, thus enabling cross-window information interaction.

[0053] The features computed through cross-window attention are mapped back to the original channel dimension through a linear projection layer and then fused with the residuals by adding them element by element with the input features.

[0054] The residual fused features are mapped through two fully connected layers of a multilayer perceptron to enhance the nonlinear expressive power of the features, resulting in deep enhanced features.

[0055] Preferably, the context aggregation module is used to extract the local structure and global semantics of the optic disc under different receptive fields, so as to enhance multi-scale perception and context-consistent expression of the optic disc region, specifically:

[0056] Receive multi-scale features of the input;

[0057] Upsampling is performed on the high-level small-scale features in the multi-scale features to align with the spatial resolution;

[0058] Perform 1×1 convolutions on the spatially aligned feature maps to align the channels, resulting in aligned multi-scale features.

[0059] The aligned multi-scale features are then subjected to global average pooling to obtain a global context vector, which is then input into a two-layer fully connected network to generate channel attention weights.

[0060] The channel attention weights are multiplied by the aligned multi-scale features using a channel dimension weighted multiplication to obtain spatially weighted features;

[0061] The spatially weighted features are input into several parallel convolutional branches to capture structural information from different receptive fields, thereby obtaining multi-scale convolutional features.

[0062] After weighted fusion of the multi-scale convolutional features and the spatially weighted features, layer normalization and 1×1 convolution are performed to obtain aggregated features;

[0063] The aggregated features are residually connected with the input multi-scale features to obtain context-enhanced features.

[0064] Preferably, the feature fusion neck includes several upsampling modules, several Concat modules, several C2f modules, Conv convolution modules, SCDown modules, and C2fCIB modules;

[0065] The upsampling module is used to perform upsampling on the input features to achieve spatial scale alignment.

[0066] The Concat module is used to perform cross-scale splicing and fusion of input features to obtain fused features;

[0067] The Conv convolution module is used to perform stepwise downsampling and compression of features;

[0068] The C2f module is used for residual connections to enhance channel interaction and gradient flow of local features;

[0069] The SCDown module is used to combine convolutional downsampling and channel splitting fusion mechanisms to compress the feature map scale while maintaining the integrity of channel information.

[0070] The C2fCIB module combines the feature recombination capability of C2f with the context enhancement capability of CIB to strengthen the global semantics and contextual consistency of deep features and output the concatenated features.

[0071] Preferably, the multi-scale detection head includes a Detect module, which is used to independently perform prediction on the feature map of each scale and generate a set of candidate boxes for the optic disc corresponding to each scale;

[0072] The candidate bounding boxes for the visual disc at all scales are subjected to non-maximum suppression to remove highly overlapping and low-confidence redundant boxes, resulting in the final effective visual disc detection target.

[0073] Preferably, S4 specifically comprises:

[0074] Based on the previous frame state of the target detected by the visual disc, a linear prediction is performed using a Kalman filter model to generate the predicted trajectory of the target in the current frame; wherein, the previous frame state includes the target center position, velocity, and scale of the previous frame.

[0075] The candidate bounding boxes of the target detected by the visual disc are received. The matching cost between the candidate bounding boxes and the predicted trajectory is calculated based on the similarity of the appearance features of the candidate bounding boxes, the geometric distance of the target center and the boundary overlap, and the target trajectory is established.

[0076] Lifecycle maintenance of established target trajectories: When the number of consecutive hits on a target trajectory reaches a set consecutive hit threshold, update the appearance features of the candidate bounding box of the target trajectory to suppress template drift caused by short-term glare; when the number of consecutive misses on a target trajectory exceeds a set temporary loss threshold, the target trajectory enters a temporary loss state; when the target trajectory exceeds the set maximum miss threshold and has not recovered, terminate the tracking of the target trajectory.

[0077] When the target trajectory enters a temporary loss state and the number of consecutive missed detection frames reaches the set ReID threshold, the target candidate box of the current frame is compared with the cached appearance features to perform ReID cosine similarity retrieval.

[0078] When the matching score of the cosine similarity retrieval result exceeds the set matching threshold, the current target trajectory is restored to its valid state and re-enters the matching process; otherwise, the target trajectory is terminated.

[0079] Output the trajectory tracking results of the valid targets in each video frame; wherein, the tracking results include the center coordinates of the video disk, the bounding box size, the detection confidence, the trajectory number and the timestamp for each frame.

[0080] Preferably, when calculating the matching cost, the gating threshold is adaptively adjusted by combining the detection confidence and velocity residual to achieve the optimal association between the candidate box and the existing trajectory;

[0081] The formula for calculating the matching cost is as follows:

[0082] ;

[0083] ;

[0084] in, This represents the matching cost between the i-th trajectory and the j-th candidate box; , , These are the weighting coefficients; , , Let represent the appearance similarity between the i-th trajectory and the j-th candidate box, the geometric distance between the target center and the boundary overlap, respectively; Let i be the cache appearance features of the i-th temporarily lost trajectory; The appearance features of the j-th current candidate box;

[0085] The formula for the gate threshold is:

[0086] ;

[0087] in, Represents the dynamic gating threshold for the i-th trajectory; The Mahalanobis distance; Let be the detection confidence score of the j-th candidate box; , These are the detection confidence level and the velocity residual weighting coefficient, respectively. The deviation between the predicted velocity of trajectory i and the actual velocity of candidate box j;

[0088] When updating the appearance features of the candidate bounding boxes for the target trajectory, historical appearance features of the trajectory are stored in a sliding cache queue, and updates are only performed when the number of consecutive hit frames of the trajectory exceeds a set threshold. The formula is as follows:

[0089] ;

[0090] in, The appearance features of the cache center after the current frame is updated; The cache length for historical appearance features; This refers to the appearance feature of the k-th cached history.

[0091] The formula for ReID cosine similarity retrieval is:

[0092] ;

[0093] in, The cache appearance features of the i-th temporarily lost trajectory Compared with the appearance features of the j-th current candidate box Cosine similarity; represents the Euclidean norm, used to calculate the magnitude of a vector; T is the transpose symbol.

[0094] The present invention also provides a real-time continuous tracking device for a video disc, comprising:

[0095] The video sequence acquisition unit is used to acquire video sequences of ophthalmic surgeries.

[0096] The preprocessing unit is used to preprocess the video sequence.

[0097] The disc target detection unit is used to perform multi-scale feature extraction and disc target detection on the preprocessed video sequence to obtain the disc detection target.

[0098] The trajectory tracking unit is used to perform inter-frame target trajectory association based on Kalman filter prediction and appearance feature matching of the visual disc detection target, and to maintain the target trajectory by combining the appearance feature cache update mechanism and ReID cosine similarity retrieval, and output the trajectory tracking result; wherein, the trajectory tracking result includes continuous tracking results of visual disc position, confidence, trajectory identifier and timestamp.

[0099] The present invention also provides a real-time continuous video disc tracking device, including a processor and a memory, wherein the memory stores a computer program that can be executed by the processor to realize a real-time continuous video disc tracking method as described above.

[0100] The present invention also provides a computer-readable storage medium storing computer-readable instructions, which, when executed by a processor of the device on which the computer-readable storage medium is located, implement a video disc real-time continuous tracking method as described above.

[0101] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0102] This invention embeds the Swing Transformer module and integrates the context aggregation (CAM) module into the YOLO backbone network, which enables the detection module to significantly improve the detection accuracy and boundary clarity of small-scale, weakly textured visual disk targets while maintaining real-time inference speed.

[0103] This invention introduces an adaptive gating matching mechanism, an appearance feature sliding cache and steady-state update mechanism, and an on-demand ReID retrieval mechanism into the DeepSORT trajectory tracking framework, enabling the tracking module to maintain the temporal continuity of the trajectory, identity consistency, and occlusion recovery capability in complex surgical scenarios such as strong reflection, partial occlusion, and low contrast.

[0104] The overall framework of this invention achieves high-precision, high-robustness, and high-continuity real-time tracking of the optic disc region without sacrificing real-time performance through deep collaboration between detection and tracking. It is suitable for applications such as ophthalmic surgical navigation, intraoperative monitoring, and postoperative intelligent analysis. Attached Figure Description

[0105] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0106] Figure 1 This is a flowchart illustrating a real-time continuous tracking method for a video disc provided in Example 1.

[0107] Figure 2 This is an overall structural diagram of a real-time continuous tracking method for a video disc provided in Embodiment 1.

[0108] Figure 3 This is a structural diagram of the CAST-YOLO detection model provided in Example 1.

[0109] Figure 4 This is a schematic diagram of the Swing Transformer encoding structure provided in Example 1.

[0110] Figure 5 This is a structural diagram of the context aggregation CAM module provided in Example 1.

[0111] Figure 6 This is a structural diagram of the A-DeepSORT target tracking model provided in Example 1.

[0112] Figure 7 This is a schematic diagram of a real-time continuous tracking device for a video disc provided in Embodiment 2.

[0113] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Detailed Implementation

[0114] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0115] Example 1

[0116] Embodiment 1 of the present invention provides a real-time continuous tracking method for a video disc, which can be implemented by a real-time continuous tracking device for a video disc (hereinafter referred to as the tracking device), and in particular, executed by one or more processors within the tracking device.

[0117] In this embodiment, the tracking device may be an electronic device equipped with a processor, the processor having a computer program for the real-time continuous tracking method of the video disc and the computer program being executable, such as a computer, smartphone, smart tablet, workstation, etc., which are not limited here.

[0118] like Figures 1-2 As shown, a method for real-time continuous tracking of a video disc includes steps S1 to S4.

[0119] S1, acquire video sequences of ophthalmic surgeries.

[0120] This step involves acquiring and preprocessing ophthalmic surgical video data to ensure the stability and consistency of the input frames, providing a reliable data foundation for subsequent detection and tracking. Specifically, the input data selected in this invention is a sequence of ophthalmic surgical videos acquired using a high-resolution microscope. For example, the preferred video frame rate is 30–60 frames per second, and the resolution is not less than 640×480, to ensure the continuity and detail integrity of the target area (optic disc) in both temporal and spatial dimensions.

[0121] S2, preprocessed video sequence.

[0122] The preprocessing includes: pixel normalization, brightness enhancement, noise suppression, region cropping, and data normalization;

[0123] First, the input video stream is extracted frame by frame at a fixed frame rate and converted into a uniform image sequence (RGB three-channel image) to ensure the consistency of input data from different video sources.

[0124] The pixel normalization process involves processing the images extracted frame by frame from the video sequence using linear normalization. The expression is as follows:

[0125] ;

[0126] in, Represents the pixel-normalized image in coordinates Pixel value at; Represents the original video frame; and These represent the minimum and maximum pixel intensities in the current frame, respectively.

[0127] This normalization process effectively reduces the impact of brightness differences between different imaging devices.

[0128] To improve the model's robustness under complex lighting conditions, brightness equalization and noise suppression operations are performed on the input frames. The brightness enhancement uses a contrast adjustment function based on adaptive histogram equalization to process the pixel-normalized video frames, as shown in the formula:

[0129] ;

[0130] in, For the brightness-enhanced image in coordinates Pixel value at; Indicates the contrast limit threshold; This indicates the grid size into which the image is divided into multiple local blocks; This represents an adaptive histogram equalization algorithm with limited contrast.

[0131] The noise suppression is achieved through Gaussian filtering to smooth and reduce noise, thereby suppressing high-frequency reflections and instrument interference signals in the image. The formula is as follows:

[0132] ;

[0133] in, This indicates the Gaussian filtered image in coordinates Pixel value at; The standard deviation of the Gaussian kernel is represented by k; k is the radius of the convolution window. This represents the row offset within the convolution window; This represents the column offset within the convolution window.

[0134] Based on prior knowledge that the visual disc target is stably distributed at the image center in a microsurgical scenario, the region cropping is based on the coordinates of the field of view center. A fixed region (ROI) is set for cropping and used as the input range for subsequent detection. The formula is as follows:

[0135] ;

[0136] in, Indicates the coordinates of the center of the field of view; , Define the width and height of the ROI; This is the image after region cropping. This constraint reduces background redundancy, significantly improving detection speed and accuracy.

[0137] The cropped and enhanced images are uniformly scaled to the target size and then subjected to batch tensor quantization to obtain a preprocessed video sequence, which provides stable temporal information support for subsequent trajectory prediction and state estimation.

[0138] Through the above preprocessing steps, the spatial scale, illumination distribution, and noise level of the video data are standardized, providing a unified and clearly defined input source for the subsequent CAST-YOLO detection model. This ensures that the model has excellent generalization performance and robustness under different microscopic equipment, lighting environments, and video quality conditions.

[0139] S3 performs multi-scale feature extraction and visual disc target detection on the preprocessed video sequence to obtain the visual disc detection target.

[0140] This step is used to achieve multi-scale feature extraction and high-precision detection of the optic disc region in microsurgical video frames. The CAST-YOLO detection model is used for multi-scale feature extraction and optic disc target detection on the preprocessed video sequence. For example... Figure 3 As shown, the CAST-YOLO detection model proposed in this invention consists of a feature extraction backbone network, a feature fusion neck, and a multi-scale detection head. The Swing Transformer and context aggregation module are introduced into the YOLO backbone network to improve the perception ability and robustness of the optic disc target in complex surgical scenarios.

[0141] In CAST-YOLO, "CAST" stands for "Context Aggregation Swin Transformer." By introducing the hierarchical self-attention mechanism and context aggregation module of the Swin Transformer into the YOLO architecture, multi-scale feature extraction and global context modeling of the optic disc region in microsurgical videos are achieved, thereby improving detection accuracy and boundary recognition capabilities.

[0142] The feature extraction backbone network includes several Conv convolutional modules, several C2f modules, several SCDown modules, SPPF modules, and PSA modules, and introduces a Swing Transformer module and a context aggregation module.

[0143] The Conv convolution module is used to perform stepwise downsampling and feature extraction on the input features;

[0144] The C2f module is used for residual connections to enhance channel interaction and gradient flow of local features;

[0145] The SCDown module is used to combine convolutional downsampling and channel splitting fusion mechanisms to compress the feature map scale while maintaining the integrity of channel information.

[0146] The SPPF module is used to aggregate global context information through multi-scale pooling to enhance the adaptability of deep features to view disks of different sizes while maintaining computational efficiency.

[0147] The PSA module is used to enhance features in key regions through a pixel-level attention mechanism, suppress background noise interference, and output globally enhanced deep features.

[0148] The Swing Transformer module is used to achieve efficient modeling of global structure and long-range dependencies in deep feature layers;

[0149] The context aggregation module is used to extract the local structure and global semantics of the optic disc under different receptive fields, so as to enhance the multi-scale perception and context-consistent expression of the optic disc region.

[0150] The input video frames are encoded step by step through the feature extraction backbone network combined with windowing and shift self-attention mechanism, and the deep features are fused with multi-scale context weighting to obtain multi-scale core feature maps from low-level texture to high-level semantics.

[0151] At the neck of the feature fusion, a feature pyramid structure based on upsampling and cross-layer stitching is adopted to perform cross-scale feature fusion and enhancement on the multi-scale core feature map to generate a multi-scale detection feature map, so as to improve the detection capability of small targets and boundary regions of the visual disk.

[0152] In the multi-scale detection head section, the visual disc region is predicted for each of the multi-scale detection feature maps, and the final effective visual disc detection target is output.

[0153] The Swin Transformer module consists of layer normalization, multi-head self-attention, and multi-layer perception.

[0154] The layer normalization is used to normalize each channel of the input features;

[0155] The multi-head self-attention is used to capture the inter-pixel correlation of the layer-normalized features within a local window, and to establish long-range dependencies across windows through a shift window mechanism.

[0156] In the multilayer perceptron, the output features of multi-head self-attention are nonlinearly transformed through two fully connected layers to enhance the semantic expressive power of the features.

[0157] A convolutional layer is connected in series after the Swin Transformer module to enhance the feature preservation capability of the viewport boundary region.

[0158] The specific calculation process in the Swin Transformer module is as follows:

[0159] The input feature map is divided into several non-overlapping local windows (i.e., patch partitions) according to a preset window size, so that self-attention calculation can be performed independently in each window;

[0160] Within each window, the window features are flattened into a feature sequence and then layer normalization is performed.

[0161] The normalized feature sequence is mapped to a query matrix, a key matrix, and a value matrix through three independent linear projection layers, respectively.

[0162] The correlation between pixels within the window is calculated based on the query matrix and the key matrix to obtain the attention weight matrix;

[0163] The attention weight matrix and the value matrix are weighted and summed to obtain the feature sequence after self-attention enhancement within the window;

[0164] Then, a shift window mechanism is used to periodically shift the window boundary of the current stage so that the windows of adjacent stages partially overlap with the original window.

[0165] Self-attention is calculated for the offset window so that pixels from different windows can enter the same offset window, thus enabling cross-window information interaction.

[0166] The features computed through cross-window attention are mapped back to the original channel dimension through a linear projection layer and then fused with the residuals by adding them element by element with the input features.

[0167] The residual fused features are mapped through two fully connected layers of a multilayer perceptron to enhance the nonlinear expressive power of the features.

[0168] The context aggregation module is used to extract the local structure and global semantics of the optic disc under different receptive fields, so as to enhance multi-scale perception and context-consistent expression of the optic disc region, specifically:

[0169] Receive multi-scale features of the input;

[0170] Upsampling is performed on the high-level small-scale features in the multi-scale features to align with the spatial resolution;

[0171] Perform 1×1 convolutions on the spatially aligned feature maps to align the channels, resulting in aligned multi-scale features.

[0172] The aligned multi-scale features are then subjected to global average pooling to obtain a global context vector, which is then input into a two-layer fully connected network to generate channel attention weights.

[0173] The channel attention weights are multiplied by the aligned multi-scale features using a channel dimension weighted multiplication to obtain spatially weighted features;

[0174] The spatially weighted features are input into several parallel convolutional branches to capture structural information from different receptive fields, thereby obtaining multi-scale convolutional features.

[0175] After weighted fusion of the multi-scale convolutional features and the spatially weighted features, layer normalization and 1×1 convolution are performed to obtain aggregated features;

[0176] The aggregated features are residually connected with the input multi-scale features to obtain context-enhanced features.

[0177] like Figure 3 As shown, the input frames are preprocessed to a uniform size of 640×640×3 and then fed into the CAST-YOLO backbone network for step-by-step feature encoding. The backbone network uses a depth scaling factor of 0.33 and a width scaling factor of 0.25, with a maximum number of channels of 1024.

[0178] Layer 0 uses a convolutional module (Conv, 3×3, stride=2, 64 channels), and the output feature map size is 320×320×64, which realizes initial downsampling and basic texture extraction;

[0179] The first layer uses a convolutional module (3×3, stride=2, 128 channels) to output 160×160×128, obtaining shallow edge and low-level semantic features;

[0180] The second layer consists of a C2f module repeated three times, maintaining a size of 160×160×128, used to enhance local feature representation and channel interaction;

[0181] The third layer uses convolution (3×3, stride=2, 256 channels) and outputs 80×80×256 as the initial scale of P3;

[0182] The fourth layer consists of a C2f(256) module repeated 6 times, maintaining 80×80×256, to extract mesoscale structure and optic disc edge information;

[0183] The 5th layer is SCDown (3×3, stride=2, 512 channels), with an output of 40×40×512, which serves as the P4 scale;

[0184] The 6th layer consists of a C2f(512) module repeated 6 times, maintaining a size of 40×40×512, which aggregates deep semantics and contextual information;

[0185] The 7th layer is SCDown (3×3, stride=2, 1024 channels), with an output of 20×20×1024, which serves as the P5 scale;

[0186] Layer 8 introduces, for example Figure 4 The Swin Transformer module shown strictly follows the evolutionary pattern of progressively shrinking feature map size and doubling channel dimensions. The input image H×W×3 is first patched and mapped to initial features using linear embedding. Then, it goes through four processing stages: In the first stage, the feature map is maintained at a resolution of H / 4×W / 4×C. Window-based multi-head self-attention is used to capture details and suppress computational redundancy in the local area. In the second and third stages, the feature map size is reduced to H / 8×W / 8 and H / 16×W / 16 respectively through patch merging operations, and the channels are expanded to 2C and 4C. A shift window mechanism is introduced to break the window boundary restrictions and achieve cross-region feature interaction. Finally, in the fourth stage, the feature map is further compressed to H / 32×W / 32×8C (i.e., the deep high-dimensional semantic features mentioned above). After highly abstract feature reconstruction in this space, the prediction result is output through global average pooling and a classifier.

[0187] Considering the shortcomings of pure attention structures in extracting local edge and texture features, this invention adds a 3×3 convolutional layer after the SwinTransformer module as a compensatory structure to enhance the expression of local details and spatial continuity, so that the output features have stronger hierarchical perception and directional consistency, providing semantically rich and structurally complete feature inputs for the subsequent SPPF and CAM modules.

[0188] The 9th layer consists of three repeated C2f (1024) modules, which refine and fuse the Swin output.

[0189] The 10th layer is SPPF (pooling kernel 5), which implements global context aggregation of multiple receptive fields on a 20×20×1024 scale;

[0190] The 11th layer introduces, for example... Figure 5 The context aggregation (CAM) module shown is used to achieve adaptive fusion of semantic context and cross-scale information integration in a deep feature space. This module aims to enhance the network's multi-scale perception ability and context-consistent expression of the visual disc region in complex microsurgical scenarios.

[0191] Specifically, the CAM module first performs dual spatial and channel normalization on the multi-scale features from the previous layer, and then extracts a global semantic description vector through global average pooling to characterize the global dependency between the view disk region and the background region. Subsequently, 1×1 convolutions are used to reduce the dimensionality of the features, compressing channel redundancy and reducing computational complexity. Based on contextual modeling, CAM internally sets up multi-scale convolutional branches with kernel sizes of 3×3, 5×5, and 7×7 in parallel to extract local structural and spatial texture features under different receptive fields. The outputs of each branch are weighted and fused to form aggregated features with multi-scale robustness, and further channel fusion and feature integration are achieved through 1×1 convolutions to achieve adaptive integration of local details and global semantics. Finally, the module superimposes the fused features with the input features through residual connections, which not only maintains the stability of gradient flow but also enhances the model's ability to represent boundaries and weakly textured regions.

[0192] After processing by the CAM module, the output features have stronger contextual consistency and multi-scale semantic aggregation capabilities, providing the detection head with high-resolution feature inputs that are structurally clear and semantically rich, thereby effectively improving the detection accuracy and robustness of the CAST-YOLO network for the optic disc region in complex surgical scenarios.

[0193] The 12th layer is a PSA (1024) module, which performs pixel-level attention enhancement on key regions and finally outputs globally enhanced features. .

[0194] This invention achieves multi-scale feature representation from low-level texture to high-level semantics through the stepwise stacking of Conv+C2f+SCDown, under the premise of controllable parameter quantity.

[0195] In the feature fusion stage, a feature pyramid structure based on upsampling and cross-layer stitching is adopted to fuse deep global features with mid-to-shallow detailed features at multiple scales, thereby improving the detection capability of small targets and boundary regions on the view disk. The feature fusion neck includes several upsampling modules, several Concat modules, several C2f modules, Conv convolution modules, SCDown modules, and C2fCIB modules.

[0196] The upsampling module is used to perform upsampling on the input features to achieve spatial scale alignment between deep and mid-level features;

[0197] The Concat module is used to perform cross-scale feature concatenation and fusion on the input features to obtain fused features;

[0198] The Conv convolution module is used to perform step-by-step downsampling and compression of the input features;

[0199] The C2f module is used for residual connections to enhance channel interaction and gradient flow of local features;

[0200] The SCDown module is used to combine convolutional downsampling and channel splitting fusion mechanisms to compress the feature map scale while maintaining the integrity of channel information.

[0201] The C2fCIB module combines the feature reorganization capability of C2f (multi-branch feature reorganization block) with the context enhancement capability of CIB (context information block) to strengthen the global semantics and contextual consistency of deep features and output the concatenated features.

[0202] 13th floor After performing a 2x upsampling, we obtained 40×40×1024.

[0203] Layer 14 concatenates the upsampled features with the 40×40×512 features output from Layer 6 of Backbone to form a fused feature of 40×40×1536.

[0204] The 15th layer compresses and reorganizes the channels through C2f (e.g., 512 channels) to obtain 40×40×512, forming the P4 fusion feature layer.

[0205] Layer 16 upsamples the P4 fusion features to 80×80×512.

[0206] The 17th layer is concatenated with the 80×80×256 feature output from the 4th layer of the backbone network to obtain an 80×80×768 feature.

[0207] The 18th layer is processed by C2f (e.g., 256 channels) to generate an 80×80×256 P3 fusion feature layer for small target detection.

[0208] Layer 19 performs a 3×3 convolution with stride=2 on the P3 fusion features and downsamples them back to 40×40×256.

[0209] The P4 fusion features of layer 20 and layer 15 are spliced ​​together to obtain 40×40×768, which is then output as 40×40×512 through layer 21 C2f (e.g., 512 channels) to form the final P4 detection feature layer.

[0210] Layer 22 performs SCDown on the P4 feature (e.g., 3×3, stride=2, 512 channels), outputting 20×20×512.

[0211] The SPPF output of layer 23 and layer 10 of Backbone is spliced ​​together to obtain 20×20×1024, resulting in 20×20×1536.

[0212] The 24th layer outputs 20×20×1024 through C2fCIB (e.g., 1024 channels), forming the final P5 detection feature layer. The core operation of the fusion process is shown in the following formula:

[0213] ;

[0214] in, This is to fuse features, which is the final P5 detection feature layer; For deep semantic features; These are the skip connection features corresponding to the scale; For upsampling operation, For channel splicing, Used to achieve cross-scale feature recombination and noise suppression.

[0215] In the multi-scale detection head section, a Detect module is included, which independently performs prediction on the feature map at each scale to generate a set of candidate bounding boxes for the visual disc at each scale. The candidate bounding boxes at all scales are then subjected to non-maximum suppression to remove highly overlapping and low-confidence redundant boxes, resulting in the final effective visual disc detection target.

[0216] This invention employs a three-scale joint detection strategy, utilizing feature maps at three scales, P3, P4, and P5, to predict candidate bounding boxes for small, medium, and large targets, respectively.

[0217] like Figure 3 As shown, the 80×80×256 output of the 18th layer is used as the P3 / 8-small detection feature map.

[0218] The 40×40×512 output from layer 21 is used as the P4 / 16-medium detection feature map.

[0219] The 20×20×1024 output from layer 24 is used as the P5 / 32-large detection feature map.

[0220] The Detect module at layer 25 receives the features from the three layers mentioned above and outputs multi-scale detection results.

[0221] For each scale, the detection head predicts the center coordinates (x, y), width and height (w, h), confidence level c, and class probability p of the bounding box to form a candidate set. All candidate boxes at all scales are processed by non-maximum suppression (NMS) to remove highly overlapping and low-confidence redundant boxes, resulting in the final set of effective detection results.

[0222] S4, the target detection of the visual disc is based on Kalman filter prediction and appearance feature matching. An adaptive gating matching mechanism is introduced to associate the target trajectory between frames. The target trajectory is maintained by combining the appearance feature cache update mechanism and ReID cosine similarity retrieval, and the trajectory tracking result is output. The adaptive gating matching mechanism is based on the gated strategy of Mahalanobis distance, and introduces detection confidence and velocity residual as dynamic adjustment factors. The trajectory tracking result includes continuous tracking results of visual disc position, confidence, trajectory identifier and timestamp.

[0223] This embodiment uses the Target Trajectory Tracking (A-DeepSORT) module for target trajectory tracking. It introduces an adaptive gating matching mechanism, an appearance feature sliding cache steady-state update mechanism, and an on-demand ReID retrieval mechanism on the basis of the traditional DeepSORT framework, which effectively enhances the trajectory continuity and tracking stability of the target in occlusion, reflection and dynamic deformation scenarios.

[0224] This step, based on the high-confidence optic disc candidate boxes output by the CAST-YOLO detection model, enables temporal tracking, trajectory maintenance, and occlusion recovery of the optic disc region in microsurgical videos. This stage employs the A-DeepSORT model of this invention, which, through structural reconstruction and strategy optimization based on the traditional DeepSORT framework, forms a closed-loop tracking mechanism of "prediction-matching-management-re-identification" to address challenges such as strong reflections, local occlusion, and rapid displacement in microsurgical scenarios.

[0225] Specifically, such as Figure 6 As shown, the current state prediction first uses a Kalman filter model to perform linear prediction based on the previous frame state of the target detected by the visual disc, generating the predicted trajectory of the target in the current frame, providing dynamic constraints for subsequent matching calculations. The previous frame state includes the target's center position, velocity, and scale. This prediction process maintains trajectory continuity even when the target is briefly occluded or missed.

[0226] The system receives candidate boxes output by the CAST-YOLO detection model, calculates the matching cost between the candidate boxes and the predicted trajectory based on the similarity of the candidate boxes' appearance features, the geometric distance of the target center, and the overlap of the boundaries, and generates three types of candidate results: "unmatched trajectory", "unmatched detection", and "matched trajectory".

[0227] The Intersection over Union (IoU) is calculated for candidate results of "unmatched trajectories" and "unmatched detections" to further filter out more reliable matching pairs and update the matching status (retaining high IoU trajectories as "matched trajectories" and marking low IoU trajectories as "unmatched trajectories"). If the number of consecutive matches for an "unmatched trajectory" is greater than the set matching threshold, the trajectory is determined to be "confirmed," triggering a Kalman filter update to correct the trajectory status based on the current detection results and ensure trajectory accuracy; otherwise, the trajectory is determined to be "unconfirmed," the filter is not updated temporarily, and candidate matches continue to be tracked.

[0228] For new target trajectories that do not match the detection, the process enters the "New Trajectory (Unconfirmed)" stage, initializes the tracking status of the new trajectory, and waits for subsequent frames to re-match. For unconfirmed trajectories, if deletion is required, the "Unconfirmed → Delete" process is executed; for confirmed trajectories, they are re-identified as needed.

[0229] Through the above cycle of "prediction-matching-update-management", the trajectory sequence of the target is continuously output.

[0230] When calculating the matching cost, the gating threshold is adaptively adjusted by combining the detection confidence and velocity residual, so that the matching range can be adaptively shrunk or widened according to the target motion state, thereby maintaining stable association under rapid displacement or low contrast conditions, achieving optimal association between candidate boxes and existing trajectories, and establishing the target trajectory.

[0231] The expression for calculating the matching cost is:

[0232] ;

[0233] ;

[0234] in, This represents the matching cost between the i-th trajectory and the j-th candidate box; , , These are the weighting coefficients; , , Let represent the appearance similarity between the i-th trajectory and the j-th candidate box, the geometric distance between the target center and the boundary overlap, respectively; Let i be the cache appearance features of the i-th temporarily lost trajectory; Let be the appearance feature of the j-th current candidate box.

[0235] The adaptive gating matching mechanism is based on a Mahalanobis distance-based gating strategy, and introduces the detection confidence level c and the velocity residual. As a dynamic adjustment factor, the formula for the gating threshold is:

[0236] ;

[0237] in, Represents the dynamic gating threshold for the i-th trajectory; The Mahalanobis distance; Let be the detection confidence score of the j-th candidate box; , These are the detection confidence level and the velocity residual weighting coefficient, respectively. The deviation between the predicted velocity of trajectory i and the actual velocity of candidate box j.

[0238] When the target moves smoothly, the threshold is automatically tightened to improve accuracy; when there is occlusion or visual jitter, the threshold is appropriately widened to prevent accidental deletion.

[0239] Lifecycle maintenance is performed on the established target trajectory: when the number of consecutive hits on the target trajectory reaches the set consecutive hit threshold (e.g., set to 3), the appearance features of the candidate box of the target trajectory are updated to suppress template drift caused by short-term reflection; when the number of consecutive misses on the target trajectory exceeds the set temporary loss threshold (e.g., set to 15), the target trajectory enters a temporary loss state; when the target trajectory exceeds the set maximum miss threshold (e.g., set to 30) and has not recovered, the tracking of the target trajectory is terminated to maintain the uniqueness of the trajectory number and the stability of the state.

[0240] The appearance template update employs a sliding cache and steady-state fusion strategy, performing exponential smooth updates only during the stable phase to improve appearance representation consistency. When updating the appearance features of candidate bounding boxes for the target trajectory, historical appearance features of the trajectory are stored in a sliding cache queue. And only when the number of consecutive frames hit by the trajectory exceeds a set threshold (e.g. Figure 6 of The update is performed when the time is right, and the formula is:

[0241] ;

[0242] in, The appearance features of the cache center after the current frame is updated; The cache length for historical appearance features; Let k be the appearance feature of the cached history.

[0243] When the target trajectory enters a temporary loss state and the number of consecutive missed detection frames reaches the set ReID threshold (e.g. Figure 6 of When set to 20), the system automatically enables the lightweight re-identification module, performing ReID cosine similarity retrieval between the target candidate box of the current frame and the cached appearance features.

[0244] The formula for ReID cosine similarity retrieval is:

[0245] ;

[0246] in, The cache appearance features of the i-th temporarily lost trajectory Compared with the appearance features of the j-th current candidate box Cosine similarity; represents the Euclidean norm, used to calculate the magnitude of a vector; T is the transpose symbol.

[0247] When the matching score of the cosine similarity retrieval result exceeds the set matching threshold (e.g., 0.7), the current target trajectory is restored to its valid state and re-enters the matching process; otherwise, the target trajectory is terminated.

[0248] This mechanism enables rapid identity recovery after prolonged occlusion, strong light reflection, or when the target temporarily leaves the field of vision, ensuring trajectory continuity.

[0249] Through the coordinated operation of the above steps, the A-DeepSORT model significantly improves the temporal consistency, appearance stability, and occlusion recovery capabilities of the optic disc target in complex dynamic scenes while maintaining real-time processing performance. The final output is the trajectory tracking results of the effective target in each video frame; these tracking results include the center coordinates of the optic disc, bounding box size, detection confidence, trajectory number, and timestamp for each frame, providing high-precision, low-latency data support for intraoperative navigation and postoperative analysis.

[0250] In a preferred embodiment, the method of the present invention was tested. The dataset used in the experiment came from real microsurgical fundus surgery videos from a provincial hospital, covering various ophthalmic diseases such as macular hole, epiretinal membrane, retinal detachment, and diabetic retinopathy. It involved microsurgical records from multiple different patients, demonstrating strong diversity and representativeness. The video data was acquired using a clinical microscope system at a frame rate of 30fps and a resolution of 1920×1080 pixels. After standardized format conversion and frame extraction, more than 3000 high-quality image samples were obtained, with a training set to validation set ratio of approximately 7:3. To prevent data leakage and ensure model generalization, surgical videos from the same patient were not simultaneously included in both the training and validation sets.

[0251] During data construction, each surgical video was first manually screened to remove invalid segments and excessively blurred frames. Then, based on the dynamic changes in the optic disc region during the surgical phases, an unequal-interval frame sampling strategy was adopted: during phases of drastic optic disc changes (such as traction, resection, liquefaction, etc.), the frame sampling density was increased to 3–5 fps to meticulously capture motion and occlusion processes; while during stable phases, it was reduced to 1 fps to control the number of samples and avoid redundancy. This strategy ensured that each frame was representative and effectively covered the spatial and brightness changes of the optic disc under different intraoperative conditions.

[0252] Each image was manually annotated by professionals with experience in medical image annotation, using bounding box annotation to accurately identify the location and extent of the optic disc region. The annotation process underwent two rounds of cross-checking to ensure consistency and accuracy, with the optic disc region positioning error controlled within 3 pixels.

[0253] The constructed dataset not only includes surgical scenes under normal lighting and with clear visibility, but also systematically covers various complex situations in microsurgery: surgical instrument occlusion, simulating scenarios where probes, forceps, or vitrectomy instruments obstruct the optic disc during actual operations; drastic lighting changes and specular reflections, reflecting high-brightness interference caused by intraoperative light source movement or angle changes; non-uniform light source distribution interference, manifested as local overexposure or vignetting; and tissue deformation and retinal folds, reflecting structural deformation and reflection changes caused by surgical operations. This highly diverse, noisy, and realistic sample distribution allows the proposed joint detection and tracking model to fully learn stable feature representations of the optic disc under different lighting, occlusion, and dynamic deformation conditions, significantly enhancing the model's robustness and generalization ability.

[0254] In comparative experiments, to comprehensively verify the performance of this invention in optic disc detection and continuous tracking in microsurgical videos, a multi-dimensional evaluation system was designed for both the detection and tracking phases. In the detection phase, five mainstream evaluation metrics were used: Precision (P), Recall (R), single-scale average precision (mAP@0.5), multi-scale average precision (mAP@0.5:0.95), and single-frame inference time (ms). P and R measure the model's accuracy and completeness in detection, respectively; mAP@0.5 reflects the model's overall detection performance at an IoU threshold of 0.5, while mAP@0.5:0.95 further examines the model's comprehensive performance at different IoU thresholds, demonstrating its robust detection capability for optic discs at different scales; single-frame inference time evaluates the model's real-time performance, with lower values ​​indicating faster inference speed. These metrics simultaneously measure the model's comprehensive performance in terms of detection accuracy, recall, and inference efficiency.

[0255] During the tracking phase, to objectively evaluate the performance of the CAST-YOLO+A-DeepSORT joint framework in continuous tracking tasks in the visual disk region, four typical metrics were used: Mean Intersection over Union (Mean IoU), Normalized Precision (Norm Prec), Area Under the Curve (AUC), and Overlap Precision 0.75 (OP@0.75). Mean IoU measures the degree of spatial overlap between the predicted and ground truth trajectories and is a core indicator for measuring temporal localization accuracy. NormPrec represents the average precision under normalized distance constraints, reflecting the overall tracking stability of the model. AUC evaluates the model's comprehensive robustness at different thresholds by calculating the area under the precision-overlap rate curve. OP@0.75 reflects the proportion of times the model can still correctly match at high overlap thresholds (IoU ≥ 0.75), measuring its adaptability to fine-grained tracking scenarios.

[0256] The experimental environment consisted of Ubuntu 24.04 LTS system, Python 3.9.20, and CUDA 12.5. All network architectures were trained on a single RTX 3090 GPU. During the training phase of the CAST-YOLO detection model, the AdamW optimizer was used with an initial learning rate of 1e-3, a batch size of 32, and a total of 300 training epochs.

[0257] The effectiveness of the CAST-YOLO detection model proposed in this invention for optic disc region detection in ophthalmic surgical videos has been fully validated. To comprehensively evaluate the detection performance of this invention, a systematic comparison was conducted with several mainstream object detection algorithms, including YOLOv3-tiny, YOLOv5n, YOLOv6n, and YOLOv8n detection models based on convolutional neural networks, YOLOv10n / m based on a hybrid structure, and the RT-DETR (ResNet50 / ResNet101) detection model based on the Transformer framework. All experiments were conducted on the same dataset and in the same hardware environment. The experimental results are shown in Table 1.

[0258] Table 1. Performance Comparison of CAST-YOLO (ours) with Mainstream Detection Models

[0259]

[0260] As shown in Table 1, the CAST-YOLO(ours) proposed in this invention achieves excellent performance across all detection metrics. In the visual disc detection task, CAST-YOLO achieves a precision (P) of 0.961 and a recall (R) of 0.955, representing improvements of 2.3% and 2.1% respectively compared to YOLOv10n, significantly reducing false positive and false negative rates. Regarding the key detection metric mAP@0.5, the detection model of this invention achieves 0.989, a 1.6% improvement compared to YOLOv10n's 0.973, indicating significant improvements in both target localization and classification accuracy. Furthermore, in terms of inference speed, CAST-YOLO's single-frame inference time is only 0.9ms, representing an approximately 18% improvement in inference efficiency compared to YOLOv10n, achieving a highly efficient balance between detection accuracy and real-time performance.

[0261] Compared to the RT-DETR series models, CAST-YOLO also demonstrates significant advantages in detection accuracy and inference speed. Compared to RT-DETR-ResNet50 and RT-DETR-ResNet101, CAST-YOLO improves recall by 1.0% and 1.2%, respectively, and mAP@0.5 by 1.2% and 1.1%, respectively, while inference speed is improved by approximately 9.8 times and nearly 10 times, respectively. These results fully demonstrate that this invention maintains lightweight design and computational controllability while still possessing superior detection accuracy and extremely high real-time response capabilities.

[0262] In summary, CAST-YOLO achieves overall optimization of detection accuracy, recall, and inference speed without increasing computational complexity. Based on the comprehensive evaluation metrics, this invention demonstrates stronger overall performance and application potential in ophthalmic surgical video optic disc detection tasks, providing high-precision, low-latency technical support for intraoperative visual navigation, real-time tracking, and intelligent assisted decision-making.

[0263] To verify the effectiveness and robustness of the proposed real-time continuous optic disc tracking method based on CAST-YOLO and A-DeepSORT in actual microsurgery scenarios, in another preferred embodiment, a comparative experiment was conducted on a self-built ophthalmic surgery video test dataset. The baseline for comparison was the YOLOv10 + DeepSORT joint framework, and the method of this invention is denoted as Ours (CAST-YOLO+A-DeepSORT).

[0264] The experiment selected four representative video sequences of microsurgical fundus surgery, denoted as test1.mp4 to test4.mp4. These four videos covered various typical intraoperative scenarios: test1 represented a relatively stable normal visual field; test2 included significant strong reflections and localized overexposure areas; test3 featured a low-contrast background and partial tissue occlusion; and test4 included frequent surgical instrument interference and partial visual field occlusion. Each video had a resolution of at least 1280×720 and a frame rate of 30fps. The actual optic disc trajectory was annotated frame-by-frame by experienced annotators to serve as the ground truth for evaluation. This configuration effectively covers the main changes in the optic disc target during ophthalmic surgery under conditions of illumination changes, specular reflection, instrument occlusion, and tissue deformation, demonstrating good representativeness and practical engineering significance. The experimental results are shown in Table 2.

[0265] Table 2. Performance Evaluation Results of CAST-YOLO + A-DeepSORT Algorithm

[0266]

[0267] As shown in Table 2, the method of the present invention improves all indicators of the four test videos to varying degrees compared with the baseline method, and the overall performance is more stable and robust. Specifically: In test1.mp4, the present invention demonstrates superior temporal consistency compared to the baseline even in scenarios without severe interference, with improvements in Mean IoU and Normalized Precision, indicating that the present invention can achieve more accurate trajectory fitting in basic scenarios. In test2.mp4, a highly reflective scene, the present invention's Mean IoU, Normalized Precision, and AUC are all significantly higher than the baseline, with OP@0.75 improving to near-full-range effective tracking levels. This proves that the high-confidence detection results provided by CAST-YOLO, combined with A-DeepSORT's adaptive gating and appearance steady-state update mechanism, can effectively suppress trajectory jitter and loss caused by sudden changes in illumination and specular reflection. In test3.mp4, a low-contrast and partially occluded scene, the baseline method exhibits significant tracking interruptions and accuracy degradation. However, the present invention, through on-demand ReID retrieval and trajectory temporary loss management mechanisms, effectively recovers occluded or temporarily missed visual disc targets, with improved Mean IoU and Normalized Precision. Prec is significantly improved; in the test4.mp4 scenario with frequent device intervention and local occlusion, the present invention consistently outperforms the baseline in MeanIoU, AUC and OP@0.75, demonstrating the ability to enhance identity preservation and trajectory continuity in complex dynamic scenarios.

[0268] The combined results from the four videos show that the proposed CAST-YOLO and A-DeepSORT joint framework achieves stable and quantifiable performance improvements in trajectory spatial accuracy, temporal continuity, high threshold overlap rate, and adaptability to scenes with strong reflection, occlusion, and low contrast, while maintaining real-time processing capabilities.

[0269] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0270] This invention embeds the Swing Transformer module and integrates the context aggregation (CAM) module into the YOLO backbone network, which enables the detection module to significantly improve the detection accuracy and boundary clarity of small-scale, weakly textured visual disk targets while maintaining real-time inference speed.

[0271] This invention introduces an adaptive gating matching mechanism, an appearance feature sliding cache and steady-state update mechanism, and an on-demand ReID retrieval mechanism into the DeepSORT trajectory tracking framework, enabling the tracking module to maintain the temporal continuity of the trajectory, identity consistency, and occlusion recovery capability in complex surgical scenarios such as strong reflection, partial occlusion, and low contrast.

[0272] The overall framework of this invention achieves high-precision, high-robustness, and high-continuity real-time tracking of the optic disc region without sacrificing real-time performance through deep collaboration between detection and tracking. It is suitable for applications such as ophthalmic surgical navigation, intraoperative monitoring, and postoperative intelligent analysis.

[0273] Example 2

[0274] like Figure 7 As shown, the second embodiment of the present invention also provides a real-time continuous tracking device for a video disc, comprising:

[0275] The video sequence acquisition unit is used to acquire video sequences of ophthalmic surgeries.

[0276] The preprocessing unit is used to preprocess the video sequence.

[0277] The disc target detection unit is used to perform multi-scale feature extraction and disc target detection on the preprocessed video sequence to obtain the disc detection target.

[0278] The trajectory tracking unit is used to perform inter-frame target trajectory association based on Kalman filter prediction and appearance feature matching of the visual disc detection target, and to maintain the target trajectory by combining the appearance feature cache update mechanism and ReID cosine similarity retrieval, and output the trajectory tracking result; wherein, the trajectory tracking result includes continuous tracking results of visual disc position, confidence, trajectory identifier and timestamp.

[0279] Example 3

[0280] The third embodiment of the present invention also provides a real-time continuous video disc tracking device, which includes a memory and a processor. The memory stores a computer program, which can be executed by the processor to realize the real-time continuous video disc tracking method described above.

[0281] Example 4

[0282] The fourth embodiment of the present invention also provides a computer-readable storage medium storing computer-readable instructions, which, when executed by the processor of the device where the computer-readable storage medium is located, implement the video disc real-time continuous tracking method as described above.

[0283] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for real-time continuous tracking of the optic disc, characterized in that, The method comprises the following steps: S1, acquiring a video sequence of an ophthalmic surgery; S2, preprocessing the video sequence; S3, performing multi-scale feature extraction and optic disc target detection on the preprocessed video sequence to obtain an optic disc detection target; S4, based on Kalman filter prediction and appearance feature matching, introducing an adaptive gating matching mechanism to associate inter-frame target trajectories, combining an appearance feature cache update mechanism and ReID cosine similarity retrieval to maintain the target trajectories, and outputting a trajectory tracking result; wherein the adaptive gating matching mechanism is based on a gating strategy based on Mahalanobis distance, and introduces detection confidence and velocity residual as dynamic adjustment factors; the trajectory tracking result includes continuous tracking results of optic disc position, confidence, trajectory identification and timestamp; When performing multi-scale feature extraction and optic disc target detection on the preprocessed video sequence, a CAST-YOLO detection model is used for processing; The CAST-YOLO detection model is embedded with a Swin Transformer module and integrates a context aggregation module in a YOLO backbone network, and is composed of a feature extraction backbone network, a feature fusion neck, and a multi-scale detection head; The feature extraction backbone network comprises a plurality of Conv convolution modules, a plurality of C2f modules, a plurality of SCDown modules, an SPPF module, and a PSA module, and introduces a Swin Transformer module and a context aggregation module; The Conv convolution module is used for step-by-step down-sampling and feature extraction of the input video frame; The C2f module is used for residual connection to enhance channel interaction and gradient flow of local features; The SCDown module is used to combine convolution down-sampling and channel splitting and fusion mechanisms to compress the feature map scale while maintaining the integrity of the channel information; The SPPF module is used to aggregate global context information through multi-scale pooling to enhance the adaptability of deep features to different size optic discs while maintaining computational efficiency; The PSA module is used to enhance features in key areas through a pixel-level attention mechanism, suppress background noise interference, and output globally enhanced deep features; The Swin Transformer module is used to efficiently model the global structure and long-range dependencies at the deep feature layer; The context aggregation module is used to extract the local structure and global semantics of the optic disc under different receptive fields to strengthen the multi-scale perception and context consistency expression of the optic disc region; The input video frame is encoded by the feature extraction backbone network combined with windowing and shift self-attention mechanism, and the deep features are multi-scale context weighted fused to obtain multi-scale core feature maps from low-level texture to high-level semantics; In the feature fusion neck, a feature pyramid structure based on up-sampling and cross-layer splicing is used to perform cross-scale feature fusion and enhancement on the multi-scale core feature maps to generate multi-scale detection feature maps, thereby improving the detection capability of small optic disc targets and boundary regions; In the multiscale detection head part, the multiscale detection feature maps are respectively subjected to optic disc region prediction, and finally effective optic disc detection targets are output.

2. The method of claim 1, wherein The preprocessing comprises pixel standardization, brightness enhancement, noise suppression, region cropping and data normalization. The pixel standardization is performed on the images extracted from the video sequence frame by frame through linear normalization, and the expression is as follows: ； wherein, represents the pixel value of the pixel-normalized image at coordinates ; represents the original video frame; and represent the minimum and maximum values of the pixel intensity in the current frame, respectively; The brightness enhancement is performed on the video frames after pixel standardization by using a contrast adjustment function based on adaptive histogram equalization, and the formula is as follows: ； wherein, is a pixel value of the image at coordinates after brightness enhancement; represents a contrast limit threshold value; represents a grid size for dividing the image into a plurality of local blocks; represents an adaptive histogram equalization algorithm with contrast limited; The noise suppression is performed by using a Gaussian filter for smoothing and noise reduction to suppress high-frequency reflection and instrument interference signals in the image, and the formula is as follows: ； wherein, represents a pixel value of the Gaussian filtered image at coordinates ; represents a standard deviation of the Gaussian kernel; k is a radius of a convolution window; represents a row offset within the convolution window; represents a column offset within the convolution window; The region cropping is performed based on the field center coordinates by setting a fixed region ROI for cropping and serving as an input range for subsequent detection, and the formula is as follows: ； wherein, is the image after region cropping; denotes the field of view center coordinates; , is the ROI width and height; The region cropped and enhanced image is uniformly scaled to a target size and subjected to batch tensorization processing to obtain a preprocessed video sequence.

3. The method of claim 1, wherein, The SwinTransformer module is composed of layer normalization, multi-head self-attention and a multi-layer perception mechanism. The layer normalization is used for normalizing each channel of the input feature. The multi-head self-attention is used for capturing the pixel correlation of the features after layer normalization within a local window and establishing long-range dependencies across windows through a shift window mechanism. In the multi-layer perception mechanism, the output features of the multi-head self-attention are subjected to nonlinear transformation through two fully connected layers to enhance the semantic expression ability of the features. A convolution layer is connected in series after the Swin Transformer module to strengthen the feature preservation ability of the optic disc boundary region. In the Swin Transformer module, the specific calculation process is as follows: The input feature map is divided into a plurality of non-overlapping local windows according to a preset window size, so as to independently perform self-attention calculation in each window. In each window, the window features are flattened into a feature sequence and then subjected to layer normalization. The normalized feature sequence is mapped into a query matrix, a key matrix and a value matrix through three independent linear projection layers. The correlation between pixels within the window is calculated based on the query matrix and the key matrix to obtain an attention weight matrix. The attention weight matrix and the value matrix are weighted and summed to obtain the feature sequence after self-attention enhancement within the window. Subsequently, based on the feature sequence after self-attention enhancement within the window, a shift window mechanism is used to periodically shift the window boundary of the current stage, so that the windows of adjacent stages partially overlap with the original window. Self-attention is calculated for the shifted window to enable pixels of different original windows to enter the same shifted window, thereby realizing cross-window information interaction. The features after cross-window attention calculation are mapped back to the original channel dimension through a linear projection layer and are subjected to element-wise residual fusion with the input features. The features after residual fusion are mapped through two fully connected layers of the multi-layer perception mechanism to enhance the nonlinear expression ability of the features, thereby obtaining deep enhanced features.

4. The method of claim 3, wherein, The context aggregation module is configured to extract local structures and global semantics of the optic disc under different receptive fields, so as to strengthen multi-scale perception and context consistency expression of the optic disc region, and specifically comprises the following steps: receiving input multi-scale features; performing up-sampling on high-layer small-scale features in the multi-scale features to align spatial resolution; performing 1*1 convolution on the spatially aligned feature maps respectively to perform channel alignment, so as to obtain aligned multi-scale features; performing global average pooling on the aligned multi-scale features to obtain a global context vector, and inputting the global context vector into two fully connected networks to generate channel attention weights; performing channel dimension weighting multiplication on the channel attention weights and the aligned multi-scale features, so as to obtain spatial weighted features; inputting the spatial weighted features into a plurality of parallel convolution branches respectively to capture structure information of different receptive fields, so as to obtain multi-scale convolution features; performing layer normalization and 1*1 convolution on the multi-scale convolution features and the spatial weighted features after weighted fusion, so as to obtain aggregated features; performing residual connection on the aggregated features and the input multi-scale features, so as to obtain context strengthened features.

5. The method of claim 1, wherein, The feature fusion neck comprises a plurality of up-sampling modules, a plurality of Concat modules, a plurality of C2f modules, a Conv convolution module, an SCDown module and a C2fCIB module; The up-sampling module is configured to perform up-sampling on input features to realize spatial scale alignment. The Concat module is configured to perform cross-scale splicing fusion on input features to obtain fused features. The Conv convolution module is configured to perform step-by-step down-sampling and compression scale on features. The C2f module is configured to perform residual connection to enhance channel interaction and gradient flow of local features. The SCDown module is configured to combine convolution down-sampling and channel split fusion mechanisms to compress feature map scale while maintaining the integrity of channel information. The C2fCIB module is configured to combine the feature reorganization capability of C2f and the context enhancement capability of CIB to strengthen global semantics and context consistency of deep features, and output spliced features.

6. The method of claim 1, wherein, The multi-scale detection head part comprises a Detect module configured to independently perform prediction on feature maps of each scale to generate a set of optic disc candidate boxes corresponding to each scale. All scale sets of optic disc candidate boxes are subjected to non-maximum suppression processing to remove redundant boxes with high overlap and low confidence, so as to obtain final effective optic disc detection targets.

7. The method of claim 6, wherein, The S4 specifically comprises the following steps: According to a previous frame state of the optic disc detection target, a predicted trajectory of the current frame target is generated through a Kalman filter model for linear prediction, wherein the previous frame state comprises a target center position, a speed and a scale of the previous frame; candidate boxes of the optic disc detection target are received, and a matching cost of the candidate boxes and the predicted trajectory is calculated through appearance feature similarity of the candidate boxes, target center geometric distance and boundary overlap degree, so as to establish a target trajectory. The life cycle of the established target trajectory is maintained: when the number of consecutive hits of the target trajectory reaches a set continuous hit threshold, the appearance feature of the candidate box of the target trajectory is updated to suppress template drift caused by short-time reflection; when the number of consecutive frames of missed hits of the target trajectory exceeds a set temporary loss threshold, the target trajectory enters a temporary loss state; when the target trajectory exceeds a set maximum miss threshold and has not recovered, the tracking of the target trajectory is terminated; When the target trajectory enters the temporary loss state and the number of consecutive missed detection frames reaches a set ReID threshold, the target candidate box of the current frame is subjected to ReID cosine similarity retrieval with the cached appearance feature; When the matching score of the cosine similarity retrieval result exceeds a set matching threshold, the effective state of the current target trajectory is recovered and the target trajectory participates in matching again; otherwise, the target trajectory is terminated; The tracking result of the valid target in each video frame is output; wherein the tracking result includes the center coordinates, the boundary box size, the detection confidence, the track number and the timestamp of each frame of the optic disc.

8. The method of claim 7, wherein, In the calculation of the matching cost, the detection confidence and the speed residual are combined to adaptively adjust the gating threshold, so as to realize the optimal association of the candidate box and the existing trajectory. The formula of the matching cost is: ； ； wherein, represents the matching cost of the ith track and the jth candidate box; , , is a weight coefficient; , , respectively represent the appearance similarity, the target center geometric distance and the boundary overlap degree of the ith track and the jth candidate box; is the cache appearance feature of the ith temporary track; is the jth current candidate box appearance feature; The formula of the gating threshold is: ； wherein, denotes the dynamic gating threshold of the i-th trajectory; is the Mahalanobis distance; is the detection confidence of the j-th candidate box; , are the detection confidence and the velocity residual weight coefficient, respectively; is the deviation between the predicted velocity of the trajectory i and the actual velocity of the candidate box j. In the updating of the appearance feature of the candidate box of the target trajectory, the historical appearance features of the trajectory are stored based on a sliding cache queue, and the updating is only performed when the number of consecutive hit frames of the trajectory exceeds a set threshold, and the formula is: ； wherein, a cached center appearance feature updated for the current frame; a cache length for the historical appearance features; a kth cached historical appearance feature; The formula of the ReID cosine similarity retrieval is: ； wherein, cache appearance feature of the ith temporary trajectory cosine similarity with the jth current candidate box appearance feature denotes the Euclidean norm for calculating the length of a vector; T is the transpose symbol.​ 9. An apparatus for optic disc real-time continuous tracking, for implementing the method of any one of claims 1-8, characterized in that, It comprises: A video sequence acquisition unit configured to acquire a video sequence of an ophthalmic surgery; A preprocessing unit configured to preprocess the video sequence; An optic disc target detection unit configured to perform multi-scale feature extraction and optic disc target detection on the preprocessed video sequence to obtain an optic disc detection target; A trajectory tracking unit configured to perform Kalman filter prediction and appearance feature matching based on the optic disc detection target, introduce an adaptive gating matching mechanism to associate the target trajectory between frames, and combine an appearance feature cache updating mechanism and ReID cosine similarity retrieval to maintain the target trajectory, and output a trajectory tracking result; wherein the trajectory tracking result comprises continuous tracking results of the position, confidence, track identification and timestamp of the optic disc.

Images