Detection method for small sample strabismus detection based on fusion network of meta learning and image processing

Abstract

The fusion network based on meta learning and image processing is used for few-sample squint detection, and the problem that performance of traditional deep learning is limited under data scarcity is solved. Iris relative offset ratio (IRDR) features are extracted through an image processing algorithm, and precise classification under the condition of few samples is realized in combination with a meta-learning framework. The method comprises the specific steps of obtaining and preprocessing eye image data, and constructing a network model combining image processing and metric learning. The image processing module carries out gray scale and HSV conversion through a double-branch process, and carries out light reflection point detection and pupil area fitting circle calculation to obtain IRDR features. And the IRDR features are combined with meta-learning results, so that the squint classification accuracy is improved. According to the method, the accuracy rate of 86.2% is achieved in a 5-way 1-shot task, the detection performance under the condition of few samples is remarkably improved, and support is provided for clinical strabismus screening.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of computer vision and medical image analysis, specifically a detection method for strabismus detection based on a fusion network of meta-learning and image processing for small sample strabismus detection, which is suitable for automatic screening and classification of strabismus in scenarios where medical data is scarce. Background Technology

[0002] Strabismus is a common eye disease characterized by the inability of both eyes to work in parallel coordination when focusing on a target. This not only affects visual function but can also lead to serious consequences such as amblyopia and loss of stereoscopic vision, with a particularly high incidence in children. Traditional strabismus diagnosis relies on clinical examinations by ophthalmologists (such as the cover test, prism test, and Hershberg test), but these methods are highly dependent on physician experience, subjective, and difficult to implement in areas with limited primary healthcare resources, easily leading to many missed or misdiagnosed cases. With the development of computer vision and artificial intelligence technologies, automated strabismus screening methods have become a research hotspot. Early methods were based on image processing techniques, calculating eye alignment parameters by extracting features such as iris position and corneal reflector (CLR), but these methods are sensitive to image quality and have limited generalization ability. In recent years, deep learning methods have been widely used in medical image analysis due to their powerful feature learning capabilities. Some studies have attempted to use convolutional neural networks (CNNs) to achieve automatic strabismus classification, but these methods usually require a large amount of labeled data for training. However, the acquisition and labeling of strabismus images are limited by medical ethics, patient privacy, and professional physician resources, resulting in a significant data scarcity problem that severely restricts model performance and practicality.

[0003] Few-shot learning techniques offer a new approach to addressing the problem of data scarcity. Meta-learning, through its "learning how to learn" mechanism, enables models to quickly adapt to new tasks from a limited number of samples, making it particularly suitable for scenarios where data acquisition is difficult, such as medical imaging. Prototype networks, a classic method of meta-learning, achieve few-shot classification by constructing category prototypes (supporting the mean features of the samples). However, existing prototype network-based methods still suffer from inaccurate feature extraction when processing strabismus images: traditional convolutional operations struggle to capture subtle geometric deformations of the eye and lack targeted attention to key medical features, resulting in limited classification accuracy. Therefore, there is an urgent need for a technical solution that can accurately extract key features from strabismus images under few-shot conditions and improve classification performance to meet the demand for efficient, accurate, and automated detection methods in clinical strabismus screening. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] This invention aims to overcome the performance limitations of traditional deep learning methods in strabismus detection due to data scarcity, as well as the inaccurate feature extraction of existing meta-learning methods. It provides a detection method for strabismus detection based on a fusion network of meta-learning and image processing for small sample strabismus detection. Through innovative network structure design, it achieves efficient and accurate classification of strabismus types under small sample conditions.

[0006] (II) Technical Solution

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A detection method for few-sample strabismus detection based on a fusion network of meta-learning and image processing includes the following steps:

[0009] Step 1: Obtain the original dataset of eye images, preprocess and divide the dataset to obtain the dataset for meta-training and the dataset for meta-testing;

[0010] Step 2: Construct an image processing module, the model including a two-branch image processing flow and the construction of IRDR features;

[0011] Step 3: Combine the image processing module with metric-based meta-learning, and use the dataset used for meta-training in step (1) to train the entire network model to obtain the trained model;

[0012] Step 4: Input the dataset used for meta-testing in step (1) into the trained network model and output the small sample strabismus detection results;

[0013] The image processing module in step (2) includes a dual-branch image processing flow; the image after cropping the eye region is converted to grayscale and HSV respectively, the grayscale image is binarized and other operations are performed to obtain the light reflection point detection result, and the pupil region fitting circle is calculated to obtain the center of the pupil region fitting circle of the HSV image.

[0014] The Iris Relative Displacement Ratio (IRDR) feature is obtained from an image processing module containing the detection results of light reflection points and the fitted center of the pupil region. In a normal situation, the light reflection points in both eyes coincide with the pupil center. If the corresponding positions of the light reflection points in the iris differ significantly between the two eyes, it indicates strabismus. For strabismus, we establish a polar coordinate system by using the light reflection point in the eye on the side with the different positions as the origin of the polar coordinates, and the ray from that point to the light reflection point on the other side as the polar axis. The polar coordinates of the pupil center on the origin side are denoted as... The iris relative offset ratio (IRDR) characteristic is defined by the following formula:

[0015]

[0016] in This represents the distance (in pixels) between the pupil center and the corneal light reflector (CLR). This distance directly reflects the degree of eye deviation.

[0017] This denominator represents the approximate diameter of the iris (in pixels). It is introduced to normalize the displacement, eliminating scale effects caused by factors such as shooting distance and individual differences in eye size, making IRDR a comparable metric relatively independent of image resolution.

[0018] The metric-based meta-learning method divides the embedding vectors obtained from the embedding network into support set features and query set features. A class prototype matrix is ​​constructed using the support set features, and the query set features are then combined to classify strabismus types. The input consists of eye images of various categories, including esotropia, exotropia, hypertropia, hypotropia, and normal eye. These input images first enter the embedding network, which extracts features to generate embedding features. These embedding features are then divided into support set features and query set features. The support set features are fed into the prototype computation layer to construct the class prototype matrix. The query set features are input into the classification headers and combined with the class prototype matrix to obtain predicted logits. Meanwhile, the query set ground truth labels, after label smoothing, are used together with the predicted logarithms to calculate the cross-entropy loss. Finally, the parameters of the entire network are optimized through backpropagation to improve the performance of few-sample strabismus detection.

[0019] The image processing module described in step (3) is combined with metric-based meta-learning. First, the high-dimensional deep feature vector output by the meta-learning branch is processed. Principal component analysis is used for dimensionality reduction. During model training and inference, the deep features of each sample are reduced. Using the PCA transformation matrix learned offline Projecting it into a lower-dimensional space yields the dimensionality-reduced features. Obtain the depth features after dimensionality reduction. IRDR features extracted from image processing branch Then, a splicing operation is used to merge them:

[0020]

[0021] Further, according to the method of claim 1, the original dataset of eye images in step (1) includes the mini-ImageNet dataset, the EyeTracking dataset, and the STRABISMUS dataset; the preprocessing includes extracting the eye region using the Dlib facial feature detection library, cropping the eye image containing the eyelids, sclera, and iris based on the 68-point facial feature localization system, and expanding the cropping range to retain periocular features; in the dataset partitioning, the mini-ImageNet dataset and the EyeTracking dataset are used as the meta-training dataset, and the STRABISMUS dataset is used as the meta-test dataset, and samples from the same patient in the meta-test dataset do not appear simultaneously in the training set and the validation set.

[0022] Further, according to the method of claim 1, the few-sample strabismus detection task in step (4) is a 5-way 1-shot task (5 categories, 1 supporting sample for each category), and the detection results are evaluated by accuracy, sensitivity and specificity.

[0023] (III) Beneficial Effects

[0024] This invention offers several significant advantages: Regarding small-sample detection capabilities, it leverages a meta-learning framework based on metric learning, constructs a class prototype matrix using support sets, and combines query set features to achieve efficient classification, thus addressing the problem of scarce strabismus medical data samples. (5-way) The system maintains detection accuracy even in 1-shot tasks, reducing reliance on large-scale labeled data. Regarding feature fusion to improve diagnostic accuracy, high-dimensional deep features from the meta-learning branch are dimensionality-reduced using PCA and then fused with IRDR features from the image processing branch. This preserves deep semantic information of the image while incorporating key physiological indicators of iris shift, avoiding the limitations of single features and improving the differentiation of strabismus types. In terms of data adaptability and robustness, the multi-source dataset is reasonably divided, with independent meta-training and meta-testing data covering diverse scenarios. The preprocessing stage accurately extracts the eye region and retains key features. The normalization design of the IRDR features eliminates interference from differences in shooting distance and eye size, adapting to images acquired under different conditions. In terms of clinical practical value, it covers multiple strabismus types and normal eye classifications. The detection results are evaluated through multiple dimensions, including accuracy, meeting clinical diagnostic needs and providing efficient and reliable technical support for strabismus screening in areas with scarce medical resources. Attached Figure Description

[0025] Figure 1 This is a simplified flowchart of the present invention;

[0026] Figure 2 This is a schematic diagram of the network structure of the present invention;

[0027] Figure 3 This is a schematic diagram illustrating the establishment of the polar coordinate system in this invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] Example

[0030] like Figure 1 As shown in the figure, this embodiment provides a detection method for few-sample strabismus detection based on a fusion network of meta-learning and image processing, which includes the following steps:

[0031] Step 1: Obtain the original dataset of eye images, preprocess and divide the dataset to obtain the dataset for meta-training and the dataset for meta-testing;

[0032] Step 2: Construct an image processing module, the model including a two-branch image processing flow and the construction of IRDR features;

[0033] Step 3: Combine the image processing module with metric-based meta-learning, and use the dataset used for meta-training in step (1) to train the entire network model to obtain the trained model;

[0034] Step 4: Input the dataset used for meta-testing in step (1) into the trained network model and output the small sample strabismus detection results;

[0035] The image processing module in step (2) includes a dual-branch image processing flow; the image after cropping the eye region is converted to grayscale and HSV respectively, the grayscale image is binarized and other operations are performed to obtain the light reflection point detection result, and the pupil region fitting circle is calculated to obtain the center of the pupil region fitting circle of the HSV image.

[0036] The Iris Relative Displacement Ratio (IRDR) feature is obtained from an image processing module containing the detection results of light reflection points and the fitted center of the pupil region. In a normal situation, the light reflection points in both eyes coincide with the pupil center. If the corresponding positions of the light reflection points in the iris differ significantly between the two eyes, it indicates strabismus. For strabismus, we establish a polar coordinate system by using the light reflection point in the eye on the side with the different positions as the origin of the polar coordinates, and the ray from that point to the light reflection point on the other side as the polar axis. The polar coordinates of the pupil center on the origin side are denoted as... The iris relative offset ratio (IRDR) characteristic is defined by the following formula:

[0037]

[0038] in This represents the distance (in pixels) between the pupil center and the corneal light reflector (CLR). This distance directly reflects the degree of eye deviation.

[0039] This denominator represents the approximate diameter of the iris (in pixels). It is introduced to normalize the displacement, eliminating scale effects caused by factors such as shooting distance and individual differences in eye size, making IRDR a comparable metric relatively independent of image resolution.

[0040] The metric-based meta-learning method divides the embedding vectors obtained from the embedding network into support set features and query set features. A class prototype matrix is ​​constructed using the support set features, and the query set features are then combined to classify strabismus types. The input consists of eye images of various categories, including esotropia, exotropia, hypertropia, hypotropia, and normal eye. These input images first enter the embedding network, which extracts features to generate embedding features. These embedding features are then divided into support set features and query set features. The support set features are fed into the prototype computation layer to construct the class prototype matrix. The query set features are input into the classification headers and combined with the class prototype matrix to obtain predicted logits. Meanwhile, the query set ground truth labels, after label smoothing, are used together with the predicted logarithms to calculate the cross-entropy loss. Finally, the parameters of the entire network are optimized through backpropagation to improve the performance of few-sample strabismus detection.

[0041] The image processing module described in step (3) is combined with metric-based meta-learning. First, the high-dimensional deep feature vector output by the meta-learning branch is processed. Principal component analysis is used for dimensionality reduction. During model training and inference, the deep features of each sample are reduced. Using the PCA transformation matrix learned offline Projecting it into a lower-dimensional space yields the dimensionality-reduced features. Obtain the depth features after dimensionality reduction. IRDR features extracted from image processing branch Then, a splicing operation is used to merge them:

[0042]

[0043] 2. The method according to claim 1, wherein the original dataset of eye images in step (1) includes the mini-ImageNet dataset, the EyeTracking dataset, and the STRABISMUS dataset; the preprocessing includes extracting the eye region using the Dlib facial feature detection library, cropping the eye image containing the eyelids, sclera, and iris based on the 68-point facial feature localization system, and expanding the cropping range to retain periorbital features; in the dataset partitioning, the mini-ImageNet dataset and the EyeTracking dataset are used as the meta-training dataset, the STRABISMUS dataset is used as the meta-test dataset, and samples from the same patient in the meta-test dataset do not appear simultaneously in the training set and the validation set.

[0044] In this embodiment, data augmentation refers to performing operations such as rotation and mirroring on the original images in the STRABISMUS dataset to achieve data augmentation.

[0045] The method in this embodiment is compared with that of traditional embedding networks for detecting small sample strabismus datasets. The image size input to the network is size=128 pixels × 512 pixels; 5 classes are labeled, namely upslope, downslope, leftslope, rightslope, and normal, with corresponding class labels of 0, 1, 2, 3, and 4, and named HYPERTROPIA, HYPOTROPIA, EXOTROPIA, ESOTROPIA, and NORMAL. The ratio of training set to validation set is 4:1; the network settings parameters are shown in Table 1.

[0046] Table 1. Unified parameter settings

[0047] Image size Training rounds Batch size 128×512 60 2

[0048] The model was implemented using PyTorch 3.8.20, with an Intel(R) Xeon(R) CPU and an NVIDIA GeForce RTX 3090 Ti GPU. During training, a stochastic gradient descent (SGD) optimizer was used to jointly optimize the parameters of the embedding network and the classifier head. The initial learning rate was set to 0.001, the momentum parameter to 0.9, and the weight decay coefficient to 5e-4. Nesterov was enabled to accelerate gradient updates and improve convergence efficiency. The learning rate scheduling strategy employed cosine annealing, with the maximum iteration period set to the total number of training epochs and the minimum learning rate decayed to 0.00001. Periodic dynamic adjustment of the learning rate prevented the model from getting trapped in local optima, enhancing the stability and convergence of the training process. The total training epochs were 60, the batch size was 2, and the label smoothing coefficient ε = 0.1. Accuracy, sensitivity, and specificity were used to evaluate the network model's detection results. The ablation experiment results are shown in Table 2.

[0049] Table 2 Evaluation Table of Network Model Detection Results

[0050] Module Accuracy (%) Sensitivity (%) Specificity (%) Meta-learning networks based on metric learning (base) 60.2 58.5 61.9 base+ image processing module 71.3 69.2 73.4 base+IRDR features 79.5 77.1 81.9 Complete model 86.2 84.3 88.1

[0051] The results show that the network model in this embodiment improves the accuracy by 26%, the sensitivity by 25.8%, and the specificity by 26.2% compared to the basic model, proving that the present application has improved the detection accuracy, sensitivity, and specificity of strabismus detection in small samples.

[0052] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for few-shot strabismus detection based on a meta-learning and image processing fusion network, characterized in that, The method comprises the following steps: Step 1: obtaining an eye image original data set, pre-processing and dividing the data set to obtain a data set for meta-training and a data set for meta-testing; Step 2: constructing an image processing module, wherein the model comprises a double-branch image processing process and construction of an IRDR feature; Step 3: combining the image processing module and meta-learning based on metric learning, training the entire network model by using the data set for meta-training in step (1), and obtaining a trained model; Step 4: inputting the data set for meta-testing in step (1) into the trained network model to output a small sample strabismus detection result; In step (2), the image processing module comprises a double-branch image processing process; The image after the eye region is cropped is subjected to grayscale conversion and HSV conversion, and the grayscale image is subjected to binarization and other operations to obtain a light reflection point detection result, and the pupil region of the HSV image is fitted to obtain a pupil region fitting center; The iris relative displacement ratio (IRDR) feature is an image containing light reflection point detection results and pupil region fitting circle center results obtained by the received image processing module. In the image, the coincidence of the light reflection points and the pupil centers on both sides of the eye is a normal condition, and if the corresponding positions of the light reflection points on both sides of the eye are different in the iris, it is a strabismus condition. For the strabismus condition, we take the light reflection point on the side with the different corresponding positions as the origin of the polar coordinate, and take the ray from the light reflection point to the light reflection point on the other side as the polar axis to establish a polar coordinate system. The polar coordinates of the pupil center on the origin side are marked as The iris relative displacement ratio (IRDR) feature is defined by the following formula: wherein represents the distance (in pixels) between the Pupil Center and the Corneal Light Reflection point (CLR). This distance directly reflects the degree of eye misalignment; : represents the approximate diameter of the iris (in pixels). The purpose of introducing this denominator is to normalize the displacement to eliminate the scale effect caused by the shooting distance, individual eye size difference and other factors, so that IRDR becomes a comparable measurement index relative to the image resolution; The meta-learning based on metric learning divides the embedding vector obtained by the embedding network into support set features and query set features, constructs a class prototype matrix through the support set features, and completes strabismus type classification in combination with the query set features. The input part is an eye image of each category, covering esotropia (ESOTROPIA), exotropia (EXOTROPIA), hypertropia (HYPERTROPIA), hypotropia (HYPOTROPIA) and normal eye (NORMAL) and the like. These input images first enter the embedding network, and the embedding network extracts features from the input images to generate embedding features (Embedding Feature). Then, the embedding features are divided into support set features (Support Set Feature) and query set features (Query Set Feature). The support set features are transmitted to the prototype computation layer (Prototype Computation Layer) to construct a class prototype matrix (Class Prototype Matrix). The query set features are input into the classification head (Classification_heads) to obtain predicted logits (Predicted Logits) in combination with the class prototype matrix. Meanwhile, the real labels of the query set (QuerySet Groud Truth Labels) are processed by label smoothing, and are used together with the predicted logits to calculate the cross-entropy loss (Cross-Entropy Loss). Finally, the entire network is optimized through the back propagation mechanism to improve the performance of the small sample strabismus detection. The image processing module described in step (3) is combined with metric-based meta-learning. First, the high-dimensional deep feature vector output by the meta-learning branch is processed. Principal component analysis is used for dimensionality reduction. During model training and inference, the deep features of each sample are reduced. Using the PCA transformation matrix learned offline Projecting it into a lower-dimensional space yields the dimensionality-reduced features. Obtain the depth features after dimensionality reduction. IRDR features extracted from image processing branch Then, a splicing operation is used to merge them:

2. The method of claim 1, wherein, The eye image original data set in step (1) includes a mini-ImageNet data set, an EyeTracking data set and a STRABISMUS data set; the preprocessing includes extracting an eye region by using a Dlib face feature detection library, cutting an eye image containing eyelids, sclera and iris based on a 68-point face feature positioning system, and expanding the cutting range to retain eye-related features; In the data set division, the mini-ImageNet data set and the EyeTracking data set are used as meta-training data sets, and the STRABISMUS data set is used as a meta-test data set, and samples of the same patient in the meta-test data set do not appear in the training set and the validation set at the same time.

3. The method of claim 1, wherein, In step (4), the few-shot strabismus detection task is a 5-way 1-shot task (5 categories, 1 support sample per category), and the detection result is evaluated by accuracy, sensitivity and specificity.

Images