Graph neural network refined strabismus diagnosis method, system and equipment based on causal feature selection and medium

Abstract

The invention discloses a causal feature selection-based graph neural network refined strabismus diagnosis method, system and device and a medium, and the method comprises the steps: obtaining nine face photos of a patient, and carrying out the image preprocessing of the nine face photos, so as to construct a nine-eye bitmap; detecting an orbit part of each eye position in the nine-eye-position map by adopting a target detection algorithm so as to extract a characteristic variable related to strabismus refined diagnosis in each eye position; a causal feature selection algorithm is adopted to screen out key feature variables having a direct causal relationship with strabismus refined diagnosis from the feature variables related to each eye position; and inputting the key feature variable of each eye position into a graph neural convolutional network model, and carrying out training optimization in a strabismus disease data set through a propagation formula to output a refined diagnosis result. According to the method, irrelevant details on the whole face image are removed to extract main eye areas to construct the nine-eye bitmap, so that the calculation cost is reduced; the key feature variable of each eye position is extracted by adopting a causal feature algorithm, and the key feature variable of each eye position is learned through the graph neural convolutional network model, so that the graph neural convolutional network model is more interpretable, and refined strabismus diagnosis is realized.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, system, device and medium for fine-tuning strabismus diagnosis based on a graph neural network for causal feature selection. Background Art

[0002] Strabismus is a common eye misalignment disorder characterized by incorrect eye position when focusing on objects. This condition is particularly prevalent in children and, if left untreated, can progress to severe binocular vision impairment. In addition to the direct impact on vision, strabismus can have a profound psychological impact on those affected. Fortunately, surgical intervention can effectively treat strabismus. However, the symptoms of strabismus vary, and the type of surgery required depends on an accurate diagnosis. Current diagnostic methods for strabismus rely heavily on expert eye examinations, such as the Cover test and the Hirschberg test. These examinations require specialized techniques and equipment, which are usually only available in larger hospitals or specialty clinics.

[0003] With the development of digital image technology, more and more researchers use the eye position images of strabismus patients to diagnose strabismus. For example: AHT algorithm, CCLRR algorithm, HOCORC algorithm, U-Net iris segmentation algorithm and other strabismus classification algorithms. Although the above model algorithms have a certain accuracy, they also have some shortcomings: First, for the strabismus diagnosis task, since the incidence of various complex diseases of strabismus is much lower than that of common diseases, this makes it more difficult to collect complex diseases, making it impossible for the existing strabismus detection model to make a refined diagnosis of strabismus. Most of the work only achieves strabismus detection, that is, judging whether there is strabismus; second, due to the difficulty of clinical data collection, most of the existing strabismus detection models are only trained and tested on small sample data sets, and cannot achieve good generalization performance; third, the existing deep learning model belongs to the "black box model" and has the problem of poor interpretability. Even the mainstream deep learning interpretability method, such as heat map, is difficult to feedback effective diagnostic basis to doctors. Summary of the invention

[0004] Based on this, the purpose of the present invention is to provide a graph neural network based on causal feature selection for refined strabismus diagnosis method, system, equipment and medium. The present invention combines expert priors and medical consensus to model causal knowledge, and combines causal knowledge with graph neural networks to make the graph neural network model more interpretable, thereby achieving refined strabismus diagnosis.

[0005] In a first aspect, the present invention provides a graph neural network refined strabismus diagnosis method based on causal feature selection, comprising the following steps: Step S11, obtaining nine facial photos of the patient, and performing image preprocessing on the nine facial photos respectively to construct a nine-eye bitmap, wherein the nine facial photos are photos of the patient gazing in specific directions, and the specific directions include upper left, directly above, upper right, directly left, directly in front, directly right, lower left, directly below, and lower right; Step S12, using a target detection algorithm to detect the orbital part of each eye position in the nine eye position maps, so as to extract characteristic variables related to the refined diagnosis of strabismus in each eye position; Step S13, using a causal feature selection algorithm to screen out key feature variables that have a direct causal relationship with the refined diagnosis of strabismus from the feature variables related to each eye position, the key feature variables including the X coordinate of the center point of the left eye iris, the X coordinate of the left eye reflection point, the X coordinate of the center point of the right eye iris, the X coordinate of the right eye reflection point, the upper eye position iris center distance, the middle eye position iris center distance and the lower eye position iris center distance; Step S14, input the key feature variables of each eye position into the graph neural convolutional network model, and perform training optimization in the strabismus disease dataset through the propagation formula to output a refined diagnosis result, wherein the propagation formula Z is expressed as: ; In the formula, and is a nonlinear activation function, is the node connection matrix, X is the input feature matrix, is the weight matrix of the first layer of the graph neural convolutional network, is the weight matrix of the second layer of the graph neural convolutional network.

[0006] Preferably, in step S11, the step of performing image preprocessing on the nine facial photos respectively includes: Step S111, using the dlib library to detect facial landmarks of each facial image, and calculating the horizontal distance between the two eyes based on the facial landmarks and vertical distance ; Step S112, define the horizontal distance centered at the midpoint between the two eyes and vertical distance The rectangle with a side length is the main eye area of ​​each facial image, and a nine-eye bitmap is constructed with nine main eye areas, where the length of the main eye area is × , the width is × , relaxation coefficient =1.2, relaxation coefficient =1.4.

[0007] Preferably, in step S11, the nine facial photos are obtained by taking photos of the patient while he / she is gazing in a specific direction; Alternatively, a video recording device is used to collect video of the patient. During the video recording process, the patient's eyeballs turn in sequence according to the directions of the prompt arrows in the video recording device; Nine facial photos are obtained from the collected videos according to the video frames.

[0008] Preferably, in step S12, the relevant feature variables include position information of left and right irises, position information of points around left and right eye frames, position information of left and right reflection points, angular deviation of the upper and lower positions of both eyes, the Euclidean distance between the centers of both irises, the Euclidean distance between the positions of the reflection points of both eyes, position information of left and right eye sockets, proportion of iris area in the left and right eye sockets, angle of the left and right iris centers relative to the center of the eye sockets, angle difference of the iris centers in the same row relative to the center of the eye sockets, and angle difference of the left and right iris centers relative to the center of the eye sockets.

[0009] Preferably, in step S14, the construction of the strabismus disease dataset for training optimization includes the following steps: Acquire nine facial photos of a plurality of patients, and perform image preprocessing on the nine facial photos of each patient to construct a corresponding nine-eye bitmap; Senior ophthalmologists make fine-grained diagnosis on the nine-eye bitmaps of each patient, and use the fine-grained diagnosis of each patient as the true annotation of the nine-eye bitmap of the corresponding patient to construct a strabismus disease dataset; The strabismus dataset was divided into training set, validation set and test set with a split ratio of 6:3:1.

[0010] Preferably, the step S14 specifically includes: Step S141, designing and encoding nine network nodes of a graph neural convolutional network model by imitating nine gaze directions of the nine-eye bitmap; Step S142, inputting the key feature variables of each eye position into the network node at the corresponding position, extracting, interactively comparing and combining the feature variables of all its neighbors through each key feature variable, so as to learn and obtain the intraocular detail features of each eye position and capture the inter-eye variation features in different gaze directions; Step S143, training optimization is performed in the strabismus disease dataset by propagating the formula to output a refined diagnosis result.

[0011] In a second aspect, the present invention provides a graph neural network refined strabismus diagnosis system based on causal feature selection, comprising: An image preprocessing module is used to obtain nine facial photos of the patient and perform image preprocessing on the nine facial photos respectively to construct a nine-eye bitmap, wherein the nine facial photos are photos of the patient gazing in specific directions, and the specific directions include upper left, directly above, upper right, directly left, directly in front, directly right, lower left, directly below and lower right; A feature variable extraction module is used to detect the orbital part of each eye position in the nine-eye position map using a target detection algorithm to extract feature variables related to the refined diagnosis of strabismus in each eye position; A feature variable screening module is used to screen out key feature variables with a direct causal relationship with the refined diagnosis of strabismus from the feature variables related to each eye position by using a causal feature selection algorithm, wherein the key feature variables include the X coordinate of the center point of the left eye iris, the X coordinate of the left eye reflection point, the X coordinate of the center point of the right eye iris, the X coordinate of the right eye reflection point, the upper eye position iris center distance, the middle eye position iris center distance and the lower eye position iris center distance; The training optimization module is used to input the key feature variables of each eye position into the graph neural convolutional network model, and perform training optimization in the strabismus disease dataset through the propagation formula to output a refined diagnosis result, wherein the propagation formula Z is expressed as: ; In the formula, and is a nonlinear activation function, is the node connection matrix, X is the input feature matrix, is the weight matrix of the first layer of the graph neural convolutional network, is the weight matrix of the second layer of the graph neural convolutional network.

[0012] Preferably, the training optimization module includes: A network node design module, used to design and encode nine network nodes of a graph neural convolutional network model by imitating the nine gaze directions of the nine-eye bitmap; A feature learning and capturing module is used to input the key feature variables of each eye position into the network node at the corresponding position, and extract, interactively compare and combine the feature variables of all its neighbors through each key feature variable, so as to learn the intraocular detail features of each eye position and capture the inter-eye variation features in different gaze directions; The training optimization submodule is used to perform training optimization in the strabismus disease dataset through the propagation formula to output a refined diagnosis result.

[0013] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the graph neural network refined strabismus diagnosis method based on causal feature selection as described in the first aspect.

[0014] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the graph neural network refined strabismus diagnosis method based on causal feature selection as described in the first aspect.

[0015] Compared with the prior art, the present invention has the following beneficial effects: First, the nine-eye bitmap is constructed by removing irrelevant details on the entire facial image to extract the main eye area, thereby reducing the computational cost; Second, a causal feature algorithm is used to extract the key feature variables of each eye position, and the key feature variables of each eye position are learned through a graph neural convolutional network model, making the graph neural convolutional network model more interpretable, thereby achieving refined strabismus diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the process of the graph neural network refined strabismus diagnosis method based on causal feature selection in the present invention; Figure 2 for Figure 1 A schematic diagram of the process of the refined strabismus diagnosis method; Figure 3 The causal graph of extroversion and vertical strabismus obtained based on the causal feature selection algorithm HCM; Figure 4 The causal graph of AV sign-related features obtained based on the causal feature selection algorithm HCM; Figure 5 for Figure 2 Schematic diagram of the construction method of the nine-eye bitmap; Figure 6 It is a distribution diagram of several typical nine-eye bitmaps in the present invention and different fine-grained strabismus categories in the data set; Figure 7 A schematic diagram of the composition of the graph neural network refined strabismus diagnosis system based on causal feature selection in the present invention; Figure 8 The heat map corresponding to the nine eye bitmaps of a patient; Fig. 9 In order to refine the strabismus diagnosis method of the present invention Figure 8 The heat map in the diagnosis results. DETAILED DESCRIPTION

[0017] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art may think of other obvious variations.

[0018] See also Figure 1 and Figure 2 , an embodiment of the present invention provides a graph neural network refined strabismus diagnosis method based on causal feature selection, comprising the following steps: Step S11, obtaining nine facial photos of the patient, and performing image preprocessing on the nine facial photos respectively to construct a nine-eye bitmap, wherein the nine facial photos are photos of the patient gazing in specific directions, and the specific directions include upper left, directly above, upper right, directly left, directly in front, directly right, lower left, directly below, and lower right; Step S12, using a target detection algorithm to detect the orbital part of each eye position in the nine eye position maps, so as to extract characteristic variables related to the refined diagnosis of strabismus in each eye position; Step S13, using a causal feature selection algorithm to screen out key feature variables that have a direct causal relationship with the refined diagnosis of strabismus from the feature variables related to each eye position, the key feature variables including the X coordinate of the center point of the left eye iris, the X coordinate of the left eye reflection point, the X coordinate of the center point of the right eye iris, the X coordinate of the right eye reflection point, the upper eye position iris center distance, the middle eye position iris center distance and the lower eye position iris center distance; Step S14, input the key feature variables of each eye position into the graph neural convolutional network model, and perform training optimization in the strabismus disease dataset through the propagation formula to output a refined diagnosis result, wherein the propagation formula Z is expressed as: ; In the formula, and is a nonlinear activation function, is the node connection matrix, X is the input feature matrix, is the weight matrix of the first layer of the graph neural convolutional network, is the weight matrix of the second layer of the graph neural convolutional network.

[0019] It should be noted that for complex disease recognition, complex disease recognition is divided into two stages according to pathological characteristics. The first stage detects whether it is esotropia, exotropia or vertical strabismus, and the second stage detects whether it is A sign, V sign or non-AV sign. Due to the presence of a lot of interference or mixed information in the image, the traditional image recognition model mistakenly learns invalid features, resulting in poor prediction performance. Therefore, it is hoped that clear key features can be extracted from complex image information so that the model can only focus on learning effective information in strabismus diagnosis.

[0020] In step S12, the nine-eye bitmap is subjected to the current mainstream open source target detection algorithm YOLO-X to extract a series of characteristic variables that may be related to the identification of complex diseases, such as the iris and eye sockets of the child that the doctor will pay attention to. The relevant characteristic variables include the position information of the left and right irises, the position information of the points around the left and right eye frames, the position information of the left and right reflection points, the angle deviation of the upper and lower positions of both eyes, the Euclidean distance of the centers of the irises of both eyes, the Euclidean distance of the positions of the reflection points of both eyes, the position information of the left and right eye sockets, the proportion of the iris area in the left and right eye sockets, the angle of the left and right iris centers relative to the center of the eye sockets, the angle difference of the iris centers in the same row relative to the center of the eye sockets, and the angle difference of the left and right iris centers relative to the center of the eye sockets. The relevant characteristic variables are shown in Table 1 below: Table 1 Characteristic variables related to the refined diagnosis of strabismus In step S13, the present invention uses the causal feature selection algorithm HCM to screen the extracted strabismus related features. The causal feature selection algorithm HCM is a mature and open source causal feature selection algorithm, and its algorithm framework can be divided into three parts: Causal skeleton learning: Combine the PC-Stable algorithm and the mixed-type randomized independence test (MRCIT) to determine an edge-less causal skeleton graph.

[0021] Based on the causal skeleton diagram obtained in the first stage, the greedy search strategy and the Cross-Validation based Mixed Information Criterion (CVMIC) are combined to determine the direction of the upper edge of the skeleton diagram and obtain a causal diagram.

[0022] Based on the causal graph obtained in the second stage, a mixed random independence test (MRCIT) is performed to delete some causal edges with low credibility, and finally a more complete causal graph is obtained.

[0023] The present invention uses the causal feature selection algorithm HCM to perform feature screening on strabismus-related variables to obtain two causal graphs, in which the direct parent node of the target variable Y ( Figure 3 , Figure 4 The red nodes in the figure are the causal features of strabismus diagnosis, that is, only these seven variables are needed to complete the refined diagnosis of strabismus in the present invention, and the key feature variables include the X coordinate of the center point of the left eye iris, the X coordinate of the left eye reflected light point, the X coordinate of the center point of the right eye iris, the X coordinate of the right eye reflected light point, the upper eye iris center distance, the middle eye iris center distance and the lower eye iris center distance.

[0024] See also Figure 3, where Y is the disease category label, which is divided into: esotropia, exotropia, and vertical strabismus; X1 is the X-coordinate of the center point of the left eye iris; X3 is the X-coordinate of the center point of the right eye iris; X5 is the X-coordinate of the left eye reflection point; and X7 is the X-coordinate of the right eye reflection point.

[0025] See also Figure 4 , where Y is the disease category label, which is divided into: A sign, V sign, non-AV sign; X11 is the difference in the distance between the center points of the left and right irises in the upper eye position and the lower eye position in the left direction; X12 is the difference in the distance between the center points of the left and right irises in the lower eye position in the center direction; X13 is the difference in the distance between the center points of the left and right irises in the lower eye position in the right direction.

[0026] In step S14, based on the above-mentioned screening of key characteristic variables of strabismus symptoms through causal features, they are constructed as nodes of the graph neural network. Since the graph neural network can utilize the characteristics of the graph structure, the present invention embeds the extracted key characteristic variables into the node design of the graph neural network, that is, the input data only needs these 7 key variables to realize the refined identification of strabismus on the graph neural network. At the same time, we can reverse the identification results to the specific node information, provide the basis for network identification, and improve the interpretability of the model.

[0027] See also Figure 5 In a preferred embodiment of the present invention, in step S11, the steps of performing image preprocessing on the nine facial photos respectively include: Step S111, using the dlib library to detect facial landmarks of each facial image, and calculating the horizontal distance between the two eyes based on the facial landmarks and vertical distance ; Step S112, define the horizontal distance centered at the midpoint between the two eyes and vertical distance The rectangle with a side length is the main eye area of ​​each facial image, and a nine-eye bitmap is constructed with nine main eye areas, where the length of the main eye area is × , the width is × , relaxation coefficient =1.2, relaxation coefficient =1.4.

[0028] It should be noted that in facial photos, the area occupied by the eyes is relatively small. Applying the recognition model directly on the entire facial image may introduce irrelevant details, such as hairstyle, nose, and mouth, which may confuse the diagnosis process and unnecessarily increase the computational cost. To solve this problem, image preprocessing is performed to extract the main eye area. Furthermore, by setting the relaxation coefficient and relaxation coefficient To adjust the corresponding horizontal distance and vertical distance .

[0029] In a preferred embodiment of the present invention, in step S11, nine facial photos are obtained by taking photos of the patient while he or she is gazing in a specific direction.

[0030] Specifically, enter the diagnostic software (such as the mini program in the mobile phone) interface, click the diagnostic button, and the shooting interface will pop up. Then place the face in the portrait frame, and then follow the arrow directions (upper left, directly above, upper right, directly left, directly right, lower left, directly below, and lower right), and click the red button to take photos in sequence to obtain nine facial photos.

[0031] In another preferred embodiment of the present invention, a video recording device is used to collect video of the patient. During the video collection process, the patient's eyeballs turn in sequence according to the direction of the prompt arrow in the video recording device; Nine facial photos are obtained from the collected videos according to the video frames.

[0032] Specifically, enter the diagnostic software (such as a small program in a mobile phone) interface, click the diagnostic button, and the recording interface pops up. Then place the face in the portrait frame and click the red button to record in sequence. During the recording process, 8 arrows will appear to indicate the direction (upper left, directly above, upper right, directly left, directly right, lower left, directly below, and lower right). Turn your eyes to the direction of the arrows and then complete the recording. After the video is collected, the mediapipe face recognition library will be called to extract the eye area based on the image taken from the video frame to obtain nine facial photos.

[0033] In a preferred embodiment of the present invention, in step S14, the construction of the strabismus disease dataset for training optimization includes the following steps: Acquire nine facial photos of a plurality of patients, and perform image preprocessing on the nine facial photos of each patient to construct a corresponding nine-eye bitmap; Senior ophthalmologists make fine-grained diagnosis on the nine-eye bitmaps of each patient, and use the fine-grained diagnosis of each patient as the true annotation of the nine-eye bitmap of the corresponding patient to construct a strabismus disease dataset; The strabismus dataset was divided into training set, validation set and test set with a split ratio of 6:3:1.

[0034] See also Figure 5 and Figure 6It should be noted that existing strabismus recognition datasets are usually affected by limited sample size and coarse-grained annotations. In the present invention, the applicant meets the needs of fine-grained strabismus recognition by compiling a large-scale dataset containing 1,075 strabismus patients from Xinhua Hospital Affiliated to Shanghai Jiao Tong University School of Medicine. Each patient contributed nine facial photos, which were taken when they were staring in a specific direction: upper left, up, upper right, left, front, right, lower left, bottom, and lower right. The resolution of each photo is 2592×1944 pixels. Subsequently, these images are processed in the image preprocessing stage to construct nine eye bitmaps (such as Figure 5 Each diagram was resized to 2400 × 900 pixels. In addition, each patient received a fine-grained diagnosis from a senior ophthalmologist, which served as the ground truth annotation for the dataset. For clarity and illustration, Figure 6 Several typical nine-eye images are shown, and the distribution of different fine-grained squint categories in our dataset is illustrated. We divide the dataset into training, validation, and testing subsets using a 6:3:1 split ratio.

[0035] Please refer to Figure 2 In a preferred embodiment of the present invention, the step S14 specifically includes: Step S141, designing and encoding nine network nodes of a graph neural convolutional network model by imitating nine gaze directions of the nine-eye bitmap; Step S142, inputting the key feature variables of each eye position into the network node at the corresponding position, extracting, interactively comparing and combining the feature variables of all its neighbors through each key feature variable, so as to learn and obtain the intraocular detail features of each eye position and capture the inter-eye variation features in different gaze directions; Step S143, training optimization is performed in the strabismus disease dataset by propagating the formula to output a refined diagnosis result.

[0036] See also Figure 7 The present invention provides a graph neural network refined strabismus diagnosis system based on causal feature selection, comprising: An image preprocessing module is used to obtain nine facial photos of the patient and perform image preprocessing on the nine facial photos respectively to construct a nine-eye bitmap, wherein the nine facial photos are photos of the patient gazing in specific directions, and the specific directions include upper left, directly above, upper right, directly left, directly in front, directly right, lower left, directly below and lower right; A feature variable extraction module is used to detect the orbital part of each eye position in the nine-eye position map using a target detection algorithm to extract feature variables related to the refined diagnosis of strabismus in each eye position; A feature variable screening module is used to screen out key feature variables with a direct causal relationship with the refined diagnosis of strabismus from the feature variables related to each eye position by using a causal feature selection algorithm, wherein the key feature variables include the X coordinate of the center point of the left eye iris, the X coordinate of the left eye reflection point, the X coordinate of the center point of the right eye iris, the X coordinate of the right eye reflection point, the upper eye position iris center distance, the middle eye position iris center distance and the lower eye position iris center distance; The training optimization module is used to input the key feature variables of each eye position into the graph neural convolutional network model, and perform training optimization in the strabismus disease dataset through the propagation formula to output a refined diagnosis result, wherein the propagation formula Z is expressed as: ; In the formula, and is a nonlinear activation function, is the node connection matrix, X is the input feature matrix, is the weight matrix of the first layer of the graph neural convolutional network, is the weight matrix of the second layer of the graph neural convolutional network.

[0037] Preferably, the training optimization module includes: A network node design module, used to design and encode nine network nodes of a graph neural convolutional network model by imitating the nine gaze directions of the nine-eye bitmap; A feature learning and capturing module is used to input the key feature variables of each eye position into the network node at the corresponding position, and extract, interactively compare and combine the feature variables of all its neighbors through each key feature variable, so as to learn the intraocular detail features of each eye position and capture the inter-eye variation features in different gaze directions; The training optimization submodule is used to perform training optimization in the strabismus disease dataset through the propagation formula to output a refined diagnosis result.

[0038] The contents of the above method embodiments are all applicable to the present system embodiments. The functions implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are the same as those of the above method embodiments, which will not be repeated here.

[0039] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, a graph neural network refined strabismus diagnosis method based on causal feature selection in the above embodiment is implemented. Wherein, the computer-readable storage medium includes but is not limited to any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card or optical card. In other words, the storage device includes any medium that stores or transmits information in a readable form by a device (such as a computer, a mobile phone, etc.), which can be a read-only memory, a disk or an optical disk, etc.

[0040] As an embodiment, the computer device includes a processor, a memory, and a computer program, wherein the computer program is stored in the memory and configured to be executed by the processor, and the computer program is configured to execute a graph neural network refined strabismus diagnosis method based on causal feature selection in the above-mentioned embodiment.

[0041] The present invention is described in detail below with specific embodiments: In this paper, the proposed strabismus diagnosis method is implemented using the PyTorch library, which is a popular choice for deep learning research. The computing platform used in the experiment includes an Intel Xeon Gold 5115CPU running at 2.40 GHz, supplemented by an NVIDIA GeForce RTX 2080Ti GPU and 32GB RAM. In the training phase, we use the adaptive moment estimation (Adam) optimizer with a learning rate set to 0.01 and a weight decay of 0.001. Each iteration processes 32 samples to effectively update the model parameters.

[0042] Consistent with standard practice for classification tasks, the evaluation metrics in the present invention include overall accuracy, directional accuracy, and angular accuracy. In addition, in order to comprehensively evaluate the performance of the model of the present invention in different recognition targets (i.e., directional strabismus recognition and angular change recognition), the corresponding accuracy of each task was calculated simultaneously. After evaluating the test subset, TP (True Positive), FP (False Positive), TN (True Negative), and FN (False Negative) were calculated based on the provided annotations, which enable the present invention to quantify the effectiveness of the evaluation model in correctly identifying strabismus types and angular changes.

[0043] 1) Overall accuracy: In actual diagnosis scenarios, accuracy metrics play a vital role in evaluating model performance, where correct diagnosis requires accurate identification of each fine-grained symptom. Mathematically, the overall accuracy Acc is defined as: 2) Directional accuracy: Directional accuracy is a key indicator for evaluating the performance of the model in predicting directional squint. Mathematically, the directional accuracy Acc Dir Defined as: Where TP Dir , FP Dir TN Dir and FN Dir Directional accuracy for true positives, false positives, true negatives, and false negatives for directional squint samples, respectively.

[0044] 3) Angle accuracy: The angle accuracy Acc of the squint angle change classification in this invention Deg Defined as: Where TP Deg , FP Deg TN Deg and FN Deg Angular accuracy of true positive, false positive, true negative, and false negative for classifying samples with respect to angular variation, respectively.

[0045] The applicant compared the method of the present invention with the most advanced end-to-end classification models, including VGG [Verydeepconvolutional networks for large-scale image recognition, ultra-deep convolutional network model for large-scale image recognition], DenseNet [Densely connected convolutional networks, densely connected convolutional network model], ViT [An image is worth 16x16 words: Transformers forimage recognition at scale, transformer model for large-scale image recognition], and Swin ViT [Swintransformer: Hierarchical vision transformer using shifted windows, hierarchical visual transformer model using shifted windows] and other models for fine-grained squint recognition. To ensure a fair comparison, a hierarchical classifier was uniformly integrated into each model during the comparison process. Key components such as data partitioning, data enhancement, and training settings remained consistent in all methods.

[0046] Table 2 Performance comparison between the proposed model and mainstream classification models The experimental results are shown in Table 2. Several observations can be drawn from these results. First, compared with other mainstream classification models, the method of the present invention shows superior performance, which verifies its effectiveness. Second, by using causal feature selection to screen out key variables and exclude the influence of other confounding variables, the method of the present invention has made substantial improvements over other mainstream classification models, which emphasizes the effectiveness of the method of the present invention.

[0047] The graph neural convolutional network model based on causal feature selection proposed in the present invention significantly reduces the computational cost. In order to verify its effectiveness compared with the benchmark VGG model and ViT model, the present invention conducted a comparative analysis, as shown in Table 3. The results show that the computational cost of the method of the present invention is only 1‰ of that of other models.

[0048] Table 3 Comparison of computational costs of different models See also Figure 8 and Fig. 9 ,from Figure 8 It is difficult to obtain accurate diagnostic results from heat maps. Fig. 9It can be seen that after the patient's nine eye positions were analyzed by the method of the present invention, there were problems with the iris center point and the light reflection point of eye position 5, and the diagnosis result was: exotropia. Therefore, the present invention refines the strabismus diagnosis method based on the graph neural network with causal feature selection, making the graph neural convolutional network model more interpretable.

[0049] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions only describe the principles of the present invention. The present invention may be subject to various changes and improvements without departing from the spirit and scope of the present invention. These changes and improvements fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the attached claims and their equivalents.

Claims

1. A graph neural network refined strabismus diagnosis method based on causal feature selection, characterized in that: The following steps are involved: Step S11, obtaining nine facial photos of the patient, and performing image preprocessing on the nine facial photos respectively to construct a nine-eye bitmap, wherein the nine facial photos are photos of the patient gazing in specific directions, and the specific directions include upper left, directly above, upper right, directly left, directly in front, directly right, lower left, directly below, and lower right; Step S12, using a target detection algorithm to detect the orbital part of each eye position in the nine eye position maps, so as to extract characteristic variables related to the refined diagnosis of strabismus in each eye position; Step S13, using a causal feature selection algorithm to screen out key feature variables that have a direct causal relationship with the refined diagnosis of strabismus from the feature variables related to each eye position, the key feature variables including the X coordinate of the center point of the left eye iris, the X coordinate of the left eye reflection point, the X coordinate of the center point of the right eye iris, the X coordinate of the right eye reflection point, the upper eye position iris center distance, the middle eye position iris center distance and the lower eye position iris center distance; Step S14, input the key feature variables of each eye position into the graph neural convolutional network model, and perform training optimization in the strabismus disease dataset through the propagation formula to output a refined diagnosis result, wherein the propagation formula Z is expressed as: ； In the formula, and is a nonlinear activation function, is the node connection matrix, X is the input feature matrix, is the weight matrix of the first layer of the graph neural convolutional network, is the weight matrix of the second layer of the graph neural convolutional network.

2. According to claim 1, a graph neural network refined strabismus diagnosis method based on causal feature selection is characterized in that: In step S11, the steps of performing image preprocessing on the nine facial photos respectively include: Step S111, using the dlib library to detect the facial landmarks of each facial image, and calculate the horizontal distance between the two eyes based on the facial landmarks and vertical distance ; Step S112, define the horizontal distance centered at the midpoint between the two eyes and vertical distance The rectangle with a side length is the main eye area of ​​each facial image, and a nine-eye bitmap is constructed with nine main eye areas, where the length of the main eye area is × , the width is × , relaxation coefficient =1.2, relaxation coefficient =1.

4.

3. The method for diagnosing strabismus using a graph neural network based on causal feature selection according to claim 1, characterized in that: In step S11, nine facial photos are obtained by taking photos of the patient's gaze in a specific direction respectively; Alternatively, a video recording device is used to collect video of the patient. During the video recording process, the patient's eyeballs turn in sequence according to the directions of the prompt arrows in the video recording device; Nine facial photos are obtained from the collected videos according to the video frames.

4. The method for diagnosing strabismus using a graph neural network based on causal feature selection according to claim 1, characterized in that: In step S12, the relevant feature variables include the position information of the left and right irises, the position information of the points around the left and right eye frames, the position information of the left and right reflection points, the angular deviation of the upper and lower positions of both eyes, the Euclidean distance between the centers of the irises of both eyes, the Euclidean distance between the positions of the reflection points of both eyes, the position information of the left and right eye sockets, the proportion of the iris area in the left and right eye sockets, the angle of the left and right iris centers relative to the center of the eye sockets, the angle difference of the iris centers in the same row relative to the center of the eye sockets, and the angle difference of the left and right iris centers relative to the center of the eye sockets.

5. The method for diagnosing strabismus using a graph neural network based on causal feature selection according to claim 1, characterized in that: In step S14, the construction of the strabismus disease dataset for training optimization includes the following steps: Acquire nine facial photos of a plurality of patients, and perform image preprocessing on the nine facial photos of each patient to construct a corresponding nine-eye bitmap; Senior ophthalmologists make fine-grained diagnosis on the nine-eye bitmaps of each patient, and use the fine-grained diagnosis of each patient as the true annotation of the nine-eye bitmap of the corresponding patient to construct a strabismus disease dataset; The strabismus dataset was divided into training set, validation set and test set with a split ratio of 6:3:

1.

6. The method for diagnosing strabismus using a graph neural network based on causal feature selection according to claim 1, characterized in that: The step S14 specifically includes: Step S141, designing and encoding nine network nodes of a graph neural convolutional network model by imitating nine gaze directions of the nine-eye bitmap; Step S142, inputting the key feature variables of each eye position into the network node at the corresponding position, extracting, interactively comparing and combining the feature variables of all its neighbors through each key feature variable, so as to learn and obtain the intraocular detail features of each eye position and capture the inter-eye variation features in different gaze directions; Step S143, training optimization is performed in the strabismus disease dataset by propagating the formula to output a refined diagnosis result.

7. A graph neural network refined strabismus diagnosis system based on causal feature selection, characterized in that: include: An image preprocessing module is used to obtain nine facial photos of the patient and perform image preprocessing on the nine facial photos respectively to construct a nine-eye bitmap, wherein the nine facial photos are photos of the patient gazing in specific directions, and the specific directions include upper left, directly above, upper right, directly left, directly in front, directly right, lower left, directly below and lower right; A feature variable extraction module is used to detect the orbital part of each eye position in the nine-eye position map using a target detection algorithm to extract feature variables related to the refined diagnosis of strabismus in each eye position; A feature variable screening module is used to screen out key feature variables with a direct causal relationship with the refined diagnosis of strabismus from the feature variables related to each eye position by using a causal feature selection algorithm, wherein the key feature variables include the X coordinate of the center point of the left eye iris, the X coordinate of the left eye reflection point, the X coordinate of the center point of the right eye iris, the X coordinate of the right eye reflection point, the upper eye position iris center distance, the middle eye position iris center distance and the lower eye position iris center distance; The training optimization module is used to input the key feature variables of each eye position into the graph neural convolutional network model, and perform training optimization in the strabismus disease dataset through the propagation formula to output a refined diagnosis result, wherein the propagation formula Z is expressed as: ； In the formula, and is a nonlinear activation function, is the node connection matrix, X is the input feature matrix, is the weight matrix of the first layer of the graph neural convolutional network, is the weight matrix of the second layer of the graph neural convolutional network.

8. The graph neural network refined strabismus diagnosis system based on causal feature selection according to claim 7 is characterized in that: The training optimization module includes: A network node design module, used to design and encode nine network nodes of a graph neural convolutional network model by imitating the nine gaze directions of the nine-eye bitmap; A feature learning and capturing module is used to input the key feature variables of each eye position into the network node at the corresponding position, and extract, interactively compare and combine the feature variables of all its neighbors through each key feature variable, so as to learn the intraocular detail features of each eye position and capture the inter-eye variation features in different gaze directions; The training optimization submodule is used to perform training optimization in the strabismus disease dataset through the propagation formula to output a refined diagnosis result.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: The processor executes the computer program to implement the graph neural network refined strabismus diagnosis method based on causal feature selection as described in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the graph neural network refined strabismus diagnosis method based on causal feature selection as described in any one of claims 1 to 6.

Images