A method and system for locating the pupil center of an optometer

Abstract

The application relates to the technical field of image processing, and discloses a method and system for locating a pupil center of an optometry instrument. The method comprises the following steps: S1: collecting pupil image information on the optometry instrument, and marking a preset area on the pupil image information, so as to take the preset area as a feature area for learning of a detection network; S2: marking an intersection of center lines of each point on the preset area as a target center point for learning of the detection network; S3: taking the marked pupil image information as a data set, dividing the data set into a training set and a test set according to a preset proportion, and training and testing the marked pupil image information through a face detection neural network; and S4: predicting a prediction center point and a prediction area position of the preset area through a target detection network. The method directly predicts an output result through supervised learning of a neural network, so that the position of the pupil center point is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a method and system for locating the pupil center of an optometry instrument. BACKGROUND

[0002] At present, the target detection technology in the field of deep learning has been developing rapidly and is widely used in urban transportation, medical treatment and other fields.

[0003] Optometry optometry can be used for checking the soft corneal contact lens fitting, which does not need to dilate the pupils and can quickly measure the refractive power. However, the accuracy of the optometry instrument is affected by many factors, such as the user's eye rotation causing the eye to fixate on the target in the optometry instrument, which will inevitably affect the accuracy of the refractive power test results, and even cause a large difference in the degree of repeated tests. Therefore, real-time positioning of the user's eye center position is crucial for obtaining accurate refractive power test results. SUMMARY

[0004] In view of the above status of the prior art, the technical problem to be solved by the present application is to provide a method and system for locating the pupil center of an optometry instrument, in order to assist the pupil center point detection device in automatically locating the pupil center point under different lighting environments to calculate the refractive power of the human eye. The output results are directly predicted through supervised learning of the neural network, so as to obtain the position of the pupil center point.

[0005] The present application provides a method and system for locating the pupil center of an optometry instrument, which specifically comprises the following technical solutions:

[0006] A method for locating the pupil center of an optometry instrument, comprising the steps of:

[0007] S1: Collecting pupil image information on the optometry instrument, and labeling a preset area on the pupil image information, the preset area being used as a feature area for learning of the detection network;

[0008] S2: Labeling the intersection of the center lines of each point in the preset area as a target center point for learning of the detection network;

[0009] S3: Labeling the pupil image information as a data set, and dividing the data set into a training set and a test set according to a preset ratio, and training and testing the labeled pupil image information through a face detection neural network;

[0010] S4: Predicting the predicted center point and the predicted area position of the preset area through the target detection network;

[0011] S5: Performing cross-entropy loss function on the predicted area position through the target detection network, and performing Euclidean distance judgment on the predicted center point position predicted in step S4 to predict the error between the predicted center point and the target center point;

[0012] S6: The weight parameters in the object detection network are updated in reverse by using the chain rule, so that the object detection network can continuously learn image features and output more accurate center point and region positions.

[0013] Specifically, this is to assist the pupil center point detection device in automatically locating the pupil center point under different lighting conditions in order to calculate the refractive power of the human eyeball.

[0014] Specifically, firstly, pupil images with eight light spots are captured by an optometry device. For each image, the circumscribed square region of the eight light spots is labeled as the feature region to be learned by the object detection network. The intersection of the center lines of the eight circular light spots is labeled as the learning target of the detection network. The labeled dataset is divided with a training set to a test set ratio of 8:2. The object detection network is constructed, and a face detection neural network or a general object detection network is used for training and testing on the labeled images. For the circumscribed square predicted by the detection network, the intersection of the two diagonals is calculated as the predicted center point. The detection network uses the cross-entropy loss function for the circumscribed square and Euclidean distance to measure the difference between the predicted and true center points.

[0015] Further, step S1 includes:

[0016] S11: Labelmg is used to annotate pupil image information;

[0017] S12: Mark eight light points on the pupil image information and generate a circular outer square region as the preset area.

[0018] Specifically, the annotation needs to be applied to the circumscribed square of the eight light points detected by the refractometer, as well as the intersection of the diagonals of the square. Use the Labelmg toolbox to annotate the image, obtaining the coordinates of the top-left and bottom-right corners of the square, as well as the coordinates of its center point.

[0019] Further, step S2 includes:

[0020] S21: Obtain the intersection of the diagonals of the circumscribed square of the circle, and use the coordinates of the upper left and lower right corners of the square in the pupil image information, as well as the coordinates of the center point, as the target center point.

[0021] Further, step S3 includes:

[0022] S31: Train the labeled pupil image information using the YOLO-V5 object detection network.

[0023] Further, step S4 includes:

[0024] S41: Input the pupil image matrix into the target detection network, and output the coordinates of the upper left and lower right corners of the predicted circumscribed square and the position coordinates of the predicted center point;

[0025] S42: Based on the coordinates of the upper left and lower right corners of the predicted circumscribed square and the position coordinates of the predicted center point, predict the location of the predicted center point and the predicted area.

[0026] Specifically, the YOLO-V5 real-time object detection network is used to train the labeled images. The network input is an image matrix, and the output is the coordinates of the top-left and bottom-right corners of the predicted circumscribed square, as well as the coordinates of a predicted center point. The errors between the predicted center point and the predicted square position and the ground truth label are calculated, and the weight parameters in the network are updated in reverse using the chain rule, allowing the network to continuously learn image features and output more accurate center point and circumscribed square positions.

[0027] Further, step S5 includes:

[0028] S51: The loss function for predicting the location of the target detection network is cross-entropy.

[0029] L = -(ylogp + (1-y)log(1-p))

[0030] Where y represents the target value, p represents the predicted value, and L represents the cross-entropy;

[0031] S52: The error in predicting the center point location is calculated by determining the absolute distance between the predicted center point and the target center point in multidimensional space using Euclidean distance.

[0032]

[0033] Where, vector x k The vector y represents the predicted center point location. k The label indicates the location, where X and Y represent the coordinates of the top left and bottom right corners of the circumscribed square, n represents the total number of pupil image information entries labeled, and k represents the number of the labeled pupil image information entries.

[0034] Specifically, to make the input image insensitive to changes in illumination and to eliminate the impact of changes in illumination conditions on the eight light points, the image input to the detection network uses random brightness variations to enhance its anti-interference ability.

[0035] A system for locating the pupil center in an optometry device includes:

[0036] Acquisition module: used to acquire pupil image information from the optometry instrument, and mark a preset area on the pupil image information, using the preset area as the feature area for the detection network to learn;

[0037] Labeling module: Used to label the intersection of the center lines of each point in the preset area as the target center point for the detection network to learn;

[0038] The training module is used to divide the labeled pupil image information into a training set and a test set according to a preset ratio, and to train the labeled pupil image information using a face detection neural network.

[0039] Prediction module: Predicts the center point and location of the preset area using an object detection network;

[0040] Judgment module: The target detection network applies cross-entropy loss function to the predicted region location and uses Euclidean distance to determine the error between the predicted center point and the target center point.

[0041] Output module: The weight parameters in the object detection network are updated in reverse using the chain rule, so that the object detection network can continuously learn image features and output more accurate center point and region locations.

[0042] Furthermore, the acquisition module includes:

[0043] First annotation unit: Labelmg is used to annotate pupil image information;

[0044] Generation unit: Marks eight light points on the pupil image information and generates a circular outer square region as the preset area;

[0045] The annotation module includes:

[0046] The second annotation unit: annotates the intersection of the diagonals of the circumscribed square of the circle, and obtains the coordinates of the upper left and lower right corners of the square in the pupil image information, as well as the coordinates of the center point, as the target center point.

[0047] Furthermore, the partitioning training module includes:

[0048] Training unit: The YOLO-V5 object detection network is used to train the labeled pupil image information.

[0049] Furthermore, the prediction module includes:

[0050] Input unit: Used to input the pupil image matrix into the object detection network, and outputs the coordinates of the upper left and lower right corners of the predicted circumscribed square and the position coordinates of the predicted center point;

[0051] Prediction Unit: Based on the coordinates of the upper left and lower right corners of the predicted circumscribed square and the position coordinates of the predicted center point, the prediction center point and the prediction area are predicted.

[0052] The present invention employing the above-described technical solution has at least the following beneficial effects:

[0053] This invention relates to a method and system for locating the pupil center in an optometry instrument. First, image information of the pupil center position determined by the optometry instrument is acquired. Then, the circumscribed squares of eight light beam spots generated by the optometry instrument are marked on the image information. Simultaneously, the center point of intersection of the eight light spots is also marked as the target point. The marked image information is then divided into training and validation sets in an 8:2 ratio, and a target detection network is constructed to train and test on the marked image information. The target detection network finally outputs the predicted center point position for each image. This method can meet real-time detection requirements and has high robustness to illumination intensity and occlusion conditions, enabling accurate pupil center positioning. Attached Figure Description

[0054] Figure 1 The flowchart of the method for locating the pupil center of an optometer provided in Embodiment 1 of the present invention Figure 1 ;

[0055] Figure 2 The flowchart of the method for locating the pupil center of an optometer provided in Embodiment 1 of the present invention Figure 2 ;

[0056] Figure 3 A schematic diagram showing the marked positions of the method for locating the pupil center of an optometer according to Embodiment 1 of the present invention;

[0057] Figure 4 This is a schematic diagram of the predicted position of the method for locating the pupil center of an optometer according to Embodiment 1 of the present invention.

[0058] Figure 5 The system structure for pupil center positioning of an optometer provided in Embodiment 2 of the present invention Figure 1 ;

[0059] Figure 6 The system structure for pupil center positioning of an optometer provided in Embodiment 2 of the present invention Figure 2 . Detailed Implementation

[0060] The following are specific embodiments of the present invention, which are described in conjunction with the accompanying drawings. However, the present invention is not limited to these embodiments.

[0061] Example 1

[0062] This embodiment provides a method for locating the center of the pupil in an optometer, such as... Figure 1 and Figure 2 As shown, this method includes the following steps:

[0063] S1: Collect pupil image information from the optometry device, and mark a preset area on the pupil image information, using the preset area as the feature area for the detection network to learn;

[0064] S2: Mark the intersection of the center lines of each point in the preset area as the target center point for the detection network to learn;

[0065] S3: Use the labeled pupil image information as a dataset, and divide the dataset into a training set and a test set according to a preset ratio. Use a face detection neural network to train and test the labeled pupil image information.

[0066] S4: Predict the center point and location of the preset area using the object detection network;

[0067] S5: Apply the cross-entropy loss function to the predicted region location through the target detection network, and use Euclidean distance to determine the error between the predicted center point and the target center point based on the predicted center point location in step S4.

[0068] S6: The weight parameters in the object detection network are updated in reverse by using the chain rule, so that the object detection network can continuously learn image features and output more accurate center point and region positions.

[0069] Specifically, this is to assist the pupil center point detection device in automatically locating the pupil center point under different lighting conditions in order to calculate the refractive power of the human eyeball.

[0070] See Figure 1 and Figure 3 Specifically, firstly, pupil images with eight light spots are captured by an optometry device. For each image, the circumscribed square region of the eight light spots is labeled as the feature region to be learned by the object detection network. The intersection of the center lines of the eight circular light spots is labeled as the learning target of the detection network. The labeled dataset is divided with a training set to a test set ratio of 8:2. The object detection network is constructed, and a face detection neural network or a general object detection network is used for training and testing on the labeled images. For the circumscribed square predicted by the detection network, the intersection of the two diagonals is calculated as the predicted center point. The detection network uses the cross-entropy loss function for the circumscribed square and Euclidean distance to measure the difference between the predicted and true center points.

[0071] Step S1 includes: S11: Labeling the pupil image information using Labelmg; S12: Labeling eight light points on the pupil image information and generating a circular outer square region as a preset region.

[0072] Step S2 includes: S21: Obtain the intersection of the diagonals of the circumscribed square of the circle, and obtain the coordinates of the upper left and lower right corners of the square in the pupil image information and the coordinates of the center point as the target center point.

[0073] Step S3 includes: S31: Training the labeled pupil image information using the YOLO-V5 object detection network.

[0074] See Figure 4 Step S4 includes:

[0075] S41: Input the pupil image matrix into the target detection network, and output the coordinates of the upper left and lower right corners of the predicted circumscribed square and the position coordinates of the predicted center point;

[0076] S42: Based on the coordinates of the upper left and lower right corners of the predicted circumscribed square and the position coordinates of the predicted center point, predict the location of the predicted center point and the predicted area.

[0077] Step S5 includes:

[0078] S51: The loss function for predicting the location of the target detection network is cross-entropy.

[0079] L = -(ylogp + (1-y)log(1-p))

[0080] Where y represents the target value, p represents the predicted value, and L represents the cross-entropy;

[0081] S52: The error in predicting the center point location is calculated by determining the absolute distance between the predicted center point and the target center point in multidimensional space using Euclidean distance.

[0082]

[0083] Where, vector x k The vector y represents the predicted center point location. k The label indicates the location, where X and Y represent the coordinates of the top left and bottom right corners of the circumscribed square, n represents the total number of pupil image information entries labeled, and k represents the number of the labeled pupil image information entries.

[0084] This method first acquires image information of the pupil center position determined by an optometry instrument, and then labels the circumscribed squares of the eight light beams generated by the optometry instrument on the image information. Simultaneously, the center point of the intersection of the eight light beams is also labeled as the target point. The labeled image information is then divided into training and validation sets in an 8:2 ratio, and a target detection network is constructed to train and test on the labeled image information. The target detection network finally outputs the predicted center point position for each image. This method can meet real-time detection requirements and has high robustness to illumination intensity and occlusion conditions, enabling accurate acquisition of the pupil center position.

[0085] Example 2

[0086] This embodiment provides a system for locating the pupil center in an optometry device, such as... Figure 5 and Figure 6 As shown, this system includes:

[0087] Acquisition module: used to acquire pupil image information from the optometry instrument, and mark a preset area on the pupil image information, using the preset area as the feature area for the detection network to learn;

[0088] Labeling module: Used to label the intersection of the center lines of each point in the preset area as the target center point for the detection network to learn;

[0089] The training module is used to divide the labeled pupil image information into a training set and a test set according to a preset ratio, and to train the labeled pupil image information using a face detection neural network.

[0090] Prediction module: Predicts the center point and location of the preset area using an object detection network;

[0091] Judgment module: The target detection network applies cross-entropy loss function to the predicted region location and uses Euclidean distance to determine the error between the predicted center point and the target center point.

[0092] Output module: The weight parameters in the object detection network are updated in reverse using the chain rule, so that the object detection network can continuously learn image features and output more accurate center point and region locations.

[0093] See Figure 6 The acquisition module includes:

[0094] First annotation unit: Use Labelmg to annotate pupil image information;

[0095] Generation unit: Marks eight light points on the pupil image information and generates a circular outer square region as the preset area;

[0096] The annotation module includes:

[0097] The second annotation unit: annotates the intersection of the diagonals of the circumscribed square of the circle, and obtains the coordinates of the upper left and lower right corners of the square in the pupil image information, as well as the coordinates of the center point, as the target center point.

[0098] Specifically, the annotation needs to be applied to the circumscribed square of the eight light points detected by the refractometer, as well as the intersection of the diagonals of the square. Use the Labelmg toolbox to annotate the image, obtaining the coordinates of the top-left and bottom-right corners of the square, as well as the coordinates of its center point.

[0099] The training module is divided into: Training unit: trained using the YOLO-V5 object detection network on labeled pupil image information.

[0100] Specifically, the YOLO-V5 real-time object detection network is used to train the labeled images. The network input is an image matrix, and the output is the coordinates of the top-left and bottom-right corners of the predicted circumscribed square, as well as the coordinates of a predicted center point. The errors between the predicted center point and the predicted square position and the ground truth label are calculated, and the weight parameters in the network are updated in reverse using the chain rule, allowing the network to continuously learn image features and output more accurate center point and circumscribed square positions.

[0101] The prediction module includes:

[0102] Input unit: Used to input the pupil image matrix into the object detection network, and outputs the coordinates of the upper left and lower right corners of the predicted circumscribed square and the position coordinates of the predicted center point;

[0103] Prediction Unit: Based on the coordinates of the upper left and lower right corners of the predicted circumscribed square and the position coordinates of the predicted center point, the prediction center point and the prediction area are predicted.

[0104] Specifically, to make the input image insensitive to changes in illumination and to eliminate the impact of changes in illumination conditions on the eight light points, the image input to the detection network uses random brightness variations to enhance its anti-interference ability.

[0105] Unlike current methods that rely on Haar-like feature extraction for coarse localization or use the RANSAC algorithm to estimate the pupil ellipse, this system employs deep learning detection techniques. Instead of complex mathematical calculations and fitting, it utilizes supervised learning of neural networks to directly predict the result, thereby obtaining the position of the center point.

[0106] This system first acquires image information of the pupil center position determined by an optometry instrument, and then labels the circumscribed squares of the eight light beams generated by the optometry instrument on the image information. Simultaneously, the center point of the intersection of the eight light beams is also labeled as the target point. The labeled image information is then divided into training and validation sets in an 8:2 ratio, and a target detection network is constructed to train and test on the labeled image information. The target detection network finally outputs the predicted center point position for each image. This system can meet real-time detection requirements and has high robustness to illumination intensity and occlusion conditions, enabling accurate acquisition of the pupil center position.

[0107] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A method for locating the center of the pupil in an optometer, characterized in that, Including the following steps: S1: Collect pupil image information from the optometry device, and mark a preset area on the pupil image information, using the preset area as the feature area for the detection network to learn; S2: Mark the intersection of the center lines of each point in the preset area as the target center point for the detection network to learn; S3: Use the labeled pupil image information as a dataset, and divide the dataset into a training set and a test set according to a preset ratio. Use a face detection neural network to train and test the labeled pupil image information. S4: Predict the center point and location of the preset area using the object detection network; S5: Apply the cross-entropy loss function to the predicted region location through the target detection network, and use Euclidean distance to determine the error between the predicted center point and the target center point based on the predicted center point location in step S4. S6: The weight parameters in the object detection network are updated in reverse by using the chain rule of differentiation, so that the object detection network can continuously learn image features and output more accurate center point and region positions; Step S1 includes: S11: Labelmg is used to annotate pupil image information; S12: Mark eight light points on the pupil image information and generate a circular outer square region as the preset region; Step S5 includes: S51: The loss function for predicting the location of the target detection network is cross-entropy. L = -(ylogp + (1-y)log(1-p)) Where y represents the target value, p represents the predicted value, and L represents the cross-entropy; S52: The error in predicting the center point location is calculated by determining the absolute distance between the predicted center point and the target center point in multidimensional space using Euclidean distance. Where, vector x k The vector y represents the predicted center point location. k The label indicates the location, where X and Y represent the coordinates of the top left and bottom right corners of the circumscribed square, n represents the total number of pupil image information entries labeled, and k represents the number of the labeled pupil image information entries.

2. The method for locating the pupil center in an optometer according to claim 1, characterized in that, Step S2 includes: S21: Obtain the intersection of the diagonals of the circumscribed square of the circle, and use the coordinates of the upper left and lower right corners of the square in the pupil image information, as well as the coordinates of the center point, as the target center point.

3. The method for locating the pupil center in an optometer according to claim 1, characterized in that, Step S3 includes: S31: Train the labeled pupil image information using the YOLO-V5 object detection network.

4. The method for locating the pupil center in an optometer according to claim 1, characterized in that, Step S4 includes: S41: Input the pupil image matrix into the target detection network, and output the coordinates of the upper left and lower right corners of the predicted circumscribed square and the position coordinates of the predicted center point; S42: Based on the coordinates of the upper left and lower right corners of the predicted circumscribed square and the position coordinates of the predicted center point, predict the location of the predicted center point and the predicted area.

5. A system for locating the pupil center in an optometry instrument, based on the method for locating the pupil center in an optometry instrument according to any one of claims 1-4, characterized in that, include: Acquisition module: used to acquire pupil image information from the optometry instrument, and mark a preset area on the pupil image information, using the preset area as the feature area for the detection network to learn; Labeling module: Used to label the intersection of the center lines of each point in the preset area as the target center point for the detection network to learn; The training module is used to divide the labeled pupil image information into a training set and a test set according to a preset ratio, and to train the labeled pupil image information using a face detection neural network. Prediction module: Predicts the center point and location of the preset area using an object detection network; Judgment module: The target detection network applies cross-entropy loss function to the predicted region location and uses Euclidean distance to determine the error between the predicted center point and the target center point. Output module: The weight parameters in the object detection network are updated in reverse using the chain rule, so that the object detection network can continuously learn image features and output more accurate center point and region locations.

6. The system for locating the pupil center of an optometer according to claim 5, characterized in that: The acquisition module includes: First annotation unit: Use Labelmg to annotate pupil image information; Generation unit: Marks eight light points on the pupil image information and generates a circular outer square region as the preset area; The annotation module includes: The second annotation unit: annotates the intersection of the diagonals of the circumscribed square of the circle, and obtains the coordinates of the upper left and lower right corners of the square in the pupil image information, as well as the coordinates of the center point, as the target center point.

7. The system for locating the pupil center of an optometer according to claim 6, characterized in that, The partitioning training module includes: Training unit: The YOLO-V5 object detection network is used to train the labeled pupil image information.

8. The system for locating the pupil center of an optometer according to claim 7, characterized in that, The prediction module includes: Input unit: Used to input the pupil image matrix into the object detection network, and outputs the coordinates of the upper left and lower right corners of the predicted circumscribed square and the position coordinates of the predicted center point; Prediction Unit: Based on the coordinates of the upper left and lower right corners of the predicted circumscribed square and the position coordinates of the predicted center point, the prediction center point and the prediction area are predicted.

Images