A pupil positioning method and related device

Abstract

The present application discloses a pupil positioning method and related devices, which relate to the field of image processing. After acquiring each frame of the eye image of the detection object, the pupil parameters of the eye image are detected, and the moving direction and moving speed of the motion control are determined according to the pupil parameters and the coordinates of the target point, so as to control the movement of the motion control according to the moving direction and moving speed until the pupil center of the detection object reaches the target point and the pupil clarity of the eye image is the highest, that is, the moving speed is adjusted for each frame of the eye image acquired, and the moving speed of the moving control is gradual, so that the entire pupil positioning movement process is smoother, the frustration caused by speed switching is reduced, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a pupil positioning method and related devices. Background Art

[0002] The pupil is an important part of the optical structure of the human eye. When ophthalmic instruments are performing eye tests, they need to move the motion controls of the instrument in multiple dimensions to locate the pupil of the test subject to the target point. However, the movement speed of existing ophthalmic instruments is usually a pre-set fixed gear, and switching speeds at different gears will cause a sense of frustration, especially in a small range near the target point. Oscillations will occur, causing great discomfort to the moving head and affecting the user experience. Moreover, since the movement speed only has a fixed gear to choose from, the instrument cannot accurately locate the pupil target point in real time, which is not conducive to improving the detection efficiency.

[0003] In addition, the initial distance between the test object and the ophthalmic instrument is random, so the forward and backward movement direction is uncertain during positioning. When the image at the starting position is very blurry, the clarity and other judgment indicators are in an oscillating state, which will cause the instrument to be unable to accurately judge the movement direction and may move in the wrong direction; even if the movement direction is correct, it will oscillate back and forth near the clearest position of the image, making the movement unsmooth, causing discomfort to the test object or generating noise. Summary of the invention

[0004] In view of the above problems, the present application provides a pupil positioning method and related devices, which make the movement process of pupil positioning smoother, reduce the frustration caused by speed switching, and improve user experience. The specific solution is as follows:

[0005] The first aspect of the present application provides a pupil positioning method, comprising:

[0006] Acquire a frame of eye image of the detection object, and detect pupil parameters of the eye image;

[0007] Determining the moving direction and moving speed of the motion control according to the pupil parameter and the coordinates of the target point;

[0008] Controlling the movement of the motion control in real time according to the moving direction and the moving speed;

[0009] Acquire the next frame of the eye image of the detection object, and perform the above-mentioned operations until the pupil center of the detection object reaches the target point and the pupil clarity of the eye image is the highest.

[0010] In a possible implementation, detecting pupil parameters of the eye image includes:

[0011] Performing bright spot detection on the eye image to obtain at least two bright spot circles;

[0012] Selecting at least two target bright spot circles that meet the reflection spot characteristics from the at least two bright spot circles;

[0013] Calculating the bright spot areas of the at least two target bright spot circles and the bright spot center distance between the target bright spot circles;

[0014] Calculate the pupil clarity of the pupil area.

[0015] In a possible implementation, determining the moving direction and moving speed of the motion control according to the pupil parameter and the coordinates of the target point includes:

[0016] Comparing the coordinates of the pupil center with the coordinates of the target point to determine the left-right moving direction and the up-down moving direction of the motion control;

[0017] Determining a remaining estimated distance between the pupil center and the target point in a left-right direction and a remaining estimated distance in a top-bottom direction according to the coordinates of the pupil center, the coordinates of the target point, and an estimated pixel ratio, wherein the estimated pixel ratio is a ratio of an actual size of a reference object in an image captured at the target position to a corresponding image pixel size;

[0018] The remaining estimated distance between the pupil center and the target point in the left and right directions is divided by the duration of the movement cycle to obtain the left and right movement speed of the motion control, wherein the duration of the movement cycle is the time for detecting pupil parameters for one frame of the eye image, and the left and right movement speed of the motion control does not exceed a preset left and right movement speed upper limit;

[0019] The remaining estimated distance between the pupil center and the target point in the up-down direction is divided by the duration of the movement cycle to obtain the up-down movement speed of the motion control, and the up-down movement speed of the motion control does not exceed the preset up-down movement speed;

[0020] When the remaining estimated distance between the pupil center and the target point in the left-right direction is less than a threshold, the motion control stops moving in the left-right direction;

[0021] When the remaining estimated distance between the pupil center and the target point in the up-down direction is less than a threshold, the motion control stops moving in the up-down direction.

[0022] In a possible implementation, the pupil locating method further includes:

[0023] For each test subject, respectively obtaining test eye images of the test subject photographed at different shooting distances;

[0024] Respectively obtain a bright spot distance curve, a bright spot size curve and a clarity curve corresponding to each of the test subjects, wherein the bright spot distance curve represents the bright spot center distance corresponding to each frame of the test eye image of the test subject, the bright spot size curve represents the bright spot area corresponding to each frame of the test eye image of the test subject, and the clarity curve represents the pupil clarity corresponding to each frame of the test eye image of the test subject;

[0025] Aligning the bright spot distance curves corresponding to all the test objects with a reference point as the center to obtain a modified bright spot distance curve, wherein the reference point is a point corresponding to the maximum pupil clarity in the clarity curve or the minimum bright spot area in the bright spot size curve;

[0026] Taking an average value of the bright spot center distances corresponding to the same horizontal coordinate position in the corrected bright spot distance curve to obtain an average value of the bright spot center distances corresponding to each frame of the test eye image;

[0027] Performing linear fitting on the average value of the bright spot center distance corresponding to each frame of the test eye image to obtain a bright spot distance reference curve, wherein the bright spot center distance corresponding to the reference point in the bright spot distance reference curve is the reference bright spot center distance;

[0028] A relative actual distance curve is obtained, where the relative actual distance curve represents the relative actual distance corresponding to the bright spot distance difference between each point on the bright spot distance reference curve and the reference point.

[0029] In a possible implementation, if the current eye image is the first frame of the eye image, determining the moving direction and moving speed of the motion control according to the pupil parameter and the coordinates of the target point includes:

[0030] Comparing the bright spot center distance in the pupil parameter with the reference bright spot center distance to determine the forward and backward movement direction of the motion control;

[0031] The relative actual distance corresponding to the difference between the bright spot center distance in the pupil parameter and the reference bright spot center distance is read from the relative actual distance curve as the initial moving distance distanceZ, and the ratio of the initial moving distance to the duration of the moving cycle is determined as the forward and backward moving speed of the motion control, the duration of the moving cycle is the time for detecting pupil parameters for one frame of the eye image, and the forward and backward moving speed of the motion control does not exceed a preset forward and backward moving speed upper limit.

[0032] In a possible implementation, if the current eye image is not the first frame of the eye image, determining the moving direction and moving speed of the motion control according to the pupil parameter and the coordinates of the target point includes:

[0033] If the pupil clarity of the current eye image is less than the current highest clarity or the bright spot area of ​​the current eye image is greater than the current minimum bright spot area, calculate the clarity and / or bright spot area of ​​multiple consecutive frames of eye images in the current moving direction, record the number of the multiple consecutive frames as num, and when num is less than a first preset value, the forward and backward moving direction of the motion control remains unchanged; when num reaches the first preset value, change the forward and backward moving direction of the motion control, reduce the forward and backward moving speed of the motion control, and calculate the remaining estimated moving distance in the forward and backward direction; the current highest clarity is the highest clarity value in the acquired eye image, and the current minimum bright spot area is the minimum bright spot area in the acquired eye image;

[0034] If the pupil clarity of the current eye image is greater than the current highest clarity and / or the bright spot area of ​​the current eye image is less than the current minimum bright spot area, the pupil clarity of the current eye image is determined as the current highest clarity or the bright spot area of ​​the current eye image is determined as the current minimum bright spot area, the forward and backward moving direction and forward and backward moving speed of the motion control are kept unchanged, and the remaining estimated distance in the forward and backward direction is calculated;

[0035] When the remaining estimated distance in the front-rear direction is less than a threshold value, the motion control stops moving in the front-rear direction.

[0036] In a possible implementation, when num is less than a first preset value, the forward and backward moving direction of the motion control remains unchanged; when num reaches the first preset value, the forward and backward moving direction of the motion control is changed, the forward and backward moving speed of the motion control is reduced, and the remaining estimated moving distance in the forward and backward direction is calculated, including:

[0037] If the forward and backward moving direction has not been changed at present, and num is less than the first preset value, the forward and backward moving direction of the motion control remains unchanged, the forward and backward moving speed vz=speed1 / (num+m) of the motion control is determined, and the remaining estimated distance distancez=vz×estimateTime×(N-num) between the pupil center and the target point in the forward and backward direction is determined, where m is an integer greater than or equal to 1, estimateTime is the duration of the moving cycle, the duration of the moving cycle is the time for detecting pupil parameters for one frame of the eye image, speed1 is the forward and backward moving speed of the motion control in the moving cycle after the previous frame of the eye image, N is a positive integer greater than num, and vz does not exceed the preset upper limit of the forward and backward moving speed;

[0038] If the forward and backward moving direction has not been changed and num reaches a first preset value, the forward and backward moving direction of the motion control is changed, the forward and backward moving speed of the motion control is set to speed2, and the remaining estimated distance distancez between the pupil center and the target point in the forward and backward direction is set to the initial moving distance distanceZ, where speed2 is less than speed1;

[0039] If after the first change in the forward and backward movement direction, and num is less than the second preset value, the forward and backward movement speed vz=speed2 / (num+m') of the motion control is determined, and the remaining estimated distance distancez=vz×estimateTime×(N-num) between the pupil center and the target point in the forward and backward direction is determined, where m' is an integer greater than m, and vz does not exceed the preset forward and backward movement speed upper limit;

[0040] If after the first change of the forward and backward moving direction, and num reaches the second preset value, the forward and backward moving direction of the motion control is changed, the forward and backward moving speed of the motion control is set to speed3, and the remaining estimated distance between the pupil center and the target point in the forward and backward direction is distancez=min(distanceZ / 2, the third preset value), where speed3 is less than speed2;

[0041] If after the second change in the forward and backward moving direction, and num is less than the fourth preset value n, the forward and backward moving speed vz=(distancez- speed3×estimateTime) / (n×estimateTime) of the motion control is determined, and the remaining estimated distance distancez=distancez- speed3×estimateTime between the pupil center and the target point in the forward and backward direction is determined, and vz does not exceed the preset upper limit of the forward and backward moving speed.

[0042] In a possible implementation, determining the moving direction and moving speed of the motion control according to the pupil parameter and the coordinates of the target point further includes:

[0043] When the motion control stops moving in the left-right, up-down, and forward-backward directions, it enters the fine-tuning stage:

[0044] Determine the left-right moving direction, left-right moving speed, up-down moving direction, and up-down moving speed of the motion control according to the coordinates of the pupil center and the coordinates of the target point, and multiply the left-right moving speed and up-down moving speed of the motion control by a first correction coefficient, respectively, wherein the first correction coefficient is less than 1;

[0045] Performing Gaussian curve fitting on the pupil clarity of the acquired eye image and the front and rear absolute position of the motion control, and determining the front and rear absolute position of the target corresponding to the maximum pupil clarity;

[0046] Comparing the current front-to-back absolute position with the target front-to-back absolute position to determine the front-to-back moving direction of the motion control;

[0047] Multiplying the current forward and backward movement speed of the motion control by a second correction coefficient, wherein the second correction coefficient is less than 1;

[0048] When the motion control moves to the target point in the left-right moving direction and the up-down moving direction and moves to the target front-back absolute position in the front-back moving direction, the motion control is controlled to stop moving.

[0049] In a possible implementation, the pupil locating method further includes:

[0050] If the remaining estimated distance between the pupil center and the target point in the left-right direction, the up-down direction, or the front-back direction exceeds the limit value of the motion control, the motion control is controlled to decelerate and move to the limit boundary of the motion control or the motion control is controlled to stop moving.

[0051] A second aspect of the present application provides a pupil positioning device, comprising:

[0052] A pupil detection unit, used to acquire an eye image of a detection object in real time and detect pupil parameters of each frame of the eye image;

[0053] A movement parameter determination unit, used to determine the movement direction and movement speed of the motion control according to the pupil parameter and the coordinates of the target point;

[0054] A movement control unit is used to control the movement of the motion control in real time according to the movement direction and the movement speed until the pupil center of the detection object reaches the target point and the pupil clarity of the eye image is the highest.

[0055] By means of the above technical scheme, the present application provides a pupil positioning method and related devices, which detect the pupil parameters of the eye image after acquiring each frame of the eye image of the detection object, determine the moving direction and moving speed of the motion control according to the pupil parameters and the coordinates of the target point, and thus control the movement of the motion control according to the moving direction and moving speed until the pupil center of the detection object reaches the target point and the pupil clarity of the eye image is the highest, that is, the moving speed is adjusted for each frame of the eye image acquired, and the moving speed of the motion control is continuously and gradually changed, so that the entire pupil positioning movement process is smoother, reducing the frustration caused by speed switching, and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0057] Figure 1 A schematic diagram of a flow chart of a pupil positioning method provided in an embodiment of the present application;

[0058] Figure 2 A schematic diagram of the angle range provided for an embodiment of the present application;

[0059] Figure 3 A schematic diagram of a bright spot distance curve provided in an embodiment of the present application;

[0060] Figure 4 A schematic diagram of a bright spot size curve provided in an embodiment of the present application;

[0061] Figure 5 A schematic diagram of a clarity curve provided in an embodiment of the present application;

[0062] Figure 6 A schematic diagram of the structure of a pupil locating device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0064] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0065] The present application embodiment provides a pupil positioning method. The pupil positioning method of the present application embodiment is described in detail below with reference to the accompanying drawings.

[0066] Reference Figure 1 , Figure 1 A flow chart of a pupil positioning method provided in an embodiment of the present application is shown as follows: Figure 1 As shown, a pupil positioning method provided in an embodiment of the present application may include steps 101 to 103, and these steps are described in detail below.

[0067] 101: Acquire a frame of eye image of the detection object, and detect pupil parameters of the eye image;

[0068] The pupil parameters of the eye image include bright spot area, bright spot center distance, pupil center coordinates, pupil clarity, etc. Among them, the bright spot is the reflected light spot of the light source on the cornea.

[0069] 102: Determine the moving direction and moving speed of the motion control according to the pupil parameter and the coordinates of the target point;

[0070] The moving direction and moving speed of the motion control include: left and right moving direction and left and right moving speed, up and down moving direction and up and down moving speed, forward and backward moving direction and forward and backward moving speed.

[0071] The left-right moving direction and the left-right moving speed, and the up-down moving direction and the up-down moving speed are determined by comparing the coordinates of the pupil center in the pupil parameters with the coordinates of the target point.

[0072] As for the forward and backward moving direction and speed, since the purpose of the movement is to maximize the pupil clarity of the eye image, it is necessary to determine whether the current forward and backward moving direction is correct based on the changing trend of the pupil clarity of the eye image during the movement process, thereby determining the forward and backward moving direction and speed.

[0073] 103: Control the movement of the motion control in real time according to the moving direction and moving speed;

[0074] After determining the left and right moving direction and left and right moving speed, the up and down moving direction and up and down moving speed, the forward and backward moving direction and forward and backward moving speed, the left and right moving speed vector V is determined according to the left and right moving direction and the left and right moving speed. x , corresponding to the movement on the X axis; determine the up and down movement speed vector V according to the up and down movement direction and up and down movement speed y , corresponding to the movement on the Y axis; the forward and backward movement speed vector V is determined according to the forward and backward movement direction and the forward and backward movement speed z , corresponding to the movement on the Z axis. Finally, we get the velocity vector vec{V} of the motion control = (V x ,V y , V z ), thereby controlling the movement of the motion control according to the velocity vector vec{V}.

[0075] 104: Obtain the next frame of eye image of the detection object, and detect pupil parameters of the eye image;

[0076] 105: Determine whether the pupil center of the detected object reaches the target point and the pupil clarity of the eye image is the highest; if so, the process ends, if not, return to execute step 102.

[0077] The end point of the motion control moving up, down, left, and right is the target point (x, y), and the end point of the motion control moving forward and backward is the position where the pupil clarity of the eye image is the highest.

[0078] The present embodiment discloses a pupil positioning method, which detects pupil parameters of the eye image after acquiring each frame of the eye image of the detection object, determines the moving direction and moving speed of the motion control according to the pupil parameters and the coordinates of the target point, and controls the movement of the motion control according to the moving direction and moving speed until the pupil center of the detection object reaches the target point and the pupil clarity of the eye image is the highest, that is, the three moving directions and speeds are calculated and adjusted in real time for each frame of the eye image acquired, so that the moving speed of the motion control is continuously and gradually changed, the whole moving process is smoother, the frustration caused by speed switching is reduced, the user experience is improved, and the detection efficiency is improved.

[0079] Each step in the above embodiment is described in detail below.

[0080] In a possible implementation, detecting pupil parameters of the eye image in step 101 in the above embodiment includes the following steps 1011-1014:

[0081] 1011: Perform bright spot detection on the eye image to obtain at least two bright spot circles;

[0082] In order to reduce the amount of calculation and improve the accuracy, pupil candidate points (points that may correspond to the pupil) can be obtained first, and then bright spot circles in the eye image are detected within a certain range near the pupil candidate points. Exemplarily, the pupil candidate points in the eye image are obtained using an existing target detection algorithm, and then the eye image is gray-processed, and the bright spot circles in the eye image are detected using the Hough circle detection algorithm.

[0083] 1012: Select at least two target bright spot circles that meet the reflection spot characteristics from at least two bright spot circles;

[0084] The characteristics of the reflected light spot are consistent with those of the light source.

[0085] The number of target bright spots is related to the light source. For example, if the light source is an LED lamp, the more LED lamps there are, the more light spots will be reflected on the cornea. For example, if the light source is two LED lamps, since the LED lamps are set in parallel, the reflected light spot feature is two bright spots of similar size and height. For each detected bright spot, the method of screening the target bright spot that meets the reflected light spot feature is as follows:

[0086] A1: Take the center of the bright spot circle as the center, the radius of the bright spot circle as the starting point, and the step length as 1 to search the boundary outward.

[0087] Because the Hough circle detection is performed on the eye image after the boundary is extracted by the Sobel operator and binarized, the pixel values ​​of the binarized eye image are only 0 and 255. Therefore, the stopping condition is that the pixel value of the current step minus the pixel value of the previous step is equal to 255, and the search range does not exceed 10 times the radius of the bright spot circle.

[0088] A2: Divide the circle of the bright spot into 360 angles. Execute the steps in A1 for each angle. If a certain angle triggers the stop condition, save the coordinates of the point corresponding to the current step. This point is the boundary point of the bright spot circle.

[0089] The coordinate system here is the image coordinate system, with the upper left corner of the eye image as the origin, the right as the positive direction of the X axis, and the downward as the positive direction of the Y axis.

[0090] A3: If the number of saved points is less than the set threshold (for example, 360×70%), that is, there are too few boundary points to form a circle, then it is determined that the bright spot circle is not the boundary circle corresponding to the reflection spot; otherwise, a circle is fitted for all saved points, and the mean of the fitting error (the absolute value of the difference between the distance from the point to the center of the circle and the radius of the fitting circle) is calculated. If the mean is greater than the set threshold, it means that the circle fitted by these boundary points is not the bright spot circle corresponding to the reflection spot; if the mean of the fitting error is not greater than the set threshold, it means that the bright spot circle is the bright spot circle corresponding to the reflection spot.

[0091] The thresholds corresponding to images with different resolutions are different. For example, when the image resolution is 640×480, the threshold can be set to 3-5 pixels.

[0092] A4: The bright spot circle that passes the above screening and meets the characteristics of the reflected light spot is the target bright spot circle.

[0093] 1013: Calculate the bright spot areas of at least two target bright spot circles and the bright spot center distance between the target bright spot circles;

[0094] It can be understood that the bright spot area and the bright spot center distance can be obtained according to the coordinates of the center of the bright spot circle and the radius.

[0095] 1014: Calculate pupil clarity of the pupil area.

[0096] Taking the light source as two LED lamps, that is, there are two target bright spots as an example, a possible implementation of calculating the pupil clarity of the pupil area includes the following steps B1-B8:

[0097] B1: Fill the target bright spot circle according to its center and radius (fill with black, i.e., set the pixel value to 0) to avoid interference of the bright spot in the subsequent pupil boundary search process.

[0098] B2: Normalize and binarize the padded image.

[0099] B3: Take the average of the centers of the two target bright spots to obtain the first pupil center.

[0100] Exemplarily, if there are three target bright spot circles, the first pupil center is the geometric center of the bright spot pattern formed by the three target bright spot circles.

[0101] B4: Calculate the angle range to be avoided based on the center and radius of the two target bright spots and the center of the first pupil, and do not search for the pupil boundary within this angle range;

[0102] Specifically, Figure 2 As shown, the intersection of a straight line parallel to the y-axis and passing through the center of the target bright spot circle and the boundary of the target bright spot circle is obtained, and then the angle between the intersection and the straight line of the pupil center is calculated to obtain the angle between the two intersection points of a bright spot (as shown in α1 and α2 in the figure), which is the angle range to be avoided.

[0103] B5: With the initial radius as half of the absolute difference in the x-coordinates of the two target bright spot centers, the step length is 1, and the boundary is searched from the center of the first pupil outward. The stopping condition is that the pixel value of the current step minus the pixel value of the previous step is equal to -255, and the search range does not exceed 10 times the radius.

[0104] B6: Divide the circle around the center of the first pupil into 360 angles, and execute the steps in B5 for each angle. If a certain angle triggers the stop condition, the coordinates of the corresponding point of the current step are saved.

[0105] B7: If the number of saved points is greater than the set threshold (e.g., 360×70%), the saved points are fitted with an ellipse, and the resulting ellipse constitutes the pupil area.

[0106] B8: Calculate pupil clarity based on the fitted ellipse information.

[0107] Specifically, the fitted ellipse boundary is the pupil edge, and the ellipse center is the precise second pupil center. The pixel means of a preset number of points before and after each pupil edge point are calculated along the direction from the second pupil center position to the pupil edge point, and the difference between the pixel means of the preset number of points before and after is taken as the clarity of the pupil edge point. The clarity of all pupil edge points is statistically analyzed, and the statistical result is used as the pupil clarity.

[0108] In this way, pupil parameters including pupil center, pupil clarity, target bright spot circle area, and target bright spot circle center distance were obtained.

[0109] In addition, in a possible implementation, the left-right moving direction and left-right moving speed, and the up-down moving direction and up-down moving speed of the motion control are determined according to the pupil parameters and the coordinates of the target point, which specifically includes the following steps 1021-1025:

[0110] 1021: Compare the coordinates of the pupil center with the coordinates of the target point to determine the left-right moving direction and the up-down moving direction of the motion control;

[0111] Taking the coordinates of the target point as (x, y) and the coordinates of the pupil center in the eye image as (cx, cy) as an example, the positive or negative difference between the pupil center and the target point coordinates on the x-axis (cx-x) indicates the left-right movement direction of the motion control. If the difference is positive, the motion control is determined to move to the right, otherwise, the motion control is determined to move to the left. Similarly, the positive or negative difference between the pupil center and the target point coordinates on the y-axis (cy-y) indicates the up-down movement direction of the motion control. If the difference is positive, the motion control is determined to move downward, otherwise, the motion control is determined to move upward.

[0112] 1022: Determine a remaining estimated distance between the pupil center and the target point in the left-right direction and a remaining estimated distance in the up-down direction according to the coordinates of the pupil center, the coordinates of the target point, and the estimated pixel ratio;

[0113] The estimated pixel ratio estimateDis is the ratio of the actual size of the reference object in the image captured at the target position to the corresponding image pixel size. The target position is the clearest position for capturing the reference object, which may not be the clearest position for capturing the reference object. When the target position is the clearest position for capturing the reference object, estimateDis can only represent the approximate correspondence between the pixel and the actual size at the clearest position. The deviation will be large at other positions, but it can be used as an estimate in mobile control. The reference object can be a simulated eye of known size.

[0114] The remaining estimated distance between the pupil center and the target point in the left and right directions is abs((cx-x) ×estimateDis).

[0115] The remaining estimated distance between the pupil center and the target point in the up and down directions is abs((cy-y) ×estimateDis).

[0116] 1023: Divide the remaining estimated distance between the pupil center and the target point in the left and right directions by the duration of the movement cycle to obtain the left and right movement speed of the motion control;

[0117] The duration of the moving cycle estimateTime is the time for detecting pupil parameters for a frame of eye image, specifically the time for the image to be processed through steps 1011-1014.

[0118] The left and right moving speed of the motion control does not exceed the preset left and right moving speed upper limit maxSpeedX, that is, the left and right moving speed of the motion control is the minimum value of vx=abs((cx-x) ×estimateDis) / estimateTime and maxSpeedX.

[0119] 1024: Divide the remaining estimated distance between the pupil center and the target point in the up-down direction by the duration of the movement cycle to obtain the up-down movement speed of the motion control, and the up-down movement speed of the motion control does not exceed the preset up-down movement speed;

[0120] The up and down moving speed of the motion control does not exceed the preset up and down moving speed upper limit maxSpeedY, that is, the up and down moving speed of the motion control is the minimum value of vy=abs((cy-y) × estimateDis) / estimateTime and maxSpeedY.

[0121] 1025: When the remaining estimated distance between the pupil center and the target point in the left-right direction is less than a threshold, the motion control stops moving in the left-right direction; when the remaining estimated distance between the pupil center and the target point in the up-down direction is less than a threshold, the motion control stops moving in the up-down direction.

[0122] The threshold in the above step 1025 may be 0.1 mm, indicating that the motion control has reached the target point within the allowable error range in the left-right direction and the up-down direction.

[0123] The determination of the forward and backward moving direction and forward and backward moving speed is relatively complicated. When the light source is fixed, the bright spot distance curves in different eyes are relatively similar. The bright spot distance reference curve can be determined based on the bright spot distance curves of multiple test subjects. Therefore, the forward and backward initial moving direction determination value can be set based on the bright spot distance reference curve. Specifically, the following steps C1-C6 are included:

[0124] C1: for each test subject, obtain test eye images of the test subject taken at different shooting distances;

[0125] For each test subject, each shooting distance corresponds to a frame of test eye image.

[0126] C2: Obtain the bright spot distance curve, bright spot size curve and clarity curve corresponding to each test subject respectively. The bright spot distance curve represents the target bright spot circle center distance corresponding to each frame of the test eye image of the test subject. The bright spot size curve represents the target bright spot circle area corresponding to each frame of the test eye image of the test subject. The clarity curve represents the pupil clarity corresponding to each frame of the test eye image of the test subject.

[0127] The bright spot distance curve is as follows Figure 3 As shown in the figure, the horizontal axis represents the distance from the clearest position of each frame of the eye image, indicating the shooting order from far to near, and the vertical axis is the distance from the center of the target bright spot circle, that is, the bright spot distance. Figure 3 It can be seen that: according to the principle that things get smaller when they are farther away and larger when they are closer, the distance between the centers of bright spots increases as the shooting distance gets closer.

[0128] The bright spot size curve is as follows Figure 4 As shown, the horizontal axis represents the distance before and after the clearest position of each frame of the test eye image, indicating the shooting order from far to near, and the vertical axis is the area size of the target bright spot circle, that is, the bright spot area.

[0129] The clarity curve is as follows Figure 5 As shown, the horizontal axis represents the distance before and after the clearest position of each frame of the test eye image, indicating the shooting order from far to near, and the vertical axis is the pupil clarity corresponding to the test eye image.

[0130] As the shooting distance increases from far to near, the image changes from blurry to clear and then to blurry again. Correspondingly, the pupil clarity curve shows a trend of first rising and then falling, and as the shooting distance increases from far to near, the bright spot area curve shows a trend of first decreasing and then increasing. Therefore, it can be found that the points corresponding to the maximum pupil clarity in the clarity curve and the minimum bright spot area in the bright spot size curve are very close.

[0131] C3: Align the bright spot distance curves corresponding to all test objects with the reference point as the center to obtain the modified bright spot distance curve. The reference point is the point corresponding to the maximum pupil clarity in the clarity curve or the minimum bright spot area in the bright spot size curve;

[0132] C4: taking the average of the bright spot center distances corresponding to the same horizontal coordinate position in the corrected bright spot distance curve, and obtaining the average bright spot center distance corresponding to each frame of the test eye image;

[0133] C5: Perform linear fitting on the average value of the bright spot center distance corresponding to each frame of the test eye image to obtain a bright spot distance reference curve. The bright spot center distance corresponding to the reference point in the bright spot distance reference curve is the reference bright spot center distance;

[0134] Since the data of multiple test objects are integrated, the bright spot distance reference curve has universal reference value.

[0135] C6: Obtain a relative actual distance curve, which represents the relative actual distance corresponding to the difference between the bright spot distance of each point on the bright spot distance reference curve and the reference point.

[0136] Since the motion control moves at a constant speed when shooting the test object and the relative positions of different shooting distances are known, the actual distance corresponding to every two frames of test eye images in the bright spot distance reference curve can be obtained, and the relative actual distance corresponding to the bright spot distance difference between each point on the bright spot distance reference curve and the reference point can be obtained.

[0137] Based on this, if the current eye image is the first frame of the eye image, determining the forward and backward moving direction and forward and backward moving speed of the motion control includes the following steps D1-D2:

[0138] D1: Compare the bright spot center distance in the pupil parameters of the first frame of the eye image with the reference bright spot center distance to determine the forward and backward movement direction of the motion control;

[0139] According to the principle that objects are smaller when they are far away and larger when they are near, if the bright spot center distance in the pupil parameters in the first frame of the eye image is greater than the reference bright spot center distance, it means that the current shooting position is forward relative to the estimated clearest shooting position, and the motion control should be moved backward, otherwise it should be moved forward.

[0140] D2: The relative actual distance corresponding to the difference between the bright spot center distance in the pupil parameter and the reference bright spot center distance is read from the relative actual distance curve as the initial moving distance distanceZ, and the ratio of the initial moving distance distanceZ to the duration of the moving cycle estimateTime is determined as the forward and backward moving speed of the motion control. The duration of the moving cycle estimateTime is the time for detecting pupil parameters for the first frame of the eye image, specifically the time for the image to be processed by steps 1011-1014.

[0141] The relative actual distance corresponding to the difference between the bright spot center distance in the pupil parameter and the reference bright spot center distance is read from the relative actual distance curve, that is, the relative actual distance between the current shooting position and the estimated clearest shooting position.

[0142] Since the forward and backward movement speed of the motion control does not exceed the preset forward and backward movement speed upper limit maxSpeedZ, the forward and backward movement speed of the motion control is the minimum value of vz=distanceZ / estimateTime and maxSpeedZ.

[0143] If the current eye image is not the first frame of the eye image, determining the forward and backward moving direction and forward and backward moving speed of the motion control includes the following steps D3-D5:

[0144] D3: If the pupil clarity of the current eye image is less than the current highest clarity and / or the bright spot area of ​​the current eye image is greater than the current minimum bright spot area, it means that as the motion control moves forward and backward, the pupil clarity does not improve, and the current forward and backward moving direction of the motion control may be wrong. However, since the clarity curve may have local fluctuations, in order to avoid misjudgment, the clarity and / or bright spot area of ​​multiple consecutive frames (the number is recorded as num) of eye images in the current moving direction are calculated, and a first preset value of num is set. When num does not reach the first preset value, the forward and backward moving direction of the motion control remains unchanged. When num reaches the first preset value, if the clarity and / or bright spot area still does not meet the requirements, it means that the current forward and backward moving direction is wrong, then the forward and backward moving direction of the motion control is changed, the forward and backward moving speed of the motion control is reduced, and the remaining estimated moving distance in the forward and backward direction is calculated. The current highest clarity is the highest clarity value in the acquired eye image, and the current minimum bright spot area is the minimum bright spot area in the acquired eye image.

[0145] D4: If the pupil clarity of the current eye image is greater than the current highest clarity and / or the bright spot area of ​​the current eye image is less than the current minimum bright spot area, it means that the current forward and backward moving direction is correct, and the movement in this direction can be continued, and the pupil clarity of the current eye image is determined as the current highest clarity and / or the bright spot area of ​​the current eye image is determined as the current minimum bright spot area, and num is cleared to zero, the forward and backward moving direction of the motion control is kept unchanged, the forward and backward moving speed of the motion control is kept unchanged, and the remaining estimated distance in the forward and backward direction is calculated;

[0146] D5: When the remaining moving distance in the front-back direction is less than the threshold, the motion control stops moving in the front-back direction.

[0147] The threshold here may be 0.2 mm. When the remaining moving distance in the front-to-back direction is less than the threshold, it indicates that the motion control has reached the estimated clearest shooting position within the allowable error range in the front-to-back direction.

[0148] In a possible implementation, the current eye image is not the first frame of the eye image, and steps D3-D5 of determining the forward and backward moving direction and forward and backward moving speed of the motion control include the following three stages:

[0149] (1) First return determination stage

[0150] If the pupil clarity of the current eye image is greater than the current highest clarity and / or the bright spot area of ​​the current eye image is less than the current minimum bright spot area, the pupil clarity of the current eye image is determined as the current highest clarity and / or the bright spot area of ​​the current eye image is determined as the current minimum bright spot area. At this time, the forward and backward movement direction of the motion control remains unchanged, and the forward and backward movement speed is the minimum value of vz=speed1 and maxSpeedZ. Speed1 is the forward and backward movement speed of the motion control in the movement cycle after the previous frame of the eye image. The remaining estimated distance between the pupil center and the target point in the forward and backward direction is distancez=vz×estimateTime×N, where N is set according to the empirical value, and N>num.

[0151] If the pupil clarity of the current eye image is less than the current highest clarity and / or the bright spot area of ​​the current eye image is greater than the current minimum bright spot area, update the number of eye images num in the current moving direction, that is, num+1. When num is less than the first preset value (such as 5), the forward and backward moving direction of the motion control remains unchanged, the forward and backward moving speed is the minimum value of vz=speed1 / (num+m) and maxSpeedZ, and the remaining estimated distance distancez between the center of the pupil and the target point in the forward and backward direction is distancez=vz×estimateTime×(N-num), where m is an integer greater than or equal to 1, such as m=1. When num reaches the first preset value, change the forward and backward moving direction of the motion control, set the forward and backward moving speed of the motion control to speed2, and set the remaining estimated distance distancez between the center of the pupil and the target point in the forward and backward direction to the initial moving distance distanceZ in the above step D2, and enter the second return determination stage, where speed2 is less than speed1.

[0152] (2) Second return determination stage

[0153] If the pupil definition of the current eye image is greater than the current highest definition and / or the bright spot area of ​​the current eye image is less than the current minimum bright spot area, the pupil definition of the current eye image is determined as the current highest definition or the bright spot area of ​​the current eye image is determined as the current minimum bright spot area. At this time, the forward and backward movement direction of the motion control remains unchanged, the forward and backward movement speed is the minimum value of vz=speed2 and maxSpeedZ, and the remaining estimated distance between the pupil center and the target point in the forward and backward direction is distancez=vz×estimateTime×N.

[0154] If the pupil clarity of the current eye image is less than the current highest clarity and / or the bright spot area of ​​the current eye image is greater than the current minimum bright spot area, update the number of eye images num in the current moving direction, that is, num+1. When num is less than the second preset value (can be less than the first preset value, such as 4), the forward and backward moving direction of the motion control remains unchanged, and the forward and backward moving speed is the minimum value of vz=speed2 / (num+m') and maxSpeedZ, where m' is an integer greater than m, such as m'=2, that is, the speed is reduced relative to the first return determination stage, and the remaining estimated distance distancez between the pupil center and the target point in the forward and backward direction is distancez=vz×estimateTime×(N-num). When num reaches the second preset value, change the forward and backward moving direction of the motion control, set the forward and backward moving speed of the motion control to speed3, and the remaining estimated distance distancez between the pupil center and the target point in the forward and backward direction is distancez=min(distanceZ / 2, the third preset value), and enter the stop determination stage, where speed3 is less than speed2, and the third preset value can be 0.3mm.

[0155] (3) Stop judgment phase

[0156] If the pupil clarity of the current eye image is greater than the current maximum clarity and / or the bright spot area of ​​the current eye image is smaller than the current minimum bright spot area, the pupil clarity of the current eye image is determined as the current maximum clarity or the bright spot area of ​​the current eye image is determined as the current minimum bright spot area, and the movement of the motion control in the forward and backward directions is stopped.

[0157] If the pupil clarity of the current eye image is less than the current highest clarity and / or the bright spot area of ​​the current eye image is greater than the current minimum bright spot area, update the number of eye images num in the current moving direction, that is, num+1. When num is less than the fourth preset value n (can be less than the second preset value, such as n=2), the forward and backward moving direction of the motion control remains unchanged, the forward and backward moving speed of the motion control vz=(distancez- speed3×estimateTime) / (n×estimateTime) is determined, and the remaining estimated distance distancez=distancez- speed3×estimateTime between the pupil center and the target point in the forward and backward direction is determined. When num reaches the fourth preset value or the remaining estimated distance in the forward and backward direction is less than a threshold value (such as 0.2mm), the motion control stops moving in the forward and backward direction.

[0158] Further, after the motion control stops moving in the left-right direction, the up-down direction, and the front-back direction, the fine-tuning stage can be entered.

[0159] During the fine-tuning stage, for the left and right movement directions and the up and down movement directions, the left and right movement direction, left and right movement speed, up and down movement direction and up and down movement speed of the motion control are determined according to the coordinates of the pupil center and the coordinates of the target point, and the left and right movement speed and up and down movement speed of the motion control are multiplied by the first correction coefficient respectively. The first correction coefficient is less than 1, and the motion control is controlled to move slowly.

[0160] For the forward and backward movement direction, each frame of eye image is previously acquired to obtain the forward and backward absolute position of the motion control. Therefore, the pupil clarity and forward and backward absolute position of the acquired eye image can be fitted with a Gaussian curve to determine the forward and backward absolute position of the target corresponding to the maximum pupil clarity. Among them, considering that the motion control does not go very far in the forward and backward direction, that is, the curve only drops a little and then returns, so strictly speaking, there is only half of the curve, so there is a situation where the direct fitting effect is not good. In this case, by directly presetting half the width of the curve, such as the Gaussian curve, an initial standard deviation is given, that is, the sigma value is limited and then the Gaussian curve is fitted. The sigma value is obtained when obtaining the bright spot distance reference curve. Specifically, the standard deviation obtained by linearly fitting the average value of the bright spot center distance corresponding to each frame of the test eye object in the above step C5 is the sigma value.

[0161] Then compare the current front and back absolute positions with the target front and back absolute positions to determine the front and back moving direction of the motion control, multiply the current front and back moving speed of the motion control by a second correction coefficient, the second correction coefficient is less than 1, and when the motion control moves to the target point in the left and right moving directions and the up and down moving directions and the motion control moves to the target front and back absolute positions in the front and back moving directions, control the motion control to stop moving.

[0162] It should be noted that if the remaining estimated distance between the pupil center and the target point in the left-right direction, up-down direction, or front-back direction exceeds the limit value of the motion control, the motion control is controlled to decelerate and move to the limit boundary of the motion control or the motion control is controlled to stop moving, and a prompt is given that the target point cannot be reached to ensure the safety of the machine operation.

[0163] A pupil positioning method provided by an embodiment of the present application is introduced above, and a device for executing the pupil positioning method will be introduced below.

[0164] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of a pupil positioning device provided in an embodiment of the present application. Figure 6 As shown, the pupil positioning device comprises:

[0165] The pupil detection unit 601 is used to obtain an eye image of a detection object in real time and detect pupil parameters of each frame of the eye image;

[0166] A movement parameter determination unit 602, configured to determine a movement direction and a movement speed of the motion control according to the pupil parameter and the coordinates of the target point;

[0167] The movement control unit 603 is used to control the movement of the motion control in real time according to the movement direction and the movement speed until the pupil center of the detection object reaches the target point and the pupil clarity of the eye image is the highest.

[0168] In a possible implementation, the pupil detection unit 601 is specifically used to perform bright spot detection on the eye image to obtain at least two bright spot circles; select at least two target bright spot circles that meet the characteristics of the reflected light spot from the at least two bright spot circles; calculate the bright spot areas of the at least two target bright spot circles and the bright spot center distances between the target bright spot circles; and calculate the pupil clarity of the pupil area.

[0169] In a possible implementation, the movement parameter determination unit 602 includes:

[0170] A first direction determination subunit, used for comparing the coordinates of the pupil center with the coordinates of the target point to determine the left-right moving direction and the up-down moving direction of the motion control;

[0171] a first distance determination subunit, for determining a remaining estimated distance between the pupil center and the target point in a left-right direction and a remaining estimated distance in a top-bottom direction according to the coordinates of the pupil center, the coordinates of the target point, and an estimated pixel ratio, wherein the estimated pixel ratio is a ratio of an actual size of a reference object in an image captured at the target position to a corresponding image pixel size;

[0172] The first speed determination subunit is used to divide the remaining estimated distance between the pupil center and the target point in the left-right direction by the duration of the movement cycle to obtain the left-right movement speed of the motion control, the duration of the movement cycle is the time for detecting pupil parameters for one frame of the eye image, and the left-right movement speed of the motion control does not exceed a preset left-right movement speed upper limit; when the remaining estimated distance between the pupil center and the target point in the left-right direction is less than a threshold, the left-right movement speed of the motion control is set to 0;

[0173] The second speed determination subunit is used to divide the remaining estimated distance between the pupil center and the target point in the up and down direction by the duration of the movement cycle to obtain the up and down movement speed of the motion control, and the up and down movement speed of the motion control does not exceed the preset up and down movement speed; when the remaining estimated distance between the pupil center and the target point in the up and down direction is less than a threshold value, the up and down movement speed of the motion control is set to 0.

[0174] In a possible implementation, the pupil locating device further includes:

[0175] A test image acquisition unit, used for acquiring, for each test subject, test eye images of the test subject shot at different shooting distances;

[0176] a curve acquisition unit, used to respectively acquire a bright spot distance curve, a bright spot size curve and a clarity curve corresponding to each of the test subjects, wherein the bright spot distance curve represents the bright spot center distance corresponding to each frame of the test eye image of the test subject, the bright spot size curve represents the bright spot area corresponding to each frame of the test eye image of the test subject, and the clarity curve represents the pupil clarity corresponding to each frame of the test eye image of the test subject;

[0177] A curve correction unit, used for aligning the bright spot distance curves corresponding to all test objects with a reference point as the center to obtain a corrected bright spot distance curve, wherein the reference point is a point corresponding to the maximum pupil clarity in the clarity curve or the minimum bright spot area in the bright spot size curve;

[0178] a bright spot distance reference curve acquisition unit, configured to average the bright spot center distances corresponding to the same horizontal coordinate position in the modified bright spot distance curve to obtain the average bright spot center distance corresponding to each frame of the test eye image; perform linear fitting on the average bright spot center distance corresponding to each frame of the test eye image to obtain a bright spot distance reference curve, wherein the bright spot center distance corresponding to the reference point in the bright spot distance reference curve is the reference bright spot center distance;

[0179] The relative actual distance curve acquisition unit is used to acquire a relative actual distance curve, wherein the relative actual distance curve represents the relative actual distance corresponding to the bright spot distance difference between each point on the bright spot distance reference curve and the reference point.

[0180] In a possible implementation, the movement parameter determination unit 602 includes:

[0181] A second direction determination subunit, used for comparing the bright spot center distance in the pupil parameter with the reference bright spot center distance, to determine the forward and backward moving direction of the motion control;

[0182] The first parameter determination subunit is used to read the relative actual distance corresponding to the difference between the bright spot center distance in the pupil parameter and the reference bright spot center distance from the relative actual distance curve as the initial moving distance distanceZ, and determine the ratio of the initial moving distance to the duration of the moving cycle as the forward and backward moving speed of the motion control, the duration of the moving cycle is the time for detecting pupil parameters for one frame of the eye image, and the forward and backward moving speed of the motion control does not exceed a preset forward and backward moving speed upper limit.

[0183] In a possible implementation, the movement parameter determination unit 602 includes:

[0184] The second parameter determination subunit is used to calculate the clarity and / or bright spot area of ​​multiple consecutive frames of eye images in the current moving direction if the pupil clarity of the current eye image is less than the current highest clarity or the bright spot area of ​​the current eye image is greater than the current minimum bright spot area, and record the number of the multiple consecutive frames as num; when num is less than a first preset value, the forward and backward moving direction of the motion control remains unchanged; when num reaches the first preset value, the forward and backward moving direction of the motion control is changed, the forward and backward moving speed of the motion control is reduced, and the remaining estimated moving distance in the forward and backward direction is calculated; the current highest clarity is the highest clarity value in the acquired eye image, and the current minimum bright spot area is the minimum bright spot area in the acquired eye image;

[0185] a third parameter determination subunit, configured to determine the pupil clarity of the current eye image as the current highest clarity or the bright spot area of ​​the current eye image as the current minimum bright spot area if the pupil clarity of the current eye image is greater than the current highest clarity and / or the bright spot area of ​​the current eye image is less than the current minimum bright spot area, keep the forward and backward moving direction and forward and backward moving speed of the motion control unchanged, and calculate the remaining estimated distance in the forward and backward direction;

[0186] When the remaining estimated distance in the front-to-back direction is less than a threshold, the motion control stops moving in the front-to-back direction.

[0187] In a possible implementation, the second parameter determination subunit is specifically used to, if the forward and backward moving direction has not been changed at present, and num is less than a first preset value, the forward and backward moving direction of the motion control remains unchanged, determine the forward and backward moving speed vz=speed1 / (num+m) of the motion control, and determine the remaining estimated distance distancez=vz×estimateTime×(N-num) between the pupil center and the target point in the forward and backward direction, where m is an integer greater than or equal to 1, estimateTime is the duration of the moving cycle, the duration of the moving cycle is the time for detecting pupil parameters for one frame of the eye image, speed1 is the forward and backward moving speed of the motion control in the moving cycle after the previous frame of the eye image, N is a positive integer greater than num, and vz does not exceed the preset upper limit of the forward and backward moving speed; if the forward and backward moving direction has not been changed at present, and num reaches the first preset value, change the forward and backward moving direction of the motion control, set the forward and backward moving speed of the motion control to speed2, and set the remaining estimated distance distancez between the pupil center and the target point in the forward and backward direction to the initial moving distance distanceZ, where speed2 is less than speed1; if after the first change in the forward and backward moving direction, and num is less than the second preset value, determine the forward and backward moving speed vz=speed2 / (num+m') of the motion control, and determine the remaining estimated distance distancez=vz×estimateTime×(N-num) between the center of the pupil and the target point in the forward and backward direction, where m' is an integer greater than m, and vz does not exceed the preset upper limit of the forward and backward moving speed; if after the first change in the forward and backward moving direction, and num reaches the second preset value, change the forward and backward moving direction of the motion control, set the forward and backward moving speed of the motion control to speed3, and the remaining estimated distance distancez=min(distanceZ / 2, third preset value) between the center of the pupil and the target point in the forward and backward direction, where speed3 is less than speed2; if after the second change in the forward and backward moving direction, and num is less than the fourth preset value n, determine the forward and backward moving speed vz=(distancez-speed3×estimateTime) / (n×estimateTime), and determine the remaining estimated distance distancez=distancez-speed3×estimateTime between the pupil center and the target point in the front-back direction.

[0188] In a possible implementation, the movement parameter determination unit 602 further includes:

[0189] A fine-tuning subunit is used to enter a fine-tuning stage after the motion control stops moving in the left-right, up-down and front-to-back directions: determine the left-right moving direction, left-right moving speed, up-down moving direction and up-down moving speed of the motion control according to the coordinates of the pupil center and the coordinates of the target point, and multiply the left-right moving speed and up-down moving speed of the motion control by a first correction coefficient, wherein the first correction coefficient is less than 1; perform Gaussian curve fitting on the pupil clarity of the acquired eye image and the front-to-back absolute position of the motion control to determine the target front-to-back absolute position corresponding to the maximum pupil clarity; compare the current front-to-back absolute position with the target front-to-back absolute position to determine the front-to-back moving direction of the motion control; multiply the current front-to-back moving speed of the motion control by a second correction coefficient, wherein the second correction coefficient is less than 1; and control the motion control to stop moving when the motion control moves to the target point in the left-right moving direction and the up-to-down moving direction and moves to the target front-to-back absolute position in the front-to-back moving direction.

[0190] In one possible implementation, the mobile control unit 603 is also used to control the motion control to decelerate and move to the limit boundary of the motion control or control the motion control to stop moving if the remaining estimated distance between the pupil center and the target point in the left-right direction, the up-down direction, or the front-back direction exceeds the limit value of the motion control.

[0191] The present embodiment discloses a pupil positioning device, which detects pupil parameters of the eye image after acquiring each frame of the eye image of the detection object, determines the moving direction and moving speed of the motion control according to the pupil parameters and the coordinates of the target point, and controls the movement of the motion control according to the moving direction and moving speed until the pupil center of the detection object reaches the target point and the pupil clarity of the eye image is the highest, that is, the moving speed is adjusted each time a frame of the eye image is acquired, and the moving speed of the mobile control is gradual, so that the entire pupil positioning movement process is smoother, the frustration caused by speed switching is reduced, and the user experience is improved.

[0192] An embodiment of the present application also provides an electronic device, including a processor and a memory, wherein the processor and the memory communicate via a bus;

[0193] The memory is used to store program code and transmit the program code to the processor;

[0194] The processor is used to execute a pupil positioning method as described in any one of the implementation modes in the above embodiments according to the instructions in the program code.

[0195] An embodiment of the present application also provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any pupil positioning method provided in the embodiment of the present application.

[0196] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any pupil positioning method provided in the embodiment of the present application.

[0197] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0198] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0199] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0200] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A pupil positioning method, characterized in that: include: Acquire a frame of eye image of the detection object, and detect pupil parameters of the eye image; Determining the moving direction and moving speed of the motion control according to the pupil parameter and the coordinates of the target point; Controlling the movement of the motion control in real time according to the moving direction and the moving speed; Acquire the next frame of the eye image of the detection object, and perform the above-mentioned operations until the pupil center of the detection object reaches the target point and the pupil clarity of the eye image is the highest.

2. The pupil positioning method according to claim 1, characterized in that: The detecting pupil parameters of the eye image comprises: Performing bright spot detection on the eye image to obtain at least two bright spot circles; Selecting at least two target bright spot circles that meet the reflection spot characteristics from the at least two bright spot circles; Calculating the bright spot areas of the at least two target bright spot circles and the bright spot center distance between the target bright spot circles; Calculate the pupil clarity of the pupil area.

3. The pupil positioning method according to claim 1, characterized in that: The step of determining the moving direction and moving speed of the motion control according to the pupil parameter and the coordinates of the target point includes: Comparing the coordinates of the pupil center with the coordinates of the target point to determine the left-right moving direction and the up-down moving direction of the motion control; Determining a remaining estimated distance between the pupil center and the target point in a left-right direction and a remaining estimated distance in a top-bottom direction according to the coordinates of the pupil center, the coordinates of the target point, and an estimated pixel ratio, wherein the estimated pixel ratio is a ratio of an actual size of a reference object in an image captured at the target position to a corresponding image pixel size; The remaining estimated distance between the pupil center and the target point in the left and right directions is divided by the duration of the movement cycle to obtain the left and right movement speed of the motion control, wherein the duration of the movement cycle is the time for detecting pupil parameters for one frame of the eye image, and the left and right movement speed of the motion control does not exceed a preset left and right movement speed upper limit; The remaining estimated distance between the pupil center and the target point in the up-down direction is divided by the duration of the movement cycle to obtain the up-down movement speed of the motion control, and the up-down movement speed of the motion control does not exceed the preset up-down movement speed; When the remaining estimated distance between the pupil center and the target point in the left-right direction is less than a threshold, the motion control stops moving in the left-right direction; When the remaining estimated distance between the pupil center and the target point in the up-down direction is less than a threshold, the motion control stops moving in the up-down direction.

4. The pupil positioning method according to claim 1, characterized in that: The pupil positioning method also includes: For each test subject, respectively obtaining test eye images of the test subject photographed at different shooting distances; Respectively obtain a bright spot distance curve, a bright spot size curve and a clarity curve corresponding to each of the test subjects, wherein the bright spot distance curve represents the bright spot center distance corresponding to each frame of the test eye image of the test subject, the bright spot size curve represents the bright spot area corresponding to each frame of the test eye image of the test subject, and the clarity curve represents the pupil clarity corresponding to each frame of the test eye image of the test subject; Aligning the bright spot distance curves corresponding to all the test objects with a reference point as the center to obtain a modified bright spot distance curve, wherein the reference point is a point corresponding to the maximum pupil clarity in the clarity curve or the minimum bright spot area in the bright spot size curve; Taking an average value of the bright spot center distances corresponding to the same horizontal coordinate position in the corrected bright spot distance curve to obtain an average value of the bright spot center distances corresponding to each frame of the test eye image; Performing linear fitting on the average value of the bright spot center distance corresponding to each frame of the test eye image to obtain a bright spot distance reference curve, wherein the bright spot center distance corresponding to the reference point in the bright spot distance reference curve is the reference bright spot center distance; A relative actual distance curve is obtained, where the relative actual distance curve represents the relative actual distance corresponding to the bright spot distance difference between each point on the bright spot distance reference curve and the reference point.

5. The pupil positioning method according to claim 4, characterized in that: If the current eye image is the first frame of the eye image, determining the moving direction and moving speed of the motion control according to the pupil parameter and the coordinates of the target point includes: Comparing the bright spot center distance in the pupil parameter with the reference bright spot center distance to determine the forward and backward movement direction of the motion control; The relative actual distance corresponding to the difference between the bright spot center distance in the pupil parameter and the reference bright spot center distance is read from the relative actual distance curve as the initial moving distance distanceZ, and the ratio of the initial moving distance to the duration of the moving cycle is determined as the forward and backward moving speed of the motion control, the duration of the moving cycle is the time for detecting pupil parameters for one frame of the eye image, and the forward and backward moving speed of the motion control does not exceed a preset forward and backward moving speed upper limit.

6. The pupil positioning method according to claim 4, characterized in that: If the current eye image is not the first frame of the eye image, determining the moving direction and moving speed of the motion control according to the pupil parameter and the coordinates of the target point includes: If the pupil clarity of the current eye image is less than the current highest clarity or the bright spot area of ​​the current eye image is greater than the current minimum bright spot area, calculate the clarity and / or bright spot area of ​​multiple consecutive frames of eye images in the current moving direction, record the number of the multiple consecutive frames as num, and when num is less than a first preset value, the forward and backward moving direction of the motion control remains unchanged; when num reaches the first preset value, change the forward and backward moving direction of the motion control, reduce the forward and backward moving speed of the motion control, and calculate the remaining estimated moving distance in the forward and backward direction; the current highest clarity is the highest clarity value in the acquired eye image, and the current minimum bright spot area is the minimum bright spot area in the acquired eye image; If the pupil clarity of the current eye image is greater than the current highest clarity and / or the bright spot area of ​​the current eye image is less than the current minimum bright spot area, the pupil clarity of the current eye image is determined as the current highest clarity or the bright spot area of ​​the current eye image is determined as the current minimum bright spot area, the forward and backward moving direction and forward and backward moving speed of the motion control are kept unchanged, and the remaining estimated distance in the forward and backward direction is calculated; When the remaining estimated distance in the front-rear direction is less than a threshold value, the motion control stops moving in the front-rear direction.

7. The pupil positioning method according to claim 6, characterized in that: When num is less than a first preset value, the forward and backward moving direction of the motion control remains unchanged; when num reaches the first preset value, the forward and backward moving direction of the motion control is changed, the forward and backward moving speed of the motion control is reduced, and the remaining estimated moving distance in the forward and backward direction is calculated, including: If the forward and backward moving direction has not been changed at present, and num is less than the first preset value, the forward and backward moving direction of the motion control remains unchanged, the forward and backward moving speed vz=speed1 / (num+m) of the motion control is determined, and the remaining estimated distance distancez=vz×estimateTime×(N-num) between the pupil center and the target point in the forward and backward direction is determined, where m is an integer greater than or equal to 1, estimateTime is the duration of the moving cycle, the duration of the moving cycle is the time for detecting pupil parameters for one frame of the eye image, speed1 is the forward and backward moving speed of the motion control in the moving cycle after the previous frame of the eye image, N is a positive integer greater than num, and vz does not exceed the preset upper limit of the forward and backward moving speed; If the forward and backward moving direction has not been changed and num reaches a first preset value, the forward and backward moving direction of the motion control is changed, the forward and backward moving speed of the motion control is set to speed2, and the remaining estimated distance distancez between the pupil center and the target point in the forward and backward direction is set to the initial moving distance distanceZ, where speed2 is less than speed1; If after the first change in the forward and backward movement direction, and num is less than the second preset value, the forward and backward movement speed vz=speed2 / (num+m') of the motion control is determined, and the remaining estimated distance distancez=vz×estimateTime×(N-num) between the pupil center and the target point in the forward and backward direction is determined, where m' is an integer greater than m, and vz does not exceed the preset forward and backward movement speed upper limit; If after the first change of the forward and backward moving direction, and num reaches the second preset value, the forward and backward moving direction of the motion control is changed, the forward and backward moving speed of the motion control is set to speed3, and the remaining estimated distance between the pupil center and the target point in the forward and backward direction is distancez=min(distanceZ / 2, the third preset value), where speed3 is less than speed2; If after the second change in the forward and backward moving direction, and num is less than the fourth preset value n, the forward and backward moving speed vz=(distancez-speed3×estimateTime) / (n×estimateTime) of the motion control is determined, and the remaining estimated distance distancez=distancez-speed3×estimateTime between the pupil center and the target point in the forward and backward direction is determined, and vz does not exceed the preset upper limit of the forward and backward moving speed.

8. The pupil positioning method according to claim 3 or 6, characterized in that: The step of determining the moving direction and moving speed of the motion control according to the pupil parameter and the coordinates of the target point further includes: When the motion control stops moving in the left-right, up-down, and forward-backward directions, it enters the fine-tuning stage: Determine the left-right moving direction, left-right moving speed, up-down moving direction, and up-down moving speed of the motion control according to the coordinates of the pupil center and the coordinates of the target point, and multiply the left-right moving speed and up-down moving speed of the motion control by a first correction coefficient, respectively, wherein the first correction coefficient is less than 1; Performing Gaussian curve fitting on the pupil clarity of the acquired eye image and the front and rear absolute position of the motion control, and determining the front and rear absolute position of the target corresponding to the maximum pupil clarity; Comparing the current front-to-back absolute position with the target front-to-back absolute position to determine the front-to-back moving direction of the motion control; Multiplying the current forward and backward movement speed of the motion control by a second correction coefficient, wherein the second correction coefficient is less than 1; When the motion control moves to the target point in the left-right moving direction and the up-down moving direction and moves to the target front-back absolute position in the front-back moving direction, the motion control is controlled to stop moving.

9. The pupil positioning method according to any one of claims 3 to 7, characterized in that: The pupil positioning method also includes: If the remaining estimated distance between the pupil center and the target point in the left-right direction, the up-down direction, or the front-back direction exceeds the limit value of the motion control, the motion control is controlled to decelerate and move to the limit boundary of the motion control or the motion control is controlled to stop moving.

10. A pupil locating device, characterized in that: include: A pupil detection unit, used to acquire an eye image of a detection object in real time and detect pupil parameters of each frame of the eye image; A movement parameter determination unit, used to determine the movement direction and movement speed of the motion control according to the pupil parameter and the coordinates of the target point; A movement control unit is used to control the movement of the motion control in real time according to the movement direction and the movement speed until the pupil center of the detection object reaches the target point and the pupil clarity of the eye image is the highest.

Images