Pupil recognition method

By extracting and fitting pupil contour points, eliminating invalid data, and using least squares method to fit circles, the problem of difficulty in pupil recognition in the existing technology is solved, and the accurate identification of unobstructed and obstructed pupils is achieved, which improves the Identify success rate.

CN119014805AActive Publication Date: 2024-11-26Guangzhou Huangpu District He Eye Health Industry Technology Research Institute +1

Patent Information

Application Number
CN202411436980.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-15
Publication Date
2024-11-26
Estimated Expiration
2044-10-15

AI Technical Summary

Technical Problem

Existing pupil recognition methods are difficult to accurately locate under light changes and ptosis in the elderly, resulting in a low recognition success rate.

Method used

By extracting the contour points set of the subject's pupil, the invalid data of the obstructed part was eliminated, the circle was fitted using the least squares method, and the center position and diameter of the pupil were obtained, so as to realize the identification of the unobstructed and obstructed pupils.

Benefits of technology

It improves the success rate of pupil recognition in fully automatic ophthalmic medical devices, and can accurately identify normal and obscured pupils.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119014805A_ABST
    Figure CN119014805A_ABST
Patent Text Reader

Abstract

The invention discloses a pupil recognition method, relates to the field of image processing, and particularly belongs to the improvement of a pupil recognition method in full-automatic medical instruments in the ophthalmology department. Coarse positioning of pupils of a testee is achieved in a complex environment, and an effective area containing the pupils of the testee is intercepted; calculating the difference value between the average value of the pixel gray values in the effective area and the standard deviation of the pixel gray values, determining the segmentation threshold value of the intercepted effective area, and realizing accurate segmentation of the pupil of the testee; and extracting a contour point set of the pupil of the testee, rejecting invalid data in the shielded pupil contour of the testee, fitting a circle by using a least square method, and obtaining the center position coordinate of the pupil of the testee and the diameter of the pupil of the testee. The pupil in a normal state can be recognized, meanwhile, the shielded pupil can be recognized, and the success rate of the pupil recognition method in the full-automatic medical instrument for the ophthalmology department is greatly increased.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the field of ophthalmic examination image processing, and in particular to a pupil recognition method. Background Art

[0002] At present, with the rapid development of image processing technology, image processing technology has begun to be widely used in medical devices. Among them, when fully automatic ophthalmic examination equipment examines the eyes of the subject, it is necessary to accurately identify the pupil of the subject. Most of the existing pupil positioning methods currently use traditional image processing methods such as Hough transform method, ellipse fitting method, gradient vector method, etc. Because the shapes of the subject's eyes are different, different shadows are formed under light, which makes it difficult to accurately locate the pupil. The pupil of the subject can only be identified in the entire image collected. The realization of such a pupil recognition method is based on the sharp contrast between the collected subject's pupil part and other collected parts, so that the above-mentioned traditional image processing method can be used to realize pupil recognition, which greatly reduces the success rate of pupil recognition of the subject.

[0003] In addition, the subjects of ophthalmological examinations include a special group such as the elderly. As they age, the probability of eye diseases increases. The drooping eyelids of the elderly often block the pupils. During an ophthalmological examination, even if the doctor asks them to open their eyes wide and stare still, this is difficult for the elderly. This also increases the difficulty of pupil identification. Summary of the invention

[0004] In view of the above problems, the present invention proposes a pupil recognition method, which extracts the contour point set of the subject's pupil, removes invalid data in the obscured subject's pupil contour, and uses the least squares method to fit the circle to obtain the center position coordinates of the subject's pupil and the diameter of the subject's pupil. Not only can the normal unobstructed pupil be recognized, but also the obscured pupil can be recognized.

[0005] The technical solution adopted by the present invention is as follows:

[0006] A pupil recognition method, characterized in that it comprises the following steps:

[0007] S1 (collection step), collecting the image of the subject's eye area and graying the image;

[0008] S2 (precise positioning step), through the horizontal grayscale projection curve and the vertical projection curve of the subject's eye image, the horizontal position and the vertical position of the subject's pupil are determined respectively, so as to achieve precise positioning of the subject's pupil;

[0009] S3 (interception step), capturing an effective area image around the subject's pupil, which is a first effective area image, completely including the subject's pupil;

[0010] S4 (separation step), determining a segmentation threshold, performing threshold segmentation on the intercepted first effective area to obtain a second effective area;

[0011] S5 (recognition step), identifying the contour of the obtained second effective area, and screening out the contour of the subject's pupil;

[0012] S6 (fitting step), performs least squares circle fitting on the subject's pupil contour to obtain the center coordinates and radius of the pupil; the unobstructed subject's pupil contour point data are all valid data; the obstructed subject's pupil contour point data contain invalid data, and the invalid data is eliminated.

[0013] In this recognition method, by extracting the contour point set of the subject's pupil, removing the invalid data in the occluded subject's pupil contour, and fitting the circle using the least squares method, the center position coordinates of the subject's pupil and the diameter of the subject's pupil are obtained. Not only can the normal unoccluded pupil be recognized, but also the occluded pupil can be recognized, greatly improving the success rate of the pupil recognition method in ophthalmic fully automatic medical devices.

[0014] Optionally, step S2 includes the following steps:

[0015] S21, the image of the subject's eye area is a matrix of m×n resolution, according to the formula Find the horizontal grayscale projection curve of the image, where the horizontal coordinate is the column number of the matrix, and the vertical coordinate is the sum of the elements in each column of the matrix, where i, j are the rows and columns of the matrix, 0<i≤m, 0<j≤n, and M(i,j) represents the grayscale value of the pixel in the i-th column and j-th row;

[0016] S22, according to the formula Find the grayscale projection curve in the vertical direction of the image, where the horizontal coordinate is the row number of the matrix and the vertical coordinate is the sum of the elements in each row of the matrix, where i, j are the rows and columns of the matrix, 0<i≤m, 0<j≤n, and M(i,j) represents the grayscale value of the pixel in the i-th column and j-th row;

[0017] S23, determining the pupil position information of the subject, and analyzing the pupil position information from the horizontal grayscale projection curve and the vertical grayscale projection curve of the subject's eye area image.

[0018] Optionally, the S23 includes the following steps:

[0019] S231, perform first-order difference on the horizontal grayscale projection curve of the subject's eye area image obtained in S21 to find the position of the trough in the horizontal grayscale projection curve of the subject's eye area image, use array locs_h[] to record the position i of the trough, and use array pks_h[] to record the grayscale projection value y(i) corresponding to the trough position; filter out the grayscale projection value y(i) in the grayscale projection value array pks_h[] that is greater than half of the maximum horizontal grayscale projection value and is the minimum horizontal grayscale projection value, and at the same time determine the trough position i corresponding to the minimum grayscale projection value y(i) in the array locs_h[], and determine the horizontal coordinate of the subject's pupil position.

[0020] S232, perform first-order difference on the vertical grayscale projection curve of the subject's eye image in S22 to find the position of the trough in the vertical grayscale projection curve of the subject's eye image, use array locs_v[] to record the position j of the trough, and use array pks_v[] to record the grayscale projection value x(j) corresponding to the trough position j; filter out the grayscale projection value x(j) in the grayscale projection value array pks_v[] that is greater than half of the maximum vertical grayscale projection value and is the minimum vertical grayscale projection value, and at the same time determine the trough position j corresponding to the minimum vertical grayscale projection value x(j) in the array locs_v[] to determine the vertical coordinate of the subject's pupil position.

[0021] Optionally, the S3 step includes the following steps:

[0022] According to the position information of the subject's pupil (i, j),

[0023] Calculate the difference a between the horizontal coordinate i and the minimum value min_H of the non-zero area in the horizontal grayscale projection curve of the image of the subject's eye area;

[0024] Calculate the difference b between the maximum value max_H of the non-zero area in the horizontal grayscale projection curve of the image of the subject's eye area and the horizontal coordinate i;

[0025] Calculate the difference c between the ordinate j and the minimum value min_V of the non-zero area in the vertical grayscale projection curve of the image of the subject's eye area;

[0026] Calculate the difference d between the maximum value max_V of the non-zero area in the horizontal grayscale projection curve of the image of the subject's eye area and the vertical coordinate j;

[0027] Calculate the minimum value of a and b min(a,b), and the minimum value of c and d min(c,d) respectively.

[0028] The pupil of a normal person in a dark room can reach 5-7mm. Assuming that the pixel diameter of the largest pupil in the image is D, if min(a,b)≥D and min(c,d)≥D, the first effective area of ​​the subject's pupil is intercepted as:

[0029] [i - D, j - D, 2D, 2D];

[0030] If min(a, b) ≥ D and min(c, d) < D, then the first effective region of the subject's pupil is intercepted as:

[0031] [i - D, min(c, d), 2D, 2 * min(c, d)];

[0032] If min(a, b) < D and min(c, d) ≥ D, then the first effective region of the subject's pupil is intercepted as:

[0033] [min(a, b), j - D, 2 * min(a, b), 2D];

[0034] If min(a, b) < D and min(c, d) < D, then the first effective region of the subject's pupil is intercepted as:

[0035] [min(a, b), min(c, d), 2 * min(a, b), 2 * min(c, d)];

[0036] Optionally, step S4 includes the following steps

[0037] S41, calculate the average value of the grayscale values of all pixels in the first effective region according to the average value calculation formula;

[0038] S42, calculate the standard deviation of the grayscale values of the pixels in the first effective region according to the standard deviation formula;

[0039] S43, use the difference between the average value of the grayscale values of the pixels in the first effective region and the standard deviation of the grayscale values of the pixels in the first effective region as the grayscale threshold for threshold segmentation of the first effective region, and perform threshold segmentation on the first effective region to obtain the second effective region.

[0040] Optionally, step S5 includes the following steps: detect the contours in the second effective region image obtained by using the findContours function in opencv.

[0041] Optionally, step S6 includes the following steps

[0042] S61, determine the state of the subject's pupil by judging the relationship between the width W in the horizontal direction and the height H in the vertical direction of the subject's pupil contour. If W ≠ H, then the state of the subject's pupil is the occluded state; otherwise, the state of the subject's pupil is unoccluded. All the pupil contour point data of the unoccluded subject are valid data, and there are invalid data in the pupil contour point data of the occluded subject.剔除无效数据;

[0043] It should be noted that in the original text, "剔除无效数据" is a Chinese phrase, and I have translated it as "剔除无效数据" in the English translation for now. If there is a more accurate English expression for this operation, it can be adjusted accordingly. Also, please check if there are any specific requirements or corrections needed for the translation based on the context of the patent.S62, calculating the best fitting circle center coordinates and radius for the valid data of the subject's pupil contour points obtained in step S61;

[0044] S63, determining the center position coordinates of the subject's pupil according to the center coordinates obtained in S62 and the starting point coordinates of the first effective area intercepted in the intercepting step.

[0045] The beneficial effect of the present invention is that by extracting the contour point set of the subject's pupil, eliminating the invalid data in the obstructed subject's pupil contour, and fitting the circle using the least square method, the center position coordinates of the subject's pupil and the diameter of the subject's pupil are obtained. Not only can the normal unobstructed pupil be identified, but also the obstructed pupil can be identified, greatly improving the success rate of the pupil recognition method in the fully automatic ophthalmic medical device. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0047] Figure 1a It is the image of the eye area of ​​a normal subject;

[0048] Figure 1b It is an image of the eye area of ​​the subject being covered;

[0049] Figure 2a It is the horizontal grayscale projection curve of the image of the eye area of ​​a normal subject;

[0050] Figure 2b It is the horizontal grayscale projection curve of the image of the part of the subject's eyes that is blocked;

[0051] Figure 3a It is the vertical grayscale projection curve of the image of the eye area of ​​a normal subject;

[0052] Figure 3b It is the vertical grayscale projection curve of the image of the part of the eye of the subject being covered;

[0053] Figure 4 This is a schematic diagram of the effective pupil area of ​​a normal subject;

[0054] Figure 5 It is a schematic diagram of blocking the effective area of ​​the pupil of the subject;

[0055] Figure 6 It is a schematic diagram of the standard deviation range;

[0056] Figure 7aThis is a schematic diagram of the first effective area segmentation of a normal subject;

[0057] Figure 7b is a schematic diagram of segmentation of the first effective area of ​​the occluded subject;

[0058] Figure 8a This is a schematic diagram of the contour recognition and screening results of the second effective area of ​​a normal subject;

[0059] Figure 8b It is a schematic diagram of the contour recognition and screening results of the second effective area of ​​the occluded subject;

[0060] Figure 9a This is a schematic diagram of the horizontal and vertical dimensions of the pupil contour of a normal subject;

[0061] Figure 9b It is a schematic diagram of the horizontal and vertical dimensions of the pupil contour of the blocked subject. DETAILED DESCRIPTION

[0062] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0063] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0064] S1, collect the eye pictures of the subject and convert the images into grayscale;

[0065] like Figure 1a As shown in , the image of the subject's eyes is collected without any obstruction; Figure 1b As shown in , it is an image of the occlusion state of the subject's eyes.

[0066] S2, realize accurate positioning of the subject's pupil. Because the grayscale around the subject's pupil changes significantly in the collected subject's eye image, the horizontal and vertical positions of the subject's pupil can be determined respectively through the horizontal grayscale projection curve and the vertical projection curve of the subject's eye image, thereby realizing accurate positioning of the subject's pupil.

[0067] The specific implementation is as follows:

[0068] S21, the image of the subject's eye area is a matrix with a resolution of m×n. The horizontal grayscale projection curve of the image is calculated according to formula (1), where the horizontal coordinate is the column number of the matrix and the vertical coordinate is the sum of the elements in each column of the matrix (that is, the sum of the pixel values ​​in each column of the image).

[0069]

[0070] Where i and j are the rows and columns of the matrix, 0<i<m, 0<j<n, and M(i, j) represents the grayscale value of the pixel in the i-th column and j-th row.

[0071] The horizontal grayscale projection curve of the image in the eye area of ​​a normal subject is as follows: Figure 2a As shown; the horizontal grayscale projection curve of the image in the image where the subject's eyes are blocked is as follows Figure 2b shown.

[0072] S22, according to formula (2), the grayscale projection curve in the vertical direction of the image is obtained, where the horizontal coordinate is the row number of the matrix, and the vertical coordinate is the sum of the elements in each row of the matrix (that is, the sum of the pixel values ​​in each row of the image).

[0073]

[0074] Where i and j are the rows and columns of the matrix, 0<i≤m, 0<j≤n, and M(i,j) represents the grayscale value of the pixel in the i-th column and j-th row.

[0075] The vertical grayscale projection curve of the image in the eye area of ​​a normal subject is as follows: Figure 3a As shown; the horizontal grayscale projection curve of the image in the image where the subject's eyes are blocked is as follows Figure 3b shown.

[0076] S23, determining the pupil position information of the subject.

[0077] From the horizontal grayscale projection curve and vertical grayscale projection curve of the subject's eye image, the pupil is located at the trough of the curve. The first-order difference at the trough of the curve is d y [i]<0,d y [i+1]>0 phenomenon. Find all d in the first-order difference of the curve y [i]<0,d y For points [i+1]>0, use the array locs[] to record the position i of the trough, and use the array pks[] to record the grayscale projection value corresponding to the trough position i. Use the bubble sorting method to find the grayscale projection value in the array pks[] that is greater than half of the maximum grayscale projection value and the minimum grayscale projection value, as well as the corresponding position i.

[0078] S231, performing a first-order difference on the horizontal grayscale projection curve of the subject's eye image in step S21, and obtaining:

[0079] d y =y'(i)(0<i≤m)

[0080] With dy [i]<0,d y [i+1]>0(0<i<m) condition is used to find the position of the trough in the horizontal grayscale projection curve of the subject's eye image, use the array locs_h[] to record the position i of the trough, and use the array pks_h[] to record the grayscale projection value y(i) corresponding to the trough position i.

[0081] The grayscale projection value in the pks_h[] array is judged to see whether it is greater than half of the maximum horizontal grayscale projection value of the subject's eye area image, and bubble sorting is performed to screen out the grayscale projection value that is greater than half of the maximum horizontal grayscale projection value and is the minimum grayscale projection value y(i). In the array locs_h[], the trough position i corresponding to the minimum grayscale projection value y(i) is determined, that is, the horizontal coordinate of the subject's pupil position is determined.

[0082] S232, performing a first-order difference on the vertical grayscale projection curve of the subject's eye image in step S21, and obtaining:

[0083] d x =x'(j) (0<j≤n)

[0084] With d x [j]<0,d x [j+1]>0(0<j<n) condition is used to find the position of the trough in the vertical grayscale projection curve of the subject's eye image, use the array locs_v[] to record the position i of the trough, and use the array pks_v[] to record the grayscale projection value x(j) corresponding to the trough position j.

[0085] The grayscale projection value in the pks_v[] array is judged to be greater than half of the maximum horizontal grayscale projection value of the subject's eye area image, and bubble sorting is performed to screen out the grayscale projection value that is greater than half of the maximum horizontal grayscale projection value and is the minimum grayscale projection value x(j). In the array locs_v[], the trough position j corresponding to the minimum grayscale projection value x(j) is determined, that is, the vertical coordinate of the subject's pupil position is determined.

[0086] S3, capturing the effective area image around the subject's pupil, which is the first effective area image, completely including the subject's pupil.

[0087] According to the position information of the subject's pupil (i, j),

[0088] Calculate the difference a=i-min_H between the horizontal coordinate i and the minimum value min_H of the non-zero area in the horizontal grayscale projection curve of the image of the subject's eye area;

[0089] Calculate the difference b = max_H - i between the maximum value max_H of the non-zero region in the horizontal grayscale projection curve of the eye region image of the subject and the abscissa i;

[0090] Calculate the difference c = j - min_V between the ordinate j and the minimum value min_V of the non-zero region in the vertical grayscale projection curve of the eye region image of the subject;

[0091] Calculate the difference d = max_V - j between the maximum value max_V of the non-zero region in the horizontal grayscale projection curve of the eye region image of the subject and the ordinate i;

[0092] Calculate the minimum values min(a, b) of a and b respectively, and the minimum values min(c, d) of c and d.

[0093] In the case of a normal person's pupil in a dark room, it can reach 5 - 7 mm. Assume that the pixel diameter of the maximum pupil in the picture is D. If min(a, b) ≥ D and min(c, d) ≥ D, then the first effective region of the subject's pupil is intercepted as:

[0094] [i - D, j - D, 2D, 2D];

[0095] If min(a, b) ≥ D and min(c, d) < D, then the first effective region of the subject's pupil is intercepted as:

[0096] [i - D, min(c, d), 2D, 2 * min(c, d)];

[0097] If min(a, b) < D and min(c, d) ≥ D, then the first effective region of the subject's pupil is intercepted as:

[0098] [min(a, b), j - D, 2 * min(a, b), 2D];

[0099] If min(a, b) < D and min(c, d) < D, then the first effective region of the subject's pupil is intercepted as:

[0100] [min(a, b), min(c, d), 2 * min(a, b), 2 * min(c, d)];

[0101] S4 Determine an appropriate segmentation threshold and perform threshold segmentation on the first effective region intercepted in step S3.

[0102] In mathematics, the standard deviation is a measure of the dispersion of a set of data. It represents the degree of deviation of each data point from the mean. The larger the standard deviation, the greater the degree of deviation of the data points from the mean and the greater the dispersion of the data distribution.

[0103] The formula for calculating the mean:

[0104]

[0105] Standard deviation calculation formula:

[0106]

[0107] The mean-variance method, the mean plus or minus the standard deviation is a mathematical statistical method used to explain the distribution of data. It can help us quickly understand the specific location of a certain point of data, including the maximum value, minimum value, data at a specific location, etc. The significance of the mean plus or minus the standard deviation is to describe the distribution range of the data. In general, the mean plus or minus one standard deviation can include about 68% of the data points. If two standard deviations are added or subtracted, about 95% of the data points can be included. This range is called the "standard deviation range." Combined with the attached Figure 6 According to the standard deviation range diagram in and the data in Table 1 below, steps S41 to S43 are performed.

[0108] Table 1

[0109] Standard coefficient Pixel value ratio 1 68% 2 95% 3 99%

[0110] S41, calculating the average grayscale values ​​of all pixels in the first effective area according to the average value calculation formula:

[0111]

[0112] Wherein, N represents the number of pixels in the first effective area, represents the sum of the grayscale values ​​of all pixels in the first effective area, X i Represents the grayscale value of the i-th pixel in the first valid area.

[0113] S42, calculating the standard deviation of the grayscale values ​​of pixels in the first effective area according to the standard deviation formula:

[0114]

[0115] Where N represents the number of pixels in the first effective area, X i represents the gray value of the i-th pixel in the first valid area, represents the average grayscale value of pixels in the first effective area, Represents the sum of squares of the differences between the grayscale value of each pixel in the first valid area and the average grayscale value of the pixels.

[0116] S43: Taking the difference between the average grayscale value of the pixels in the first effective area and the standard deviation of the grayscale value of the pixels in the first effective area as the grayscale threshold for threshold segmentation of the first effective area.

[0117] Right now:

[0118]

[0119] Perform threshold segmentation on the first valid area to obtain the second valid area:

[0120] This process can be seen in Figure 7a as well as Figure 7b As shown in Figure 7a The diagram shown in FIG. 1 is a schematic diagram of a process of performing threshold segmentation on the first effective area of ​​a normal subject to obtain the second effective area. Figure 7b Shown in FIG. 1 is a schematic diagram of a process of performing threshold segmentation on a first effective area of ​​a normal subject to obtain a second effective area.

[0121] S5, identifying the contour of the second effective area obtained in step S4, and screening out the contour of the subject's pupil.

[0122] S51, using the findContours() function in opencv to detect the contours in the second valid area image obtained in step S4.

[0123] S52, the right picture is the result of detecting the contours in the left picture. Multiple contours will be detected, but there will be differences in size, area and other features between the contours, and there is a small contour included in the contour of the subject's pupil. The contour of the subject's pupil is screened out. The contour of the pupil of the unobstructed subject is close to a circle; while the lower half of the pupil contour of the subject obstructed by the upper eyelid is still close to a circle. For this process, please refer to the attached Figure 8a And attached Figure 8b As shown in the process, Figure 8a What is shown in the figure is the contour recognition and screening process of normal subjects. Figure 8b The contour recognition and screening process of the occluded subjects shown in .

[0124] S6, performing least squares circle fitting on the pupil contour of the subject obtained in step S5 to obtain the center coordinates (x, y) and radius r of the pupil.

[0125] When the subject's pupil is not blocked, the contour curve of the subject's pupil is close to a circular curve; after the subject's pupil is partially blocked by the upper eyelid, the contour curve of the lower part that is not blocked is still close to a circular curve.

[0126] Fitting a circle is one of the common tasks in image processing and computer vision. Its purpose is to find an optimal circle to describe the distribution characteristics of a given set of data points.

[0127] The least squares method for fitting a circle is a common mathematical optimization method that determines the best fitting circle by minimizing the sum of the squares of the distances from the data points to the fitting circle. Specifically, for a given set of data points, the parameters of the best fitting circle can be obtained by solving an optimization problem that minimizes the objective function.

[0128] S61, preprocessing the pupil contour of the subject screened out in step S5

[0129] The state of the subject's pupil is determined by judging the relationship between the horizontal width W and the vertical height H of the subject's pupil contour. If W≠H, the subject's pupil state is blocked; otherwise, the subject's pupil state is not blocked.

[0130] The unobstructed pupil contour point data of the test subject are all valid data. This state is as shown in the attached Figure 9a As shown in;

[0131] There are invalid data in the pupil contour point data of the occluded subject, and the invalid data needs to be eliminated. This state is as shown in the attached Figure 9b As shown in . Find the contour point coordinates (Xmin, Ymin1) when the X coordinate is the minimum value and the Y coordinate is the minimum value of the pupil contour in the horizontal direction of the subject, and (Xmax, Ymin2) when the X coordinate is the maximum value and the Y coordinate is the minimum value. The contour point data between Xmin and Xmax, with the Y coordinate greater than Ymin1 or greater than Ymin2, is valid data. S62, for the valid data (xmin, Ymin1) of the pupil contour point of the subject obtained in step S61 i ,y i )i=1,2,...,n, calculate the best fitting circle center coordinates (x0,y0) and radius r.

[0132] The equation of a circle can be expressed as:

[0133] (x-x0) 2 +(y-y0) 2 =r 2 (1)

[0134] For the least squares circle fitting, the optimization objective function of the square error is:

[0135]

[0136] Where: (x i ,y i )i=1,2,...,n are the coordinates of the feature points on the arc; n is the number of feature points involved in fitting.

[0137] On the premise of maintaining the characteristics of this optimization objective function, we need to use a slightly different and improved method to define the square of the error, which avoids the square root and obtains a direct solution to the minimization problem, defined as follows:

[0138]

[0139] Then (2) can be rewritten as:

[0140]

[0141] Let,B=-2y0,

[0142] That is, formula (3) can be expressed as:

[0143]

[0144] According to the principle of least squares, the parameters A, B, and C should make E obtain the minimum value. According to the method of finding the minimum value, A, B, and C should satisfy

[0145]

[0146] To solve the system of equations, first eliminate the parameter C, then

[0147] Mode have to

[0148]

[0149] Mode have to

[0150]

[0151] make

[0152]

[0153] Write equations (7) and (8) in matrix form

[0154]

[0155] According to formula (14) and formula (6), we can get:

[0156]

[0157] Thus, the best fitting center coordinates (x0, y0) and the fitting value of radius r are obtained:

[0158]

[0159] S63, according to the coordinates of the center of the fitting circle obtained in step S62 Determine the central position coordinates of the pupil of the subject based on the starting point coordinates of the first valid region intercepted in step S3:

[0160] Assume that the pixel diameter of the maximum pupil in the picture is D. If min(a, b) ≥ D and min(c, d) ≥ D, then the central position coordinates of the pupil of the subject,

[0161]

[0162] If min(a, b) ≥ D and min(c, d) < D, then the central position coordinates of the pupil of the subject,

[0163]

[0164] If min(a, b) < D and min(c, d) ≥ D, then the central position coordinates of the pupil of the subject,

[0165]

[0166] If min(a, b) < D and min(c, d) < D, then the central position coordinates of the pupil of the subject,

[0167]

[0168] The diameter of the pupil of the subject

[0169] The above-described embodiments merely represent some embodiments of the present invention. Their descriptions are relatively specific and detailed, but should not be construed as limiting the scope of the patent of the present invention. It should be noted that for those skilled in the art, they can still modify the technical solutions recorded in the foregoing embodiments or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A pupil recognition method, characterized in that: The following steps are included: The acquisition step is to acquire the image of the subject's eye area and grayscale the image; The precise positioning step is to determine the horizontal and vertical positions of the subject's pupils through the horizontal grayscale projection curve and the vertical projection curve of the subject's eye image, respectively, so as to achieve precise positioning of the subject's pupils; The interception step is to capture the image of the effective area around the pupil of the subject, which is the first effective area image and completely includes the pupil of the subject; A segmentation step, determining a segmentation threshold, performing threshold segmentation on the intercepted first effective area to obtain a second effective area; an identification step of identifying the contour of the obtained second effective area and screening out the contour of the subject's pupil; In the fitting step, the least squares circle fitting is performed on the pupil contour of the subject to obtain the center coordinates and radius of the pupil; the unobstructed pupil contour point data of the subject are all valid data; the obstructed pupil contour point data of the subject contain invalid data, and the invalid data are eliminated.

2. A pupil recognition method according to claim 1, characterized in that: The precise positioning steps include the following steps: S21, the image of the subject's eye area is a matrix of m×n resolution, according to the formula Find the horizontal grayscale projection curve of the image, where the horizontal coordinate is the column number of the matrix, and the vertical coordinate is the sum of the elements in each column of the matrix, where i, j are the rows and columns of the matrix, 0<i≤m, 0<j≤n, and M(i,j) represents the grayscale value of the pixel in the i-th column and j-th row; S22, according to the formula Find the grayscale projection curve in the vertical direction of the image, where the horizontal coordinate is the row number of the matrix and the vertical coordinate is the sum of the elements in each row of the matrix, where i, j are the rows and columns of the matrix, 0<i≤m, 0<j≤n, and M(i,j) represents the grayscale value of the pixel in the i-th column and j-th row; S23, determining the pupil position information of the subject, and analyzing the pupil position information from the horizontal grayscale projection curve and the vertical grayscale projection curve of the subject's eye area image.

3. A pupil recognition method according to claim 2, characterized in that: The S23 comprises the following steps: S231, perform first-order difference on the horizontal grayscale projection curve of the subject's eye image obtained in S21 to find the position of the trough in the horizontal grayscale projection curve of the subject's eye image, use array locs_h[] to record the position i of the trough, and use array pks_h[] to record the grayscale projection value y(i) corresponding to the trough position i; filter out the grayscale projection value y(i) in the grayscale projection value array pks_h[] that is greater than half of the maximum horizontal grayscale projection value and is the minimum horizontal grayscale projection value, and at the same time determine the trough position i corresponding to the minimum grayscale projection value y(i) in the array locs_h[], and determine the horizontal coordinate of the subject's pupil position. S232, perform first-order difference on the vertical grayscale projection curve of the subject's eye image in S22 to find the position of the trough in the vertical grayscale projection curve of the subject's eye image, use array locs_v[] to record the position i of the trough, and use array pks_v[] to record the grayscale projection value x(j) corresponding to the trough position j; filter out the grayscale projection value x(j) in the grayscale projection value array pks_v[] that is greater than half of the maximum vertical grayscale projection value and is the minimum vertical grayscale projection value, and at the same time determine the trough position j corresponding to the minimum vertical grayscale projection value x(j) in the array locs_v[] to determine the vertical coordinate of the subject's pupil position.

4. A pupil recognition method according to claim 1, characterized in that: The interception step includes the following steps, According to the position information of the subject's pupil (i, j), Calculate the difference a=i-min_H between the horizontal coordinate i and the minimum value min_H of the non-zero area in the horizontal grayscale projection curve of the image of the subject's eye area; Calculate the difference b=max_H-i between the maximum value max_H of the non-zero area in the horizontal grayscale projection curve of the image of the subject's eye area and the horizontal coordinate i; Calculate the difference c = j - min_V between the ordinate j and the minimum value min_V of the non-zero region in the vertical gray projection curve of the eye part image of the subject; Calculate the difference d = max_V - j between the maximum value max_V of the non-zero region in the horizontal gray projection curve of the eye part image of the subject and the ordinate i; Calculate the minimum values min(a, b) of a and b respectively, and the minimum values min(c, d) of c and d. Assume that the pixel diameter of the largest pupil in the picture is D. If min(a, b) ≥ D and min(c, d) ≥ D, then the first effective region of the subject's pupil is intercepted as: [i - D, j - D, 2D, 2D]; If min(a, b) ≥ D and min(c, d) < D, then the first effective region of the subject's pupil is intercepted as: [i - D, min(c, d), 2D, 2 * min(c, d)]; If min(a, b) < D and min(c, d) ≥ D, then the first effective region of the subject's pupil is intercepted as: [min(a, b), j - D, 2 * min(a, b), 2D]; If min(a, b) < D and min(c, d) < D, then the first effective region of the subject's pupil is intercepted as: [min(a, b), min(c, d), 2 * min(a, b), 2 * min(c, d)].

5. A pupil recognition method according to claim 1, characterized in that: The separation step includes the following steps. S41. Calculate the average value of all pixel gray values in the first effective region according to the average value calculation formula. S42. Calculate the standard deviation of the pixel gray values in the first effective region according to the standard deviation formula. S43. Use the difference between the average value of the pixel gray values in the first effective region and the standard deviation of the pixel gray values in the first effective region as the gray threshold for threshold segmentation of the first effective region, and perform threshold segmentation on the first effective region to obtain the second effective region.

6. A pupil recognition method according to claim 1, characterized in that: The recognition step includes the following steps: Detect the contours in the second effective region image obtained by using the findContours function in opencv.

7. A pupil recognition method according to claim 1, characterized in that: The fitting step includes the following steps. S61. Determine the state of the subject's pupil by judging the relationship between the width W in the horizontal direction and the height H in the vertical direction of the pupil contour of the subject. If W ≠ H, then the state of the subject's pupil is an occluded state; On the contrary, the state of the subject's pupil is unoccluded. All the pupil contour point data of the unoccluded subject are valid data, and there are invalid data in the pupil contour point data of the occluded subject.剔除无效数据; S62. Calculate the best-fitting center coordinates and radius for the valid pupil contour point data of the subject obtained in step S61. S63. Determine the central position coordinates of the subject's pupil according to the center coordinates obtained in S62 and the starting coordinates of the first effective region intercepted in the interception step.

Citation Information

Patent Citations

  • Pupil positioning method and related device

    CN116797661A

  • Rapid pupil center positioning method for infrared image

    CN117830387A

  • Pre-processing method for iris image

    CN1885314A

  • Method for processing pupil tracking image

    US20200305705A1

Cited By

  • Focusing shooting method, focusing shooting system and multi-wavelength imaging device

    CN119922419A

  • Focusing photographing method, focusing photographing system, and multi-wavelength imaging device

    CN119922419B

  • Computer password protection system based on identity recognition

    CN119961910A