Pupil center detection method, device, equipment and storage medium

Abstract

The present application discloses a pupil center detection method, apparatus, device, and storage medium. The method includes: obtaining a facial image to be detected; determining an eye region based on the facial image to be detected; determining a pupil region based on the eye region; the pupil region includes a first pupil region and a second pupil region; the first pupil region and the second pupil region respectively correspond to the pupils of the two eyes in the facial image to be detected; if there is reflection in the first pupil region and there is no reflection in the second pupil region, determining the second pupil corresponding to the second pupil region; determining a brightness function based on the second pupil; determining an initial first pupil boundary corresponding to the first pupil region based on the brightness function; adjusting the initial first pupil boundary to obtain a target first pupil boundary; and determining a first pupil center corresponding to the first pupil region based on the target first pupil boundary. In this way, the influence of eyeglass reflection on pupil center positioning can be reduced, thereby improving the accuracy and reliability of pupil center positioning.

Description

Technical Field

[0001] The present application relates to the field of gaze tracking technology, and in particular to a pupil center method, apparatus, device, and storage medium. Background Art

[0002] Gaze tracking technology, also known as eye tracking technology, can use image processing technology to locate the pupil position, obtain the coordinates of the pupil center, calculate the eye's gaze point, and thus track eye movement. Gaze tracking technology can provide a more convenient, fast, and natural way for people and machines to interact, and has broad application prospects in the field of human-computer interaction. Currently, gaze tracking technology can be divided into two categories: contact and non-contact. Because contact gaze tracking technology devices, such as head-mounted eye trackers, contact lenses, and other devices, come into contact with the user, they are highly invasive, have low comfort levels, and have limited application scenarios. Therefore, research on gaze tracking technology mainly focuses on non-contact gaze tracking technology.

[0003] Typically, non-contact gaze tracking technology uses a camera to capture an eye image and then uses image processing techniques to determine the pupil center location. Prior art research on non-contact gaze tracking technology has mostly relied on near-eye images that are free of reflections. These images offer high resolution and low noise. However, in practice, eye images can be affected by reflections from glasses, which can severely impact image quality and lead to low pupil center location accuracy. Furthermore, in most cases, cameras are unable to capture eye images at close range, resulting in images taken from a distance, resulting in low resolution and high noise. Summary of the Invention

[0004] The embodiments of the present application provide a pupil center detection method, apparatus, device, and storage medium, which can reduce the impact of eyeglass reflections on pupil center positioning and improve the accuracy and reliability of pupil center positioning.

[0005] In one aspect, an embodiment of the present application provides a method for detecting a pupil center, the method comprising:

[0006] Get the face image to be detected;

[0007] determining an eye region based on the face image to be detected;

[0008] Determining a pupil area based on the eye area; the pupil area includes a first pupil area and a second pupil area; the first pupil area and the second pupil area respectively correspond to pupils of two eyes in the face image to be detected;

[0009] If there is reflection in the first pupil area and there is no reflection in the second pupil area, determining the second pupil corresponding to the second pupil area;

[0010] determining a brightness function based on the second pupil;

[0011] determining an initial first pupil boundary corresponding to the first pupil region based on the brightness function;

[0012] Adjusting the initial first pupil boundary to obtain a target first pupil boundary;

[0013] A first pupil center corresponding to the first pupil region is determined based on the target first pupil boundary.

[0014] Furthermore, determining the pupil area based on the eye area includes:

[0015] Determine the gradient data of each pixel in the eye area;

[0016] Based on the gradient data of each pixel point in the eye area, performing noise filtering on the eye area to obtain a filtered eye area and the gradient data of each pixel point in the filtered eye area;

[0017] determining gradient field response data for each pixel in the filtered eye area based on the gradient data for each pixel in the filtered eye area;

[0018] Determine the pixel point in the filtered eye area where the gradient field response data is greater than or equal to the preset gradient field response data as the initial pupil center;

[0019] The pupil area is determined based on the initial pupil center and preset range rules.

[0020] Furthermore, after determining the pupil area based on the eye area, if there is reflection in the first pupil area and there is no reflection in the second pupil area, before determining the second pupil corresponding to the second pupil area, the method further includes:

[0021] Determine the pixel value and gradient data of each pixel in the pupil area;

[0022] Based on the gradient data of each pixel in the pupil area, the pupil area is subjected to noise filtering to obtain a filtered pupil area and the gradient data of each pixel in the filtered pupil area;

[0023] determining the variance of the gradient data of the filtered pupil region;

[0024] Binarization is performed on the pixel points in the filtered pupil area, dividing the pixel points in the filtered pupil area into a first pixel point and a second pixel point; the pixel value of the first pixel point is a first preset pixel value, and the pixel value of the second pixel point is a fourth preset number of pixel point values;

[0025] If the variance of the gradient data of the filtered pupil area is greater than the preset variance, and the number of first pixel points is greater than the first preset number, it is determined that there is reflection in the pupil area; or, if the variance of the gradient data of the filtered pupil area is less than or equal to the preset variance, or the number of first pixel points is less than or equal to the first preset number, it is determined that there is no reflection in the pupil area.

[0026] Furthermore, the second pupil includes a second pupil boundary and a second pupil center; and determining the second pupil corresponding to the second pupil region includes:

[0027] Performing region growing processing on the second pupil region to obtain a second pupil mask;

[0028] Filtering the pixels in the second pupil area based on the second pupil mask to obtain initial second pupil pixels;

[0029] Performing connected domain statistics on the initial second pupil pixel points to obtain the number of pixels in multiple connected domains formed by the initial second pupil pixel points;

[0030] Determine a target second pupil pixel point based on a connected domain formed by the initial second pupil pixel point, the number of which is greater than or equal to a preset number of pixels;

[0031] Determine the second pupil boundary based on the target second pupil pixel point and parabolic interpolation method;

[0032] A second pupil center is determined based on the second pupil boundary.

[0033] Furthermore, determining a brightness function based on the second pupil includes:

[0034] Based on the second pupil boundary, determining a second preset number of boundary points;

[0035] Obtaining pixel values ​​of pixels that are separated from the boundary point by a third preset number of pixels and located at a preset position of the boundary point, and using pixel values ​​whose gradient data is greater than or equal to the first preset gradient data as the pixel values ​​of the boundary point;

[0036] A brightness function is determined based on the pixel values ​​of the boundary points.

[0037] Furthermore, determining an initial first pupil boundary corresponding to the first pupil area based on the brightness function includes:

[0038] Determining a first pixel point in the first pupil area as an iris pixel point;

[0039] Determining a first pupil pixel value based on the brightness function and the pixel value of the iris pixel point;

[0040] Determine a pixel whose distance from the iris pixel is within a fourth preset number of pixels and whose pixel value is the first pupil pixel value as the first pupil pixel;

[0041] Repeat the steps: determine the second pixel point in the first pupil area as the current iris pixel point; determine the first pupil pixel value based on the brightness function and the pixel value of the iris pixel point; determine the pixel point whose distance from the iris pixel point is within a fourth preset number of pixels and whose pixel value is the first pupil pixel value as the first pupil pixel point; until all pixels in the first pupil area are determined as iris pixels, and the initial first pupil boundary is obtained based on the first pupil pixel point.

[0042] Furthermore, the initial first pupil boundary is adjusted to obtain the target first pupil boundary, including:

[0043] Determine the pixel point at the center position corresponding to the initial first pupil boundary as the third pixel point; the third pixel point does not belong to the pixel points included in the initial first pupil boundary;

[0044] Determine the fourth pixel point among the pixels included in the initial first pupil boundary as the target pixel point;

[0045] Determine the direction from the third pixel point to the target pixel point as the moving direction;

[0046] Move the target pixel point along the moving direction until the gradient data of the target pixel point is greater than the second preset gradient data, and then stop moving;

[0047] Repeat the steps of: determining the fifth pixel point among the pixels included in the initial first pupil boundary as the current target pixel point; determining the direction from the third pixel point to the target pixel point as the moving direction; moving the target pixel point in the moving direction until the gradient data of the target pixel point is greater than the second preset gradient data, and then stopping the movement; until all the pixels included in the initial first pupil boundary are determined as target pixels;

[0048] The initial first pupil boundary after the position shift is determined as the target first pupil boundary.

[0049] On the other hand, an embodiment of the present application provides a pupil detection device, the device comprising:

[0050] An image acquisition module is used to acquire the face image to be detected;

[0051] An eye region determination module, configured to determine the eye region based on the face image to be detected;

[0052] A pupil region determination module is configured to determine a pupil region based on an eye region; the pupil region includes a first pupil region and a second pupil region; the first pupil region and the second pupil region respectively correspond to the pupils of the two eyes in the face image to be detected;

[0053] A second pupil determination module is configured to determine a second pupil corresponding to the second pupil area if there is reflection in the first pupil area and no reflection in the second pupil area;

[0054] a brightness function determination module, configured to determine a brightness function based on the second pupil;

[0055] an initial first pupil boundary determining module, configured to determine an initial first pupil boundary corresponding to the first pupil region based on a brightness function;

[0056] a target first pupil boundary determination module, configured to adjust the initial first pupil boundary to obtain a target first pupil boundary;

[0057] The first pupil center determination module is used to determine the first pupil center corresponding to the first pupil area based on the target first pupil boundary.

[0058] Furthermore, the pupil area determination module is used to determine the gradient data of each pixel in the eye area; based on the gradient data of each pixel in the eye area, the eye area is noise filtered to obtain the filtered eye area and the gradient data of each pixel in the filtered eye area; based on the gradient data of each pixel in the filtered eye area, the gradient field response data of each pixel in the filtered eye area is determined; the pixel in the filtered eye area whose gradient field response data is greater than or equal to the preset gradient field response data is determined as the initial pupil center; and the pupil area is determined based on the initial pupil center and the preset range rule.

[0059] Furthermore, the device further includes:

[0060] A reflection determination module is used to determine the pixel value and gradient data of each pixel in the pupil area; based on the gradient data of each pixel in the pupil area, perform noise filtering on the pupil area to obtain the filtered pupil area and the gradient data of each pixel in the filtered pupil area; determine the variance of the gradient data of the filtered pupil area; perform binarization processing on the pixels in the filtered pupil area to divide the pixels in the filtered pupil area into a first pixel and a second pixel; the pixel value of the first pixel is a first preset pixel value, and the pixel value of the second pixel is a fourth preset number of pixel values; if the variance of the gradient data of the filtered pupil area is greater than the preset variance, and the number of the first pixels is greater than the first preset number, it is determined that reflection exists in the pupil area; or if the variance of the gradient data of the filtered pupil area is less than or equal to the preset variance, or the number of the first pixels is less than or equal to the first preset number, it is determined that no reflection exists in the pupil area.

[0061] Further, the second pupil includes a second pupil boundary and a second pupil center;

[0062] The second pupil determination module is used to perform region growing processing on the second pupil area to obtain a second pupil mask; filter the pixels in the second pupil area based on the second pupil mask to obtain initial second pupil pixels; perform connected domain statistics on the initial second pupil pixels to obtain pixel values ​​of multiple connected domains formed by the initial second pupil pixels; determine the target second pupil pixel based on the connected domains formed by the initial second pupil pixels, whose pixel values ​​are greater than or equal to a third preset pixel value; determine the second pupil boundary based on the target second pupil pixel and the parabolic interpolation method; and determine the second pupil center based on the second pupil boundary.

[0063] Furthermore, the brightness function determination module is used to determine a second preset number of boundary points based on the second pupil boundary; obtain the pixel values ​​of pixel points that are separated from the boundary points by a third preset number of pixels and located at a preset position of the boundary points, and use the pixel values ​​whose gradient data in the pixel values ​​is greater than or equal to the first preset gradient data as the pixel values ​​of the boundary points; and determine the brightness function based on the pixel values ​​of the boundary points.

[0064] Furthermore, the initial first pupil boundary determination module is used to determine the first pixel point in the first pupil area as an iris pixel point; determine the first pupil pixel value based on the brightness function and the pixel value of the iris pixel point; determine the pixel point whose distance from the iris pixel point is within a fourth preset number of pixels and whose pixel value is the first pupil pixel value as the first pupil pixel point; repeat the steps of: determining the second pixel point in the first pupil area as the current iris pixel point; determining the first pupil pixel value based on the brightness function and the pixel value of the iris pixel point; determining the pixel point whose distance from the iris pixel point is within a fourth preset number of pixels and whose pixel value is the first pupil pixel value as the first pupil pixel point; until all the pixels in the first pupil area are determined as iris pixels, ellipse fitting is performed on the first pupil pixel point to obtain the initial first pupil boundary.

[0065] Furthermore, the first pupil boundary determination module is used to determine the pixel point at the center position corresponding to the initial first pupil boundary as the third pixel point; the third pixel point does not belong to the pixel points included in the initial first pupil boundary; determine the fourth pixel point among the pixel points included in the initial first pupil boundary as the target pixel point; determine the direction from the third pixel point to the target pixel point as the moving direction; move the target pixel point in the moving direction until the gradient data of the target pixel point is greater than the second preset gradient data, and stop moving; repeat the steps: determine the fifth pixel point among the pixel points included in the initial first pupil boundary as the current target pixel point; determine the direction from the third pixel point to the target pixel point as the moving direction; move the target pixel point in the moving direction until the gradient data of the target pixel point is greater than the second preset gradient data, and stop moving; until all the pixel points included in the initial first pupil boundary are determined as target pixel points; determine the initial first pupil boundary after position movement as the target first pupil boundary.

[0066] On the other hand, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded by the processor and executes the above pupil center detection method.

[0067] On the other hand, an embodiment of the present application provides a computer storage medium, which stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to implement the pupil center detection method as described above.

[0068] The pupil center detection method, apparatus, device, and storage medium provided in the embodiments of the present application have the following technical effects:

[0069] Acquire a facial image to be detected; determine an eye region based on the facial image to be detected; determine a pupil region based on the eye region; the pupil region includes a first pupil region and a second pupil region; the first pupil region and the second pupil region respectively correspond to the pupils of the two eyes in the facial image to be detected; if there is reflection in the first pupil region and there is no reflection in the second pupil region, determine the second pupil corresponding to the second pupil region; determine a brightness function based on the second pupil; determine an initial first pupil boundary corresponding to the first pupil region based on the brightness function; adjust the initial first pupil boundary to obtain a target first pupil boundary; and determine a first pupil center corresponding to the first pupil region based on the target first pupil boundary. In this way, the influence of eyeglass reflection on pupil center positioning can be reduced, thereby improving the accuracy and reliability of pupil center positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0071] Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present application;

[0072] Figure 2 1 is a flow chart of a pupil center detection method provided in an embodiment of the present application;

[0073] Figure 3 1 is a flow chart of a pupil center detection method provided in an embodiment of the present application;

[0074] Figure 4 1 is a flow chart of a pupil center detection method provided in an embodiment of the present application;

[0075] Figure 5 1 is a flow chart of a pupil center detection method provided in an embodiment of the present application;

[0076] Figure 6 1 is a flow chart of a pupil center detection method provided in an embodiment of the present application;

[0077] Figure 7 1 is a flow chart of a pupil center detection method provided in an embodiment of the present application;

[0078] Figure 8 1 is a flow chart of a pupil center detection method provided in an embodiment of the present application;

[0079] Figure 9This is a schematic diagram of target pixel adjustment for a pupil center detection method provided in an embodiment of the present application;

[0080] Figure 10 This is a practical effect diagram of a pupil center detection method provided by an embodiment of the present application;

[0081] Figure 11 This is a practical effect diagram of a pupil center detection method provided by an embodiment of the present application;

[0082] Figure 12 This is a practical effect diagram of a pupil center detection method provided by an embodiment of the present application;

[0083] Figure 13 This is a practical effect diagram of a pupil center detection method provided by an embodiment of the present application;

[0084] Figure 14 This is a practical effect diagram of a pupil center detection method provided by an embodiment of the present application.

[0085] Figure 15 1 is a schematic structural diagram of a pupil center detection device provided in an embodiment of the present application;

[0086] Figure 16 This is a hardware structure block diagram of a server for a pupil center detection method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0087] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0088] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0089] See also Figure 1 , Figure 1 This is a schematic diagram of an application environment provided by an embodiment of the present application, including a camera 101, a server 102 and a client 103, wherein the camera 101 is used to shoot a face image to be detected, the server 102 receives the face image to be detected shot by the camera 101 and processes it to obtain the pupil center of the eye in the face image to be detected, and the client 103 receives the pupil center of the eye in the face image to be detected obtained by the server 102 and performs corresponding operations based on these pupil centers.

[0090] Specifically, the server 102 obtains the face image to be detected captured by the camera 101; determines the eye area based on the face image to be detected; determines the pupil area based on the eye area, wherein the pupil area includes a first pupil area and a second pupil area, and the first pupil area and the second pupil area respectively correspond to the pupils of the two eyes in the face image to be detected; if there is reflection in the first pupil area and there is no reflection in the second pupil area, determines the second pupil corresponding to the second pupil area, and determines the brightness function based on the second pupil; determines the initial first pupil boundary corresponding to the first pupil area based on the brightness function; adjusts the initial first pupil boundary to obtain the target first pupil boundary; determines the first pupil center corresponding to the first pupil area based on the target first pupil boundary, and sends the detection result to the client 103.

[0091] In the embodiment of the present application, the camera 101 may be an infrared camera, and accordingly, the facial image to be detected is an infrared image. The camera 101 may also be other types of cameras.

[0092] In the embodiment of the present application, the server 102 may also be another computer terminal having the same functions as the server, or a similar computing device. Furthermore, the server 102 may be replaced by a server system, a computing platform, or a server cluster comprising multiple servers. The server 102 may be located on the device or terminal where the client 103 is located, or it may be a remote control center that uniformly receives the facial images to be detected sent by each camera 101, processes these facial images to be detected, and then sends the corresponding detection results to the corresponding client 103.

[0093] In the embodiment of the present application, the client 103 can be any device or terminal that uses eye tracking technology, such as a smart phone, AR / VR, smart driving car, or other device or terminal that can perform human-computer interaction.

[0094] The following describes a specific embodiment of a pupil center detection method of the present application. Figure 2It is a flowchart of a pupil center detection method provided in an embodiment of the present application. This specification provides method operation steps such as the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many steps, and does not represent the only execution order. When the actual system or server product is executed, it can be executed in sequence or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment). Specifically, as Figure 2 As shown, the method may include:

[0095] S201: Obtain a face image to be detected.

[0096] S203: Determine the eye region based on the face image to be detected.

[0097] In the embodiment of the present application, the eye area corresponds to the area where the eyes are located on the face of the face image to be detected.

[0098] As an optional implementation, one eye corresponds to one eye region. For a face image to be detected with two eyes, there is a first eye region and a second eye region.

[0099] As an optional implementation, the eye area recognition model can be trained in advance through deep learning based on the image training set. During detection, the face image to be detected is input into the eye area recognition model to obtain the eye area corresponding to the face image to be detected.

[0100] As an optional implementation, the eye region recognition model outputs the coordinates of the upper left corner of the rectangle corresponding to the eye region and the length and width of the rectangle. An eye region can be uniquely determined based on the coordinates of the upper left corner, length and width of the rectangle.

[0101] S205: Determine the pupil area based on the eye area.

[0102] In the embodiment of the present application, the pupil area corresponds to the area where the pupil is located in the eyes of the face image to be detected, that is, there is one pupil area corresponding to one eye area.

[0103] As an optional implementation, Figure 3 FIG. 1 is a flow chart of a pupil center detection method provided in an embodiment of the present application, such as Figure 3 As shown, step S205: determining the pupil area based on the eye area may include the following steps:

[0104] S301: Determine the gradient data of each pixel in the eye area.

[0105] In the embodiment of the present application, the gradient data includes the x-direction gradient value, the y-direction gradient value, the gradient amplitude and the gradient angle. The gradient data characterizes the pixel brightness, that is, the changing trend of the pixel value.

[0106] S303: Based on the gradient data of each pixel point in the eye area, noise filtering is performed on the eye area to obtain a filtered eye area and gradient data of each pixel point in the filtered eye area.

[0107] In the embodiment of the present application, considering that if the image to be detected is a long-distance image, its resolution is relatively low and the noise interference contained in the image is large, which will greatly affect the detection precision and accuracy, the eye area is noise filtered based on the gradient data of each pixel point in the eye area.

[0108] As an optional implementation, by setting a first high threshold and a first low threshold, pixel points corresponding to gradient data that are not between the first high threshold and the first low threshold are filtered out to perform noise filtering on the eye area.

[0109] As an optional implementation, the first high threshold value may be set to a gradient amplitude of 0.9, and the first low threshold value may be set to a gradient amplitude of 0.55.

[0110] As an optional embodiment, when performing noise filtering on the eye region, the pixel values ​​corresponding to the gradient data that is not between the first high threshold and the first low threshold are set to a first preset value, and the pixel values ​​corresponding to the gradient data that is between the first high threshold and the first low threshold are set to a second preset value. Non-maximum suppression is performed on the pixel whose value is the first preset value. Specifically, the pixel with the largest gradient amplitude is selected along the gradient direction, centered at the pixel, and the pixel is replaced.

[0111] As an optional implementation, the first preset value may be 255, indicating that the pixel is white; the second preset value may be 0, indicating that the pixel is black.

[0112] S305: Determine gradient field response data of each pixel point in the filtered eye area based on the gradient data of each pixel point in the filtered eye area.

[0113] S307: Determine the pixel point in the filtered eye area whose gradient field response data is greater than or equal to the preset gradient field response data as the initial pupil center.

[0114] In the embodiment of the present application, since the gradient field response data of the pupil center is the largest, the gradient field response data of each pixel point in the filtered eye area is calculated, and the pixel point whose gradient field response data is equal to the preset gradient field response data is taken as the initial pupil center, and the pupil center is preliminarily roughly located.

[0115] As an optional implementation manner, the preset gradient field response data is the maximum data among the gradient field response data of all pixel points in the eye area after filtering.

[0116] S309: Determine the pupil area based on the initial pupil center and preset range rules.

[0117] In the embodiment of the present application, based on the initial pupil center and the preset range rule, the approximate range of the pupil, that is, the pupil area, is first determined, and then the pupil center is further finely positioned based on the pupil area.

[0118] As an optional implementation, the preset range rule may be to use the coordinates of the initial pupil center as the center point coordinates to form a rectangle with a preset length and a preset width.

[0119] As an optional implementation, in the preset range rule, the preset length and the preset width can be set to a length of 40 pixels.

[0120] As an optional implementation, in the preset range rule, the preset length and the preset width can be adjusted according to actual conditions and are not limited to the above-mentioned lengths.

[0121] As an optional implementation, Figure 4 FIG. 1 is a flow chart of a pupil center detection method provided in an embodiment of the present application, such as Figure 4 As shown, in step S205: after determining the pupil area based on the eye area, the process further includes determining whether there is reflection in the pupil area, which may specifically include the following steps:

[0122] S401: Determine the pixel value and gradient data of each pixel point in the pupil area.

[0123] S403: Based on the gradient data of each pixel point in the pupil area, noise filtering is performed on the pupil area to obtain a filtered pupil area and the gradient data of each pixel point in the filtered pupil area.

[0124] In the embodiment of the present application, by setting a second high threshold and a second low threshold, pixel points corresponding to the gradient data that are not between the second high threshold and the second low threshold are filtered out to perform noise filtering on the pupil area.

[0125] As an optional implementation, the second high threshold value may be set to any value greater than or equal to the first high threshold value, and the second low threshold value may be set to any value less than or equal to the first low threshold value.

[0126] As an optional implementation, the second high threshold value may be set to a gradient amplitude of 0.9, and the second low threshold value may be set to a gradient amplitude of 0.55.

[0127] S405: Determine the variance of the gradient data of the filtered pupil area.

[0128] In this embodiment of the present application, the variance of the gradient data of the filtered pupil area includes the variance of the x-direction gradient value and the y-direction gradient value. The variance of the x-direction gradient value and the y-direction gradient value of the filtered pupil area are calculated separately, and the variance of the x-direction gradient value and the y-direction gradient value of the filtered pupil area is used as one of the conditions for determining the presence of reflection in the pupil area.

[0129] S407: Binarization is performed on the pixel points in the filtered pupil area, and the pixel points in the filtered pupil area are divided into first pixel points and second pixel points.

[0130] In the embodiment of the present application, based on a binarized threshold segmentation method, the pixels in the filtered pupil area are divided into two parts according to the threshold, namely, first pixels and second pixels. The first pixels are pixels with larger pixel values, representing the white background in the pupil area, while the second pixels are pixels with smaller pixel values, representing the pupil and iris targets in the pupil area, thereby separating the targets in the pupil area from the background.

[0131] As an optional implementation, the pixel values ​​of the first pixel points are set to the first preset pixel values, and the pixel values ​​of the second pixel points are set to the fourth preset number of pixel point values.

[0132] As an optional implementation, the maximum inter-class variance method can be used for binarization, and a segmentation threshold can be set to divide the pixels in the filtered pupil area into a first pixel and a second pixel. The inter-class variance of the segmented pupil area is calculated to be the largest. When the inter-class variance is , it indicates that the threshold segmentation at this time can fully distinguish the target from the iris, that is, the pupil and the iris, and the probability of pixel misclassification is minimized. At this time, the pixel value of the pixel classified as the first pixel is set to a first preset pixel value, and the pixel value of the pixel classified as the second pixel is set to a fourth preset number of pixel values.

[0133] S409: Determine whether there is reflection in the pupil area based on the variance of the gradient data of the filtered pupil area and the number of first pixels.

[0134] In the embodiment of the present application, the variance of the gradient data of the filtered pupil area and the number of first pixels are used as two conditions for determining pupil area reflectivity. A greater variance of the gradient data of the filtered pupil area indicates a greater difference in pixel values ​​among the pixels in the pupil area, and a greater number of pixels with significantly different pixel values ​​from pupil-related pixels, i.e., white reflective pixels. A greater number of first pixels indicates a greater number of white reflective pixels in the pupil area.

[0135] As an optional embodiment, if the variance of the gradient data of the filtered pupil area is greater than the preset variance, and the number of first pixel points is greater than the first preset number, it is determined that there is reflection in the pupil area; or, if the variance of the gradient data of the filtered pupil area is less than or equal to the preset variance, or the number of first pixel points is less than or equal to the first preset number, it is determined that there is no reflection in the pupil area.

[0136] As an optional implementation, the preset variance may be set to 25 for both the x-direction gradient value and the y-direction gradient value, and the first preset number may be set to 180 pixels.

[0137] As an optional embodiment, for the first eye region and the second eye region, the pupil region includes a first pupil region and a second pupil region, and the first pupil region and the second pupil region respectively correspond to the pupils of the two eyes in the face image to be detected. The first pupil region corresponds to the first pupil of the first eye region, and the second pupil region corresponds to the second pupil of the second eye region.

[0138] As an optional implementation, for the first eye area and the second eye area, the above steps S203 and S205 are respectively performed to obtain the corresponding first pupil area and second pupil area, and it is determined whether there is reflection in the first pupil area and the second pupil area.

[0139] S207: If there is reflection in the first pupil area and there is no reflection in the second pupil area, determine the second pupil corresponding to the second pupil area.

[0140] In the embodiment of the present application, due to the presence of reflections in the first pupil area, the reflective pixels will cause great interference to the pupil detection in the first pupil area. Therefore, the first pupil cannot be determined directly based on the first pupil area, but it is necessary to first determine the second pupil corresponding to the second pupil area, and then infer the first pupil from the second pupil.

[0141] In the embodiment of the present application, the pupil center includes the pupil boundary and the pupil center. Then, the first pupil includes the first pupil boundary and the first pupil center, and the second pupil includes the second pupil boundary and the second pupil center.

[0142] As an optional implementation, Figure 5 FIG. 1 is a flow chart of a pupil center detection method provided in an embodiment of the present application, such as Figure 5 As shown, determining the second pupil corresponding to the second pupil area may specifically include the following steps:

[0143] S501: Performing region growing processing on the second pupil region to obtain a second pupil mask.

[0144] In the embodiment of the present application, image segmentation is performed on the second pupil region based on region growing, and a second pupil mask is obtained based on the second pupil region after image segmentation.

[0145] As an optional implementation, based on the segmentation threshold used in step S407 above, a pixel value histogram is determined for pixels with values ​​between 0 and the segmentation threshold, i.e., the second pixel value. The pixel value corresponding to the maximum peak in the histogram is obtained from the pixel value histogram and determined as the first threshold t1. Using the maximum inter-class variance method, a second threshold t2 is determined based on pixel values ​​between 0 and t1. The resulting growth threshold t is the average of the first threshold t1 and the second threshold t2.

[0146] As an optional implementation, the average value of the pixel values ​​of the pixels located at the edge of the second pupil area is calculated, and the difference between the average value and the growth threshold t is set as the growth interval.

[0147] As an optional implementation, the row and column where the initial second pupil center is located in the second pupil area are set as growth seed points.

[0148] As an optional embodiment, based on the aforementioned growth threshold t, growth interval, and growth seed point, region growing is performed, and image segmentation is performed on the second pupil region. The portion covered by the region growth is determined as the first region, and the portion not covered by the region growth is determined as the second region. In the second pupil region, the pixel values ​​of the pixels in the first region are set to a first preset pixel value, and the pixel values ​​of the pixels in the second region are set to a fourth preset number of pixel values. The resulting image is determined as the second pupil mask.

[0149] S503: Filtering the pixels in the second pupil area based on the second pupil mask to obtain initial second pupil pixels.

[0150] In the embodiment of the present application, the second pupil mask is associated with pixels in the second pupil region in a one-to-one correspondence. Pixels in the second pupil region corresponding to pixels in the second pupil mask with pixel values ​​of a first preset value are retained, while pixels in the second pupil region corresponding to pixels in the second pupil mask with pixel values ​​of a second preset value are discarded. The retained pixels in the second pupil region are determined as initial second pixels.

[0151] As an optional implementation, before filtering the pixels in the second pupil region based on the second pupil mask, the pixels in the second pupil region may be further refined. A third high threshold and a third low threshold are set to filter the pixels in the second pupil region. The average pixel value of the pixels located at the edge of the second pupil region is then calculated, and pixels with values ​​greater than the average are discarded.

[0152] As an optional implementation, the above-mentioned refinement process is performed at least once.

[0153] As an optional implementation manner, the third high threshold value may be set to any value greater than or equal to the second high threshold value, and the third low threshold value may be set to any value less than or equal to the second low threshold value.

[0154] As an optional implementation, the third high threshold value may be set to a gradient amplitude of 0.9, and the third low threshold value may be set to a gradient amplitude of 0.4.

[0155] S505: Perform connected domain statistics on the initial second pupil pixel points to obtain the number of pixel points in multiple connected domains formed by the initial second pupil pixel points.

[0156] In this embodiment of the present application, multiple connected domains formed by the initial second pupil pixels are counted, and the number of pixels contained in these connected domains is calculated. In image processing, a connected domain generally refers to adjacent foreground pixels in an image with the same pixel value. That is, in this embodiment of the present application, a connected domain is a pixel region formed by multiple adjacent pixels in the initial second pupil pixels.

[0157] S507: Determine a target second pupil pixel point based on a connected domain formed by the initial second pupil pixel point, the number of which is greater than or equal to a preset number of pixels.

[0158] In the embodiment of the present application, since the image corresponding to the pupil should be a completely circular black area, theoretically, the connected domain corresponding to the pupil should be a circular pixel area composed of multiple adjacent pixels with pixel values ​​of 0 or smaller. Therefore, for the initial second pupil pixels, the pixels contained in the connected domain with a small number of pixels are not the second pupil pixels. Therefore, the pixels contained in the connected domain with a number of pixels less than the preset number of pixels in the initial second pupil pixels are discarded, while the pixels contained in the connected domain with a number of pixels greater than or equal to the preset number of pixels are retained, and the target second pupil pixels are obtained based on this.

[0159] S509: Determine the second pupil boundary based on the target second pupil pixel point and parabolic interpolation method.

[0160] In the embodiment of the present application, the second pupil boundary can be obtained based on the pixel points at the edge position of the target second pupil pixel points. However, since it is composed only of pixels, the smallest unit is a pixel, and the accuracy is not high enough. Therefore, sub-pixels are introduced to divide each pixel into more similar units to improve accuracy. Based on the concept of sub-pixels, parabolic interpolation is used to supplement the sub-pixels between the pixel points at the edge position of the target second pupil pixel points to obtain the second pupil boundary, thereby improving the accuracy and reliability of the second pupil boundary. At the same time, it can also improve the accuracy and reliability of the second pupil center determined based on the second pupil boundary, thereby improving the accuracy and reliability of pupil center detection.

[0161] S511: Determine a second pupil center based on the second pupil boundary.

[0162] In the embodiment of the present application, based on the ellipse fitting method, an ellipse corresponding to the second pupil boundary is fitted, and the center point of the fitted ellipse is used as the second pupil center.

[0163] As an optional implementation, in the ellipse fitting process, in order to make the fitted conic curve finally become an ellipse and ensure the linear least squares, a discriminator (b 2 -4ac) is a negative value. At the same time, considering that the Kuhn-Tucker condition cannot be completely solved, it is impossible to find the parameters that minimize the quadratic form of the non-convex inequality, so the parameters are scaled and the discriminator (b 2 -4ac) is set equal to -1 to approximate the minimization of the quadratic form subject to the non-convex inequality.

[0164] S209: Determine a brightness function based on the second pupil.

[0165] In the embodiment of the present application, due to the presence of reflections in the first pupil region, the first pupil cannot be directly derived from the first pupil region. Since the brightness information relationship between the pupil and iris in the eye region, that is, the pixel value relationship, should be consistent for different eyes in the same face image to be detected, a brightness function is constructed based on the pixel values ​​of the pupil position and the iris position in the second pupil, and the first pupil is inferred in the first pupil region based on the brightness function.

[0166] As an optional implementation, Figure 6 FIG. 1 is a flow chart of a pupil center detection method provided in an embodiment of the present application, such as Figure 6 As shown, step S209: determining the brightness function based on the second pupil may include the following steps:

[0167] S601: Determine a second preset number of boundary points based on the second pupil boundary.

[0168] In the embodiment of the present application, a second preset number of boundary points on the second pupil boundary are selected, and pixel values ​​of the pupil and iris positions are obtained based on these boundary points.

[0169] As an optional implementation, the second preset number of boundary points are evenly distributed on the boundary of the second pupil to avoid the positions of the selected boundary points being too concentrated, resulting in unrepresentative data and affecting the detection of the first pupil.

[0170] S603: Obtain pixel values ​​of pixels that are separated from the boundary point by a third preset number of pixels and located at a preset position of the boundary point, and use pixel values ​​whose gradient data is greater than or equal to the first preset gradient data as the pixel values ​​of the boundary point.

[0171] In an embodiment of the present application, taking into account the brightness changes at the junction of the pupil and the iris, that is, taking into account the gradient of the boundary points, for these selected boundary points, the pixel values ​​of the pixel points that are separated from them by a third preset number of pixels and located at their preset positions are obtained, and the pixel values ​​whose gradient data among these pixel values ​​is greater than or equal to the first preset gradient data are used as the pixel values ​​of the boundary points.

[0172] As an optional implementation, the third preset number is 3, and the preset directions are at least four directions: up, down, left, and right.

[0173] As an optional implementation, if a pixel point that is separated from a boundary point by a third preset number of pixels and located at a preset position of the boundary point, as well as a boundary point, falls on a non-pupil or iris area such as the corneal reflection surface or eyelid, the pixel point will be discarded and its pixel value will not be stored.

[0174] As an optional embodiment, the pixel value of the boundary point includes a pupil pixel value and an iris pixel value. If the boundary point is located at the intersection of the pupil and the iris, then among the pixels at the preset location, at least one pixel is located in the pupil region and at least one pixel is located in the iris region. Based on this, a pupil pixel value and an iris pixel value can be determined respectively and used as the pixel value of the boundary point.

[0175] As an optional implementation, it is possible to determine whether the area in which the pixel point is located is the pupil area or the iris area based on the pixel value of the pixel point at the preset orientation of the boundary point or the gradient data between the pixel point and the corresponding boundary point. Specifically, the color of the pupil area is usually darker than the color of the iris area, and the corresponding pixel value of the pixel point located in the pupil area is usually smaller than the pixel value of the pixel point in the iris area. By comparing the pixel values ​​of these pixels, it is possible to determine in which area these pixels are located. For the same reason, the pixel value change trend between the pixel point located in the pupil area and the boundary point should be different from the pixel value change trend between the pixel point located in the iris area and the boundary point, and the corresponding gradient data also have differences. By comparing them, it is possible to determine in which area these pixels are located.

[0176] S605: Determine a brightness function based on the pixel values ​​of the boundary points.

[0177] In an embodiment of the present application, based on the pixel values ​​of the boundary points, i.e., their corresponding pupil pixel values ​​and iris pixel values, a first-order function fitting is used to obtain a first-order function of the pupil pixel value with respect to the iris pixel value, i.e., a brightness function.

[0178] S211: Determine an initial first pupil boundary corresponding to the first pupil area based on the brightness function.

[0179] In an embodiment of the present application, based on the brightness function, the first pupil pixel point in the first pupil area is extracted. The first pupil pixel point is the pixel point in the first pupil area that may correspond to the first pupil. The initial first pupil boundary is determined by the first pupil pixel point. The initial first pupil boundary is a roughly positioned first pupil boundary, and there is a certain deviation between it and the first pupil, which needs to be adjusted subsequently.

[0180] As an optional implementation, Figure 7 FIG. 1 is a flow chart of a pupil center detection method provided in an embodiment of the present application, such as Figure 7 As shown, step S211: determining an initial first pupil boundary corresponding to the first pupil area based on a brightness function may include the following steps:

[0181] S701: Determine a first pixel point in a first pupil area as an iris pixel point.

[0182] In the embodiment of the present application, the first pixel point is any point in the first pupil area.

[0183] S703: Determine a first pupil pixel value based on the brightness function and the pixel value of the iris pixel point.

[0184] S705: Determine a pixel whose distance from the iris pixel is within a fourth preset number of pixels and whose pixel value is the first pupil pixel value as the first pupil pixel.

[0185] In the embodiment of the present application, within a range of a fourth preset number of pixels away from the iris pixel point, a pixel point having a pixel value equal to the first pupil pixel value is searched and determined as the first pupil pixel point.

[0186] S707: Repeat steps: determine the second pixel point in the first pupil area as the current iris pixel point; determine the first pupil pixel value based on the brightness function and the pixel value of the iris pixel point; determine the pixel point whose distance from the iris pixel point is within a fourth preset number of pixels and whose pixel value is the first pupil pixel value as the first pupil pixel point; until all pixels in the first pupil area are determined as iris pixels, and the initial first pupil boundary is obtained based on the first pupil pixel point.

[0187] In the embodiment of the present application, all pixels in the first pupil region are traversed to find all pixels that may correspond to the first pupil, i.e., all first pupil pixels. Among the found first pupil pixels, first pupil pixels that form a circular arrangement are searched. If the arrangement of these first pupil pixels is consistent with the shape of the first pupil, these first pupil pixels can be considered to correspond to the first pupil. Pixels at the edge of these first pupil pixels are determined as pixels of the initial first pupil boundary, thereby obtaining the initial first pupil boundary.

[0188] S213: Adjust the initial first pupil boundary to obtain a target first pupil boundary.

[0189] In an embodiment of the present application, the initial first pupil boundary is a boundary calculated based on the brightness function between the pupil pixel value and the iris pixel value in the second pupil. There is a certain difference between it and the actual first pupil boundary. Therefore, it is adjusted based on the initial first pupil boundary to obtain the target first pupil boundary. The target first pupil boundary is the first pupil boundary finally detected.

[0190] As an optional implementation, Figure 8 FIG. 1 is a flow chart of a pupil center detection method provided in an embodiment of the present application, such as Figure 8 As shown, step S213: adjusting the initial first pupil boundary to obtain the target first pupil boundary may include the following steps:

[0191] S801: Determine the pixel point at the center position corresponding to the initial first pupil boundary as the third pixel point.

[0192] In the embodiment of the present application, the third pixel point is not included in the pixel points included in the initial first pupil boundary. The third pixel point is equivalent to the center point of the ellipse corresponding to the initial first pupil boundary.

[0193] S803: Determine the fourth pixel point among the pixels included in the initial first pupil boundary as the target pixel point.

[0194] S805: Determine the direction from the third pixel point to the target pixel point as the moving direction.

[0195] S807: Move the target pixel point along the moving direction until the gradient data of the target pixel point is greater than the second preset gradient data, and then stop moving.

[0196] In the embodiment of the present application, the target pixel moves along the moving direction, and moves one pixel at a time. In other words, the target pixel moves along a unit vector that is consistent with the moving direction.

[0197] In an embodiment of the present application, the target pixel point can move outward along the moving direction based on the current position, that is, the target pixel point moves along the moving direction to the outside of the initial first pupil boundary; the target pixel point can also move inward along the direction opposite to the moving direction based on the current position, that is, the target pixel point moves in the opposite direction along the moving direction to the inside of the initial first pupil boundary.

[0198] In an embodiment of the present application, the second preset gradient data can be the largest gradient data among all the gradient data of the target pixel point, so that when the movement is finally stopped, the gradient data of the target pixel point is the largest, which is consistent with the maximum gradient data at the pupil-iris boundary, so that when the movement is finally stopped, the target pixel point falls exactly on the actual pupil boundary.

[0199] S809: Repeat the steps: determine the fifth pixel point among the pixels included in the initial first pupil boundary as the current target pixel point; determine the direction from the third pixel point to the target pixel point as the moving direction; move the target pixel point in the moving direction until the gradient data of the target pixel point is greater than the second preset gradient data, and stop moving; until all the pixels included in the initial first pupil boundary are determined as target pixel points.

[0200] In the embodiment of the present application, all pixel points on the initial first pupil boundary are moved and adjusted so that when the movement finally stops, all pixel points fall exactly on the actual pupil boundary.

[0201] As an optional implementation, Figure 9 FIG. 4 shows a schematic diagram of target pixel adjustment of a pupil center detection method provided by an embodiment of the present application. Figure 9As shown in the figure, the square area represents the first pupil area, the dark gray circular area represents the actual location of the first pupil, the gray ring area represents the location of the iris corresponding to the first pupil, the elliptical curve represents the initial first pupil boundary, and the center point of the elliptical curve represents the third pixel corresponding to the initial first pupil boundary. Pixels on the initial first pupil boundary are moved one pixel at a time in the direction from the third pixel to the pixel until the gradient data at the pixel is greater than or equal to the second preset gradient data, corresponding to the actual location of the first pupil.

[0202] S811: Determine the initial first pupil boundary after the position is moved as the target first pupil boundary.

[0203] S215: Determine a first pupil center corresponding to the first pupil area based on the target first pupil boundary.

[0204] In an embodiment of the present application, based on ellipse fitting, the center point of the ellipse corresponding to the target first pupil boundary is determined, and the center point is used as the first pupil center.

[0205] As an optional implementation, Figure 10-14 This is a practical effect diagram of a pupil center detection method provided by an embodiment of the present application, such as Figure 10-14 As shown, there is no reflection in the left eye, but there is reflection in the right eye. Figure 10 In order to determine the actual effect diagram of the eye area and pupil area from the face image to be detected, the left and right eyes have an eye area and a pupil area respectively, where the outer rectangle corresponds to the eye area and the inner rectangle corresponds to the pupil area. Figure 11 This is the actual effect of the pupil and eye mask corresponding to the non-reflective eye on the left. Figure 12 This is the actual effect diagram of the pupil boundary of the non-reflective eye on the left. The pupil boundary of the non-reflective eye finally obtained can well match the actual pupil boundary. Figure 13 This is the actual effect diagram of the pupil boundary of the reflective eye on the right. The pupil boundary of the reflective eye finally obtained can basically match the actual pupil boundary. Figure 14 This is an actual rendering of the pupil center of the right reflective eye. The final located pupil of the reflective eye is highly consistent with the actual pupil center. This shows that the pupil center detection method provided by the embodiment of the present application can accurately locate the pupil center of the reflective eye when processing the face image to be detected.

[0206] The present application also provides a pupil center detection device. Figure 15 : is a structural diagram of a pupil center detection device provided in an embodiment of the present application, such as Figure 15 As shown, the device includes:

[0207] Image acquisition module 1501, used to acquire a face image to be detected;

[0208] An eye region determination module 1502 is configured to determine an eye region based on a face image to be detected;

[0209] The pupil region determining module 1503 is configured to determine a pupil region based on the eye region; the pupil region includes a first pupil region and a second pupil region; the first pupil region and the second pupil region respectively correspond to the pupils of the two eyes in the face image to be detected;

[0210] A second pupil determination module 1504 is configured to determine a second pupil corresponding to the second pupil region if there is reflection in the first pupil region and there is no reflection in the second pupil region;

[0211] a brightness function determining module 1505, configured to determine a brightness function based on the second pupil;

[0212] An initial first pupil boundary determining module 1506 is configured to determine an initial first pupil boundary corresponding to the first pupil region based on a brightness function;

[0213] A target first pupil boundary determination module 1507 is configured to adjust the initial first pupil boundary to obtain a target first pupil boundary;

[0214] The first pupil center determining module 1508 is configured to determine a first pupil center corresponding to the first pupil region based on the target first pupil boundary.

[0215] As an optional embodiment, the pupil area determination module is used to determine the gradient data of each pixel in the eye area; based on the gradient data of each pixel in the eye area, perform noise filtering on the eye area to obtain a filtered eye area and gradient data of each pixel in the filtered eye area; based on the gradient data of each pixel in the filtered eye area, determine the gradient field response data of each pixel in the filtered eye area; determine the pixel in the filtered eye area whose gradient field response data is greater than or equal to the preset gradient field response data as the initial pupil center; and determine the pupil area based on the initial pupil center and a preset range rule.

[0216] As an optional embodiment, the device further includes:

[0217] A reflection determination module is used to determine the pixel value and gradient data of each pixel in the pupil area; based on the gradient data of each pixel in the pupil area, perform noise filtering on the pupil area to obtain the filtered pupil area and the gradient data of each pixel in the filtered pupil area; determine the variance of the gradient data of the filtered pupil area; perform binarization processing on the pixels in the filtered pupil area to divide the pixels in the filtered pupil area into a first pixel and a second pixel; the pixel value of the first pixel is a first preset pixel value, and the pixel value of the second pixel is a fourth preset number of pixel values; if the variance of the gradient data of the filtered pupil area is greater than the preset variance, and the number of the first pixels is greater than the first preset number, it is determined that reflection exists in the pupil area; or if the variance of the gradient data of the filtered pupil area is less than or equal to the preset variance, or the number of the first pixels is less than or equal to the first preset number, it is determined that no reflection exists in the pupil area.

[0218] As an optional implementation, the second pupil includes a second pupil boundary and a second pupil center;

[0219] The second pupil determination module is used to perform region growing processing on the second pupil area to obtain a second pupil mask; filter the pixels in the second pupil area based on the second pupil mask to obtain initial second pupil pixels; perform connected domain statistics on the initial second pupil pixels to obtain the number of pixels in multiple connected domains formed by the initial second pupil pixels; determine the target second pupil pixel based on the connected domains formed by the initial second pupil pixels, where the number of pixels is greater than or equal to the preset number of pixels; determine the second pupil boundary based on the target second pupil pixel and the parabolic interpolation method; and determine the second pupil center based on the second pupil boundary.

[0220] As an optional embodiment, the brightness function determination module is used to determine a second preset number of boundary points based on the second pupil boundary; obtain the pixel values ​​of pixel points that are separated from the boundary points by a third preset number of pixels and located at a preset position of the boundary points, and use the pixel values ​​whose gradient data in the pixel values ​​is greater than or equal to the first preset gradient data as the pixel values ​​of the boundary points; and determine the brightness function based on the pixel values ​​of the boundary points.

[0221] As an optional implementation, an initial first pupil boundary determination module is used to determine the first pixel point in the first pupil area as an iris pixel point; determine the first pupil pixel value based on a brightness function and the pixel value of the iris pixel point; determine the pixel point whose distance from the iris pixel point is within a fourth preset number of pixels and whose pixel value is the first pupil pixel value as the first pupil pixel point; repeat the steps of: determining the second pixel point in the first pupil area as the current iris pixel point; determining the first pupil pixel value based on a brightness function and the pixel value of the iris pixel point; determining the pixel point whose distance from the iris pixel point is within a fourth preset number of pixels and whose pixel value is the first pupil pixel value as the first pupil pixel point; until all pixels in the first pupil area are determined as iris pixels, and the initial first pupil boundary is obtained based on the first pupil pixel point.

[0222] As an optional implementation, the first pupil boundary determination module is used to determine the pixel point at the center position corresponding to the initial first pupil boundary as the third pixel point; the third pixel point does not belong to the pixel points included in the initial first pupil boundary; determine the fourth pixel point among the pixel points included in the initial first pupil boundary as the target pixel point; determine the direction from the third pixel point to the target pixel point as the moving direction; move the target pixel point in the moving direction until the gradient data of the target pixel point is greater than the second preset gradient data, and stop moving; repeat the steps: determine the fifth pixel point among the pixel points included in the initial first pupil boundary as the current target pixel point; determine the direction from the third pixel point to the target pixel point as the moving direction; move the target pixel point in the moving direction until the gradient data of the target pixel point is greater than the second preset gradient data, and stop moving; until all the pixel points included in the initial first pupil boundary are determined as target pixel points; determine the initial first pupil boundary after position movement as the target first pupil boundary.

[0223] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Taking running on a server as an example, Figure 16 This is a hardware structure diagram of a server for pupil center detection method provided in an embodiment of the present application. Figure 16As shown, the server 1600 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPUs) 1610 (the processor 1610 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 1630 for storing data, and one or more storage media 1620 (such as one or more mass storage devices) for storing application programs 1623 or data 1622. Among them, the memory 1630 and the storage medium 1620 can be temporary storage or permanent storage. The program stored in the storage medium 1620 may include one or more modules, each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1610 can be configured to communicate with the storage medium 1620 to execute a series of instruction operations in the storage medium 1620 on the server 1600. The server 1600 may also include one or more power supplies 1660, one or more wired or wireless network interfaces 1650, one or more input and output interfaces 1640, and / or one or more operating systems 1621, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, etc.

[0224] The input / output interface 1640 can be used to receive or send data via a network. Specific examples of the aforementioned network may include a wireless network provided by the communication provider of the server 1600. In one embodiment, the input / output interface 1640 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one embodiment, the input / output interface 1640 can be a radio frequency (RF) module for wirelessly communicating with the Internet.

[0225] It can be understood by those skilled in the art that Figure 16 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 16 More or fewer components than shown, or with Figure 16 Different configurations shown.

[0226] An embodiment of the present application also provides a pupil center detection device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement a pupil center detection method.

[0227] An embodiment of the present application also provides a storage medium, which can be set in a server to store at least one instruction, at least one program, code set or instruction set related to a pupil center detection method in a method embodiment. The at least one instruction, the at least one program, the code set or instruction set is loaded and executed by the processor to implement the pupil center detection method provided by the above method embodiment.

[0228] Optionally, in this embodiment, the storage medium may be located in at least one of a plurality of network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0229] As can be seen from the embodiments of the pupil center detection method, apparatus, device, and storage medium provided by the present application, the present application obtains a face image to be detected; determines an eye region based on the face image to be detected; determines a pupil region based on the eye region; the pupil region includes a first pupil region and a second pupil region; the first pupil region and the second pupil region respectively correspond to the pupils of the two eyes in the face image to be detected; if there is reflection in the first pupil region and there is no reflection in the second pupil region, determines the second pupil corresponding to the second pupil region; determines a brightness function based on the second pupil; determines an initial first pupil boundary corresponding to the first pupil region based on the brightness function; adjusts the initial first pupil boundary to obtain a target first pupil boundary; and determines a first pupil center corresponding to the first pupil region based on the target first pupil boundary. In this way, the influence of eyeglass reflection on pupil center positioning can be reduced, thereby improving the accuracy and reliability of pupil center positioning.

[0230] It should be noted that the order of the embodiments of the present application described above is for descriptive purposes only and does not represent the superiority or inferiority of the embodiments. The above description is of specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0231] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the device embodiments are generally similar to the method embodiments, so the description is relatively simple. For relevant parts, refer to the description of the method embodiments.

[0232] Those skilled in the art will understand that all or part of the steps of implementing the above embodiments can be completed by hardware, or can be completed by a program to instruct the relevant hardware, and the program can be stored in a computer-readable storage medium, and the above-mentioned storage medium can be a read-only memory, a disk or an optical disk, etc.

[0233] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

Claims

1. A pupil center detection method, characterized in that: The method comprises: Get the face image to be detected; determining an eye region based on the face image to be detected; Determining a pupil area based on the eye area; the pupil area includes a first pupil area and a second pupil area; the first pupil area and the second pupil area respectively correspond to pupils of two eyes in the face image to be detected; If there is light reflection in the first pupil area and there is no light reflection in the second pupil area, determining a second pupil corresponding to the second pupil area; the second pupil includes a second pupil boundary; determining a brightness function based on the second pupil; determining an initial first pupil boundary corresponding to the first pupil area based on the brightness function; Adjusting the initial first pupil boundary to obtain a target first pupil boundary; Determining a first pupil center corresponding to the first pupil area based on the target first pupil boundary; The determining of a brightness function based on the second pupil comprises: determining a second preset number of boundary points based on the second pupil boundary; Obtaining pixel values ​​of pixels that are separated from the boundary point by a third preset number of pixels and located at a preset position of the boundary point, and using pixel values ​​whose gradient data is greater than or equal to the first preset gradient data as the pixel values ​​of the boundary point; The brightness function is determined based on the pixel values ​​of the boundary points.

2. The pupil center detection method according to claim 1, characterized in that: The determining the pupil area based on the eye area includes: Determining gradient data of each pixel point in the eye area; performing noise filtering on the eye region based on the gradient data of each pixel in the eye region to obtain a filtered eye region and the gradient data of each pixel in the filtered eye region; determining gradient field response data of each pixel point in the filtered eye area based on the gradient data of each pixel point in the filtered eye area; Determining a pixel in the filtered eye region where the gradient field response data is greater than or equal to a preset gradient field response data as an initial pupil center; The pupil area is determined based on the initial pupil center and a preset range rule.

3. The pupil center detection method according to claim 1, wherein: After determining the pupil area based on the eye area, if there is reflection in the first pupil area and there is no reflection in the second pupil area, and before determining the second pupil corresponding to the second pupil area, the method further includes: Determining pixel value and gradient data of each pixel point in the pupil area; performing noise filtering on the pupil area based on the gradient data of each pixel in the pupil area to obtain a filtered pupil area and the gradient data of each pixel in the filtered pupil area; determining a variance of the gradient data of the filtered pupil region; Binarization is performed on the pixel points in the filtered pupil area to divide the pixel points in the filtered pupil area into a first pixel point and a second pixel point; the pixel value of the first pixel point is a first preset pixel value, and the pixel value of the second pixel point is a fourth preset number of pixel point values; If the variance of the gradient data of the filtered pupil area is greater than the preset variance, and the number of the first pixel points is greater than the first preset number, it is determined that there is reflection in the pupil area; or, if the variance of the gradient data of the filtered pupil area is less than or equal to the preset variance, or the number of the first pixel points is less than or equal to the first preset number, it is determined that there is no reflection in the pupil area.

4. The pupil center detection method according to claim 1, wherein: The second pupil includes a second pupil boundary and a second pupil center; and determining the second pupil corresponding to the second pupil area includes: performing region growing processing on the second pupil region to obtain a second pupil mask; Filtering the pixels in the second pupil area based on the second pupil mask to obtain initial second pupil pixels; Performing connected domain statistics on the initial second pupil pixel points to obtain the number of pixel points in multiple connected domains formed by the initial second pupil pixel points; Determining a target second pupil pixel point based on a connected domain formed by the initial second pupil pixel point, the number of the pixel points being greater than or equal to a preset number of pixel points; Determining the second pupil boundary based on the target second pupil pixel point and parabolic interpolation method; The second pupil center is determined based on the second pupil boundary.

5. The pupil center detection method according to claim 1, characterized in that: The determining an initial first pupil boundary corresponding to the first pupil area based on the brightness function includes: Determining a first pixel point in the first pupil area as an iris pixel point; determining a first pupil pixel value based on the brightness function and the pixel value of the iris pixel point; Determine a pixel whose distance from the iris pixel is within a fourth preset number of pixels and whose pixel value is the first pupil pixel value as the first pupil pixel; Repeat the steps of: determining the second pixel point in the first pupil area as the current iris pixel point; determining the first pupil pixel value based on the brightness function and the pixel value of the iris pixel point; determining the pixel point whose distance from the iris pixel point is within the fourth preset number of pixels and whose pixel value is the first pupil pixel value as the first pupil pixel point; until all pixels in the first pupil area are determined as the iris pixel points, and obtaining the initial first pupil boundary based on the first pupil pixel point.

6. The pupil center detection method according to claim 1, characterized in that: The adjusting the initial first pupil boundary to obtain a target first pupil boundary includes: Determining a pixel point at a center position corresponding to the initial first pupil boundary as a third pixel point; the third pixel point does not belong to the pixel points included in the initial first pupil boundary; Determining a fourth pixel point among the pixels included in the initial first pupil boundary as a target pixel point; Determine the direction from the third pixel point to the target pixel point as the moving direction; Moving the target pixel point along the moving direction until the gradient data of the target pixel point is greater than the second preset gradient data, and then stopping the movement; Repeat the steps of: determining the fifth pixel point among the pixels included in the initial first pupil boundary as the current target pixel point; determining the direction from the third pixel point to the target pixel point as the movement direction; moving the target pixel point in the movement direction until the gradient data of the target pixel point is greater than the second preset gradient data, and then stopping the movement; until all the pixels included in the initial first pupil boundary are determined as the target pixel point; The initial first pupil boundary after the position shift is determined as the target first pupil boundary.

7. A pupil detection device, characterized in that: The device comprises: An image acquisition module is used to acquire the face image to be detected; An eye region determination module, configured to determine an eye region based on the face image to be detected; a pupil region determining module, configured to determine a pupil region based on the eye region; the pupil region comprising a first pupil region and a second pupil region; the first pupil region and the second pupil region respectively corresponding to the pupils of the two eyes in the face image to be detected; a second pupil determination module, configured to determine a second pupil corresponding to the second pupil area if there is reflection in the first pupil area and no reflection in the second pupil area; the second pupil includes a second pupil boundary; a brightness function determining module, configured to determine a brightness function based on the second pupil; an initial first pupil boundary determining module, configured to determine an initial first pupil boundary corresponding to the first pupil area based on the brightness function; a target first pupil boundary determination module, configured to adjust the initial first pupil boundary to obtain a target first pupil boundary; A first pupil center determining module, configured to determine a first pupil center corresponding to the first pupil area based on the target first pupil boundary; The determining of a brightness function based on the second pupil comprises: determining a second preset number of boundary points based on the second pupil boundary; Obtaining pixel values ​​of pixels that are separated from the boundary point by a third preset number of pixels and located at a preset position of the boundary point, and using pixel values ​​whose gradient data is greater than or equal to the first preset gradient data as the pixel values ​​of the boundary point; The brightness function is determined based on the pixel values ​​of the boundary points.

8. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded by the processor and executes the pupil center detection method according to any one of claims 1-6.

9. A computer storage medium, characterized in that The computer storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the pupil center detection method according to any one of claims 1 to 6.

Images