Pupil center positioning method, device, equipment and storage medium

Abstract

The embodiments of the present invention disclose a pupil center positioning method, apparatus, device and storage medium. The method comprises: performing coarse positioning processing on an acquired original eye image to obtain a coarse pupil positioning position of the eye, and obtaining a target eye image containing the coarse pupil positioning position; in the target eye image, using the coarse pupil positioning position as a seed point, adopting at least one set step size to perform region expansion to obtain at least one candidate pupil region of the eye; performing a screening operation on each candidate pupil region, selecting a target pupil region, and determining the target pupil center position of the eye based on the target pupil region, thereby solving the problem of low accuracy caused by setting a fixed threshold for detecting the pupil center positioning position, screening out the target pupil region from the candidate pupil regions, improving the accuracy of determining the target pupil region, and the determined target pupil region is not affected by pupil shape and light, is suitable for images under complex backgrounds, and has high robustness.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of eye tracking technology, and in particular to a pupil center positioning method, apparatus, device, and storage medium. Background Art

[0002] With the rapid development of society, gaze tracking technology has been widely used in real life. Gaze tracking technology uses eye movement to estimate gaze direction or gaze placement. Its application has permeated various areas of human life. For example, in the field of human-computer interaction, eye tracking technology has become an important means of interaction between humans and machines. People with disabilities can use eye tracking technology to perform functions such as typing. In the transportation field, driver fatigue can be determined by eye movement or pupil changes. In the field of virtual reality, gaze tracking technology can achieve scene gaze point rendering, improving the user experience. Pupil center detection is a core component of gaze tracking technology. Existing gaze tracking technologies first need to determine the location of the pupil center, based on which the gaze direction can be calculated.

[0003] Existing pupil center positioning algorithms generally perform binarization operations on images based on a fixed threshold. Due to many factors such as irregular pupil shape, uneven lighting, and large changes in viewing angle in images actually taken against complex backgrounds, the fixed threshold method has low applicability and robustness in images against complex backgrounds. Inaccurate screening may occur when determining the pupil area, resulting in low pupil center positioning accuracy and other shortcomings. Summary of the Invention

[0004] The present invention provides a pupil center positioning method, device, equipment and storage medium to achieve accurate screening of the target pupil area and improve the accuracy of pupil positioning.

[0005] In a first aspect, an embodiment of the present invention provides a pupil center positioning method, the pupil center positioning method comprising:

[0006] Performing coarse positioning processing on the acquired original eye image to obtain a coarse pupil positioning position of the eye, and obtaining a target eye image containing the coarse pupil positioning position;

[0007] In the target eye image, using the pupil coarse location as a seed point, performing region expansion with at least one set step size to obtain at least one candidate pupil region of the eye;

[0008] A screening operation is performed on each of the candidate pupil regions to select a target pupil region, and a target pupil center position of the eye is determined based on the target pupil region.

[0009] In a second aspect, an embodiment of the present invention further provides a pupil center locating device, the pupil center locating device comprising:

[0010] a coarse positioning position determination module, configured to perform coarse positioning processing on the acquired original eye image, obtain a coarse positioning position of the pupil of the eye, and obtain a target eye image containing the coarse positioning position of the pupil;

[0011] a candidate region determination module, configured to, in the target eye image, use the pupil coarse location as a seed point and perform region expansion using at least one set step size to obtain at least one candidate pupil region of the eye;

[0012] The pupil center determination module is used to screen the candidate pupil areas, select a target pupil area, and determine the target pupil center position of the eye based on the target pupil area.

[0013] In a third aspect, an embodiment of the present invention further provides a device, comprising:

[0014] one or more processors;

[0015] a storage device for storing one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement a pupil center positioning method as described in any one of the embodiments of the present invention.

[0017] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a pupil center positioning method as described in any one of the embodiments of the present invention.

[0018] Embodiments of the present invention provide a pupil center positioning method, apparatus, device and storage medium, which obtains the pupil coarse positioning position of the eye by performing coarse positioning processing on the acquired original eye image, and obtains a target eye image containing the pupil coarse positioning position; in the target eye image, the pupil coarse positioning position is used as a seed point, and at least one set step size is used to perform area expansion to obtain at least one candidate pupil area of the eye; a screening operation is performed on each of the candidate pupil areas, a target pupil area is selected, and the target pupil center position of the eye is determined based on the target pupil area, thereby solving the problem of low accuracy caused by setting a fixed threshold for detecting the pupil center positioning position, and performing pupil coarse positioning on the original eye image, and then performing area expansion based on the coarse positioning position to determine each candidate pupil area, and screening out the target pupil area from the candidate pupil area, thereby improving the accuracy of determining the target pupil area, and the determined target pupil area is not affected by pupil shape and light, is suitable for images under complex backgrounds, and has high robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flow chart of a pupil center positioning method in Example 1 of the present invention;

[0020] Figure 2 This is an example diagram of an original eye image in a pupil center location method in the first embodiment of the present invention;

[0021] Figure 3 This is a flow chart of a pupil center positioning method in Embodiment 2 of the present invention;

[0022] Figure 4 This is an example diagram of a mask area in a pupil center location method in the second embodiment of the present invention;

[0023] Figure 5 This is an example diagram of a target region of interest in a pupil center positioning method in the second embodiment of the present invention;

[0024] Figure 6 This is an example diagram of a target eye image in a pupil center positioning method in the second embodiment of the present invention;

[0025] Figure 7 exemplifying each candidate pupil region in a pupil center location method in the second embodiment of the present invention;

[0026] Figure 8 This is a schematic structural diagram of a pupil center positioning device in Embodiment 3 of the present invention;

[0027] Figure 9 This is a structural diagram of a device in embodiment 4 of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0029] Example 1

[0030] Figure 1 This is a flowchart of a pupil center location method provided in Example 1 of the present invention. This embodiment is applicable to situations where the pupil center position is determined. The method can be performed by a pupil center location device and specifically includes the following steps:

[0031] Step S110 , performing coarse positioning processing on the acquired original eye image to obtain the coarse positioning position of the pupil of the eye, and obtaining a target eye image including the coarse positioning position of the pupil.

[0032] In this embodiment, the original eye image can be understood as an image of the user including the eyes captured by a camera. For example, Figure 2 Provided is an example diagram of an original eye image in a pupil center positioning method. In the diagram, A1 is the portion of the original eye image that does not include the user's eyes. When capturing the user's eye image, due to limitations of the device itself, part of the captured original eye image is the user's eye image, and the other part is the occluded area of the device. A2 is an image that includes the user's eyes, which may be the user's upper and lower eyelids and the user's eyes. A3 is the user's eyes. A4 is the user's pupils. The coarse pupil positioning position can be understood as the position obtained by predicting the pupil center based on the eye image. The target eye image can be understood as the eye image that includes the coarse pupil positioning position, which may be the original eye image or an eye image processed from the original eye image.

[0033] The user is photographed using a camera, such as an infrared camera, to obtain an original eye image and send it to a computer processing unit. The computer processing unit performs corresponding image processing on the original eye image, such as filtering, denoising, and other image processing methods to remove factors that affect the accuracy of the original eye image. The processed eye image is then coarsely located using methods such as convolution to obtain the coarse pupil location. The coarse pupil location is mapped to the original eye image to obtain a target eye image containing the coarse pupil location, or the processed eye image containing the coarse pupil location is used as the target eye image. The coarse location of the original eye image can also be performed by selecting the point with the minimum grayscale value in the eye from the original eye image as the coarse pupil location. When determining the coarse pupil location using the original eye image, it is also possible to directly perform coarse location without performing any image processing.

[0034] Step S120 : In the target eye image, using the roughly located pupil position as a seed point, performing region expansion with at least one set step size to obtain at least one candidate pupil region of the eye.

[0035] In this embodiment, region expansion is achieved through a region growing algorithm, starting from a point and expanding to form a region. The region growing algorithm is the process of growing groups of pixels or regions into larger regions. Starting from a set of seed points, regions are grown from these points by merging adjacent pixels with similar attributes to each seed point, such as intensity, grayscale, texture, color, etc., into this region. The set step size can be understood as a pre-set step size for region growing, which can be 1, 2, 3, etc.; the candidate pupil regions can be understood as the regions containing the pupil obtained by region growing.

[0036] In the target eye image, the pupil's coarse location is used as a seed point, and region expansion is performed by setting a step size to obtain a candidate pupil region. When performing region expansion, multiple set step sizes can be used and the region expansion can be performed multiple times to obtain multiple candidate pupil regions.

[0037] Step S130 : screening the candidate pupil regions, selecting a target pupil region, and determining the target pupil center position of the eye based on the target pupil region.

[0038] In this embodiment, the target pupil area can be understood as the final pupil area selected from multiple candidate pupil areas; the target pupil center position can be understood as the final pupil center position of the user's eye.

[0039] The multiple candidate pupil regions obtained through region expansion are screened to select the fullest candidate pupil region that best matches the actual pupil region as the target pupil region. The target pupil center position of the eye is ultimately determined based on the target pupil region. The target pupil region can be selected by comparing the pixel counts of the candidate pupil regions, the ratio of the actual area to the theoretical area, the perimeter of the circumscribed rectangle, the center of mass, and other methods. The target pupil center position can be determined using methods such as center of mass and ellipse fitting.

[0040] An embodiment of the present invention provides a pupil center positioning method, which performs coarse positioning processing on the acquired original eye image to obtain the coarse pupil positioning position of the eye, and obtains a target eye image containing the coarse pupil positioning position; in the target eye image, the coarse pupil positioning position is used as a seed point, and at least one set step size is used to perform area expansion to obtain at least one candidate pupil area of the eye; a screening operation is performed on each of the candidate pupil areas, a target pupil area is selected, and the target pupil center position of the eye is determined based on the target pupil area, thereby solving the problem of low accuracy caused by setting a fixed threshold for detecting the pupil center positioning position. By performing coarse pupil positioning on the original eye image, and then performing area expansion based on the coarse positioning position to determine each candidate pupil area, the target pupil area is screened out from the candidate pupil area, thereby improving the accuracy of determining the target pupil area. The determined target pupil area is not affected by pupil shape and light, is suitable for images under complex backgrounds, and has high robustness.

[0041] Example 2

[0042] Figure 3 This is a flowchart of a pupil center positioning method provided in Example 2 of the present invention. The technical solution of this embodiment is further refined on the basis of the above technical solution, and specifically includes the following steps:

[0043] Step S210: downsample and perform low-value filtering on the original eye image to obtain an intermediate image to be processed.

[0044] In this embodiment, the intermediate image to be processed can be understood as an image obtained by processing the original eye image and removing the effects of noise and the like.

[0045] The original eye image is downsampled to obtain a reduced image. The sampling frequency can be set according to the needs and is not limited in the embodiment of the present application. After the downsampling process, low-value filtering is performed to obtain an intermediate image to be processed with noise such as highlights removed.

[0046] Downsampling reduces the size of the original eye image, increasing computational speed and thus efficiency. When the camera captures the user's original eye image, there may be highlights due to reflections from the cornea and other factors. Low-value filtering removes these highlights and improves image processing accuracy.

[0047] Step S220 : determining the target region of interest based on the grayscale values of each region of the intermediate image to be processed and the grayscale threshold.

[0048] In this embodiment, the regional grayscale value can be understood as the grayscale value of each pixel in the image to be processed; the grayscale threshold is pre-set according to the actual application scenario; and the target region of interest can be understood as the region containing the entire eye image.

[0049] The grayscale threshold is used to determine whether the pixel corresponding to the grayscale value of the region is an eye image. The target region of interest is determined based on all the pixels that make up the eye image. In other words, the target region of interest is obtained by removing most of the non-eye image portion of the intermediate image to be processed.

[0050] Optionally, determining the target region of interest based on the grayscale values of each region of the intermediate image to be processed in combination with a grayscale threshold can be implemented as follows:

[0051] Determine whether the grayscale value of each area is less than the grayscale threshold. If so, determine the area formed by the pixel points corresponding to the grayscale value of each area as the mask area; determine the coordinate value of each pixel point in the mask area; determine the target area of interest based on the maximum horizontal coordinate, minimum horizontal coordinate, maximum vertical coordinate and minimum vertical coordinate of each coordinate value.

[0052] In this embodiment, the mask area can be understood as the area in the intermediate image to be processed that only contains the eye image. Determine whether the regional grayscale value corresponding to each pixel point is less than the grayscale threshold. If it is less than the grayscale threshold, this pixel point is the eye image, and an area is formed by all the pixel points corresponding to the regional grayscale value less than the grayscale threshold, and this area is determined as the mask area. A rectangle is determined based on the maximum horizontal coordinate a1, the minimum horizontal coordinate a2, the maximum vertical coordinate b1 and the minimum vertical coordinate b2 in the coordinate values of each pixel point in the mask area. This rectangle is the target region of interest, that is, the target region of interest is the circumscribed rectangle of the mask area. For example, Figure 4 An example diagram of a mask area in a pupil center positioning method is provided, where A1 is the portion of the original eye image that does not include the user's eyes; A2 is the mask area, which only includes the image of the user's eyes. Figure 4 The four points a1, a2, b1, and b2 are the maximum horizontal coordinate, minimum horizontal coordinate, maximum vertical coordinate, and minimum vertical coordinate of the coordinate values of each pixel point, respectively. Figure 5 An example diagram of a target region of interest in a pupil center positioning method is provided, in which B1 is the portion of the target region of interest that does not include the user's eyes; B2 is a mask area that only includes an image of the user's eyes; B3 is the user's eyes; and B4 is the user's pupil.

[0053] Step S230: performing a convolution operation on the target region of interest and a preset pupil template to determine a rough pupil location of the eye.

[0054] In this embodiment, the preset pupil template can be understood as a pre-set convolution template. The target region of interest is convolved with the preset pupil template to obtain an extreme value, which is used as the coarse pupil location. For example, the convolution operation uses an integral image, and the minimum value is used as the coarse pupil location. Determining the coarse pupil location through convolution operation on the integral image can improve computational efficiency. The size of the integral image is set or selected according to the pupil size. For example, the core of the integral image is selected as 17×17 pixels, and the outer core is selected as 23×23 pixels.

[0055] Step S240: Map the pupil coarse location to the original eye image to obtain a regression image.

[0056] In this embodiment, the regression image can be understood as the original eye image containing the coarse pupil location. The coarse pupil location is determined based on the downsampled original eye image. Directly performing region expansion based on the downsampled original eye image would result in certain errors. Therefore, the coarse pupil location is mapped to the corresponding position in the original eye image to obtain a regression image, and region expansion is performed based on the obtained regression image. Mapping the coarse pupil location to the original eye image to obtain the regression image can eliminate the error caused by downsampling and reduce errors.

[0057] Step S250 , intercepting the regression image according to a set ratio, and performing low-value filtering on the intercepted image to obtain a target eye image containing a rough pupil location.

[0058] The method of intercepting the regression image according to the set ratio can be to use the rough pupil location as the reference point, expand a certain distance to the left, right, upward, and downward respectively, and intercept the regression image; or to use the rough pupil location as the center of mass, expand a certain size rectangle and intercept the regression image. The intercepted image is low-value filtered to remove the highlights in the image, and the target eye image containing the rough pupil location is obtained. For example, Figure 6 An example diagram of a target eye image in a pupil center positioning method is provided, where C1 represents the upper and lower eyelids and other parts of the target eye image; C2 represents the user's eye; C3 represents the user's pupil; and C4 represents the coarse positioning position of the pupil.

[0059] Step S260: In the target eye image, using the roughly located pupil position as a seed point, performing region expansion with at least one set step size to obtain at least one candidate pupil region of the eye.

[0060] Step S270: Select at least one preset screening rule from the preset rule set, and use each preset screening rule to screen corresponding candidate target areas from each candidate pupil area.

[0061] In this embodiment, the preset rule set can be understood as a set that includes at least one preset filtering rule, and the preset filtering rule can be filtering by pixel points, area ratio, perimeter and center of mass; the candidate target area can be understood as the candidate area among the candidate pupil areas that can be used as the target pupil area.

[0062] For example, Figure 7 An example diagram of candidate pupil regions in a pupil center localization method is provided. The diagram shows 13 candidate pupil regions obtained by sequentially expanding the region with set step sizes increasing from 1 to 13. In the diagram, D1 represents the upper and lower eyelids, etc., of the target eye image; D2 represents the user's eye; D3 represents the user's pupil; and D4 represents the candidate pupil region. The candidate target region screening method is illustrated using four preset screening rules as an example: Using the first screening rule, candidate target regions 9, 10, and 11 are determined; using the second screening rule, candidate target regions 10, 11, and 12 are determined; using the third screening rule, candidate target regions 10, 11, 12, and 13 are determined; and using the fourth screening rule, candidate target regions 10 and 11 are determined (the number n represents the nth candidate pupil region, and each candidate pupil region corresponds to a set step size).

[0063] Optionally, using a preset screening rule to screen each target area from each candidate pupil area can be implemented in the following manner:

[0064] When the preset screening rule is pixel point screening, the pixel point increase value of each candidate pupil area is determined, and the set of candidate pupil areas whose corresponding pixel point increase value is less than the first preset threshold is determined; when the number of areas in the candidate pupil area set is greater than the first preset number, each candidate pupil area in the candidate pupil area set is set as the target area.

[0065] In this embodiment, pixel screening can be understood as screening by pixel; the first preset threshold value can be 20 pixels, 30 pixels, etc. The number of regions can be understood as the number of candidate pupil regions included in the candidate pupil region set; the first preset number can be 1, 2, 3, etc. The candidate pupil region set can be understood as a set that includes at least one candidate pupil region.

[0066] When the preset screening rule is pixel screening, the pixels of each candidate pupil area are determined, and then the pixel increase value between two adjacent candidate pupil areas is determined. It is determined whether the pixel increase value is less than a first preset threshold. If so, a candidate pupil area set is formed based on the candidate pupil areas corresponding to the pixel increase values less than the first preset threshold. It is determined whether the number of areas in the candidate pupil area set is greater than a first preset number. If so, each candidate pupil area in the candidate pupil area set is determined as a target area.

[0067] When the region is expanded according to the set step size in the target eye image, the set step size is 1-13 as an example, and a candidate pupil region is obtained when the region is expanded for each set step size. For example, Figure 7 For example, when forming a set of candidate pupil regions, each candidate pupil region corresponding to a pixel increase value less than a first preset threshold value can be considered as a region in the candidate pupil region set, or each candidate pupil region corresponding to a pixel increase value less than the first preset threshold value for n consecutive times can be considered as a region in the candidate pupil region set. For example, if the first preset number of times is 3, and the pixel increase values of the 10th, 11th, 12th, and 13th candidate pupil regions are less than the first preset threshold value, the 10th, 11th, 12th, and 13th candidate pupil regions are selected as target regions.

[0068] Optionally, using a preset screening rule to screen each target area from each candidate pupil area can be implemented in the following manner:

[0069] When the preset screening rule is area ratio screening, the circumscribed polygon corresponding to each candidate pupil area is determined, and the maximum inscribed area of the circumscribed polygon is calculated as the theoretical area; the area ratio is determined based on the area of the candidate pupil area and the corresponding theoretical area, and a set of candidate pupil areas whose corresponding area ratios are greater than a second preset threshold is determined; when the number of areas in the candidate pupil area set is greater than the second preset number, each candidate pupil area in the candidate pupil area set is determined as a target area.

[0070] In this embodiment, area ratio screening can be understood as screening by calculating the ratio of the actual area to the theoretical area; the circumscribed polygon can be a circumscribed quadrilateral, a circumscribed pentagon, etc., and the maximum inscribed area can be the area of the inscribed ellipse of the circumscribed quadrilateral. Preferably, the second preset threshold can be 0.95.

[0071] When the preset screening rule is area ratio screening, the circumscribed polygon corresponding to each candidate pupil region is determined and the maximum inscribed area of the circumscribed polygon is calculated, which is used as the theoretical area of the candidate pupil region. The ratio of the area of the candidate pupil region to the corresponding theoretical area is calculated and used as the area ratio. It is determined whether the area ratio of each candidate pupil region is greater than a second preset threshold. If so, a candidate pupil region set is formed based on the candidate pupil regions corresponding to each area ratio greater than the second preset threshold. It is determined whether the number of regions in the candidate pupil region set is greater than a second preset number. If so, each candidate pupil region in the candidate pupil region set is determined as a target region.

[0072] Optionally, using a preset screening rule to screen each target area from each candidate pupil area can be implemented in the following manner:

[0073] When the preset screening rule is perimeter screening, the circumscribed rectangle corresponding to each candidate pupil area is determined, the perimeter change is determined based on the perimeter of each circumscribed rectangle, and a set of candidate pupil areas whose corresponding perimeter change is less than a third preset threshold is determined; when the number of areas in the candidate pupil area set is greater than the third preset number, each candidate pupil area in the candidate pupil area set is defined as the target area.

[0074] In this embodiment, perimeter screening can be understood as screening by comparing the perimeters of the circumscribed rectangles of the candidate pupil regions.

[0075] When the preset screening rule is perimeter screening, the circumscribed rectangle corresponding to each candidate pupil area is determined and the perimeter of each circumscribed rectangle is calculated. The perimeter change corresponding to each candidate pupil area is determined based on the perimeters of two adjacent circumscribed rectangles. It is determined whether the perimeter change of each candidate pupil area is less than a third preset threshold. If so, a candidate pupil area set is formed based on the candidate pupil areas corresponding to each perimeter change less than the third preset threshold. It is determined whether the number of areas in the candidate pupil area set is greater than a third preset number. If so, each candidate pupil area in the candidate pupil area set is determined as a target area.

[0076] Optionally, using a preset screening rule to screen each target area from each candidate pupil area can be implemented in the following manner:

[0077] When the preset screening rule is centroid screening, the centroid of each candidate pupil area is determined, and the offset between adjacent centroids is determined, and a set of candidate pupil areas whose corresponding offsets are less than a fourth preset threshold is determined; when the number of areas in the candidate pupil area set is greater than the fourth preset number, each candidate pupil area in the candidate pupil area set is defined as a target area.

[0078] In this embodiment, centroid screening can be understood as screening by the centroid of the candidate pupil regions.

[0079] When the preset screening rule is centroid screening, the centroid of each candidate pupil area and the offset between adjacent centroids are determined, and it is determined whether the offset of each candidate pupil area is less than a fourth preset threshold. If so, a candidate pupil area set is formed based on the candidate pupil areas corresponding to each offset less than the fourth preset threshold. It is determined whether the number of areas in the candidate pupil area set is greater than a fourth preset number. If so, each candidate pupil area in the candidate pupil area set is determined as a target area.

[0080] In the embodiment of the present application, the first preset number, the second preset number, the third preset number and the fourth preset number may be the same or different, and the embodiment of the present application does not make any specific limitation on this.

[0081] Step S280: Determine the region with the highest number of appearances among the candidate target regions as the pre-selected region.

[0082] In this embodiment, the pre-selected region can be understood as a region selected from the candidate target regions for final determination of the target pupil region. Taking the exemplary example selected in step S270 as an example, the candidate target regions are 9, 10, 11, 12, and 13, where 9 appears once, 10 appears four times, 11 appears four times, 12 appears two times, and 13 appears once. The regions with the highest number of appearances are 10 and 11. Therefore, the 10th and 11th candidate pupil regions are selected as the pre-selected regions.

[0083] Step S290: Determine whether the number of pre-selected areas is one. If so, execute step S291; otherwise, execute step S292.

[0084] Step S291: Determine the preselected area as the target pupil area, and execute step S293.

[0085] Step S292: Determine the pre-selected area with the largest step length as the target pupil area.

[0086] If there is only one pre-selected area, the only pre-selected area is used as the target pupil area. If there is not one pre-selected area, the area with the largest step size among the pre-selected areas is determined as the target pupil area. For example, the pre-selected areas are the 10th candidate pupil area and the 11th candidate pupil area, the step size of the 10th candidate pupil area is set to 10, and the step size of the 11th candidate pupil area is set to 11, and the 11th candidate pupil area is used as the target pupil area.

[0087] Step S293: extracting pupil edge points of the target pupil area using a preset edge extraction algorithm.

[0088] In this embodiment, the preset edge extraction algorithm may be a Laplacian operator, a Sobel operator, a Robert operator, a Marr operator, a Canny operator, a Shen-Castan operator, etc. The image containing the target pupil area is processed by the preset edge extraction algorithm to extract each pupil edge point of the target pupil area.

[0089] Step S294: Perform a convex hull operation on each pupil edge point to obtain the pupil edge area.

[0090] For a plane point set or a polygon, the convex hull refers to the smallest convex figure or smallest convex region that contains it. Convex hull calculation methods can be Graham scanning method, Jarvis step method, etc. The pupil edge region corresponding to each pupil edge point is obtained through convex hull calculation.

[0091] After determining the pupil edge point, the pupil edge area is obtained through convex hull operation, without considering whether the pupil shape is regular. This avoids the problem of low accuracy caused by irregular pupil shape when fitting an ellipse through an ellipse fitting algorithm and taking the ellipse center as the target pupil center position.

[0092] Step S295: Determine the centroid of the pupil edge area as the target pupil center position of the eye.

[0093] The embodiment of the present invention provides a pupil center positioning method, which solves the problem of low accuracy caused by setting a fixed threshold to detect the pupil center positioning position. By performing coarse pupil positioning on the original eye image, and then performing regional expansion based on the coarse positioning position to determine each candidate pupil area, the target pupil area is screened out from the candidate pupil area, thereby improving the accuracy of determining the target pupil area. The determined target pupil area is not affected by pupil shape and light, is suitable for images under complex backgrounds, and has high robustness. The size of the original eye image is reduced by downsampling processing, the calculation speed is increased, and thus the efficiency is improved. The highlights in the original eye image are removed by low-value filtering, thereby improving the image processing accuracy. Mapping the coarse pupil positioning position to the original eye image to obtain a regression image can eliminate the error value caused by downsampling and reduce the error.

[0094] Example 3

[0095] Figure 8 This is a structural diagram of a pupil center positioning device provided in Embodiment 3 of the present invention. The device includes: a coarse positioning position determination module 31, a candidate area determination module 32 and a pupil center determination module 33.

[0096] Among them, the coarse positioning position determination module is used to perform coarse positioning processing on the acquired original eye image, obtain the coarse positioning position of the pupil of the eye, and obtain a target eye image containing the coarse positioning position of the pupil; the candidate area determination module is used to use the coarse positioning position of the pupil as a seed point in the target eye image, adopt at least one set step size to perform area expansion, and obtain at least one candidate pupil area of the eye; the pupil center determination module is used to perform screening operations on each of the candidate pupil areas, select the target pupil area, and determine the target pupil center position of the eye based on the target pupil area.

[0097] An embodiment of the present invention provides a pupil center positioning device, which solves the problem of low accuracy caused by setting a fixed threshold to detect the pupil center positioning position. By performing rough pupil positioning on the original eye image, and then expanding the area according to the rough positioning position to determine each candidate pupil area, the target pupil area is screened out from the candidate pupil area, thereby improving the accuracy of determining the target pupil area. The determined target pupil area is not affected by pupil shape and light, is suitable for images under complex backgrounds, and has high robustness.

[0098] Furthermore, the coarse positioning position determination module 31 includes:

[0099] An image processing unit, configured to perform downsampling and low-value filtering on the original eye image to obtain an intermediate image to be processed;

[0100] A region determination unit, configured to determine a target region of interest based on the grayscale value of each region of the intermediate image to be processed in combination with a grayscale threshold;

[0101] A position determination unit is used to perform a convolution operation on the target region of interest and a preset pupil template to determine a rough positioning position of the pupil of the eye.

[0102] Furthermore, the area determination unit is specifically used to: determine whether the grayscale value of each area is less than the grayscale threshold, and if so, determine the area formed by the pixel points corresponding to the grayscale value of each area as the mask area; determine the coordinate value of each pixel point in the mask area; and determine the target area of interest based on the maximum horizontal coordinate, minimum horizontal coordinate, maximum vertical coordinate and minimum vertical coordinate of each coordinate value.

[0103] Furthermore, the coarse positioning position determination module 31 includes:

[0104] A mapping unit, configured to map the pupil coarse location to the original eye image to obtain a regression image;

[0105] The interception unit is used to intercept the regression image according to a set ratio, and perform low-value filtering on the intercepted image to obtain a target eye image containing the rough positioning position of the pupil.

[0106] Furthermore, the pupil center determination module 33 includes:

[0107] a screening unit, configured to select at least one preset screening rule from a preset rule set, and use each of the preset screening rules to screen a corresponding candidate target area from each candidate pupil area;

[0108] A pre-selected region determining unit, configured to determine the region with the highest number of appearances among the candidate target regions as the pre-selected region;

[0109] The target area determination unit is configured to determine, if the number of the pre-selected areas is one, the pre-selected area as the target pupil area; otherwise, determine the pre-selected area with the largest step size as the target pupil area.

[0110] Furthermore, the screening unit is specifically used to: when the preset screening rule is pixel screening, determine the pixel increase value of each candidate pupil area, and determine the set of candidate pupil areas whose corresponding pixel increase values are less than a first preset threshold; when the number of areas in the candidate pupil area set is greater than a first preset number, each candidate pupil area in the candidate pupil area set is a target area.

[0111] Furthermore, the screening unit is specifically used to: when the preset screening rule is area ratio screening, determine the circumscribed polygon corresponding to each candidate pupil area, and calculate the maximum inscribed area of the circumscribed polygon as the theoretical area; determine the area ratio according to the area of the candidate pupil area and the corresponding theoretical area, and determine a set of candidate pupil areas whose corresponding area ratios are greater than a second preset threshold; when the number of areas in the candidate pupil area set is greater than a second preset number, each candidate pupil area in the candidate pupil area set is a target area.

[0112] Furthermore, the screening unit is specifically used to: when the preset screening rule is perimeter screening, determine the circumscribed rectangle corresponding to each candidate pupil area, determine the perimeter change according to the perimeter of each circumscribed rectangle, and determine a set of candidate pupil areas whose corresponding perimeter change is less than a third preset threshold; when the number of areas in the candidate pupil area set is greater than a third preset number, each candidate pupil area in the candidate pupil area set is a target area.

[0113] Furthermore, the screening unit is specifically used to: when the preset screening rule is centroid screening, determine the centroid of each candidate pupil area, determine the offset between each adjacent centroid, and determine a set of candidate pupil areas whose corresponding offset is less than a fourth preset threshold; when the number of areas in the candidate pupil area set is greater than the fourth preset number, each candidate pupil area in the candidate pupil area set is a target area.

[0114] Furthermore, the pupil center determination module 33 includes:

[0115] An edge point extraction unit, configured to extract pupil edge points of the target pupil area using a preset edge extraction algorithm;

[0116] an edge region determining unit, configured to perform a convex hull operation on each pupil edge point to obtain a pupil edge region;

[0117] The pupil center determining unit is used to determine the centroid of the pupil edge area as the target pupil center position of the eye.

[0118] The pupil center locating device provided in the embodiment of the present invention can execute the pupil center locating method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0119] Example 4

[0120] Figure 9 A schematic diagram of the structure of a device provided in the fourth embodiment of the present invention is shown in FIG. Figure 9 As shown, the device includes a processor 40, a memory 41, an input device 42 and an output device 43; the number of processors 40 in the device can be one or more. Figure 9 In the embodiment, a processor 40 is used as an example; the processor 40, the memory 41, the input device 42 and the output device 43 in the device can be connected by a bus or other means. Figure 9 The bus connection is taken as an example.

[0121] The memory 41, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the pupil center localization method in the embodiments of the present invention (for example, the coarse location determination module 31, the candidate region determination module 32, and the pupil center determination module 33 in the pupil center localization device). The processor 40 executes the software programs, instructions, and modules stored in the memory 41 to execute various functional applications and data processing of the device, thereby implementing the aforementioned pupil center localization method.

[0122] The memory 41 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data created based on the use of the terminal, etc. Furthermore, the memory 41 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some instances, the memory 41 may further include memory remotely located relative to the processor 40, and these remote memories may be connected to the device via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0123] The input device 42 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device 43 may include a display device such as a display screen.

[0124] Example 5

[0125] Embodiment 5 of the present invention further provides a storage medium containing computer-executable instructions, wherein the computer-executable instructions, when executed by a computer processor, are used to perform a pupil center location method, the method comprising:

[0126] Performing coarse positioning processing on the acquired original eye image to obtain a coarse pupil positioning position of the eye, and obtaining a target eye image containing the coarse pupil positioning position;

[0127] In the target eye image, using the pupil coarse location as a seed point, performing region expansion with at least one set step size to obtain at least one candidate pupil region of the eye;

[0128] A screening operation is performed on each of the candidate pupil regions to select a target pupil region, and a target pupil center position of the eye is determined based on the target pupil region.

[0129] Of course, the storage medium containing computer-executable instructions provided in an embodiment of the present invention is not limited to the method operations described above, and can also execute related operations in the pupil center positioning method provided in any embodiment of the present invention.

[0130] Through the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented with the help of software and necessary general-purpose hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0131] It is worth noting that in the embodiment of the above-mentioned pupil center positioning device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0132] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will appreciate that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions are possible for those skilled in the art without departing from the scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the scope of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A pupil center positioning method, characterized in that: include: Performing coarse positioning processing on the acquired original eye image to obtain a coarse pupil positioning position of the eye, and obtaining a target eye image containing the coarse pupil positioning position; In the target eye image, using the pupil coarse location as a seed point, performing region expansion with at least one set step size to obtain at least one candidate pupil region of the eye; Performing a screening operation on each of the candidate pupil regions to select a target pupil region, and determining a target pupil center position of the eye based on the target pupil region; The coarse positioning processing is performed on the original eye image to obtain the coarse positioning position of the pupil of the eye, including: Performing downsampling and low-value filtering on the original eye image to obtain an intermediate image to be processed; Determine the target region of interest according to the grayscale value of each region of the intermediate image to be processed in combination with a grayscale threshold; Performing a convolution operation on the target region of interest and a preset pupil template to determine a rough pupil location of the eye; The determining of the target region of interest according to the grayscale values of each region of the intermediate image to be processed in combination with a grayscale threshold comprises: Determine whether the grayscale value of each region is less than a grayscale threshold, and if so, determine the region formed by the pixel points corresponding to the grayscale value of each region as a mask region; Determining the coordinate value of each pixel in the mask area; Determine the target region of interest according to the maximum abscissa, the minimum abscissa, the maximum ordinate, and the minimum ordinate among the coordinate values; The obtaining of the target eye image including the pupil coarse location comprises: Mapping the pupil coarse location to the original eye image to obtain a regression image; The regression image is intercepted according to a set ratio, and the intercepted image is subjected to low-value filtering to obtain a target eye image containing the rough positioning position of the pupil.

2. The method according to claim 1, characterized in that The screening operation on each candidate pupil area to select a target pupil area includes: Selecting at least one preset screening rule from a preset rule set, and using each of the preset screening rules to screen a corresponding candidate target area from each candidate pupil area; Determine the area with the highest number of appearances in each candidate target area as the pre-selected area; If the number of the pre-selected areas is one, the pre-selected area is determined to be the target pupil area; otherwise, the pre-selected area with the largest step size is determined to be the target pupil area.

3. The method according to claim 2, characterized in that The target areas are screened from the candidate pupil areas using preset screening rules, including: When the preset screening rule is pixel screening, determining a pixel increase value of each candidate pupil area, and determining a set of candidate pupil areas whose corresponding pixel increase values are less than a first preset threshold; When the number of regions in the candidate pupil region set is greater than a first preset number, each candidate pupil region in the candidate pupil region set is used as a target region.

4. The method according to claim 3, characterized in that The target areas are screened from the candidate pupil areas using preset screening rules, including: When the preset screening rule is area ratio screening, determining the circumscribed polygons corresponding to the candidate pupil regions, and calculating the maximum inscribed area of the circumscribed polygons as the theoretical area; determining an area ratio based on the areas of the candidate pupil regions and the corresponding theoretical areas, and determining a set of candidate pupil regions having corresponding area ratios greater than a second preset threshold; When the number of regions in the candidate pupil region set is greater than a second preset number, each candidate pupil region in the candidate pupil region set is used as a target region.

5. The method according to claim 2, characterized in that The target areas are screened from the candidate pupil areas using preset screening rules, including: When the preset screening rule is perimeter screening, determining the circumscribed rectangles corresponding to the candidate pupil regions, determining perimeter changes based on the perimeters of the circumscribed rectangles, and determining a set of candidate pupil regions whose corresponding perimeter changes are less than a third preset threshold; When the number of regions in the candidate pupil region set is greater than a third preset number, each candidate pupil region in the candidate pupil region set is used as a target region.

6. The method according to claim 2, characterized in that The target areas are screened from the candidate pupil areas using preset screening rules, including: When the preset screening rule is centroid screening, determining the centroids of the candidate pupil regions, determining the offsets between adjacent centroids, and determining a set of candidate pupil regions whose corresponding offsets are less than a fourth preset threshold; When the number of regions in the candidate pupil region set is greater than a fourth preset number, each candidate pupil region in the candidate pupil region set is used as a target region.

7. The method according to claim 1, characterized in that Determining the target pupil center position of the eye according to the target pupil area includes: Extracting pupil edge points of the target pupil area using a preset edge extraction algorithm; Performing a convex hull operation on each pupil edge point to obtain a pupil edge area; The centroid of the pupil edge area is determined as the target pupil center position of the eye.

8. A pupil center positioning device, characterized in that: include: a coarse positioning position determination module, configured to perform coarse positioning processing on the acquired original eye image, obtain a coarse positioning position of the pupil of the eye, and obtain a target eye image containing the coarse positioning position of the pupil; a candidate region determination module, configured to, in the target eye image, use the pupil coarse location as a seed point and perform region expansion using at least one set step size to obtain at least one candidate pupil region of the eye; a pupil center determination module, configured to screen each of the candidate pupil regions, select a target pupil region, and determine a target pupil center position of the eye based on the target pupil region; The coarse positioning module includes: An image processing unit, configured to perform downsampling and low-value filtering on the original eye image to obtain an intermediate image to be processed; A region determination unit, configured to determine a target region of interest based on the grayscale value of each region of the intermediate image to be processed in combination with a grayscale threshold; a position determination unit, configured to perform a convolution operation on the target region of interest and a preset pupil template to determine a rough pupil location of the eye; The region determination unit is specifically configured to: determine whether the grayscale value of each region is less than a grayscale threshold; if so, determine the region formed by the pixel points corresponding to the grayscale value of each region as a mask region; determine the coordinate value of each pixel point in the mask region; and determine the target region of interest based on the maximum horizontal coordinate, the minimum horizontal coordinate, the maximum vertical coordinate, and the minimum vertical coordinate of each coordinate value; Furthermore, the coarse positioning position determination module includes: A mapping unit, configured to map the pupil coarse location to the original eye image to obtain a regression image; The interception unit is used to intercept the regression image according to a set ratio, and perform low-value filtering on the intercepted image to obtain a target eye image containing the rough positioning position of the pupil.

9. A device, characterized in that The device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the pupil center location method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the pupil center positioning method according to any one of claims 1 to 7 is implemented.

Images