A pupil positioning method, device, electronic device and storage medium

Abstract

The application discloses a pupil positioning method and device, electronic equipment and storage medium. The method comprises the following steps: determining a candidate region containing a pupil from an eye region image. Mask processing is performed on the candidate region to determine a pupil detection region corresponding to the pupil from the candidate region. Based on the edge points of the candidate region, the pupil edge points corresponding to the pupil detection region are determined. The pupil edge points corresponding to the pupil detection region are connected to obtain at least one pupil connected region. The pupil edge points corresponding to each of the at least one pupil connected region are fitted to obtain a target pupil region. Based on the target pupil region, the pupil center position corresponding to the pupil is determined. The method can improve the accuracy and stability of pupil positioning in a real environment.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a pupil positioning method, apparatus, electronic device, and storage medium. Background Technology

[0002] Head-mounted eye trackers are of great significance for studying human behavior in many practical dynamic tasks, and accurate pupil center localization algorithms are essential for gaze tracking under real-world conditions. Currently, most head-mounted eye trackers use the dark pupil method to locate the pupil. While this method can accurately locate the pupil center under controlled indoor conditions, it leads to a decrease in accuracy and stability under outdoor conditions with environmental interference. Summary of the Invention

[0003] This application provides a pupil positioning method, apparatus, electronic device, and storage medium, which can improve the accuracy and stability of pupil center positioning.

[0004] On the one hand, this application provides a pupil localization method, the method comprising:

[0005] Identify candidate regions containing the pupil from images of the eye region;

[0006] The candidate regions are masked to determine the pupil detection region corresponding to the pupil.

[0007] Based on the edge points of the candidate region, the pupil edge points corresponding to the pupil detection region are determined; the edge points of the candidate region are obtained by performing edge detection on the candidate region;

[0008] Connect the pupil edge points corresponding to the pupil detection area to obtain at least one pupil connected region;

[0009] The pupil edge points corresponding to each of the at least one pupil connected region are fitted to obtain the target pupil region;

[0010] Based on the target pupil region, determine the pupil center position corresponding to the pupil.

[0011] In an optional embodiment, the pupil edge points corresponding to the pupil detection region include a first pupil edge point and a second pupil edge point, and determining the pupil edge points corresponding to the pupil detection region based on the edge points of the candidate region includes:

[0012] Based on the first gradient threshold, edge point filtering is performed on the edge points of the candidate region to obtain the first candidate edge points;

[0013] Determine the first pupil edge point within the pupil detection area and the non-edge points outside the pupil detection area from the first candidate edge points;

[0014] Based on a first pupil edge point that matches a preset edge point determination template, a second pupil edge point that matches the edge point determination template is determined from the non-edge points.

[0015] In an optional embodiment, the step of connecting the pupil edge points corresponding to the pupil detection area to obtain at least one pupil connected region includes:

[0016] Based on the edge points, determine the connection method corresponding to the template, perform connection operations on the first pupil edge points and the second pupil edge points to obtain at least one initial pupil connected region;

[0017] The initial pupil connectivity region that meets the preset region conditions is taken as the pupil connectivity region.

[0018] In an optional embodiment, the step of fitting the pupil edge points corresponding to each of the at least one pupil connected region to determine the target pupil region includes:

[0019] The pupil edge point corresponding to each of the at least one pupil connected region is taken as the third pupil edge point;

[0020] The third pupil edge point is fitted to obtain the first pupil region;

[0021] Based on the first pupil region, edge point filtering is performed on the edge points of the third pupil to obtain the edge points of the fourth pupil;

[0022] The target pupil region is obtained by fitting the edge points of the fourth pupil.

[0023] In an optional embodiment, the step of filtering the edge points of the third pupil based on the first pupil region to obtain the fourth pupil edge points includes:

[0024] Based on the distance between the third pupil edge point and the first pupil region, edge point filtering is performed on the third pupil edge point to obtain the fifth pupil edge point;

[0025] Based on the distance between the fifth pupil edge point and the first pupil region, the discrete distribution parameters of the fifth pupil edge point are determined;

[0026] When the discrete distribution parameter is less than a preset parameter threshold, the edge point of the fifth pupil is filtered based on the distance between the edge point of the fifth pupil and the first pupil region to obtain the edge point of the fourth pupil.

[0027] In an optional embodiment, after determining the discrete distribution parameters of the fifth pupil edge point based on the distance between the fifth pupil edge point and the first pupil region, the method further includes:

[0028] If the discrete distribution parameter is greater than or equal to a preset parameter threshold, the noise distribution information of the fifth pupil edge point in the first pupil region is determined;

[0029] When the noise distribution information indicates the presence of noise within the first pupil region, edge point filtering is performed on the fifth pupil edge point based on the distance between the fifth pupil edge point and the target inscribed curve to obtain the sixth pupil edge point; the target inscribed curve is determined based on the circumscribed curve corresponding to the region boundary information, and the region boundary information is the boundary information of the region constructed by the fifth pupil edge point;

[0030] When the discrete distribution parameter is less than a preset parameter threshold, the step of filtering the fifth pupil edge point based on the distance between the fifth pupil edge point and the first pupil region to obtain the fourth pupil edge point includes:

[0031] If the discrete distribution parameter corresponding to the sixth pupil edge point is less than the preset parameter threshold, the sixth pupil edge point is filtered based on the distance between the sixth pupil edge point and the first pupil region to obtain the fourth pupil edge point.

[0032] In an optional embodiment, after determining the noise distribution information of the fifth pupil edge point in the first pupil region when the discrete distribution parameter is less than a preset parameter threshold, the method further includes:

[0033] When the noise distribution information indicates that there is noise outside the first pupil region, the edge point of the fifth pupil is filtered based on the reference line determined by the focal point of the first pupil region to obtain the edge point of the seventh pupil.

[0034] The edge points of the seventh pupil are fitted to determine the region of the second pupil;

[0035] Obtain target ray information starting from the center position of the curve in the second pupil region;

[0036] Based on the gradient information of the fifth pupil edge point on the target ray information, edge point filtering is performed on the fifth pupil edge point to obtain the eighth pupil edge point;

[0037] The fitting process for the fourth pupil edge point to obtain the target pupil region includes:

[0038] The target pupil region is obtained by fitting the edge point of the eighth pupil.

[0039] In an optional embodiment, the candidate region includes a pupil spot, and the method further includes:

[0040] Determine the initial spot center position corresponding to the pupil spot from the candidate region;

[0041] The first spot detection region corresponding to the center position of the initial spot is processed to obtain the second spot detection region.

[0042] Based on the second spot detection area, the center position of the target spot corresponding to the pupil spot is determined.

[0043] In an optional embodiment, determining the initial spot center position corresponding to the pupil spot from the candidate region includes:

[0044] Starting from the detection centroid, a region diffusion process is performed to obtain the region from the candidate region where the grayscale information of the pixels meets the preset grayscale threshold, thus obtaining the third spot detection region; the detection centroid is the centroid detected when acquiring the eye region image;

[0045] Based on the second gradient threshold, edge points in the candidate region are filtered to obtain second candidate edge points;

[0046] The first spot edge point within the third spot detection area is determined from the second candidate edge point;

[0047] Connect the edge points of the first light spot to obtain the connected region of the light spot;

[0048] The centroid of the connected region of the light spot is taken as the initial center position of the light spot.

[0049] In an optional embodiment, the step of performing region processing on the first spot detection region corresponding to the initial spot center position to obtain the second spot detection region includes:

[0050] The first spot detection area is integrated to determine the regional response parameters corresponding to the edge points of the spot in the first spot detection area.

[0051] Based on the region response parameters, edge point filtering is performed on the edge points of the light spot in the first light spot detection region to obtain the edge points of the second light spot.

[0052] The detection area of ​​the second spot is determined based on the centroid corresponding to the edge point of the second spot.

[0053] In an optional embodiment, determining the center position of the target spot corresponding to the pupil spot based on the second spot detection area includes:

[0054] Gradient processing is performed on the second spot detection region to determine the gradient parameters corresponding to the edge points of the spot in the second spot detection region.

[0055] Based on the gradient parameters, edge point filtering is performed on the edge points of the spot in the second spot detection area to obtain the edge points of the third spot;

[0056] Based on the centroid corresponding to the edge point of the third light spot, the center position of the target light spot corresponding to the pupil light spot is determined.

[0057] On the other hand, a pupil positioning device is provided, the device comprising:

[0058] The candidate region determination module is used to determine candidate regions containing the pupil from the eye region image;

[0059] A mask region determination module is used to perform mask processing on the candidate region and determine the pupil detection region corresponding to the pupil from the candidate region;

[0060] The pupil edge point determination module is used to determine the pupil edge points corresponding to the pupil detection area based on the edge points of the candidate area; the edge points of the candidate area are obtained by performing edge detection on the candidate area;

[0061] The pupil edge point connection module is used to connect the pupil edge points corresponding to the pupil detection area to obtain at least one pupil connected area.

[0062] An edge point fitting module is used to fit the pupil edge points corresponding to each of the at least one pupil connected regions to obtain the target pupil region.

[0063] The pupil center determination module is used to determine the pupil center position corresponding to the pupil based on the target pupil region.

[0064] On the other hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement a pupil localization method as described above.

[0065] On the other hand, a computer-readable storage medium is provided, the storage medium including a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement a pupil localization method as described above.

[0066] On the other hand, a computer program product is provided, including a computer program that, when executed by a processor, implements the pupil positioning method described above.

[0067] This application provides a pupil localization method, apparatus, electronic device, and storage medium. The method can determine candidate regions containing pupils from an image of an eye region. The candidate regions are masked to determine pupil detection regions corresponding to the pupils. Based on the edge points of the candidate regions, pupil edge points corresponding to the pupil detection regions are determined. The pupil edge points corresponding to the pupil detection regions are connected to obtain at least one connected pupil region. The pupil edge points corresponding to each of the at least one connected pupil region are fitted to obtain a target pupil region. Based on the target pupil region, the pupil center position corresponding to the pupil is determined. This method can detect pupil centers in real-world environments, such as non-reflective pupil centers, reflective pupil centers, and double-image pupil centers, by filtering the pupil edge points, thus improving the accuracy and stability of pupil localization. Attached Figure Description

[0068] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 This is a schematic diagram illustrating an application scenario of a pupil localization method provided in an embodiment of this application;

[0070] Figure 2 A flowchart illustrating a pupil localization method provided in an embodiment of this application;

[0071] Figure 3 A flowchart illustrating the determination of pupil edge points in a pupil localization method provided in this application embodiment;

[0072] Figure 4 A schematic diagram of an edge point determination template in a pupil localization method provided in this application embodiment;

[0073] Figure 5 A flowchart illustrating the determination of a target pupil region in a pupil localization method provided in this application embodiment;

[0074] Figure 6 This is a flowchart illustrating edge point filtering based on a first pupil region in a pupil localization method provided in an embodiment of this application.

[0075] Figure 7 This is a flowchart illustrating the removal of noise within the first pupil region in a pupil localization method provided in an embodiment of this application.

[0076] Figure 8 This is a flowchart illustrating the removal of noise outside the first pupil region in a pupil localization method provided in an embodiment of this application.

[0077] Figure 9 This is a schematic diagram of subpixel interpolation in a pupil localization method provided in an embodiment of this application;

[0078] Figure 10 A flowchart illustrating the determination of the pupil spot center in a pupil localization method provided in this application embodiment;

[0079] Figure 11 This is a flowchart illustrating the determination of the initial pupil spot center in a pupil localization method provided in an embodiment of this application;

[0080] Figure 12 This is a flowchart illustrating the determination of the second light spot detection area in a pupil localization method provided in an embodiment of this application;

[0081] Figure 13 This is a flowchart illustrating the determination of the center position of a target light spot in a pupil localization method provided in an embodiment of this application;

[0082] Figure 14 This is a schematic diagram of the logical structure of pupil positioning and light spot positioning in a pupil positioning method provided in an embodiment of this application;

[0083] Figure 15 This is a schematic diagram illustrating the output of the pupil center position and the target spot center position in a pupil positioning method provided in an embodiment of this application.

[0084] Figure 16 This is a schematic diagram illustrating the detection results of pupil localization and light spot localization in a real-world scenario in a pupil localization method provided in this application embodiment.

[0085] Figure 17 This is a schematic diagram of the structure of a pupil positioning device provided in an embodiment of this application;

[0086] Figure 18 This is a schematic diagram of the hardware structure of a device for implementing the method provided in the embodiments of this application. Detailed Implementation

[0087] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0088] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. Furthermore, the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein.

[0089] It is understood that the user information and related data involved in the specific implementation of this application are all data that the user has authorized or agreed to obtain, and the collection, use and processing of the related data comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0090] Please see Figure 1 The illustration shows an application scenario diagram of a pupil localization method provided in the embodiments of this application. The application scenario includes a client 110 and a server 120. The client 110 sends a face image to the server 120. After the server 120 performs pupil localization and pupil spot localization based on the face image, it sends the localization result to the client 110.

[0091] In this embodiment of the disclosure, client 110 includes physical devices such as smartphones, desktop computers, tablets, laptops, digital assistants, and smart wearable devices, and may also include software running on the physical device, such as applications. Smart wearable devices may include head-mounted eye trackers for gaze tracking.

[0092] In this embodiment of the disclosure, server 120 may include a standalone server, a distributed server, or a server cluster consisting of multiple servers. Server 120 may include a network communication unit, a processor, and memory, etc.

[0093] Please see Figure 2 It demonstrates a pupil localization method applicable to the server side, the method comprising:

[0094] S210. Identify candidate regions containing the pupil from the eye region image;

[0095] In an optional embodiment, a face image is acquired. The face image can be an image containing only brightness and darkness contrast, such as an infrared image. After acquiring the face image, key points around the eyes can be detected from the face image to obtain an eye region image. The eye region image can be a rectangular image with a preset side length. An eye region recognition model can be used to detect the face image and key points around the eyes. The output of the eye region recognition model can be the location information and side length information of the eye region image. The location information can be the angular coordinates of the eye region image. For example, the output is (x, y, w, h), where (x, y) is the coordinate of the upper left corner of the eye region image, w is the width of the eye region image, and h is the height of the eye region image. The eye region recognition model can be used to detect regions of interest in a face image and identify the location and category probability of the region of interest. The eye region recognition model can be a YOLO model, which is a model that divides the face image into a grid and performs object detection processing on the resulting grid regions to obtain the eye region image.

[0096] The center point of the human eye in the eye region image is used as the center point of the candidate region to generate a candidate region of a preset size. For example, a rectangular region of a preset size is generated using the center point of the human eye as the center point of the candidate region, and this rectangular region of the preset size is used as the candidate region.

[0097] S220. Perform masking on the candidate region and determine the pupil detection region corresponding to the pupil from the candidate region;

[0098] In an optional embodiment, image enhancement processing is performed on the candidate region to obtain a region-enhanced image. The region-enhanced image is converted into a grayscale histogram, and histogram regions with grayscale values ​​below a preset first grayscale threshold are obtained. The maximum value in the histogram region is used as a binarization threshold. The region-enhanced image is binarized based on the binarization threshold to obtain a binarized image.

[0099] Detect connected regions in the binarized image. If two connected regions are obtained, the image composed of pixels in the two connected regions that are less than the binarization threshold is taken as the pupil detection region. The image composed of pixels that are less than the binarization threshold can be a connected region marked as 1. If more than two connected regions are obtained, calculate the gray-scale mean in each connected region and take the connected region with the smallest gray-scale mean as the pupil detection region.

[0100] S230. Based on the edge points of the candidate region, determine the pupil edge points corresponding to the pupil detection region; the edge points of the candidate region are obtained by performing edge detection on the candidate region;

[0101] In an optional embodiment, edge points in the candidate region are determined, and the candidate region and the pupil detection region are overlapped. Edge points covered by the pupil detection region and edge points related to the pupil detection region in other regions besides the pupil detection region are determined. The edge points covered by the pupil detection region and the edge points related to the pupil detection region are taken as the pupil edge points corresponding to the pupil detection region.

[0102] In an optional embodiment, please refer to Figure 3 The pupil edge points corresponding to the pupil detection area include the first pupil edge point and the second pupil edge point. Based on the edge points of the candidate area, the pupil edge points corresponding to the pupil detection area are determined as follows:

[0103] S310. Based on the first gradient threshold, edge point filtering is performed on the edge points of the candidate region to obtain the first candidate edge points;

[0104] S320. Determine the first pupil edge point within the pupil detection area and the non-edge points outside the pupil detection area from the first candidate edge points;

[0105] S330. Based on the first pupil edge point that matches the preset edge point determination template, determine the second pupil edge point that matches the edge point determination template from the non-edge points.

[0106] In an optional embodiment, the first pupil edge point is an edge point covered by the pupil detection area, and the second pupil edge point is an edge point related to the pupil detection area. Gradient processing is performed on the candidate regions to obtain gradient information for each edge point. Edge points whose gradient information satisfies a first gradient threshold are selected as first candidate edge points. The first gradient threshold may include a third gradient threshold and a fourth gradient threshold, where the fourth gradient threshold is greater than the third gradient threshold. Therefore, edge points whose gradient information is greater than or equal to the third gradient threshold and less than or equal to the fourth gradient threshold can be selected as first candidate edge points.

[0107] The candidate region and the pupil detection region are overlapped. The first candidate edge point within the pupil detection region is taken as the first pupil edge point, and the first candidate edge point outside the pupil detection region is taken as the non-edge point.

[0108] The edge point determination template includes a first preset edge point position and a second preset edge point position. If the first preset edge point position matches any pupil edge point among the first pupil edge points, and a non-edge point matches the second preset edge point position, then that non-edge point can be used as the second pupil edge point. There can be multiple first preset edge point positions; please refer to [link to relevant documentation]. Figure 4 ,like Figure 4The diagram shows a template for determining edge points, where O represents the preset position of the first edge point and X represents the preset position of the second edge point. The template includes two preset positions of the first edge point and one preset position of the second edge point. If the preset positions of the first edge points on both sides match the first pupil edge point, and the preset position of the second edge point in the middle matches a non-edge point, then that non-edge point can be used as the second pupil edge point.

[0109] There can be multiple edge point determination templates. In the first candidate edge points, edge points are traversed based on each edge point determination template to obtain the first pupil edge point matched by each edge point determination template. Based on the first pupil edge point matched by each edge point determination template, edge point filtering is performed on non-edge points to determine the second pupil edge point matched by each edge point determination template from the non-edge points.

[0110] Determining the second pupil edge point using an edge point determination template, and supplementing the first pupil edge point determined by the pupil detection area, can improve the accuracy of determining the pupil edge point.

[0111] S240. Connect the pupil edge points corresponding to the pupil detection area to obtain at least one pupil connected region;

[0112] In an optional embodiment, the pupil edge points corresponding to the pupil detection area are connected to obtain edge point connection results. These results are then filtered to obtain at least one connected pupil region. The edge point connection results can be the initial connected pupil region; region filtering of this initial connected pupil region yields at least one connected pupil region.

[0113] In an optional embodiment, connecting the pupil edge points corresponding to the pupil detection region to obtain at least one connected pupil region includes:

[0114] Based on the edge points, the connection method corresponding to the template is determined, and the first pupil edge points and the second pupil edge points are connected to obtain at least one initial pupil connected region.

[0115] The initial pupil connectivity region that meets the preset region conditions is taken as the pupil connectivity region.

[0116] In an optional embodiment, based on the arrangement of the preset positions of the first and second edge points in the edge point determination template, a first pupil edge point matching the preset position of the first edge point and a connection method between the second pupil edge points matching the preset position of the second edge point can be obtained. Different edge point determination templates correspond to different connection methods. By connecting the first and second pupil edge points using the connection methods corresponding to different edge point determination templates, at least one initial pupil connected region can be obtained.

[0117] For each initial pupil connected region, the bounding rectangle is calculated to obtain the minimum bounding rectangle corresponding to each initial pupil connected region. If the aspect ratio of the minimum bounding rectangle corresponding to the initial pupil connected region is greater than or equal to a preset aspect ratio threshold, the initial pupil connected region is taken as the pupil connected region. If the aspect ratio of the minimum bounding rectangle corresponding to the initial pupil connected region is less than the preset aspect ratio threshold, the pupil edge points corresponding to the initial pupil connected region are filtered out from the first pupil edge points and the second pupil edge points.

[0118] The connection method corresponding to the template is determined based on the edge points. The initial pupil connected region is determined and filtered based on the preset region conditions to improve the accuracy of the pupil edge points, thereby improving the accuracy of pupil center positioning.

[0119] S250. Fit the pupil edge points corresponding to each of at least one pupil connected region to obtain the target pupil region;

[0120] In an optional embodiment, fitting processing is performed on the pupil edge points corresponding to at least one pupil connected region, so that the pupil edge points corresponding to different pupil connected regions can be fitted into the same region, thereby obtaining the target pupil region.

[0121] The fitting process can be performed multiple times. After the first fitting process, the pupil edge points corresponding to the first fitting process are filtered to obtain the pupil edge points corresponding to the second fitting process. The area obtained after the second fitting process can be used as the target pupil area.

[0122] In an optional embodiment, please refer to Figure 5 Fitting is performed on the pupil edge points corresponding to at least one pupil connected region to determine the target pupil region, including:

[0123] S510. Take the pupil edge point corresponding to each of at least one pupil connected region as the third pupil edge point;

[0124] S520. Fit the edge points of the third pupil to obtain the first pupil region;

[0125] S530. Based on the first pupil region, perform edge point filtering on the edge points of the third pupil to obtain the edge points of the fourth pupil;

[0126] S540. Fit the edge points of the fourth pupil to obtain the target pupil region.

[0127] In an optional embodiment, at least one pupil edge point corresponding to each pupil connected region is designated as a third pupil edge point. Fitting the third pupil edge points yields a closed curve formed by these points, which represents the first pupil region. Based on the positional relationship between the first pupil region and the third pupil edge points, edge point filtering can be performed on the third pupil edge points to obtain a fourth edge point whose positional relationship with the first pupil region satisfies a preset positional relationship condition.

[0128] Multiple fitting processes can be performed on the edge points of the third pupil. In each fitting process, a portion of the pupil edge points are obtained from the third pupil edge points and fitted to obtain the first pupil region corresponding to each fitting process. Based on the first pupil region corresponding to each fitting process, edge point filtering is performed on the third pupil edge points, and the pupil edge points obtained after multiple edge point filtering are used as the fourth pupil edge points.

[0129] By fitting the edge points of the fourth pupil, a closed curve formed by the edge points of the fourth pupil can be obtained, which is the target pupil region.

[0130] Edge point filtering is performed based on the first pupil region to further improve the accuracy of pupil edge points, thereby improving the accuracy of pupil center positioning.

[0131] In an optional embodiment, please refer to Figure 6 Based on the first pupil region, edge point filtering is performed on the edge points of the third pupil to obtain the edge points of the fourth pupil, including:

[0132] S610. Based on the distance between the edge point of the third pupil and the region of the first pupil, edge point filtering is performed on the edge point of the third pupil to obtain the edge point of the fifth pupil;

[0133] S620. Determine the discrete distribution parameters of the fifth pupil edge point based on the distance between the fifth pupil edge point and the first pupil region;

[0134] S630. When the discrete distribution parameter is less than the preset parameter threshold, the edge point of the fifth pupil is filtered based on the distance between the edge point of the fifth pupil and the region of the first pupil to obtain the edge point of the fourth pupil.

[0135] In an optional embodiment, the distance between the edge point of the third pupil and the first pupil region is determined. This distance can be the distance of a perpendicular line segment drawn from the edge point of the third pupil to the boundary of the first pupil region. Alternatively, it can be the distance between the intersection point of the line connecting the edge point of the third pupil to the center of the first pupil region and the boundary of the first pupil region, and the distance between this intersection point and the edge point of the third pupil. The distances between all subsequent points and regions are described similarly.

[0136] Edge point filtering is performed on the third pupil edge points to remove those whose distance from the first pupil region is greater than a first preset distance threshold, and those whose distance from the first pupil region is less than or equal to the first preset distance threshold are designated as the fifth pupil edge points. By filtering the third pupil edge points, pupil edge points that are far from the first pupil region can be filtered out.

[0137] Based on the distance between the fifth pupil edge point and the first pupil region, the discrete distribution parameters of the fifth pupil edge point can be determined. These discrete distribution parameters represent at least the distribution of the fifth pupil edge point within and outside the first pupil region, and can be the variance of the distance. If the discrete distribution parameter is less than a preset threshold, it indicates that the fifth pupil edge point is the true edge point corresponding to the pupil, and a pupil region satisfying a preset accuracy can be obtained based on the fifth pupil edge point. By determining the discrete distribution of the fifth pupil edge point, it can be verified whether the fifth pupil edge point satisfies the distribution of edge points in the true pupil region.

[0138] Edge point filtering is performed on the fifth pupil edge points to remove fifth pupil edge points whose distance from the first pupil region is greater than the second preset distance threshold, and fifth pupil edge points whose distance from the first pupil region is less than or equal to the second preset distance threshold are used as fourth pupil edge points.

[0139] Filtering is performed based on the distance between the first pupil region and the pupil edge points to remove pupil edge points that are too far away. Based on the discrete distribution parameters of the pupil edge points, the accuracy of the filtered pupil edge points is determined. Thus, the pupil center position can be calculated when the accuracy of the pupil edge points meets the distribution conditions, thereby improving the accuracy and effectiveness of the pupil center position.

[0140] In an optional embodiment, please refer to Figure 7 After determining the discrete distribution parameters of the fifth pupil edge point based on the distance between the fifth pupil edge point and the first pupil region, the method further includes:

[0141] S710. When the discrete distribution parameter is greater than or equal to the preset parameter threshold, determine the noise distribution information of the fifth pupil edge point in the first pupil region;

[0142] S720. When the noise distribution information indicates that there is noise in the first pupil area, the edge point of the fifth pupil is filtered based on the distance between the edge point of the fifth pupil and the inner curve of the target to obtain the edge point of the sixth pupil; the inner curve of the target is determined based on the outer curve corresponding to the region boundary information, and the region boundary information is the boundary information of the region constructed by the edge point of the fifth pupil;

[0143] When the discrete distribution parameter is less than a preset parameter threshold, edge point filtering is performed on the fifth pupil edge point based on the distance between the fifth pupil edge point and the first pupil region to obtain the fourth pupil edge point, which includes:

[0144] S730. If the discrete distribution parameter corresponding to the edge point of the sixth pupil is less than the preset parameter threshold, the edge point of the sixth pupil is filtered based on the distance between the edge point of the sixth pupil and the first pupil region to obtain the edge point of the fourth pupil.

[0145] In an optional embodiment, if the discrete distribution parameter is greater than or equal to a preset parameter threshold, it indicates that there are noisy edge points in the fifth pupil edge points. Image noise processing can be performed on the first pupil region to determine the noise distribution information of the fifth pupil edge points in the first pupil region.

[0146] When noise distribution information indicates the presence of noise within the first pupil region, the boundary information of the region corresponding to the fifth pupil edge point is obtained. This region can be an irregularly shaped area. The bounding curve corresponding to the region boundary information can be the minimum bounding rectangle. The bounding rectangle is calculated for the region boundary information to obtain the minimum bounding rectangle corresponding to the region boundary information. The inscribed circle of the minimum bounding rectangle corresponding to the region boundary information is determined, and this inscribed circle is used as the target inscribed curve. Edge point filtering is performed on the fifth pupil edge points to remove pupil edge points located near the target inscribed curve, thereby removing noise edge points inside the first pupil region.

[0147] Based on the distance between the fifth pupil edge point and the target inscribed curve, fifth pupil edge points whose distance to the target inscribed curve is less than the third preset distance threshold are removed, and fifth pupil edge points whose distance to the target inscribed curve is greater than or equal to the third preset distance threshold are taken as sixth pupil edge points.

[0148] Based on the distance between the sixth pupil edge point and the first pupil region, the discrete distribution parameters corresponding to the sixth pupil edge point are determined. If the discrete distribution parameters corresponding to the sixth pupil edge point are less than a preset parameter threshold, edge point filtering is performed on the sixth pupil edge point based on the distance between the sixth pupil edge point and the first pupil region to obtain the fourth pupil edge point.

[0149] By using the target inline curve to remove noise within the first pupil region, the fitting error of the pupil edge points can be reduced, thereby improving the stability of pupil center position determination in the presence of noise.

[0150] In an optional embodiment, please refer to Figure 8 When the discrete distribution parameter is less than a preset parameter threshold, after determining the noise distribution information of the fifth pupil edge point in the first pupil region, the method further includes:

[0151] S810. When the noise distribution information indicates that there is noise outside the first pupil region, the edge points of the fifth pupil are filtered based on the reference line determined by the focus of the first pupil region to obtain the edge points of the seventh pupil.

[0152] S820. Fit the edge points of the seventh pupil to determine the region of the second pupil;

[0153] S830. Obtain target ray information starting from the center position of the curve in the second pupil region;

[0154] S840. Based on the gradient information of the fifth pupil edge point on the target ray information, perform edge point filtering on the fifth pupil edge point to obtain the eighth pupil edge point;

[0155] By fitting the edge points of the fourth pupil, the target pupil region is obtained, including:

[0156] S850. Fit the edge points of the eighth pupil to obtain the target pupil region.

[0157] In an optional embodiment, when noise information indicates the presence of noise outside the first pupil region, a vertical line between the two focal points in the first pupil region can be determined and used as a reference line. Based on the distance between the fifth pupil edge point and the reference line, edge point filtering is performed on the fifth pupil edge point to obtain a seventh pupil edge point. If the line connecting at least two pupil edge points intersects the vertical line corresponding to the reference line, a first distance and a second distance from each of the at least two pupil edge points to the vertical line corresponding to the reference line can be determined. The pupil edge point corresponding to the first distance is removed from the fifth pupil edge points, and the pupil edge point corresponding to the second distance is retained, wherein the first distance is greater than the second distance.

[0158] By fitting the edge points of the seventh pupil, a closed curve formed by these points can be obtained, which represents the second pupil region. The center position of this curve within the second pupil region is then obtained. Using this center position as the starting point and a preset angular interval as the axis, target ray information is determined. The seventh pupil edge point with the largest gradient on each target ray is identified, thus defining the boundary between the dark area corresponding to the pupil and other regions. This seventh pupil edge point with the largest gradient is then used as the eighth pupil edge point, which is the boundary point of the target pupil region. Therefore, fitting the eighth pupil edge point yields the target pupil region.

[0159] Before fitting the edge points of the eighth pupil, sub-pixel interpolation can be performed on them. Using each edge point of the eighth pupil as the center, a first pixel region is determined in the eye region image. From this first pixel region, a second and third pixel region are obtained. The second pixel region can be the upper left corner of the first pixel region, and the third pixel region can be the upper right corner. The gradient values ​​of the center points of the second and third pixel regions are then calculated. Figure 9 As shown, Figure 9 The diagram illustrates subpixel interpolation. The first pixel region is a 5x5 pixel area, while the second and third pixel regions are both 3x3 pixel areas. Point A is the edge of the eighth pupil, point B is the center of the second pixel region, and point C is the center of the third pixel region.

[0160] Based on the gradient values ​​of each eighth pupil edge point, the gradient value of the second pixel region center point, and the gradient value of the third pixel region center point, parabolic interpolation is performed to obtain the sub-pixel coordinates corresponding to the ninth pupil edge point. Fitting is then performed based on the ninth pupil edge point to obtain the target pupil region.

[0161] By refitting the second pupil region and removing noise outside the first pupil region, the fitting error of the pupil edge points can be reduced, and the stability of pupil center position determination can be improved in the presence of noise.

[0162] S260. Based on the target pupil region, determine the pupil center position corresponding to the pupil.

[0163] In an optional embodiment, the target pupil region represents the region obtained by curve fitting of the edge points obtained after at least one edge point filtering. The target pupil region can be an elliptical region, in which case the center of the elliptical region can be determined as the center position of the pupil.

[0164] In an optional embodiment, the candidate region includes the pupil spot; see [link to relevant documentation]. Figure 10 The method also includes:

[0165] S1010. Determine the initial spot center position corresponding to the pupil spot from the candidate region;

[0166] S1020. Perform region processing on the first spot detection area corresponding to the initial spot center position to obtain the second spot detection area;

[0167] S1030. Based on the second spot detection area, determine the center position of the target spot corresponding to the pupil spot.

[0168] In an optional embodiment, when determining the pupil spot, the candidate region used to determine the pupil can be used as the candidate region corresponding to the pupil spot. A filtering operation is performed on the candidate region to remove image noise. Since the grayscale distribution of the pupil spot conforms to a Gaussian distribution, Gaussian filtering can be used to remove image noise from the candidate region.

[0169] The candidate region is used to determine the connected region containing the pupil spot, and the initial spot center position is determined based on this connected region. Region processing is then performed on the first spot detection region corresponding to the initial spot center position to filter out spot edge points. Based on the filtered spot edge points in the first spot detection region, a second spot detection region is obtained. Region processing is then performed on the second spot detection region to filter out spot edge points. Based on the filtered spot edge points in the second spot detection region, the target spot center position corresponding to the pupil spot is determined.

[0170] First, determine a rough initial spot center position, and then refine the position based on the rough initial spot center position to obtain the target spot center position, which can improve the accuracy of spot center positioning.

[0171] In an optional embodiment, please refer to Figure 11 Determining the initial spot center position corresponding to the pupil spot from the candidate region includes:

[0172] S1110. Perform region diffusion processing starting from the detection centroid, and obtain the region where the grayscale information of the pixels in the candidate region meets the preset grayscale threshold to obtain the third spot detection region; the detection centroid is the centroid detected when acquiring the eye region image;

[0173] S1120. Based on the second gradient threshold, edge points in the candidate region are filtered to obtain second candidate edge points;

[0174] S1130. Determine the first spot edge point within the third spot detection area from the second candidate edge points;

[0175] S1140. Perform a connection operation on the edge points of the first light spot to obtain the connected region of the light spot;

[0176] S1150. Take the centroid of the connected region of the light spot as the initial center position of the light spot.

[0177] In an optional embodiment, the grayscale histogram corresponding to the candidate region is obtained, and the maximum value in the grayscale histogram is determined. Then, the value with the largest inter-class variance between 0 and the maximum value is determined. Based on the maximum value and the value with the largest inter-class variance, a preset grayscale threshold can be determined. The specific formula is as follows:

[0178] T = (t1 + t2) / 2

[0179] Where T is the preset grayscale threshold, t1 is the maximum value, and t2 is the value with the largest inter-class variance.

[0180] The region diffusion interval is determined based on the average grayscale value of edge points in the candidate region and a preset grayscale threshold. Starting from the detection centroid, the row and column where the detection centroid is located are used as seed points, and region diffusion processing is performed based on the region diffusion interval. This allows the region from the candidate region to obtain the grayscale information of pixels that meets the preset grayscale threshold.

[0181] Gradient processing is performed on the candidate regions to obtain the gradient information of each edge point. Edge points whose gradient information satisfies a second gradient threshold are selected as second candidate edge points. The second gradient threshold may include a fifth gradient threshold and a sixth gradient threshold, where the sixth gradient threshold is greater than the fifth threshold. Edge points whose gradient information is greater than or equal to the fifth gradient threshold and less than or equal to the sixth gradient threshold are selected as second candidate edge points. The fifth gradient threshold can be 0.9, and the sixth gradient threshold can be 1. From the second candidate edge points, edge points whose grayscale values ​​are less than the average grayscale value corresponding to the second candidate edge point are removed. The second candidate edge points covered by the third spot detection area are then selected as the first spot edge points.

[0182] Connectivity statistics are performed on the edge points of the first light spot to obtain at least one connected region of the first light spot. The connected regions of the first light spot with gray values ​​greater than a preset gray value threshold are taken as the connected regions of the second light spot. Morphological processing is performed on the connected regions of the second light spot to smooth their boundaries, thus obtaining the connected regions of the light spots. This morphological processing method can be an opening operation.

[0183] Calculate the centroid of the connected region of the light spot and use it as the initial center position of the light spot.

[0184] Based on the grayscale values ​​in the image, the third spot detection area is determined. Edge point filtering is performed based on the third spot detection area to obtain more accurate spot edge points. Based on the spot connected region determined by the spot edge points, the initial spot center position is determined, thereby improving the accuracy of the initial spot center position and further improving the accuracy of the target spot center position.

[0185] In an optional embodiment, please refer to Figure 12 The first spot detection region corresponding to the initial spot center position is processed to obtain the second spot detection region, which includes:

[0186] S1210. Perform integration processing on the first spot detection area to determine the regional response parameters corresponding to the edge points of the spot in the first spot detection area;

[0187] S1220. Based on the region response parameters, edge point filtering is performed on the edge points of the light spot in the first light spot detection region to obtain the edge points of the second light spot;

[0188] S1230. Determine the detection area of ​​the second spot based on the centroid corresponding to the edge point of the second spot.

[0189] In an optional embodiment, an integral processing is performed on the first spot detection region to obtain an integral map corresponding to the first spot detection region. This integral map includes regional response parameters corresponding to the spot edge points in the first spot detection region. A preset percentage value of the maximum regional response parameter in the integral map is obtained. This preset percentage value can be 80%, and this preset percentage value of the maximum regional response parameter is used as the regional response threshold. Edge point filtering is performed on the spot edge points in the first spot detection region to remove spot edge points with regional response parameters less than the regional response threshold, and spot edge points with regional response parameters greater than or equal to the regional response threshold are used as second spot edge points.

[0190] The centroid is calculated based on the edge points of the second light spot, and the detection area of ​​the second light spot is determined by the centroid corresponding to the edge points of the second light spot. The detection area of ​​the second light spot matches the actual pupil light spot area better than the detection area of ​​the first light spot.

[0191] By using the regional response parameters obtained through integration processing, edge point filtering is performed on the edge points of the first spot detection region. The efficiency of edge point filtering can be improved by using integral image processing, thereby improving the efficiency of spot center localization.

[0192] In an optional embodiment, please refer to Figure 13 Based on the second spot detection area, determining the center position of the target spot corresponding to the pupil spot includes:

[0193] S1310. Perform gradient processing on the second spot detection area to determine the gradient parameters corresponding to the edge points of the spot in the second spot detection area;

[0194] S1320. Based on the gradient parameters, edge point filtering is performed on the edge points of the spot in the second spot detection area to obtain the edge points of the third spot;

[0195] S1330. Based on the centroid corresponding to the edge point of the third spot, determine the center position of the target spot corresponding to the pupil spot.

[0196] In an optional embodiment, gradient processing is performed on the second spot detection region to determine the gradient parameters corresponding to the edge points of the spot in the second spot detection region. The edge point corresponding to the maximum gradient parameter in the second spot detection region is obtained, i.e., the point with the greatest brightness-dark contrast in the second spot detection region, thus obtaining the edge point of the third spot. The centroid corresponding to the edge point of the third spot is calculated, and sub-pixel processing is performed on this centroid to obtain the center position of the target spot corresponding to the pupil spot.

[0197] Using the centroid corresponding to the edge point of the third light spot as the center, determine the first pixel region in the eye region image. From the first pixel region, obtain the second and third pixel regions. The second pixel region can be the area at the upper left corner of the first pixel region, and the third pixel region can be the area at the upper right corner of the first pixel region. Calculate the gradient value at the center point of the second pixel region and the gradient value at the center point of the third pixel region.

[0198] Based on the gradient values ​​of the centroid corresponding to the edge point of the third spot, the gradient value of the center point of the second pixel region, and the gradient value of the center point of the third pixel region, parabolic interpolation is performed to obtain the sub-pixel coordinates corresponding to the center position of the target spot.

[0199] Gradient processing is used to obtain the boundary points of the spot region, namely the edge points of the third spot. Sub-pixel processing is then performed on the centroid corresponding to the edge points of the third spot to determine the center position of the target spot. This allows the center position of the target spot to achieve sub-pixel accuracy, thereby improving the accuracy of spot center positioning.

[0200] In an optional embodiment, please refer to Figure 14 ,like Figure 14The diagram illustrates the logical structure of pupil localization and target spot localization. After determining the pupil center position and the target spot center position, these positions can be detected based on preset constraints to obtain detection results. If the detection result indicates a pass, the pupil center position and target spot center position are output; otherwise, the data that failed the detection is adjusted. Constraints can include angle and distance conditions. The angle and distance between the pupil center positions and the target spot center positions in the left and right eyes are calculated. If both the angle and distance conditions are satisfied, a detection result indicating a pass can be generated. Figure 15 The diagram shows the positions of the output pupil center and the target spot center. Figure 16 The diagram shows the detection results of pupil localization and spot localization in a real scene. Due to lighting conditions in the real scene, the obtained eye area image is subject to environmental interference from the eyeglass frame and the reflection of the eyeglasses. Under the presence of environmental interference, the pupil localization method described above can identify the center position of the pupil corresponding to the white dot in the white box area and the center position of the target spot corresponding to the black dot in the white box area.

[0201] This application provides a pupil localization method, which includes: determining a candidate region containing the pupil from an eye region image; performing masking processing on the candidate region to determine a pupil detection region corresponding to the pupil from the candidate region; determining pupil edge points corresponding to the pupil detection region based on the edge points of the candidate region; performing a connection operation on the pupil edge points corresponding to the pupil detection region to obtain at least one connected pupil region; performing fitting processing on the pupil edge points corresponding to each of the at least one connected pupil region to obtain a target pupil region; and determining the pupil center position corresponding to the pupil based on the target pupil region. This method avoids environmental interference that exists when determining the pupil center by brightness contrast by filtering the pupil edge points, thereby enabling the detection of pupil centers in real environments such as non-reflective pupil centers, reflective pupil centers, and double-image pupil centers, improving the accuracy and stability of pupil localization.

[0202] This application also provides a pupil positioning device; please refer to [link to relevant documentation]. Figure 17 The device includes:

[0203] Candidate region determination module 1710 is used to determine candidate regions containing pupils from an eye region image;

[0204] The mask region determination module 1720 is used to perform mask processing on the candidate region and determine the pupil detection region corresponding to the pupil from the candidate region.

[0205] The pupil edge point determination module 1730 is used to determine the pupil edge points corresponding to the pupil detection area based on the edge points of the candidate area; the edge points of the candidate area are obtained by performing edge detection on the candidate area;

[0206] The pupil edge point connection module 1740 is used to connect the pupil edge points corresponding to the pupil detection area to obtain at least one pupil connected area.

[0207] The edge point fitting module 1750 is used to fit the pupil edge points corresponding to at least one pupil connected region to obtain the target pupil region.

[0208] The pupil center determination module 1760 is used to determine the pupil center position corresponding to the target pupil region.

[0209] In an optional embodiment, the pupil edge points corresponding to the pupil detection area include a first pupil edge point and a second pupil edge point, and the pupil edge point determination module includes:

[0210] The first candidate edge point acquisition unit is used to filter the edge points of the candidate region based on the first gradient threshold to obtain the first candidate edge points;

[0211] The first pupil edge point determination unit is used to determine the first pupil edge point within the pupil detection area and the non-edge points outside the pupil detection area from the first candidate edge points;

[0212] The second pupil edge point determination unit is used to determine a second pupil edge point that matches the edge point determination template from non-edge points based on a first pupil edge point that matches a preset edge point determination template.

[0213] In an optional embodiment, the pupil edge point connection module includes:

[0214] The initial pupil connected region determination unit is used to determine the connection method corresponding to the template based on the edge points, and to perform connection operations on the first pupil edge points and the second pupil edge points to obtain at least one initial pupil connected region.

[0215] The pupil connectivity region filtering unit is used to select the initial pupil connectivity region that meets the preset region conditions as the pupil connectivity region.

[0216] In an optional embodiment, the edge point fitting module includes:

[0217] The third pupil edge point determination unit is used to determine the pupil edge points corresponding to at least one pupil connected region as the third pupil edge points.

[0218] The first fitting processing unit is used to fit the edge points of the third pupil to obtain the first pupil region;

[0219] The fourth pupil edge point determination unit is used to perform edge point filtering on the third pupil edge point based on the first pupil region to obtain the fourth pupil edge point;

[0220] The second fitting processing unit is used to fit the edge points of the fourth pupil to obtain the target pupil region.

[0221] In an optional embodiment, the fourth pupil edge point determination unit includes:

[0222] The fifth pupil edge point determination unit is used to perform edge point filtering on the third pupil edge point based on the distance between the third pupil edge point and the first pupil region to obtain the fifth pupil edge point;

[0223] The discrete distribution parameter determination unit is used to determine the discrete distribution parameters of the fifth pupil edge point based on the distance between the fifth pupil edge point and the first pupil region;

[0224] The first edge point filtering unit is used to filter the edge point of the fifth pupil based on the distance between the fifth pupil edge point and the first pupil region when the discrete distribution parameter is less than the preset parameter threshold, so as to obtain the fourth pupil edge point.

[0225] In an optional embodiment, the fourth pupil edge point determination unit further includes:

[0226] The noise distribution information determination unit is used to determine the noise distribution information of the fifth pupil edge point in the first pupil region when the discrete distribution parameter is greater than or equal to a preset parameter threshold.

[0227] The sixth pupil edge point determination unit is used to filter the fifth pupil edge point based on the distance between the fifth pupil edge point and the target inscribed curve when the noise distribution information indicates that there is noise in the first pupil area, so as to obtain the sixth pupil edge point; the target inscribed curve is determined based on the circumscribed curve corresponding to the region boundary information, and the region boundary information is the boundary information of the region constructed by the fifth pupil edge point;

[0228] The first edge point filtering unit includes:

[0229] The second edge point filtering unit is used to filter the edge point of the sixth pupil based on the distance between the sixth pupil edge point and the first pupil region when the discrete distribution parameter corresponding to the sixth pupil edge point is less than the preset parameter threshold, so as to obtain the fourth pupil edge point.

[0230] In an optional embodiment, the fourth pupil edge point determination unit further includes:

[0231] The seventh pupil edge point determination unit is used to filter the edge points of the fifth pupil based on the reference line determined by the focus of the first pupil region when the noise distribution information indicates that there is noise outside the first pupil region, so as to obtain the edge points of the seventh pupil.

[0232] The third fitting processing unit is used to fit the edge points of the seventh pupil to determine the region of the second pupil.

[0233] The target ray information acquisition unit is used to acquire target ray information starting from the center position of the curve in the second pupil region;

[0234] The eighth pupil edge point determination unit is used to filter the edge points of the fifth pupil based on the gradient information of the fifth pupil edge points on the target ray information, and obtain the edge points of the eighth pupil.

[0235] The second fitting processing unit includes:

[0236] The target pupil region is obtained by fitting the edge points of the eighth pupil.

[0237] In an optional embodiment, the candidate region includes a pupil spot, and the device further includes:

[0238] The initial spot center position determination module is used to determine the initial spot center position corresponding to the pupil spot from the candidate region;

[0239] The detection area processing module is used to perform area processing on the first spot detection area corresponding to the center position of the initial spot to obtain the second spot detection area.

[0240] The target spot center position determination module is used to determine the target spot center position corresponding to the pupil spot based on the second spot detection area.

[0241] In an optional embodiment, the initial spot center position determination module includes:

[0242] The first region determination unit is used to perform region diffusion processing starting from the detection centroid, and to obtain the region where the grayscale information of the pixels in the candidate region meets the preset grayscale threshold, thus obtaining the third spot detection region; the detection centroid is the centroid detected when acquiring the eye region image;

[0243] The second candidate edge point acquisition unit is used to filter edge points in the candidate region based on the second gradient threshold to obtain the second candidate edge points;

[0244] The first spot edge point determination unit is used to determine the first spot edge point within the third spot detection area from the second candidate edge points;

[0245] The spot connectivity region determination unit is used to connect the edge points of the first spot to obtain the spot connectivity region;

[0246] The initial center determination unit is used to determine the centroid of the connected region of the light spot as the initial center position of the light spot.

[0247] In an optional embodiment, the detection area processing module includes:

[0248] The region integration processing unit is used to perform integration processing on the first spot detection region to determine the region response parameters corresponding to the spot edge points in the first spot detection region.

[0249] The second spot edge point determination unit is used to perform edge point filtering on the spot edge points in the first spot detection area based on the region response parameters to obtain the second spot edge points;

[0250] The second region determination unit is used to determine the detection region of the second spot based on the centroid corresponding to the edge point of the second spot.

[0251] In an optional embodiment, the target spot center location determination module includes:

[0252] The regional gradient processing unit is used to perform gradient processing on the second spot detection region and determine the gradient parameters corresponding to the edge points of the spot in the second spot detection region.

[0253] The third spot edge point determination unit is used to perform edge point filtering on the spot edge points in the second spot detection area based on gradient parameters to obtain the third spot edge points;

[0254] The target center determination unit is used to determine the center position of the target spot corresponding to the pupil spot based on the centroid corresponding to the edge point of the third spot.

[0255] The apparatus provided in the above embodiments can execute the method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in the above embodiments can be found in a pupil localization method provided in any embodiment of this application.

[0256] This embodiment also provides a computer-readable storage medium storing computer-executable instructions, which are loaded by a processor and executed by the pupil localization method described above in this embodiment.

[0257] This embodiment also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method provided in the various optional implementations of the pupil positioning described above.

[0258] This embodiment also provides an electronic device, which includes a processor and a memory, wherein the memory stores a computer program adapted to be loaded by the processor and executed as described above in this embodiment of the pupil localization method.

[0259] The device may be a computer terminal, a mobile terminal, or a server, and may also participate in constituting the apparatus or system provided in the embodiments of this application. For example... Figure 18 As shown, server 18 may include one or more processors 1802 (shown as 1802a, 1802b, ..., 1802n in the figure) 1802 (processor 1802 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPLD, etc.), a memory 1804 for storing data, and a transmission device 1806 for communication functions. In addition, it may also include an input / output interface (I / O interface). Those skilled in the art will understand that... Figure 18 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, server 18 may also include components that are more... Figure 18 The more or fewer components shown, or having the same Figure 18 The different configurations shown.

[0260] It should be noted that the aforementioned one or more processors 1802 and / or other data processing circuitry are generally referred to herein as “data processing circuitry.” This data processing circuitry may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the server 18.

[0261] The memory 1804 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method described in the embodiments of this application. The processor 1802 executes various functional applications and data processing by running the software programs and modules stored in the memory 1804, thereby realizing the above-described method for generating temporal behavior capture boxes based on self-attention networks. The memory 1804 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1804 may further include memory remotely located relative to the processor 1802, and these remote memories can be connected to the server 18 via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0262] This specification provides the operational steps of the methods described in the embodiments or flowcharts, but more or fewer operational steps may be included based on conventional or non-inventive labor. The steps and order listed in the embodiments are merely one possible execution order among many steps and do not represent the only execution order. In actual system or interrupt product execution, the methods shown in the embodiments or drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0263] The structure shown in this embodiment is only a partial structure related to the solution of this application and does not constitute a limitation on the device to which the solution of this application is applied. Specific devices may include more or fewer components than shown, or combinations of certain components, or arrangements of different components. It should be understood that the methods, apparatuses, etc., disclosed in this embodiment can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or unit modules through some interfaces.

[0264] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0265] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this specification can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0266] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A pupil positioning method, characterized by, The method comprises: determining a candidate region containing a pupil from an eye region image; masking the candidate region to determine a pupil detection region corresponding to the pupil from the candidate region; determining pupil edge points corresponding to the pupil detection region based on edge points of the candidate region, wherein the edge points of the candidate region are obtained by performing edge detection on the candidate region; performing a connection operation on the pupil edge points corresponding to the pupil detection region to obtain at least one pupil connected region; performing fitting processing on the pupil edge points corresponding to each of the at least one pupil connected region to obtain a target pupil region; determining a pupil center position corresponding to the pupil based on the target pupil region; The pupil edge points corresponding to the pupil detection region include first pupil edge points and second pupil edge points, and the determination of the pupil edge points corresponding to the pupil detection region based on the edge points of the candidate region comprises: performing edge point filtering on the edge points of the candidate region based on a first gradient threshold to obtain first candidate edge points; determining first pupil edge points within the pupil detection region and non-edge points outside the pupil detection region from the first candidate edge points; determining second pupil edge points matching a preset edge point determination template from the non-edge points based on first pupil edge points matching the edge point determination template.

2. The pupil positioning method of claim 1, wherein, The connection operation on the pupil edge points corresponding to the pupil detection region to obtain at least one pupil connected region comprises: performing a connection operation on the first pupil edge points and the second pupil edge points based on a connection mode corresponding to the edge point determination template to obtain at least one initial pupil connected region; regarding the initial pupil connected region satisfying a preset region condition as the pupil connected region.

3. The pupil positioning method of claim 1, wherein, The fitting processing on the pupil edge points corresponding to each of the at least one pupil connected region to determine the target pupil region comprises: regarding the pupil edge points corresponding to each of the at least one pupil connected region as third pupil edge points; performing fitting processing on the third pupil edge points to obtain a first pupil region; performing edge point filtering on the third pupil edge points based on the first pupil region to obtain fourth pupil edge points; performing fitting processing on the fourth pupil edge points to obtain the target pupil region.

4. The pupil positioning method of claim 3, wherein, The edge point filtering on the third pupil edge points based on the first pupil region to obtain fourth pupil edge points comprises: performing edge point filtering on the third pupil edge points based on the distance between the third pupil edge points and the first pupil region to obtain fifth pupil edge points; determining a discrete distribution parameter of the fifth pupil edge points based on the distance between the fifth pupil edge points and the first pupil region; in a case where the discrete distribution parameter is less than a preset parameter threshold, performing edge point filtering on the fifth pupil edge points based on the distance between the fifth pupil edge points and the first pupil region to obtain the fourth pupil edge points.

5. The pupil positioning method of claim 4, wherein, The method further includes: obtaining noise distribution information of the fifth pupil edge point in the first pupil region when the discrete distribution parameter is greater than or equal to the preset parameter threshold; in a case where the noise distribution information indicates that there is noise in the first pupil region, performing edge point filtering on the fifth pupil edge point based on a distance between the fifth pupil edge point and a target inscribed curve to obtain sixth pupil edge points; the target inscribed curve is determined based on an inscribed curve corresponding to region boundary information; the region boundary information is boundary information of a region constructed by the fifth pupil edge point; The edge point filtering on the fifth pupil edge point based on the distance between the fifth pupil edge point and the first pupil region when the discrete distribution parameter is less than the preset parameter threshold to obtain the fourth pupil edge point includes: in a case where the discrete distribution parameter of the sixth pupil edge point is less than the preset parameter threshold, performing edge point filtering on the sixth pupil edge point based on a distance between the sixth pupil edge point and the first pupil region to obtain the fourth pupil edge point.

6. The pupil positioning method of claim 5, wherein, The method further includes: in a case where the noise distribution information indicates that there is noise outside the first pupil region, performing edge point filtering on the fifth pupil edge point based on a reference straight line determined based on a focal point of the first pupil region to obtain seventh pupil edge points; performing fitting processing on the seventh pupil edge points to determine a second pupil region; obtaining target ray information with a curve center position of the second pupil region as a starting point; performing edge point filtering on the fifth pupil edge point based on gradient information of the fifth pupil edge point on the target ray information to obtain eighth pupil edge points; The fitting processing on the fourth pupil edge points to obtain the target pupil region includes: performing fitting processing on the eighth pupil edge points to obtain the target pupil region.

7. The pupil positioning method of claim 1, wherein, The candidate region includes a pupil spot, and the method further includes: determining an initial spot center position corresponding to the pupil spot from the candidate region; performing region processing on a first spot detection region corresponding to the initial spot center position to obtain a second spot detection region; based on the second spot detection region, determining a target spot center position corresponding to the pupil spot.

8. The pupil positioning method of claim 7, wherein, The determination of the initial spot center position corresponding to the pupil spot from the candidate region includes: performing region diffusion processing with a detection gravity center as a starting point to obtain a third spot detection region from a region in which pixel point gray scale information satisfies a preset gray scale threshold; the detection gravity center is a gravity center detected when the eye region image is obtained. perform edge point filtering on the edge points in the candidate region based on a second gradient threshold to obtain second candidate edge points; determine first glint edge points within the third glint detection region from the second candidate edge points; perform a connection operation on the first glint edge points to obtain a glint connected region; take the center of gravity of the glint connected region as the initial glint center position.

9. A pupil positioning device, characterized in that The apparatus includes: a candidate region determination module configured to determine a candidate region containing a pupil from an eye region image; a mask region determination module configured to perform mask processing on the candidate region to determine a pupil detection region corresponding to the pupil from the candidate region; a pupil edge point determination module configured to determine pupil edge points corresponding to the pupil detection region based on edge points of the candidate region, the edge points of the candidate region being obtained by performing edge detection on the candidate region; a pupil edge point connection module configured to perform a connection operation on the pupil edge points corresponding to the pupil detection region to obtain at least one pupil connected region; an edge point fitting module configured to perform fitting processing on the pupil edge points corresponding to the at least one pupil connected region respectively to obtain a target pupil region; a pupil center determination module configured to determine a pupil center position corresponding to the pupil based on the target pupil region; The pupil edge point connection module is specifically configured to: perform edge point filtering on the edge points of the candidate region based on a first gradient threshold to obtain first candidate edge points; determine first pupil edge points within the pupil detection region and non-edge points outside the pupil detection region from the first candidate edge points; determine second pupil edge points matching a preset edge point determination template from the non-edge points based on the first pupil edge points matching the edge point determination template.

Images