Pupil determination method, apparatus, device, and storage medium

Abstract

The application discloses a pupil determination method, device and equipment and a storage medium. Standard pupil imaging information is acquired; wherein the standard pupil imaging information comprises a standard pupil imaging contour set and a standard pupil center imaging point set; wherein the pupil imaging contour and the pupil center imaging point are in one-to-one correspondence; pupil image information is determined according to an acquired eye image; wherein the pupil image information comprises a pupil image contour and a pupil image center; a target pupil contour is determined according to the pupil image information and the standard pupil imaging information; and a target pupil center is determined according to the standard pupil center imaging point corresponding to the target pupil contour and the pupil image center. The actual pupil center can be obtained by the above scheme of determining the actual pupil center through the pupil image information and the pupil imaging information, so that the accuracy of eye movement tracking can be improved.

Description

Technical Field

[0001] The present invention relates to the field of eye-tracking technology, and in particular to a method, apparatus, device and storage medium for determining pupil size. Background Technology

[0002] During eye tracking, it is necessary to locate the center of the pupil. The accuracy of the pupil center location directly affects the eye tracking effect and is a very critical element in the entire function.

[0003] In related technologies, the pupil center is typically obtained by performing a series of image processing steps on an image of the eye. When the pupil is photographed from different angles, each part of the pupil will undergo a certain degree of distortion, causing a deviation between the pupil center determined by the image and the actual pupil center. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and storage medium for determining the pupil, which can determine the actual pupil center based on the pupil image center, thereby improving the accuracy of eye tracking.

[0005] In a first aspect, embodiments of the present invention provide a pupil determination method, comprising: acquiring standard pupil imaging information; wherein the standard pupil imaging information includes a standard pupil imaging contour set and a standard pupil center imaging point set; wherein the pupil imaging contour corresponds one-to-one with the pupil center imaging point; determining pupil image information based on acquired eye images; wherein the pupil image information includes a pupil image contour and a pupil image center; determining a target pupil contour based on the pupil image information and the standard pupil imaging information; and determining a target pupil center based on the standard pupil center imaging point corresponding to the target pupil contour and the pupil image center.

[0006] Secondly, embodiments of the present invention also provide a pupil determination device, comprising: a standard pupil imaging information acquisition module, used to acquire standard pupil imaging information; wherein the standard pupil imaging information includes a standard pupil imaging contour set and a standard pupil center imaging point set; wherein the pupil imaging contour corresponds one-to-one with the pupil center imaging point; a pupil image information determination module, used to determine pupil image information based on acquired eye images; wherein the pupil image information includes a pupil image contour and a pupil image center; a target pupil contour determination module, used to determine a target pupil contour based on the pupil image information and the standard pupil imaging information; and a target pupil center determination module, used to determine a target pupil center based on the standard pupil center imaging point corresponding to the target pupil contour and the pupil image center.

[0007] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0008] At least one processor; and

[0009] A memory communicatively connected to the at least one processor; wherein,

[0010] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the pupil determination method according to any embodiment of the present invention.

[0011] Fourthly, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute and implement the pupil determination method according to any embodiment of the present invention.

[0012] The technical solution of this embodiment acquires standard pupil imaging information, which includes a standard pupil imaging contour set and a standard pupil center imaging point set, wherein the pupil imaging contour and the pupil center imaging point correspond one-to-one. Pupil image information is determined based on the acquired eye image, wherein the pupil image information includes the pupil image contour and the pupil image center. A target pupil contour is determined based on the pupil image information and the standard pupil imaging information. The target pupil center is determined based on the standard pupil center imaging point corresponding to the target pupil contour and the pupil image center. This scheme, which determines the actual pupil center using pupil image information and pupil imaging information, can obtain the actual pupil center, thereby improving the accuracy of eye tracking. Attached Figure Description

[0013] Figure 1 A flowchart of a pupil determination method provided in an embodiment of the present invention;

[0014] Figure 2 This is a partial pupil imaging outline and pupil center imaging point provided in an embodiment of the present invention;

[0015] Figure 3 This is a schematic diagram of the sub-region and neighboring region where the pupil image center is located, provided in an embodiment of the present invention;

[0016] Figure 4 This is a schematic diagram of a pupil determination device provided in an embodiment of the present invention;

[0017] Figure 5 This is a schematic diagram of the structure of an electronic device that implements the pupil determination method of this invention. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0019] Figure 1 This is a flowchart of a pupil determination method provided in Embodiment 1 of the present invention. This embodiment is applicable to determining the pupil center. The method can be executed by a pupil determination device and specifically includes the following steps:

[0020] S110, Obtain standard pupil imaging information.

[0021] The standard pupil imaging information includes a standard pupil imaging contour set and a standard pupil center imaging point set; the pupil imaging contour and the pupil center imaging point correspond one-to-one. The standard pupil imaging information can be obtained by imaging the pupil edge and pupil center of a virtual eyeball using a virtual camera. The pupil imaging contour can be composed of multiple imaging points corresponding to the pupil edge in the virtual eyeball. The virtual camera and virtual eyeball can be simulated using pre-defined simulation software.

[0022] Optionally, the method for obtaining standard pupil imaging information may be: acquiring a virtual eyeball and a virtual camera; controlling the virtual eyeball to move to multiple states; determining the pupil imaging contour of the virtual eyeball in each state using the virtual camera, and obtaining a set of pupil imaging contours; determining the pupil center imaging point of the virtual eyeball in each state using the virtual camera, and obtaining a set of pupil center imaging points.

[0023] The virtual camera is used to simulate a real camera for taking pictures. The virtual eyeball is used to simulate a real human eyeball. Both the virtual eyeball and the virtual camera are obtained through simulation software. Specifically, in the simulation software, the corresponding virtual eyeball is deployed according to preset eyeball positions, the virtual camera is deployed according to corresponding parameters, and other optical components that affect the optical path between the virtual eyeball and the virtual camera are deployed. These optical components may include the cornea, refractive lenses, plane mirrors, curved mirrors, etc. The virtual camera can be a virtual infrared camera. The multiple states can be understood as the multiple states corresponding to the translation of the virtual eyeball in the x-axis, y-axis, and z-axis directions and / or the rotation angles around the x-axis and y-axis directions and / or changes in pupil diameter.

[0024] Specifically, after acquiring a virtual eyeball and a virtual camera through simulation software, the virtual eyeball can be moved in multiple directions (up / down, left / right) and / or its pupil diameter can be controlled to shrink or enlarge, resulting in multiple states of the virtual eyeball. Based on the camera imaging principle, the pupil contour of the virtual eyeball in multiple states can be obtained through the virtual camera, thus obtaining a set of pupil imaging contours composed of these multiple states. Simultaneously, the pupil center imaging point of the virtual eyeball in multiple states can also be obtained through the virtual camera, thus obtaining a set of pupil center imaging points composed of these multiple states.

[0025] Optionally, the virtual eyeball can be controlled to move to multiple states by: controlling the virtual eyeball to move within a set range of motion with a first set step size and / or controlling the virtual eyeball to scale within a set range of scaling with a second set step size, so that the virtual eyeball moves to multiple states.

[0026] The defined range of motion includes translation along the three coordinate axes, and rotation about the x-axis and y-axis. The three coordinate axes are the x-axis, y-axis, and z-axis. For the coordinate system, a right-handed coordinate system can be used as a reference, with x-positive horizontally to the right, y-positive vertically downwards, and z-positive horizontally forwards.

[0027] The first set step size can be understood as the distance the virtual eyeball moves each time within the set range of motion. Taking translation along the x-axis as an example, if the set range of motion is between -3 mm and 3 mm, starting at 0 mm, and the first set step size is 0.5 mm, then the position after the first translation can be 0.5 mm, and the position after the second translation can be 1 mm.

[0028] The scaling range can be understood as the range of pupil diameter, and the second scaling step can be understood as the size of the pupil diameter scaling each time. For example, if the scaling range is set between 2 mm and 10 mm, and the initial pupil diameter is 5 mm, and the second scaling step is 1 mm, then the pupil diameter after the first scaling can be 6 mm, and the pupil diameter after the second scaling can be 7 mm.

[0029] For example, the virtual eyeball can be controlled to translate along the three coordinate axes as follows: the x-axis translation range can be between -3 mm and 3 mm, with a corresponding first set step size of 0.5 mm; the y-axis translation range can be between -3 mm and 3 mm, with a corresponding first set step size of 0.5 mm; and the z-axis translation range can be between -3 mm and 3 mm, with a corresponding first set step size of 0.5 mm. The virtual eyeball can also be controlled to rotate around the x-axis and y-axis as follows: the rotation angle around the x-axis can be between -5 and 5 degrees, with a corresponding first set step size of 0.5 degrees; the rotation angle around the y-axis can also be between -5 and 5 degrees, with a corresponding first set step size of 0.5 degrees.

[0030] Specifically, the virtual eyeball can be obtained in multiple states based on the translation of the three coordinate axes, the rotation around the x-axis and the y-axis, and the scaling of the virtual eyeball, that is, based on the changes in the six dimensions.

[0031] In this embodiment, the virtual eyeballs in multiple states are not only obtained based on the translation of the three coordinate axes, the rotation around the x-axis and the y-axis, but also take into account the pupil's dilation and contraction. Therefore, in practical applications, when the pupil diameter changes due to various factors, the actual pupil center obtained by this solution can also achieve good results when performing eye tracking, making it applicable to a wider range of scenarios.

[0032] Optionally, after acquiring the virtual eyeball and virtual camera, the method further includes: determining the pupil imaging region on the imaging plane of the virtual camera and dividing the pupil imaging region into multiple sub-regions; correspondingly, determining the pupil imaging contour of the virtual eyeball in each state using the virtual camera to obtain a set of pupil imaging contours, including: determining the pupil imaging contour of the virtual eyeball falling into the pupil imaging region in each state using the virtual camera to obtain a set of pupil imaging contours; correspondingly, determining the pupil center imaging point of the virtual eyeball in each state using the virtual camera to obtain a set of pupil center imaging points, including: determining the pupil center imaging point of the virtual eyeball falling into the pupil imaging region in each state using the virtual camera to obtain a set of pupil center imaging points.

[0033] The imaging plane can be understood as the plane on which the image taken by the virtual camera lies. The pupil imaging area can be understood as the pupil imaging area corresponding to the virtual eyeball in various states within the imaging plane.

[0034] Specifically, the pupil imaging region can be determined in the imaging plane of the virtual camera according to the set range of motion of the virtual eyeball, and the pupil imaging region can be divided into multiple sub-regions to facilitate subsequent calculations. Correspondingly, the pupil imaging region can be a region composed of multiple squares, such as 10*10 or 100*100 squares; this embodiment does not impose such limitations. After determining the pupil imaging region, the pupil outline of the virtual eyeball can be projected onto the pupil imaging region using the virtual camera, and the pupil imaging outlines projected onto the pupil imaging region in multiple states can be determined, thereby obtaining a set of pupil imaging outlines. Correspondingly, the center of the pupil of the virtual eyeball can be projected onto the pupil imaging region using the virtual camera, and the pupil center imaging point projected onto the pupil imaging region in multiple states can be determined, thereby obtaining a set of pupil center imaging points. For example, Figure 2 This image shows a partial pupil imaging contour and a pupil center imaging point provided in an embodiment of the present invention. Figure 2 As shown, the pupil imaging region consists of 10*10 sub-regions, including multiple pupil imaging contours and pupil center imaging points.

[0035] S120. Determine pupil image information based on the collected eye images.

[0036] The pupil image information includes the pupil image outline and the pupil image center. Specifically, eye images can be captured using a real camera, and pupil image information can be obtained using relevant pupil detection methods. For example, deep learning methods and computer graphics operations can be used to fit an ellipse to the pupil in the captured eye image, thus obtaining the pupil image outline. Correspondingly, the center of the ellipse can be the pupil image center, such as the intersection of the major and minor axes of the ellipse or its centroid.

[0037] S130. Determine the target pupil outline based on pupil image information and standard pupil imaging information.

[0038] Specifically, the pupil image contour can be matched with multiple standard pupil imaging contours in a set of standard pupil imaging contours, and the standard pupil imaging contour with the highest matching degree can be used as the target pupil contour.

[0039] Optionally, the method for determining the target pupil contour based on pupil image information and standard pupil imaging information can be as follows: obtain the neighborhood region of the sub-region where the pupil image center is located, and determine it as the target region; obtain the standard pupil imaging contour corresponding to the standard pupil center imaging point falling into the target region, as the first standard pupil imaging contour subset; extract standard pupil imaging contours from the first standard pupil imaging contour subset whose size difference from the pupil image contour size is less than a set threshold, to obtain the second standard pupil imaging contour subset; and determine the standard pupil imaging contour with the highest similarity to the pupil image contour in the second standard pupil imaging contour subset as the target pupil contour.

[0040] Here, the neighborhood region can be understood as the sub-region adjacent to the sub-region where the pupil image center is located. For example, Figure 3 This is a schematic diagram of the sub-region and neighboring region where the pupil image center is located, provided in an embodiment of the present invention. Figure 3 The dots in the diagram can represent the center of the pupil image. The sub-region containing the pupil image center and its neighboring region can be composed of 3*3 sub-regions. The first standard pupil imaging contour subset can be a subset of the standard pupil imaging contour set, and the second standard pupil imaging contour subset can be multiple standard pupil imaging contours selected from the first standard pupil imaging contour subset.

[0041] Specifically, since the imaging planes corresponding to the pupil image center and the standard pupil center imaging point are the same, by dividing the pupil imaging region into multiple sub-regions, the sub-region containing the pupil image center can be determined, and the neighborhood of this sub-region can be used as the target region. After determining the target region, the standard pupil center imaging point falling within the target region can be identified first, and then the standard pupil imaging contour corresponding to this standard pupil center imaging point can be obtained and used as the first standard pupil imaging contour subset. After determining the first standard pupil imaging contour subset, a second standard pupil imaging contour subset can be selected from the first standard pupil imaging contour subset. Specifically, the selection method can be: extracting standard pupil imaging contours from the first standard pupil imaging contour subset whose area difference between the bounding rectangle and the bounding rectangle of the pupil image contour is less than a set threshold; or, extracting standard pupil imaging contours from the first standard pupil imaging contour subset whose area difference between the contour (ellipse) and the pupil image contour area is less than a set threshold. Alternatively, a standard pupil imaging profile can be extracted from the first standard pupil imaging profile subset, provided that the difference between the lengths of its major and minor axes and the lengths of the pupil image profile's major and minor axes is simultaneously less than a set threshold. After obtaining the second standard pupil imaging profile subset, the pupil image profile is compared with all pupil imaging profiles in the second standard pupil imaging profile subset for similarity calculation, and the standard pupil imaging profile with the highest similarity in the second standard pupil imaging profile subset is taken as the target pupil profile. This embodiment does not limit the method used for calculating the similarity between the pupil image profile and the second standard pupil imaging profile subset; for example, structural similarity (SSIM) algorithms can be used.

[0042] Optionally, extracting standard pupil imaging contours whose size differs from the pupil image contour size by less than a set threshold from the first standard pupil imaging contour subset to obtain a second standard pupil imaging contour subset includes: determining the area of ​​the first circumscribed rectangle of each standard pupil imaging contour in the first standard pupil imaging contour subset; determining the area of ​​the second circumscribed rectangle of the pupil image contour; determining the difference between the area of ​​the first circumscribed rectangle and the area of ​​the second circumscribed rectangle; and extracting standard pupil imaging contours whose difference is less than a set threshold from the first standard pupil imaging contour subset to obtain the second standard pupil imaging contour subset.

[0043] Specifically, the area of ​​the first bounding rectangle of each standard pupil imaging contour in the first standard pupil imaging contour subset can be calculated, then the area of ​​the second bounding rectangle of the pupil image contour can be calculated, and the difference between the areas of the first and second bounding rectangles can be calculated. Each difference is compared with a set threshold, and the first standard pupil imaging contour subsets whose differences are less than the set threshold are selected as the second standard pupil imaging contour subsets. In this embodiment, determining the second standard pupil imaging contour subset by calculating the area of ​​the bounding rectangle can reduce computational complexity and improve computational efficiency.

[0044] Optionally, the standard pupil imaging contour with the highest similarity to the pupil image contour in the second standard pupil imaging contour subset is determined as the target pupil contour, including: traversing the standard pupil imaging contours in the second standard pupil imaging contour subset, scaling the pupil image contour so that the size of the scaled pupil image contour is equal to the size of the traversed standard pupil imaging contour; determining the similarity between the scaled pupil image contour and the traversed standard pupil imaging contour; and determining the standard pupil imaging contour with the highest similarity as the target pupil contour.

[0045] Scaling can be understood as proportionally reducing or enlarging. Dimensions can be the area of ​​the circumscribed rectangle, the area of ​​the outline (ellipse), the length of the major and minor axes, etc. This embodiment does not limit the way dimensions are represented.

[0046] Specifically, the standard pupil imaging contours in the second standard pupil imaging contour subset are traversed one by one. During the traversal, the pupil image contours are scaled proportionally so that the size of the scaled pupil image contour is equal to the size of the traversed standard pupil imaging contours. The similarity between the scaled pupil contour and the traversed standard pupil imaging contours is calculated, and the standard pupil imaging contour with the highest similarity is taken as the target pupil contour. In this embodiment, the pupil image contours are scaled proportionally during the similarity calculation to facilitate the calculation of the standard pupil imaging contours and pupil image contours in the second standard pupil imaging contour subset.

[0047] For example, if there are two standard pupil imaging contours in the second standard pupil imaging contour subset, when traversing the first standard pupil imaging contour, the pupil image contour is scaled proportionally so that the size of the scaled pupil image contour is equal to the size of the traversed first standard pupil imaging contour. Then, the similarity between the scaled pupil image contour and the first standard pupil imaging contour is calculated, which can be recorded as the first similarity. When traversing the second standard pupil imaging contour, the pupil image contour is scaled proportionally so that the size of the scaled pupil image contour is equal to the size of the traversed second standard pupil imaging contour. Then, the similarity between the scaled pupil image contour and the second standard pupil imaging contour is calculated, which can be recorded as the second similarity. Comparing the first similarity and the second similarity, the standard pupil imaging contour corresponding to the highest similarity value is determined as the target pupil contour.

[0048] S140. Determine the center of the target pupil based on the standard pupil center imaging point corresponding to the target pupil outline and the center of the pupil image.

[0049] In this embodiment, the center of the target pupil is determined based on the standard pupil center imaging point corresponding to the pupil outline and the center of the pupil image, which can improve the accuracy of determining the pupil center.

[0050] Optionally, the method for determining the target pupil center based on the standard pupil center imaging point corresponding to the target pupil contour and the pupil image center can be as follows: determine the offset based on the center of the target pupil contour and the pupil image center; adjust the standard pupil center imaging point corresponding to the target pupil contour based on the offset to obtain the target pupil center.

[0051] Specifically, by subtracting the center coordinates of the target pupil outline from the center coordinates of the pupil image, the offset between the center of the target pupil outline and the center of the pupil image can be determined. Adding this offset to the standard pupil center imaging point corresponding to the target pupil outline yields the center of the target pupil. For example, assuming the center coordinates of the target pupil outline are point A(X1, Y1), the center coordinates of the pupil image are point B(X2, Y2), and the coordinates of the standard pupil center imaging point corresponding to the target pupil outline are point C(X3, Y3), then the offset between the center of the target pupil outline and the center of the pupil image can be (X1-X2, Y1-Y2), and the center of the target pupil can be (X3+(X1-X2), Y3+(Y1-Y2)).

[0052] The technical solution of this embodiment acquires standard pupil imaging information, which includes a standard pupil imaging contour set and a standard pupil center imaging point set. The pupil imaging contour and the pupil center imaging point correspond one-to-one. Pupil image information is determined based on the acquired eye image, including the pupil image contour and the pupil image center. The target pupil contour is determined based on the pupil image information and the standard pupil imaging information. The target pupil center is determined based on the standard pupil center imaging point corresponding to the target pupil contour and the pupil image center. This scheme, which determines the actual pupil center using pupil image information and pupil imaging information, can obtain the actual pupil center, thereby improving the accuracy of eye tracking. Furthermore, this solution can also solve the problem of calculating the actual pupil center when photographing the pupil from various angles, especially when distortion occurs during large-angle pupil photography.

[0053] Figure 4 This is a schematic diagram of a pupil determination device provided in an embodiment of the present invention. Figure 4 As shown, the device includes: a standard pupil imaging information acquisition module 401, a pupil image information determination module 402, a target pupil contour determination module 403, and a target pupil center determination module 404;

[0054] The standard pupil imaging information acquisition module 401 is used to acquire standard pupil imaging information; wherein, the standard pupil imaging information includes a standard pupil imaging contour set and a standard pupil center imaging point set; wherein, the pupil imaging contour and the pupil center imaging point correspond one-to-one.

[0055] The pupil image information determination module 402 is used to determine pupil image information based on the acquired eye image; wherein, the pupil image information includes the pupil image outline and the pupil image center;

[0056] The target pupil contour determination module 403 is used to determine the target pupil contour based on the pupil image information and the standard pupil imaging information;

[0057] The target pupil center determination module 404 is used to determine the target pupil center based on the standard pupil center imaging point corresponding to the target pupil outline and the pupil image center.

[0058] The technical solution of this embodiment acquires standard pupil imaging information through a standard pupil imaging information acquisition module. This standard pupil imaging information includes a standard pupil imaging contour set and a standard pupil center imaging point set. The pupil imaging contour and the pupil center imaging point correspond one-to-one. A pupil image information determination module determines pupil image information based on the acquired eye image. This pupil image information includes a pupil image contour and a pupil image center. A target pupil contour determination module determines a target pupil contour based on the pupil image information and the standard pupil imaging information. A target pupil center determination module determines the target pupil center based on the standard pupil center imaging point corresponding to the target pupil contour and the pupil image center. This scheme, which determines the actual pupil center using pupil image information and pupil imaging information, can obtain the actual pupil center, thereby improving the accuracy of eye tracking.

[0059] Optionally, the standard pupil imaging information acquisition module is specifically used to: acquire virtual eyeballs and virtual cameras;

[0060] The virtual eyeball is controlled to move to multiple states; the pupil imaging contour of the virtual eyeball in each state is determined by the virtual camera to obtain a set of pupil imaging contours; the pupil center imaging point of the virtual eyeball in each state is determined by the virtual camera to obtain a set of pupil center imaging points.

[0061] Optionally, the standard pupil imaging information acquisition module is also used to: control the virtual eyeball to move within a set range of motion with a first set step size and / or control the virtual eyeball to scale within a set range of scaling with a second set step size, so that the virtual eyeball moves to multiple states; wherein, the set range of motion includes: translation along three coordinate axes, rotation around the x-axis and rotation around the y-axis.

[0062] Optionally, the standard pupil imaging information acquisition module is also used to: determine the pupil imaging area on the imaging plane of the virtual camera, and divide the pupil imaging area into multiple sub-regions;

[0063] Optionally, the standard pupil imaging information acquisition module is further configured to: determine the pupil imaging contour of the virtual eyeball falling into the pupil imaging area in each state through the virtual camera, and obtain a set of pupil imaging contours;

[0064] Optionally, the standard pupil imaging information acquisition module is further configured to: determine the pupil center imaging point of the virtual eyeball falling into the pupil imaging area in each state through the virtual camera, and obtain a set of pupil center imaging points.

[0065] Optionally, the target pupil contour determination module is specifically used for: obtaining the neighborhood region of the sub-region where the pupil image center is located, and determining it as the target region; obtaining the standard pupil imaging contour corresponding to the standard pupil center imaging point falling into the target region, as a first standard pupil imaging contour subset; extracting standard pupil imaging contours from the first standard pupil imaging contour subset whose size difference from the pupil image contour size is less than a set threshold, to obtain a second standard pupil imaging contour subset; and determining the standard pupil imaging contour with the highest similarity to the pupil image contour in the second standard pupil imaging contour subset as the target pupil contour.

[0066] Optionally, the target pupil contour determination module is further configured to: determine the area of ​​the first circumscribed rectangle of each standard pupil imaging contour in the first standard pupil imaging contour subset; determine the area of ​​the second circumscribed rectangle of the pupil image contour; determine the difference between the area of ​​the first circumscribed rectangle and the area of ​​the second circumscribed rectangle; extract the standard pupil imaging contours whose difference is less than a set threshold from the first standard pupil imaging contour subset to obtain the second standard pupil imaging contour subset.

[0067] Optionally, the target pupil contour determination module is further configured to: traverse the standard pupil imaging contours in the second standard pupil imaging contour subset, scale the pupil image contours so that the size of the scaled pupil image contours is equal to the size of the traversed standard pupil imaging contours; determine the similarity between the scaled pupil image contours and the traversed standard pupil imaging contours; and determine the standard pupil imaging contour with the highest similarity as the target pupil contour.

[0068] Optionally, the target pupil center determination module is specifically used to: determine an offset based on the center of the target pupil contour and the center of the pupil image; and adjust the standard pupil center imaging point corresponding to the target pupil contour based on the offset to obtain the target pupil center.

[0069] The above-described apparatus can execute the methods provided in all the foregoing embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the above methods. Technical details not described in detail in this embodiment can be found in the methods provided in all the foregoing embodiments of the present invention.

[0070] Figure 5A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0071] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0072] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0073] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the method pupil determination.

[0074] In some embodiments, the method pupil determination may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the method pupil determination described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform method pupil determination by any other suitable means (e.g., by means of firmware).

[0075] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0076] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0077] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0078] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0079] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0080] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0081] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0082] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for determining pupil size, characterized in that, include: Obtain standard pupil imaging information; wherein, the standard pupil imaging information includes a standard pupil imaging contour set and a standard pupil center imaging point set; wherein, the pupil imaging contour and the pupil center imaging point correspond one-to-one; The process of acquiring standard pupil imaging information includes: acquiring a virtual eyeball and a virtual camera; controlling the virtual eyeball to move to multiple states; determining the pupil imaging contour of the virtual eyeball in each state using the virtual camera to obtain a set of pupil imaging contours; and determining the pupil center imaging point of the virtual eyeball in each state using the virtual camera to obtain a set of pupil center imaging points. After acquiring the virtual eyeball and virtual camera, the method further includes: determining the pupil imaging region on the imaging plane of the virtual camera, and dividing the pupil imaging region into multiple sub-regions; Pupil image information is determined based on the acquired eye images; wherein, the pupil image information includes the pupil image outline and the pupil image center; The target pupil outline is determined based on the pupil image information and the standard pupil imaging information; The determination of the target pupil contour based on the pupil image information and the standard pupil imaging information includes: obtaining a neighborhood region of the sub-region where the center of the pupil image is located, and determining it as the target region; obtaining the standard pupil imaging contour corresponding to the standard pupil center imaging point falling into the target region, as a first standard pupil imaging contour subset; extracting standard pupil imaging contours from the first standard pupil imaging contour subset whose size difference from the size of the pupil image contour is less than a set threshold to obtain a second standard pupil imaging contour subset; and determining the standard pupil imaging contour with the highest similarity to the pupil image contour in the second standard pupil imaging contour subset as the target pupil contour. The center of the target pupil is determined based on the standard pupil center imaging point corresponding to the target pupil outline and the center of the pupil image.

2. The method according to claim 1, characterized in that, Controlling the virtual eyeball to move to multiple states, including: Control the virtual eyeball to move within a set range of motion with a first set step size and / or control the virtual eyeball to scale within a set range of scaling with a second set step size, so that the virtual eyeball moves to multiple states; wherein, the set range of motion includes: translation along three coordinate axes, rotation around the x-axis and rotation around the y-axis.

3. The method according to claim 1, characterized in that, The pupil imaging contour of the virtual eyeball is determined by the virtual camera in various states, resulting in a pupil imaging contour set, including: The pupil imaging contour of the virtual eyeball falls into the pupil imaging area in each state by using the virtual camera, and a pupil imaging contour set is obtained. Accordingly, the pupil center imaging point of the virtual eyeball in each state is determined by the virtual camera, and a set of pupil center imaging points is obtained, including: The virtual camera determines the pupil center imaging points that fall into the pupil imaging area under various states, thus obtaining a set of pupil center imaging points.

4. The method according to claim 1, characterized in that, A second standard pupil imaging profile subset is obtained by extracting standard pupil imaging profiles whose size differs from the pupil image profile size by less than a set threshold from the first standard pupil imaging profile subset, including: Determine the area of ​​the first bounding rectangle of each standard pupil imaging profile in the first standard pupil imaging profile subset; Determine the area of ​​the second bounding rectangle of the pupil image contour; Determine the difference between the area of ​​the first circumscribed rectangle and the area of ​​the second circumscribed rectangle; Extract the standard pupil imaging profiles whose difference is less than a set threshold from the first standard pupil imaging profile subset to obtain the second standard pupil imaging profile subset.

5. The method according to claim 1, characterized in that, The target pupil contour is determined by identifying the standard pupil imaging contour with the highest similarity to the pupil image contour in the second standard pupil imaging contour subset, including: Traverse the standard pupil imaging contours in the second standard pupil imaging contour subset, and scale the pupil image contours so that the size of the scaled pupil image contours is equal to the size of the traversed standard pupil imaging contours. Determine the similarity between the scaled pupil image contour and the traversed standard pupil imaging contour; The standard pupil imaging contour with the highest similarity is determined as the target pupil contour.

6. The method according to claim 1, characterized in that, Determining the target pupil center based on the standard pupil center imaging point corresponding to the target pupil contour and the pupil image center includes: The offset is determined based on the center of the target pupil outline and the center of the pupil image; The standard pupil center imaging point corresponding to the target pupil contour is adjusted based on the offset to obtain the target pupil center.

7. A pupil determining device, characterized in that, include: A standard pupil imaging information acquisition module is used to acquire standard pupil imaging information; wherein, the standard pupil imaging information includes a standard pupil imaging contour set and a standard pupil center imaging point set; wherein, the pupil imaging contour and the pupil center imaging point correspond one-to-one. Specifically, the standard pupil imaging information acquisition module is used to: acquire a virtual eyeball and a virtual camera; control the virtual eyeball to move to multiple states; determine the pupil imaging contour of the virtual eyeball in each state through the virtual camera, and obtain a set of pupil imaging contours; and determine the pupil center imaging point of the virtual eyeball in each state through the virtual camera, and obtain a set of pupil center imaging points. The standard pupil imaging information acquisition module is also used to: determine the pupil imaging area on the imaging plane of the virtual camera, and divide the pupil imaging area into multiple sub-regions; A pupil image information determination module is used to determine pupil image information based on acquired eye images; wherein, the pupil image information includes the pupil image outline and the pupil image center; A target pupil contour determination module is used to determine the target pupil contour based on the pupil image information and the standard pupil imaging information; Specifically, the target pupil contour determination module is used to: obtain the neighborhood region of the sub-region where the center of the pupil image is located, and determine it as the target region; obtain the standard pupil imaging contour corresponding to the standard pupil center imaging point falling into the target region, as a first standard pupil imaging contour subset; extract standard pupil imaging contours from the first standard pupil imaging contour subset whose size difference with the pupil image contour size is less than a set threshold, to obtain a second standard pupil imaging contour subset; and determine the standard pupil imaging contour with the highest similarity to the pupil image contour in the second standard pupil imaging contour subset as the target pupil contour. The target pupil center determination module is used to determine the target pupil center based on the standard pupil center imaging point corresponding to the target pupil outline and the center of the pupil image.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the pupil determination method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the pupil determination method according to any one of claims 1-6.

Images