Method for determining cornea center and interpupillary distance and related device

Abstract

The present disclosure provides a method for determining a cornea center and an interpupillary distance and a related device. Provided is the method for determining a cornea center, which comprises: acquiring a plurality of camera parameters of a camera, wherein the camera parameters comprise a starting point position and a space linear equation penetrating through the starting point position; acquiring an eye image acquired by the camera, wherein the eye image comprises a first light spot formed by a light source, and the light source is arranged around the camera; determining, according to pixel point coordinates of the first light spot in the eye image and the plurality of camera parameters, a direction vector from a virtual point to the cornea center; determining, according to a positional relationship among the light source, a second light spot formed by the light source on the cornea, and the cornea center, a distance from the virtual point to the cornea center; and determining, according to the direction vector from the virtual point to the cornea center and the distance from the virtual point to the cornea center, three-dimensional positional information of the cornea center.

Description

Method for determining corneal center and pupil distance and related equipment

[0001] This application claims priority to the Chinese invention patent application with application number 202311767248.6 filed on December 20, 2023 and titled “Method for determining corneal center and pupil distance and related equipment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present disclosure relates to the field of extended reality technology, and in particular to a method for determining a corneal center and pupil distance and related equipment. Background Art

[0003] Extended Reality (XR) refers to the use of computers to combine the real and virtual worlds to create a virtual environment where humans and machines can interact. XR technology can further encompass augmented reality (AR), virtual reality (VR), and mixed reality (MR), leveraging hardware devices combined with various technologies to fuse virtual content with real-world scenes.

[0004] Generally, an extended reality system provides users with wearable devices to enable human-computer interaction. The wearable device can be a head-mounted wearable device. In some scenarios, the wearable device can collect images of the human eye to perform calculations to achieve gaze tracking or pupil distance estimation. Summary of the Invention

[0005] The present disclosure proposes a method for determining the corneal center and the pupil distance and related devices to solve or partially solve the above problems.

[0006] In a first aspect of the present disclosure, a method for determining the center of the cornea is provided, comprising: acquiring multiple camera parameters of a camera, the camera parameters including a starting position and an equation of a spatial straight line passing through the starting position; acquiring an eye image captured by the camera, the eye image including a first light spot formed by a light source, and the light source being arranged around the camera; determining a direction vector from a virtual point to the center of the cornea based on the pixel coordinates of the first light spot in the eye image and the multiple camera parameters; determining a distance from the virtual point to the center of the cornea based on a positional relationship between the light source, a second light spot formed by the light source on the cornea, and the center of the cornea; and determining three-dimensional position information of the center of the cornea based on the direction vector from the virtual point to the center of the cornea and the distance from the virtual point to the center of the cornea.

[0007] In a second aspect of the present disclosure, a method for determining pupillary distance is provided, comprising: determining three-dimensional position information of a corneal center according to the method described in the first aspect; determining two-dimensional position information of a pupil center in an eye image; determining three-dimensional position information of the pupil center based on the three-dimensional position information of the corneal center and the two-dimensional position information of the pupil center; and determining pupillary distance based on the three-dimensional position information of the corneal center and the three-dimensional position information of the pupil center.

[0008] In a third aspect of the present disclosure, a cornea center determination device is provided, comprising: a first acquisition module, configured to: acquire multiple camera parameters of a camera, wherein the camera parameters include a starting position and a spatial straight line equation passing through the starting position; a second acquisition module, configured to: acquire an eye image captured by the camera, wherein the eye image includes a first light spot formed by a light source, and the light source is arranged around the camera; a first determination module, configured to: determine a direction vector from a virtual point to the cornea center based on the pixel coordinates of the first light spot in the eye image and the multiple camera parameters; a second determination module, configured to: determine a distance from the virtual point to the cornea center based on a positional relationship between the light source, a second light spot formed by the light source on the cornea, and the cornea center; a third determination module, configured to: determine three-dimensional position information of the cornea center based on the direction vector from the virtual point to the cornea center and the distance from the virtual point to the cornea center.

[0009] In a fourth aspect of the present disclosure, a pupillary distance determination device is provided, comprising: a first determination module configured to determine three-dimensional position information of a corneal center according to the method described in the first aspect; a second determination module configured to determine two-dimensional position information of a pupil center in an eye image; a third determination module configured to determine three-dimensional position information of the pupil center based on the three-dimensional position information of the corneal center and the two-dimensional position information of the pupil center; and a fourth determination module configured to determine the pupillary distance based on the three-dimensional position information of the corneal center and the three-dimensional position information of the pupil center.

[0010] In a fifth aspect of the present disclosure, a computer device is provided, comprising one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the programs include instructions for executing the method according to the first aspect or the second aspect.

[0011] In a sixth aspect of the present disclosure, a non-volatile computer-readable storage medium containing a computer program is provided, which, when executed by one or more processors, enables the processors to execute the method described in the first aspect or the second aspect.

[0012] In a seventh aspect of the present disclosure, a computer program product is provided, comprising computer program instructions, which, when executed on a computer, cause the computer to execute the method described in the first aspect or the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0014] FIG1A shows a schematic diagram of an exemplary system provided by an embodiment of the present disclosure.

[0015] FIG1B shows a schematic diagram of an exemplary head-mounted wearable device.

[0016] 1C and 1D are schematic diagrams showing exemplary human eye images.

[0017] FIG2A shows a schematic diagram of an exemplary wearable device provided by an embodiment of the present disclosure.

[0018] FIG2B shows a schematic diagram of another exemplary wearable device provided by an embodiment of the present disclosure.

[0019] FIG2C shows a schematic diagram of another exemplary wearable device provided by an embodiment of the present disclosure.

[0020] FIG3 is a schematic diagram showing exemplary camera parameters according to an embodiment of the present disclosure.

[0021] FIG4 shows a schematic diagram of an exemplary eyeball model according to an embodiment of the present disclosure.

[0022] FIG5A shows a flowchart of an exemplary method provided by an embodiment of the present disclosure.

[0023] FIG5B shows a schematic diagram of an exemplary eyeball model for solving the corneal center according to an embodiment of the present disclosure.

[0024] FIG5C is a flowchart illustrating an exemplary method for determining a direction vector from a virtual point to the center of the cornea according to an embodiment of the present disclosure.

[0025] FIG5D shows a flowchart of an exemplary method for determining the distance from the virtual point to the center of the cornea according to an embodiment of the present disclosure.

[0026] FIG5E is a flowchart illustrating an exemplary method for determining the distance from the virtual point to the center of the cornea according to an embodiment of the present disclosure.

[0027] FIG5F shows a flowchart of another exemplary method provided by an embodiment of the present disclosure.

[0028] FIG5G shows a schematic diagram of an exemplary eyeball model for solving the pupil center according to an embodiment of the present disclosure.

[0029] FIG5H shows a flowchart of an exemplary method for determining a pupil center according to an embodiment of the present disclosure.

[0030] FIG5I is a schematic diagram illustrating an exemplary scenario for solving refracted light according to an embodiment of the present disclosure.

[0031] FIG6 shows a schematic diagram of the hardware structure of an exemplary computer device provided by an embodiment of the present disclosure.

[0032] FIG7 shows a schematic diagram of an exemplary device provided by an embodiment of the present disclosure.

[0033] FIG8 shows a schematic diagram of another exemplary device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0035] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should have the usual meanings understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative position relationships. When the absolute position of the described object changes, the relative position relationship may also change accordingly.

[0036] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0037] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0038] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0039] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0040] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0041] FIG1A shows a schematic diagram of an exemplary augmented reality system 100 provided by an embodiment of the present disclosure.

[0042] Extended Reality (XR) refers to the use of computers to combine the real and virtual worlds to create a virtual environment where humans and machines can interact. XR technology can further encompass augmented reality (AR), virtual reality (VR), and mixed reality (MR), leveraging hardware devices combined with various technologies to fuse virtual content with real-world scenes.

[0043] As shown in Figure 1A, the system 100 may include various types of wearable devices, such as head-mounted wearable devices (e.g., VR / AR glasses or head-mounted displays (HMD)) 104, operating handles 108, etc. In some scenarios, a camera / camera 110 for taking photos of the operator (user) 102 may also be provided. In some embodiments, when the aforementioned device does not have a processing function, the system 100 may also include an external control device 112 for providing a processing function. The control device 112, for example, may be a computer device such as a mobile phone or a computer. In some embodiments, when any of the aforementioned devices serves as a control device or a main control device, it may communicate with other devices in the system 100 through wired or wireless communication to achieve information interaction.

[0044] In system 100, user 102 can interact with augmented reality system 100 using head-mounted wearable device 104 and operating handle 108. In some scenarios, system 100 can use images captured by camera 110 to identify user 102's posture and gestures, and then interact with user 102 based on the identified posture and gestures. In some embodiments, user 130 can also implement gesture input using bare hands. Head-mounted wearable device 104 can capture front-facing images in real time using a camera or other device located in front of head-mounted wearable device 104, and recognize user 130's gestures by recognizing these images.

[0045] In some embodiments, as shown in FIG1A , the system 100 may further communicate with a server 114 and obtain data, such as images, audio, and video, from the server 114 and output the data through the head-mounted wearable device 104 , for example, by displaying images or videos on a display screen of the head-mounted wearable device 104 , playing audio and video audio using a speaker of the head-mounted wearable device 104 , etc. In some embodiments, as shown in FIG1A , the server 114 may retrieve the required data, such as images, audio, and video, from a database server 116 for storing data.

[0046] In some embodiments, a collection unit for collecting information may be provided on the head-mounted wearable device 104. The collection unit may be of various types.

[0047] In some embodiments, the acquisition unit may further include an environment acquisition unit and a positioning tracking unit, wherein the environment acquisition unit may be used to acquire environmental information around (for example, in front of) the wearable device 104, and the positioning tracking unit may be used to track the wearable device 104. Optionally, the environment acquisition unit may include but is not limited to a three-color camera (for example, an RGB camera), a depth camera, a binocular camera, a laser or other photosensitive element, and the positioning tracking unit may include but is not limited to visual simultaneous localization and mapping (visual SLAM), an inertial measurement unit (IMU), a global positioning system (GPS), ultra-wideband wireless communication technology (UWB), a laser or other modules.

[0048] In some embodiments, the head-mounted wearable device 104 may also be provided with a speed sensor, an acceleration sensor, an angular velocity sensor (e.g., a gyroscope), etc. for collecting speed information or acceleration information of the head-mounted wearable device 104. For another example, the operating handle 108 may also be provided with a speed sensor, an acceleration sensor, an angular velocity sensor (e.g., a gyroscope), etc. for collecting speed information or acceleration information of the operating handle 108. It should be noted that, in addition to being provided on the head-mounted wearable device 104 and the operating handle 108, the aforementioned collection unit may also be provided directly on a body part of the interactive user 102 by attachment without relying on a hardware device, thereby collecting relevant information of the body part, such as speed, acceleration, or angular velocity information, or information collected by other sensors or collection units (e.g., human eye images (including pupil images), etc.).

[0049] In some embodiments, the head-mounted wearable device 104 may also be provided with a camera or a camera for taking photos of the operator (user) 102 (eg, photos of hands or feet) and images of the environment.

[0050] In some embodiments, the system 100 can identify the posture and gestures of the user 102 through the collected information, and then can perform corresponding interactions based on the identified user posture and gestures.

[0051] FIG1B shows a schematic diagram of an exemplary head-mounted wearable device 104 .

[0052] As shown in FIG1B , the head-mounted wearable device 104 may include a lens barrel 1042, inside which a display screen 1044 for displaying an image and an optical component 1046 for processing the light path may be provided. Optionally, the optical component 1046 may further include a plurality of lenses (e.g., lenses 1046A and 1046B). The combination of the plurality of lenses may project light emitted by the display screen 1044 into the human eye 1022, so that the human eye 1022 may view the image displayed on the display screen 1044. It will be understood that FIG1B only exemplarily shows a single-sided structure of the head-mounted wearable device 104. In order to achieve binocular display, the head-mounted wearable device 104 may include two lens barrel structures arranged side by side.

[0053] In some embodiments, as shown in FIG1B , the head-mounted wearable device 104 may also be provided with a camera 1048 for capturing images of the human eye. The camera 1048 may be a charge coupled device (CCD) image sensor, a complementary metal oxide semiconductor (CMOS) image sensor, or the like.

[0054] Optionally, the camera 1048 may be an eye tracking (ET) camera, and the human eye images collected by the camera may be used to implement functions such as pupil distance estimation and eye tracking.

[0055] As shown in FIG1B , in the related art, camera 1048 is usually set outside the lens barrel and usually only one camera is set for each lens barrel. In addition, in order to better capture the complete image of the human eye and not affect the human eye's observation of the screen 1022, the common deployment position of camera 1048 is generally at the outer corner of the eye, the wing of the nose, or directly below the eye. Referring to FIG1B , if camera 1048 is close to the outside of the device, the camera deployment position shown in FIG1B is the outer corner of the eye. If camera 1048 is close to the inside of the device, the camera deployment position shown in FIG1B is the wing of the nose.

[0056] However, the inventors of the present disclosure discovered that the camera mounting method used in the related art easily causes camera 1048 to be mounted at a large angle relative to eye 1022. This results in a large angle α between the orientation of camera 1048 and the normal viewing direction of eye 1022 (i.e., camera 1048 is tilted at a high degree). This makes it difficult for the captured eye image to reflect the normal viewing angle of the eye, as shown in Figures 1C and 1D. Furthermore, within the range of eye movement, a single camera 1048 is unable to capture the complete range of eye movement.

[0057] In some cases, the user may need to wear glasses before using the head-mounted wearable device 104. However, since the camera 1048 is set outside the lens barrel 1042 (especially when the camera 1048 is set at the nose wing position), the camera 1048 is higher than the lens barrel 1042, which makes it easy to collide with or squeeze the glasses, affecting the wearing comfort of the head-mounted wearable device 104 and possibly scratching the user's glasses. At the same time, the imaging of the camera 1048 is easily affected by the edge of the glasses. The refraction of light passing through the edge of the glasses reduces the image clarity of the camera 1048 and forms many refracted light spots in the image, which reduces the clarity of the image captured by the camera and produces large distortion, thereby affecting the accuracy of the subsequent pupil distance estimation algorithm and the line of sight estimation algorithm. Alternatively, when the glasses have a frame, the camera is easily blocked by the edge of the glasses and cannot capture a complete eye image, resulting in the failure of the pupil distance estimation function and the line of sight estimation function. In particular, when there is only one camera corresponding to the lens barrel, this problem will be further aggravated.

[0058] In view of this, an embodiment of the present disclosure provides a wearable device in which a camera is disposed inside a lens barrel, which can solve or partially solve the above-mentioned problems to a certain extent.

[0059] FIG2A shows a schematic diagram of an exemplary wearable device 200 provided by an embodiment of the present disclosure.

[0060] As shown in FIG2A , similarly, the wearable device 200 may also include a lens barrel 202, a display screen 204 disposed inside the lens barrel 202, and an optical assembly 206. The optical assembly 206 may further include multiple lenses (e.g., lenses 206A and 206B). The combination of the multiple lenses may project light emitted by the display screen 204 into the human eye 1022, so that the human eye 1022 can view the image displayed on the display screen 204.

[0061] Unlike the wearable device 104 shown in FIG1B , the camera 208 of the wearable device 200 is arranged inside the lens barrel 202 and toward the light-emitting side of the lens barrel. Since the camera 208 is placed inside the lens barrel 202, it will not affect the wearing of glasses, thereby improving the comfort of the wearable device 200. At the same time, as shown in FIG2A , since the camera 208 is arranged inside the lens barrel 202, the distance between the camera 208 and the human eye 1022 is extended, so that the installation angle of the camera 208 relative to the human eye 1022 becomes smaller, and then the angle β between the direction of the camera 208 and the normal view direction of the human eye 1022 is reduced, so that the camera 208 has a better observation angle, and the collected human eye image can better reflect the image of the human eye normal view angle, and the imaging quality is better. In addition, since the camera 208 is placed inside the lens barrel 202, the glasses will not interfere with the imaging of the camera 208, further improving the imaging quality. The improvement in imaging quality also improves the accuracy of algorithms such as pupil distance estimation or line of sight tracking.

[0062] FIG2B shows a schematic diagram of another exemplary wearable device 200 provided by an embodiment of the present disclosure.

[0063] As shown in FIG2B , in some embodiments, unlike FIG2A , camera 208 faces display screen 204 , and wearable device 200 may further include a reflective structure 210 , which may be a reflective film, a reflective mirror, or other reflective structure. By reflecting light from reflective structure 210 , camera 208 can still capture images of the human eye. Furthermore, because an additional reflection process is added to the optical path, the observation angle γ is further reduced, enabling camera 208 to better capture images.

[0064] FIG2C shows a schematic diagram of another exemplary wearable device 200 provided by an embodiment of the present disclosure.

[0065] As shown in FIG2C , in some embodiments, unlike FIG2B , reflective structure 210 can be a semi-transparent, semi-reflective film that transmits half of the light and reflects the other half. Thus, reflective structure 210 can reflect light from the eye outside the lens barrel into camera 208 and transmit light from display screen 204. As can be seen from FIG2B , the use of reflective structure 210 in FIG2C avoids the problem of light blocking caused by the opacity of the reflective structure. Furthermore, the location of reflective structure 210 does not need to be specifically designed based on the location of camera 208; it can simply be placed on the light-emitting side of display screen 204.

[0066] In some embodiments, as shown in Figures 2A to 2C, the wearable device 200 may further include a point light source 212 further provided on the outside of the lens 202. The light emitted by the point light source 212 can form reflected light on the cornea of ​​the human eye 1022 so as to be collected by the camera 208, and then form a light spot in the image obtained by the camera 208. According to the position of the light spot in the image, it can be used to assist in locating the center of the cornea and the position of the pupil. In some embodiments, in order to make the positioning more accurate, the number of point light sources 212 can be two or more. Optionally, the point light source 212 can be a light emitting diode (LED). In some embodiments, the light emitted by the point light source 212 may not be visible light (for example, it may be infrared light) or may be visible light with relatively weak brightness, thereby improving the user's comfort.

[0067] It can be understood that FIG. 2A to FIG. 2C only exemplarily illustrate a single-side structure of the wearable device 200. In order to achieve binocular display, the wearable device 200 may include two lens barrel structures arranged side by side.

[0068] It can be seen from the above embodiments that the embodiment of the present disclosure not only avoids the aforementioned adverse effects by setting the camera 208 inside the wearable device 200, but also increases the distance between the camera and the eye, thereby having a better camera observation angle and obtaining better imaging quality (i.e., the camera can capture higher quality eye images), which is beneficial to improving the accuracy of the pupil distance estimation algorithm and the line of sight estimation algorithm, and at the same time can also improve the wearing comfort of the wearable device 200.

[0069] Furthermore, the inventors of the present disclosure discovered that, compared with placing the camera outside the lens barrel, when the camera is placed inside the lens barrel, the imaging distortion is affected not only by the camera module itself but also by the optical components in the lens barrel when the camera is imaging.

[0070] As shown in Figures 2A to 2C , compared to the camera 1048 in Figure 1B , the optical component 206 or a portion of the optical component 206 within the lens barrel is also present in the optical path from the camera 208 to the human eye 1022 (the position may vary depending on the relative position of the camera 208 and the lenses in the optical component 206; for example, in addition to the position shown in Figures 2A to 2C , the camera 208 may also be positioned between lens 206A and lens 206B). Therefore, when the camera 208 images the human eye 1022, the optical component 206 or a portion of the optical component 206 affects or changes the optical path from the camera 208 to the human eye 1022, resulting in inconsistent refractive indices at various locations within the lens barrel. This causes the optical axis of the camera 208 to not completely coincide with the optical axis of the optical component 206 within the lens barrel, resulting in potentially asymmetrical distortion of the image captured by the camera 208 (i.e., the distortion produced at various locations in the image captured by the camera is different).

[0071] Furthermore, since the optical path from the camera 208 to the human eye 1022 is changed, the camera 208 does not have a unified projection center, making it impossible to use a central camera model to fit the projection process of the camera in the lens barrel to process the distortion, making it difficult to calibrate the camera parameters.

[0072] In view of this, an embodiment of the present disclosure further provides a camera parameter calibration method, which can use a non-camera center model to implement parameter calibration of the camera in the lens barrel.

[0073] FIG3 shows a schematic diagram of exemplary camera parameters according to an embodiment of the present disclosure.

[0074] As shown in Figure 3, the camera parameters obtained by calibration are the spatial straight line equations corresponding to some pixel points (that is, each pixel point in the camera imaging corresponds to a direction vector and a starting point position in the three-dimensional space). In some embodiments, in order to save the amount of calculation, only some key pixel points (control points) are sampled and their spatial straight line equations and their starting point positions are saved as camera parameters. The camera parameters corresponding to other pixel points can be obtained by quadratic spline interpolation. As shown in Figure 3, the spatial straight lines corresponding to each pixel point do not strictly intersect at one point (for example, point A and point B are the intersection points obtained by the convergence of different spatial straight lines), but are distributed in a range interval, indicating that the camera is non-central, that is, there is no unique projection center.

[0075] Therefore, in order to ensure the accuracy of the algorithm, when using a non-parametric camera calibration model to calculate the pupil position and line of sight direction, for the pixel points in the image captured by the camera, since the camera parameters only retain the spatial straight line equations corresponding to the control points, it is necessary to perform quadratic spline interpolation on the starting point position and direction vector in the camera parameters to restore the spatial straight line equations corresponding to all pixel points. The calculation is large and complex.

[0076] In order to simplify the algorithm and save computational effort, in some embodiments, a central camera approximation can be performed under the full camera field of view. By forcibly fitting the non-central camera model with the central camera model, a camera center is obtained, thereby eliminating the need for quadratic spline interpolation of the starting point position, thereby saving computational effort.

[0077] In some scenarios, after the camera parameters are known, gaze tracking or pupil distance estimation can be completed based on the camera parameters.

[0078] FIG4 shows a schematic diagram of an exemplary eyeball model according to an embodiment of the present disclosure.

[0079] As shown in Figure 4, the eye is simplified into an eyeball model. The eyeball is a sphere that rotates around the eyeball's rotation center (abbreviated as the eyeball rotation center). The cornea is also simplified into a sphere. The eyeball rotation center, corneal center, and pupil center are collinear. This line is the line of sight. The line of sight can be determined by calculating any two points among the eyeball rotation center, corneal center, and pupil center. Pupillary distance estimation can be achieved by calculating the distance between the centers of the left and right corneas.

[0080] One approach to calculating the corneal center is corneal reflection calculation. In this approach, infrared light sources (e.g., infrared LEDs) are placed around the camera. The cornea is approximated as a spherical mirror, which reflects the light from the light source, forming a light spot. This light spot can then be reflected in the image captured by the camera.

[0081] According to the laws of reflection and refraction of light, based on the intrinsic and extrinsic parameters of the camera, the position of the light source, the light spot, and the pixel position of the pupil in the image formed by the camera, the three-dimensional coordinates of the center of the cornea and the center of the pupil can be solved, thereby obtaining the pupil distance and line of sight direction.

[0082] Generally, a camera is approximated as a pinhole imager, that is, the camera has a fixed center of gravity, and the camera intrinsic parameter is a matrix. Based on this matrix, 2D pixels can be directly projected onto the 3D camera plane through matrix multiplication, or 3D points can be projected onto the pixel plane to calculate the correspondence between pixels and 3D points (lines).

[0083] Returning to FIG3 , as shown in the figure, the camera parameters obtained by calibration using a non-central camera model are the spatial straight line equations corresponding to some pixel points (that is, each pixel point of the camera imaging corresponds to a direction vector and a starting point position in three-dimensional space). In some embodiments, in order to save computational effort, only some key pixel points (control points) are sampled and their spatial straight line equations and their starting point positions are saved as camera parameters, and the camera parameters corresponding to other pixel points can be obtained using quadratic spline interpolation. As shown in FIG3 , the spatial straight lines corresponding to each pixel point do not strictly intersect at one point (for example, point A and point B are the intersection points obtained by the convergence of different spatial straight lines), but are distributed in a range, indicating that the camera is non-central, that is, there is no unique projection center.

[0084] Therefore, the inventors of the present disclosure found that in the related art, if the camera parameters of the wearable device are calibrated based on a non-central camera model, it is difficult to use the traditional method to calibrate the camera parameters based on the non-central camera model and solve the corneal center through the light-light source (LED) coplanarity scheme.

[0085] In view of this, an embodiment of the present disclosure provides a method for determining the center of the cornea to solve or partially solve the above-mentioned problem.

[0086] FIG5A shows a flowchart of an exemplary method 500 provided in an embodiment of the present disclosure.

[0087] The method 500 may be used to determine the center of the cornea. As shown in FIG. 5A , the method 500 may further include the following steps 502 , 504 , 506 , 508 , and 510 .

[0088] In step 502, a plurality of camera parameters of the camera are obtained. In some embodiments, the camera parameters include a starting point position and a spatial straight line equation passing through the starting point position, as shown in FIG3 .

[0089] At step 504, an eye image captured by the camera is acquired. In some embodiments, the eye image includes a first light spot formed by a light source.

[0090] In step 506 , a direction vector from the virtual point to the corneal center is determined based on the pixel coordinates of the first light spot in the eye image and the multiple camera parameters.

[0091] FIG5B shows a schematic diagram of an exemplary eyeball model for solving the corneal center according to an embodiment of the present disclosure.

[0092] As shown in FIG5B , the eyeball model includes a cornea center c and a pupil center p. As shown in FIG5B , a light source l is provided around the camera. i(The figure shows light sources l1 and l2), the light source can form a second light spot q on the cornea i (The second light spots q1 and q2 are shown in the figure), the second light spot q i is the light source l in three-dimensional space i The reflected light spot formed on the cornea by the emitted light. In this embodiment, the first light spot refers to the image formed by the second light spot in the eye image, that is, the light spot image formed by the second light spot in the eye image.

[0093] Optionally, as shown in FIG5B , a common virtual point may be defined The virtual point It can be used to characterize the common starting point of the plurality of camera parameters. Specifically, all starting points of the plurality of camera parameters s i Through virtual point and virtual points To the starting position s i In some embodiments, as shown in FIG5C , step 506 of determining the direction vector from the virtual point to the corneal center based on the pixel coordinates of the first light spot in the eye image and the multiple camera parameters may further include the following steps 5062 and 5064.

[0094] In step 5062, the starting point position and direction vector of the light passing through the first light spot are determined based on the pixel coordinates of the first light spot in the eye image and the multiple camera parameters.

[0095] g i =[g xi ,g yi ] is the light source numbered i (denoted as l i ) is calculated by combining the pixel coordinates of the first light spot imaged in the eye image with the plurality of camera parameters (as shown in FIG3 ), and the starting position and direction vector of the 3D light passing through the first light spot can be obtained. The starting position is s i , the direction vector is

[0096] In step 5064, the direction vector from the virtual point to the cornea center is determined based on the starting position and direction vector of the light ray passing through the first light spot and the position of the light source.

[0097] In some embodiments, step 5064 of determining the direction vector from the virtual point to the center of the cornea based on the starting position and direction vector of the light ray passing through the first light spot and the position of the light source may further include: determining the direction vector from the virtual point to the center of the cornea based on the coplanar relationship among the center of the cornea, the starting position and direction vector of the light ray passing through the first light spot and the position of the light source.

[0098] Specifically, according to the law of light reflection, the corneal center c and the starting point s i , direction vector Light source i Coplanar.

[0099] In some embodiments, a first equation for solving the direction vector from the virtual point to the corneal center may be determined first.

[0100] According to the coplanar relationship, we can get:

[0101] in, n i For virtual points To the starting position s i The distance, so the first equation is:

[0102] Then, the first equation may be solved to obtain a direction vector from the virtual point to the corneal center.

[0103] The first equation can be decomposed into a system of equations:

[0104] Finally, it becomes solving Ax=b, where:

[0105] Next, define (0,0,0) as a virtual point

[0106] The problem becomes solving the above linear equations, which can be solved in a variety of ways. As an optional embodiment, the matrix least squares method is used to solve the above linear equations. It can be seen that Ax=b will be converted into the following formula: T A) -1 A T b

[0107] The x obtained by solving is the direction vector from the virtual point to the center of the cornea

[0108] In the embodiment of the present disclosure, based on the particularity of the camera parameters, the process of determining the corneal center is broken down into two parts by constructing a virtual point, thereby simplifying the process of determining the corneal center and improving the accuracy, without being restricted by the camera position and distortion.

[0109] The above-mentioned embodiment of step 506 has described an exemplary method for determining the direction vector from a virtual point to the center of the cornea (the first part of the process of determining the center of the cornea), and the following embodiment of step 508 continues to introduce an exemplary method for determining the distance from the virtual point to the center of the cornea (the second part of the process of determining the center of the cornea).

[0110] In step 508, the distance from the virtual point to the center of the cornea is determined based on the positional relationship between the light source, the second light spot formed by the light source on the cornea, and the center of the cornea.

[0111] In some embodiments, determining the distance from the virtual point to the center of the cornea based on the positional relationship between the light source, the second light spot formed by the light source on the cornea, and the center of the cornea includes:

[0112] The distance from the virtual point to the center of the cornea is determined based on the positional relationship between the light source, the second light spot formed by the light source on the cornea, and the center of the cornea, in combination with the starting position and direction vector of the light passing through the first light spot and the position of the light source.

[0113] In some embodiments, as shown in FIG5D , based on the positional relationship between the light source, the second light spot formed by the light source on the cornea, and the center of the cornea, combined with the starting position and direction vector of the light passing through the first light spot and the position of the light source, the distance from the virtual point to the center of the cornea is determined, which may further include the following steps 5082 and 5084.

[0114] In step 5082, a relationship between the distance from the virtual point to the center of the cornea and the distance from the starting position of the light ray passing through the first light spot to the second light spot is constructed.

[0115] In some embodiments, a second equation for the distance from the starting position to the second light spot can be constructed based on the positional relationship between the light source, the second light spot formed by the light source on the cornea, and the center of the cornea, combined with the starting position and direction vector of the light passing through the first light spot. The second equation is a function of the distance from the virtual point to the center of the cornea.

[0116] Assume that the distance from the virtual point to the center of the cornea (i.e. The length of k c , assuming the starting position s iTo the reflected light spot q i The distance is k qi , combined with FIG5B , the following formula can be listed:

[0117] a.

[0118] b. c.||q i -c||=R.

[0119] According to formulas a, b, and c, we can get the second equation (i.e., k qi It's about k c function):

[0120] Then, a third equation may be constructed based on the positional relationship among the second light spot, the corneal center, and the starting position of the light ray passing through the first light spot:

[0121] ||q i -s i ||<||cs i ||.

[0122] Next, a fourth equation may be constructed based on the positional relationship among the second light spot, the light source, the direction vector of the light passing through the first light spot, and the corneal center:

[0123] The second equation, the third equation and the fourth equation can be considered as the relationship equations described in this embodiment.

[0124] In step 5084, based on the direction vector from the virtual point to the corneal center, combined with the starting position and direction vector of the light ray passing through the first light spot and the position of the light source, the distance from the virtual point to the corneal center that satisfies the relationship is determined.

[0125] In some embodiments, determining the distance from the virtual point satisfying the relationship to the center of the cornea may further include: determining the distance from the virtual point satisfying the relationship to the center of the cornea based on Newton's method or least squares method.

[0126] As an optional embodiment, the distance from the virtual point to the cornea center may be determined based on Newton's method or least squares method according to the second equation, the third equation and the fourth equation.

[0127] In some embodiments, as shown in FIG5E , step 5088 of determining the distance from the virtual point to the center of the cornea according to the second equation, the third equation, and the fourth equation may further include the following steps 50882 and 50884 .

[0128] In step 50882, the second equation is substituted into the third equation and the fourth equation respectively.

[0129] In step 50884, the third equation and the fourth equation are solved using Newton's method or least squares method to obtain the distance k from the virtual point to the center of the cornea. c The value of .

[0130] In step 510, the three-dimensional position information of the cornea center is determined according to the direction vector from the virtual point to the cornea center and the distance from the virtual point to the cornea center.

[0131] According to the formula The coordinates of the corneal center c can be obtained.

[0132] It can be seen from the above embodiments that the corneal center determination method provided by the embodiments of the present disclosure can solve the corneal center based on the camera parameters calibrated by the non-central camera model, without being restricted by the camera position, distortion, etc.

[0133] In some embodiments, specifically, based on the particularity of camera parameters, by constructing a virtual point, the process of determining the corneal center is decomposed into two parts (determining the direction vector from the virtual point to the corneal center and determining the distance from the virtual point to the corneal center), thereby simplifying the corneal center determination process and improving accuracy, and is not limited by camera position and distortion.

[0134] It should be noted that the method of the embodiments of the present disclosure can be performed by a single device, such as a computer or server. The method of the embodiments of the present disclosure can also be applied in a distributed scenario, where multiple devices cooperate to perform the method. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiments of the present disclosure, and the multiple devices will interact with each other to complete the method.

[0135] It should be noted that the above description is limited to some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0136] The disclosed embodiment also provides a method for determining pupil distance.

[0137] FIG5F shows a flowchart of an exemplary method 520 provided in an embodiment of the present disclosure.

[0138] The method 520 may be used to determine the pupil distance. As shown in FIG. 5F , the method 520 may further include the following steps 522 , 524 , 526 , and 528 .

[0139] In step 522 , three-dimensional position information of the center of the cornea may be determined according to any embodiment or permutation or combination of the method 500 .

[0140] At step 524 , two-dimensional position information of the pupil center is determined in the eye image.

[0141] For example, by detecting the pupil center image in the eye image, the two-dimensional position information of the pupil center is obtained.

[0142] In step 526, the three-dimensional position information of the pupil center is determined based on the three-dimensional position information of the cornea center and the two-dimensional position information of the pupil center.

[0143] FIG5G shows a schematic diagram of an exemplary eyeball model for solving the pupil center according to an embodiment of the present disclosure.

[0144] 5G , the eyeball model includes a corneal center c, a corneal radius R, and a pupil center p. In some embodiments, the corneal center c and the radius R are known parameters.

[0145] In some embodiments, as shown in FIG5H , the step of determining the three-dimensional position information of the pupil center further includes steps 5262 , 5264 , and 5266 .

[0146] In step 5262, the position of the refraction point r of the pupil center on the corneal surface is determined.

[0147] Specifically, since the pixel coordinates of the refraction point r in the eye image (ie, the two-dimensional position information of the pupil center) are known, the corresponding projection light can be obtained, with the starting point being o and the direction being v.

[0148] Since the corneal center c and radius R are known, the intersection of the projection light and the corneal surface can be obtained, which is the position of the required refraction point r.

[0149] The intersection point is calculated as follows:

[0150] Assume that x is a point on a sphere with radius r, and the ray Passing through x, we have the equation:

[0151] ||xc|| 2 =r 2 and

[0152] have to:

[0153] d is the value to be sought

[0154] According to the solution of the quadratic equation In this equation:

[0155] c=(oc) 2 -r 2 Then we can judge whether there is an intersection point or the corresponding ray length by judging Δ.

[0156] Since the direction of the ray is known, the intersection point can be found. Since the required point is on the side close to the starting point of the ray, the sign can be determined.

[0157] In step 5264, the direction vector of the refracted light is determined based on the position of the refraction point r of the pupil center on the corneal surface and the refractive index of the cornea.

[0158] Since the position of the refraction point r and the refractive index n1 of the cornea are known, and the distance K from the center of the pupil to the center of the cornea is known (the pupil is located on a sphere with center c and radius K), according to the light refraction formula sin(i1)*N1=sin(i2)*N2, the direction vector of the refracted light can be obtained.

[0159] The following describes how to calculate the direction vector of the refracted light, given the direction vector of the incident light and the refractive index.

[0160] FIG5I is a schematic diagram illustrating an exemplary scenario for solving refracted light according to an embodiment of the present disclosure.

[0161] As shown in Figure 5I, assuming that is the incident unit vector, is the refraction unit vector, O is the refraction point, the medium refractive index is n1 and n2 respectively, and the interface direction vector is (to the right), the normal vector is (downward), the angle of incidence is θ1, and the angle of refraction is θ2.

[0162] The refraction formula n1sin(θ1)=n2sin(θ2) is known. and the normal vector Under the premise of

[0163] Solution The process is as follows:

[0164] Projecting Q and P onto the normal to obtain Q' and P', we can get:

[0165] According to the trigonometric formula:

[0166] therefore:

[0167] Extract n1 / n2 and we get:

[0168] Since QO is known, Given that n1 and n2 are known, we can get Direction vector.

[0169] In step 5266, the three-dimensional position information of the pupil center is determined based on the direction vector of the refracted light.

[0170] Specifically, the direction vector of the refracted light is known To find the coordinates of a point on the refracted ray whose distance from the 3D position of the cornea center is K, the problem becomes: find the intersection of the refracted ray and a sphere with a radius of K and a 3D position of the cornea center as its center. The calculation method is as follows:

[0171] Assume that x is a point on a sphere with radius r, and the ray Passing through x, we have the equation:

[0172] ||xc|| 2 =r 2 and

[0173] have to:

[0174] d is the value to be sought

[0175] According to the solution of the quadratic equation In this equation:

[0176] c=(oc) 2 -r 2 , we can determine whether there is an intersection point or the corresponding ray length by judging Δ.

[0177] Since the direction of the ray is known, the intersection point can be found. Since the required point is on the side close to the starting point of the ray, the sign can be determined.

[0178] The obtained intersection point is the three-dimensional position information of the pupil center.

[0179] In step 528, the pupil distance is determined based on the three-dimensional position information of the pupil center.

[0180] Specifically, the pupil distance can be obtained by using the coordinates of the pupil centers of both eyes calculated using the aforementioned steps, thereby achieving pupil distance estimation.

[0181] It should be noted that the method of the embodiments of the present disclosure can be performed by a single device, such as a computer or server. The method of the embodiments of the present disclosure can also be applied in a distributed scenario, where multiple devices cooperate to perform the method. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiments of the present disclosure, and the multiple devices will interact with each other to complete the method.

[0182] It should be noted that the above description is limited to some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0183] The present disclosure also provides a computer device for implementing the above-mentioned method 500 or method 520. FIG6 shows a schematic diagram of the hardware structure of an exemplary computer device 600 provided in an embodiment of the present disclosure. The computer device 600 can be used to implement the head-mounted wearable device 104 of FIG1A , the wearable device 200 of FIG2A to FIG2C , the external device 112 of FIG1A , and the server 114 of FIG1A . In some scenarios, the computer device 600 can also be used to implement the database server 116 of FIG1A .

[0184] 6 , computer device 600 may include a processor 602, a memory 604, a network module 606, a peripheral interface 608, and a bus 610. The processor 602, the memory 604, the network module 606, and the peripheral interface 608 are connected to each other in communication within the computer device 600 via the bus 610.

[0185] The processor 602 may be a central processing unit (CPU), an image processor, a neural network processor (NPU), a microcontroller (MCU), a programmable logic device (PLD), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or one or more integrated circuits. The processor 602 may be used to perform functions related to the technology described in this disclosure. In some embodiments, the processor 602 may also include multiple processors integrated into a single logical component. For example, as shown in FIG6 , the processor 602 may include multiple processors 602a, 602b, and 602c.

[0186] The memory 604 can be configured to store data (e.g., instructions, computer codes, etc.). As shown in Figure 6, the data stored in the memory 604 can include program instructions (e.g., program instructions for implementing the method 500 or method 520 of the embodiment of the present disclosure) and data to be processed (e.g., the memory can store configuration files of other modules, etc.). The processor 602 can also access the program instructions and data stored in the memory 604, and execute the program instructions to operate on the data to be processed. The memory 604 can include a volatile storage device or a non-volatile storage device. In some embodiments, the memory 604 can include a random access memory (RAM), a read-only memory (ROM), an optical disc, a magnetic disk, a hard disk, a solid-state drive (SSD), a flash memory, a memory stick, etc.

[0187] The network interface 606 can be configured to provide the computer device 600 with communication with other external devices via a network. The network can be any wired or wireless network capable of transmitting and receiving data. For example, the network can be a wired network, a local wireless network (e.g., Bluetooth, WiFi, near field communication (NFC)), a cellular network, the Internet, or a combination thereof. It will be understood that the type of network is not limited to the specific examples above.

[0188] The peripheral interface 608 can be configured to connect the computer device 600 to one or more peripheral devices to enable information input and output. For example, the peripheral devices can include input devices such as a keyboard, a mouse, a touchpad, a touch screen, a microphone, and various sensors, and output devices such as a display, a speaker, a vibrator, and an indicator light.

[0189] The bus 610 may be configured to transmit information between various components of the computer device 600 (e.g., the processor 602, the memory 604, the network interface 606, and the peripheral interface 608), such as an internal bus (e.g., a processor-memory bus), an external bus (USB port, PCI-E bus), etc.

[0190] It should be noted that although the architecture of the computer device 600 shown above only includes the processor 602, memory 604, network interface 606, peripheral interface 608, and bus 610, in a specific implementation, the architecture of the computer device 600 may also include other components necessary for normal operation. In addition, those skilled in the art will understand that the architecture of the computer device 600 may only include the components necessary to implement the embodiments of the present disclosure, and does not necessarily include all the components shown in the figure.

[0191] The present disclosure also provides a device for determining the corneal center. FIG7 shows a schematic diagram of an exemplary device 700 provided by the present disclosure. As shown in FIG7 , the device 700 can be used to implement the method 500 and can further include the following modules.

[0192] The first acquisition module 702 is configured to: acquire multiple camera parameters of the camera, wherein the camera parameters include a starting position and a spatial straight line equation passing through the starting position; the second acquisition module 704 is configured to: acquire an eye image captured by the camera, wherein the eye image includes a first light spot formed by a light source, and the light source is arranged around the camera; the first determination module 706 is configured to: determine the direction vector from the virtual point to the corneal center based on the pixel coordinates of the first light spot in the eye image and the multiple camera parameters; the second determination module 708 is configured to: determine the distance from the virtual point to the corneal center based on the positional relationship between the light source, the second light spot formed by the light source on the cornea, and the corneal center; the third determination module 710 is configured to: determine the three-dimensional position information of the corneal center based on the direction vector from the virtual point to the corneal center and the distance from the virtual point to the corneal center.

[0193] For the convenience of description, the above devices are described as being functionally divided into various modules. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0194] The apparatus of the above embodiment is used to implement the corresponding method 500 in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0195] The present disclosure also provides an apparatus for determining pupillary distance. FIG8 shows a schematic diagram of an exemplary apparatus 800 provided by the present disclosure. As shown in FIG8 , the apparatus 800 can be used to implement method 520 and can further include the following modules.

[0196] The first determination module 802 is configured to determine the three-dimensional position information of the cornea center according to the method 500 described. The second determination module 804 is configured to determine the two-dimensional position information of the pupil center in the eye image. The third determination module 806 is configured to determine the three-dimensional position information of the pupil center according to the three-dimensional position information of the cornea center and the two-dimensional position information of the pupil center. The fourth determination module 808 is configured to determine the pupil distance according to the three-dimensional position information of the pupil center.

[0197] For the convenience of description, the above devices are described as being functionally divided into various modules. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0198] The apparatus of the above embodiment is used to implement the corresponding method 520 in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0199] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute method 500 or method 520 as described in any of the above embodiments.

[0200] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0201] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute method 500 or method 520 as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0202] Based on the same inventive concept, corresponding to method 500 or method 520 in any of the above embodiments, the present disclosure further provides a computer program product comprising computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processor to perform method 500 or method 520. For the execution entities corresponding to the steps in each embodiment of method 500 or method 520, the processors that execute the corresponding steps can belong to the corresponding execution entities.

[0203] The computer program product of the above embodiment is used to enable the computer and / or the processor to execute method 500 or method 520 as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0204] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Within the scope of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0205] In addition, to simplify the description and discussion, and so as not to obscure the embodiments of the present disclosure, known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, devices may be shown in the form of block diagrams to avoid obscuring the embodiments of the present disclosure, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure are to be implemented (i.e., these details should be fully within the purview of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it will be apparent to those skilled in the art that the embodiments of the present disclosure may be implemented without these specific details or with variations in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0206] Although the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0207] The embodiments of the present disclosure are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method for determining a corneal center, comprising: Acquire a plurality of camera parameters of the camera, wherein the camera parameters include a starting point position and a spatial straight line equation passing through the starting point position; Acquire an eye image captured by the camera, wherein the eye image includes a first light spot formed by a light source, and the light source is arranged around the camera; Determine a direction vector from a virtual point to a corneal center according to the pixel coordinates of the first light spot in the eye image and the plurality of camera parameters; Determining the distance from the virtual point to the center of the cornea according to the positional relationship between the light source, the second light spot formed by the light source on the cornea, and the center of the cornea; The three-dimensional position information of the cornea center is determined according to the direction vector from the virtual point to the cornea center and the distance from the virtual point to the cornea center.

2. The method of claim 1, wherein: Determining a direction vector from a virtual point to a corneal center according to the pixel coordinates of the first light spot in the eye image and the plurality of camera parameters includes: Determining a starting point position and a direction vector of a light ray passing through the first light spot according to the pixel coordinates of the first light spot in the eye image and the plurality of camera parameters; The direction vector from the virtual point to the cornea center is determined according to the starting point position and the direction vector of the light passing through the first light spot and the position of the light source.

3. The method of claim 2, wherein: Determining the direction vector from the virtual point to the center of the cornea according to the starting point position and the direction vector of the light passing through the first light spot and the position of the light source includes: The direction vector from the virtual point to the cornea center is determined according to the coplanar relationship among the cornea center, the starting point position and the direction vector of the light passing through the first light spot, and the position of the light source.

4. The method of claim 2, wherein: Determining the distance from the virtual point to the center of the cornea according to the positional relationship between the light source, the second light spot formed by the light source on the cornea, and the center of the cornea includes: The distance from the virtual point to the cornea center is determined based on the positional relationship between the light source, the second light spot formed by the light source on the cornea, and the cornea center, combined with the starting point position and direction vector of the light passing through the first light spot and the position of the light source.

5. The method of claim 4, wherein: Determining the distance from the virtual point to the center of the cornea according to the positional relationship between the light source, the second light spot formed by the light source on the cornea, and the center of the cornea, in combination with the starting point position and the direction vector of the light passing through the first light spot and the position of the light source, comprises: Constructing a relationship between the distance from the virtual point to the center of the cornea and the distance from the starting point of the light passing through the first light spot to the second light spot; According to the direction vector from the virtual point to the cornea center, combined with the starting position and direction vector of the light passing through the first light spot and the position of the light source, the distance from the virtual point to the cornea center that satisfies the relationship is determined.

6. The method of claim 5, wherein: Determining the distance from the virtual point to the cornea center that satisfies the relationship comprises: Based on Newton's method or least squares method, the distance from the virtual point to the cornea center that satisfies the relationship is determined.

7. A method for determining pupil distance, comprising: Determining the three-dimensional position information of the center of the cornea according to the method described in any one of claims 1 to 6; Determining two-dimensional position information of the center of the pupil in the eye image; Determining the three-dimensional position information of the pupil center according to the three-dimensional position information of the cornea center and the two-dimensional position information of the pupil center; The pupil distance is determined according to the three-dimensional position information of the pupil center.

8. A device for determining the center of a cornea, comprising: A first acquisition module is configured to: acquire a plurality of camera parameters of a camera, wherein the camera parameters include a starting point position and a spatial straight line equation passing through the starting point position; A second acquisition module is configured to: acquire an eye image captured by the camera, wherein the eye image includes a first light spot formed by a light source, and the light source is arranged around the camera; A first determination module is configured to: determine a direction vector from a virtual point to a corneal center according to the pixel coordinates of the first light spot in the eye image and the plurality of camera parameters; A second determination module is configured to: determine the distance from the virtual point to the center of the cornea according to the positional relationship between the light source, the second light spot formed by the light source on the cornea, and the center of the cornea; The third determination module is configured to determine the three-dimensional position information of the cornea center according to the direction vector from the virtual point to the cornea center and the distance from the virtual point to the cornea center.

9. A device for determining pupil distance, comprising: A first determination module is configured to: determine the three-dimensional position information of the center of the cornea according to the method according to any one of claims 1 to 6; A second determination module is configured to: determine two-dimensional position information of a pupil center in the eye image; A third determination module is configured to: determine the three-dimensional position information of the pupil center according to the three-dimensional position information of the cornea center and the two-dimensional position information of the pupil center; The fourth determination module is configured to determine the pupil distance according to the three-dimensional position information of the pupil center.

10. A computer device comprising one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the programs include instructions for executing the method according to any one of claims 1 to 6 or the method according to claim 7.

11. A non-volatile computer-readable storage medium containing a computer program, which, when executed by one or more processors, causes the processors to perform the method according to any one of claims 1 to 6 or the method according to claim 7.

12. A computer program product, comprising computer program instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 6 or the method according to claim 7.

Images