Screen-off unlocking method and device

Abstract

The application discloses a screen-off unlocking method and device, which is used for completing unlocking in a screen-off state and is more favorable for privacy protection. The method is applied to a device with a screen and supporting an eye tracking function, and comprises the following steps: continuously detecting whether a wake-up condition of the eye tracking function is met in the screen-off state; starting the eye tracking function when the wake-up condition is met; tracking an eye movement trajectory by using the eye tracking function in the screen-off state; the tracked eye movement trajectory is a target eye movement trajectory; and performing an unlocking operation on the screen when the target eye movement trajectory is consistent with a preset eye movement trajectory. The whole unlocking process of the embodiment of the application is completed in the screen-off state: continuously detecting whether the wake-up condition of the eye tracking function is met in the screen-off state, and starting the function when the wake-up condition is met. After the function is started, the eye tracking is still performed in the screen-off state to obtain the eye movement trajectory, and the unlocking operation is performed when the obtained eye movement trajectory is consistent with the preset eye movement trajectory.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of eye tracking, in particular to a screen-off unlocking method and device. BACKGROUND

[0002] Currently, the technology of eye tracking based on a camera is very mature and can be applied to screen unlocking of an electronic device.

[0003] The existing method of unlocking the screen of an electronic device by using the eye tracking function is that a user manually turns on the screen and then gazes at fixed positions on the screen in a predetermined order to unlock the screen.

[0004] The above method needs to manually turn on the screen in advance and then gaze at the visible area on the screen to unlock the screen. The screen is bright during the unlocking process, which may leak the unlocking information and is not conducive to privacy protection. SUMMARY

[0005] To solve the above problems, the present application provides a screen-off unlocking method and device to complete the unlocking in a screen-off state and is more conducive to privacy protection.

[0006] To achieve the above purpose, the present application provides the following technical solutions.

[0007] A screen-off unlocking method applied to a device with a screen and supporting an eye tracking function, comprising the following steps:

[0008] In a screen-off state, continuously detecting whether the wake-up condition of the eye tracking function is met;

[0009] When the wake-up condition is met, the eye tracking function is started;

[0010] In the screen-off state, the eye tracking function is used to track an eye movement trajectory. The tracked eye movement trajectory is a target eye movement trajectory;

[0011] If the target eye movement trajectory is consistent with a preset eye movement trajectory, an unlocking operation is performed on the screen.

[0012] Optionally, the target eye movement trajectory comprises a positioning point that is successively gazed at among N preset positioning points.

[0013] Optionally, the N positioning points comprise light points in a light point array displayed in the screen-off state, or the N positioning points comprise the vertex of the screen and the midpoint of each edge line.

[0014] Optionally, the device comprises an infrared light source and an infrared camera; the preset eye movement trajectory is recorded in an unlocking information recording process; if the N positioning points comprise vertices of the screen and midpoints of each side line, the unlocking information recording process comprises: position calibration of the infrared camera; performing a calibration process to obtain personal calibration coefficients of the user; prompting the user to gaze at a preset position; after recognizing that the user gazes at the preset position, collecting eye images of the user based on the infrared light source and the infrared camera; identifying the gaze axis and gaze position of the eye using the eye images and the personal calibration coefficients; recording the corresponding positioning point of the gaze position in the N positioning points; after receiving a stop command, terminating the unlocking information recording process.

[0015] Optionally, the device comprises an infrared light source and an infrared camera; the preset eye movement trajectory is recorded in an unlocking information recording process; if the N positioning points comprise light points in a light point array, the unlocking information recording process comprises: position calibration of the infrared camera; performing a calibration process to obtain personal calibration coefficients of the user; prompting the user to gaze at a preset position; after recognizing that the user gazes at the preset position, displaying the light point array on the screen; collecting eye images of the user based on the infrared light source and the infrared camera; identifying the gaze axis and gaze position of the eye using the eye images and the personal calibration coefficients; recording the corresponding light point of the gaze position in the light point array; after receiving a stop command, terminating the unlocking information recording process.

[0016] Optionally, before tracking the eye movement trajectory using the eye tracking function, the method further comprises: obtaining IPD pupillary distance information; matching the obtained IPD pupillary distance information with preset IPD pupillary distance information; if the matching fails, determining that the unlocking fails; the step of tracking the eye movement trajectory using the eye tracking function is performed after the matching succeeds.

[0017] Optionally, the preset IPD information is recorded in an unlocking information recording process.

[0018] Optionally, the wake-up condition comprises at least one of: detecting that there is a moving object in front of the screen, and detecting that the distance between the object and the screen satisfies a preset condition.

[0019] Optionally, the device comprises an infrared light source and an infrared camera; the tracking of the eye movement trajectory using the eye tracking function comprises: collecting eye images of the user based on the infrared light source and the infrared camera; identifying the gaze axis and gaze position of the eye using the eye images; determining the corresponding positioning point of the gaze position in the N positioning points as the gazed positioning point.

[0020] An off-screen unlocking device applied to a device with a screen and supporting eye tracking function, comprising:

[0021] A detection unit for:

[0022] In the off-screen state, continuously detecting whether the wake-up condition of the eye tracking function is met;

[0023] When the wake-up condition is met, the eye tracking function is started;

[0024] An unlocking unit for:

[0025] In the off-screen state, the eye tracking function is used to track the eye movement trajectory, and the tracked eye movement trajectory is a target eye movement trajectory;

[0026] If the target eye movement trajectory is consistent with the preset eye movement trajectory, an unlocking operation is performed on the screen.

[0027] An off-screen unlocking device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the off-screen unlocking method according to any one of the above embodiments when executing the program.

[0028] A storage medium, wherein the storage medium stores computer executable instructions, and the computer executable instructions are loaded and executed by a processor to implement the steps of the off-screen unlocking method according to any one of the above embodiments.

[0029] Compared with the prior art, the embodiment of the present application continuously detects whether the wake-up condition of the eye tracking function is met in the off-screen state, and starts the function when the condition is met. After starting, the eye tracking is still performed in the off-screen state to obtain the eye movement trajectory, and the unlocking operation is performed when the obtained eye movement trajectory is consistent with the preset eye movement trajectory. The entire unlocking process of the embodiment of the present application is completed in the off-screen state, which is not easy to leak the unlocking information, thereby being more secure and confidential, and being more conducive to privacy protection. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0031] Figure 1 An exemplary flow of the off-screen unlocking method provided by the embodiment of the present application;

[0032] Figure 2 An exemplary arrangement of the positioning points provided by the embodiment of the present application;

[0033] Figure 3 Another exemplary arrangement of the positioning points provided for the embodiments of the present application;

[0034] Figure 4a Another exemplary flow of the screen-off unlocking method provided for the embodiments of the present application;

[0035] Figure 4b Another exemplary flow of the screen-off unlocking method provided for the embodiments of the present application;

[0036] Figure 5a Another exemplary distribution of the positioning points provided for the embodiments of the present application;

[0037] Figure 5b Another exemplary distribution of the positioning points provided for the embodiments of the present application;

[0038] Figure 6 An exemplary flow of the eye movement trajectory correction and input unlocking using the positioning points provided for the embodiments of the present application; Figure 5a An exemplary flow of the eye movement trajectory correction and input unlocking using the positioning points provided for the embodiments of the present application;

[0039] Figure 7 An exemplary flow of the eye movement trajectory correction and input unlocking using the positioning points provided for the embodiments of the present application; Figure 5b An exemplary flow of the eye movement trajectory correction and input unlocking using the positioning points provided for the embodiments of the present application;

[0040] Figure 8 An exemplary structure of the screen-off unlocking device provided for the embodiments of the present application;

[0041] Figure 9 An exemplary structure of the screen-off unlocking device provided for the embodiments of the present application. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0043] The terms “first” and “second” and the like in the specification and claims of the present application and the above drawings are used to distinguish different objects, but are not used to describe a specific order. In addition, the terms “include” and “have” and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not set to the listed steps or units, but can include steps or units not listed.

[0044] The screen-off unlocking method and the screen-off unlocking device provided in the embodiments of the present application are applied to a device (referred to as a terminal device) having a screen and supporting an eye tracking function, which exemplarily includes but is not limited to a smart phone, a tablet, a computer, and the like.

[0045] Eye tracking can also be referred to as gaze tracking, which is a technology for estimating a line of sight and / or a gaze point of an eye by measuring an eye movement.

[0046] The terminal device can use one or more eye tracking algorithms for eye tracking. The eye tracking algorithm is used to calculate gaze information. Specifically, the eye tracking algorithm can calculate a line of sight direction and an eye position, and obtain a gaze point coordinate on the screen according to the line of sight direction and the eye position.

[0047] The hardware devices used in cooperation with the eye tracking algorithm exemplarily include a camera, a light source, and the like, which are also deployed on the terminal device and serve the eye tracking algorithm.

[0048] The camera includes a camera for shooting an eye image, such as an environmental camera, an infrared camera (940 nm), and can also include a depth camera.

[0049] A set of eye tracking algorithms and the corresponding hardware devices can also be regarded as an eye tracking system.

[0050] The embodiments of the present application will be further described in detail below based on the common aspects related to the present application in the above description.

[0051] Embodiment One

[0052] In the embodiment one of the present application, a screen-off unlocking method is provided to complete unlocking in a screen-off state. Please refer to Figure 1 The screen-off unlocking method includes:

[0053] S1: In the screen-off state, continuously detecting whether the wake-up condition of the eye tracking function is met;

[0054] Step S1 is executed by the screen-off unlocking device. When the electronic device such as a smart phone is in a screen-off (commonly referred to as black screen) state, the eye tracking function is not started in real time, but continuously detects whether the wake-up condition of the eye tracking function is met.

[0055] S2: When the wake-up condition is met, the eye tracking function is started;

[0056] In one example, the wake-up condition can include at least one of detecting that there is a moving object in front of the screen, and detecting that the distance between the object and the screen meets a preset condition.

[0057] S3: In the screen-off state, the eye tracking function is used to track an eye movement trajectory (referred to as a target eye movement trajectory);

[0058] In one example, the target eye movement trajectory can include: in the preset N positioning points, the positioning points are successively fixated.

[0059] Wherein, N is a positive integer not less than 2. Those skilled in the art can flexibly design the value of N according to the needs, for example, design its value to be 9. In addition, the specific positions of the N positioning points, the shape of the arrangement (for example Figure 2 rectangular in the figure, Figure 3 circular in the figure) and the like can be flexibly designed.

[0060] S4: If the target eye movement trajectory is consistent with the preset eye movement trajectory, perform an unlocking operation on the screen.

[0061] The user can pre-input the eye movement trajectory (motion trajectory) used for unlocking as the preset eye movement trajectory.

[0062] Compared with the prior art, the embodiment of the present application continuously detects whether the wake-up condition of the eye tracking function is met in the screen-off state, and when the condition is met, the function is turned on. After being turned on, the eye tracking is still performed in the screen-off state to obtain an eye movement trajectory, and when the obtained eye movement trajectory is consistent with the preset eye movement trajectory, an unlocking operation is performed. The entire unlocking process of the embodiment of the present application is completed in the screen-off state, and the unlocking information is not easy to be leaked, so that the security and privacy are higher, and the privacy protection is more beneficial.

[0063] Embodiment two

[0064] This embodiment focuses on how to use the eye tracking function to track an eye movement trajectory, please refer to Figure 4a The above screen-off unlocking method exemplarily includes:

[0065] S41: In the screen-off state, continuously detect whether the wake-up condition of the eye tracking function is met.

[0066] S42: When the wake-up condition is met, turn on the eye tracking function;

[0067] Steps S41-S42 are the same as the foregoing S1-S2, and will not be repeated here.

[0068] S43: Capture the eye image of the user based on the infrared light source and the infrared camera;

[0069] The eye image here refers to an image containing the eyes, such as a front, side, or only eye image.

[0070] It should be noted that for some eye movement tracking algorithms based on pupil-corneal reflection method (PCCR), the eye map is specifically an eye feature image containing a light spot. When collecting: the infrared light source is directed to the eye, forming a reflection point, i.e. a light spot (also known as Purkinje spot), on the cornea. An infrared camera captures an eye image with a light spot.

[0071] S44: extracting eye feature information from the eye image;

[0072] Specifically, the eye feature information can be obtained by processing the eye image.

[0073] With different eye movement tracking algorithms, the eye feature information contains different feature information.

[0074] For example, according to the eye movement tracking algorithm based on the shape of the pupil position, the gaze information (gaze axis and gaze position) is calculated by the principal axis direction of the pupil and the pupil position. Therefore, the eye feature information can include the principal axis direction of the pupil, the pupil position, the major axis length of the pupil, the minor axis length of the pupil, etc.

[0075] For another example, a certain feature vector fitting method works as follows:

[0076] Extract the center position of the pupil, the center position of the left corner of the eye, and the center position of the right corner of the eye;

[0077] Subtract the center position of the left corner of the eye from the center position of the pupil as feature vector A;

[0078] Subtract the center position of the right corner of the eye from the center position of the pupil as feature vector B.

[0079] Construct a mapping function (several equation groups) between vectors A, B and the gaze point.

[0080] Based on the given feature vectors A and B, and the known gaze information, polynomial fitting is performed to obtain the unknown coefficients in the mapping function (the solution of the unknown coefficients can be completed in the calibration process). After obtaining the unknown coefficients, the current extracted eye feature information is input into the mapping function, and the current gaze point can be obtained (this process is the tracking process).

[0081] The feature vector fitting method requires the eye feature information to include the center position of the pupil, the center position of the left corner of the eye, and the center position of the right corner of the eye.

[0082] Since there are many types of eye movement tracking algorithms, they will not be described one by one here.

[0083] S45: identifying the gaze axis and gaze position of the eye according to the eye feature information;

[0084] In one example, the gaze axis and gaze position of the eye can be identified by using a pupil-corneal reflection method (PCCR) based eye tracking algorithm, which works as follows: an infrared light source is directed to the eye, and a reflection point, i.e. a glint (also known as Purkinje image), is formed on the cornea. An infrared camera captures eye images with the glint.

[0085] As the eyeball rotates, the relative position relationship between the center of the pupil and the glint changes accordingly, and the eye images with the glint captured by the infrared camera also reflect such position change relationship. The line of sight / gaze point can be estimated according to the position change relationship.

[0086] In the existing unlocking method based on the gaze point of the human eye, the eye movement recognition accuracy based on the glint feature is low, and the unlocking success rate is low, which is not suitable for the complex use scenarios of portable mobile devices. In contrast, the eye movement recognition based on the glint has higher recognition accuracy and efficiency.

[0087] S46: Determine the positioning point corresponding to the gaze position in the N positioning points as the gazed positioning point.

[0088] Taking an example of 9 positioning points (positioning points 1-9), assuming that the currently calculated gaze position corresponds to the positioning point 1, the positioning point 1 is taken as the gazed positioning point.

[0089] The sequentially gazed positioning points constitute the current eye movement trajectory of the user, i.e. the target eye movement trajectory.

[0090] S47: If the target eye movement trajectory is consistent with the preset eye movement trajectory, perform the unlocking operation on the screen.

[0091] Step S47 is the same as the aforementioned S4, and will not be described here.

[0092] Of course, if they are not consistent, it can be determined that the unlocking fails.

[0093] In this embodiment, the infrared camera based on the infrared light source captures the eye image, and calculates the gaze point (gaze axis and gaze position) according to the eye image, and determines the corresponding positioning point based on the identified gaze position. In this process, it is not necessary to turn on the screen.

[0094] Embodiment Three

[0095] In order to further protect the privacy of the user, please refer to Figure 4b Before the eye movement trajectory is tracked by using the eye tracking function (for example, step S45), the following steps can also be included:

[0096] S51: Obtain the IPD (interpupillary distance) information;

[0097] The pupil distance information can be obtained according to the eye image, and the pupil distance information can be part of the eye feature information.

[0098] The pupil distance information refers to the distance between the centers of the pupils of the two eyes when a person gazes at a far place.

[0099] S52: match the obtained IPD pupil distance information with preset IPD pupil distance information; if the matching fails, go to S53, otherwise, go to S45;

[0100] The preset IPD information can be entered in the unlocking information entry process.

[0101] The unlocking information entry process will be introduced later in this paper.

[0102] S53: determine that the unlocking fails;

[0103] The step of tracking the eye movement trajectory using the eye tracking function (for example, step S3 or steps S45-S46) is performed after the matching succeeds.

[0104] It should be noted that the pupil distance information is almost unique for each person, but due to measurement accuracy, it can only be used as an auxiliary item. Generally, the pupil distance information can exclude most non-personal unlocking information, so in this embodiment, IPD detection is performed, and after the detection passes, it can be determined that the user is the person himself, and then the eye movement tracking is performed. In this way, the false positive rate can be reduced, the user's privacy can be better protected, and the unlocking operation efficiency can be improved and energy consumption can be saved.

[0105] Embodiment Four

[0106] In Embodiment One, the target eye movement trajectory includes: the positioning points that are successively gazed at among the preset N positioning points.

[0107] Optionally, please refer to Figure 5a The above N positioning points include: the vertices of the screen and the midpoints of each edge line of the screen;

[0108] Or, please refer to Figure 5b The above N positioning points can include: light points in the light point array displayed in the screen-off state.

[0109] In addition, before performing the screen-off unlocking, it is also necessary to perform the correction in advance, and the eye movement trajectory used for unlocking is entered in the unlocking information entry process.

[0110] The specific methods of correction and pre-input of the eye movement trajectory used for unlocking will be introduced in detail according to the embodiments.

[0111] Embodiment Five

[0112] Figure 6 shows the method based on Figure 5aAn exemplary procedure for correction and inputting the eye movement trajectory using the indicated positioning points, comprising:

[0113] S61: Position calibration of the infrared camera;

[0114] Specifically, the camera can be position calibrated using a calibration board, which is a prior art and will not be described here.

[0115] S62: Perform calibration procedure to obtain personal calibration coefficient of the user;

[0116] The personal calibration coefficient is a parameter used in the line-of-sight estimation algorithm to calculate the final result of the line-of-sight, and is related to data such as the pupil radius, corneal curvature, and angle difference between the visual axis and optical axis of the user.

[0117] The calculation principle of the personal calibration coefficient is as follows:

[0118] Step a: Display several feature points on the screen;

[0119] Step b: Acquire eye images during the user's gaze at the feature points, and extract eye feature information therefrom;

[0120] The eye feature information is described above and will not be described here.

[0121] Step c: Use the line-of-sight estimation algorithm to inversely solve the personal calibration coefficient based on the acquired eye feature information and calibration point coordinates.

[0122] S63: Prompt the user to gaze at a preset position;

[0123] In one example, the preset position can be the center of the camera, and the center point of the camera can be used as the origin point in the subsequent steps.

[0124] Of course, other positions can also be flexibly selected by those skilled in the art as the preset position, which will not be described here.

[0125] S64: After recognizing that the user gazes at the preset position, acquire the eye images of the user based on the infrared light source and the infrared camera;

[0126] S64 is similar to S43 described above, and will not be described here.

[0127] S65: Identify the gaze axis and gaze position of the eye using the eye images and the personal calibration coefficient;

[0128] Taking the eigenvector fitting method as an example, the personal calibration coefficient can be used to solve the unknown coefficients in the mapping function, and the eye feature information extracted from the eye images can be input into the mapping function to obtain the current gaze point. The detailed process is described above in step S44.

[0129] The aforementioned preset IPD information can be entered in the unlocking information entry process.

[0130] After the eye feature information is extracted, the IPD information in the eye feature information can be entered as the preset IPD information.

[0131] S66: record the positioning point corresponding to the gaze position in the N positioning points;

[0132] S66 is similar to S46 described above, and thus is not described again.

[0133] S67: terminate the unlocking information entry process after receiving the stop command.

[0134] In one example, the stop command can be issued by the user touching the stop button.

[0135] In this embodiment, the positioning points are the vertices of the screen and the midpoints of each side. Before entry, the position of the infrared camera is calibrated, and a calibration process is performed to make subsequent operations more accurate. Then, the eye image of the user is collected, the gaze position is identified, and the positioning point corresponding to the gaze position is recorded. The plurality of recorded positioning points constitute the preset eye movement trajectory. When the stop command is received, the process is terminated, and the final eye movement trajectory for unlocking is obtained. The screen does not need to display any picture during the entire unlocking process.

[0136] Embodiment Six

[0137] Figure 7 An exemplary process for correcting and inputting the eye movement trajectory for unlocking based on the positioning points shown in FIG. 6 is shown, including: Figure 5b

[0138] S71: calibrate the position of the infrared camera;

[0139] S72: perform a calibration process to obtain the personal calibration coefficient of the user;

[0140] S73: prompt the user to gaze at the preset position;

[0141] S71-S73 are the same as S61-S63 described above, and thus are not described again.

[0142] S74: display the light point array on the screen after identifying that the user gazes at the preset position;

[0143] The light point array can be seen in FIG. 7, which is exemplary. Figure 5b

[0144] S75: collect the eye image of the user based on the infrared light source and the infrared camera;

[0145] ​​S75 is similar to S43, and is not described herein again.

[0146] S76: identifying the gaze axis and the gaze position of the eye using the eye image and the personal calibration coefficient;

[0147] S76 is similar to S65, and is not described herein again.

[0148] S77: recording the light spot corresponding to the gaze position in the light spot array;

[0149] Taking the light spot array containing 9 light spots (light spots 1-9) as an example, assuming that the currently calculated gaze position corresponds to light spot 1, then record light spot 1 (as the gaze point).

[0150] The light spots successively gazed at constitute the preset eye movement trajectory.

[0151] The foregoing mentioned that the preset IPD information can be entered in the unlocking information entry process.

[0152] Then, after the eye feature information is extracted, the IPD information in the eye feature information can be entered as the preset IPD information.

[0153] S78: terminating the unlocking information entry process after receiving the stop command.

[0154] It can be seen that in the embodiment, the gaze point is a light spot in the light spot array displayed in the screen-off state. Before entry, the position of the infrared camera is calibrated, and the calibration process is performed to make the subsequent operation more accurate. Then, the light spot array is displayed to facilitate the user to gaze. The eye image of the user is collected, the gaze position is identified, the light spot corresponding to the gaze position is recorded, and the unlocking information entry is realized.

[0155] Embodiment seven

[0156] The embodiment provides an off-screen unlocking device, which is applied to a terminal device with a screen and supporting an eye movement tracking function.

[0157] Please refer to Figure 8 The above device comprises:

[0158] A detection unit 1 is configured to continuously detect whether the wake-up condition of the eye movement tracking function is met in the screen-off state.

[0159] The wake-up condition comprises at least one of the following: detecting that there is a moving object in front of the screen, and detecting that the distance between the object and the screen meets a preset condition.

[0160] An unlocking unit 2 is configured to:

[0161] In the screen-off state, the eye tracking function is used to track the eye movement trajectory; the tracked eye movement trajectory is a target eye movement trajectory;

[0162] If the target eye movement trajectory is consistent with the preset eye movement trajectory, an unlocking operation is performed on the screen.

[0163] The eye tracking device involved in the embodiments of the present application is deployed in a terminal device, and whether the wake-up condition of the eye tracking function is met can be continuously detected in the screen-off state. When the condition is met, the function is enabled. After being enabled, the eye tracking is still performed in the screen-off state to obtain an eye movement trajectory. When the obtained eye movement trajectory is consistent with the preset eye movement trajectory, an unlocking operation is performed. The entire unlocking process of the embodiments of the present application is completed in the screen-off state, and the unlocking information is not easy to be leaked, so that the security and privacy are higher, and the privacy protection is more favorable.

[0164] For specific content, please refer to the foregoing description, which is not repeated here.

[0165] In one example, before the eye tracking function is used to track the eye movement trajectory, the unlocking unit 2 can also be used to:

[0166] Obtain IPD pupillary distance information;

[0167] Match the obtained IPD pupillary distance information with preset IPD pupillary distance information;

[0168] If the matching fails, it is determined that the unlocking fails;

[0169] The operation of using the eye tracking function to track the eye movement trajectory is performed after the matching succeeds.

[0170] The preset IPD information is entered in the unlocking information entry process. For specific content, please refer to the foregoing description, which is not repeated here.

[0171] In one example, the target eye movement trajectory includes: a positioning point that is successively fixated in the preset N positioning points.

[0172] The skilled in the art can flexibly set the setting of the positioning points, for example, please refer to Figure 5a The N positioning points can include: light points in the light point array displayed in the screen-off state; for another example, please refer to Figure 5b The N positioning points can include: the vertices of the screen and the midpoints of each edge. For specific content, please refer to the foregoing description, which is not repeated here.

[0173] The terminal device includes: an infrared light source and an infrared camera. The preset eye movement trajectory is entered in the unlocking information entry process.

[0174] Therefore, please refer to Figure 9The screen-off unlocking device can further comprise a unlocking information input unit 3 configured to input unlocking information.

[0175] In one example, if the N locating points comprise vertices of the screen and midpoints of each edge line, the unlocking information input procedure performed by the unlocking information input unit 3 can exemplarily comprise:

[0176] Position calibration of the infrared camera;

[0177] Performing a calibration procedure to obtain a personal calibration coefficient of the user;

[0178] Prompting the user to gaze at a preset position;

[0179] After recognizing that the user gazes at the preset position, capturing an eye image of the user based on the infrared light source and the infrared camera;

[0180] Identifying a gaze axis and a gaze position of the eye using the eye image and the personal calibration coefficient;

[0181] Recording a locating point corresponding to the gaze position in the N locating points;

[0182] Terminating the unlocking information input procedure after receiving a stop command.

[0183] For details, please refer to the foregoing description, which will not be repeated here.

[0184] If the N locating points comprise light points in the light point array, the unlocking information input procedure performed by the unlocking information input unit 3 can exemplarily comprise:

[0185] Position calibration of the infrared camera;

[0186] Performing a calibration procedure to obtain a personal calibration coefficient of the user;

[0187] Prompting the user to gaze at a preset position;

[0188] After recognizing that the user gazes at the preset position, displaying the light point array on the screen;

[0189] Capturing an eye image of the user based on the infrared light source and the infrared camera;

[0190] Identifying a gaze axis and a gaze position of the eye using the eye image and the personal calibration coefficient;

[0191] Recording a light point corresponding to the gaze position in the light point array;

[0192] Terminating the unlocking information input procedure after receiving a stop command.

[0193] For details, please refer to the foregoing description, which will not be repeated here.

[0194] In yet another example, in terms of tracking the eye movement trajectory using the eye tracking function, the unlocking unit 2 can be specifically used for:

[0195] acquiring the eye image of the user based on the infrared light source and the infrared camera;

[0196] identifying the gaze axis and the gaze position of the eye using the eye image;

[0197] determining the positioning point corresponding to the gaze position among the N positioning points as the positioned positioning point.

[0198] For specific content, please refer to the foregoing description, which will not be repeated here.

[0199] It is mentioned above that the preset IPD information can be entered in the unlocking information entry process.

[0200] Then the unlocking information entry unit 3 can also be used to enter the IPD information in the eye feature information as the preset IPD information.

[0201] Embodiment eight

[0202] The embodiment of the application provides a storage medium, the storage medium stores computer executable instructions, and the computer executable instructions are loaded and executed by a processor to realize the steps of the screen-out unlocking method in any one of the embodiments one to six.

[0203] Embodiment nine

[0204] The embodiment of the application provides a processor, which is used for running a program, wherein the program is executed to perform the screen-out unlocking method steps in any one of the embodiments one to six.

[0205] Embodiment ten

[0206] The embodiment of the application provides a terminal device, which comprises a processor, a memory, and a program stored in the memory and executable on the processor, and the processor performs the screen-out unlocking method steps in any one of the embodiments one to six when executing the program.

[0207] Those skilled in the art should understand that the embodiments of the application can be provided as a method, a system, or a computer program product. Therefore, the application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0208] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure One one or more flow or blocks Figure One means for functionally implementing the steps listed in the flowchart block or blocks.

[0209] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure One one or more flow or blocks Figure One means for functionally implementing the steps listed in the flowchart block or blocks.

[0210] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure One one or more flow or blocks Figure One Figure One means for functionally implementing the steps listed in the flowchart block or blocks.

[0211] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0212] The memory can include non-persistent memory and / or volatile memory, such as a random access memory (RAM) including a cache area for the temporary storage of data. The memory can also include non-volatile memory, such as read only memory (ROM) for storing structural information and / or instruction code. Both volatile and non-volatile memory can be implemented as a flash memory, a magnetic memory, an optical memory, and / or any non-transitory computer readable storage medium.

[0213] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0214] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusions, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0215] Those skilled in the art will appreciate that embodiments of the present application can be provided as a method, system or computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code.

[0216] The above merely provides embodiments of the present application and is not intended to limit the present application. Various modifications and changes can be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the scope of the claims of the present application.

Claims

1. A screen-off unlock method, characterized by, The application is applied to a device with a screen and an eye tracking function, and comprises the following steps: In an off-screen state, continuously detecting whether the wake-up condition of the eye tracking function is met, the wake-up condition comprising: detecting that there is a moving object in front of the screen; When the wake-up condition is met, the eye tracking function is started; IPD information is obtained, and the obtained IPD information is matched with preset IPD information; if the matching fails, it is determined that the unlocking fails; In the off-screen state, if the matching succeeds, the eye tracking function is used to track the eye movement trajectory; the tracked eye movement trajectory is a target eye movement trajectory, and the target eye movement trajectory comprises: in a preset N positioning point, the positioning point successively gazed at, and the N positioning point comprises: the vertex of the screen and the midpoint of each edge line; If the target eye movement trajectory is consistent with the preset eye movement trajectory, an unlocking operation is performed on the screen.

2. The method of claim 1, wherein The N positioning points further comprise: light points in an array of light points displayed in the off-screen state.

3. The method of claim 2, wherein The device comprises: an infrared light source and an infrared camera; The preset eye movement trajectory is entered in an unlocking information entry process; If the N positioning points comprise the vertex of the screen and the midpoint of each edge line, the unlocking information entry process comprises: Position calibration is performed on the infrared camera; A calibration process is performed to obtain a personal calibration coefficient of the user; The user is prompted to gaze at a preset position; After recognizing that the user gazes at the preset position, an eye image of the user is collected based on the infrared light source and the infrared camera; The eye image and the personal calibration coefficient are used to identify the gaze axis and the gaze position of the eye; The gaze position corresponding to the positioning point in the N positioning points is recorded; After receiving a stop command, the unlocking information entry process is terminated.

4. The method of claim 2, wherein The device comprises: an infrared light source and an infrared camera; The preset eye movement trajectory is entered in an unlocking information entry process; If the N positioning points comprise light points in the array of light points, the unlocking information entry process comprises: Position calibration is performed on the infrared camera; A calibration process is performed to obtain a personal calibration coefficient of the user; The user is prompted to gaze at a preset position; After recognizing that the user gazes at the preset position, the array of light points is displayed on the screen; An eye image of the user is collected based on the infrared light source and the infrared camera; The eye image and the personal calibration coefficient are used to identify the gaze axis and the gaze position of the eye; The gaze position corresponding to the light point in the array of light points is recorded; After receiving a stop command, the unlocking information entry process is terminated.

5. The method of claim 1, wherein, The preset IPD information is entered in an unlocking information entry process.

6. The method according to any one of claims 1 to 5, wherein, The wake-up condition further comprises: detecting that the distance between the object and the screen meets a preset condition.

7. The method of claim 2, wherein The device comprises: an infrared light source and an infrared camera; The use of the eye tracking function to track the eye movement trajectory comprises: An eye image of the user is collected based on the infrared light source and the infrared camera; An eye gaze axis and a gaze position of an eye are recognized using an eye image; A gaze position corresponding to the gaze position in the N positioning points is determined as a positioning point being gazed at.

8. A screen-off unlocking device, characterized in that, Applied to a device with a screen and supporting an eye movement tracking function, comprising: A detection unit, configured to: In an off-screen state, continuously detect whether an eye movement tracking function wake-up condition is met, the wake-up condition comprising: detecting that a moving object exists in front of the screen; When the wake-up condition is met, the eye movement tracking function is started; An unlocking unit, configured to: acquire IPD pupillary distance information; match the acquired IPD pupillary distance information with preset IPD pupillary distance information; if the matching fails, determine that the unlocking fails; The unlocking unit is also configured to: In an off-screen state, if the matching succeeds, the eye movement tracking function is used to track an eye movement trajectory; the tracked eye movement trajectory is a target eye movement trajectory, the target eye movement trajectory comprising: in the preset N positioning points, the positioning points being gazed at in turn, the N positioning points comprising: the vertices of the screen and the midpoints of each edge line; If the target eye movement trajectory is consistent with a preset eye movement trajectory, an unlocking operation is performed on the screen.

Images