Eyeball tracking method and device, electronic equipment and readable storage medium

Abstract

The invention discloses an eyeball tracking method and device, electronic equipment and a readable storage medium, which are used for improving the accuracy of eyeball positioning. The method comprises the following steps: acquiring an eyeball image in real time through a camera; determining an eyeball tracking mode according to the initial frame of the eyeball image; and performing eyeball positioning on the eyeball image based on the eyeball tracking mode to obtain eyeball position information.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to an eye tracking method, device, electronic device and readable storage medium. Background Art

[0002] Eye tracking technology is a key technology widely used in scenarios such as augmented reality, virtual reality, medical detection, and human-computer interaction in recent years. One of its core tasks is to accurately obtain the spatial position information of the eyes in the image. Existing eye positioning methods mostly rely on traditional image processing algorithms, such as gray-scale distribution analysis, shape fitting, Hough circle detection, etc. to infer the center of the eyes. These methods can achieve certain accuracy in an ideal environment with high-quality static images and uniform illumination, but in practical applications, they often face the influence of complex factors such as image noise interference, illumination changes, partial occlusion, and insufficient image resolution, resulting in unstable eye positioning results and decreased accuracy.

[0003] In addition, to improve processing efficiency, some systems only analyze the global features of the image, lacking in-depth mining and dynamic response to local features, making it difficult to adapt to the small changes in the eye position in consecutive frames, and prone to drift, jitter, or misjudgment. Especially when there is no effective ROI area set or the initial position estimation is inaccurate, the calculation results of the eye center fluctuate greatly, affecting the response accuracy of subsequent tracking or control systems.

[0004] In summary, the existing technology is difficult to achieve stable and high-precision eye positioning in complex environments, and there is an urgent need for an eye image processing solution that can enhance the positioning accuracy in multi-frame dynamic scenarios to improve the overall performance of the eye tracking system. Summary of the Invention

[0005] This application provides an eye tracking method, device, electronic device and readable storage medium, which can improve the accuracy of eye positioning.

[0006] In the first aspect of this application, an eye tracking method is provided, including: Real-time obtaining an eye image through a camera; Determining an eye tracking method according to the initial frame of the eye image; Performing eye positioning on the eye image based on the eye tracking method to obtain eye position information.

[0007] Optionally, the determining an eye tracking method according to the initial frame of the eye image includes: Determining the eye tracking method according to whether initial position information is obtained from the initial frame of the eye image, where the initial position information is the information of the eye position in the initial frame.

[0008] Optionally, when the initial position information is not obtained, performing eye positioning on the eye image based on the eye tracking method to obtain eye position information, including: Separating the RGB channels of the eye image and extracting the blue channel to obtain a blue channel image; Determining the ROI based on the blue channel image; Calculating the gradient vector of each pixel point in the ROI; Calculating the first displacement vector of each pixel point in the ROI according to the center point of the ROI and the gradient vector; Calculating the voting value of each pixel point in the ROI according to the gradient vector and the first displacement vector; Determining the eye position information according to the voting value.

[0009] Optionally, when the initial position information is obtained, performing eye positioning on the eye image based on the eye tracking method to obtain eye position information, including: Extracting multi-channel features of the eye image; Invoking a preset filter to process the multi-channel features to generate a response map; Determining the maximum response point according to the response map; Calculating a second displacement vector according to the maximum response point; Calculating the eye position information according to the second displacement vector and the initial position information.

[0010] Optionally, after determining the maximum response point according to the response map, the method further includes: Calculating a tracking value according to the maximum response point; Judging whether the tracking is successful according to the tracking value and a preset threshold; The calculating the second displacement vector according to the maximum response point includes: When it is determined that the tracking is successful, calculating the second displacement vector according to the maximum response point.

[0011] Optionally, after performing eye positioning on the eye image based on the eye tracking method to obtain eye position information, the method further includes: Comparing the eye position information of the current frame with the eye position information of the previous frame to determine whether to update the eye position information of the current frame.

[0012] Optionally, after performing eye positioning on the eye image based on the eye tracking method to obtain eye position information, the method further includes: Send the eye position information to the light engine control system, so that the light engine control system calculates the irradiation position according to the eye position information and irradiates the eye according to the irradiation position.

[0013] The second aspect of this application provides an eye tracking device, including: An acquisition unit for acquiring an eye image in real time through a camera; A determination unit for determining an eye tracking method according to an initial frame of the eye image; A positioning unit for performing eye positioning on the eye image based on the eye tracking method to obtain eye position information.

[0014] Optionally, the determination unit is specifically configured to: Determine the eye tracking method according to whether initial position information is obtained from the initial frame of the eye image, where the initial position information is information on the eye position in the initial frame.

[0015] Optionally, when the initial position information is not obtained, the positioning unit includes: A separation module for separating the RGB channels of the eye image and extracting the blue channel to obtain a blue channel image; A first determination module for determining the ROI based on the blue channel image; A first calculation module for calculating the gradient vector of each pixel point in the ROI; A second calculation module for calculating a first displacement vector of each pixel point in the ROI according to the center point of the ROI and the gradient vector; A third calculation module for calculating a voting value of each pixel point in the ROI according to the gradient vector and the first displacement vector; A second determination module for determining eye position information according to the voting value.

[0016] Optionally, when the initial position information is obtained, the positioning unit includes: An extraction module for extracting multi-channel features of the eye image; An invocation module for invoking a preset filter to process the multi-channel features to generate a response map; A third determination module for determining a maximum response point according to the response map; A fourth calculation module for calculating a second displacement vector according to the maximum response point; A fifth calculation module for calculating eye position information according to the second displacement vector and the initial position information.

[0017] Optionally, the positioning unit further includes: A sixth calculation module, configured to calculate a tracking value according to the maximum response point; A determination module, configured to determine whether the tracking is successful according to the tracking value and a preset threshold; The fourth calculation module is specifically configured to: When it is determined that the tracking is successful, calculate a second displacement vector according to the maximum response point.

[0018] Optionally, the device further includes an update unit, and the update unit is configured to: Compare the eye position information of the current frame with the eye position information of the previous frame to determine whether to update the eye position information of the current frame.

[0019] Optionally, the device further includes a sending unit, and the sending unit is configured to: Send the eye position information to an optical engine control system, so that the optical engine control system calculates an irradiation position according to the eye position information and irradiates the eyes according to the irradiation position.

[0020] A third aspect of the present application provides an electronic device, including: A processor, a memory, an input / output unit, and a bus; The processor is connected to the memory, the input / output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method in the first aspect and any possible implementation manner of the first aspect.

[0021] A fourth aspect of the present application provides a computer-readable storage medium, on which a program is stored, and when the program is executed on a computer, the computer is caused to execute the method in the first aspect and any possible implementation manner of the first aspect.

[0022] As can be seen from the above technical solutions, the present application has the following advantages: In the embodiments of the present application, by determining the most suitable eye tracking method according to the initial frame, a strategy highly matching the current environment and image quality can be selected in the subsequent positioning process, thereby significantly improving the accuracy of eye positioning. At the same time, this embodiment can also reduce the data acquisition delay and continuously and stably output eye position information, providing reliable data for subsequent interactions and enhancing the fluency and accuracy of the usage experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic flowchart of an embodiment of an eye tracking method in the present application; Figure 2 It is a schematic flowchart of an embodiment of determining an eye tracking method in the present application; Figure 3 Schematic flowchart of an embodiment for obtaining eye position information in this application; Figure 4 Schematic flowchart of another embodiment for obtaining eye position information in this application; Figure 5 Schematic flowchart of an embodiment for determining whether tracking is successful in this application; Figure 6 Schematic flowchart of an embodiment for updating the eye position information of the current frame in this application; Figure 7 Schematic structural diagram of an embodiment of an eye tracking device in this application; Figure 8 Schematic structural diagram of an embodiment of an electronic device in this application. Detailed implementation manners

[0024] The embodiments of this application provide an eye tracking method, device, electronic device and readable storage medium, which are used to improve the accuracy of eye positioning.

[0025] The method of this application can be applied to a server, a terminal or other devices with logical processing capabilities, and this application does not make any limitation in this regard. For the convenience of description, the following takes the terminal as the execution subject for description.

[0026] Next, the embodiments in this application will be described with reference to the accompanying drawings.

[0027] Please refer to Figure 1 , Figure 1 which is an embodiment of the eye tracking method provided by this application, and this embodiment includes: 101. Obtain an eye image in real time through a camera; The terminal calls the camera driver interface, turns on the camera and sets an appropriate resolution (such as 640×480 or higher) and frame rate (such as 30fps), enters the loop acquisition mode, and continuously reads each frame of image to be processed from the camera buffer. Further, in order to improve the subsequent processing quality, the terminal can preprocess each frame of image, including using Gaussian filtering to remove noise, automatic exposure or brightness correction to stabilize the image brightness, etc.

[0028] 102. Determine the eye tracking method according to the initial frame of the eye image; The terminal analyzes the overall position and image quality of the eye in the first frame image, determines which eye tracking method is suitable in the current environment, and saves the information of this method locally in the terminal, so as to use the same strategy for subsequent images.

[0029] 103. Perform eye positioning on the eye image based on the eye tracking method to obtain the eye position information.

[0030] The terminal determines and locates the eye position in each frame of the image according to the previously determined eye tracking method, and continuously updates the eye position information of the current frame.

[0031] In this embodiment, by determining the most suitable eye tracking method according to the initial frame, the terminal can select a strategy that highly matches the current environment and image quality in the subsequent positioning process, thereby significantly improving the accuracy of eye positioning. At the same time, this embodiment can also reduce the data acquisition delay and continuously and stably output the eye position information, providing reliable data for subsequent interactions and enhancing the fluency and accuracy of the user experience.

[0032] Please refer to Figure 2 , in some embodiments of the present application, step 102 in the above embodiment of determining the eye tracking method according to the initial frame of the eye image may include the following steps: 201. Determine the eye tracking method according to whether the initial position information is obtained from the initial frame of the eye image, where the initial position information is the information of the eye position in the initial frame.

[0033] The terminal analyzes the initial frame of the eye image and attempts to extract the initial position information of the eye. If the eye position in the initial frame is successfully extracted, it indicates that the image is clear and the features are obvious, and the terminal selects a tracking method with higher accuracy and relying on the initial position information accordingly; if the initial position information is not extracted, it means that the image quality is low or the features are not obvious, and the terminal switches to a more robust and adaptable tracking method to ensure the stability of the subsequent eye positioning process. Among them, the eye tracking methods include the no-initial ROI method and the with-initial ROI method.

[0034] In practical applications, the terminal can use a pre-trained model to extract the initial position information of the initial frame of the eye image, or use the method of manual marking to extract the initial position information. If it is extracted, it can be preliminarily determined to use the with-initial ROI method for eye tracking subsequently. At this time, the algorithm corresponding to the with-initial ROI method can be initialized, and the initialization of this algorithm may include saving the initial ROI, creating a Hanning window, extracting multi-channel features (including grayscale, HOG, and color features), initializing the channel weights to a uniform distribution, creating a Gaussian response function, and creating an initial filter model, etc. If the algorithm initialization is successful, it is finally determined to select the with-initial ROI method for eye tracking. Otherwise, the no-initial ROI method is used for eye tracking.

[0035] In this embodiment, the terminal determines whether the initial frame contains eye position information, thereby matching a suitable eye tracking method, so that the subsequent eye positioning process can maintain a high accuracy under different image conditions. At the same time, the terminal's dynamic switching strategy based on image quality helps to improve the overall stability and adaptability of eye tracking.

[0036] Please refer to Figure 3 , in some embodiments of the present application, when the no-initial-ROI method is selected for eye tracking, step 103 in the above embodiment for eye positioning of the eye image based on the eye tracking method to obtain eye position information may include the following steps: 301. Separate the RGB channels of the eye image and extract the blue channel to obtain a blue channel image; After the terminal obtains the eye image, it first splits the image into three channels: red, green, and blue, and extracts the blue channel image for subsequent processing. The blue channel has more obvious contrast characteristics for the edges and structures of the eye region, which helps to enhance the effect of subsequent gradient calculations. This operation can be directly completed with an image channel separation function to obtain an image containing only the blue component.

[0037] 302. Determine the ROI based on the blue channel image; The terminal determines the ROI in the image based on the blue channel result of the current image to reduce the amount of computation and improve the positioning accuracy. Specifically, if the approximate position coordinates of the eye are already known, the ROI area is set as a local rectangular area centered on this point, and the width and height are usually 20% of the image size; if the approximate position coordinates of the eye are unknown, the ROI is default set as the central area of the image (for example, 20% - 80% horizontally and 10% - 90% vertically), and the ROI area is processed with median filtering (such as a 5×5 window) to reduce noise.

[0038] 303. Calculate the gradient vector of each pixel point in the ROI; The terminal calculates the change amplitude (i.e., gradient) of each pixel point in the ROI area in the horizontal and vertical directions respectively, and further calculates its gradient amplitude and direction. To eliminate interference, the terminal sets a dynamic threshold obtained based on histogram analysis, and only retains the pixel points whose gradient amplitude is greater than this threshold, and normalizes the gradient vectors of these points to unit vectors.

[0039] The terminal calculates the image gradients in the horizontal and vertical directions for each pixel point in the ROI area to obtain the gradient vector of each point. The calculation method can use the central difference method, and perform normalization processing on the gradient amplitude, only retaining the gradient information of the pixel points whose amplitude is greater than the dynamic threshold of.

[0040] Formula 1 Formula 2 Formula 3 Formula 4 Formula 5 wherein, is the blue channel image value; is the gradient in the x direction; is the gradient in the y direction; is the gradient magnitude; is the dynamic threshold; 、 are the normalized effective gradient components.

[0041] 304. Calculate the first displacement vector of each pixel point in the ROI according to the center point of the ROI and the gradient vector; The terminal sets the center point of the ROI as the candidate eyeball center. For each pixel point with an effective gradient, it calculates the displacement vector from this center point to this pixel point and performs normalization processing to prepare for subsequent voting.

[0042] 305. Calculate the voting value of each pixel point in the ROI according to the gradient vector and the first displacement vector; The terminal calculates the voting value of each pixel point for the candidate eyeball center. The method is to take the dot product of the gradient vector and the normalized displacement vector, take the square of the non - negative part, and then multiply it by the weight of this pixel point, and finally accumulate it into the voting value corresponding to the candidate center point. The calculation formula of the voting value is as follows: Formula 6 wherein, is the set of all pixel points in the ROI whose normalized gradient vectors are not zero; , is the displacement vector of point relative to the candidate center; is the length of the first displacement vector; is the weight of point , usually using the result of image inverse Gaussian blur, then is the cumulative voting system of candidate center .

[0043] 306. Determine the eyeball position information according to the voting value.

[0044] The terminal searches for the position of the pixel point with the largest cumulative voting value in the voting map, considers that its corresponding position is the most likely eyeball center position, and outputs its coordinates as the eyeball position information in this frame of image.

[0045] In this embodiment, the terminal extracts enhanced image features through the blue channel, defines the ROI area based on a priori or default policies, calculates the effective gradient, constructs a displacement vector for gradient voting, and finally locates the center of the eyeball by accumulating the voting values, achieving a technical path for efficiently locating the eyeball without template matching or complex models. Each step cooperates closely to make the eyeball positioning more robust and more adaptable to light changes and image noise, thereby improving the accuracy and stability of obtaining the eyeball position information.

[0046] Please refer to Figure 4 , in some embodiments of the present application, when the initial ROI method is selected for eyeball tracking, step 103 in the above embodiment performs eyeball positioning on the eyeball image based on the eyeball tracking method to obtain the eyeball position information, which may include the following steps: 401. Extract multi-channel features of the eyeball image; The terminal extracts multiple feature channels from the current eyeball image frame. Each feature channel reflects the information of the image in different perceptual dimensions, such as texture, edge, or color, etc., to prepare for subsequent filtering and matching.

[0047] 402. Call a preset filter to process the multi-channel features to generate a response map; The terminal calls a preset filter (i.e., the filter model during algorithm initialization in step 201), convolves each channel feature with the corresponding filter, and weights and sums the convolution results of all channels to obtain a comprehensive response map.

[0048] Formula 7 Where, represents the image feature of the i-th channel; represents the corresponding filter; represents the weight coefficient of the i-th channel; represents the two-dimensional convolution operation; n represents the number of channels; represents the point at the response value, which is used to reflect the matching degree.

[0049] 403. Determine the maximum response point according to the response map; The terminal traverses the response map , finds the position with the maximum response value, and uses it as the most likely position of the target in the current frame.

[0050] Formula 8 Where, represents the coordinates of the point with the maximum response value in the response map; is used to return the maximum position.

[0051] 404. Calculate the second displacement vector based on the maximum response point; The terminal calculates the offset of the maximum response point relative to the initial position as the second displacement vector and limits it within the maximum allowable displacement range.

[0052] Formula 9 Formula 10 Formula 11 Where, represents the width of the response map; represents the height of the response map; represents the offset from the center point of the response map to the maximum response point; represents the x-axis offset; represents the y-axis offset; represents the maximum allowable displacement, which is usually set to half the size of the target area.

[0053] 405. Calculate the eye position information based on the second displacement vector and the initial position information.

[0054] The terminal sets the eye position information of the current frame as the sum of the position of the previous frame and the second displacement vector, thereby updating the eye center position. The calculation formula is as follows: Formula 12 Where, represents the eye center coordinates of the previous frame; represents the eye center coordinates of the current frame; represents the second displacement vector.

[0055] In this embodiment, the terminal realizes precise positioning based on image response by extracting multi-channel features, applying a filter to generate a response map, and calculating the position information of the eye according to the maximum response point; among them, the maximum response point provides the position indication with the highest intensity, and the displacement vector calculated according to this point combined with the initial position information can dynamically correct the eye center coordinates, so that the system can maintain strong target recognition ability and positioning stability even under certain degrees of image blur, occlusion, or illumination change conditions, thereby improving the accuracy and real-time performance of eye tracking.

[0056] Please refer to Figure 5 , in some embodiments of the present application, after step 403 in the above embodiment determines the maximum response point according to the response map, the eye tracking method may further include the following steps: 501. Calculate the tracking value according to the maximum response point; After the terminal obtains the maximum response point, it calculates the contrast between the peak value of the response map and the surrounding background area as the tracking value for this frame. Using the PSR (Peak to Sidelobe Ratio) calculation method, the larger the PSR value, the more obvious the difference between the peak and the background, and the better the tracking effect. The calculation formula is as follows: Formula 13 Among them, represents the maximum value in the response map, corresponding to the maximum response point; represents the average value of the sidelobe area (i.e., the remaining area after excluding the peak area); represents the standard deviation of the sidelobe area.

[0057] The terminal will delimit a peak area centered on the maximum response point (set according to requirements, usually with a size of 5*5). After excluding this area, it calculates the mean and standard deviation of the remaining pixels in the sidelobe area to obtain the PSR value.

[0058] 502. Determine whether the tracking is successful based on the tracking value and the preset threshold; The terminal compares the calculated PSR value with the preset threshold. If it is greater than the threshold, it determines that the tracking of this frame is successful; otherwise, it considers the tracking to be failed, thereby triggering a repositioning mechanism or using the position of the previous frame for correction.

[0059] When it is determined that the tracking is successful, the terminal continues to execute step 404 below.

[0060] In this embodiment, the terminal calculates the PSR value after obtaining the maximum response point and compares it with the preset threshold. It only confirms that the tracking is successful when the PSR is higher than the threshold, thereby automatically filtering out low-quality positioning before the result is output, improving the accuracy of the eye position information; at the same time, the threshold judgment enables the terminal to trigger repositioning in a timely manner or correct using the position of the previous frame when the tracking fails, reducing the jitter caused by displacement jumps and improving the overall stability and robustness of the tracking.

[0061] Please refer to Figure 6 , in some embodiments of the present application, after performing eye positioning on the eye image based on the eye tracking method in step 103 of the above embodiment to obtain the eye position information, the eye tracking method may further include the following steps: 601. Compare the eye position information of the current frame with the eye position information of the previous frame to determine whether to update the eye position information of the current frame.

[0062] After obtaining the eye position information of the current frame, the terminal compares this information with the eye position of the previous frame to determine whether to perform an update operation. Specifically, the terminal uses a position smoothing strategy. Instead of directly adopting the current detection value as the eye position in each frame, it combines the smoothing result of the previous frame and controls the change trend of the positioning result through an update mechanism. If the difference between the current detection result and the smoothed value of the previous frame exceeds the set threshold, it is regarded as a valid displacement, and the terminal will update the eye position of the current frame; if the difference is small, it is considered that there may be detection errors or jitters, and the terminal will continue to maintain the result of the previous frame to reduce error accumulation or screen jitter caused by high-frequency noise.

[0063] For example, the terminal can calculate the smoothed value of the current frame eye position information based on the previous frame eye position information and the current frame eye position information, and then calculate the pixel offset between this smoothed value and the previous frame eye position information. If the offset is greater than the set threshold, this smoothed value is used as the latest eye position system information. The specific calculation formula is as follows: Formula 14 Formula 15 Among them, is the current smoothed value; is the smoothed value at the previous moment; is the current measured value; is the influencing factor, which can be adjusted according to the actual situation; is the preset threshold of pixel offset.

[0064] 602. Send the eye position information to the light engine control system so that the light engine control system calculates the irradiation position according to the eye position information and irradiates the eyes according to the irradiation position.

[0065] After obtaining the accurate eye position information, the terminal uses this information as a control parameter and transmits it to the light engine control system to guide the light engine to perform the irradiation operation. To ensure the real-time and accuracy of the control, the terminal uses a specific transmission mechanism to timely transmit the pixel coordinates of the eye center position and trigger the light engine control system to perform position conversion. After receiving this coordinate, the light engine control system maps the pixel coordinates to the control coordinates required by the light engine control system according to the conversion relationship between the camera coordinates and the light engine coordinates, and adjusts the irradiation position and light direction accordingly to ensure that the light spot accurately falls on the target area.

[0066] For example, the terminal obtains the position of the center of the eyeball in the current frame image as pixel coordinates (x, y), and transmits it to the optical engine control system in a form with a delay of less than 5 milliseconds. The optical engine control system uses the coordinate transformation matrix in the configuration file to convert the pixel coordinates into target control coordinates (x′, y′), and the transformation formula is as follows: Formula 16 Among them, (x, y) is the pixel position in the camera coordinate system (that is, the eyeball position information sent by the terminal); (x′, y′) is the position required by the optical engine control system (that is, the irradiation information calculated by the optical engine control system); 、 、 、 、 、 are preset values used to describe the translation, rotation, and scaling relationships of the affine transformation.

[0067] It should be noted that in this embodiment, the terminal can either only execute any one of step 601 or step 602, or execute both step 601 and step 602. Moreover, there is no strict execution order between step 601 and step 602. If the terminal executes both, then the terminal can either execute step 601 first and then step 602, or execute step 602 first and then step 601.

[0068] In this embodiment, when the terminal determines whether to update the eyeball position information of the current frame, it compares with the previous frame data to ensure that the positioning result of the current frame is more reliable, thereby improving the accuracy of the final eyeball positioning; at the same time, the terminal timely transmits the updated eyeball position information to the optical engine control system, and cooperates with coordinate transformation and fast response to achieve real-time linkage between eyeball tracking and light control.

[0069] Please refer to Figure 7 , Figure 7 which is an embodiment of the eyeball tracking device provided by this application. This embodiment includes: An acquisition unit 701, configured to acquire an eyeball image in real time through a camera; A determination unit 702, configured to determine an eyeball tracking method according to the initial frame of the eyeball image; A positioning unit 703, configured to perform eyeball positioning on the eyeball image based on the eyeball tracking method to obtain eyeball position information.

[0070] In this embodiment, by determining the most suitable eye tracking method based on the initial frame, the eye tracking device can select a strategy that highly matches the current environment and image quality during subsequent positioning, thereby significantly improving the accuracy of eye positioning. At the same time, this embodiment can also reduce the data acquisition delay and continuously and stably output the eye position information, providing reliable data for subsequent interactions and enhancing the fluency and accuracy of the usage experience.

[0071] Optionally, the determining unit 702 is specifically configured to: Determine the eye tracking method according to whether initial position information is obtained from the initial frame of the eye image, where the initial position information is the information of the eye position in the initial frame.

[0072] Optionally, when the initial position information is not obtained, the positioning unit 703 includes: A separation module, configured to separate the RGB channels of the eye image and perform blue channel extraction to obtain a blue channel image; A first determination module, configured to determine the ROI based on the blue channel image; A first calculation module, configured to calculate the gradient vector of each pixel point in the ROI; A second calculation module, configured to calculate the first displacement vector of each pixel point in the ROI according to the center point of the ROI and the gradient vector; A third calculation module, configured to calculate the voting value of each pixel point in the ROI according to the gradient vector and the first displacement vector; A second determination module, configured to determine the eye position information according to the voting value.

[0073] Optionally, when the initial position information is obtained, the positioning unit 703 includes: An extraction module, configured to extract multi-channel features of the eye image; An invocation module, configured to invoke a preset filter to process the multi-channel features to generate a response map; A third determination module, configured to determine the maximum response point according to the response map; A fourth calculation module, configured to calculate the second displacement vector according to the maximum response point; A fifth calculation module, configured to calculate the eye position information according to the second displacement vector and the initial position information.

[0074] Optionally, the positioning unit 703 further includes: A sixth calculation module, configured to calculate a tracking value according to the maximum response point; A judgment module, configured to judge whether the tracking is successful according to the tracking value and a preset threshold; The fourth calculation module is specifically configured to: When it is determined that the tracking is successful, calculate the second displacement vector according to the maximum response point.

[0075] Optionally, the device further includes an updating unit, and the updating unit is configured to: Compare the eye position information of the current frame with the eye position information of the previous frame to determine whether to update the eye position information of the current frame.

[0076] Optionally, the device further includes a sending unit, and the sending unit is configured to: Send the eye position information to the optical engine control system, so that the optical engine control system calculates the irradiation position according to the eye position information and irradiates the eyes according to the irradiation position.

[0077] In this embodiment, the functions of each unit and module correspond to the steps in the foregoing Figures 1 to 6 illustrated embodiment, and will not be described in detail here.

[0078] Please refer to Figure 8 , Figure 8 which is an embodiment of the electronic device provided by the present application. This embodiment includes: A processor 801, a memory 802, an input / output unit 803, and a bus 804; The processor 801 is connected to the memory 802, the input / output unit 803, and the bus 804; A program is stored on the memory 802, and the processor 801 calls the program to execute Figures 1 to 6 the steps in the illustrated embodiment.

[0079] In this embodiment, the function of the processor 801 corresponds to the steps in the foregoing Figures 1 to 6 illustrated embodiment, and will not be described in detail here.

[0080] The embodiment of the present application further provides a computer-readable storage medium, on which a program is stored. When the program is executed on a computer, the computer is caused to execute the method in any of the foregoing Figures 1 to 6 possible embodiments.

[0081] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments, and will not be described in detail here.

[0082] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be in electrical, mechanical, or other forms.

[0083] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0084] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0085] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

Claims

1. An eye tracking method, characterized in that, including: Obtaining an eyeball image in real time through a camera; Determining an eyeball tracking method according to an initial frame of the eyeball image; Performing eyeball positioning on the eyeball image based on the eyeball tracking method to obtain eyeball position information.

2. The method according to claim 1, wherein The determining the eyeball tracking method according to the initial frame of the eyeball image includes: Determining the eyeball tracking method according to whether initial position information is obtained from the initial frame of the eyeball image, where the initial position information is information about the position of the eyeball in the initial frame.

3. The method according to claim 2, wherein When the initial position information is not obtained, the performing eyeball positioning on the eyeball image based on the eyeball tracking method to obtain eyeball position information includes: Separating the RGB channels of the eyeball image and performing blue channel extraction to obtain a blue channel image; Determining an ROI based on the blue channel image; Calculating a gradient vector for each pixel point in the ROI; Calculating a first displacement vector for each pixel point in the ROI according to the center point of the ROI and the gradient vector; Calculating a voting value for each pixel point in the ROI according to the gradient vector and the first displacement vector; Determining the eyeball position information according to the voting value.

4. The method according to claim 2, wherein When the initial position information is obtained, the performing eyeball positioning on the eyeball image based on the eyeball tracking method to obtain eyeball position information includes: Extracting multi-channel features of the eyeball image; Invoking a preset filter to process the multi-channel features to generate a response map; Determining a maximum response point according to the response map; Calculating a second displacement vector according to the maximum response point; Calculating the eyeball position information according to the second displacement vector and the initial position information.

5. The method according to claim 4, wherein After the determining the maximum response point according to the response map, the method further includes: Calculating a tracking value according to the maximum response point; Judging whether the tracking is successful according to the tracking value and a preset threshold; The calculating the second displacement vector according to the maximum response point includes: When it is determined that the tracking is successful, calculating the second displacement vector according to the maximum response point.

6. The method according to any one of claims 1 to 5, characterized in that, After the performing eyeball positioning on the eyeball image based on the eyeball tracking method to obtain eyeball position information, the method further includes: Comparing the eyeball position information of the current frame with the eyeball position information of the previous frame to determine whether to update the eyeball position information of the current frame.

7. The method according to any one of claims 1 to 5, characterized in that, After the performing eyeball positioning on the eyeball image based on the eyeball tracking method to obtain eyeball position information, the method further includes: Sending the eyeball position information to an optical engine control system so that the optical engine control system calculates an irradiation position according to the eyeball position information and irradiates the eye according to the irradiation position.

8. An eye tracking device, characterized in that, including: An obtaining unit, configured to obtain an eyeball image in real time through a camera; A determining unit, configured to determine an eyeball tracking method according to an initial frame of the eyeball image; A positioning unit, configured to perform eyeball positioning on the eyeball image based on the eyeball tracking method to obtain eyeball position information.

9. An electronic device, characterized in that, including: A processor, a memory, an input / output unit, and a bus; The processor is connected to the memory, the input / output unit, and the bus; A program is stored in the memory, and the processor calls the program to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A program is stored on the computer-readable storage medium, and when the program is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images