A line-of-sight estimation method, apparatus and device

Abstract

This application provides a gaze estimation method, apparatus, and device. The method includes: for each device type, acquiring a first dataset corresponding to that device type, and determining a systematic bias corresponding to that device type based on the true gaze points and predicted gaze points in the first dataset; for each candidate user, acquiring a second dataset corresponding to that candidate user, correcting the predicted gaze points based on the systematic bias corresponding to the user's device type to obtain corrected gaze points; determining a user bias corresponding to that candidate user based on the true gaze points and corrected gaze points in the second dataset; and storing the systematic bias corresponding to each device type and the user bias corresponding to each candidate user; wherein the systematic bias and the user bias are used to correct the estimated gaze points in gaze estimation. Through the technical solution of this application, accurate and reliable gaze point coordinates can be obtained, avoiding problems such as large errors and inaccurate positions in gaze point coordinates.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a gaze estimation method, apparatus, and device. Background Technology

[0002] Gaze estimation is a technology related to the eyeball, eye movement, and gaze. It uses eye images or facial images as the processing object. In the process of gaze estimation, the user's gaze direction is estimated based on the eye image or facial image, and the coordinates of the gaze point that intersects with the gaze direction are obtained by combining the region coordinates.

[0003] Eye gaze estimation can be applied to various scenarios, including gaming, VR, driver assistance systems (smart cockpits), and screen-based applications (such as phone unlocking and short video effects). In gaming, eye gaze estimation facilitates interactive gameplay. In VR, the headset's built-in camera accurately estimates the user's gaze direction, enabling detailed rendering of specific areas of the scene. In driver assistance systems, analyzing the user's gaze direction helps detect driver fatigue and attention levels, improving safety. In screen-based applications, eye gaze estimation enables interactive communication. Furthermore, it allows for assessing user focus levels, rating classroom activities, products, or videos, and estimating user interests.

[0004] Gaze estimation is used to estimate the coordinates of the user's gaze point. However, these gaze point coordinates may have large errors or inaccurate positions, meaning that accurate and reliable gaze point coordinates cannot be obtained. Summary of the Invention

[0005] This application provides a line-of-sight estimation method, the method comprising:

[0006] For each device type, a first dataset corresponding to that device type is obtained. The first dataset includes the actual gaze points and predicted gaze points when the user device of that device type performs gaze estimation on the user. Based on the actual gaze points and predicted gaze points in the first dataset, the system bias corresponding to that device type is determined.

[0007] For each candidate user, a second dataset corresponding to the candidate user is obtained. The second dataset includes the actual gaze points and predicted gaze points when the user device estimates the gaze of the candidate user. The predicted gaze points are corrected based on the system bias corresponding to the device type of the user device to obtain the corrected gaze points. The user bias corresponding to the candidate user is determined based on the actual gaze points and corrected gaze points in the second dataset.

[0008] The system bias corresponding to each device type and the user bias corresponding to each candidate user are stored; wherein, the system bias and the user bias are used to correct the estimated gaze point of gaze estimation.

[0009] This application provides a line-of-sight estimation method, the method comprising:

[0010] For each device type, obtain the first dataset corresponding to that device type. The first dataset includes the actual gaze points and predicted gaze points when the user device of that device type performs gaze estimation on the user.

[0011] For each candidate user, a second dataset corresponding to that candidate user is obtained. The second dataset includes the actual gaze points and predicted gaze points when the user device performs gaze estimation for that candidate user. Based on the actual gaze points and predicted gaze points in the second dataset, the user bias corresponding to that candidate user is determined.

[0012] Determine the user bias corresponding to the predicted fixation point in the first dataset, and correct the predicted fixation point based on the user bias to obtain the corrected fixation point; determine the system bias corresponding to the device type based on the true fixation point in the first dataset and the corrected fixation point;

[0013] The system stores the user bias for each candidate user and the system bias for each device type; wherein the user bias and the system bias are used to correct the estimated gaze point of gaze estimation.

[0014] This application provides a line-of-sight estimation device, the device comprising:

[0015] The acquisition module is used to acquire a first dataset corresponding to each device type, which includes the actual gaze points and predicted gaze points when the user device of that device type performs gaze estimation on the user; and to acquire a second dataset corresponding to each candidate user, which includes the actual gaze points and predicted gaze points when the user device performs gaze estimation on the candidate user.

[0016] The determination module is used to determine the system bias corresponding to the device type based on the real fixation points and predicted fixation points in the first dataset; to correct the predicted fixation points based on the system bias corresponding to the device type in the second dataset to obtain corrected fixation points; and to determine the user bias corresponding to the candidate user based on the real fixation points in the second dataset and the corrected fixation points.

[0017] The storage module is used to store the system bias corresponding to each device type and the user bias corresponding to each candidate user. The system bias and user bias are used to correct the estimated gaze point of gaze estimation.

[0018] This application provides a line-of-sight estimation device, the device comprising:

[0019] The acquisition module is used to acquire a first dataset corresponding to each device type, which includes the actual gaze points and predicted gaze points when the user device of that device type performs gaze estimation on the user; and to acquire a second dataset corresponding to each candidate user, which includes the actual gaze points and predicted gaze points when the user device performs gaze estimation on the candidate user.

[0020] The determination module is used to determine the user bias corresponding to the candidate user based on the true fixation points and predicted fixation points in the second dataset; determine the user bias corresponding to the user corresponding to the predicted fixation point in the first dataset; correct the predicted fixation point based on the user bias to obtain the corrected fixation point; and determine the system bias corresponding to the device type based on the true fixation points in the first dataset and the corrected fixation point.

[0021] The storage module is used to store the user bias corresponding to each candidate user and the system bias corresponding to each device type. The user bias and system bias are used to correct the estimated gaze point of gaze estimation.

[0022] This application provides an electronic device, including: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions executable by the processor; the processor is configured to execute the machine-executable instructions to implement the gaze estimation method of the above example.

[0023] As can be seen from the above technical solutions, in this embodiment, the system deviation corresponding to each device type and the user deviation corresponding to each candidate user can be obtained. Based on the system deviation and user deviation, the estimated gaze point of the gaze estimation is corrected, thereby obtaining accurate and reliable gaze point coordinates and avoiding problems such as large errors and inaccurate positions of the gaze point coordinates. The gaze deviation can be distinguished into system deviation and user deviation. System deviation is the gaze deviation caused by differences in user devices, while user deviation is the gaze deviation caused by differences in users. By decoupling the system deviation and user deviation and estimating them separately, the accuracy and flexibility of gaze correction can be improved, and more accurate correction parameters can be solved, thereby improving the correction accuracy. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings of the embodiments of this application.

[0025] Figure 1 This is a flowchart illustrating a line-of-sight estimation method in one embodiment of this application;

[0026] Figure 2 This is a flowchart illustrating a line-of-sight estimation method in one embodiment of this application;

[0027] Figure 3 This is a schematic diagram illustrating the decoupling of system bias and user bias for line-of-sight correction.

[0028] Figure 4 This is a flowchart illustrating a line-of-sight estimation method in one embodiment of this application;

[0029] Figure 5 This is a schematic diagram of the line-of-sight estimation device in one embodiment of this application;

[0030] Figure 6 This is a schematic diagram of the line-of-sight estimation device in one embodiment of this application;

[0031] Figure 7 This is a hardware structure diagram of an electronic device according to one embodiment of this application. Detailed Implementation

[0032] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “the,” and “the” as used in this application and claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to any and all possible combinations comprising one or more of the associated listed items.

[0033] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" may also be interpreted as "when," "when," or "in response to a determination."

[0034] This application proposes a line-of-sight estimation method in its embodiments; see [link to relevant documentation]. Figure 1 As shown, the method may include:

[0035] Step 101: For each device type, obtain the first dataset corresponding to that device type. The first dataset includes the actual gaze points and predicted gaze points when the user device of that device type performs gaze estimation on the user. Determine the system bias corresponding to that device type based on the actual gaze points and predicted gaze points in the first dataset.

[0036] For example, determining the system bias corresponding to the device type based on the actual gaze point and the predicted gaze point in the first dataset may include, but is not limited to: determining the lateral system correction parameter based on the x-coordinate of the predicted gaze point, the y-coordinate of the predicted gaze point, and the x-coordinate of the actual gaze point; determining the longitudinal system correction parameter based on the x-coordinate of the predicted gaze point, the y-coordinate of the predicted gaze point, and the y-coordinate of the actual gaze point; and determining the system bias based on the lateral system correction parameter and the longitudinal system correction parameter.

[0037] Step 102: For each candidate user, obtain the second dataset corresponding to the candidate user. The second dataset includes the real gaze point and the predicted gaze point when the user device performs gaze estimation for the candidate user. Correct the predicted gaze point based on the system bias corresponding to the device type of the user device to obtain the corrected gaze point. Determine the user bias corresponding to the candidate user based on the real gaze point and the corrected gaze point in the second dataset.

[0038] For example, correcting the predicted gaze point based on the system bias corresponding to the device type of the user device to obtain the corrected gaze point may include, but is not limited to: if the system bias includes a lateral system correction parameter and a longitudinal system correction parameter, then determining the lateral coordinate of the corrected gaze point based on the lateral coordinate of the predicted gaze point, the ordinate of the predicted gaze point, and the lateral system correction parameter; determining the ordinate of the corrected gaze point based on the lateral coordinate of the predicted gaze point, the ordinate of the predicted gaze point, and the longitudinal system correction parameter; and determining the corrected gaze point based on the lateral coordinate of the corrected gaze point and the ordinate of the corrected gaze point.

[0039] For example, determining the user bias corresponding to the candidate user based on the real fixation point and the corrected fixation point in the second dataset may include, but is not limited to: determining the lateral user correction parameters based on the x-coordinate of the corrected fixation point, the y-coordinate of the corrected fixation point, and the x-coordinate of the real fixation point; and determining the longitudinal user correction parameters based on the x-coordinate of the corrected fixation point, the y-coordinate of the corrected fixation point, and the y-coordinate of the real fixation point; and determining the user bias based on the lateral user correction parameters and the longitudinal user correction parameters.

[0040] Step 103: Store the system bias corresponding to each device type and the user bias corresponding to each candidate user; wherein, the system bias and user bias are used to correct the estimated gaze point of gaze estimation.

[0041] In one possible implementation, after step 103, a correction request sent by the target user equipment can be received. This correction request may include the target user's user identifier, the target user equipment's device type, and the estimated gaze point when the target user equipment estimates the target user's gaze. The estimated gaze point is then corrected for system bias based on the system bias corresponding to the target user equipment's device type, resulting in a corrected estimated gaze point. Next, the corrected estimated gaze point is corrected for user bias based on the user identifier corresponding to the target user, resulting in a corrected target gaze point. Finally, the corrected target gaze point is sent to the target user equipment.

[0042] In another possible implementation, after step 103, a data request sent by the target user equipment can be received. This data request includes the user identifier of the target user and the device type of the target user equipment. The system bias corresponding to the device type and the user bias corresponding to the user identifier are sent to the target user equipment so that the target user equipment can perform system bias correction on the estimated gaze point (the estimated gaze point when the target user equipment estimates the gaze of the target user) based on the system bias, obtain the corrected estimated gaze point, and perform user bias correction on the corrected estimated gaze point based on the user bias, obtain the corrected target gaze point.

[0043] As can be seen from the above technical solutions, in this embodiment, the system deviation corresponding to each device type and the user deviation corresponding to each candidate user can be obtained. Based on the system deviation and user deviation, the estimated gaze point of the gaze estimation is corrected, thereby obtaining accurate and reliable gaze point coordinates and avoiding problems such as large errors and inaccurate positions of the gaze point coordinates. The gaze deviation can be distinguished into system deviation and user deviation. System deviation is the gaze deviation caused by differences in user devices, while user deviation is the gaze deviation caused by differences in users. By decoupling the system deviation and user deviation and estimating them separately, the accuracy and flexibility of gaze correction can be improved, and more accurate correction parameters can be solved, thereby improving the correction accuracy.

[0044] The technical solutions described above in the embodiments of this application will be explained below in conjunction with specific application scenarios.

[0045] During gaze estimation, the user's gaze direction can be estimated based on eye or facial images, and the coordinates of the gaze point intersecting this gaze direction can be obtained by combining region coordinates. However, these gaze point coordinates may have large errors or inaccurate positions, meaning that accurate and reliable gaze point coordinates cannot be obtained.

[0046] In response to the above findings, this application proposes a gaze estimation method that can obtain the system bias corresponding to each device type and the user bias corresponding to each candidate user. Based on the system bias and user bias, the estimated gaze point (i.e., gaze point coordinates) of the gaze estimation is corrected, thereby obtaining accurate and reliable gaze point coordinates and avoiding problems such as large errors and inaccurate positions of gaze point coordinates.

[0047] In this embodiment, gaze deviation can be divided into system deviation and user deviation. System deviation is gaze deviation caused by differences in user devices, while user deviation is gaze deviation caused by differences in users. By decoupling system deviation and user deviation and estimating them separately, the gaze point coordinates can be corrected based on the gaze deviation caused by differences in user devices and user deviations. This can improve the accuracy and flexibility of gaze correction, thereby improving the accuracy of gaze point coordinate correction and obtaining accurate and reliable gaze point coordinates.

[0048] Systematic bias is a bias that is related to the user's equipment but not to the user. For example, it is an error caused by the inconsistency between the data acquisition and calibration system used to train the algorithm and the data acquisition and calibration system deployed on-site. For instance, the calibration system used to train the algorithm may be biased to the left by one degree, while the calibration system deployed on-site may be biased to the right by one degree. This fixed bias is unrelated to the user and is only related to the deployed equipment (such as different car models or mobile phone models). It can be assumed that the bias is the same for a fixed car model or a fixed mobile phone model. This bias can be called systematic bias, that is, each equipment type corresponds to a systematic bias.

[0049] User bias is a bias that is related to the user but not to the user's equipment. For example, bias caused by user differences, such as the bias caused by the different deviation angles of the visual axis and optical axis of each user. Obviously, user bias is related to each user but not to the user's equipment and calibration system.

[0050] See Figure 2 The diagram shown illustrates the flow chart of a line-of-sight estimation method, which may include:

[0051] Step 201: The server obtains sample data, which includes the actual gaze point and predicted gaze point when the user device estimates the user's gaze, the device type of the user device, and the user's user identifier.

[0052] For example, the application scenarios of this gaze estimation method may include, but are not limited to, games, VR, driver assistance (smart cockpit), screen applications (such as phone unlocking, short video effects, etc.).

[0053] For example, if this gaze estimation method is applied to a game scenario, it can guide the user to gaze at a designated location. The coordinates of this location are the actual gaze point (i.e., the actual gaze point coordinates) when the user device (such as a smartphone, smart screen, AR / VR device, etc.) estimates the user's gaze. When the user gazes at the designated location, eye images or facial images can be captured. Based on these images, the user's gaze direction can be estimated, and the coordinates of the gaze point intersecting this gaze direction can be obtained by combining the region coordinates. This is the predicted gaze point (i.e., the predicted gaze point coordinates) when the user device estimates the user's gaze. Then, the user device can send the actual gaze point, the predicted gaze point, the device type of the user device, and the user's user identifier to the server.

[0054] If this gaze estimation method is applied to assisted driving scenarios, it can guide the user to gaze at a designated location. The coordinates of this location are the actual gaze point when the user device (such as an in-vehicle terminal) estimates the user's gaze. When the user gazes at the designated location, an eye image or facial image can be captured. Based on the eye or facial image, the user's gaze direction can be estimated, and the coordinates of the gaze point intersecting this gaze direction can be obtained—that is, the predicted gaze point when the user device estimates the user's gaze. Alternatively, instead of guiding the user to gaze at a designated location, the user's gaze position can be determined based on the user's actions. For example, when turning the steering wheel to the left, the user's gaze position is the center of the left rearview mirror. The coordinates of this gaze position are the actual gaze point when the user device estimates the user's gaze. An eye image or facial image corresponding to the user's gaze position can be captured. Based on the eye or facial image, the user's gaze direction can be estimated, and the coordinates of the gaze point intersecting this gaze direction can be obtained—that is, the predicted gaze point when the user device estimates the user's gaze. Then, the user device can send this actual gaze point, this predicted gaze point, the user device type, and the user's user identifier to the server.

[0055] If this gaze estimation method is applied to scenarios such as VR and screen-based applications, the actual gaze point and the predicted gaze point, the device type of the user's device, and the user's user identifier can be sent to the server without any restrictions.

[0056] For example, the user equipment can send the real gaze point and the predicted gaze point, the device type of the user equipment, and the user's user identifier to the server. Since only text information is provided, and there is no need to provide images and the user's real identity information, the amount of data transmission can be reduced, network bandwidth can be saved, and the security of user data can be protected.

[0057] Step 202: For each device type, the server obtains the first dataset corresponding to that device type based on the sample data. The first dataset may include multiple sets of gaze point data. Each set of gaze point data may include the actual gaze point and the predicted gaze point when the user device of that device type estimates the user's gaze.

[0058] For example, the server can cluster all sample data according to device type to obtain sample data corresponding to the same device type, and parse the fove point data (i.e., real fove points and predicted fove points) from the sample data. Then, the multiple fove point data are combined into a first dataset. See Table 1 and Table 2 for two examples of the first dataset. There are no restrictions on the content of this first dataset.

[0059] Table 1

[0060]

[0061]

[0062] Table 2

[0063]

[0064] In summary, for each device type, the server can obtain the first dataset corresponding to that device type, and the first dataset includes multiple sets of gaze point data. These gaze point data can include the actual gaze points and predicted gaze points when the user device of that device type performs gaze estimation for multiple users.

[0065] Step 203: For each candidate user, the server obtains the second dataset corresponding to the candidate user based on the sample data. The second dataset may include multiple sets of gaze point data. Each set of gaze point data may include the actual gaze point and the predicted gaze point when the user device performs gaze estimation for the candidate user.

[0066] For example, the server can cluster all sample data according to user identifiers (one user identifier corresponds to one candidate user) to obtain sample data corresponding to the same user identifier, and parse the fixation data (i.e., real fixation and predicted fixation) from the sample data. Then, multiple fixation data are combined into a second dataset, as shown in Tables 3 and 4, which are two examples of the second dataset.

[0067] Table 3 Table 4

[0068]

[0069] In summary, for each user identifier (i.e., candidate user), the server can obtain a second dataset corresponding to that user identifier. This second dataset includes multiple sets of gaze point data, which may include the actual gaze points and predicted gaze points when the user device estimates the gaze of the candidate user.

[0070] For example, the first dataset includes multiple sets of fixation point data, each set of which may include the actual and predicted fixation points when the user device estimates the user's gaze. The second dataset includes multiple sets of fixation point data, each set of which may include the actual and predicted fixation points when the user device estimates the gaze of a candidate user. The user device corresponding to the fixation point data in the first dataset and the user device corresponding to the fixation point data in the second dataset may be the same user device or different user devices. For instance, the fixation point data in the first dataset is used to determine system bias and is fixation point data for the same device type, such as user device 1, user device 2, and user device 3 corresponding to the same device type, and the first dataset includes the fixation point data corresponding to user device 1, user device 2, and user device 3. The fixation point data in the second dataset is used to determine user bias and is fixation point data for the same candidate user, such as user device 1 and user device 4 corresponding to the same candidate user, and the second dataset includes the fixation point data corresponding to user device 1 and user device 4.

[0071] Step 204: For each device type, the server determines the system bias corresponding to that device type based on the true fixation points and predicted fixation points in the first dataset corresponding to that device type.

[0072] For example, lateral system correction parameters can be determined based on the x-coordinate of the predicted gaze point, the y-coordinate of the predicted gaze point, and the x-coordinate of the actual gaze point, and longitudinal system correction parameters can be determined based on the same parameters. The system bias corresponding to this device type is then determined based on the lateral and longitudinal system correction parameters.

[0073] For example, the predicted gaze coordinates (x and y) can be used as input values ​​to a first function, and the actual gaze coordinates (x and y) as output values ​​to determine the parameter values ​​of the first function, which then serve as lateral system correction parameters. Similarly, the predicted gaze coordinates (x and y) can be used as input values ​​to a second function, and the actual gaze coordinates (y) as output values ​​to determine the parameter values ​​of the second function, which then serve as longitudinal system correction parameters. These lateral and longitudinal system correction parameters can then be used as the system bias corresponding to this device type.

[0074] In one possible implementation, the first function can be referred to as Equation (1), and the second function can be referred to as Equation (2), where (x, y) represents the x and y coordinates of the predicted gaze point, serving as the input values ​​for the first and second functions. The x and y coordinates of the true gaze point are used as the output values ​​of the first and second functions. a, b, and c represent the lateral system correction parameters, which are the parameter values ​​of the first function. d, e, and f represent the longitudinal system correction parameters, which are the parameter values ​​of the second function.

[0075]

[0076]

[0077] Because multiple sets of fixation point data can be obtained, each set of fixation point data includes the true fixation point (x, y) and the predicted fixation point. Therefore, multiple sets of fixation point data can be substituted into formulas (1) and (2) to obtain the lateral system correction parameters a, b, c and the longitudinal system correction parameters d, e, f. For example, the following equation can be constructed based on multiple sets of fixation point data, and then the lateral system correction parameters a, b, c and the longitudinal system correction parameters d, e, f can be obtained by the least squares method. Since a, b, c, d, e, f are obtained from data from multiple different users, they can be used as system biases.

[0078]

[0079] ...

[0081]

[0082]

[0083] Step 205: For each candidate user, the second dataset may include the actual gaze point and the predicted gaze point when the user device estimates the gaze of the candidate user. The server corrects the predicted gaze point based on the system bias corresponding to the device type of the user device to obtain the corrected gaze point.

[0084] For example, referring to Table 3 or Table 4, for the second dataset corresponding to candidate users of user identifier s1, the predicted fixation point a12 can be corrected based on the system bias corresponding to device type a to obtain the corrected fixation point a12', and the predicted fixation point a32 can be corrected based on the system bias corresponding to device type a to obtain the corrected fixation point a32'. For the second dataset corresponding to candidate users of user identifier s2, the predicted fixation point a22 can be corrected based on the system bias corresponding to device type a to obtain the corrected fixation point a22', and the predicted fixation point b12 can be corrected based on the system bias corresponding to device type b to obtain the corrected fixation point b12'. After correcting the predicted fixation points, the second dataset shown in Table 5 can be obtained.

[0085] Table 5

[0086]

[0087]

[0088] For example, correcting the predicted gaze point based on the system bias corresponding to the device type to obtain the corrected gaze point may include: if the system bias includes a lateral system correction parameter and a longitudinal system correction parameter, the lateral system correction parameter can be used to determine the lateral coordinate of the corrected gaze point; the longitudinal system correction parameter can be used to determine the longitudinal coordinate of the corrected gaze point; and then, the corrected gaze point is determined based on the lateral and longitudinal system correction parameters.

[0089] For example, as shown in formulas (1) and (2), the system deviation corresponding to the device type may include lateral system correction parameters a, b, c and longitudinal system correction parameters d, e, f. (x, y) represents the abscissa and ordinate of the predicted fixation point. The (x, y) of the predicted fixation point and the lateral system correction parameters a, b, c can be substituted into formula (1) to obtain the abscissa of the corrected fixation point. The predicted fixation point (x, y) and the longitudinal system correction parameters d, e, and f can be substituted into formula (2) to obtain the ordinate of the corrected fixation point. Obviously, the x-axis and ordinate Composition of corrected fixation points That is, the predicted fixation point is corrected to obtain the corrected fixation point.

[0090] In summary, the predicted gaze point can be corrected based on the system bias corresponding to the device type to obtain the corrected gaze point. This yields both the true gaze point and the corrected gaze point for the same candidate user, and the corrected gaze point is obtained after correcting the predicted gaze point using the system bias. By correcting the predicted gaze point using the system bias to obtain the corrected gaze point, and then determining the user bias of the candidate user based on the true gaze point and the corrected gaze point, the system bias and user bias can be decoupled.

[0091] Step 206: For each candidate user, the server determines the user bias corresponding to the candidate user based on the true fixation point and the corrected fixation point in the second dataset corresponding to the candidate user.

[0092] For example, the predicted fixation points in the second dataset can be corrected using systematic bias to obtain corrected fixation points. That is, the second dataset includes true fixation points and corrected fixation points, so that the user bias corresponding to the candidate user can be determined based on the true fixation points and corrected fixation points in the second dataset.

[0093] For example, lateral user correction parameters can be determined based on the x-coordinate of the corrected gaze point, the y-coordinate of the corrected gaze point, and the x-coordinate of the actual gaze point, and longitudinal user correction parameters can be determined based on the x-coordinate of the corrected gaze point, the y-coordinate of the corrected gaze point, and the y-coordinate of the actual gaze point. The user bias corresponding to the candidate user is then determined based on the lateral and longitudinal user correction parameters.

[0094] For example, the x and y coordinates of the corrected gaze point can be used as input values ​​to a third function, and the x coordinate of the actual gaze point can be used as the output value of the third function to determine its parameter values, which then serve as the lateral user correction parameters. Similarly, the x and y coordinates of the corrected gaze point can be used as input values ​​to a fourth function, and the y coordinate of the actual gaze point can be used as the output value of the fourth function to determine its parameter values, which then serve as the longitudinal user correction parameters. Finally, the lateral and longitudinal user correction parameters can be used as the user bias corresponding to the candidate user.

[0095] In one possible implementation, the third function can be referred to as Equation (3), and the fourth function can be referred to as Equation (4), where (x,y) represents the x and y coordinates of the corrected gaze point, serving as the input values ​​for the third and fourth functions. The x and y coordinates of the true gaze point are used as the output values ​​of the third and fourth functions. g, h, and i represent the lateral user correction parameters, which are the parameter values ​​of the third function. j, k, and l represent the longitudinal user correction parameters, which are the parameter values ​​of the fourth function.

[0096]

[0097]

[0098] Because multiple sets of fixation point data can be obtained, each set of fixation point data includes the true fixation point (x, y) and the corrected fixation point. Therefore, multiple sets of fixation point data can be substituted into formulas (3) and (4) to obtain the lateral user correction parameters g, h, i and the longitudinal user correction parameters j, k, l. For example, the lateral user correction parameters g, h, i and the longitudinal user correction parameters j, k, l can be obtained by the least squares method. Since g, h, i, j, k, l are obtained from data of the same user, they can be called user bias.

[0099] Step 207: The server stores the system bias corresponding to each device type and the user bias corresponding to each candidate user; wherein, the system bias and user bias are used to correct the estimated gaze point of gaze estimation.

[0100] For example, the server can store the system bias corresponding to each device type and the user bias corresponding to each candidate user in the database. See Table 6 for an example of system bias and user bias.

[0101] Table 6

[0102]

[0103]

[0104] In one possible implementation, the server can correct the estimated gaze point in gaze estimation based on stored system and user biases. For example, the target user device (such as a smartphone, smart screen, AR / VR device in a gaming scenario, or an in-vehicle terminal in an assisted driving scenario) can acquire eye or facial images, estimate the target user's gaze direction based on the eye or facial images, and obtain the coordinates of the gaze point intersecting with that gaze direction, i.e., the estimated gaze point when the target user device estimates the target user's gaze. Then, the target user device can send a correction request to the server, which receives the correction request. The correction request may include the target user's user identifier, the target user device's device type, and the estimated gaze point when the target user device estimates the target user's gaze.

[0105] The server performs system bias correction on the estimated fixation point based on the system bias corresponding to the device type of the target user's device, thus obtaining the corrected estimated fixation point. For example, as shown in formulas (1) and (2), the system bias corresponding to the device type may include lateral system correction parameters a, b, and c and longitudinal system correction parameters d, e, and f. (x, y) represents the abscissa and ordinate of the estimated fixation point. The (x, y) of the estimated fixation point and the lateral system correction parameters a, b, and c can be substituted into formula (1) to obtain the corrected abscissa. The estimated fixation point (x, y) and the longitudinal system correction parameters d, e, and f can be substituted into formula (2) to obtain the corrected ordinate. Obviously, the x-axis and ordinate The corrected estimated fixation point is formed.

[0106] The server performs user bias correction on the estimated fixation point based on the user bias corresponding to the user identifier of the target user, and obtains the corrected target fixation point. For example, as shown in formulas (3) and (4), the user bias corresponding to the user identifier may include horizontal user correction parameters g, h, i and vertical user correction parameters j, k, l. The horizontal and vertical coordinates of the estimated fixation point can be substituted into formula (3) to obtain the horizontal coordinate of the target fixation point, and the horizontal and vertical coordinates of the estimated fixation point can be substituted into formula (4) to obtain the vertical coordinate of the target fixation point. In this way, the corrected target fixation point can be obtained.

[0107] The server can send the corrected target gaze point to the target user device, so that the target user device can obtain an accurate and reliable target gaze point and then perform processing based on the target gaze point.

[0108] In another possible implementation, the target user equipment (User Equipment) can correct the estimated gaze point based on stored system and user biases. For example, the User Equipment can send a data request to the server, which may include the user identifier of the target user and the device type of the User Equipment. The target user is the user using the User Equipment, and there can be at least one target user. The User Equipment can send data requests to the server periodically or on certain triggering conditions, such as a new user using the User Equipment or the User Equipment being idle; there are no restrictions on these triggering conditions.

[0109] After receiving a data request, the server can query the system deviation corresponding to the device type and the user deviation corresponding to the user identifier from the local machine, and send the system deviation and the user deviation to the target user device, which will then store the system deviation corresponding to the device type and the user deviation corresponding to the user identifier.

[0110] The target user device (User Device) can acquire eye or facial images, estimate the user's gaze direction based on these images, and obtain the coordinates of the gaze point intersecting this gaze direction—the estimated gaze point when the User Device estimates the user's gaze. Then, the User Device can query the system bias corresponding to the device type and the user bias corresponding to the user identifier locally. Based on the system bias, the estimated gaze point is corrected to obtain the corrected estimated gaze point. Finally, based on the user bias, the corrected estimated gaze point is corrected to obtain the corrected target gaze point.

[0111] See Figure 3 The diagram illustrates the process of decoupling system bias and user bias for gaze correction. The <User ID, True Gaze Point, Predicted Gaze Point, Device ID> is uploaded to the server. The Device ID can be the device type, and the User ID can be the user identifier. The server clusters this information according to device type, obtaining the <True Gaze Point, Predicted Gaze Point> for different users within the same device type. This data is used to estimate the system bias corresponding to each device type. Then, the information is clustered according to user identifier, obtaining the <True Gaze Point, System Bias Corrected Predicted Gaze Point> for the same user's device or different devices within the same user identifier. The system bias corrected predicted gaze point is the corrected gaze point.

[0112] For example, to decouple system bias from user bias, after obtaining the system bias, the predicted gaze point can be corrected based on the system bias to obtain the corrected predicted gaze point. Thus, when determining user bias, it is based on the corrected predicted gaze point. For instance, after obtaining the <true gaze point and the system bias-corrected predicted gaze point> for the same user device or different devices under the same user identifier, the user bias corresponding to each user identifier can be estimated using this data.

[0113] After obtaining the system bias and user bias, they can be stored in the database. When a new user uses the calibration function, the user's device sends a calibration request. Based on the device type and user ID, the corresponding system bias and user bias are retrieved from the database. Then, the system bias is calibrated first, followed by the user bias calibration, to obtain the final calibration result.

[0114] In one possible implementation, all data involved in this embodiment (such as the actual and predicted gaze points when the user device estimates the user's gaze, the device type of the user device, the user's user identifier, etc.) are only obtained and used with the knowledge and authorization of the relevant user.

[0115] This application proposes a gaze estimation method, which may include: for each device type, obtaining a first dataset corresponding to that device type, the first dataset may include the true gaze points and predicted gaze points when a user device of that device type performs gaze estimation on a user. For each candidate user, obtaining a second dataset corresponding to that candidate user, the second dataset may include the true gaze points and predicted gaze points when a user device performs gaze estimation on that candidate user, and determining the user bias corresponding to that candidate user based on the true gaze points and predicted gaze points in the second dataset. Determining the user bias corresponding to the predicted gaze points in the first dataset, and correcting the predicted gaze points based on the user bias to obtain corrected gaze points; determining the system bias corresponding to that device type based on the true gaze points in the first dataset and the corrected gaze points. Storing the user bias corresponding to each candidate user and the system bias corresponding to each device type; wherein, the user bias and system bias are used to correct the estimated gaze points of gaze estimation.

[0116] See Figure 4 The diagram shown is a flowchart of the line-of-sight estimation method, which may include:

[0117] Step 401: The server obtains sample data, which includes the actual gaze point and predicted gaze point when the user device estimates the user's gaze, the device type of the user device, and the user's user identifier.

[0118] Step 402: For each device type, the server obtains the first dataset corresponding to that device type based on the sample data. The first dataset may include multiple sets of gaze point data. Each set of gaze point data may include the actual gaze point and the predicted gaze point when the device of that device type estimates the user's gaze.

[0119] Step 403: For each candidate user, the server obtains the second dataset corresponding to the candidate user based on the sample data. The second dataset may include multiple sets of gaze point data. Each set of gaze point data may include the actual gaze point and the predicted gaze point when the user device performs gaze estimation for the candidate user.

[0120] Step 404: For each candidate user, the server determines the user bias corresponding to the candidate user based on the true fixation points and predicted fixation points in the second dataset corresponding to the candidate user.

[0121] For example, determining the user bias corresponding to the candidate user based on the real gaze point and the predicted gaze point in the second dataset may include, but is not limited to: determining the lateral user correction parameter based on the x-coordinate of the predicted gaze point, the y-coordinate of the predicted gaze point, and the x-coordinate of the real gaze point; and determining the longitudinal user correction parameter based on the x-coordinate of the predicted gaze point, the y-coordinate of the predicted gaze point, and the y-coordinate of the real gaze point; and determining the user bias based on the lateral user correction parameter and the longitudinal user correction parameter.

[0122] For example, the processing of step 404 is similar to that of step 206, except that the corrected gaze point is replaced with the predicted gaze point. For instance, multiple sets of gaze point data (each set of gaze point data includes the real gaze point and the predicted gaze point) can be substituted into formulas (3) and (4) to obtain the lateral user correction parameters g, h, i and the longitudinal user correction parameters j, k, l, which are the user biases.

[0123] Step 405: For the first dataset corresponding to each device type, the first dataset may include the actual gaze point and the predicted gaze point when the user device of that device type estimates the user's gaze. The server corrects the predicted gaze point based on the user's corresponding user bias to obtain the corrected gaze point.

[0124] For example, correcting the predicted fixation point based on the user's corresponding user bias to obtain a corrected fixation point may include, but is not limited to: if the user bias includes a lateral user correction parameter and a longitudinal user correction parameter, then determining the lateral coordinate of the corrected fixation point based on the lateral coordinate of the predicted fixation point, the longitudinal coordinate of the predicted fixation point, and the lateral user correction parameter; determining the longitudinal coordinate of the corrected fixation point based on the lateral coordinate of the predicted fixation point, the longitudinal user correction parameter, and the longitudinal user correction parameter; and determining the corrected fixation point based on the lateral coordinate of the corrected fixation point and the longitudinal coordinate of the corrected fixation point.

[0125] For example, the processing procedure in step 405 is similar to that in step 205, except that the system bias is replaced by the user bias, that is, the predicted fixation point is corrected based on the user bias, which will not be repeated here.

[0126] Step 406: For each device type, the server determines the system bias corresponding to that device type based on the true gaze points and corrected gaze points in the first dataset corresponding to that device type.

[0127] For example, determining the system bias corresponding to the device type based on the real gaze point and the corrected gaze point in the first dataset may include, but is not limited to: determining the lateral system correction parameter based on the x-coordinate of the corrected gaze point, the y-coordinate of the corrected gaze point, and the x-coordinate of the real gaze point; determining the longitudinal system correction parameter based on the x-coordinate of the corrected gaze point, the y-coordinate of the corrected gaze point, and the y-coordinate of the real gaze point; and determining the system bias based on the lateral system correction parameter and the longitudinal system correction parameter.

[0128] For example, the processing of step 406 is similar to that of step 204, except that the predicted fixation point is replaced with the corrected fixation point. For instance, multiple sets of fixation point data (each set of fixation point data includes the real fixation point and the corrected fixation point) can be substituted into formulas (1) and (2) to obtain the lateral system correction parameters a, b, c and the longitudinal system correction parameters d, e, f, which are the system biases.

[0129] Step 407: The server stores the user bias for each candidate user and the system bias for each device type; wherein, the user bias and system bias are used to correct the estimated gaze point of gaze estimation.

[0130] In one possible implementation, the target user equipment can send a correction request to the server, and the server can receive the correction request sent by the target user equipment. The correction request may include the user identifier of the target user, the device type of the target user equipment, and the estimated gaze point when the target user equipment performs gaze estimation on the target user.

[0131] The server performs user bias correction on the estimated fixation point based on the user bias corresponding to the user identifier of the target user, and obtains the corrected estimated fixation point. For example, as shown in formulas (3) and (4), the user bias corresponding to the user identifier may include horizontal user correction parameters g, h, i and vertical user correction parameters j, k, l. The horizontal and vertical coordinates of the estimated fixation point can be substituted into formula (3) to obtain the corrected horizontal coordinate, and the horizontal and vertical coordinates of the estimated fixation point can be substituted into formula (4) to obtain the corrected vertical coordinate. The corrected horizontal and vertical coordinates constitute the corrected estimated fixation point.

[0132] The server performs system bias correction on the estimated fixation point based on the system bias corresponding to the device type of the target user device, thus obtaining the corrected target fixation point. For example, as shown in formulas (1) and (2), the system bias corresponding to the device type may include lateral system correction parameters a, b, and c and longitudinal system correction parameters d, e, and f. The abscissa and ordinate of the estimated fixation point can be substituted into formula (1) to obtain the abscissa of the target fixation point, and the abscissa and ordinate of the estimated fixation point can be substituted into formula (2) to obtain the ordinate of the target fixation point. In this way, the corrected target fixation point can be obtained.

[0133] The server can send the corrected target gaze point to the target user device, so that the target user device can obtain an accurate and reliable target gaze point and then perform processing based on the target gaze point.

[0134] In another possible implementation, the target user equipment can send a data request to the server. The data request may include the user identifier of the target user and the device type of the target user equipment. After receiving the data request, the server can query the system deviation corresponding to the device type and the user deviation corresponding to the user identifier from its local machine, and send the system deviation and the user deviation to the target user equipment. The target user equipment stores the system deviation corresponding to the device type and the user deviation corresponding to the user identifier.

[0135] The target user equipment (User Equipment) can acquire eye or facial images and estimate the gaze point of the target user based on these images. Then, the User Equipment can query the system bias corresponding to the device type and the user bias corresponding to the user identifier locally. Based on this user bias, the estimated gaze point is corrected to obtain the corrected estimated gaze point. Finally, based on the system bias, the corrected estimated gaze point is corrected to obtain the corrected target gaze point.

[0136] As can be seen from the above technical solutions, in this embodiment, the system deviation corresponding to each device type and the user deviation corresponding to each candidate user can be obtained. Based on the system deviation and user deviation, the estimated gaze point of the gaze estimation is corrected, thereby obtaining accurate and reliable gaze point coordinates and avoiding problems such as large errors and inaccurate positions of the gaze point coordinates. The gaze deviation can be distinguished into system deviation and user deviation. System deviation is the gaze deviation caused by device differences, while user deviation is the gaze deviation caused by user differences. By decoupling the system deviation and user deviation and estimating them separately, the accuracy and flexibility of gaze correction can be improved, and more accurate correction parameters can be solved, thereby improving the correction accuracy.

[0137] Based on the same concept as the methods described above, this application proposes a line-of-sight estimation device, see [link to relevant documentation]. Figure 5 The diagram shown is a structural schematic of the line-of-sight estimation device, which may include:

[0138] The acquisition module 51 is used to acquire a first dataset corresponding to each device type, the first dataset including the real gaze points and predicted gaze points when the user device of the device type performs gaze estimation on the user; and to acquire a second dataset corresponding to each candidate user, the second dataset including the real gaze points and predicted gaze points when the user device performs gaze estimation on the candidate user.

[0139] The determination module 52 is used to determine the system bias corresponding to the device type based on the real fixation points and predicted fixation points in the first dataset; to correct the predicted fixation points based on the system bias corresponding to the device type in the second dataset to obtain corrected fixation points; and to determine the user bias corresponding to the candidate user based on the real fixation points in the second dataset and the corrected fixation points.

[0140] The storage module 53 is used to store the system bias corresponding to each device type and the user bias corresponding to each candidate user. The system bias and user bias are used to correct the estimated gaze point of the gaze estimation.

[0141] For example, when determining the system bias corresponding to the device type based on the actual gaze point and the predicted gaze point in the first dataset, the determining module 52 is specifically used to: determine the lateral system correction parameters based on the abscissa of the predicted gaze point, the ordinate of the predicted gaze point, and the abscissa of the actual gaze point, and determine the longitudinal system correction parameters based on the abscissa of the predicted gaze point, the ordinate of the predicted gaze point, and the ordinate of the actual gaze point; and determine the system bias based on the lateral system correction parameters and the longitudinal system correction parameters.

[0142] For example, when the determining module 52 corrects the predicted fixation point based on the system deviation corresponding to the device type to obtain the corrected fixation point, it is specifically used to: if the system deviation includes a lateral system correction parameter and a longitudinal system correction parameter, then determine the lateral coordinate of the corrected fixation point based on the lateral coordinate of the predicted fixation point, the longitudinal coordinate of the predicted fixation point, and the lateral system correction parameter; determine the longitudinal coordinate of the corrected fixation point based on the lateral coordinate of the predicted fixation point, the longitudinal system correction parameter; and determine the corrected fixation point based on the lateral coordinate of the corrected fixation point and the longitudinal coordinate of the corrected fixation point.

[0143] For example, when determining the user deviation corresponding to the candidate user based on the real gaze point and the corrected gaze point in the second dataset, the determining module 52 is specifically used to: determine the lateral user correction parameters based on the abscissa of the corrected gaze point, the ordinate of the corrected gaze point, and the abscissa of the real gaze point, and determine the longitudinal user correction parameters based on the abscissa of the corrected gaze point, the ordinate of the corrected gaze point, and the ordinate of the real gaze point; and determine the user deviation based on the lateral user correction parameters and the longitudinal user correction parameters.

[0144] For example, the apparatus further includes: a processing module, configured to receive a data request sent by a target user device, the data request including a user identifier of the target user and a device type of the target user device; send the system bias corresponding to the device type and the user bias corresponding to the user identifier to the target user device, so that the target user device performs system bias correction on the estimated fixation point based on the system bias to obtain a corrected estimated fixation point, and performs user bias correction on the corrected estimated fixation point based on the user bias to obtain a corrected target fixation point.

[0145] Based on the same concept as the methods described above, this application proposes a line-of-sight estimation device, see [link to relevant documentation]. Figure 6 The diagram shown is a structural schematic of the line-of-sight estimation device, which may include:

[0146] The acquisition module 61 is used to acquire a first dataset corresponding to each device type, the first dataset including the actual gaze points and predicted gaze points when the user device of the device type performs gaze estimation on the user; and to acquire a second dataset corresponding to each candidate user, the second dataset including the actual gaze points and predicted gaze points when the user device performs gaze estimation on the candidate user.

[0147] The determination module 62 is used to determine the user bias corresponding to the candidate user based on the real fixation points and predicted fixation points in the second dataset; determine the user bias corresponding to the user corresponding to the predicted fixation point in the first dataset; correct the predicted fixation point based on the user bias to obtain the corrected fixation point; and determine the system bias corresponding to the device type based on the real fixation points in the first dataset and the corrected fixation point.

[0148] The storage module 63 is used to store the user bias corresponding to each candidate user and the system bias corresponding to each device type. The user bias and system bias are used to correct the estimated gaze point of the gaze estimation.

[0149] For example, the apparatus further includes: a processing module, configured to receive a data request sent by a target user device, the data request including a user identifier of the target user and a device type of the target user device; send the user bias corresponding to the user identifier and the system bias corresponding to the device type to the target user device, so that the target user device performs user bias correction on the estimated fixation point based on the user bias to obtain a corrected estimated fixation point, and performs system bias correction on the corrected estimated fixation point based on the system bias to obtain a corrected target fixation point.

[0150] Based on the same concept as the above method, this application proposes an electronic device, see [link to previous application]. Figure 7 As shown, the electronic device includes a processor 71 and a machine-readable storage medium 72, the machine-readable storage medium 72 storing machine-executable instructions that can be executed by the processor 71; the processor 71 is used to execute the machine-executable instructions to implement the line-of-sight estimation method disclosed in the above example of this application.

[0151] Based on the same concept as the above method, this application also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the line-of-sight estimation method disclosed in the above examples of this application.

[0152] The aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0153] The systems, devices, modules, or units described in the above embodiments can be implemented by a computer entity or by a product with a certain function. A typical implementation device is a computer, which can be a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0154] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0155] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0156] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0157] Furthermore, these computer program instructions can also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0158] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0159] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A line-of-sight estimation method, characterized in that, The method includes: For each device type, a first dataset corresponding to that device type is obtained. The first dataset includes the actual gaze points and predicted gaze points when the user device of that device type performs gaze estimation on the user. Based on the actual gaze points and predicted gaze points in the first dataset, the system bias corresponding to that device type is determined. For each candidate user, a second dataset corresponding to the candidate user is obtained. The second dataset includes the actual gaze points and predicted gaze points when the user device estimates the gaze of the candidate user. The predicted gaze points are corrected based on the system bias corresponding to the device type of the user device to obtain the corrected gaze points. The user bias corresponding to the candidate user is determined based on the actual gaze points and corrected gaze points in the second dataset. The system bias corresponding to each device type and the user bias corresponding to each candidate user are stored; wherein, the system bias and the user bias are used to correct the estimated gaze point of gaze estimation.

2. The method according to claim 1, characterized in that, The system bias for determining the device type based on the true and predicted fixation points in the first dataset includes: The lateral system correction parameters are determined based on the x-coordinate of the predicted gaze point, the y-coordinate of the predicted gaze point, and the x-coordinate of the actual gaze point; and the longitudinal system correction parameters are determined based on the x-coordinate of the predicted gaze point, the y-coordinate of the predicted gaze point, and the y-coordinate of the actual gaze point. The system deviation is determined based on the lateral system correction parameters and the longitudinal system correction parameters.

3. The method according to claim 1, characterized in that, The method of correcting the predicted gaze point based on the system bias corresponding to the device type of the user equipment to obtain the corrected gaze point includes: If the system bias includes lateral system correction parameters and longitudinal system correction parameters, then the lateral system correction parameters are used to determine the lateral coordinate of the corrected fixation point; and the longitudinal system correction parameters are used to determine the lateral coordinate of the corrected fixation point. The corrected gaze point is determined based on the x-coordinate and y-coordinate of the corrected gaze point.

4. The method according to claim 1, characterized in that, The process of determining the user bias corresponding to the candidate user based on the true fixation points and corrected fixation points in the second dataset includes: The horizontal user correction parameters are determined based on the horizontal coordinate of the corrected gaze point, the vertical coordinate of the corrected gaze point, and the horizontal coordinate of the actual gaze point; and the vertical user correction parameters are determined based on the horizontal coordinate of the corrected gaze point, the vertical coordinate of the corrected gaze point, and the vertical coordinate of the actual gaze point. The user bias is determined based on the lateral user correction parameters and the longitudinal user correction parameters.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: The system receives a data request sent by a target user device, the data request including the user identifier of the target user and the device type of the target user device; it sends the system bias corresponding to the device type and the user bias corresponding to the user identifier to the target user device, so that the target user device performs system bias correction on the estimated fixation point based on the system bias to obtain the corrected estimated fixation point, and performs user bias correction on the corrected estimated fixation point based on the user bias to obtain the corrected target fixation point.

6. A line-of-sight estimation method, characterized in that, The method includes: For each device type, obtain the first dataset corresponding to that device type. The first dataset includes the actual gaze points and predicted gaze points when the user device of that device type performs gaze estimation on the user. For each candidate user, a second dataset corresponding to that candidate user is obtained. The second dataset includes the actual gaze point and the predicted gaze point when the device performs gaze estimation for that candidate user. Based on the actual gaze point and the predicted gaze point in the second dataset, the user bias corresponding to that candidate user is determined. Determine the user bias corresponding to the predicted fixation point in the first dataset, and correct the predicted fixation point based on the user bias to obtain the corrected fixation point; determine the system bias corresponding to the device type based on the true fixation point in the first dataset and the corrected fixation point; The system stores the user bias for each candidate user and the system bias for each device type; wherein the user bias and the system bias are used to correct the estimated gaze point of gaze estimation.

7. The method according to claim 6, characterized in that, The method further includes: The system receives a data request sent by a target user device, the data request including the user identifier of the target user and the device type of the target user device; it sends the user bias corresponding to the user identifier and the system bias corresponding to the device type to the target user device, so that the target user device performs user bias correction on the estimated fixation point based on the user bias to obtain the corrected estimated fixation point, and performs system bias correction on the corrected estimated fixation point based on the system bias to obtain the corrected target fixation point.

8. A line-of-sight estimation device, characterized in that, The device includes: The acquisition module is used to acquire a first dataset corresponding to each device type, which includes the actual gaze points and predicted gaze points when the user device of that device type performs gaze estimation on the user; and to acquire a second dataset corresponding to each candidate user, which includes the actual gaze points and predicted gaze points when the user device performs gaze estimation on the candidate user. The determination module is used to determine the system bias corresponding to the device type based on the real fixation points and predicted fixation points in the first dataset; to correct the predicted fixation points based on the system bias corresponding to the device type in the second dataset to obtain corrected fixation points; and to determine the user bias corresponding to the candidate user based on the real fixation points in the second dataset and the corrected fixation points. The storage module is used to store the system bias corresponding to each device type and the user bias corresponding to each candidate user. The system bias and user bias are used to correct the estimated gaze point of gaze estimation.

9. A line-of-sight estimation device, characterized in that, The device includes: The acquisition module is used to acquire a first dataset corresponding to each device type, which includes the actual gaze points and predicted gaze points when the user device of that device type performs gaze estimation on the user; and to acquire a second dataset corresponding to each candidate user, which includes the actual gaze points and predicted gaze points when the user device performs gaze estimation on the candidate user. The determination module is used to determine the user bias corresponding to the candidate user based on the true fixation points and predicted fixation points in the second dataset; determine the user bias corresponding to the user corresponding to the predicted fixation point in the first dataset; correct the predicted fixation point based on the user bias to obtain the corrected fixation point; and determine the system bias corresponding to the device type based on the true fixation points in the first dataset and the corrected fixation point. The storage module is used to store the user bias corresponding to each candidate user and the system bias corresponding to each device type. The user bias and system bias are used to correct the estimated gaze point of gaze estimation.

10. An electronic device, characterized in that, include: A processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; The processor is configured to execute machine-executable instructions to implement the method of any one of claims 1-7.

Images