Eye tracking method and system

Abstract

The present application relates to an eye movement tracking method and a system. The method comprises: receiving a target interface, eyeball data of a user, and eyeball movement information of the user at different points in time within a target period of time and transmitted by an eye tracking device, and generating line of sight information of the user at each point in time on the basis of the eyeball data of the user and the eyeball movement information at each point in time; identifying a line of sight interaction range corresponding to each piece of line of sight line information on the target interaction interface on the basis of the line of sight information at each point in time, and identifying target interaction information of the user within each line of sight interaction range; sorting the target interaction information according to the chronological order of the points in time to obtain an interaction sequence, and generating target interaction content for the user on the basis of the interaction sequence. By utilizing the present method, the accuracy of identified target interaction content can be improved.

Description

Eye tracking method and system Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an eye tracking method and system. Background Art

[0002] With the development of digital display technology, diversified display terminal devices play a key role in modern communication, entertainment and information dissemination. In order to improve the interaction efficiency between users and display terminals, the technology of identifying user eye movement information and determining the interaction between users and the content displayed on the interactive interface of the display terminal has gradually developed and been put into production. Therefore, how to improve the recognition accuracy of eye movement information is the current research focus.

[0003] Traditional eye tracking technology projects electromagnetic waves into the eyeballs and receives the intensity of the reflected waves, determining the user's eye movements and, therefore, the user's control commands for the interactive interface. However, due to factors such as corneal thickness and pupil divergence, the error rate of identifying eye movements solely through electromagnetic waves is high, resulting in low accuracy in the interaction content corresponding to the identified eye movement information.

[0004] Summary of the Invention

[0005] Based on this, it is necessary to provide an eye tracking method, system, computer device, computer-readable storage medium and computer program product to address the above technical problems.

[0006] In a first aspect, the present application provides an eye tracking method. The method comprises:

[0007] receiving eye movement information of a user at different time points in a target time period, eye data of the user, and a target interactive interface transmitted by an eye tracking device, and generating gaze information of the user at each time point based on the eye movement information at each time point and the eye data of the user;

[0008] Based on the sight line information at each time point, identifying, on the target interaction interface, a sight line interaction range corresponding to each sight line information, and identifying the target interaction information of the user in each sight line interaction range;

[0009] The target interaction information is sorted according to the order of each time point to obtain an interaction sequence, and the target interaction content of the user is generated based on the interaction sequence.

[0010] Optionally, generating the user's sight line information at each time point based on the eye movement information at each time point and the user's eye data includes:

[0011] Based on the user's eye information, identifying the user's eye focus information, and identifying the viewing angle change angle corresponding to each eye movement information, and the viewing angle change rate corresponding to each eye movement information;

[0012] Based on the user's eye focus information, construct the user's visual center point and the user's visual range information, and for each time point, determine the user's eye direction information corresponding to the time point based on the visual angle change angle at the time point and the visual angle change rate at the time point;

[0013] Based on the eyeball direction information of the user, the center point of the user's sight line, and the sight range of the user, the sight line information of the user at the time point is determined.

[0014] Optionally, the identifying, on the target interaction interface, a line of sight interaction range corresponding to each line of sight information based on the line of sight information at each time point includes:

[0015] Obtaining spatial position information of the user's eyeball and spatial position information of the target interaction interface, and calculating the spatial distance between the user's eyeball and the target interaction interface;

[0016] Calculating, based on the user's visual range information, visual range change information corresponding to a unit distance of the user, and calculating, based on the spatial distance, a target visual range of the user;

[0017] Based on the target visual range of the user and the visual information of the user at each time point, a visual interaction range corresponding to each visual information is determined.

[0018] Optionally, determining a sight interaction range corresponding to each sight information based on the target sight range of the user and the sight information of the user at each time point includes:

[0019] For each time point, projecting the sight line information of the time point to the spatial layer of the interaction interface between the user and the target;

[0020] Connecting the sight line information with the spatial position information of the user's eyeball and the eyeball of the target interactive interface respectively to obtain the interactive area of ​​the target sight line range on the target interactive interface;

[0021] In the interaction area, determining a central interaction position point corresponding to the sight line center point, and based on the interaction position point, determining interaction probabilities of different interaction positions in the interaction area according to a preset interaction range determination strategy;

[0022] The central interaction location point and the interaction areas corresponding to all interaction location points containing interaction probabilities are used as the sight line interaction range of the sight line information corresponding to the time point.

[0023] Optionally, identifying target interaction information of the user in each line of sight interaction range includes:

[0024] For the visual interaction range, identifying the interaction content information corresponding to each interaction position point within the visual interaction range, and determining the interaction probability weight of each interaction content information according to the interaction type to which each interaction content information belongs;

[0025] Based on the interaction probability of each interaction position point within the visual interaction range and the interaction probability weight of the interaction content information corresponding to each interaction position point, the target interaction probability of each interaction content information is calculated, and the interaction content information greater than a preset interaction probability threshold is screened as the target interaction information.

[0026] Optionally, generating target interaction content of the user based on the interaction sequence includes:

[0027] Identifying an interactive function of each target interactive content and an interactive result of each target interactive content, and arranging each of the interactive functions and each of the interactive results according to the interaction sequence to obtain an interactive function sequence and an interactive result sequence;

[0028] Analyzing the interaction logic information and interaction trend information of the interaction function sequence, and analyzing the target interaction results corresponding to the interaction result sequence;

[0029] Based on the interaction logic information and the interaction trend information, the interaction target of the user is determined, and based on the target interaction result and the interaction target of the user, the target interaction content of the user is determined.

[0030] Optionally, analyzing the interaction logic information and interaction trend information of the interaction function sequence includes:

[0031] Based on the interactive function sequence, the interactive functions are sorted by distribution to obtain interactive function distribution information, and curvature trend information of the interactive function distribution information is identified to obtain the interactive trend information;

[0032] In the interactive function sequence, based on the sample interaction logic between every two interactive functions in the database, the sub-interaction logic between two adjacent interactive functions is identified, and all the sub-interaction logics are sorted according to the interactive function sequence to obtain a sub-interaction logic sequence;

[0033] Deduplication processing is performed on each of the sub-interaction logics to obtain a target sub-interaction logic sequence, and interaction logic information is obtained based on all the sub-interaction logics and the target sub-interaction logic sequence.

[0034] Optionally, analyzing the target interaction result corresponding to the interaction result sequence includes:

[0035] Identify association information between two adjacent interaction results in the interaction result sequence, and identify progressive information of the association information between each interaction result; and predict a target interaction result of the interaction result sequence based on the progressive information and the last interaction result in the interaction result sequence.

[0036] In a second aspect, the present application further provides an eye tracking system. The system comprises:

[0037] an acquisition module, configured to receive eye movement information of a user at different time points in a target time period, eye data of the user, and a target interactive interface transmitted by an eye tracking device, and generate gaze information of the user at each time point based on the eye movement information and eye data of the user at each time point;

[0038] an identification module for identifying, on the target interaction interface, a sight line interaction range corresponding to each sight line information based on the sight line information at each time point, and identifying target interaction information of the user in each sight line interaction range;

[0039] The generating module is used to sort the target interaction information according to the order of each time point to obtain an interaction sequence, and generate the target interaction content of the user based on the interaction sequence.

[0040] Optionally, the acquisition module is specifically configured to:

[0041] Based on the user's eye information, identifying the user's eye focus information, and identifying the viewing angle change angle corresponding to each eye movement information, and the viewing angle change rate corresponding to each eye movement information;

[0042] Based on the user's eye focus information, construct the user's visual center point and the user's visual range information, and for each time point, determine the user's eye direction information corresponding to the time point based on the visual angle change angle at the time point and the visual angle change rate at the time point;

[0043] Based on the eyeball direction information of the user, the center point of the user's sight line, and the sight range of the user, the sight line information of the user at the time point is determined.

[0044] Optionally, the identification module is specifically configured to:

[0045] Obtaining spatial position information of the user's eyeball and spatial position information of the target interaction interface, and calculating the spatial distance between the user's eyeball and the target interaction interface;

[0046] Calculating, based on the user's visual range information, visual range change information corresponding to a unit distance of the user, and calculating, based on the spatial distance, a target visual range of the user;

[0047] Based on the target visual range of the user and the visual information of the user at each time point, a visual interaction range corresponding to each visual information is determined.

[0048] Optionally, the identification module is specifically configured to:

[0049] For each time point, projecting the sight line information of the time point to the spatial layer of the interaction interface between the user and the target;

[0050] Connecting the sight line information with the spatial position information of the user's eyeball and the eyeball of the target interactive interface respectively to obtain the interactive area of ​​the target sight line range on the target interactive interface;

[0051] In the interaction area, determining a central interaction position point corresponding to the sight line center point, and based on the interaction position point, determining interaction probabilities of different interaction positions in the interaction area according to a preset interaction range determination strategy;

[0052] The central interaction location point and the interaction areas corresponding to all interaction location points containing interaction probabilities are used as the sight line interaction range of the sight line information corresponding to the time point.

[0053] Optionally, the identification module is specifically configured to:

[0054] For the visual interaction range, identifying the interaction content information corresponding to each interaction position point within the visual interaction range, and determining the interaction probability weight of each interaction content information according to the interaction type to which each interaction content information belongs;

[0055] Based on the interaction probability of each interaction position point within the visual interaction range and the interaction probability weight of the interaction content information corresponding to each interaction position point, the target interaction probability of each interaction content information is calculated, and the interaction content information greater than a preset interaction probability threshold is screened as the target interaction information.

[0056] Optionally, the generating module is specifically configured to:

[0057] Identifying an interactive function of each target interactive content and an interactive result of each target interactive content, and arranging each of the interactive functions and each of the interactive results according to the interaction sequence to obtain an interactive function sequence and an interactive result sequence;

[0058] Analyzing the interaction logic information and interaction trend information of the interaction function sequence, and analyzing the target interaction results corresponding to the interaction result sequence;

[0059] Based on the interaction logic information and the interaction trend information, the interaction target of the user is determined, and based on the target interaction result and the interaction target of the user, the target interaction content of the user is determined.

[0060] Optionally, the generating module is specifically configured to:

[0061] Based on the interactive function sequence, the interactive functions are sorted by distribution to obtain interactive function distribution information, and curvature trend information of the interactive function distribution information is identified to obtain the interactive trend information;

[0062] In the interactive function sequence, based on the sample interaction logic between every two interactive functions in the database, the sub-interaction logic between two adjacent interactive functions is identified, and all the sub-interaction logics are sorted according to the interactive function sequence to obtain a sub-interaction logic sequence;

[0063] Deduplication processing is performed on each of the sub-interaction logics to obtain a target sub-interaction logic sequence, and interaction logic information is obtained based on all the sub-interaction logics and the target sub-interaction logic sequence.

[0064] Optionally, the generating module is specifically configured to:

[0065] Identify association information between two adjacent interaction results in the interaction result sequence, and identify progressive information of the association information between each interaction result; and predict a target interaction result of the interaction result sequence based on the progressive information and the last interaction result in the interaction result sequence.

[0066] The above-mentioned eye tracking method, system, computer device, storage medium, and computer program product receive eye movement information of a user at different time points in a target time period, the user's eye data, and a target interaction interface transmitted by an eye tracking device. Based on the eye movement information and the user's eye data at each time point, the method generates the user's gaze information at each time point. Based on the gaze information at each time point, the method identifies the gaze interaction range corresponding to each gaze information on the target interaction interface, and identifies the user's target interaction information within each gaze interaction range. The method sorts the target interaction information according to the order of each time point to obtain an interaction sequence, and generates the user's target interaction content based on the interaction sequence. By receiving the eye movement information and the user's eye data at different time points in the target time period, the method generates the user's gaze information at each time point. While ensuring the ultimate efficiency of eye movement, the method generates the user's gaze information based on the user's eye data, thereby improving the accuracy of the generated gaze information at each time point. The method then determines the user's target interaction information for each gaze interaction range on the target interaction interface based on the gaze information, improving the error tolerance and accuracy of the identified target interaction information. Finally, by sorting the target interaction information and analyzing the user's target interaction content, the accuracy of identifying target interaction information is improved while also improving the accuracy of judging the meaning of gaze, thereby comprehensively improving the accuracy of identifying target interaction content. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] FIG1 is a schematic flow chart of an eye tracking method according to an embodiment;

[0068] FIG2 is a block diagram of an eye tracking system according to an embodiment; DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0070] The eye tracking method provided in the embodiments of the present application can be applied in an application environment corresponding to the eye gaze tracking technology shown in Figure 1. This solution can be applied to a terminal, a server, or a system comprising a terminal and a server, and implemented through interaction between the terminal and the server. The terminal may include, but is not limited to, various personal computers, laptops, tablet computers, etc. The terminal generates the user's gaze information at each time point by receiving eye movement information and the user's eye data at different time points within a target time period. This generates the user's gaze information based on the user's eye data while ensuring ultimate eye movement efficiency, thereby improving the accuracy of the generated gaze information at each time point. The gaze information is then used to determine the target interaction information for each gaze interaction range of the user within the target interaction interface, thereby improving the error tolerance and accuracy of the identified target interaction information. Finally, by sorting each target interaction information and analyzing the user's target interaction content, the accuracy of identifying the target interaction information is improved, while also improving the accuracy of determining the gaze meaning. This comprehensively improves the accuracy of the identified target interaction content.

[0071] In one embodiment, as shown in FIG1 , an eye tracking method is provided, which is described by taking the application of the method to a terminal as an example, including the following steps:

[0072] Step S101: receiving the eye movement information of the user at different time points in the target time period, the user's eye data, and the target interaction interface transmitted by the eye tracking device, and generating the user's line of sight information at each time point based on the eye movement information and the user's eye data at each time point.

[0073] In this embodiment, with user authorization, the terminal presets multiple collection time points for a single time period. Upon reaching the start of the target time period, the terminal collects eye movement information transmitted by the eye tracking device, the user's eye data, and the target interaction interface, according to the time intervals between each collection time point. The eye tracking device is a device that identifies the user's eye data and eye movement data. This device may be, but is not limited to, an eye tracking eye, an eye tracking headset, or an eye tracking instrument. Eye movement information includes information about the angle and rate of eye movement; eye data includes, but is not limited to, corneal thickness, pupil diameter, eye diameter, and eye shape parameters. The target interaction interface is the real-world interface corresponding to the terminal device with which the user wishes to interact, such as a laptop display, projection screen, computer monitor, or mobile phone display. The gaze information at each time point is a three-dimensional line formed by a radial radius centered at the user's eye and radiating from the pupil diameter. The specific process for generating gaze information will be described in detail later.

[0074] Step S102 : Based on the sight line information at each time point, the sight line interaction range corresponding to each sight line information is identified in the target interaction interface, and the target interaction information of the user in each sight line interaction range is identified.

[0075] In this embodiment, based on the gaze information at each point in time, the terminal identifies the gaze interaction range corresponding to each gaze information on the target interaction interface. The terminal then searches for target interaction information within this gaze interaction range, indicating the target interaction information that the user needs to interact with the target interaction interface. This target interaction information represents the interaction content within the target interaction interface. The specific identification process will be described in detail later.

[0076] Step S103 , sorting the target interaction information according to the order of each time point to obtain an interaction sequence, and generating the user's target interaction content based on the interaction sequence.

[0077] In this embodiment, the terminal sorts the target interaction information according to the order of each time point to obtain an interaction sequence, and then generates the user's target interaction content based on the interaction sequence. The target interaction content is the interaction information that the user needs to complete based on the target interaction interface. For example, the target interaction content may include turning on the computer, turning on the air conditioner, moving the chair, typing and generating voice, displaying pictures, etc.

[0078] Based on the above scheme, by receiving the eye movement information at different time points within the target time period and the eye data of the user, the user's line of sight information at each time point is generated, thereby generating the user's line of sight information based on the user's eye data while ensuring the ultimate efficiency of the eye movement, thereby improving the accuracy of the generated line of sight information at each time point. Then, the target interaction information of the user in each line of sight interaction range of the target interaction interface is determined by the line of sight information, thereby improving the fault tolerance rate of the identified target interaction information and the accuracy of the identified target interaction information. Finally, by sorting each target interaction information and analyzing the user's target interaction content, the accuracy of the line of sight meaning judgment is improved while improving the accuracy of the identified target interaction information. Thus, the accuracy of the identified target interaction content is comprehensively improved.

[0079] Optionally, based on the eye movement information at each time point and the user's eye data, the user's line of sight information at each time point is generated, including: based on the user's eye information, identifying the user's eye focus information, and identifying the viewing angle change angle corresponding to each eye movement information, and the viewing angle change rate corresponding to each eye movement information; based on the user's eye focus information, constructing the user's line of sight center point and the user's line of sight range information, and for each time point, determining the user's eye direction information corresponding to the time point based on the viewing angle change angle at the time point and the viewing angle change rate at the time point; determining the user's line of sight information at the time point based on the user's eye direction information, the user's line of sight center point, and the user's line of sight range.

[0080] In this embodiment, the terminal identifies the user's eye focus information based on the user's eye information. Specifically, based on the user's pupil diameter and corneal thickness, the terminal generates the user's eye focus point and the relative positional relationship between the eye focus point and the user's eye. The relative positional relationship between the eye focus point and the user's eye is used as the user's eye focus information. The terminal then identifies the viewing angle change and viewing angle change rate corresponding to each piece of eye deviation information. The angle change range is 360 degrees, with each degree as the unit of angle change. The viewing angle change rate is the average rate of viewing angle change between two acquisition time points. Next, based on the user's eye focus information, the terminal constructs the user's visual center point and the user's visual range information. The visual range information is a radial range with the user's eye as the point and the pupil diameter as the radial radius. Next, for each time point, the terminal determines the user's eye direction information corresponding to that time point based on the viewing angle change and viewing angle change rate at that time point. Finally, the terminal determines the user's sight information at that time point based on the user's eye direction information, the user's sight center point, and the user's sight range.

[0081] Based on the above solution, by determining the eye direction information, the sight center point, and the sight range in the user's sight information, the practicality, comprehensiveness, and accuracy of the generated user's sight information are improved.

[0082] Optionally, based on the line of sight information at each time point, the line of sight interaction range corresponding to each line of sight information is identified in the target interaction interface, including: obtaining the spatial position information of the user's eyeball and the spatial position information of the target interaction interface, and calculating the spatial distance between the user's eyeball and the target interaction interface; based on the user's line of sight range information, calculating the user's line of sight range change information corresponding to the unit distance, and based on the spatial distance, calculating the user's target line of sight range; based on the user's target line of sight range and the user's line of sight information at each time point, determining the line of sight interaction range corresponding to each line of sight information.

[0083] In this embodiment, the terminal obtains the spatial position information of the user's eyeball and the spatial position information of the target interactive interface, and calculates the spatial distance between the user's eyeball and the target interactive interface. The spatial position is three-dimensional position information in a three-dimensional space within a three-dimensional coordinate system established in the geodetic coordinate system. Then, based on the user's visual range information, the terminal calculates the visual range change information corresponding to the user's unit distance. The unit distance is a distance preset in the terminal, for example, 1cm, 2cm, 5cm, etc. The visual range change information corresponding to the unit distance is the width of the change in the diameter of the visual range between the unit distances.

[0084] Specifically, the angle between the user's eye point and the pupil boundary is identified to obtain the radial angle. Based on this radial angle and the distance from the eye point every two units, the length of the other right-angled side with the extension of the line of sight center as the right angle and the radial angle as the diagonal angle is calculated to obtain the radius change width. Then, the radius change width is multiplied by 2 to obtain the diameter change width.

[0085] Then, the terminal calculates the user's target sight range based on the spatial distance. Finally, the terminal determines the sight interaction range corresponding to each sight information based on the user's target sight range and the user's sight information at each time point.

[0086] Based on the above solution, the line of sight interaction range corresponding to each line of sight information is determined through spatial distance and line of sight range change information, which improves the accuracy of the calculated interaction range.

[0087] Optionally, based on the user's target visual range and the user's visual information at each time point, the visual interaction range corresponding to each visual line of sight information is determined, including: for each time point, projecting the visual line information at the time point to the spatial layer of the user and the target interaction interface; connecting the visual line information with the spatial position information of the user's eyeball and the eyeball of the target interaction interface respectively to obtain the interaction area of ​​the target visual line range in the target interaction interface; in the interaction area, determining the central interaction position point corresponding to the visual line center point, and based on the interaction position point, determining the interaction probability of different interaction position points in the interaction area according to a preset interaction range determination strategy; using the central interaction position point and the interaction areas corresponding to all interaction position points containing interaction probabilities as the visual line interaction range of the visual line information corresponding to the time point.

[0088] In this embodiment, for each time point, the terminal projects the gaze information at that time point onto the spatial layer between the user and the target interaction interface. The terminal then connects the gaze information with the spatial position information of the user's eyes and the eyes on the target interaction interface, respectively, to obtain the interaction area of ​​the target gaze range within the target interaction interface. The terminal then determines the central interaction location corresponding to the gaze center point within the interaction area. Based on the interaction location, the terminal determines the interaction probabilities of different interaction locations within the interaction area according to a preset interaction range determination strategy. The preset interaction range determination strategy is based on the interaction area, with the interaction probability change corresponding to each unit proportional range change. For example, if the interaction area is a range with a diameter of 10 cm and the interaction range determination strategy is to reduce the interaction probability by 10% for every 1 cm, the interaction probabilities of each interaction location within the interaction area are: 100% for the central interaction location, 90% for interaction locations within a radius of 0-1 cm, 80% for interaction locations within a radius of 1-2 cm, etc., and 0% for interaction locations within a radius of 9-1 cm.

[0089] Finally, the terminal uses the central interaction location point and the interaction areas corresponding to all interaction locations that contain interaction probabilities as the line of sight interaction range of the line of sight information corresponding to the time point.

[0090] Based on the above solution, the interaction probability of each interaction point in the line of sight interaction range is identified through the line of sight interaction range and the interaction range determination strategy, thereby improving the accuracy of identifying the user's target interaction information.

[0091] Optionally, identifying the user's target interaction information in each line of sight interaction range includes: for the line of sight interaction range, identifying the interaction content information corresponding to each interaction position point within the line of sight interaction range, and determining the interaction probability weight of each interaction content information according to the interaction type to which each interaction content information belongs; calculating the target interaction probability of each interaction content information based on the interaction probability of each interaction position point within the line of sight interaction range and the interaction probability weight of the interaction content information corresponding to each interaction position point, and screening the interaction content information greater than a preset interaction probability threshold as the target interaction information.

[0092] In this embodiment, the terminal identifies the interactive content information corresponding to each interactive location within the visual interaction range and determines an interaction probability weight for each interactive content information based on the interaction type to which it belongs. Interactive content information includes, but is not limited to, images, text, buttons, links, and blank spaces, and interaction types include, but are not limited to, browsing, clicking, reading, resizing, and jumping. The terminal presets an interaction probability weight for each interaction type and then determines an interaction probability weight for each interactive content information based on the interaction type corresponding to each interactive content information. For example, the interaction probability weight for browsing is 20%, the interaction probability weight for blank spaces is 0%, the interaction probability weight for reading is 50%, and the interaction probability weight for clicking is 80%. The terminal then multiplies the interaction probability for each interactive location within the visual interaction range by the interaction probability weight for the interactive content information corresponding to each interactive location to obtain a target interaction probability for each interactive content information. Finally, the terminal presets an interaction probability threshold and selects interactive content information with a probability greater than the preset interaction probability threshold as target interaction information.

[0093] Based on the above solution, the interaction probability of the interaction content information of each interaction location point is adjusted by the interaction probability weight of each interaction content information, thereby improving the accuracy of screening target interaction information and increasing the fault tolerance rate of screening target interaction information.

[0094] Optionally, based on the interaction sequence, the user's target interaction content is generated, including: identifying the interaction function of each target interaction content and the interaction result of each target interaction content, and arranging and processing each interaction function and each interaction result according to the interaction sequence to obtain an interaction function sequence and an interaction result sequence; analyzing the interaction logic information and interaction trend information of the interaction function sequence, and analyzing the target interaction results corresponding to the interaction result sequence; determining the user's interaction goal based on the interaction logic information and interaction trend information, and determining the user's target interaction content based on the target interaction result and the user's interaction goal.

[0095] In this embodiment, the terminal identifies the interactive function for each target interactive content and the interaction result for each target interactive content. The interactive function is the functional information corresponding to the target content. For example, the interactive function corresponding to the close button is closing the page, and the interactive function corresponding to the input information content is inputting information. The interaction result is the result corresponding to each interactive function. For example, the interaction result corresponding to closing the page is ending the display of the current page, and the interaction result corresponding to the input information is obtaining the information entered by the user. The terminal then arranges each interactive function and each interaction result according to the interaction sequence, obtaining an interactive function sequence and an interaction result sequence. The terminal analyzes the interaction logic information and interaction trend information of the interactive function sequence and analyzes the target interaction result corresponding to the interaction result sequence. The specific analysis process will be described in detail later. Next, based on the interaction logic information and interaction trend information, the terminal determines the user's interaction goal. This interaction goal is the user's purpose for interacting with the target interactive interface. For example, playing a TV series, replying to a message, playing an audio message, etc. Finally, based on the target interaction results and the user's interaction goal, the terminal determines the user's target interactive content. The target interaction content is a set of interaction content and interaction sequence that the user needs to interact with in the current target interaction interface.

[0096] Based on the above solution, the target interaction content is analyzed through the interaction sequence, which improves the recognition accuracy of the meaning of eye contact.

[0097] Optionally, the interaction logic information and interaction trend information of the interaction function sequence are analyzed, including: based on the interaction function sequence, sorting the interaction functions by distribution to obtain interaction function distribution information, and identifying the curvature trend information of the interaction function distribution information to obtain interaction trend information; in the interaction function sequence, based on the sample interaction logic between every two interaction functions in the database, identifying the sub-interaction logic between two adjacent interaction functions, and sorting all the sub-interaction logics according to the interaction function sequence to obtain a sub-interaction logic sequence; deduplicating each sub-interaction logic to obtain a target sub-interaction logic sequence, and obtaining interaction logic information based on all the sub-interaction logics and the target sub-interaction logic sequence.

[0098] In this embodiment, the terminal sorts the interaction functions based on the interaction function sequence to obtain interaction function distribution information, and identifies the curvature trend of the interaction function distribution information to obtain interaction trend information. Then, within the interaction function sequence, the terminal identifies the sub-interaction logic between two adjacent interaction functions based on the sample interaction logic between each two interaction functions in the database, and sorts all the sub-interaction logics according to the interaction function sequence to obtain a sub-interaction logic sequence. Finally, the terminal deduplicates each sub-interaction logic to obtain a target sub-interaction logic sequence, and based on all the sub-interaction logics and the target sub-interaction logic sequence, obtains interaction logic information. This interaction logic information is used to characterize the user's interaction intent with the target interaction interface.

[0099] Based on the above solution, the interactive logic information is identified through the interactive function sequence, thereby improving the accuracy of identifying the interactive logic information.

[0100] Optionally, analyzing the target interaction result corresponding to the interaction result sequence includes: identifying the association information between two adjacent interaction results in the interaction result sequence, and identifying the progressive information of the association information between each interaction result, and predicting the target interaction result of the interaction result sequence based on the progressive information and the last interaction result in the interaction result sequence.

[0101] In this embodiment, the terminal identifies the association information between two adjacent interaction results in the interaction result sequence and identifies the progressive information associated with the association information between each interaction result. The progressive information is the progressive interaction content following the Lian Ge interaction result. For example, the progressive information between opening a transaction interface and entering information is the transaction amount being entered, and the progressive information between opening a browser and selecting a news link is the news content being browsed. Finally, based on the progressive information and the last interaction result in the interaction result sequence, the terminal predicts the target interaction result of the interaction result sequence.

[0102] Based on the above solution, by identifying the correlation between two interaction results, the progressive information is determined and the target interaction result is obtained, thereby improving the accuracy of identifying the target interaction result.

[0103] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0104] Based on the same inventive concept, embodiments of the present application also provide an eye-tracking system for implementing the aforementioned eye-tracking method. The solution provided by this system is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more eye-tracking system embodiments provided below can be found in the above-described limitations of the eye-tracking method and will not be further elaborated here.

[0105] In one embodiment, as shown in FIG2 , an eye tracking system is provided, including: an acquisition module 210 , a recognition module 220 , and a generation module 230 , wherein:

[0106] An acquisition module 210 is configured to receive eye movement information of a user at different time points in a target time period, eye data of the user, and a target interactive interface transmitted by an eye tracking device, and generate gaze information of the user at each time point based on the eye movement information and eye data of the user at each time point;

[0107] The identification module 220 is used to identify the sight line interaction range corresponding to each sight line information on the target interaction interface based on the sight line information at each time point, and identify the target interaction information of the user in each sight line interaction range;

[0108] The generating module 230 is configured to sort the target interaction information according to the order of each time point to obtain an interaction sequence, and generate the target interaction content of the user based on the interaction sequence.

[0109] Optionally, the acquisition module 210 is specifically configured to:

[0110] Based on the user's eye information, identifying the user's eye focus information, and identifying the viewing angle change angle corresponding to each eye movement information, and the viewing angle change rate corresponding to each eye movement information;

[0111] Based on the user's eye focus information, construct the user's visual center point and the user's visual range information, and for each time point, determine the user's eye direction information corresponding to the time point based on the visual angle change angle at the time point and the visual angle change rate at the time point;

[0112] Based on the eyeball direction information of the user, the center point of the user's sight line, and the sight range of the user, the sight line information of the user at the time point is determined.

[0113] Optionally, the identification module 220 is specifically configured to:

[0114] Obtaining spatial position information of the user's eyeball and spatial position information of the target interaction interface, and calculating the spatial distance between the user's eyeball and the target interaction interface;

[0115] Calculating, based on the user's visual range information, visual range change information corresponding to a unit distance of the user, and calculating, based on the spatial distance, a target visual range of the user;

[0116] Based on the target visual range of the user and the visual information of the user at each time point, a visual interaction range corresponding to each visual information is determined.

[0117] Optionally, the identification module 220 is specifically configured to:

[0118] For each time point, projecting the sight line information of the time point to the spatial layer of the interaction interface between the user and the target;

[0119] Connecting the sight line information with the spatial position information of the user's eyeball and the eyeball of the target interactive interface respectively to obtain the interactive area of ​​the target sight line range on the target interactive interface;

[0120] In the interaction area, determining a central interaction position point corresponding to the sight line center point, and based on the interaction position point, determining interaction probabilities of different interaction positions in the interaction area according to a preset interaction range determination strategy;

[0121] The central interaction location point and the interaction areas corresponding to all interaction location points containing interaction probabilities are used as the sight line interaction range of the sight line information corresponding to the time point.

[0122] Optionally, the identification module 220 is specifically configured to:

[0123] For the visual interaction range, identifying the interaction content information corresponding to each interaction position point within the visual interaction range, and determining the interaction probability weight of each interaction content information according to the interaction type to which each interaction content information belongs;

[0124] Based on the interaction probability of each interaction position point within the visual interaction range and the interaction probability weight of the interaction content information corresponding to each interaction position point, the target interaction probability of each interaction content information is calculated, and the interaction content information greater than a preset interaction probability threshold is screened as the target interaction information.

[0125] Optionally, the generating module 230 is specifically configured to:

[0126] Identifying an interactive function of each target interactive content and an interactive result of each target interactive content, and arranging each of the interactive functions and each of the interactive results according to the interaction sequence to obtain an interactive function sequence and an interactive result sequence;

[0127] Analyzing the interaction logic information and interaction trend information of the interaction function sequence, and analyzing the target interaction results corresponding to the interaction result sequence;

[0128] Based on the interaction logic information and the interaction trend information, the interaction target of the user is determined, and based on the target interaction result and the interaction target of the user, the target interaction content of the user is determined.

[0129] Optionally, the generating module 230 is specifically configured to:

[0130] Based on the interactive function sequence, the interactive functions are sorted by distribution to obtain interactive function distribution information, and curvature trend information of the interactive function distribution information is identified to obtain the interactive trend information;

[0131] In the interactive function sequence, based on the sample interaction logic between every two interactive functions in the database, the sub-interaction logic between two adjacent interactive functions is identified, and all the sub-interaction logics are sorted according to the interactive function sequence to obtain a sub-interaction logic sequence;

[0132] Deduplication processing is performed on each of the sub-interaction logics to obtain a target sub-interaction logic sequence, and interaction logic information is obtained based on all the sub-interaction logics and the target sub-interaction logic sequence.

[0133] Optionally, the generating module 230 is specifically configured to:

[0134] Identify association information between two adjacent interaction results in the interaction result sequence, and identify progressive information of the association information between each interaction result; and predict a target interaction result of the interaction result sequence based on the progressive information and the last interaction result in the interaction result sequence.

[0135] Each module in the above-mentioned eye tracking system can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0136] In one embodiment, a computer device is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the content described in any one of the methods in the first aspect is implemented.

[0137] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the content described in any one of the methods in the first aspect is implemented.

[0138] In one embodiment, a computer program product is provided, comprising a computer program, which implements the contents of any one of the methods in the first aspect when executed by a processor.

[0139] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0140] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0141] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. An eye tracking method, characterized in that: The method comprises: Receive eye movement information of a user at different time points in a target time period, eye data of the user, and a target interactive interface transmitted by an eye tracking device, and generate sight line information of the user at each time point based on the eye movement information at each time point and the eye data of the user; Based on the sight line information at each time point, in the target interaction interface, identifying the sight line interaction range corresponding to each sight line information, and identifying the target interaction information of the user in each sight line interaction range; The target interaction information is sorted according to the sequence of each time point to obtain an interaction sequence, and the target interaction content of the user is generated based on the interaction sequence.

2. The method according to claim 1, characterized in that The generating the sight line information of the user at each time point based on the eye movement information at each time point and the eye data of the user includes: Based on the eye information of the user, identifying the eye focus information of the user, and identifying the viewing angle change angle corresponding to each eye deviation information, and the viewing angle change rate corresponding to each eye deviation information; Based on the eye focus information of the user, construct the center point of the user's sight line and the user's sight range information, and for each time point, determine the user's eye direction information corresponding to the time point based on the view angle change angle at the time point and the view angle change rate at the time point; Based on the eye direction information of the user, the center point of the user's line of sight, and the line of sight range of the user, the line of sight information of the user at the time point is determined.

3. The method according to claim 2, characterized in that The step of identifying the sight line interaction range corresponding to each sight line information on the target interaction interface based on the sight line information at each time point includes: Acquire the spatial position information of the user's eyeball and the spatial position information of the target interaction interface, and calculate the spatial distance between the user's eyeball and the target interaction interface; Based on the visual range information of the user, calculating the visual range change information corresponding to the unit distance of the user, and based on the spatial distance, calculating the target visual range of the user; Based on the target sight range of the user and the sight information of the user at each time point, a sight interaction range corresponding to each sight information is determined.

4. The method according to claim 3, characterized in that The determining, based on the target sight range of the user and the sight information of the user at each time point, a sight interaction range corresponding to each sight information includes: For each time point, projecting the sight line information of the time point to the spatial layer of the interaction interface between the user and the target; Connecting the sight line information with the spatial position information of the user's eyeball and the eyeball of the target interactive interface respectively, to obtain the interactive area of ​​the target sight line range on the target interactive interface; In the interaction area, determining a central interaction position point corresponding to the sight line center point, and based on the interaction position point, determining interaction probabilities of different interaction position points in the interaction area according to a preset interaction range determination strategy; The central interaction position point and the interaction areas corresponding to all the interaction position points containing interaction probabilities are used as the sight line interaction range of the sight line information corresponding to the time point.

5. The method according to claim 4, characterized in that The identifying target interaction information of the user in each line of sight interaction range includes: For the sight line interaction range, identifying the interaction content information corresponding to each interaction position point within the sight line interaction range, and determining the interaction probability weight of each interaction content information according to the interaction type to which each interaction content information belongs; Based on the interaction probability of each interaction position point within the line of sight interaction range and the interaction probability weight of the interaction content information corresponding to each interaction position point, the target interaction probability of each interaction content information is calculated, and the interaction content information greater than a preset interaction probability threshold is screened as the target interaction information.

6. The method according to claim 1, characterized in that The generating target interaction content of the user based on the interaction sequence includes: Identify the interactive function of each target interactive content and the interactive result of each target interactive content, and arrange the interactive functions and the interactive results according to the interactive sequence to obtain an interactive function sequence and an interactive result sequence; Analyzing the interaction logic information and interaction trend information of the interaction function sequence, and analyzing the target interaction results corresponding to the interaction result sequence; Based on the interaction logic information and the interaction trend information, the interaction target of the user is determined, and based on the target interaction result and the interaction target of the user, the target interaction content of the user is determined. Allow.

7. The method according to claim 6, characterized in that The analyzing the interaction logic information and interaction trend information of the interaction function sequence includes: Based on the interactive function sequence, the interactive functions are sorted for distribution to obtain interactive function distribution information, and curvature trend information of the interactive function distribution information is identified to obtain the interactive trend information; In the interactive function sequence, based on the sample interactive logic between every two interactive functions in the database, the sub-interaction logic between two adjacent interactive functions is identified, and all the sub-interaction logics are sorted according to the interactive function sequence to obtain a sub-interaction logic sequence; Deduplication processing is performed on each of the sub-interaction logics to obtain a target sub-interaction logic sequence, and interaction logic information is obtained based on all the sub-interaction logics and the target sub-interaction logic sequence.

8. The method according to claim 6, characterized in that The analyzing the target interaction result corresponding to the interaction result sequence includes: Identify association information between two adjacent interaction results in the interaction result sequence, and identify progressive information of the association information between the interaction results, and predict a target interaction result of the interaction result sequence based on the progressive information and the last interaction result in the interaction result sequence.

9. An eye tracking system, characterized in that: The system comprises: an acquisition module, configured to receive eye movement information of a user at different time points in a target time period, eye data of the user, and a target interaction interface transmitted by an eye tracking device, and generate sight line information of the user at each time point based on the eye movement information at each time point and the eye data of the user; an identification module, configured to identify, based on the line of sight information at each time point, a line of sight interaction range corresponding to each line of sight information in the target interaction interface, and identify target interaction information of the user in each line of sight interaction range; The generating module is used to sort the target interaction information according to the sequence of each time point to obtain an interaction sequence, and generate the target interaction content of the user based on the interaction sequence.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images