A method and device for detecting gaze point of human eye movement

Abstract

The present invention discloses a method and device for detecting eye gaze points during eye movement, and relates to the field of eye movement interaction technology. It solves the problem that the existing methods cannot effectively detect the eye gaze state during eye movement. The present invention can effectively determine when the eye is in the eye gaze state during eye movement and also calculate the corresponding gaze point. The main technical scheme of the present invention is: according to a preset time interval, the intersection points of the eye movement corresponding line of sight and the screen and their corresponding pixel coordinates are continuously obtained during the process of the eye watching the screen; when the number of intersection points continuously obtained statistically reaches a preset threshold, the intersection points are used to form an intersection point set, and each intersection point and its corresponding pixel coordinates are stored in the intersection point set; it is determined whether the divergence value of the corresponding discreteness of the intersection point set is greater than the preset divergence threshold; if so, it is determined that the eye movement is in the gaze state and the corresponding gaze point and its corresponding pixel coordinates are calculated according to the intersection point set. The present invention is applied to detecting the eye state.

Description

Technical Field

[0001] The present invention relates to the field of eye movement interaction technology, and in particular to a method and device for detecting a gaze point of a human eye movement. Background Art

[0002] With the continuous innovation and development of science and technology, eye movement interaction products are favored by more and more people. Eye movement interaction is actually to record and identify the movement of the human eye through the device, and use specific eye movements as input signals to control specific task instructions.

[0003] At present, in the process of human eye movement to watch the screen, the main way to achieve how to accurately track the movement of the human eye is through obtaining eye movement information, modeling and processing, and training to simulate eye movement. That is, tracking the point where the line of sight on the screen is caused by the movement of the human eye.

[0004] However, depending on the actual psychological activities of people, the eye movement state of people is also diverse. For example, when watching the screen, the user may not be interested in some of the content displayed on the screen, or even not really read or check the screen range within the sight, but just glance at it (i.e., the eye scanning state); if they encounter content of interest, they will stare at it or check it repeatedly (i.e., the eye fixation state), etc. By analyzing the eye state, especially the fixation state, it is helpful to analyze the relationship between the eye movement and the actual content that the user cares about, and then it is more helpful to assist in the research and development of eye movement equipment performance improvement. However, according to the existing eye tracking method, it can only accurately track the point of view of the eye movement on the screen, and it is impossible to detect when the user is in the eye fixation state. Summary of the invention

[0005] In view of this, the present invention provides a method and device for detecting the gaze point of human eye movement. The main purpose is to solve the problem that the existing methods cannot effectively detect the gaze state when the human eye moves. The present invention optimizes the operation of tracking the human eye movement, can effectively determine when the human eye is in the eye gaze state during movement and also calculate the corresponding gaze point.

[0006] In order to solve the above-mentioned purpose, the present invention mainly provides the following technical solutions:

[0007] The first aspect of the present application provides a method for detecting a gaze point of a human eye movement, the method comprising:

[0008] At preset time intervals, continuously obtaining the intersection of the line of sight corresponding to the movement of the human eye and the screen and its corresponding pixel coordinates while the human eye is viewing the screen;

[0009] When the number of the intersection points continuously obtained by counting reaches a preset threshold, the intersection points are used to form an intersection point set, and the intersection point set stores each intersection point and its corresponding pixel coordinates;

[0010] Determine whether the divergence value of the discreteness corresponding to the intersection set is greater than a preset divergence threshold;

[0011] If so, it is determined that the human eye movement is in a fixation state and the corresponding fixation point and its corresponding pixel coordinates are calculated according to the intersection point set.

[0012] In some modified implementations of the first aspect of the present application, the method further includes:

[0013] determining the fixation point as a target intersection point on the screen;

[0014] Selecting an area within a preset pixel range from the target intersection on the screen as the target area;

[0015] A feature rendering operation is performed on the target area and a display effect is output.

[0016] In some modified implementations of the first aspect of the present application, the continuously acquiring the intersection of the line of sight corresponding to the movement of the human eye and the screen and the corresponding pixel coordinates thereof during the process of the human eye viewing the screen includes:

[0017] According to the time sequence of obtaining the intersection points of the sight line corresponding to the eye movement and the screen, the intersection points are stored one by one in a preset buffer;

[0018] According to the time sequence in which the intersection points are stored in the preset buffer, the intersection points are sorted to obtain an intersection point sequence.

[0019] In some modified implementations of the first aspect of the present application, when the number of intersections obtained by counting reaches a preset threshold, using the intersections to form an intersection set includes:

[0020] Setting a sliding window, wherein the sliding window length is preset and is used to circle the intersection points of the preset threshold number;

[0021] The first intersection point stored in the preset buffer is taken as the first intersection point, and the preset threshold number of target intersection points are circled in the intersection point sequence by using the sliding window;

[0022] An intersection point set is formed using the target intersection points.

[0023] In some modified implementations of the first aspect of the present application, if it is determined that the divergence value of the discreteness corresponding to the intersection set is not greater than a preset divergence threshold, the method further includes:

[0024] Triggering the sliding window to move rightward by a preset number of intersection positions to redetermine the first intersection point;

[0025] Reselecting the preset threshold number of target intersection points in the intersection point sequence in the preset buffer zone using the re-determined first intersection point;

[0026] Forming a new set of intersection points according to the re-selected target intersection points;

[0027] Determine whether the divergence value of the discreteness corresponding to the new intersection set is greater than a preset divergence threshold;

[0028] If not, the operations of triggering the sliding window to move, reselecting the target intersection, and judging whether the divergence value of the discreteness corresponding to the new intersection set composed of the target intersection is greater than the preset divergence threshold are repeatedly executed until it is judged that the divergence value is greater than the preset threshold, then it is determined that the human eye is in a gaze state and the corresponding gaze point and its corresponding pixel coordinates are calculated according to the new intersection set.

[0029] A second aspect of the present application provides a device for detecting a gaze point of a human eye movement, the device comprising:

[0030] An acquisition unit, used for continuously acquiring the intersection of the line of sight corresponding to the movement of the human eye and the screen and the corresponding pixel coordinates thereof at a preset time interval while the human eye is viewing the screen;

[0031] A forming unit, configured to form an intersection point set using the intersection points when the number of the intersection points continuously obtained by the obtaining unit reaches a preset threshold, wherein the intersection point set stores each intersection point and its corresponding pixel coordinates;

[0032] A judging unit, used to judge whether a divergence value of a discreteness corresponding to a set of intersection points formed by the constituent units is greater than a preset divergence threshold;

[0033] A determination unit, configured to determine that the human eye movement is in a fixation state when the determination unit determines that the divergence value of the discreteness corresponding to the intersection set is greater than a preset divergence threshold;

[0034] The calculation unit is used to calculate the corresponding gaze point and its corresponding pixel coordinates according to the intersection point set after the determination unit determines that the human eye movement is in a gaze state.

[0035] In some modified implementations of the second aspect of the present application, the device further includes:

[0036] The determining unit is further used to determine the gaze point as a target intersection point on the screen;

[0037] A selection unit, configured to select, on the screen, an area within a preset pixel range from the target intersection point determined by the determination unit as a target area;

[0038] A rendering unit, used for performing a feature rendering operation on the target area selected by the selection unit;

[0039] The output unit is used to output the display effect obtained by the rendering unit performing the rendering operation.

[0040] In some modified implementations of the second aspect of the present application, the acquisition unit includes:

[0041] A storage module, used for storing the intersection points one by one in a preset buffer according to the time sequence of obtaining the intersection points between the line of sight corresponding to the movement of the human eye and the screen;

[0042] The sorting module is used to sort the intersections according to the time sequence in which the intersections stored by the storage module enter the preset buffer to obtain an intersection sequence.

[0043] In some modified implementations of the second aspect of the present application, the constituent unit includes:

[0044] A setting module, used for setting a sliding window, wherein the sliding window has a preset length and is used for encircling the intersection points of the preset threshold number;

[0045] A selection module, configured to use the first intersection point stored in the preset buffer as the first intersection point, and to select the preset threshold number of target intersection points in the intersection sequence using the sliding window set by the setting module;

[0046] A composition module is used to form an intersection set using the target intersections circled by the selection module.

[0047] In some modified implementations of the second aspect of the present application, the device further includes:

[0048] a triggering unit, configured to trigger the sliding window to move rightward by a preset number of intersection positions to re-determine the first intersection point when the judging unit judges that the divergence value of the discreteness corresponding to the intersection point set is not greater than a preset divergence threshold;

[0049] A selection unit, configured to reselect the preset threshold number of target intersection points in the intersection point sequence in the preset buffer zone by using the re-determined first intersection point;

[0050] a forming unit, configured to form a new set of intersection points according to the target intersection points re-selected by the selecting unit;

[0051] The judging unit is further used to judge whether the divergence value of the discreteness corresponding to the new intersection set formed by the forming unit is greater than a preset divergence threshold;

[0052] An execution unit is used to repeatedly trigger the movement of the sliding window, reselect the target intersection, and judge whether the divergence value of the discreteness corresponding to the new intersection set composed of the target intersection is greater than the preset divergence threshold if the judgment unit judges that the divergence value of the discreteness corresponding to the new intersection set is not greater than the preset divergence threshold, until if it is judged that the divergence value is greater than the preset threshold, it is determined that the human eye is in a gaze state and the corresponding gaze point and its corresponding pixel coordinates are calculated according to the new intersection set.

[0053] A third aspect of the present application provides a storage medium, which includes a stored program, wherein when the program is executed, the device where the storage medium is located is controlled to execute the above-mentioned method for detecting the gaze point of human eye movement.

[0054] A fourth aspect of the present application provides an electronic device, the device comprising at least one processor, and at least one memory and a bus connected to the processor;

[0055] Wherein, the processor and the memory communicate with each other through the bus;

[0056] The processor is used to call the program instructions in the memory to execute the above-mentioned method for detecting the gaze point of human eye movement.

[0057] By means of the above technical solution, the technical solution provided by the present invention has at least the following advantages:

[0058] The present invention provides a method and device for detecting eye gaze points of human eye movement. The present invention periodically and continuously obtains the intersection points and pixel coordinates of the eye gaze lines of human eye movement on the screen. When the statistics reach a preset threshold number, the intersection points counted are used to form an intersection set, and it is determined whether the divergence value of the discreteness corresponding to the intersection set is greater than the preset divergence threshold. If so, it is determined that the human eye is in a gaze state and the corresponding gaze point and its corresponding pixel coordinates are calculated according to the intersection set. Compared with the prior art, the problem that the prior method cannot effectively detect the eye gaze state when the human eye moves is solved. The present invention optimizes the operation of tracking the eye movement, can effectively determine when the human eye is in an eye gaze state when it moves, and also calculates the corresponding gaze point. The gaze point is equivalent to representing a scattered point in the intersection set, and then according to the gaze point, it is helpful to know that the surrounding area is the content that the user is interested in or concerned about, and it is also convenient to analyze the relationship between the eye movement and the content that the user is interested in or concerned about.

[0059] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0061] Figure 1 A flow chart of a method for detecting a gaze point of a human eye movement provided by an embodiment of the present invention;

[0062] Figure 2 A flow chart of another method for detecting gaze points of human eye movement provided by an embodiment of the present invention;

[0063] Figure 3 The embodiment of the present invention exemplifies the use of a sliding window to circle a sequence of intersection points in a preset buffer zone;

[0064] Figure 4 The screen exemplified in the embodiment of the present invention shows the display effects of the scanning state and the fixation state;

[0065] Figure 5 A block diagram of a device for detecting eye movement fixation points provided by an embodiment of the present invention;

[0066] Figure 6 A block diagram of another device for detecting eye movement gaze points provided by an embodiment of the present invention;

[0067] Figure 7 An electronic device for detecting gaze points of human eye movements provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0069] The embodiment of the present invention provides a method for detecting a gaze point of a human eye movement. Figure 1 As shown, the method is to continuously obtain the intersection points of the line of sight corresponding to the human eye movement and the screen and the corresponding pixel coordinates when the human eye is viewing the screen, and form an intersection point set by the continuously obtained intersection points. By judging whether the divergence value of the discreteness of the intersection point set is greater than a preset divergence threshold, it is judged whether the user's eyes are in a gaze state. For this, the embodiment of the present invention provides the following specific steps:

[0070] 101. According to a preset time interval, the intersection point between the line of sight corresponding to the movement of the human eye and the screen and the corresponding pixel coordinates are continuously obtained while the human eye is viewing the screen.

[0071] In the embodiment of the present invention, when the human eye is viewing the screen, the eye tracking technology can be mainly used to obtain the point where the human eye's line of sight falls on the screen in real time, that is, the pixel coordinates of the intersection of the human eye's line of sight and the screen. Specifically, in the application, the time period for obtaining the intersection can be pre-set, and then according to the time period, the intersection of the human eye's line of sight and the screen, as well as the pixel coordinates of each intersection on the screen, are continuously obtained.

[0072] 102. When the number of intersection points continuously obtained by counting reaches a preset threshold, the intersection points are used to form an intersection point set, and each intersection point and its corresponding pixel coordinates are stored in the intersection point set.

[0073] It should be noted that merely obtaining an intersection is not sufficient to determine the user's eye status, so in an embodiment of the present invention, a threshold number is set in advance. When the number of intersections obtained statistically reaches the preset threshold, these intersections are used to form an intersection set. In the intersection set, there is a time difference between the earliest intersection obtained and the latest intersection obtained in chronological order, and the time difference represents the time the user views the screen.

[0074] 103. Determine whether the divergence value of the discreteness corresponding to the intersection point set is greater than a preset divergence threshold.

[0075] In an embodiment of the present invention, the intersection points included in the intersection set are the intersection points between the line of sight of the human eye and the screen. In fact, these intersection points are discrete points on the screen. The divergence value can be pre-set to measure the discreteness corresponding to these discrete points. If the discreteness is judged to be low, it means that the intersection density in the intersection set is relatively high, which means that: during the time when the user watches the screen corresponding to the intersection set, the user's line of sight of the human eye repeatedly falls into the screen area corresponding to the intersection set during the time when the user watches the screen, so the user is interested in or pays more attention to it. During this time period, the user is in a state of eye gaze, rather than scanning with the eye movement.

[0076] 104. If the divergence value of the discreteness corresponding to the intersection point set is determined to be greater than a preset divergence threshold, it is determined that the human eye movement is in a fixation state and the corresponding fixation point and its corresponding pixel coordinates are calculated according to the intersection point set.

[0077] In the embodiment of the present invention, the fixation point is equivalent to representing the intersection set. By using this fixation point, it can be known that during the screen viewing time corresponding to the intersection set, the content that the user is interested in or pays attention to is the area around the fixation point.

[0078] The embodiment of the present invention provides a method for detecting the gaze point of human eye movement. The embodiment of the present invention periodically and continuously obtains the intersection points and pixel coordinates of the human eye movement line of sight on the screen. When the statistics reach a preset threshold number, the intersection points counted are used to form an intersection set, and it is determined whether the divergence value of the discreteness corresponding to the intersection set is greater than the preset divergence threshold. If so, it is determined that the human eye movement is in a gaze state and the corresponding gaze point and its corresponding pixel coordinates are calculated based on the intersection set. Compared with the prior art, the problem that the prior method cannot effectively detect the gaze state when the human eye moves is solved. The embodiment of the present invention optimizes the operation of tracking the human eye movement, and can effectively determine when the human eye is in a gaze state when the human eye moves and also calculate the corresponding gaze point. The gaze point is equivalent to representing a scattered point in the intersection set, and then according to the gaze point, it is helpful to know that the surrounding area is the content that the user is interested in or concerned about, and it is also convenient to analyze the relationship between the human eye movement and the content that the user is interested in or concerned about.

[0079] In order to explain the above embodiment in more detail, the embodiment of the present invention also provides another method for detecting the gaze point of human eye movement, such as Figure 2 As shown, this method is a detailed statement and supplementary statement of the method in the above embodiment, and this embodiment of the present invention provides the following specific steps:

[0080] 201. According to a preset time interval, continuously obtain the intersection point between the line of sight corresponding to the movement of the human eye and the screen and the corresponding pixel coordinates when the human eye views the screen.

[0081] In an embodiment of the present invention, the continuously acquired intersection points between the line of sight of the human eye movement and the screen can be stored using a preset buffer with a fixed depth. The preset buffer with a fixed depth is used to receive data and follows the first-in-first-out principle of data processing, that is, for the buffer, if there is data overflow during data storage, the data that first enters the buffer will be stored.

[0082] Then, the intersections are sorted according to the time sequence of the intersections stored in the preset buffer to obtain an intersection sequence. In the embodiment of the present invention, the preset buffer is combined, and the intersections of the line of sight and the screen are obtained first, and the processing is performed first, and then according to the order of processing the intersections, it is convenient to analyze the user's eye status according to the time sequence.

[0083] 202. When the number of intersection points continuously obtained by counting reaches a preset threshold, the intersection points are used to form an intersection point set, and each intersection point and its corresponding pixel coordinates are stored in the intersection point set.

[0084] In an embodiment of the present invention, this step is further described. A sliding window of a fixed length can be used to count the number of intersections of a preset threshold, and then analyze the eye status of the user during the time when the user watches the screen and the intersection set is composed of these intersections. The specific steps are as follows:

[0085] First, a sliding window is set, the length of which is preset and is used to circle a preset threshold number of intersection points.

[0086] In the embodiment of the present invention, how to set the preset threshold can be determined by combining the actual detection demand time (such as detecting that the user's screen viewing time reaches 10 minutes, etc.) and obtaining the time interval of each intersection, and specifically adopting the following formula (1):

[0087] L = t / s Formula (1);

[0088] Among them, L is the length of the sliding window, t is the length of the time for observing the movement of human eyes, and s is the time period for obtaining the intersection of the line of sight of human eye movement and the screen. In other words, within the observation time t, combined with the time period for obtaining the intersection of the line of sight of human eye movement and the screen, a better way is to pre-set the length of the sliding window to L, which meets and is suitable for counting the intersection of the preset threshold number.

[0089] Secondly, the first intersection point stored in the preset buffer is taken as the first intersection point, and the target intersection points with a preset threshold number are circled in the intersection point sequence using the sliding window, and the target intersection points are used to form an intersection point set. Figure 3 As shown, the example uses a sliding window to circle the intersection sequence in the preset buffer, such as Figure 3 Assume that the intersection sequence in the preset buffer contains 6 intersections, and the sliding window length is sufficient to circle at most 5 intersections. Then, the intersection that is stored in the preset buffer earliest on the left is taken as the first intersection, and the sliding window is used to circle as follows: Figure 3 If the sliding window moves to the right, the intersection points in the preset buffer can be circled. Figure 3 The rightmost intersection point, that is, the sixth intersection point in the intersection sequence.

[0090] It should be noted that the advantage of using a fixed-length sliding window to circle a sequence is that when determining a first circled intersection, the intersections circled by the sliding window are not scattered, but can be automatically circled starting from the first intersection in a sorted sequence to achieve continuous intersections in the sequence. In addition, for the embodiments of the present invention, the intersections circled by the sliding window this time may not be sufficient to determine that the user is in a gaze state. Then, by sliding the sliding window, it is also convenient to determine the user's eye state in the corresponding time of other continuous intersections in the intersection sequence in the preset buffer by re-circling according to the time sequence of the intersections between the line of sight and the screen.

[0091] It is particularly important to explain that, for the operation of circling intersections when the sliding window moves, two adjacent circling operations can circle repeated intersections, and each intersection corresponds to one time when the user views the screen. Furthermore, based on the existence of repeated intersections, it also indicates that two detection time periods with overlapping time are used to detect the user's eye status. This time overlap also facilitates the most comprehensive detection of the user's eye status and avoids omissions in detection time.

[0092] 203. Determine whether the divergence value of the discreteness corresponding to the intersection point set is greater than a preset divergence threshold.

[0093] It should be noted that the pixel coordinates corresponding to each intersection point in the set of intersection points circled by the sliding window on the screen are obtained according to step 201 .

[0094] First, the specific implementation method of calculating the corresponding discreteness divergence value in combination with the pixel coordinates of these intersection points may include the following:

[0095] From all the intersections circled by the sliding window, obtain the pixel coordinates of each intersection on the screen. By comparing the horizontal and vertical coordinates of each intersection, find the maximum pixel coordinate max(x) in the horizontal coordinate and the minimum pixel coordinate min(x) in the horizontal coordinate; find the maximum pixel coordinate max(y) in the vertical coordinate and the minimum pixel coordinate min(y) in the vertical coordinate. Add the difference between the maximum coordinate value and the minimum coordinate value in the horizontal and vertical dimensions to obtain the discreteness measurement of all coordinates. The specific process can be simply expressed as the following formula (2):

[0096] div(xy)=[max(x)-min(x)]+[max(y)-min(y)] Formula (2);

[0097] Among them, div(xy) is the divergence value of the discrete degree corresponding to the intersection point included in the sliding window.

[0098] Secondly, it is determined whether the divergence value obtained by the above calculation reaches a preset threshold.

[0099] In the embodiment of the present invention, the divergence value of the discreteness corresponding to the set of intersection points selected by the sliding window is compared with a preset threshold value. Since the preset threshold value (divergence value) is pre-set according to actual needs and is used as a measurement standard for measuring the degree of discreteness, the degree of discreteness corresponding to the set of intersection points selected by the sliding window can be judged through the comparison operation.

[0100] 204. If it is determined that the divergence value of the discreteness corresponding to the intersection point set is greater than a preset divergence threshold, it is determined that the human eye movement is in a fixation state and the corresponding fixation point and its corresponding pixel coordinates are calculated according to the intersection point set.

[0101] In the embodiment of the present invention, if the divergence value of the discreteness corresponding to the intersection set is determined to be greater than the preset divergence threshold, the pixel coordinates of all the intersections in the sliding window are weighted by the average value to obtain the coordinate position on the screen where the human eye is looking. For example, formula (3) is used for the intersection {(x i ,y i )|0<=i<=p-1}:

[0102]

[0103]

[0104] Among them, the pixel coordinates of the gaze point are (c x , c y ),

[0105] Further, in an embodiment of the present invention, after the pixel coordinates of the gaze point corresponding to the intersection set are calculated, it indicates that the user can be determined to be in a gaze state and the gaze point can be calculated using the intersection set, which means that the task of detecting the user's eye state in this round is completed and ended (i.e., all intersections in the sliding window are deleted), and then a new round of tasks for detecting the user's eye state is started. Since the intersections between the line of sight and the screen are continuously obtained during the process of the human eye moving to watch the screen, the new round of tasks can re-count and obtain the preset threshold number of intersections, and it should be noted that in the new round of tasks, it should be avoided to obtain intersections that are repeated with the intersection set in the previous round of tasks, that is, to detect the user's eye state in the next time period of this round of tasks. Specifically, the operation process of re-executing a new round of tasks may include the following:

[0106] First, according to the order of time when the intersections of the line of sight corresponding to the eye movement and the screen are continuously obtained, the preset threshold number of intersections is re-counted. Specifically, the last selected intersection in the intersection set selected in the previous round of tasks can be used as a reference to find the next intersection of the intersection in the intersection sequence in the preset buffer, and this intersection is used as the starting point to use the sliding window to select the target intersection and form a new intersection set.

[0107] Secondly, it is determined whether the divergence value of the discreteness corresponding to the new intersection set is greater than the preset divergence threshold; if so, it is determined that the human eye is in a fixation state and the corresponding fixation point and its corresponding pixel coordinates are calculated based on the intersection set.

[0108] It should be noted that in the embodiments of the present invention, the application scenarios given in the embodiments are all for judging that the divergence value of the discreteness corresponding to the intersection set is greater than the preset divergence threshold, and detailed explanations are also given. However, if the divergence value of the discreteness corresponding to the intersection set composed of the target intersection points selected by the current sliding window is not greater than the preset divergence threshold, the following operation process can be specifically performed:

[0109] First, the sliding window is triggered to move rightward by a preset number of intersection positions to redetermine the first intersection, and a preset threshold number of target intersections are re-selected in the intersection sequence of the preset buffer using the redetermined first intersection.

[0110] Secondly, a new set of intersection points is formed based on the re-selected target intersection points, and it is determined whether the divergence value of the discreteness corresponding to the new set of intersection points is greater than a preset divergence threshold.

[0111] Finally, according to the above judgment conditions, a judgment operation is performed. If the divergence value of the discreteness corresponding to the new intersection set is greater than the preset divergence threshold, it is determined that the user is in a gaze state and the pixel coordinates of the gaze point are calculated according to the new intersection set.

[0112] However, if the divergence value of the discreteness corresponding to the new intersection set is still not greater than the preset divergence threshold, then the operations of triggering the sliding window movement, reselecting the target intersection, and judging whether the divergence value of the discreteness corresponding to the new intersection set composed of the target intersection is greater than the preset divergence threshold are repeated, that is, the above iterative operations are performed until it is judged that the divergence value is greater than the preset threshold, then it is determined that the human eye is in a gaze state and the corresponding gaze point and its corresponding pixel coordinates are calculated according to the determined intersection set.

[0113] 205. Determine the gaze point as the target intersection point on the screen.

[0114] 206. Select an area within a preset pixel range from the target intersection on the screen as the target area.

[0115] In an embodiment of the present invention, steps 205-206 are described in detail. The gaze point is equivalent to representing the intersection set. Using this gaze point, it can be known that during the screen viewing time corresponding to the intersection set, the content that the user is interested in or pays attention to is the area around the gaze point. Therefore, the gaze point can be used as the target intersection, so as to select the area within a preset pixel range from the target intersection on the screen as the target area. In this way, the area near the gaze point can be circled on the screen. In fact, the area near the gaze point that is circled is the range of repeated sight points when the user is in a gaze state.

[0116] 207. Perform feature rendering operation on the target area and output display effect.

[0117] In an embodiment of the present invention, in order to clearly show that the content that the user is interested in or pays attention to is different from other content on the screen, after determining the target area, a feature rendering method can be adopted for the target area, such as: high-resolution / multi-resolution rendering, rendering background color, etc. to highlight the display effect, and display the target area on the screen. With the help of such feature rendering operations, the user's experience of viewing the screen content is further improved, and it can also clearly show which content on the screen the user is more interested in.

[0118] For example, Figure 4 The screen shown in the example shows the display effect of the scanning state and the fixation state. Figure 4 The left side is in the state of eye scanning, and the display effect obtained by tracing the lines connecting the intersections of multiple eye sights and the screen is shown. Figure 4 In the middle right side, because the image is in a state of eye gaze, an example of the display effect obtained by performing high-definition rendering on the area near the gaze point corresponding to the gaze state is shown.

[0119] Furthermore, as a response to the above Figure 1 , Figure 2 The embodiment of the present invention provides a device for detecting the gaze point of human eye movement. The device embodiment corresponds to the aforementioned method embodiment. For ease of reading, the device embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the device in this embodiment can correspond to all the contents of the aforementioned method embodiment. The device is used to detect whether the human eye movement is in a gaze state and obtain the corresponding gaze point, specifically as follows Figure 5 As shown, the device comprises:

[0120] An acquisition unit 301 is used to continuously acquire the intersection of the line of sight corresponding to the movement of the human eye and the screen and the corresponding pixel coordinates thereof at a preset time interval while the human eye is viewing the screen;

[0121] A forming unit 302 is used to form an intersection point set using the intersection points when the number of the intersection points continuously obtained by the obtaining unit 301 reaches a preset threshold, and the intersection point set stores each intersection point and its corresponding pixel coordinates;

[0122] A judging unit 303, used to judge whether a divergence value of a discreteness corresponding to the intersection point set formed by the forming unit 302 is greater than a preset divergence threshold;

[0123] A determination unit 304, configured to determine that the human eye movement is in a fixation state when the determination unit 303 determines that the divergence value of the discreteness corresponding to the intersection set is greater than a preset divergence threshold;

[0124] The calculation unit 305 is used to calculate the corresponding gaze point and its corresponding pixel coordinates according to the intersection point set after the determination unit 304 determines that the human eye movement is in a gaze state.

[0125] Further, if Figure 6 As shown, the device also includes:

[0126] The determining unit 304 is further configured to determine the gaze point as a target intersection point on the screen;

[0127] A selection unit 306, configured to select an area on the screen within a preset pixel range from the target intersection point determined by the determination unit 304 as a target area;

[0128] A rendering unit 307, configured to perform a feature rendering operation on the target area selected by the selection unit 306;

[0129] The output unit 308 is used to output the display effect obtained by the rendering unit 307 performing the rendering operation.

[0130] Further, if Figure 6 As shown, the acquisition unit 301 includes:

[0131] The storage module 3011 is used to store the intersection points corresponding to the eye movement and the screen in a preset buffer one by one according to the time sequence of obtaining the intersection points;

[0132] The sorting module 3012 is used to sort the intersections according to the time sequence in which the intersections stored by the storage module 3011 enter the preset buffer to obtain an intersection sequence.

[0133] Further, if Figure 6 As shown, the component unit 302 includes:

[0134] A setting module 3021, used for setting a sliding window, wherein the sliding window length is preset and is used for circling the intersection points of the preset threshold number;

[0135] A selection module 3022 is configured to use the first intersection point stored in the preset buffer as the first intersection point, and to select the preset threshold number of target intersection points in the intersection sequence using the sliding window set by the setting module 3021;

[0136] The forming module 3023 is used to form an intersection set using the target intersections circled by the circle selecting module 3022.

[0137] Further, if Figure 6 As shown, the device also includes:

[0138] A trigger unit 309 is configured to trigger the sliding window to move rightward by a preset number of intersection positions to re-determine the first intersection point when the determination unit 303 determines that the divergence value of the discreteness corresponding to the intersection point set is not greater than a preset divergence threshold;

[0139] The circle selection unit 310 is used to re-select the preset threshold number of target intersection points in the intersection point sequence in the preset buffer zone with the re-determined first intersection point;

[0140] A forming unit 302, configured to form a new set of intersection points according to the target intersection points re-selected by the selecting unit 310;

[0141] The judging unit 303 is further used to judge whether the divergence value of the discreteness corresponding to the new intersection set formed by the forming unit 302 is greater than a preset divergence threshold;

[0142] The execution unit 311 is used to repeatedly execute the operations of triggering the sliding window to move, reselecting the target intersection, and judging whether the divergence value of the discreteness corresponding to the new intersection set composed of the target intersection is greater than the preset divergence threshold if the judgment unit 303 judges that the divergence value of the discreteness corresponding to the new intersection set is not greater than the preset divergence threshold, until it is judged that the divergence value is greater than the preset threshold, it is determined that the human eye is in a gaze state and the corresponding gaze point and its corresponding pixel coordinates are calculated according to the new intersection set.

[0143] In summary, the embodiment of the present invention provides a method and device for detecting the gaze point of human eye movement. The embodiment of the present invention periodically and continuously obtains the intersection points and pixel coordinates of the human eye movement line of sight on the screen. When the statistics reach the preset threshold number, the intersection points counted are used to form an intersection set, and it is determined whether the divergence value of the discreteness corresponding to the intersection set is greater than the preset divergence threshold. If so, it is determined that the human eye is in a gaze state and the corresponding gaze point and its corresponding pixel coordinates are calculated based on the intersection set. Compared with the prior art, the problem that the prior method cannot effectively detect the gaze state when the human eye moves is solved. The embodiment of the present invention optimizes the operation of tracking the human eye movement, and can effectively determine when the human eye is in a gaze state when the human eye moves and also calculate the corresponding gaze point. The gaze point is equivalent to representing the scattered points in the intersection set, and then according to the gaze point, it is helpful to know that the surrounding area is the content that the user is interested in or concerned about, and it is also convenient to analyze the relationship between the human eye movement and the content that the user is interested in or concerned about.

[0144] The human eye movement gaze point detection device includes a processor and a memory. The above-mentioned acquisition unit, composition unit, judgment unit, determination unit and calculation unit are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0145] The processor includes a kernel, and the kernel retrieves the corresponding program unit from the memory. One or more kernels can be set, and the problem that the existing method cannot effectively detect the eye gaze state when the human eye moves is solved by adjusting the kernel parameters. The present invention optimizes the operation of tracking the human eye movement, can effectively determine when the human eye is in the eye gaze state when moving, and also calculates the corresponding gaze point.

[0146] An embodiment of the present invention provides a storage medium on which a program is stored. When the program is executed by a processor, the method for detecting a gaze point of human eye movement is implemented.

[0147] An embodiment of the present invention provides a processor, which is used to run a program, wherein the method for detecting a gaze point of a human eye movement is executed when the program is run.

[0148] An embodiment of the present invention provides an electronic device 40, such as Figure 7 As shown, the device includes at least one processor 401, and at least one memory 402 and a bus 403 connected to the processor 401; wherein the processor 401 and the memory 402 communicate with each other through the bus 403; the processor 401 is used to call the program instructions in the memory 402 to execute the above-mentioned human eye movement gaze point detection method.

[0149] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing the following method steps:

[0150] A method for detecting eye gaze points of human eye movement, the method comprising: continuously obtaining the intersection points of the eye gaze corresponding to the eye movement and the screen and the corresponding pixel coordinates thereof at preset time intervals while the eye is viewing the screen; when the number of the intersection points continuously obtained statistically reaches a preset threshold, using the intersection points to form an intersection point set, wherein each intersection point and the corresponding pixel coordinates thereof are stored in the intersection point set; determining whether the divergence value of the discreteness corresponding to the intersection point set is greater than a preset divergence threshold; if so, determining that the eye movement is in a gaze state and calculating the corresponding gaze point and the corresponding pixel coordinates thereof according to the intersection point set.

[0151] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0152] In a typical configuration, the device includes one or more processors (CPU), memory and bus. The device may also include input / output interface, network interface and the like.

[0153] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip. The memory is an example of a computer-readable medium.

[0154] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0155] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0156] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0157] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for detecting eye movement fixation points, characterized in that: The method comprises: At preset time intervals, continuously obtaining the intersection of the line of sight corresponding to the movement of the human eye and the screen and its corresponding pixel coordinates while the human eye is viewing the screen; When the number of the intersection points continuously obtained by counting reaches a preset threshold, the intersection points are used to form an intersection point set, and the intersection point set stores each intersection point and its corresponding pixel coordinates; Determine whether the divergence value of the discreteness corresponding to the intersection set is greater than a preset divergence threshold; If so, it is determined that the human eye movement is in a fixation state and the corresponding fixation point and its corresponding pixel coordinates are calculated according to the intersection point set.

2. The method according to claim 1, characterized in that The method further comprises: determining the fixation point as a target intersection point on the screen; Selecting an area within a preset pixel range from the target intersection on the screen as the target area; A feature rendering operation is performed on the target area and a display effect is output.

3. The method according to claim 1, characterized in that The method of continuously obtaining the intersection of the line of sight corresponding to the movement of the human eye and the screen and the corresponding pixel coordinates thereof during the process of the human eye viewing the screen includes: According to the time sequence of obtaining the intersection points of the sight line corresponding to the eye movement and the screen, the intersection points are stored one by one in a preset buffer; According to the time sequence in which the intersection points are stored in the preset buffer, the intersection points are sorted to obtain an intersection point sequence.

4. The method according to claim 3, characterized in that When the number of intersection points obtained by counting reaches a preset threshold, using the intersection points to form an intersection point set includes: Setting a sliding window, wherein the sliding window length is preset and is used to circle the intersection points of the preset threshold number; The first intersection point stored in the preset buffer is taken as the first intersection point, and the preset threshold number of target intersection points are circled in the intersection point sequence by using the sliding window; An intersection point set is formed using the target intersection points.

5. The method according to claim 4, characterized in that If it is determined that the divergence value of the discreteness corresponding to the intersection set is not greater than a preset divergence threshold, the method further includes: Triggering the sliding window to move rightward by a preset number of intersection positions to redetermine the first intersection point; Reselecting the preset threshold number of target intersection points in the intersection point sequence in the preset buffer zone using the re-determined first intersection point; Forming a new set of intersection points according to the re-selected target intersection points; Determine whether the divergence value of the discreteness corresponding to the new intersection set is greater than a preset divergence threshold; If not, the operations of triggering the sliding window to move, reselecting the target intersection, and judging whether the divergence value of the discreteness corresponding to the new intersection set composed of the target intersection is greater than the preset divergence threshold are repeatedly executed until it is judged that the divergence value is greater than the preset threshold, then it is determined that the human eye is in a gaze state and the corresponding gaze point and its corresponding pixel coordinates are calculated according to the new intersection set.

6. A device for detecting eye movement gaze points, characterized in that: The device comprises: An acquisition unit, used for continuously acquiring the intersection of the line of sight corresponding to the movement of the human eye and the screen and the corresponding pixel coordinates thereof at a preset time interval while the human eye is viewing the screen; A forming unit, configured to form an intersection point set using the intersection points when the number of the intersection points continuously obtained by the obtaining unit reaches a preset threshold, wherein the intersection point set stores each intersection point and its corresponding pixel coordinates; A judging unit, used to judge whether a divergence value of a discreteness corresponding to a set of intersection points formed by the constituent units is greater than a preset divergence threshold; A determination unit, configured to determine that the human eye movement is in a fixation state when the determination unit determines that the divergence value of the discreteness corresponding to the intersection set is greater than a preset divergence threshold; The calculation unit is used to calculate the corresponding gaze point and its corresponding pixel coordinates according to the intersection point set after the determination unit determines that the human eye movement is in a gaze state.

7. The device according to claim 6, characterized in that The device also includes: The determining unit is further used to determine the gaze point as a target intersection point on the screen; A selection unit, configured to select, on the screen, an area within a preset pixel range from the target intersection point determined by the determination unit as a target area; A rendering unit, used for performing a feature rendering operation on the target area selected by the selection unit; The output unit is used to output the display effect obtained by the rendering unit performing the rendering operation.

8. The device according to claim 6, characterized in that The acquisition unit comprises: A storage module, used for storing the intersection points one by one in a preset buffer according to the time sequence of obtaining the intersection points between the line of sight corresponding to the movement of the human eye and the screen; The sorting module is used to sort the intersections according to the time sequence in which the intersections stored by the storage module enter the preset buffer to obtain an intersection sequence.

9. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the method for detecting the gaze point of human eye movement as described in any one of claims 1-5.

10. An electronic device, characterized in that: The device includes at least one processor, and at least one memory and a bus connected to the processor; Wherein, the processor and the memory communicate with each other through the bus; The processor is used to call the program instructions in the memory to execute the human eye movement gaze point detection method as described in any one of claims 1-5.

Images