Eye touch small target object selection method and device for mixed staring smooth tracking
By adopting a hybrid gaze smooth tracking method in augmented reality devices, combined with gaze and smooth tracking instructions, the interaction accuracy and stability problems of small target object selection are solved, and efficient selection of viewing targets less than 1.8° is achieved, and interaction accuracy is improved and user experience.
Patent Information
- Application Number
- CN202411140247.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-29
AI Technical Summary
Existing augmented reality devices have problems such as low interaction accuracy, large target size, and unstable interaction when selecting small target objects. Especially for target objects with a viewing angle of less than 1.8°, it is difficult to achieve efficient and stable eye-controlled interaction.
The eye-touch small-target object selection method based on interactive locking is adopted. By combining the two control instructions of gaze and smooth tracking, combined with the interactive intention locking mechanism, the state of the interactive object is adjusted and the interaction accuracy of small-target object selection is improved. and stability.
It realizes efficient selection of target objects with a size of less than 1.8°, improves interaction accuracy and stability, shortens interaction time, and improves user operation accuracy and satisfaction.
Smart Images

Figure CN119045661A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of eye-controlled small target selection, and in particular to an eye-touch-controlled small target object selection method based on interactive locking mixed gaze smooth tracking. Background Art
[0002] With the continuous development of spatial computing technology, augmented reality (AR) and virtual reality (VR) devices are increasingly being used in various fields. For people who use spatial computing and people with hand dysfunction, eye control technology has gradually become an important way of interaction. However, current augmented reality devices still have problems such as low interaction accuracy, large target object size, and unstable interaction when selecting small target objects.
[0003] In the prior art, the selection mechanism mainly includes pointing selection, semantic selection and motion correlation selection. Among them, pointing selection has the characteristics of intuitive interaction, while smooth tracking, as a motion correlation selection method, is spatially independent. However, the current augmented reality eye-controlled text selection method generally has problems such as low interaction accuracy, large target text size, and unstable interaction. This is mainly due to the low accuracy of eye movement data collected by augmented reality devices and the low accuracy of human eye pointing.
[0004] In current academic research, text selection (marking) methods in augmented reality and virtual reality mainly include hand ray, head movement pointing and other methods. The hand ray method is not applicable when the hand function is occupied, the head movement pointing method will cause greater fatigue, and the flexibility of head movement is not as good as eye movement. In the selection of small target objects, traditional gaze interaction methods are difficult to achieve precise operation, especially for target objects with a viewing angle of less than 1.8°. Experimental tests show that it is also difficult to achieve stable interaction effects using only smooth tracking as a method for selecting small-sized targets.
[0005] In addition, in the current eye-controlled interaction system for small targets, there are almost no eye-touch interaction methods that use gaze combined with smooth tracking. Therefore, how to achieve efficient and stable eye-controlled small target object selection in augmented reality devices has become a technical problem that needs to be solved urgently.
[0006] The present invention proposes an eye-touch small target object selection method based on interactive locking and hybrid gaze smooth tracking, which aims to improve the interactive accuracy and stability of small target object selection by fusing the two control instructions of gaze and smooth tracking. This method combines the gaze instruction and the smooth tracking instruction through the interactive intention locking mechanism, assigns weights to each, and adjusts the state of the interactive object according to the locking attribute of the interactive intention, thereby achieving efficient and stable small target object selection. Summary of the invention
[0007] The technical problem to be solved by the present invention is to provide an eye-touch small target object selection method based on interactive locking and hybrid gaze smooth tracking in order to achieve more efficient and precise interaction, in view of the shortcomings of the prior art when selecting small target objects on an augmented reality device.
[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is:
[0009] The eye-touch small target object selection method based on interactive locking hybrid gaze smooth tracking includes the following steps:
[0010] Step 1: Create interactive objects;
[0011] Step 2: Obtain gaze instructions and smooth pursuit instructions for eye movements;
[0012] Step 3: According to the acquired gaze instructions and smooth tracking instructions, the interaction intention locking attribute of each interaction object is calculated, and the interaction state of the eye-touch small target object is determined according to the calculated interaction intention locking attribute.
[0013] In step 3, based on the acquired gaze instructions and smooth tracking instructions, the interaction intention locking attribute of each interaction object is calculated as:
[0014]
[0015] Wherein: IL is the interaction intention; E1 represents the variable of whether there is a gaze instruction; E2 represents the variable of whether there is a smooth pursuit instruction; a1 and a2 are continuous variables, which change in real time according to the gaze instruction and the smooth pursuit instruction; w1 and w2 are weights used to regulate the influence of the two interaction modes; c is a constant, indicating that the interaction intention decreases over time; t represents the sampling time.
[0016] The continuous variable a1 is:
[0017]
[0018] Among them, a 1,min and a 1,max is the minimum and maximum value of a1; d is the Euclidean distance between the gaze point and the center point; Thr1 is the threshold used to determine whether the current gaze point is close to the center point.
[0019] Adjust the continuous variable a2 according to the variable E2:
[0020] Determine variable E2, which is used to indicate whether the threshold of the smooth tracking instruction is reached. The calculation formula is as follows:
[0021]
[0022] Wherein, C is the current motion correlation value; Thr2 is used to determine whether the motion correlation reaches the similar motion correlation threshold;
[0023] When E2=0, the continuous variable a2 is set to 0; when E2=1, the continuous variable a2 is adjusted according to the motion correlation value, and the formula is as follows:
[0024] a2=C
[0025] The continuous variable a1 can also be obtained in the following way, where a1 is:
[0026] Define a constant value a 1,constant It is used to adjust the variable a1 and determine whether the variable E1 is used to indicate whether the threshold of the gaze instruction is reached. The calculation formula is as follows:
[0027]
[0028] Where d is the Euclidean distance between the gaze point and the center point; Thr1 is the threshold used to determine whether the current gaze point is close to the center point;
[0029] When E1=1, the variable a1 is set to a constant value a 1,constant ;When E1=0, variable a1 is set to 0.
[0030] The weights w1, w2 and constant c are used to find the optimal parameter combination by testing the effects of different weight combinations (w1, w2) and constant c on user operation efficiency and accuracy.
[0031] The specific method to find the optimal combination of parameters (w1, w2) and constant c is:
[0032] Select a test sample of representative users; use a head-mounted display device with an eye tracker to complete the task of selecting a small target; record the user's operation time, selection accuracy, and subjective satisfaction rating data for each test;
[0033] Finally, the experimental results were analyzed to calculate the average operation time, accuracy and satisfaction scores under different weight combinations; the significant impact of different parameter combinations on operation efficiency and accuracy was evaluated; based on the evaluation results, a set of parameter combinations (w1, w2, c) was determined to make the system optimal in terms of operation efficiency, accuracy and user satisfaction.
[0034] In step 3, the interaction state of the eye-touch small target object is determined according to the calculated interaction intention lock attribute, including:
[0035] Control the state of the interactive object; the state control of the interactive object includes lock state control and gaze state feedback; the lock state control sets a threshold according to the lock attribute value of the interactive intention, and the state of the interactive object is locked when the threshold is reached, and the state unlocking time is set. When the unlocking time limit is reached, the locked state of the interactive object is unlocked; the gaze state feedback is to update the state of the interactive object according to whether the eye movement point falls on the interactive object; when the eye movement point falls on the interactive object, the state of the interactive object is updated to the hovering state;
[0036] Select the interactive object; the selection command is a key operation, blinking, foot pedaling, tooth clicking or voice interaction action. When the interactive object is in a locked state and a selection command is heard, the interactive object will change to a triggered state, which means that the interactive object is selected.
[0037] The present invention also provides an interactive device, comprising:
[0038] one or more processors;
[0039] A memory for storing one or more programs;
[0040] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the eye-touch small target object selection method based on interactive locking and hybrid gaze smooth tracking provided above.
[0041] The present invention also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the eye-touch small target object selection method based on interactive locking and mixed gaze smooth tracking are implemented.
[0042] Compared with the prior art, the present invention has the following beneficial effects:
[0043] The interaction accuracy is higher. The small target selection method of this solution uses eye control technology, which can select target objects with a size smaller than 1.8° visual angle. Since the technical solution uses a smooth tracking eye control interaction recognition method, it can process interaction instructions that are independent of position. Compared with existing eye control interaction technologies such as gaze, the present invention improves the interaction accuracy of the eye control text plotting system.
[0044] The interactive method is easier to use. The small target selection method of this scheme uses two interactive commands, gaze and smooth tracking, which shortens the time of interaction with smooth tracking commands alone and makes the small target interactive object easier to select. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 :Interaction flow chart;
[0046] Figure 2 :Indicator relationship diagram for interaction intention locking;
[0047] Figure 3 : Schematic diagram of delay algorithm;
[0048] Figure 4 :Interaction object description diagram;
[0049] Figure 5 : Smooth pursuit data acquisition experiment flow chart;
[0050] Figure 6 : Schematic diagram of each level of trajectory shape;
[0051] Figure 7 :Data analysis flow chart;
[0052] Figure 8 :Analysis results of different trajectory shapes;
[0053] Fig. 9 :Analysis results of different sizes of linear tracks;
[0054] Fig.10 :Analysis results of different motion cycles of linear track. DETAILED DESCRIPTION
[0055] The present invention will be described in detail below in conjunction with the accompanying drawings:
[0056] Example 1
[0057] This embodiment provides an eye-touch small target object selection method based on interactive locking and hybrid gaze smooth tracking, which integrates the two control instructions of gaze and smooth tracking, and adjusts the interactive intention locking attribute by giving weights to the gaze instruction w1 and the smooth tracking instruction w2 respectively. When the locking threshold is reached, the interactive object state is adjusted to be interactive, thereby completing the selection of the small target.
[0058] Eye-touch small target object selection method based on interactive locking hybrid gaze smooth tracking, such as Figure 1 As shown, the following steps are included:
[0059] Step 1: Generation and control of interactive objects: Create interactive objects and control the state of interactive objects;
[0060] Step 2: Eye movement behavior detection: obtaining gaze instructions and smooth pursuit instructions of eye movements;
[0061] Step 3: Interaction intention locking: According to the acquired gaze instructions and smooth tracking instructions, the interaction intention locking attributes of each interactive object are calculated, and the interaction state of the eye-touch small target object is determined according to the calculated interaction intention locking attributes.
[0062] Step 1: Generation and control of interactive objects.
[0063] Before the system (computer with desktop eye tracker, spatial computing device with head-mounted eye tracker (augmented reality, virtual reality device)) runs, the position of the small target object is identified and the interactive object is instantiated.
[0064] The specific method of generating and controlling the interactive object in step 1 is: obtaining the spatial position of the small target object, creating the interactive object at the obtained position, initializing the button state to the rest state, and controlling the state of the interactive object.
[0065] Interactive objects refer to objects with interactive functions, such as interface controls, etc. Interactive objects include motion sliders and motion tracks, such as Figure 4 As shown in the figure. The motion slider is the moving target point to be tracked, and the motion track is the path of the motion slider. The motion slider and motion track can be displayed or not and how to display them according to the interface design. The motion parameters of the interactive object include phase, speed, acceleration, trajectory shape, etc.
[0066] The premise of phase is that the motion of the interactive object is periodic motion. Phase is the starting point of the periodic motion, which is equivalent to defining the position of the motion slider with the percentage of the period.
[0067] Speed is the speed at which the motion slider moves, or it can be the angular velocity in circular motion.
[0068] Acceleration is a parameter used when the motion slider is moving at a variable speed.
[0069] The trajectory shape is the shape of the motion track, which can be a circle, rectangle, triangle, diamond, zigzag, V-shaped, complex polygon and other different shapes, such as Figure 6 shown.
[0070] The interactive object state control part controls the state performance of the interactive object, such as the color of the motion slider, to inform the user of the current state of the interactive object. The interactive object states include resting state, hovering state, tracking state, tracking hovering state, locked state, and triggered state. Resting state means that the interactive object is neither gazed at nor tracked. Hovering state means that the interactive object is gazed at. Tracking state means that the interactive object is tracked. Tracking hovering state means that the interactive object is both tracked and gazed at. Locked state means that tracking and gaze have reached the locked preset, and the state of the interactive object is locked. Triggered state means that the selection instruction is received and the interactive object is triggered.
[0071] Step 2: Eye movement behavior detection, obtaining gaze instructions and smooth pursuit instructions of eye movements.
[0072] Eye movement behavior detection includes: position proximity detection and motion correlation detection based on the gaze point position obtained in real time. Position proximity detection is to determine whether the gaze proximity index a1 reaches the threshold value based on the gaze instruction threshold value Thr1, thereby determining whether there is a gaze instruction E1. Motion correlation detection is to determine whether the motion similarity index a2 reaches the threshold value based on the smooth tracking instruction threshold value Thr2, thereby determining whether there is a smooth tracking instruction E2.
[0073] Thr1 is a threshold used to determine whether the current gaze point is close to the center point. This threshold can be selected by the spacing between small targets, or can be set according to the recommended size of the smallest gaze interaction eye control interaction object. Generally, the recommended size of the gaze interaction eye control interaction object is 2.6° to 3.3°.
[0074] Thr2 is used to determine whether the motion correlation reaches the motion correlation threshold. This threshold can be selected based on the recommended value of previous studies, for example, using the Pearson correlation coefficient algorithm, the recommended value is 0.8. A preliminary experiment can also be conducted to collect smooth pursuit eye movement data to obtain the best threshold.
[0075] In order to obtain the best motion correlation threshold index for smooth pursuit, eye movement related data of smooth pursuit can be collected to optimize system performance. This study tests different smooth pursuit algorithms, delay parameters, trajectory shape, size, speed and other parameters in order to find the best optimization method to improve user experience and efficiency.
[0076] Position proximity is an indicator of gaze eye movement behavior, and the presence or absence of gaze behavior is reflected by the gaze instruction E1. The position proximity detection method calculates the distance from the current gaze point to each interactive object in real time.
[0077] The gaze command is a command related to the current gaze point position. There are two methods, one is to steadily increase the constant value within the threshold, and the other is to linearly change the value of a1 according to the distance from the center point. The gaze point position threshold determines whether the degree of proximity is reached, changes E1, and linearly adjusts a1 according to the distance from the center point. Among them, the variable E1 determines the presence or absence of the gaze command through the threshold. If the threshold is reached, E1=1, otherwise E1=0. The variable a1 is linearly adjusted according to the distance between the gaze point and the center point.
[0078] Specifically, method 1 constant value adjustment a1 requires defining a threshold Thr1 to determine whether the current gaze point is close to the center point. Assume that the current gaze point position is (x, y) and the center point position is (x0, y0). Calculate the Euclidean distance d between the gaze point and the center point using the following formula:
[0079]
[0080] Define a constant value a 1,constant It is used to adjust the variable a1 and determine whether the variable E1 is used to indicate whether the threshold of the gaze instruction is reached. The calculation formula is as follows:
[0081]
[0082] When E1=1, the variable a1 is set to a constant value a 1,constant When E1=0, variable a1 is set to 0.
[0083] Method 2: Linearly adjust a1 according to the distance. The same method is used to obtain the Euclidean distance d between the viewpoint and the center point, and determine the variable E1. In the part of linearly adjusting the variable a1, it is necessary to define a 1,min and a 1,max is the minimum and maximum value of a1. When E1=0, the variable a1 is set to 0. When E1=1, the variable a1 is linearly adjusted according to the distance between the gaze point and the center point, and the formula is as follows:
[0084]
[0085] When the eye movement calibration accuracy is high, using method 2 can enhance the function of gaze commands and improve the efficiency of interaction. This is because method 2 allows gaze to increase the effect of gaze commands when the gaze is closer to the interaction object, thereby improving the efficiency of interaction. When the eye movement calibration accuracy is relatively low, using method 1 can increase the robustness of the interaction and make the interaction process more stable. This is because the constant value of method 1 makes the effect of gaze on lock constant, so that users can more easily understand how to operate the system.
[0086] Motion correlation is an indicator of eye movement behavior of smooth pursuit. The smooth pursuit instruction E2 reflects the presence or absence of smooth pursuit behavior, a2 represents the motion similarity index value, and C is the current motion correlation value. The motion correlation detection method calculates the correlation between the gaze point position and the position of each interactive object in real time over a period of time. The motion similarity index can be obtained by any of the following algorithms, including the Pearson correlation coefficient algorithm, the Euclidean algorithm, and the delayed Pearson correlation algorithm. Among these algorithms, the delayed Pearson correlation coefficient algorithm is more recommended.
[0087] The smooth tracking instruction is an instruction based on the motion correlation threshold judgment and numerical adjustment. The motion correlation threshold judges whether similarity is reached, changes E2, and adjusts a2 according to the motion correlation value. Among them, variable E2 judges the presence or absence of the smooth tracking instruction through the threshold. If the threshold is reached, E2 = 1, otherwise E2 = 0. Variable a2 is adjusted according to the motion correlation value.
[0088] The specific method of adjusting a2 according to the motion correlation value is as follows. First, perform motion correlation threshold judgment, and define motion correlation threshold Thr2 to judge whether the motion correlation reaches similarity. Secondly, determine variable E2, which is used to indicate whether the threshold of the smooth tracking instruction is reached. The calculation formula is as follows:
[0089]
[0090] Finally, adjust the variable a2. When E2 = 0, the variable a2 is set to 0. When E2 = 1, the variable a2 is adjusted according to the motion correlation value, and the formula is as follows:
[0091] a2=C
[0092] The three algorithms for motion correlation are shown below.
[0093] (1) Pearson correlation coefficient algorithm
[0094] Most previous studies use the Pearson correlation coefficient algorithm as a smooth pursuit recognition algorithm, which triggers commands by matching the target motion trajectory with the gaze motion trajectory, using motion correlation as a selection method. The Pearson correlation coefficient, usually expressed as ρ X,Y , is a method to measure the degree of linear correlation between two variables. Its value ranges from -1 to 1, where -1 indicates a complete negative linear correlation, 1 indicates a complete positive linear correlation, and 0 indicates no linear correlation. The specific formula is as follows, where X and Y are two variables, and is their average value.
[0095]
[0096] Vidal et al. proposed the following smooth tracking interaction algorithm based on the Pearson correlation coefficient. This is an algorithm for tracking the motion correlation between an input device and a moving target on the screen. It accepts two matrices as input and outputs a Boolean vector indicating whether the input device selects a target at each time point t. The following is a detailed explanation of the algorithm:
[0097] 1) Input:
[0098] P(In): A T×2 matrix representing the two-dimensional position of the eye gaze point at time t {x tin ,y tin}.
[0099] P(Out_i): a T×2 matrix representing each moving target at time t The two-dimensional position of .
[0100] τ: A correlation threshold, ranging from [0,1].
[0101] σ: A window size that represents the number of data points that the algorithm will consider over the time series.
[0102] 2) Output:
[0103] A Boolean vector Selected of length T, representing whether the input device selects a target Out at each time point t i , if selected, then Selected t is true.
[0104] 3) Algorithm steps:
[0105] ① A Boolean vector Selected of length T, representing whether the input device selects a target Out at each time point t i , if selected, then Selected t is true.
[0106] ②For each time point t, the algorithm first checks whether t is greater than or equal to the window size N. If not, Selected t Is set to None, indicating that no target is selected.
[0107] ③If t ≥ N, the algorithm will calculate the correlation coefficient c between the input position and each target position i This coefficient is determined by comparing the input and target trajectories in the x and y directions. Correlation coefficient c i Use the following formula to calculate:
[0108]
[0109] Where cor(a,b) is the correlation coefficient between a and b, defined as:
[0110]
[0111] Here a and b are the sum of the squares of the position differences between the input device and the target in the x or y direction, divided by the window size σ.
[0112] ④If c i If it is greater than the threshold τ, the algorithm will select the target Out with the largest correlation coefficient. k , where k is c i The index of the maximum value.
[0113] ⑤If there is no c i is greater than the threshold τ, then Selected t Is set to None, indicating that no target is selected.
[0114] (2) Euclidean algorithm
[0115] Another smooth tracking algorithm is the Euclidean algorithm. This algorithm activates the command by calculating the distance difference between the line of sight and the target point. The Euclidean distance algorithm is a method used to calculate the distance between two points in Euclidean space. It is one of the most intuitive distance concepts in geometry and is often used to measure the absolute distance between two points in multidimensional space. Given two points p and q, their coordinates in n-dimensional space are p = (p1, p2, …, p n ) and q=(q1,q2,…,q n ), the Euclidean distance d(p,q) can be calculated by the following formula, where p and q are two points in n-dimensional space:
[0116]
[0117] At low speeds, the detection rate of the Euclid algorithm is higher than the Pearson correlation coefficient. For specific groups of people, such as ALS patients who need to use eye-controlled interaction for a long time, slow and smooth tracking of targets can relieve fatigue and bring a better user experience.
[0118] In addition, although the uncalibrated motion matching system is indeed a major advantage of the Pearson correlation coefficient algorithm. From the perspective of the eye control system, a combination of several eye controls can provide the deepest visual experience interaction. The hybrid eye-controlled human-computer interaction (ECHCI) system includes technical modes such as gaze and smooth tracking, in which gaze needs to be calibrated, so in order to implement the system, eye movement calibration is an essential part of the system. Therefore, smooth tracking using Euclidean algorithm technology does not add additional problems. In these systems, the requirements for operation accuracy, task completion time and comfort are very high. Compared with the Pearson correlation coefficient, the low-speed smooth tracking of the Euclidean algorithm can lead to higher detection rates, faster task completion time and better comfort, so this patent believes that the smooth tracking of the Euclidean algorithm is also meaningful.
[0119] (3) Pearson correlation algorithm with delay
[0120] Drewes proposed a Pearson correlation algorithm with a 5-sample delay (83ms), which has a higher detection rate than the Pearson correlation algorithm without delay. From a physiological point of view, the smooth-tracking interactive object is sent to the user as a visual stimulus, and the user should have a reaction time to process the information and track the target, so the input signal of the user's eye gaze point should be later than the movement of the smooth-tracking interactive object. The delay of this algorithm is to solve this problem. By increasing the delay, the movement of the smooth-tracking interactive object and the movement of the user's eye gaze point are aligned.
[0121] To illustrate this algorithm, the author draws a schematic diagram of the delay algorithm, such as Figure 3 As shown in the figure, the horizontal axis is the number of samples, the origin is the 0 sample point, and the right is the positive direction of the sampling time. There are two sampling times, one is the number of delayed samples, and the other is the number of samples, which is the time window size in the correlation algorithm. The vertical axis is the object name, including eye movement input, which is the eye movement gaze point input data set for smooth tracking, and the interactive object movement is the motion data set for smooth tracking of the interactive object. In the delayed Pearson correlation algorithm, the number of delayed samples is set, and the correlation between the input eye movement point coordinates and the interactive object position in the calculation diagram is calculated.
[0122] Step 3: According to the acquired gaze instructions and smooth tracking instructions, the interaction intention locking attribute of each interaction object is calculated, and the interaction state of the eye-touch small target object is determined according to the calculated interaction intention locking attribute.
[0123] The interaction intention locking module combines gaze interaction and smooth tracking interaction, calculates the interaction intention locking attribute value of each interactive object, and determines whether to lock the interaction state of the interactive object based on the threshold, thereby making the interaction process more stable.
[0124] Step 3 specifically includes:
[0125] Step 31: Calculate the lock intent. In each frame, calculate and update the interaction intent lock attribute value of each interactive object. The calculation formula includes a part that increases the interaction intent lock attribute value, including gaze instructions and smooth tracking instructions, and also includes a part that decreases the interaction intent lock attribute value, which decreases the lock intent value according to time.
[0126] The specific formula for the interactive intent lock attribute is as follows:
[0127]
[0128] Wherein: IL is the interaction intention; E1 is the gaze instruction, which is the variable of whether there is a gaze instruction; E2 is the smooth pursuit instruction, which is the variable of whether there is a smooth pursuit instruction; a1 is the gaze proximity index; a2 is the motion similarity index; w1 is the gaze instruction weight; w2 is the smooth pursuit instruction weight, which is used to regulate the influence of the two interaction methods; c is a constant, indicating that the interaction intention decreases over time; t is the sampling time.
[0129] The weights w1, w2 and c can be tested using the following method.
[0130] The optimal parameter combination was found by testing the effects of different weight combinations (w1, w2) and constant c on user operation efficiency and accuracy. 30 users of different ages and professional backgrounds were selected to ensure that the sample was representative. A head-mounted display device with an eye tracker was used to complete the small target selection operation task.
[0131] The test task can be to ask users to complete a series of small target selection operations. Task design should take into account different levels of difficulty and complexity to comprehensively evaluate the interactive performance. During the test, users complete the same test task under each parameter combination. Each test records the user's operation time, selection accuracy, subjective satisfaction score and other data.
[0132] Finally, the experimental results were analyzed to calculate the average operation time, accuracy and satisfaction scores under different weight combinations. Statistical methods such as variance analysis were used to evaluate the significant impact of different parameter combinations on operation efficiency and accuracy. Based on the analysis results, a set of parameter combinations (w1, w2, c) was determined to make the system optimal in terms of operation efficiency, accuracy and user satisfaction.
[0133] Step 32: Control the state of the interactive object.
[0134] The state control of the interactive object includes lock state control and gaze state feedback.
[0135] The lock state control can set a threshold according to the lock attribute value of the interaction intention. When the threshold is reached, the state of the interactive object is locked, and the state unlocking time is set. When the unlocking time limit is reached, the locked state of the interactive object is unlocked.
[0136] Gaze state feedback updates the state of an interactive object based on whether the eye movement point falls on the interactive object. When the eye movement point falls on the interactive object, the state of the interactive object is updated to the hover state. This kind of feedback can make users feel more in control.
[0137] Step 33: Select the interactive object.
[0138] The selection command can be interactive actions of different channels such as key operation, blinking, foot pedaling, tooth clicking, voice, etc. When the interactive object is in the locked state and listens to the selection command, the interactive object will change to the triggered state, that is, the interactive object is selected.
[0139] To verify the solution of the present invention, the present invention provides the following test content:
[0140] Experimental environment and participants
[0141] The laboratory controlled the ambient light, with moderate brightness and close ambient light at different times. The laboratory was a separate space with a quiet environment. The room temperature was kept at around 20 degrees Celsius to create a good experimental environment for the subjects. Ten subjects were invited to this experiment. All the subjects had good eyesight and no physical factors that could easily cause reading difficulties, such as color blindness and color weakness. Before the experiment, it was confirmed that the subjects had had enough rest yesterday and had no eye fatigue or discomfort.
[0142] Experimental Procedure
[0143] Before the experiment began, participants received a brief introduction to the experimental background and process. Participants were required to sign a consent form to participate in the experiment. Then they were required to complete the eye movement calibration to ensure that the system could accurately track the participants' eye movements. Participants completed at least 6 training trials on the smooth pursuit interaction method before the formal experiment began to familiarize themselves with the experimental process. The experimental tasks and interaction methods in the training phase were consistent with the formal experiment.
[0144] The experiment was divided into three parts, each with 32 trials. Before each of the three parts, the smooth pursuit movement period was modified according to the test experience of the subjects. Each part included all the level combinations of the two independent variables, and the order of the combinations was randomized. A two-minute rest period was allowed after each stage. The following is an example of a single trial.
[0145] In the first step, a “+” sign appeared in the center of the screen. Participants were asked to fixate on the plus sign for 1000 milliseconds until it disappeared.
[0146] In the second step, the target interactive object with smooth tracking appeared on the screen. Participants were asked to track the moving object of the interactive object for 3 seconds, and the system automatically entered the next step.
[0147] In the third step, at the end of the task, the screen showed a gray blank background for 200 milliseconds. Then, the next task began until the entire experiment was completed.
[0148] The experimental steps for a single trial are as follows: Figure 5 The entire experiment lasted approximately 20-30 minutes for each participant.
[0149] Experimental design
[0150] An 8×2×2 within-subject design was used: 8 trajectory shapes, including Line, Zigzag, Ellipse, V, ↑→ (Upper Right), →↓ (Left Lower), ↓← (Lower Left), ←↑ (Left Right), two interaction object sizes (1.05°, 0.53°), and two smooth pursuit motion periods set in each of the three experimental parts, with a period between 0.8s and 2.5s. Each condition combination of the subtask was executed once. Trajectory shapes are shown in Figure 2. Figure 6 In order to reduce the order effect caused by the same trajectory, the order of experimental condition combinations was randomized. There were 8×2×2×3×10=960 samples in total, and 3×10=30 samples in a single experimental condition.
[0151] There are two sizes of smooth tracking interactive trajectories, the vertical distance of the large-size trajectory is 1.05°, and the vertical distance of the small-size trajectory is 0.53°. There are 8 shapes of smooth tracking interactive trajectories, including Line, Zigzag, Ellipse, V-shaped, ↑→ shape (Upper Right), →↓ shape (Left Lower), ↓← shape (Lower Left), ←↑ shape (Left Right). The first four of these 8 trajectory shapes are common forms of smooth tracking, and they are reciprocating motions, and the last four are unidirectional motions. The motion cycle of the smooth tracking moving object ranges from 0.8s to 2.5s. This comes from the 0.8-1.2s flashing cycle of the computer cursor, which is adjusted according to the feedback from the subjects in the pre-experimental test. When the subject indicates that the movement speed of the moving object in the previous test was too fast or too slow, the movement cycle of the moving object in the next test group will be appropriately adjusted.
[0152] The attributes collected in the dataset include timestamp, gaze point coordinates (focus_point_x, focus_point_y, focus_point_z), and moving object coordinates (IO_point_x, IO_point_y, IO_point_z). The experimental sampling rate is 30 Hz, the sampling time is 3 s, and each sample has 3×30=90 sampling points.
[0153] Analytical methods
[0154] In this study, a systematic approach was taken to evaluate the impact of different algorithm parameters (algorithm type, delay time) on the processing effect of time series point coordinate data. The data analysis method was divided into three main stages: input stage, processing stage and conclusion stage. Figure 7 shown.
[0155] First, the original experimental data were organized into a comprehensive test data set with the structure of {different subjects × different test conditions × repetitions × time points × interaction object coordinates × gaze point coordinates}, hereinafter referred to as the “time-sequential point coordinate data set”. This step aims to systematically integrate the experimental data for subsequent analysis. Next, a “parameter table of algorithms to be evaluated” was established, which lists all the algorithms to be evaluated and their parameters. These algorithm parameters will be used in the processing stage to perform smooth tracking detection on the data in the time-sequential point coordinate data set.
[0156] In the processing stage, each item in the algorithm parameter table to be evaluated is traversed, and smooth tracking detection is performed on each sub-segment in the time-series point coordinate dataset. Each detection uses the specific conditions of the algorithm parameter table to be evaluated (such as the xth row) to obtain a motion similarity score table related to the sub-segment time-series points. These sub-score tables are then merged into a comprehensive "time-series motion similarity score dataset". Through the analysis of the time-series motion similarity score dataset, this study extracted a comprehensive detection result dataset for the samples. This dataset contains a variety of key evaluation indicators, such as average similarity, maximum similarity, minimum similarity, similarity distribution, threshold comparison, similarity change trend and other parameters. These indicators provide a comprehensive perspective for evaluating the effects of different algorithm parameter settings.
[0157] In the conclusion stage, this study used a visual analysis method to show the evaluation results at each detection level, including the changes in other sample detection dependent variables with the delay time. This not only allows for an intuitive evaluation of the impact of different algorithm parameters on the processing effect, but also provides a basis for further optimizing the algorithm parameters.
[0158] Through this series of steps, this study not only analyzes in detail the impact of different algorithm parameters on the processing effect of time series point coordinate data, but also provides a systematic evaluation method for future related research.
[0159] (1) Pearson correlation coefficient algorithm with delay correction
[0160] This is a description of an algorithm for calculating the Pearson correlation coefficient with delay for motion similarity detection. Here is a description of the algorithm:
[0161] Input: Dataset data containing the x and y coordinates of the focus and IO points; sliding window size window_size; delay time delay_Time (in milliseconds)
[0162] Output: An array of minimum Pearson correlation coefficients for each window considering the delay, correlations
[0163] Here are the steps:
[0164] 1) The sample length of the initialized data set is n_samples.
[0165] 2) Initialize an empty list correlations.
[0166] 3) If delay_Time is equal to 0, set delaySample to 0. Otherwise, calculate delay sample delaySample=calculate_jump_delay_sample(delay_Time).
[0167] 4) For each starting index start from 0 to n_samples-window_size-delaySample+1:
[0168] ① Calculate the end index end = start + window_size.
[0169] ② Calculate the delayed start and end indexes of the focus data: start_dly = start + delaySample, end_dly = end + delaySample.
[0170] ③ Calculate the Pearson correlation coefficient of the delayed focus and the y coordinate of the current IO point: correlation_y.
[0171] ④ Calculate the Pearson correlation coefficient of the delayed focus and the x-coordinate of the current IO point: correlation_x.
[0172] ⑤Find the minimum of the two correlation coefficients: min_correlation.
[0173] ⑥Append min_correlation to the list correlations.
[0174] Returns: Convert the list correlations to an array and return it.
[0175] The algorithm traverses the dataset using a sliding window and introduces delay processing to simulate possible time differences in real situations. It calculates the Pearson correlation coefficient of the x and y coordinates of the focus and IO points after considering the delay, allowing for a more accurate detection of motion similarity. The algorithm finally outputs an array of the minimum Pearson correlation coefficients for each window after considering the delay.
[0176] (2) Euclidean distance algorithm with delay correction
[0177] This is a description of the algorithm for calculating the Euclidean distance with delay for motion similarity detection. Here is a description of the algorithm:
[0178] Input: data set data containing focus and IO point coordinates; sliding window size window_size; delay time delay_Time (in milliseconds)
[0179] Output: The concentrated Euclidean distance array distances calculated for each window taking into account the delay
[0180] Here are the steps:
[0181] 1) The sample length of the initialized data set is n_samples.
[0182] 2) Initialize an empty list distances.
[0183] 3) If delay_Time is equal to 0, set delaySample to 0. Otherwise, calculate the number of delay samples according to delay_Time: delaySample=calculate_jump_delay_sample(delay_Time).
[0184] 4) For each starting index start from 0 to n_samples-window_size-delaySample+1:
[0185] ① Calculate the end index end = start + window_size.
[0186] ② Calculate the start and end indexes of the delay: start_dly = start + delaySample, end_dly = end + delaySample.
[0187] ③ Consider the delayed extraction of window data, which are the x and y coordinates of the focus and IO points: focus_data_x, focus_data_y of the delayed window, and io_data_x, io_data_y of the original window.
[0188] ④ Center the data by subtracting the mean: focus_centered_x, focus_centered_y, io_centered_x, io_centered_y.
[0189] ⑤ Calculate the Euclidean distance between the x-coordinate and y-coordinate of the focal point and the IO point after concentration: distance_x, distance_y.
[0190] ⑥ Combine the distances of x and y to calculate the total distance of the window and normalize it: distance.
[0191] ⑦Append the calculated distance to the list distances.
[0192] Returns: Convert the list distances to an array and return it.
[0193] This algorithm introduces delay processing in the process of sliding window traversal of the data set to simulate the time difference that may exist in actual situations. It first calculates the number of delayed samples based on the delay time, and then extracts the coordinate data of the focal point and IO point considering the delay for each window. Next, the data is concentrated by subtracting the average value, and the Euclidean distance between the concentrated focal point and IO point is calculated. Finally, the algorithm outputs the concentrated Euclidean distance array for each window considering the delay.
[0194] (3) Analysis indicator description
[0195] When analyzing the time series point coordinate data, the following nine key indicators were selected to evaluate similarity and change trends. These indicators together form a comprehensive analysis framework for measuring and explaining the similarity characteristics and dynamic changes of time series data.
[0196] 1) The average similarity (mean) provides a measure of the average level of the entire time series similarity data, reflecting the overall similarity of the data.
[0197] 2) Maximum similarity (max) identifies the points with the highest similarity in the time series data, indicating the most matching part of the time series.
[0198] 3) Minimum Similarity (min) finds the point with the lowest similarity in the time series data, showing the point with the greatest difference.
[0199] 4) Similarity distribution (var) shows the dispersion of similarity data through distribution graphs, helping to identify the central tendency and potential outliers of the data.
[0200] 5) The first threshold time (first_threshold_time) refers to the specific moment when the time series data first reaches the preset threshold, which is helpful for understanding the speed at which the data reaches the critical state.
[0201] 6) The total time to reach the threshold (total_threshold_time) indicates the length of time that the data as a whole meets or exceeds a specific threshold, which is used to evaluate the persistence of the data at a key similarity level.
[0202] 7) The number of times the threshold is reached (threshold_crossings) records the number of times the data crosses the threshold, reflecting the frequency of data fluctuations.
[0203] 8) Volatility measures the fluctuation of time series similarity data in the form of standard deviation, describing the stability and range of variation of the data.
[0204] 9) Slope is quantified by the linear regression slope of the time series. The positive and negative values of the slope represent an upward or downward trend in the data, respectively.
[0205] Experimental Results
[0206] To analyze different smooth pursuit check methods and delay parameters, the following line graph was drawn, see Figure 8 The horizontal axis is the delay time in milliseconds (ms), and the vertical axis is the analysis indicators, including average similarity, maximum similarity, minimum similarity, similarity distribution, time to first reach the threshold, total time to reach the threshold, number of times to reach the threshold, volatility, and trend. The different colored lines in each figure represent different trajectory shapes. 1, 2, 3, 4, 6, 7, and 9 correspond to straight line, rectangle, zigzag, ellipse, upper left, upper right, and lower left, respectively.
[0207] Under the Pearson correlation coefficient algorithm, the analysis of each indicator shows that: in the average similarity indicator (mean), the straight track is better than other track shapes before the delay is 350 milliseconds; in the maximum similarity indicator (max), the straight track is better than other track shapes within the delay of 0-400 milliseconds; and in the minimum similarity indicator (min), the rectangular track has the smallest similarity; in the similarity distribution indicator (var), the straight track has a greater value than other track shapes within the delay of 0-350 milliseconds, and reaches a peak at the delay of 100-150 milliseconds; in the first threshold time indicator (first_threshold_time), the straight track is stable and It is superior to other trajectory shapes; in the total_threshold_time indicator, the straight line trajectory is much higher than other trajectory shapes when the delay is 0-400 milliseconds, and reaches the maximum value when the delay is about 100 milliseconds; and the number of times the threshold is reached (threshold_crossings) The indicator shows a similar situation to the total time to reach the threshold; in the volatility indicator, the volatility of the straight line trajectory is greater than that of other trajectory shapes, which is similar to the similarity distribution indicator; finally, in the trend indicator, the upward trend of the straight line trajectory is obvious, and it maintains a positive value within the delay of 0-400 milliseconds, reflecting the robustness and advantages of the straight line trajectory.
[0208] Under the Euclidean distance algorithm, the analysis of various indicators shows that: in the average similarity (mean) indicator, the straight track has a smaller value than other track shapes before the delay of 300 milliseconds, showing a smaller deviation, and is therefore superior to other shapes in this respect; in the maximum similarity (max) indicator, the straight track is relatively stable within the delay of 0-300 milliseconds and performs well; and in the minimum similarity (min) indicator, the straight track performs better than other track shapes within the delay of 0-200 milliseconds and is the best choice; in the similarity distribution (var) indicator, the straight track has a smaller value than other track shapes within the delay of 0-350 milliseconds and is also in a good position; in the first threshold time (first_threshold_time) indicator, the zigzag track has the lowest value , indicating that it can reach the detection threshold the fastest; in the total_threshold_time indicator, the straight track is much higher than other track shapes within a delay of 0-400 milliseconds, and performs well, although it decreases slightly with the increase of delay time; and in the threshold_crossings indicator, it presents a similar situation to the total_time indicator; the volatility indicator shows that the straight track is lower than other track shapes within a delay of 0-400 milliseconds, indicating that its similarity value fluctuates less; finally, in the slope indicator, the similarity value of the straight track shows a downward trend within a delay of 0-400 milliseconds, but the area is smaller than other track shapes, and it performs well.
[0209] In general, the straight line shape is better than other trajectory shapes in terms of average value, maximum value, minimum value, time to first reach the threshold, total time to reach the threshold, volatility, trend and other indicators.
[0210] To analyze different smooth pursuit check methods and delay parameters, the following line graph was drawn, see Fig. 9 and Fig.10 As shown. Similarly, the horizontal axis is the delay time in milliseconds (ms), and the vertical axis is the analysis indicators, including average similarity, maximum similarity, minimum similarity, similarity distribution, time to first reach the threshold, total time to reach the threshold, number of times to reach the threshold, volatility, and trend. The different colored lines in each figure represent the trajectory size (m) and motion period (s). The mapping of value-color is shown in the legend. In order to simplify the description, the medium trajectory size is set to 13.4mm and the small trajectory size is set to 6.7mm.
[0211] In the Pearson algorithm with trajectory size as a categorical variable, the analysis of each indicator shows the following trends: In terms of the average similarity (mean) indicator, the correlation coefficient mean of the medium size is higher than that of the small size before the delay of 150ms, so the medium size performs better; however, after the delay of 150ms, the indicator of the medium size is lower than that of the small size. In the maximum similarity (max) indicator, the value of the small size is always higher than that of the medium size, but both are greater than 0.94. In the minimum similarity (min) indicator, the value of the medium size trajectory before 150ms is higher than that of the small size, although both are negative. In the similarity distribution (var) indicator, the value of the medium size is larger than that of the small size, and it shows a peak at 100-150ms. In the first threshold time (first_threshold_time) indicator, the value of the small size is generally smaller than that of the medium size, so the small size has a certain advantage. In terms of the total time to reach the threshold (total_threshold_time) indicator, before the delay time is 200ms, the total time of the medium size is longer than that of the small size, showing the advantage of the medium size. The volatility indicator shows that before the delay is 100ms, the fluctuation of the medium size is smaller than that of the small size, so the medium size is more stable. The fluctuation of the medium size reaches its maximum value when the delay is 150ms. Finally, in the slope indicator, the values of the medium and small sizes are both positive, but the growth trend of the small size is more obvious.
[0212] In the Euclidean distance algorithm with trajectory size as a categorical variable, the analysis of each indicator shows the following trends: In terms of the average similarity (mean) indicator, the mean value of the medium size is higher than that of the small size when the delay is 0-400ms, so the small size performs better. In the maximum similarity (max) indicator, the value of the small size is basically higher than that of the medium size, although the difference between the two values is small. In the minimum similarity (min) indicator, the medium size is higher than the small size when the delay is 0-400ms, and the small size performs better. The similarity distribution (var) indicator shows that the value of the medium size is smaller than that of the small size, indicating that the medium size is more stable. In the first threshold time (first_threshold_time) indicator, the values of the small size are all smaller than the medium size, indicating that the small size has an advantage in this regard. In terms of the total time to reach the threshold (total_threshold_time) indicator, the medium size is always longer than the small size, showing the advantage of the medium size in the time to reach the threshold. In the volatility (volatility) indicator, the volatility of the medium size is smaller than that of the small size, indicating that the data volatility of the medium size is smaller. Finally, in the trend (slope) indicator, the values of medium and small sizes are both negative, but the downward trend of small size is more obvious.
[0213] In the Pearson algorithm with motion period as a categorical variable, the analysis of each index shows the following trends: in terms of the average similarity index (mean), the mean of the period of 1 second is the lowest and the performance is the worst; while the mean of the periods of 1.5 seconds and 2.0 seconds shows a trend of increasing first and then decreasing with the increase of delay time, reaching a peak at 100-150ms; and when the period is 3 seconds, the mean decreases with the increase of delay time. In terms of the maximum similarity (max) index, the maximum similarity of the period of 3 seconds is much lower than that of the other four conditions, and the other four conditions slowly decrease with the increase of delay time; while the value of the period of 1.5 seconds is more stable, and the maximum value is greater than that of other conditions. In terms of the minimum similarity (min) index, the minimum values of the periods of 1 second and 1.5 seconds are smaller than those of other conditions. The similarity distribution (var) index shows that the values of the periods of 1.5 seconds and 2 seconds are smaller, indicating less volatility. In the indicator of first_threshold_time, the value of the condition with a period of 1 second is the smallest, indicating that the threshold is reached first, but after a delay of 50ms, the value rises rapidly; the value of the condition with a period of 1.5 seconds is the second smallest, and the indicator slowly decreases after a delay of 50ms. In the indicator of total time to reach the threshold, the condition with a period of 1.5 seconds is much larger than other conditions, indicating that this condition has the longest time to reach the threshold, and the indicator under this condition gradually decreases after a delay of 50ms. The volatility indicator shows that the volatility indicators with periods of 1 second, 1.5 seconds, and 2 seconds are all small in the delay range of 0-150ms. Finally, in the slope indicator, the indicator of each condition is positive. In the delay range of 0-300ms, the values of the indicators with periods of 1.5 seconds, 2.0 seconds, and 3.0 seconds are the top three, indicating that the increase rate of the indicator is relatively fast.
[0214] In the Euclidean distance algorithm with motion period as a categorical variable, the analysis of each indicator shows that: in terms of the average similarity (mean) indicator, the mean value is the lowest when the period is 3 seconds, which means that the indicator performs best at 3 seconds, while the values of the indicator for other conditions are close. In terms of the maximum similarity (max) indicator, the periods of the first three maximum values are 1 second, 1.5 seconds, and 2 seconds, respectively, which means that shorter periods, that is, faster movements, will lead to greater Euclidean distance deviations. In terms of the minimum similarity (min) indicator, the delay is in the range of 0-300 milliseconds, and the period conditions of the first three minimum values are 1 second, 1.5 seconds, and 2 seconds. In terms of the similarity distribution (var) indicator, the indicator is the smallest when the period is 1.5 seconds, indicating that the indicator performs best under the period of 1.5 seconds. In terms of the first threshold reaching time (first_threshold_time) indicator, the indicator is the smallest when the period is 1.5 seconds, indicating that the period of 1.5 seconds is the best in reaching the threshold. In terms of the total_threshold_time indicator, the indicator is the largest when the period is 1.5 seconds, indicating that the 1.5-second period performs best in terms of the time to reach the threshold condition. In terms of the volatility indicator, the 1-second and 1.5-second periods are the two conditions with the smallest indicators, which means that the 1.5-second period is better in terms of volatility. Finally, in terms of the slope indicator, the indicators of all conditions are negative, indicating that the indicator shows a downward trend as the delay time increases. The 1-second and 1.5-second periods are the two conditions with the largest indicators, so they are relatively good conditions.
[0215] Thr2 is used to determine whether the motion correlation reaches the motion correlation threshold. According to the experimental results, when the Pearson correlation coefficient is used to detect smooth tracking, the motion correlation threshold of 0.5 can achieve better detection results. This result can be seen from Figure 8 , Fig. 9 , Fig.10 It can be seen from the first sub-graph "delay time-similarity mean" (delayTime-mean) of the algorithm "Pearson". In these three sub-graphs, when the delay time is 50-150ms, the similarity mean is around 0.5, so it can be concluded that the recommended value of the motion correlation threshold is 0.5.
[0216] Example 2
[0217] This embodiment provides an interactive device, including:
[0218] one or more processors;
[0219] A memory for storing one or more programs;
[0220] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the eye-touch small target object selection method based on interactive locking, hybrid gaze smooth tracking, and provided in Example 1.
[0221] Example 3
[0222] This embodiment provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the eye-touch small target object selection method based on interactive locking and hybrid gaze smooth tracking provided in Embodiment 1.
Claims
1. An eye-touch small target object selection method based on interactive locking hybrid gaze smooth tracking, characterized in that: The following steps are involved: Step 1: Create interactive objects; Step 2: Obtain gaze instructions and smooth pursuit instructions for eye movements; Step 3: According to the acquired gaze instructions and smooth tracking instructions, the interaction intention locking attribute of each interaction object is calculated, and the interaction state of the eye-touch small target object is determined according to the calculated interaction intention locking attribute.
2. The eye-touch small target object selection method based on interactive locking and hybrid gaze smooth tracking according to claim 1 is characterized in that: In step 3, based on the acquired gaze instructions and smooth tracking instructions, the interaction intention locking attribute of each interaction object is calculated as: Wherein: IL is the interaction intention; E1 is the gaze instruction; E2 is the smooth pursuit instruction; a1 is the gaze proximity index; a2 is the motion similarity index; w1 is the gaze instruction weight, w2 is the smooth pursuit instruction weight, which is used to regulate the influence of the two interaction methods; c is a constant, indicating that the interaction intention decreases over time; t is the sampling time.
3. The eye-touch small target object selection method based on interactive locking and hybrid gaze smooth tracking according to claim 2 is characterized in that: The gaze proximity index a1 is: Among them, a 1,min and a 1,max is the minimum and maximum value of the gaze proximity index a1; d is the Euclidean distance between the gaze point and the center point; Thr1 is the threshold used to determine whether the current gaze point is close to the center point.
4. The eye-touch small target object selection method based on interactive locking and hybrid gaze smooth tracking according to claim 2 is characterized in that: Adjust the motion similarity index a2 according to the smooth tracking instruction E2: Determine the smooth tracking instruction E2. The smooth tracking instruction E2 is used to indicate whether the threshold of the smooth tracking instruction is reached. The calculation formula is as follows: Wherein, C is the current motion correlation value; Thr2 is used to determine whether the motion correlation reaches the similar motion correlation threshold; When the smooth tracking instruction E2 = 0, the motion similarity index a2 is set to 0; when the smooth tracking instruction E2 = 1, the motion similarity index a2 is adjusted according to the motion correlation value, and the formula is as follows: a2=C Wherein, C is the motion correlation value.
5. The eye-touch small target object selection method based on interactive locking and hybrid gaze smooth tracking according to claim 2, characterized in that: The gaze proximity index a1 is: Define a constant value a 1,constant It is used to adjust the gaze proximity index a1 and determine whether the gaze instruction E1 is used to indicate whether the gaze instruction threshold is reached. The calculation formula is as follows: Where d is the Euclidean distance between the gaze point and the center point; Thr1 is the threshold used to determine whether the current gaze point is close to the center point; When the gaze instruction E1=1, the gaze proximity index a1 is set to a constant value a 1,constant ; When the gaze instruction E1=0, the gaze proximity index a1 is set to 0.
6. The eye-touch small target object selection method based on interactive locking and hybrid gaze smooth tracking according to claim 2 is characterized in that: The weights w1, w2 and constant c are used to find the optimal parameter combination by testing the effects of different weight combinations (w1, w2) and constant c on user operation efficiency and accuracy.
7. The eye-touch small target object selection method based on interactive locking and hybrid gaze smooth tracking according to claim 6 is characterized in that: The specific method to find the optimal parameter combination is: Select a test sample of representative users; use a head-mounted display device with an eye tracker to complete the task of selecting a small target; record the user's operation time, selection accuracy, and subjective satisfaction rating data for each test; Finally, the experimental results were analyzed to calculate the average operation time, accuracy and satisfaction scores under different weight combinations; the significant impact of different parameter combinations on operation efficiency and accuracy was evaluated; based on the evaluation results, a set of parameter combinations (w1, w2, c) was determined to make the system optimal in terms of operation efficiency, accuracy and user satisfaction.
8. The eye-touch small target object selection method based on interactive locking and hybrid gaze smooth tracking according to claim 4, characterized in that: In step 3, the interaction state of the eye-touch small target object is determined according to the calculated interaction intention lock attribute, including: Control the state of the interactive object; the state control of the interactive object includes lock state control and gaze state feedback; the lock state control sets a threshold according to the lock attribute value of the interactive intention, and the state of the interactive object is locked when the threshold is reached, and the state unlocking time is set. When the unlocking time limit is reached, the locked state of the interactive object is unlocked; the gaze state feedback is to update the state of the interactive object according to whether the eye movement point falls on the interactive object; when the eye movement point falls on the interactive object, the state of the interactive object is updated to the hovering state; Select the interactive object; the selection command is a key operation, blinking, foot pedaling, tooth clicking or voice interaction action. When the interactive object is in a locked state and a selection command is heard, the interactive object will change to a triggered state, which means that the interactive object is selected.
9. An interactive device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the eye-touch small target object selection method based on interactive locking, hybrid gaze smooth tracking as described in any one of claims 1-8.
10. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the eye-touch small target object selection method based on interactive locking, hybrid gaze smooth tracking, and the like as described in any one of claims 1 to 8 are implemented.
