Gaze interaction method for accelerating VR target selection

Abstract

The invention discloses a staring interaction method for accelerating VR target selection, and relates to the technical field of man-machine interaction interface design, and the method comprises the steps: S1, designing a target-assisted triggered annular menu layout and a target-assisted staring method; s2, performing verification and technical comparison on a target selection task triggered based on the target auxiliary staring; and S3, a coding optimization strategy of the auxiliary trigger graph in the high-density space. According to the staring interaction method for accelerating VR target selection, a TA-fixation method is introduced into VR eye control interaction so as to relieve the problem of "Midas touch", and specifically, an efficient famous annular menu is adopted, and an auxiliary trigger object is designed to provide feedback and confirmation for IO. After a user selects one IO from the pie-shaped menu, the user can watch the auxiliary triggering objects to confirm selection of the auxiliary triggering objects, the auxiliary triggering objects are positioned in the unity environment through the coordinate point of the center position, the adjustable accuracy threshold value of continuous position detection is set to be 0.9, and the situation that the triggering accuracy is disturbed by blink and eye jump visual behaviors is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of human-computer interaction interface design, and in particular to a gaze interaction method for accelerating VR target selection. Background Art

[0002] Virtual reality head-mounted displays (VR HWDs) are evolving into a fundamental platform for general consumers to access and engage with digital content, thanks to advanced sensor technology and increased price. The immersive nature of VR places higher demands on the naturalness and cognitive efficiency of human-computer interaction behaviors. However, traditional interaction methods, such as external controllers, buttons, or gestures, may no longer be suitable for user experience or efficiency. In addition, with the continuous advancement of eye tracking technology, eye-controlled interaction has become a significant solution for hand-free interaction in VR / AR. Compared with hand- or button-based methods, eye-controlled interaction has two key advantages: speed and intuitiveness, with eyeballs rotating at speeds exceeding 300° / s in specific movements and consistent with natural and effortless human behavior. Its importance extends to individuals with specific diseases (such as ALS or hand dysfunction) or whose hands are occupied in daily scenarios. Recently, the launch of Apple Vision Pro has aroused interest in and potential progress in gaze-based interaction technology, which enables users to interact with virtual objects through eye-hand coordination.

[0003] Interaction target triggering tasks are often used in eye-controlled interactions, requiring users to search and locate specific interactive objects on the interface and trigger this IO using specific gaze input technologies. Common gaze input technologies include gaze, tracking, gaze gestures, and blinking, but current gaze input technologies face several challenges, including the "Midas touch" problem, accuracy issues in high-density spaces, and possible eye fatigue due to prolonged use of the eyes.

[0004] So we propose a gaze interaction method to speed up VR target selection in order to solve the problems mentioned above. Summary of the invention

[0005] The purpose of the present invention is to provide a gaze interaction method for accelerating VR target selection, so as to solve the "Midas touch" problem in the current market, the accuracy problem in high-density space, and the problem of eye fatigue caused by long-term use of the eyes proposed by the above-mentioned background technology.

[0006] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: a gaze interaction method for accelerating VR target selection, including S1, designing a target-assisted triggering circular menu layout and a target-assisted gaze method; S2, target selection task verification and technology comparison based on target-assisted gaze triggering; S3, coding optimization strategy for auxiliary triggering graphics in high-density space;

[0007] S1. Design of target-assisted triggering circular menu layout and target-assisted gaze method

[0008] Complete the layout setting of the eye-controlled interactive target and the auxiliary trigger graphic. The auxiliary trigger graphic is located outside the extension line of the center point of the menu and the center point of the interactive target, and maintains a position of 0.5 times the width of the interactive target. The shapes of the auxiliary trigger graphic and the interactive target are both ring-shaped, and the width of the auxiliary trigger graphic is 0.5 times the width of the interactive target.

[0009] The triggering of the interactive target needs to meet two processes: first, monitor the overlap between the subject's gaze point and the interactive target position within 400ms to trigger the color change feedback of the auxiliary trigger graphic; second, detect the direction of the subject's gaze trajectory shifting to the auxiliary trigger graphic area, and detect the overlap between the subject's gaze point and the auxiliary trigger graphic position within 300ms to trigger the triggering behavior of the interactive target;

[0010] S2. Target selection task verification and technology comparison based on target-assisted gaze triggering

[0011] Design circular menu layouts under different spatial densities, conduct target triggering task verification based on target-assisted gaze triggering, and compare with other eye-controlled triggering technologies to determine the advantageous and inferior spatial triggering states of target-assisted gaze triggering;

[0012] For the eye control interaction interface, an interactive target triggering task is set, the icon of the circular menu is designed as the interactive target, and different spatial layout forms are designed, including the interactive target density and the spatial distribution area of ​​the menu; and the eye control selection steps and the platform data collection under the set task are determined, such as trigger time, trigger accuracy, and pupil diameter data. Combined with the experimental platform in step S1, the interactive target triggering task is carried out, and the proposed target-assisted gaze eye control triggering method is compared with the traditional gaze triggering method and eye gesture triggering method. The eye control interaction experimental process including recruiting subjects, practice experiments, and formal experiments is clarified, and the experimental platform data is collected to compare the adaptation scenarios of different methods;

[0013] S3. Coding optimization strategy for auxiliary trigger graphics in high-density space

[0014] The inferior triggering spatial state obtained in step S2 is optimized to enhance the comprehensive triggering performance of the target-assisted gaze method. Specifically, the inferior triggering spatial state is simulated, multi-level auxiliary triggering graphics encoding (spatial position and size) is set, the superior encoding method for enhancing the triggering efficiency is explored, and a coding strategy for multi-dimensional target-assisted gaze triggering is proposed.

[0015] Preferably, the circular layout menu interface design captures the position, angle and movement speed data of the user's head in real time through multiple high-sensitivity inertial measurement units (IMUs) and depth cameras integrated in a virtual reality (VR) helmet, and then dynamically adjusts the position and direction of the menu interface, thereby achieving synchronous binding of the menu interface with the user's head movement, ensuring that when the user moves his head, the menu interface can respond instantly and move accordingly in the user's field of view.

[0016] Preferably, the annular menu is composed of an interactive target and an auxiliary trigger graphic, wherein the interactive target is located inside the annular menu, and the auxiliary trigger graphic is located outside the annular menu, the center of the auxiliary trigger graphic is located at the connecting line between the center of the menu and the center of the interactive target, and the two interactive targets and the auxiliary trigger graphic maintain a spacing of 0.5 times the width of the interactive target;

[0017] The interactive target and auxiliary trigger graphics are set to gray, and one of the randomly appearing blue icons is set as the interactive target to be triggered.

[0018] Preferably, the triggering process design of the target-assisted gaze triggering target;

[0019] The visible area of ​​the interactive target and the auxiliary trigger object is the gaze detection area, in which the auxiliary trigger graphic serves as a visual cue and "confirmation button" for the trigger process. The specific trigger process is that the user first gazes at the blue trigger target, and monitors the overlap between the subject's gaze point position and the interactive target position area within 400ms to trigger the color change feedback of the auxiliary trigger graphic; if the subject's gaze trajectory direction is detected to be transferred to the auxiliary trigger graphic area, and the subject's gaze point position coincides with the auxiliary trigger graphic position within 300ms to trigger the trigger behavior of the interactive target, the interactive target will turn green if the trigger is successful.

[0020] Preferably, the menu spatial layout design is achieved by adjusting the interactive target density and the spatial distribution area of ​​the menu;

[0021] Set different numbers of interactive targets (3, 4, 5, 6, 7 and 8) in the same area; set the spatial distribution area of ​​the menu (defined as 60° field of view angle and 45° field of view angle area).

[0022] Preferably, the experiment uses a target triggering task, requiring participants to quickly and accurately find the blue interactive target in the circular menu;

[0023] After the participant finds the blue target button, the target trigger is completed. If the correct button is triggered, the button will turn green, otherwise, it will turn red. The experimental platform records the trigger time, trigger accuracy, and pupil diameter data.

[0024] Preferably, the experimental task execution stage;

[0025] The subjects performed an eye-controlled target triggering experiment. The experiment was divided into three groups. The three groups used target-assisted gaze triggering method, traditional gaze triggering (a method that only detects regional overlap), and eye gesture triggering (a method that detects the trajectory of line of sight movement). During the triggering process, a black fixation cross ("+") was first displayed for 500 milliseconds, followed by a blank screen for 1000 milliseconds, and then a triggering interface with a circular menu appeared. At this time, the timer started. The participant's task was to find the blue target button and trigger it. If the target icon was successfully triggered, the icon turned blue. If the non-target icon was triggered, the icon turned red. Once the target was triggered, the timer stopped, and the experimental program recorded the triggering time, and then the next trial began.

[0026] The experimental task ended at the prompt stage;

[0027] After the user completes the experiment, the center of the screen prompts that the experiment is complete, and the experimenter helps the user take off the VR device and fill in the NASA-TLX questionnaire and ASQ questionnaire;

[0028] Experimental data processing and analysis;

[0029] The experiment collected data on trigger time, trigger accuracy, and pupil diameter. The trigger time in the experiment was obtained by a program timer, which was the interval between the appearance of the search interface and the moment when the participant triggered the interaction target. Based on the trigger time, trigger accuracy, and pupil diameter, five virtual interface designers preliminarily screened out the advantageous spatial triggering state and inferior spatial triggering state of target-assisted gaze triggering.

[0030] Preferably, the disadvantageous trigger space result of the target-assisted gaze method obtained is set by setting typical optimized spatial parameters (i.e., high-density and small space layout), setting multi-level auxiliary trigger graphic encoding (spatial position and size), and carrying out target triggering experiments under the target-assisted gaze method based on different auxiliary graphic encoding states.

[0031] Preferably, the target triggers experimental data processing;

[0032] The significance and interaction of the size and relative position of the auxiliary trigger graphics on the trigger effectiveness are analyzed, and the trigger efficiency and accuracy in each encoding situation are further analyzed and visualized. The visualization results can be divided into efficiency-first or accuracy-first encoding strategies, providing users with encoding optimization strategies to improve the target-assisted gaze method in high-density space.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] (1) Design a target-assisted gaze method and build a virtual reality eye-controlled interaction experimental platform to realize the layout of eye-controlled interaction targets and auxiliary trigger graphics and eye-controlled interaction operations, and adjust platform parameters to achieve efficient interactive target "selection" performance;

[0035] (2) The experimental platform was developed using C# and Unity engine and runs on HTC VIVE ProEye (a virtual reality helmet equipped with a Tobii eye tracking module, with a sampling rate of 90 Hz and a tracking field of view of 110°). The graphics card connected to the computer must be NVIDIA GeForce 1060 or above, the processor must be Intel i7 or above, and the memory must be 4 GB or above;

[0036] (3) A circular fixed menu is used (i.e., the menu is always within the user's field of vision). The interactive target is a circular icon, which is located in the unity environment through the coordinate point of the center. The adjustable accuracy threshold of continuous position detection is set to 0.9 to avoid blinking and eye saccade visual behaviors interfering with the trigger accuracy.

[0037] (4) Each subject's sight was calibrated using the nine-point eye control calibration program of HTC VIVE ProEye. After calibration, the experimental prompt interface was entered to remind the subject to start the experiment and pay attention to concentration. The statistical analysis software SPSS23.0 (IBM Corporation, New York, USA) was used to calculate the average trigger time of each participant. In order to ensure data reliability, the extreme values ​​in the experimental results, that is, the values ​​that deviated from the mean ± 0.5 standard deviation, were removed. Then, the three parts of the data were subjected to mixed method analysis, and main effect analysis and interaction analysis were performed for each indicator.

[0038] (5) Based on Fitts's law, we convert the trigger time and coding parameters into throughput as the evaluation index of efficiency. The evaluation indicators of this experiment include accuracy and throughput. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic diagram of construction and optimization of the target-assisted gaze triggering method of the present invention;

[0040] Figure 2 It is a schematic diagram of the target-assisted gaze triggering process of the present invention;

[0041] Figure 3 Schematic diagram of three eye-controlled triggering technologies of the present invention;

[0042] Figure 4 is a schematic diagram of the field of view and number of targets of the circular menu layout of the present invention;

[0043] Figure 5 is a flow chart of the target triggering experiment of the present invention;

[0044] Figure 6 It is the objective data result diagram of the target triggering experiment of the present invention;

[0045] Figure 7 It is a subjective data result diagram of the target triggering experiment of the present invention;

[0046] Figure 8 It is a schematic diagram of the auxiliary triggering graphic coding material change of the present invention;

[0047] Fig. 9 It is a schematic diagram of coding optimization experimental results of the present invention;

[0048] Fig.10 It is a schematic diagram of accuracy-efficiency under different encodings of the present invention;

[0049] Fig.11 It is a schematic diagram of the recommendation of the task requirement coding of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] Embodiment 1: Figure 1-Figure 11 The technical solution shown in the figure, the present invention provides the following technical solution: a gaze interaction method for accelerating VR target selection, disclosed:

[0052] The experimental program was created using Unity and C# and ran on a HTC VIVE ProEye, a VR headset equipped with a Tobii eye-tracking module. The headset records eye-tracking information at a sampling rate of 90Hz and provides a 110° field of view. It is connected to a desktop computer powered by an Intel Core i7-10800F and an Nvidia GeForce RTX2060 graphics card.

[0053] In order to reduce the gaze bias during gaze triggering, we adopted the gaze detection tolerance method, that is, setting an adjustable and precise threshold for continuous position detection to ensure that the target is successfully triggered when the threshold is reached. In the formal experiment, the threshold was adjusted to 90%, allowing up to 10% of the data to be outside the trigger area within the specified dwell time. The form of a circular display gaze-triggered menu was adopted (that is, the menu was always within the user's field of view), and the interactive target was a circular icon, which was positioned in the unity environment by the coordinate point of the center position. In the AR / VR environment, menus can be divided into three types: surround gaze-triggered menus (that is, menus triggered by gaze in the virtual world), world gaze-triggered menus (that is, menus triggered by gaze on retrievable objects), and display gaze-triggered menus (that is, menus that follow the user's head movement). In the display gaze-triggered menu, the menu is gaze-triggered on the screen and is always within the user's field of view, providing an intuitive and fast operation method. However, in the "world-gaze-triggered menu", the menu is attached to the interactive object, which may change the distance from the menu due to the user's movement, resulting in inconsistency in experimental control. In addition, previous studies have shown that head movement is closely coupled with gaze. In this study, we wanted to mitigate the effects of unconscious head movements. Therefore, we validated our experiments in a “display-gaze-triggered menu” setting;

[0054] Design a circular menu, which consists of interactive targets and auxiliary trigger graphics. Specifically, the auxiliary trigger graphics are circular blocks similar to interactive objects, located on the side of the interactive target away from the center of the circle, and keep a distance of 0.3 times the width of the interactive object from the interactive target. There are two reasons why the auxiliary trigger object is placed on the outside of the circular menu instead of the inside: one is to prevent possible visual interference with the interactive target, and the other is to keep the consistency and intuitiveness of the gaze movement path. In terms of size, the auxiliary trigger graphic is designed to be half the width of the interactive target. The interactive target is set to gray, and one of the randomly appearing blue icons is set as the interactive target to be triggered. Target-assisted gaze triggering is an optimization design that aims to improve the accuracy of the gaze-based selection mechanism. This method uses auxiliary trigger graphics as visual cues and "confirmation buttons" for the triggering process, simplifies the selection process, and solves the problem of unintentional selection often encountered in traditional gaze-based interactions. The triggering process of the target-assisted gaze trigger target is divided into four steps: the first step is target interactive object search, which requires users to find the target interactive object in the ring menu to test their visual search ability. In order to reduce the impact of individual differences in visual search ability on the results, a prominent blue color (RGB: 87, 182, 247) was designed in the experiment. This prominent color is believed to be able to attract attention from the bottom up.

[0055] The second step, the target "selection" phase, requires the user to fixate on the target interactive object for a period of time. During this process, we set a dwell time of 400ms for the interactive target based on single trigger events (150-600ms) and test experience. The third and fourth steps, the "confirmation" phase, are completed by looking at the auxiliary trigger object. The user decides whether to trigger the interactive object based on the feedback from the auxiliary trigger object in step 2. If so, the user shifts their gaze from the interactive object to the auxiliary trigger object and maintains a dwell time of 300ms to complete the triggering of the interactive object. A successful triggering will turn the object green, while triggering a non-target object will turn it red.

[0056] Embodiment 2: Figure 1-Figure 11 The technical solution shown in the figure, the present invention provides the following technical solution: a gaze interaction method for accelerating VR target selection, disclosed:

[0057] The menu space layout design is achieved by adjusting the number of menu icons and the field of view. Set the number of interactive targets and determine the positions of 3, 4, 5, 6, 7 and 8 icons; set the interactive target position range, defined as the 60° field of view and 45° field of view areas;

[0058] The experiment used a target triggering task, requiring participants to quickly and accurately find the blue target button in the circular menu. 30 participants were recruited to perform the experiment. After the participants found the blue target button, they completed the target triggering. If the correct button was triggered, the button would turn green. Otherwise, it would turn red. The experimental platform recorded the triggering time, triggering accuracy, and pupil diameter data;

[0059] Experimental calibration and prompting stage. Each subject's sight line is calibrated through the nine-point eye control calibration program of HTC VIVE ProEye. After calibration, the experimental prompt interface is entered to remind the subject to start the experiment and pay attention to concentration;

[0060] Experimental task execution phase. The subjects performed an eye-controlled target triggering experiment and compared the designed target-assisted gaze triggering method with traditional gaze triggering and eye gesture triggering methods. In the triggering experiment, a black gaze trigger cross ("+") was first displayed for 500 milliseconds, followed by a blank screen for 1000 milliseconds, and then a trigger interface with a circular menu appeared, at which time the timer started. The participant's task was to find the blue target button and trigger it. If the target icon was successfully triggered, the icon turned blue; if the non-target icon was triggered, the icon turned red. Once the target was triggered, the timer stopped. The experimental program recorded the trigger time and then started the next trial. The experiment adopted a mixed design experimental design and used the menu button to select the task. A total of 9,000 test data were recorded (30 participants x 3 gaze input technologies x 2 field of view angles x 5 target numbers x 10 repetitions). The experimental platform recorded various indicators of each trial, including trigger accuracy, trigger time, and average pupil diameter during the selection process. Trigger time refers to the total time from the appearance of the menu interface to the completion of the trigger (including 700 milliseconds of gaze trigger time);

[0061] The end-of-experiment task prompt phase: After the user completes the experiment, the center of the screen prompts that the experiment is complete, and the experimenter helps the user to take off the VR device and fill in the "Post-Scene Questionnaire (ASQ)" and "NASA TTLX Questionnaire". The seven-point Likert scale is used to collect data on user experience and cognitive load;

[0062] Experimental data processing and analysis. The experiment collected trigger time, trigger accuracy, and pupil diameter data. The experimental procedures and questionnaires were designed to collect objective and subjective data under various conditions. The analysis of these data mainly includes two steps. First, outliers were found and then excluded from the analysis. The experimenters found and eliminated participants with significant deviations in accuracy (2 people). In addition, in each group of indicators, data points that exceeded twice the standard deviation of the mean (about 8% of the overall data) were also excluded. Secondly, a mixed design repeated measures ANOVA was performed on accuracy, which contained two within-group variables (i.e., FOV and number of targets) and one between-group variable (i.e., gaze input technology). When the sphericity assumption was violated, the Greenhaus-Geisel correction was used. These data were analyzed using SPSS Statistics 24. The results of the ANOVA test (F represents the level of significant difference between groups, and p represents the significance level of the test) are used to clarify the significance of each variable and whether the two independent variables affect each other. In this study, a P value less than 0.05 indicates statistical significance.

[0063] Experimental Results and Discussion

[0064] Trigger accuracy results: There was a significant main effect of gaze input technology (F(1,560)=512.01, p=0.000, η2 =0.434). The triggering accuracy of eye gesture input was the highest, followed by gaze-assisted triggering, while gaze-triggered triggering had the lowest triggering accuracy. Target number (F(4, 2240) = 164.87, p = 0.000, η 2 =0.227) also had a significant effect on the trigger accuracy. As the number of targets increased, the accuracy gradually decreased. The variance analysis also showed that the number of targets × eye control input technology (F(8,2240)=25.36, p=0.000, η 2 =0.083). This interaction indicates that as the number of targets increases, the trigger accuracy of the three eye control input techniques will decrease. In particular, when the number of targets exceeds 6, the accuracy of gaze triggering and target-assisted gaze triggering will decrease significantly. It can be seen that when the number of targets is the same, the accuracy of gaze triggering is the lowest, while the accuracy of gaze gesture is the highest. When the number of targets is the same, the accuracy decreases when the field of view angle changes from 60° to 45°. No interaction between FOV and gaze input technology was observed (F(2,560)=0.86, p=0.425, η 2 =0.003), nor was there an interaction between FOV and target number F(4,2240)=1.485, p=0.204, η 2 =0.003). This suggests that the effect of visual field angle on triggering accuracy did not differ significantly between gaze input techniques and target numbers.

[0065] Trigger time results: Eye control input technology (F(1,536) = 619.86, p = 0.000, η 2 =0.698) had a significant main effect on trigger time. The average results for each gaze input technique showed that both gaze triggering and target-assisted gaze triggering maintained relatively short trigger times (gaze triggering was faster than target-assisted gaze triggering), while the trigger time of gaze gesture was significantly longer than the former two. In addition, the number of targets (F(4,2144)=8.49, p=0.000, η2=0.016) and FOV (F(1,536)=10.26, p=0.001, η2=0.019) also significantly affected the trigger time. Post hoc comparisons showed that target numbers 7 and 8 were significantly different from target numbers 4, 5, and 6 (p<0.05), while there was no significant effect between target numbers 4, 5, and 6 and between target numbers 7 and 8. When the number of objects exceeded 6, the response time showed a downward trend, indicating that users would trigger the action faster in this case. There was also a significant interaction between visual angle and gaze input technology (F(2,536)=3.11, p=0.045, η 2=0.011). The triggering time of eye gesture triggering was the longest, while the triggering time of fixed gesture was the shortest. In addition, the triggering time of 60° FOV was slightly better than that of 45° FOV. However, in terms of number of targets × field of view (F(4,2144)=0.58, p=0.697, η 2 =0.001), target number × gaze input technique (F(8,2144)=1.485, p=0.120, η 2 =0.006) and target number × gaze input technique × visual angle (F(8,2144)=0.97, p=0.936, η 2 =0.001). This suggests that when these factors interact with each other, their combined effect on trigger time is not significant.

[0066] Pupil diameter data results: The results of variance analysis showed that eye control input technology had a significant main effect (F(2,539)=12.16, p=0.000, η 2 =0.043). Post hoc comparisons showed significant differences between gaze triggering and target-assisted gaze triggering (p < 0.05), and between gaze triggering and gaze gesture (p < 0.05). However, no significant differences were observed between target-assisted fixation and gaze gesture (p > 0.05). FOV (F(1,539) = 8.49, p = 0.004, η 2 =0.015) and target number (F(4,2156)=5.89, p=0.000, η 2 =0.011). Although the main effects of the two independent variables were significant, the differences in pupil diameter under different description conditions were very small, with the difference between the highest and lowest points being only 0.5. In addition, in terms of pupil diameter, the number of targets × gaze input technique (F(8,5126)=2.27, p=0.020, η 2 =0.008) and FOV×number of targets (F(4,2156)=3.06, p=0.016, η 2 =0.006). Under the same number of conditions, the pupil diameter of the gaze-triggered target slightly exceeded the pupil diameter of the other two techniques. However, there was no significant interaction between the viewing angle and the gaze input method (F(2,539)=0.49, p=0.611, η 2 =0.006) and between visual angle, target number and gaze input method (F(8,2156)=1.51, p=0.149, η 2 =0.006) had no significant interaction.

[0067] Subjective questionnaire results: The overall mean score of the ASQ scale showed that eye gesture (M=10.74, SD=4.52) scored the highest, while gaze trigger (M=8.80, SD=3.83) scored the lowest, indicating that users believed that eye gesture had the best user experience. Similar conclusions were drawn for the two ASQ indicators, satisfaction and task difficulty. When analyzing ASQ scores, repeated measures one-way ANOVA and post hoc multiple comparisons with LSD correction were used. After post hoc multiple comparisons, there was a significant difference in task difficulty between gaze trigger and eye gesture trigger (p<0.05), indicating that most subjects considered the latter to be more challenging than the former. However, the remaining indicators did not show significant differences. The total score of NASATLX showed that the highest cognitive load was eye gesture (M=24.3, SD=2.55), followed by target-assisted gaze trigger (M=22.43, SD=3.68) and gaze trigger (M=17.83, SD=3.55). Compared with the other two techniques, eye gestures showed significant improvement in physical demand (p<0.01), while gaze triggering scored significantly lower in physical demand (p<0.05), time demand (p<0.001), and effort (p<0.01) than both gaze triggering and eye gestures. This highlights the significant advantage of gaze triggering in terms of cognitive load. However, the gaze input method did not show significant differences in self-performance (p>0.05) and frustration (p>0.05). In terms of subjective ranking, the subjects believed that there was no significant difference in accuracy among the three input methods (p>0.05). However, participants significantly preferred the gaze trigger method over the gaze gesture method (preference: p>0.05).

[0068] Gaze triggering has the highest target triggering efficiency, but the lowest accuracy, and the accuracy decreases significantly with the increase in the number of targets. In contrast, eye gesture triggering has the highest accuracy, but the triggering time is slow and the user preference is low. At the same time, target-assisted gaze triggering has a more balanced overall triggering performance in terms of accuracy, triggering time, and cognitive load, and is better in terms of subjective accuracy and preference. At the same time, target-assisted gaze triggering also greatly alleviates the Midas contact problem observed in gaze triggering, especially in scenarios involving a large number of targets. In addition, the challenges posed by high-density VR spaces (i.e., small field of view and large number of targets) highlight the necessity of optimizing design solutions in eye-controlled interactions.

[0069] Embodiment 3: Figure 1-Figure 11 The technical solution shown in the figure, the present invention provides the following technical solution: a gaze interaction method for accelerating VR target selection, disclosed:

[0070] Determine the typical optimized spatial parameters (i.e., the number of menu buttons and the field of view), and set the auxiliary trigger graphic encoding parameters (the size and relative position of the graphic). In order to simulate a high-density spatial scene, we used a circular menu with eight buttons and a field of view of 45°. In terms of the menu selection task, the experimental process was consistent with the target trigger experiment. However, the encoding method of the auxiliary trigger graphic was changed, including adjusting the size and distance of the trigger target;

[0071] Conduct a target triggering experiment under the target-assisted gaze method based on different auxiliary graphic encoding states. We designed a 3 (auxiliary trigger graphic size: S, 0.8S and 0.6S) × 3 (distance between the interactive target and the auxiliary trigger graphic: 0.3D, 0.9D and 1.5D) within-group experiment. Specifically, the closer distance is 0.3 times the width of the interactive object (0.3D), and the other two distances are 0.9D and 1.5D respectively. In the menu selection task, participants will randomly see various combinations of sizes and distances. The experimental platform collects accuracy and reaction time data;

[0072] Target triggering experimental data processing. Analyze the significance and interaction of the size and relative position of the auxiliary triggering pattern on the triggering efficiency; further analyze the triggering efficiency and accuracy under each encoding condition. Based on Fitts's law, we convert the triggering time and encoding parameters into throughput as an indicator for evaluating efficiency. The calculation process of throughput is as follows:

[0073] First, the difficulty index (ID) is calculated. The Fitts difficulty index for each state is calculated using the Shannon formula, which is commonly used for tasks involving object location and selection.

[0074] ID=log2(D / W+1)

[0075] In the experiment, we used a world lock target, which provided multiple possible methods for calculating the target distance (D) and target size (W). D represents the distance from the center coordinates of the target object to the center coordinates of the auxiliary trigger object, measured in three values. Regarding W, since the auxiliary trigger objects used in the experiment were irregular in shape, we chose to use the minimum rectangle width surrounding the target for a unified representation. This representation method also contains three values.

[0076] Throughput calculation: The throughput of each trial is calculated in bits per second by calculating the ID and the movement time (MT). We focus only on throughput and only consider MT when "triggered". MT measures the time it takes for the subject to shift his gaze from the target to the auxiliary trigger object after fixating on the target. It is worth noting that MT does not include the 300 milliseconds spent on the auxiliary trigger object.

[0077] Throughput = ID / MT

[0078] In addition, the data filtering and analysis methods adopted are consistent with Experiment 1. About 6% of abnormal data points were excluded from the collected dataset, and the same mixed design repeated measures ANOVA was used. The evaluation indicators of this experiment include accuracy and throughput.

[0079] Triggering experiment results under different encoding states

[0080] Trigger accuracy results: A one-way analysis of variance (ANOVA) was performed on the trigger accuracy in all conditions. When the assumption of sphericity was violated, a Greenhaus-Geisel correction was used. The results showed a significant main effect of distance (F(2,1654) = 8.04, p = 0.000, η 2 =0.010). As the distance between the auxiliary trigger graphic and the interactive object increased, the trigger accuracy also increased. In addition, post hoc comparisons showed that there was a significant difference between 1.5D and 0.3D (p<0.05), and there was also a significant difference between 1.5D and 0.9D (p<0.05). This result suggests that there is no significant difference between 0.3D and 0.5D in terms of accuracy. However, no significant difference was observed between 0.3D and 0.9D (p>0.05). The size of the auxiliary trigger graphic (F(2,1654)=38.28, p=0.000, η 2 =0.044) also had a significant effect on trigger accuracy. In addition, the smaller the size, the higher the accuracy. The interaction between the size and distance of the auxiliary trigger object (F(2,1654)=0.55, p=0.699, η 2 =0.001) is not statistically significant.

[0081] Throughput results: The ANOVA results highlight the importance of two main effects. First, distance has a significant effect on throughput (F(2,1654)=345.44, p=0.000, η 2 =0.295), indicating that the longer the distance, the lower the throughput. In addition, a consistent decreasing pattern was shown in all three size conditions. Secondly, the size of AO (F(2,1654)=382.69, p=0.000, η 2 =0.316) also has a significant effect on throughput. The average results show that under the same distance conditions, the throughput of products with larger sizes is higher, while the throughput of products with smaller sizes is the lowest. The interaction between distance and size is also significant (F(4,1654)=5.68, p=0.000, η 2 =0.014).

[0082] The visualization results can be divided into efficiency-first or accuracy-first encoding strategies to provide users with personalized encoding recommendations. This experiment is to explore the encoding optimization strategy of auxiliary trigger objects in target-assisted gaze triggering, aiming to improve the overall triggering performance of menu targets. However, the results show that there is a conflict between accuracy-oriented encoding strategies and efficiency-oriented encoding strategies. For example, when the distance between the auxiliary gaze graphic and the interactive object is larger, it is more conducive to improving the triggering efficiency, but it will reduce the triggering accuracy. Similarly, the smaller the size of the auxiliary trigger object, the higher the triggering accuracy, but the lower the triggering efficiency.

[0083] In order to intuitively describe the accuracy and efficiency of each distance and size under different encoding strategies, we plotted accuracy-throughput scatter plots to illustrate the distribution of triggering performance for each participant. In addition, we further visualized the results. First, the coordinate system was divided into uniform block-shaped areas, and the probability density of each area was calculated using kernel density analysis, and the high-density areas were mapped to these block-shaped areas. The coordinate system was divided into uniform block-shaped areas, and the probability density of each area was calculated using kernel density analysis. The high-density areas were then mapped to these block areas and represented by color blocks. Therefore, each color block represents a specific distance-size encoding condition. Secondly, all color blocks were plotted on a graph, allowing us to intuitively see the triggering performance under all encoding conditions. Subsequently, we can pinpoint the areas on the graph where gaze triggering prioritizes accuracy or efficiency. In this way, we can derive the corresponding distance size encoding form. For example, Fig.11 The coding forms corresponding to the efficiency priority area are shown, including 0.3DS, 0.3D-0.8S and 0.9DS. In actual design work, it is crucial to determine the specific selection preference according to the task requirements and to strike a balance between the accuracy and efficiency of the triggering task. Through this process, the corresponding coding form can be determined.

[0084] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A gaze interaction method for accelerating VR target selection, including S1, designing a target-assisted triggering circular menu layout and a target-assisted gaze method; S2, target selection task verification and technology comparison based on target-assisted gaze triggering; S3, coding optimization strategy for auxiliary trigger graphics in high-density space; It is characterized by further comprising: S1. Design of target-assisted triggering circular menu layout and target-assisted gaze method Complete the layout setting of the eye-controlled interactive target and the auxiliary trigger graphic. The auxiliary trigger graphic is located outside the extension line of the center point of the menu and the center point of the interactive target, and maintains a position of 0.5 times the width of the interactive target. The shapes of the auxiliary trigger graphic and the interactive target are both ring-shaped, and the width of the auxiliary trigger graphic is 0.5 times the width of the interactive target. The triggering of the interactive target needs to meet two processes: first, monitor the overlap between the subject's gaze point and the interactive target position within 400ms to trigger the color change feedback of the auxiliary trigger graphic; second, detect the direction of the subject's gaze trajectory shifting to the auxiliary trigger graphic area, and detect the overlap between the subject's gaze point and the auxiliary trigger graphic position within 300ms to trigger the triggering behavior of the interactive target; S2. Target selection task verification and technology comparison based on target-assisted gaze triggering Design circular menu layouts under different spatial densities, conduct target triggering task verification based on target-assisted gaze triggering, and compare with other eye-controlled triggering technologies to determine the advantageous and inferior spatial triggering states of target-assisted gaze triggering; For the eye control interaction interface, an interactive target triggering task is set, the icon of the circular menu is designed as the interactive target, and different spatial layout forms are designed, including the interactive target density and the spatial distribution area of ​​the menu; and the eye control selection steps and the platform data collection under the set task are determined, such as trigger time, trigger accuracy, and pupil diameter data. Combined with the experimental platform in step S1, the interactive target triggering task is carried out, and the proposed target-assisted gaze eye control triggering method is compared with the traditional gaze triggering method and eye gesture triggering method. The eye control interaction experimental process including recruiting subjects, practice experiments, and formal experiments is clarified, and the experimental platform data is collected to compare the adaptation scenarios of different methods; S3. Coding optimization strategy for auxiliary trigger graphics in high-density space The inferior triggering spatial state obtained in step S2 is optimized to enhance the comprehensive triggering performance of the target-assisted gaze method. Specifically, the inferior triggering spatial state is simulated, multi-level auxiliary triggering graphics encoding (spatial position and size) is set, the superior encoding method for enhancing the triggering efficiency is explored, and a coding strategy for multi-dimensional target-assisted gaze triggering is proposed.

2. A gaze interaction method for accelerating VR target selection according to claim 1, characterized in that: The circular layout menu interface design captures the position, angle and movement speed data of the user's head in real time through multiple high-sensitivity inertial measurement units (IMUs) and depth cameras integrated in a virtual reality (VR) helmet, and then dynamically adjusts the position and direction of the menu interface, thereby achieving synchronous binding of the menu interface with the user's head movement, ensuring that when the user moves his head, the menu interface can respond instantly and move accordingly in the user's field of view.

3. The gaze interaction method for accelerating VR target selection according to claim 1, characterized in that: The circular menu is composed of an interactive target and an auxiliary trigger graphic, wherein the interactive target is located inside the circular menu, and the auxiliary trigger graphic is located outside the circular menu, the center of the auxiliary trigger graphic is located at the connecting line between the center of the menu and the center of the interactive target, and the two interactive targets and the auxiliary trigger graphic maintain a spacing of 0.5 times the width of the interactive target; The interactive target and auxiliary trigger graphics are set to gray, and one of the randomly appearing blue icons is set as the interactive target to be triggered.

4. The gaze interaction method for accelerating VR target selection according to claim 1, characterized in that: The triggering process design of the target-assisted gaze triggering target; The visible area of ​​the interactive target and the auxiliary trigger object is the gaze detection area, in which the auxiliary trigger graphic serves as a visual cue and "confirmation button" of the trigger process. The specific trigger process is that the user first gazes at the blue trigger target, and monitors the overlap between the subject's gaze point position and the interactive target position area within 400ms to trigger the color change feedback of the auxiliary trigger graphic; if the subject's gaze trajectory direction is detected to be transferred to the auxiliary trigger graphic area, and the subject's gaze point position coincides with the auxiliary trigger graphic position within 300ms to trigger the trigger behavior of the interactive target, the interactive target will turn green if the trigger is successful.

5. The gaze interaction method for accelerating VR target selection according to claim 1, characterized in that: The menu spatial layout design is achieved by adjusting the interactive target density and the spatial distribution area of ​​the menu; Set different numbers of interactive targets (3, 4, 5, 6, 7 and 8) in the same area; set the spatial distribution area of ​​the menu (defined as 60° field of view angle and 45° field of view angle area).

6. The gaze interaction method for accelerating VR target selection according to claim 4, characterized in that: The described experiment employed a target-triggering task that required participants to quickly and accurately search for a blue interactive target in a circular menu; After the participant finds the blue target button, the target trigger is completed. If the correct button is triggered, the button will turn green, otherwise, it will turn red. The experimental platform records the trigger time, trigger accuracy, and pupil diameter data.

7. The gaze interaction method for accelerating VR target selection according to claim 6, characterized in that: The experimental task execution phase; The subjects performed an eye-controlled target triggering experiment. The experiment was divided into three groups. The three groups used target-assisted gaze triggering method, traditional gaze triggering (a method that only detects regional overlap), and eye gesture triggering (a method that detects the trajectory of line of sight movement). During the triggering process, a black fixation cross ("+") was first displayed for 500 milliseconds, followed by a blank screen for 1000 milliseconds, and then a triggering interface with a circular menu appeared. At this time, the timer started. The participant's task was to find the blue target button and trigger it. If the target icon was successfully triggered, the icon turned blue. If the non-target icon was triggered, the icon turned red. Once the target was triggered, the timer stopped, and the experimental program recorded the triggering time, and then started the next trial. The experimental task ended at the prompt stage; After the user completes the experiment, the center of the screen prompts that the experiment is complete, and the experimenter helps the user take off the VR device and fill in the NASA-TLX questionnaire and ASQ questionnaire; Experimental data processing and analysis; The experiment collected data on trigger time, trigger accuracy, and pupil diameter. The trigger time in the experiment was obtained by a program timer, which was the interval between the appearance of the search interface and the moment when the participant triggered the interaction target. Based on the trigger time, trigger accuracy, and pupil diameter, five virtual interface designers preliminarily screened out the advantageous spatial triggering state and inferior spatial triggering state of target-assisted gaze triggering.

8. The gaze interaction method for accelerating VR target selection according to claim 7, characterized in that: The disadvantageous trigger space results of the target-assisted gaze method obtained are set by setting typical optimized spatial parameters (i.e., high-density and small space layout), setting multi-level auxiliary trigger graphic encoding (spatial position and size), and conducting target triggering experiments under the target-assisted gaze method based on different auxiliary graphic encoding states.

9. The gaze interaction method for accelerating VR target selection according to claim 7, characterized in that: The target triggers experimental data processing; The significance and interaction of the size and relative position of the auxiliary trigger graphics on the trigger effectiveness are analyzed, and the trigger efficiency and accuracy in each encoding situation are further analyzed and visualized. The visualization results can be divided into efficiency-first or accuracy-first encoding strategies, providing users with encoding optimization strategies to improve the target-assisted gaze method in high-density space.

Images