Unity line-of-sight interaction detection method and system based on intention prediction and adaptive inflation of collision bodies

Abstract

The present application relates to the technical field of virtual reality (VR), augmented reality (AR) and three-dimensional human-computer interaction, in particular to a Unity line-of-sight interaction method and system combining multi-modal data analysis and dynamic assistance. The present application provides a Unity line-of-sight interaction detection method and system based on intention prediction and collision body adaptive inflation, calculates an interaction intention score by fusing user head movement data and line-of-sight data, and dynamically adjusts the size of the detection bounding box of the target object and the response threshold value accordingly, solving the false touch and loss problems of traditional ray detection in the line-of-sight micro-vibration and dense scene. In the Unity development environment, an "adaptive adsorption mechanism" line-of-sight interaction experience more in line with ergonomics is provided.

Description

Technical Field

[0001] This invention relates to the fields of virtual reality, human-computer interaction and 3D graphics processing technology, and in particular to a Unity gaze interaction detection optimization method and system for solving gaze jitter and improving selection efficiency. Background Technology

[0002] In VR / AR applications developed using the Unity engine, traditional eye-tracking interaction primarily relies on Physics.Raycast. However, the human eye experiences physiological tremors when looking at an object, and it is difficult to maintain absolute stillness when wearing a head-mounted display. Existing technical solutions (such as simple delay determination) have the following drawbacks:

[0003] 1. Difficulty in selecting small objects: Selecting distant or small objects requires the user to maintain a high degree of head stability, as shaking can easily cause the ray to slip out of the target object.

[0004] 2. Midas Touch Issue: To prevent shaking, a relatively long time threshold is usually set, resulting in a strong sense of lag and a lack of smoothness in the interaction.

[0005] 3. Lack of intent assistance: The system cannot distinguish whether the user is "unintentionally scanning" or "intentionally aiming", resulting in mechanical target determination. Summary of the Invention

[0006] This invention provides a Unity gaze interaction detection method and system based on intent prediction and adaptive expansion of colliders, aiming to "attract" the user's gaze through algorithms, reduce aiming difficulty, and improve the smoothness of interaction.

[0007] A Unity gaze interaction detection method and system based on intent prediction and adaptive collider expansion, characterized by comprising:

[0008] S1. Real-time acquisition of user head posture data and gaze direction vector;

[0009] S2. Calculate the user's real-time gaze stability coefficient based on the relative stillness of head angular velocity and gaze direction;

[0010] S3. Emit a cone-shaped detector instead of a single ray in the line of sight to obtain a set of candidate objects within the cone's range;

[0011] S4. Combine the object's offset at the center of the field of view with the stability coefficient obtained in S2 to calculate the interaction intent score for each candidate object;

[0012] S5. Based on the intent score, dynamically expand the collider of high-scoring candidate objects to make them easier to hit;

[0013] S6. When the intent score exceeds the set threshold and the duration meets the dynamic window, trigger target locking or an interaction event;

[0014] S7. After the interaction ends, reset the size of the target object's collider and adjust the weight parameters of the intent prediction model based on the confirmation speed of this interaction;

[0015] S8. For interfering objects with extremely low intention scores but within the field of view, implement hierarchical culling or collision shrinkage processing.

[0016] Furthermore, the cone-shaped detection mechanism described in S3 is used for fault tolerance. Even if the user's line of sight is not completely aligned with the object, as long as the object is within the cone range and the intent score is high enough, it can still be captured by the system.

[0017] Furthermore, the formula for calculating the interaction intent score in S4 is as follows:

[0018]

[0019] in The score is based on intent. The distance from the point where the line of sight falls to the center of the object. The radius of the cone detection. Distance weights The angular velocity of head rotation (radians / second). For stability weights, A preset priority coefficient for an object.

[0020] Furthermore, the dynamic expansion of the collider described in S5 is an auxiliary mechanism that is not visually visible but is effective at the physical detection layer. When the system determines that the user is highly likely to want to interact with object A, the detection bounding box of object A will shift and expand towards the center of the line of sight, achieving a "line of sight magnetic attraction" effect.

[0021] Furthermore, the triggering mechanism described in S6 adopts the "energy accumulation method". The higher the intention score, the faster the energy bar fills up, thereby achieving "instant opening" for certain targets and "slow opening" for hesitant targets.

[0022] Furthermore, the parameter self-correction module records the user's average head tremor amplitude and automatically adjusts the baseline value of the stability coefficient in S2 to adapt to the operating habits of different users (such as the elderly and children).

[0023] Furthermore, the system includes a visual feedback mechanism that dynamically changes the shape of the crosshair or the brightness of the target object based on the intent score, providing interactive predictive feedback. Attached Figure Description

[0024] Figure 1 This is a system framework diagram of the present invention.

[0025] Figure 2 This is a flowchart of the core eye-tracking interaction of the present invention.

[0026] Figure 3 This is a schematic diagram of the energy accumulation triggering mechanism in this invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0028] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention.

[0029] Example 1.

[0030] like Figure 1 As shown in the system framework diagram, this embodiment provides a Unity line-of-sight interaction detection method and system based on intent prediction and adaptive collider expansion, which includes a multimodal data acquisition module, an intent scoring calculation module, an adaptive collider control module, a parameter self-correction module, and an energy accumulation discrimination module.

[0031] The multimodal data acquisition module is used for:

[0032] During the Unity Update lifecycle, the head position, rotation quaternion, and instantaneous angular velocity are obtained through InputSystem or XRNode.

[0033] The intent scoring calculation module is used for:

[0034] By combining the offset distance between the line of sight and the center of the object, and the smoothness of head movement, the score value of each object in the current field of vision is calculated in real time.

[0035] The adaptive collision control module is used for:

[0036] Based on the Score value, the target object's Collider.size or Collider.radius is dynamically modified via script, and the object is restored to its original state after the view is removed.

[0037] The parameter self-correction module is used for:

[0038] The system tracks user interaction success rate and average trigger duration in the background and dynamically adjusts the weighting coefficients in the intent scoring formula.

[0039] The energy accumulation discrimination module is used for:

[0040] The module is configured to maintain an energy state variable for the target object. It non-linearly charges or linearly decays the energy pool based on the current intent score: energy accumulates rapidly when the intent score is high; when the line of sight is briefly deviated from the target, the energy decays over time rather than immediately reaching zero. When the accumulated energy reaches the trigger threshold (full tank), an acknowledgment signal is generated.

[0041] Example 2.

[0042] like Figure 2 As shown in the logic flowchart, the core process of eye-tracking interaction provided in this embodiment is as follows:

[0043] a) During the system initialization phase, the gaze detection parameters, intent scoring weight parameters, and energy accumulation threshold parameters are loaded from the local configuration file or remote configuration service.

[0044] b) In each frame update cycle, the system collects user head posture data and gaze direction data, and constructs a cone-shaped detection body along the gaze direction with the user's current viewpoint as the origin. The system performs preliminary screening of scene objects within the coverage area of ​​the cone-shaped detection body to generate a set of candidate interactive targets.

[0045] c) Based on the set of candidate interactive targets, perform visibility judgment and occlusion relationship analysis on the objects in the set, and remove objects that are completely occluded; at the same time, determine whether the user is in a rapid scanning state based on the user's head angular velocity. When the head angular velocity is detected to exceed a preset threshold, the interaction intent score corresponding to the candidate interactive target is attenuated or its energy state is cleared to reduce the probability of false triggering.

[0046] d) For the remaining candidate objects, calculate their interaction intent scores and select the object with the highest score as the current potential interaction target.

[0047] e) Based on the interaction intent score of the potential interaction target, the collider is proportionally expanded and corresponding visual feedback is provided to the user.

[0048] f) Input the interaction intent score of the potential interaction target into the energy accumulation and discrimination module to determine whether its accumulated energy has reached the trigger threshold.

[0049] Example 3.

[0050] like Figure 3 As shown, this embodiment provides a gaze interaction triggering determination process based on energy accumulation, which can replace the traditional gaze determination method with a fixed time threshold.

[0051] Traditional timers use a 0 or 1 timer for judgment. This invention employs an integral circuit model where energy accumulates with the score as the user focuses on the target; when the gaze deviates, the energy does not immediately reset to zero, but decays linearly. This allows users to quickly recover their interaction progress after a brief blink or unexpected gaze shift, greatly improving the robustness of the user experience.

[0052] When the energy state variable reaches the preset trigger threshold, the system determines that the user has formed a clear interaction intention, and then outputs an interaction confirmation signal and triggers the corresponding functional event.

[0053] This mechanism can effectively tolerate eye tremors, blinks, and short-term head movements, significantly improving the robustness and continuity of eye contact.

[0054] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A Unity gaze interaction detection method and system based on intent prediction and adaptive expansion of colliders, characterized in that, include: S1. Real-time acquisition of user head posture data and gaze direction vector; S2. Calculate the user's real-time gaze stability coefficient based on the relative stillness of head angular velocity and gaze direction; S3. Emit a cone-shaped detector instead of a single ray in the line of sight to obtain a set of candidate objects within the cone's range; S4. Combine the object's offset at the center of the field of view with the stability coefficient obtained in S2 to calculate the interaction intent score for each candidate object; S5. Based on the intent score, dynamically expand the collider of high-scoring candidate objects to make them easier to hit; S6. When the intent score exceeds the set threshold and the duration meets the dynamic window requirement, trigger a target lock or interaction event; S7. After the interaction ends, reset the size of the target object's collider and adjust the weight parameters of the intent prediction model based on the confirmation speed of this interaction; S8. For interfering objects with extremely low intention scores but within the field of view, implement hierarchical culling or collision shrinkage processing.

2. The method according to claim 1, wherein the cone-shaped detector in S3 is implemented using Unity's Physics.SphereCast or a custom multi-ray sampling method to cover the range of line-of-sight jitter.

3. The method according to claim 1, wherein the calculation of the S4 interaction intent score integrates the Euclidean distance between the gaze point and the center of the object, the inverse correlation of head rotation angular velocity, and the interaction priority of the object itself.

4. The method according to claim 1, wherein S2 determines the user's interaction intention based on whether the ray emitted from the user's current line of sight in S1 collides with an object in the scene.

5. The method according to claim 1, wherein the dynamic expansion of the collider in S5 refers to temporarily increasing the Size / Radius property of the BoxCollider or SphereCollider in the Unity component at the logic layer to form a "magnetic" effect.

6. The method according to claim 1, wherein the time threshold of the dynamic window in S6 is inversely proportional to the intent score calculated in S4, and the higher the score, the shorter the triggering time.

7. The method according to claim 1, wherein the formula for calculating the interaction intent score in S4 is as follows:

8. A system for implementing the above method, comprising: a) Multimodal data acquisition module; b) Cone projection detection module; c) Stability analysis module; d) Intent scoring calculation module; e) Adaptive collider control module; f) Interactive event triggering module; g) Parameter self-correction module; h) Energy accumulation discrimination module.

Images