Intelligent human-computer interaction data processing method based on augmented reality

Abstract

The application discloses an intelligent man-machine interaction data processing method based on augmented reality, and relates to the technical field of augmented reality and man-machine interaction. The method obtains three-dimensional coordinate data of key skeletal points of a user's hand, calculates micro-kinematics features including hand movement acceleration features, grip closure degree features and spatial movement continuity features, and inputs the features into an intention classification network to determine a current intention state as a physical operation domain or a virtual interaction domain. When the state jumps from the virtual interaction domain to the physical operation domain, a virtual user interface collision detection calculation task is removed from a rendering main loop, and is re-added when a recovery condition is met. Meanwhile, an eye movement physiological feature and a hand high-frequency micro-vibration feature are fused to calculate a cognitive load index. According to the cognitive load index, a collision detection bounding box size is dynamically amplified, and hand position data is subjected to Kalman filtering and smoothing. The application can automatically distinguish interaction intentions without explicit instructions of a user, and reduces edge side computing power consumption.

Description

Technical Field

[0001] This invention relates to the field of augmented reality and human-computer interaction technology, specifically to an intelligent human-computer interaction data processing method based on augmented reality. Background Technology

[0002] Augmented reality (AR) devices are widely used in productivity scenarios involving both virtual and real elements, such as industrial maintenance and medical surgery. While users interact with real physical objects, a virtual user interface often floats in their field of view for interaction. Currently, mainstream AR devices typically include depth sensors, multi-view cameras, and eye-tracking modules to acquire real-time hand 3D skeletal coordinate data and eye-tracking data. Based on this data, existing human-computer interaction data processing methods usually generate interaction rays or virtual bounding boxes based on finger joint coordinates and perform geometric collision detection between these rays and colliders of virtual user interface elements. Once an intersection is detected, interactive events such as clicks, hovers, or selections are triggered. This collision detection operation is continuously and globally executed during system operation, without distinguishing whether the user's hand is currently interacting with a real physical object. The above processing method cannot address the issue at the data layer. The current technology distinguishes between user hand gestures that serve physical operation intentions and virtual interaction intentions. When a user performs hand movements near a virtual interface to complete a physical task, their hand trajectory is prone to geometric intersection with the virtual interface collision object, thus triggering an interaction event that is not intended by the user. At the same time, collision detection calculations continuously consume edge computing resources, and the fixed-size collision determination method does not consider the problem of decreased interaction success rate caused by physiological tremors of the user's hands under high cognitive load. Especially in the intent classification stage, the existing technology lacks refined utilization of the micro-kinematic features of the hands and does not dynamically adjust the interaction determination conditions according to the user's cognitive load state, resulting in the long-term existence of the above defects. There is an urgent need for a processing method that can automatically distinguish user interaction intentions at the data processing level while taking into account both computing power consumption and interaction success rate in high-pressure scenarios. Summary of the Invention

[0003] The technical problem this invention aims to solve is that existing augmented reality human-computer interaction data processing methods employ a global, continuous, and indiscriminate collision detection processing logic, which cannot distinguish at the data level whether a user's hand movements are used to manipulate real physical objects or to interact with the virtual user interface. This leads to unintentional interaction events triggered when the user's hand trajectory intersects with the virtual user interface collider during the manipulation of physical objects. Furthermore, since collision detection calculations are continuously executed globally during system operation, the computing resources at the edge of the augmented reality device are continuously occupied. In addition, existing technologies use fixed settings for the bounding box size of the virtual user interface collision detection and interaction judgment conditions, failing to consider the problem of decreased virtual interaction success rate caused by physiological tremors of the user's hands under high cognitive load.

[0004] To address the aforementioned technical problems, this invention provides an augmented reality-based intelligent human-computer interaction data processing method, comprising:

[0005] The three-dimensional coordinate data of multiple key skeletal points of the user's hand at multiple consecutive sampling times are obtained, and micro-kinematic features are calculated based on the three-dimensional coordinate data. The micro-kinematic features include hand motion acceleration features, grasp closure features, and spatial motion continuity features.

[0006] The micro-kinematic features are input into a pre-trained intent classification network for inference to obtain a first probability value that the current user intent belongs to the physical operation domain and a second probability value that belongs to the virtual interaction domain. Based on the comparison result between the first probability value and the preset intent threshold, the current intent state is determined to be either the physical operation domain or the virtual interaction domain, and the corresponding state flag is output.

[0007] The status flag at the current sampling time is compared with the status flag at the previous sampling time. When the comparison result is that the virtual interaction domain has jumped to the physical operation domain, a blocking instruction is triggered to remove the virtual user interface collision detection calculation task from the main rendering loop.

[0008] During the period when the collision detection calculation task is removed, the grip closure feature and the spatial motion continuity feature are continuously acquired. When the grip closure feature and the spatial motion continuity feature meet the preset recovery conditions, it is determined that the current intention state is restored to the virtual interaction domain, a recovery command is triggered, and the collision detection calculation task is re-added to the main rendering loop.

[0009] Based on the above technical solutions, the present invention further provides a dimension reduction and fault-tolerant processing scheme based on the cognitive load index: acquiring the user's eye movement physiological characteristics and the high-frequency micro-tremor energy characteristics of the hand, performing weighted fusion calculation based on the eye movement physiological characteristics and the high-frequency micro-tremor energy characteristics of the hand to obtain the cognitive load index, and when the cognitive load index exceeds a preset cognitive load threshold, dynamically enlarging the collision detection bounding box size of the currently active virtual interactive target according to a dynamic amplification ratio positively correlated with the cognitive load index, and simultaneously performing Kalman filtering smoothing processing on the collected key skeletal point position data of the hand, so that the degree of expansion of the collision judgment range and the smoothing suppression intensity of the hand position data are dynamically adapted synchronously with the degree of physiological tension.

[0010] Compared with the prior art, the beneficial effects of the present invention by adopting the above technical solution are as follows:

[0011] This invention refines and classifies hand micro-kinematic features, enabling it to automatically and in real-time determine whether a user is in the physical operation domain or the virtual interaction domain without requiring explicit voice or button commands. This reduces accidental touches caused by hand movements being indiscriminately identified as virtual interaction commands from the data processing source, and improves the accuracy of human-computer interaction in concurrent virtual and physical scenarios.

[0012] In existing technologies, collision detection calculation tasks are continuously executed globally without distinguishing user intent during system operation. Those skilled in the art typically only focus on the accuracy optimization of the collision detection algorithm itself and have never used the "intent state transition result" as a criterion for triggering the dynamic removal / restoration of the rendering main loop of the collision detection task. This invention breaks through this conventional implementation method by coupling the intent state transition result driven by micro-kinematic features with the scheduling of the rendering main loop task. When it is determined to be a physical operation domain, the virtual user interface collision detection calculation task is removed from the rendering main loop, and it is restored when the preset restoration condition composed of grasp closure feature and spatial motion continuity feature is met. This coupling mechanism between state transition and task scheduling is not a simple superposition of existing collision detection accuracy optimization methods and intent recognition methods, but rather produces a synergistic technical effect that is difficult to achieve simultaneously by existing technologies such as "mis-touch elimination" and "reduction of computing power consumption" when implemented independently, which is non-obvious.

[0013] This invention calculates the cognitive load index by fusing eye movement physiological characteristics and high-frequency hand tremor characteristics, and then dynamically amplifies the collision detection bounding box size of the virtual interactive target and performs Kalman filtering smoothing on the hand position data accordingly. This can improve the success rate of virtual interactive tasks in scenarios where users are under high cognitive load and experience physiological hand tremors. Attached Figure Description

[0014] Figure 1 A schematic diagram of the overall system architecture used in the augmented reality-based intelligent human-computer interaction data processing method provided in the embodiments of the present invention;

[0015] Figure 2 This is a schematic diagram of the core algorithm processing flow provided in the embodiments of the present invention;

[0016] Figure 3 This is a schematic diagram illustrating the state transitions of intentional decoupling, dynamic blocking, and recovery provided in an embodiment of the present invention;

[0017] Figure 4 This is a schematic diagram of the dimension reduction and fault tolerance processing flow based on the cognitive load index provided in an embodiment of the present invention. Detailed Implementation

[0018] The following is in conjunction with the appendix Figure 1-4The specific embodiments of the present invention will be further described below. It should be noted that the description of these embodiments is for the purpose of helping to understand the present invention, but does not constitute a limitation of the present invention.

[0019] Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0020] In this embodiment, the augmented reality-based intelligent human-computer interaction data processing method provided by the present invention is applied to an augmented reality head-mounted display device equipped with a depth sensor, a multi-view grayscale camera and an eye-tracking module. The data processing program, which is loaded and run in the form of middleware on the augmented reality operating system input subsystem, is implemented by an edge AI chip deployed locally on the device, i.e., an artificial intelligence computing module deployed on the local neural network processing unit or other embedded computing power unit. Real-time reasoning and control can be completed without relying on the cloud.

[0021] I. System Overall Architecture

[0022] like Figure 1 As shown, the processing device for implementing the method of the present invention includes a data acquisition module 110, a feature extraction module 120, an interaction intent decoupling decision module 130, a cognitive load calculation module 140, a dynamic blocking and recovery control module 150, and a dimension reduction and fault tolerance processing module 160. Each module is connected in sequence and interacts with the augmented reality operating system input subsystem 170. The feature extraction module 120 includes a micro-kinematic feature extraction unit 121 and a physiological feature extraction unit 122.

[0023] The data acquisition module 110 is connected to the hardware perception layer and is used to acquire the hand 3D skeletal coordinate data stream and eye movement data stream output by the depth sensor module, multi-view grayscale camera module, and eye-tracking module. After aligning the data streams with a unified timestamp, it distributes them to the feature extraction module 120. The micro-kinematic feature extraction unit 121 calculates hand motion acceleration features, grasp closure features, and spatial motion continuity features based on the hand 3D skeletal coordinate data stream. The physiological feature extraction unit 122 calculates pupillary tremor amplitude, blink frequency, and high-frequency hand tremor energy based on the eye movement data stream and the hand 3D skeletal coordinate data stream. The interaction intent decoupling decision module 130 embeds a lightweight classification network and outputs intent state flags based on micro-kinematic features. The cognitive load calculation module 140 outputs the values ​​from the physiological feature extraction unit 122. The system calculates the cognitive load index based on the characteristics of the system. The dynamic blocking and recovery control module 150 controls the removal and addition of collision detection calculation tasks in the augmented reality operating system input subsystem 170 according to the jump results of the intent state flag. The dimensionality reduction and fault tolerance processing module 160 processes the collision detection bounding box size and hand position data according to the cognitive load index. The augmented reality operating system input subsystem 170 receives the standard interactive events processed by the above modules and outputs them to the upper-layer application to complete the rendering response of the virtual user interface. The above modules are mounted on the original augmented reality operating system input subsystem in the form of middleware. For the upper-layer application, the format of the interactive events it receives is consistent with the existing standard interactive event format. That is, the processing process provided by this invention is transparent to the upper-layer application and does not require modification of the existing application logic.

[0024] II. Micro-kinematic feature extraction and decoupling of virtual-real interaction intent

[0025] In this embodiment, the data acquisition module 110 continuously acquires the three-dimensional coordinate sequence of 21 key skeletal points of the user's hand at a sampling frequency of 90 Hz. ,in Indicates the first Key skeletal points at time The three-dimensional coordinates are represented by 21 key skeletal points, which correspond to the joints of each finger and key parts of the palm.

[0026] The micro-kinematic feature extraction unit 121 calculates the instantaneous velocity vector of each key skeleton point based on the coordinate difference between adjacent sampling times:

[0027] ;

[0028] Then, by subtracting the velocity vector, we obtain the acceleration vector:

[0029] ;

[0030] in, In this embodiment, the sampling time interval is... Approximately Second, Key skeletal points At any moment The velocity vector, Key skeletal points At any moment The acceleration vectors of 21 key skeletal points are statistically analyzed (e.g., the average value is taken) to obtain the acceleration features of hand movement, which are used to reflect the intensity of the overall hand movement.

[0031] Microkinematic feature extraction unit 121 calculates the grasp closure feature using the mean Euclidean distance between the thumb tip keypoint and the keypoints of the other four fingers:

[0032] ;

[0033] in, The key point of the thumb tip at any time coordinates The key points of the fingertips of the index, middle, ring, and little fingers at all times coordinates Take values ​​respectively , , , , This represents the Euclidean distance, which is the straight-line distance between two three-dimensional coordinate points. The smaller the value, the tighter the grip, which is consistent with the characteristics of holding a real physical tool. The larger the value, the more loosely the hand is spread out, which matches the characteristics of a virtual click gesture. For example, when a user holds a wrench to tighten something, their five fingers are in a closed grip. The corresponding value is in the lower range, and when the user extends their index finger to click the virtual button, the other fingers naturally spread out. This corresponds to a relatively high value range.

[0034] Micro-kinematic feature extraction unit 121 in sliding time window By statistically analyzing the rate of change of the direction of the hand's center of mass trajectory, the characteristics of spatial motion continuity can be obtained:

[0035] ;

[0036] in, For the sliding time window The number of sampling frames included in this embodiment The value is 30. The trajectory of the hand's center of mass in the th... The sampling time and the first The angle between the motion direction vectors corresponding to each sampling time point The smaller the value, the more stable and continuous the hand movement is, which matches the characteristics of periodic rotational movements in physical operation scenarios, such as tightening bolts. The larger the value, the more frequently the hand movement direction changes, which is consistent with the discrete jumping characteristics of virtual user interface point selection operations.

[0037] The micro-kinematic feature extraction unit 121 extracts hand motion acceleration features and grasp closure features. Characteristics of spatial motion continuity The data is concatenated with other derived time-series features (such as the mean velocity of the hand's center of mass movement and the variance of the relative positions of key skeletal points), and each component is normalized according to its historical statistical range to obtain a standardized feature vector. This serves as the input for subsequent intention classification reasoning.

[0038] The interactive intent decoupling decision module 130 embeds a lightweight classification network, which is a pre-trained spatiotemporal graph convolutional network and a long short-term memory network. The training data for this classification network is constructed by collecting user hand skeletal coordinate sequences in various typical productivity scenarios (including industrial equipment disassembly and assembly, simulated surgical operations, etc.) and manually labeling them with scene tags (physical operations or virtual interactions). Supervised learning is used to train the network parameters, enabling the network to output stable intent classification results based on continuous temporal micro-kinematic features. During the inference phase, the interactive intent decoupling decision module 130 converts the feature vector sequences corresponding to multiple consecutive sampling times into a single sequence. Inputting this classification network for inference yields the following results:

[0039] ;

[0040] in, This represents the inference function of the classification network. These are the network parameters obtained after training the classification network. This represents the normalized exponential function, used to convert the network output into a probability distribution. These represent the first probability value and the second probability value for determining that the current user intent belongs to the physical operation domain and the virtual interaction domain, respectively. and The sum of them is .

[0041] The interaction intent decoupling decision module 130 will use the first probability value With preset intent threshold In comparison, in this embodiment Values ,when Greater than When the current user is in the physical operation domain, the corresponding first state flag is output. Not greater than When the current user is in the virtual interaction domain, the system determines that the user is in the virtual interaction domain and outputs the corresponding second status flag, which serves as the input signal for the dynamic blocking and recovery control module 150.

[0042] III. Dynamic Blocking and Recovery Mechanism

[0043] The dynamic blocking and recovery control module 150 is mounted as middleware on the event distribution layer of the augmented reality operating system input subsystem 170, and continuously subscribes to the status flags output frame by frame by the interaction intent decoupling decision module 130.

[0044] The dynamic blocking and recovery control module 150 compares the status flag at the current sampling moment with the status flag at the previous sampling moment. When it detects that the status flag changes from the second status flag of the corresponding virtual interaction domain to the first status flag of the corresponding physical operation domain, it triggers a blocking command. After executing the blocking command, the dynamic blocking and recovery control module 150 removes the virtual user interface ray projection and collision detection calculation task from the rendering main loop. That is, it no longer performs collision intersection calculation on the hand data corresponding to the sampling moment and subsequent sampling moments until the recovery condition is met. Taking the scenario of a user holding a wrench tightening a bolt in front of a virtual maintenance guidance panel as an example, during this process, the user's hand is continuously in a closed grip state and the movement trajectory is periodically rotating. The interaction intent decoupling decision module 130 will continuously determine that the current state is a physical operation domain. After detecting the state change, the dynamic blocking and recovery control module 150 immediately suspends the collision detection task, thereby avoiding geometric intersection between the user's hand or the tool being held and the collider of the virtual panel, which would trigger an unintentional click event.

[0045] During the period when the collision detection calculation task is removed, the dynamic blocking and recovery control module 150 continuously acquires the grip closure features at a second sampling frequency (10 Hz in this embodiment) that is lower than the first sampling frequency (i.e., 90 Hz) corresponding to the aforementioned consecutive multiple sampling times. With the characteristics of spatial motion continuity ,judge Whether it has recovered to above the preset closure threshold, and Whether the preset continuity threshold has been exceeded is used to determine whether the user's physical operation has ended. When the interaction intent decoupling decision module 130 re-outputs the status flag of the corresponding virtual interaction domain based on real-time characteristics, and Restored to above the aforementioned closure threshold. When the continuity threshold is exceeded, the dynamic blocking and recovery control module 150 determines that the preset recovery condition is met, triggers the recovery command, and re-adds the collision detection calculation task to the main rendering loop within less than 50 milliseconds, restoring the standard collision detection input response link. Since the above determination and recovery process is automatically completed at the data processing layer, when the user performs virtual interaction after completing physical operation, no additional operation is required to obtain a normal virtual interface response.

[0046] like Figure 3 As shown, in this embodiment, the collision detection corresponding to the virtual interaction domain is in an active state, and the collision detection corresponding to the physical operation domain is in a suspended state. The two states are bidirectionally switched according to the aforementioned transition and recovery conditions. The switching process does not require the user to actively issue voice or button commands. Furthermore, the collision detection task is temporarily removed from the main rendering loop during the suspension period, which reduces the computing power occupied by the edge side of the augmented reality device at this stage.

[0047] IV. Cognitive Load Index Calculation and Dimensionality Reduction Fault Tolerance Processing

[0048] Physiological feature extraction unit 122 obtains the user's pupil diameter sequence through eye-tracking module. Calculate the standard deviation of pupil diameter within a preset time window using the blink time series. and blinking frequency The former reflects the amplitude of pupil tremors, while the latter reflects the degree of user distraction; both are related to the user's current cognitive load and physiological tension.

[0049] The physiological feature extraction unit 122, based on the acceleration sequence of key hand bone points calculated by the micro-kinematic feature extraction unit 121, converts the acceleration signal from the time domain to the frequency domain using a fast Fourier transform, and then processes the converted signal in a preset high-frequency band (in this embodiment, ). Hertz Energy integration was performed within the physiological tremor frequency range of Hertz to obtain the high-frequency shaking component energy of the hand. This feature is used to reflect the intensity of high-frequency micro-tremors produced by the user's hand under physiological tension.

[0050] Cognitive load calculation module 140 will , , After normalizing each index according to its respective historical statistical range, a weighted fusion was performed to calculate the cognitive load index:

[0051] ;

[0052] in, , , They are respectively , , Normalized values, , , The weighting coefficients are experimentally calibrated, and in this embodiment, they are taken as values ​​respectively. , , The sum of the three is , The value range is [0,1]. The larger the value, the higher the user's current cognitive load and physiological tension.

[0053] The 160-dimensionality reduction fault-tolerant processing module will Compared with the preset cognitive load threshold In comparison, in this embodiment Values ,when Exceed When the user is determined to be in a state of high cognitive load, the dimensionality reduction and fault tolerance processing mode is triggered. No more than At that time, the system directly performs collision detection based on the default collision detection bounding box size and the original hand position data.

[0054] After triggering the dimensionality reduction fault-tolerant processing mode, the dimensionality reduction fault-tolerant processing module 160 dynamically enlarges the collision detection bounding box size of the virtual interactive target that is active in the current field of view according to the following formula:

[0055] ;

[0056] in, This is the default collision detection bounding box size for the virtual interactive target. The magnification factor is empirically calibrated in this embodiment. Values , The enlarged collision detection bounding box size can be seen from the above formula. Follow As the numerical value increases, the range of virtual interactive targets that can be hit also increases, meaning that the higher the user's cognitive load and physiological tension, the greater the probability of gestures deviating from the target area due to hand tremors.

[0057] The dimension reduction and fault-tolerant processing module 160 simultaneously introduces a Kalman filter into the high-frequency micro-tremor gesture data stream for smoothing, and the state prediction equation is:

[0058] ;

[0059] The state update equation is:

[0060] ;

[0061] in, For a moment The prior state estimate is used to represent the predicted value of the current hand position calculated based on the state at the previous moment. For a moment Posterior state estimation, This is the state transition matrix, which describes the transition relationship between the hand position states at adjacent sampling times. To control the input matrix, To control the input amount, For a moment The posterior state estimate, i.e., the smoothed hand position data obtained after filtering, is obtained. For a moment The corresponding Kalman gain matrix is ​​used to adjust the weight relationship between the predicted and observed values. For a moment The actual observed value, i.e., the raw, noisy hand position data acquired by the sensor. The observation matrix is ​​used to map the state vector to the observation space. Through the iterative calculation of the above prediction and update process, the interference of high-frequency micro-tremor components on hand position data can be effectively suppressed while preserving the true movement trend of the hand.

[0062] The 160-dimensionality reduction fault-tolerant processing module is based on the magnified collision detection bounding box size. and the hand position data after filtering and smoothing. Collision detection is performed, and high-frequency hand tremor errors are tolerated while ensuring response accuracy. The virtual interaction event is obtained and output. The interaction event is distributed to the upper-layer application through the augmented reality operating system input subsystem 170 to complete the corresponding interface response.

[0063] Taking a medical surgery scenario as an example, when surgeons are performing delicate procedures, they may experience increased pupillary tremors, decreased blinking frequency, and enhanced physiological high-frequency hand tremors due to high tension. Based on this, the cognitive load calculation module 140 calculates a higher cognitive load index. The dimensionality reduction and fault tolerance processing module 160 then performs collision detection bounding box magnification processing on the virtual control panel that needs to be interacted with, and performs Kalman filtering smoothing on the surgeon's hand position data. This ensures that even if the surgeon's hand tremors slightly, the interaction with the virtual control panel can still be completed accurately, avoiding delays in handling due to interaction failures.

[0064] V. Other Implementation Methods

[0065] In another embodiment, the microkinematic feature extraction unit 121 also combines the electromyographic sensor or pressure sensing wristband-collected user forearm muscle electrical signals or grip pressure data as a supplementary basis for determining grip closure features. After fusing the supplementary basis with the visually acquired hand skeleton data, it is input into the interaction intent decoupling decision module 130 to improve the robustness of intent decoupling in scenarios with hand occlusion or low light.

[0066] In another embodiment, the interaction intent decoupling decision module 130 uses a preset rule threshold combination to replace the lightweight classification network. That is, it sets corresponding judgment thresholds for hand motion acceleration features, grip closure features and spatial motion continuity features respectively, and performs joint judgment on the comparison results of the above features and corresponding thresholds according to a preset logical order, and directly outputs the judgment results of the physical operation domain and the virtual interaction domain. This reduces the algorithm's computational complexity to a certain extent while reducing the judgment accuracy, and is suitable for augmented reality wearable devices with relatively limited computing resources.

[0067] The technical features disclosed in the above embodiments can be combined with each other to form new implementation methods without causing contradictions. The method provided by the present invention integrates three processing mechanisms: intent decoupling, dynamic blocking recovery, and dimensionality reduction fault tolerance. It is mounted on the input subsystem of the existing augmented reality operating system in the form of middleware. While achieving a processing effect that is transparent to upper-layer applications, it also achieves the technical effects of improving the accuracy of human-computer interaction in virtual-real concurrent scenarios, reducing edge computing power consumption, and improving the success rate of interaction in high cognitive load scenarios.

[0068] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of the present invention. Therefore, any modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope defined by the claims of the present invention.

Claims

1. A data processing method for intelligent human-computer interaction based on augmented reality, characterized in that: include: The three-dimensional coordinate data of multiple key skeletal points of the user's hand at multiple consecutive sampling times are obtained, and micro-kinematic features are calculated based on the three-dimensional coordinate data. The micro-kinematic features include hand motion acceleration features, grasp closure features, and spatial motion continuity features. The micro-kinematic features are input into a pre-trained intent classification network for inference to obtain a first probability value that the current user intent belongs to the physical operation domain and a second probability value that belongs to the virtual interaction domain. Based on the comparison result between the first probability value and the preset intent threshold, the current intent state is determined to be either the physical operation domain or the virtual interaction domain, and the corresponding state flag is output. The status flag at the current sampling time is compared with the status flag at the previous sampling time. When the comparison result is that the virtual interaction domain has jumped to the physical operation domain, a blocking instruction is triggered to remove the virtual user interface collision detection calculation task from the main rendering loop. During the period when the collision detection calculation task is removed, the grip closure feature and the spatial motion continuity feature are continuously acquired. When the grip closure feature and the spatial motion continuity feature meet the preset recovery conditions, it is determined that the current intention state is restored to the virtual interaction domain, a recovery command is triggered, and the collision detection calculation task is re-added to the main rendering loop.

2. The method according to claim 1, characterized in that, The hand motion acceleration feature is obtained by further differential calculation based on the velocity vector obtained by the coordinate difference of the key bone points of the hand at adjacent sampling times. The grasp closure feature is obtained by calculating the average value of the Euclidean distance between the key point of the thumb fingertip and the key points of the other four fingers. The spatial motion continuity feature is obtained by calculating the cosine value of the angle between the motion directions of the hand centroid trajectory at adjacent sampling times within the sliding time window.

3. The method according to claim 1, characterized in that, The intent classification network is any lightweight classification network between a spatiotemporal graph convolutional network and a long short-term memory network. The intent classification network takes the feature vector sequence formed by splicing the micro-kinematic features at multiple consecutive sampling times as input and outputs the first probability value and the second probability value after normalization.

4. The method according to claim 1, characterized in that, The preset recovery conditions include: the grip closure feature recovers to above a preset closure threshold, and the spatial motion continuity feature exceeds a preset continuity threshold.

5. The method according to claim 1, characterized in that, During the period when the collision detection calculation task is removed, the grip closure feature and the spatial motion continuity feature are continuously acquired at a second sampling frequency lower than the first sampling frequency corresponding to the consecutive multiple sampling times.

6. The method according to claim 1, characterized in that, Also includes: The user's eye movement physiological characteristics and high-frequency hand tremor energy characteristics are obtained, and the cognitive load index is obtained by weighted fusion calculation based on the eye movement physiological characteristics and high-frequency hand tremor energy characteristics. When the cognitive load index exceeds the preset cognitive load threshold, the collision detection bounding box size of the currently active virtual interactive target is dynamically magnified according to a dynamic magnification ratio that is positively correlated with the cognitive load index. At the same time, the collected key hand bone point position data is smoothed by Kalman filtering, so that the degree of expansion of the collision judgment range and the smoothing suppression intensity of the hand position data are dynamically adapted to the degree of physiological tension.

7. The method according to claim 6, characterized in that, The eye movement physiological characteristics include the standard deviation of the pupil diameter and the blink frequency within a preset time window. The high-frequency micro-tremor energy characteristics of the hand are calculated based on the energy integral of the hand movement acceleration sequence after performing a fast Fourier transform within a preset high-frequency band.

8. The method according to claim 6, characterized in that, The ratio between the magnified collision detection bounding box size and the default collision detection bounding box size before magnification increases with the increase of the cognitive load index.

9. The method according to claim 6, characterized in that, The Kalman filter smoothing process includes: calculating the prior state estimate of the current sampling time based on the posterior state estimate and state transition matrix of the previous sampling time; calculating the posterior state estimate of the current sampling time based on the prior state estimate, the Kalman gain matrix, and the actual observation value of the current sampling time; and using the posterior state estimate as the smoothed hand position data.

10. An augmented reality-based intelligent human-computer interaction data processing device, characterized in that, include: The data acquisition module is used to acquire the three-dimensional coordinate data of multiple key skeletal points of the user's hand at multiple consecutive sampling times, as well as the user's eye movement data; The feature extraction module, connected to the data acquisition module, is used to calculate microkinematic features, including hand motion acceleration features, grasp closure features, and spatial motion continuity features, based on the three-dimensional coordinate data. It also calculates eye movement physiological features and high-frequency hand tremor energy features based on the eye movement data and the acceleration sequence of key hand skeletal points. The cognitive load calculation module, connected to the feature extraction module, is used to perform weighted fusion calculation based on the eye movement physiological features and the high-frequency hand tremor energy features to obtain a cognitive load index. The interaction intent decoupling decision module is connected to the feature extraction module. It is used to input the micro-kinematic features into the pre-trained intent classification network for reasoning, determine whether the current intent state is one of the physical operation domain and the virtual interaction domain, and output the corresponding state flag. The dynamic blocking and recovery control module, connected to the interaction intent decoupling decision module, is used to remove the virtual user interface collision detection calculation task from the main rendering loop when the status flag changes from the virtual interaction domain to the physical operation domain, and to re-add the collision detection calculation task to the main rendering loop when the status flag changes from the physical operation domain back to the virtual interaction domain. The dimension reduction and fault tolerance processing module is connected to the cognitive load calculation module. When the cognitive load index exceeds the preset cognitive load threshold, it dynamically enlarges the collision detection bounding box size of the currently active virtual interactive target and performs Kalman filtering smoothing on the collected key hand bone point position data.

Images