A data glove intelligent interaction method and system based on virtual scene correction

Abstract

This invention discloses a data glove intelligent interaction method and system based on virtual scene correction, relating to the field of virtual reality technology. The method includes acquiring data glove sensor information, acquiring data glove correction action information based on data glove initialization requirements, and collecting data generated by the sensors integrated into the data glove when the user performs data glove correction actions to obtain correction action data. This invention analyzes the completion rate of the user's initial actions using initial action data, improving the accuracy of subsequent model adjustments. Through a first action model and a second action model, the virtual hand model is further adjusted to ensure the realism of the virtual interaction. By using the user's data glove action information and data glove interaction command information, the virtual scene is adjusted and corrected, ensuring the visual effect of the virtual interaction, ensuring the consistency between the model and real hand movements, and reducing interaction latency.

Description

Technical Field

[0001] This invention relates to the field of virtual reality technology, specifically to a data glove intelligent interaction method and system based on virtual scene correction. Background Technology

[0002] With the continuous development of virtual reality (VR) and augmented reality (AR) technologies, natural interaction methods are becoming increasingly important. Data gloves, as a novel human-computer interaction tool, are now widely used in an increasing number of virtual reality environments. Their purpose is to acquire real-time data on the flexion and abduction angles of various effective parts of the hand, including the palm, fingers, and wrist, through built-in sensors, and to inversely represent gestures based on this data to achieve interaction with virtual scenes. However, the presence of soft tissue in the human hand means that it cannot be compared to the ordinary rigid linkages used in robotic hands. The movement of a joint in the hand not only affects the readings of the corresponding sensor but also causes changes in the readings of other sensors through the interaction of soft tissues. This necessitates decoupling calculations of the obtained inverse mapping to ensure a certain level of accuracy.

[0003] Currently, the interaction with data gloves still suffers from the inability to adjust the virtual hand model based on specific actions. It often only provides feedback on the data glove's actions based on a general virtual hand model, resulting in a certain error compared to real actions. If users make adjustments themselves, they can usually only adjust the overall proportion of the virtual hand model, and cannot adjust the nodes of the virtual hand model according to the user's hand characteristics, which affects the realism of virtual interaction. Summary of the Invention

[0004] To address the aforementioned technical issues, this paper provides a data glove intelligent interaction method and system based on virtual scene correction. This technical solution solves the problem mentioned in the background technology that the virtual hand model cannot be adjusted according to specific actions. Often, the data glove only provides feedback on actions based on a general virtual hand model, resulting in a certain error between the data glove and real actions. If the user makes adjustments, they can only adjust the overall proportion of the virtual hand model and cannot adjust the nodes of the virtual hand model according to the user's hand characteristics, which affects the realism of the virtual interaction.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A data glove intelligent interaction method based on virtual scene correction includes:

[0007] Acquire data glove sensor information, which includes sensor location information and sensor type information;

[0008] Based on the data glove initialization requirements, obtain data glove correction action information;

[0009] Based on the sensors integrated into the data glove, the data generated by the sensors when the user makes correction actions with the data glove is collected to obtain correction action data;

[0010] The virtual hand model is adjusted based on the corrected motion data and data glove sensor information;

[0011] Acquire data glove interaction instruction information, which includes interaction instruction information and interaction action information;

[0012] Acquire user data, glove movement information, and virtual scene information;

[0013] Based on the user's data glove action information and virtual scene information, and based on the data glove interaction command information, it is determined whether there is scene interaction. If so, the virtual scene is adjusted according to the user's data glove action information.

[0014] Preferably, adjusting the virtual hand model based on the corrected motion data and data glove sensor information specifically includes:

[0015] Based on the data glove, the motion information is corrected, with all fingers extended and fingers together as the initial motion;

[0016] Obtain virtual hand model information, which includes virtual hand model node information, wherein the nodes in the virtual hand model correspond to the data glove sensor;

[0017] Based on the virtual hand model information, the virtual hand model nodes are classified, with nodes located in the finger part of the virtual hand model being designated as model finger nodes, and nodes located in the palm part of the virtual hand model being designated as model palm nodes.

[0018] Among them, the node corresponding to the root of each finger in the palm node of the model is taken as the root node of the finger;

[0019] Sensor data is collected when the user performs the initial action to obtain initial action data. The initial action data includes action data corresponding to each node in the virtual hand model. The action data includes node position information and node curvature information.

[0020] Make a fist with all fingers as the first initial action;

[0021] Collect the virtual hand model when the user performs the first initialization action to obtain the first action model;

[0022] Based on the first action model, determine whether the virtual hand model needs to be adjusted. If so, adjust the overall proportion of the virtual hand model until there are no abnormalities in the first action model. If not, use the full extension of the fingers and the spread of the fingers as the second initialization action.

[0023] Among them, the first motion model anomalies include fingers passing through the palm, fingers crossing, and fingers dangling in the air;

[0024] Collect the virtual hand model when the user performs the second initialization action to obtain the second action model;

[0025] The virtual hand model is adjusted based on the initial motion data and the second motion model.

[0026] Preferably, the step of collecting sensor data when the user performs the initial action to obtain initial action data specifically includes:

[0027] Collect sensor data when the user performs the initial action to obtain initial action data;

[0028] Based on the initial motion data, and taking the model's palm node as the reference, a model reference plane is constructed based on two-dimensional plane fitting.

[0029] Obtain the distance between each model hand node and the model reference plane, and use the maximum distance between the model hand node and the model reference plane as the reference plane node offset threshold;

[0030] Based on the initial motion data corresponding to the model finger nodes and the model reference plane, calculate the distance between each model finger node and the model reference plane, and obtain the reference offset distance of the finger node.

[0031] Based on the finger node reference offset distance and the reference plane node offset threshold, determine whether the model finger node is on the model reference plane;

[0032] If the distance of the finger node's reference offset exceeds the reference plane node offset threshold, the model's finger node is not on the model's reference plane, the initial action data is unavailable, and the user must re-execute the initial action.

[0033] If the distance of the finger node's reference offset does not exceed the reference plane node offset threshold, then the model's finger node is on the model's reference plane, and the initial motion data is available.

[0034] Preferably, adjusting the virtual hand model based on the initial motion data and the second motion model specifically includes:

[0035] Based on the initial motion data, obtain the model finger node and finger root node corresponding to each finger in the virtual hand model;

[0036] Using the root node of the finger as the origin, the model finger nodes are fitted based on the linear equation to obtain the straight line corresponding to each finger in the virtual hand model;

[0037] Using the straight line of the finger node as the first coordinate axis, and taking the first coordinate axis as the reference and the root node of the finger as the origin, construct the second coordinate axis. The second coordinate axis intersects the first coordinate axis perpendicularly at the origin.

[0038] Construct a calibration plane coordinate system for each finger in the virtual hand model using the first coordinate axis, the second coordinate axis, and the origin;

[0039] Based on the second action model, obtain the second action data, which includes the model finger node data and the finger root node corresponding to each finger in the second action model;

[0040] Align the root node of the finger in the second motion data with the origin of the calibration plane coordinate system. Using the calibration plane coordinate system as a reference, fit the model finger node in the second motion data to obtain the expression of the second motion finger node.

[0041] Based on the calibration plane coordinate system and the expression of the second action finger node, obtain the coordinate axis intercept information, which represents the intercept of the expression of the second action finger node on the first coordinate axis and the second coordinate axis respectively;

[0042] Based on the coordinate axis intercept information, determine whether the virtual hand model needs adjustment;

[0043] If the coordinate axis intercept is not zero, the position of the root node of the finger in the virtual hand model is adjusted according to the coordinate axis intercept information until the coordinate axis intercept is zero.

[0044] Preferably, the step of determining whether scene interaction exists based on user data glove action information and virtual scene information, and on data glove interaction command information, specifically includes:

[0045] Based on the user data glove action information, user action node information is obtained, wherein the user action node information represents the virtual hand model node information corresponding to the user data glove action;

[0046] Based on the user action node information, obtain the node feature position information and node feature curvature information of the virtual hand model corresponding to the user's data glove action;

[0047] Based on the data glove interaction command information, obtain the command node position information and command node curvature information of the virtual hand model corresponding to each data glove interaction command;

[0048] Based on the requirements of contour overlap analysis, recognition weights are set for node position and instruction node curvature.

[0049] The node feature location information and node feature curvature information are compared with the command node location information and command node curvature information. Based on the overlap degree judgment, the node location overlap degree and node curvature overlap degree are obtained.

[0050] Based on the set recognition weights, the overlap of node positions and the overlap of node bending are weighted and summed to obtain the instruction overlap index corresponding to the user's data glove action and each data glove interaction instruction. The instruction overlap index represents the completion degree of the data glove interaction instruction.

[0051] Based on the different interaction commands of the data glove, a threshold for command overlap index is set;

[0052] Based on the instruction overlap index and the instruction overlap index threshold, it is determined whether the instruction overlap index exceeds the instruction overlap index threshold. If so, the virtual scene is preloaded based on the virtual scene information according to the data glove interaction instructions corresponding to the user's data glove actions.

[0053] Preferably, the step of determining whether scene interaction exists based on user data glove action information and virtual scene information, and on data glove interaction command information, further includes:

[0054] Based on the command overlap index, determine whether the user's data glove action has completed the data glove interaction command. If the command overlap index is 1, it means that the user's data glove action has completed the data glove interaction command, and obtain the user interaction command information.

[0055] Based on user interaction command information and virtual scene information, obtain virtual scene interaction target information;

[0056] Based on the virtual scene interaction target information, determine whether there is a virtual interaction target object. If not, adjust the virtual scene according to the virtual scene interaction target information. If yes, obtain the virtual interaction target object information, which includes the virtual object's position information.

[0057] Based on the user's interaction instructions, determine the positional relationship between the virtual interactive target object and the virtual hand model;

[0058] Based on user data and glove motion information, the position of the virtual interactive target object is corrected according to the positional relationship between the virtual interactive target object and the virtual hand model.

[0059] Furthermore, a data glove intelligent interaction system based on virtual scene correction is proposed to implement the interaction method described above, including:

[0060] The main control module is used to determine whether the virtual hand model needs adjustment based on the first action model; to determine whether the model finger nodes are on the model reference plane based on the finger node reference offset distance and the reference plane node offset threshold; to determine whether the virtual hand model needs adjustment based on the coordinate axis intercept information; to determine whether there is scene interaction based on the user data glove action information and virtual scene information, and based on the data glove interaction command information; to correct the action information based on the data glove, taking full finger extension and finger folding as the initial action; to construct the model reference plane based on the model palm node as the reference and based on two-dimensional plane fitting based on the initial action data of the model finger nodes and the model reference plane, to calculate the distance between each model finger node and the model reference plane, to obtain the finger node reference offset distance; to fit the model finger nodes based on the linear equation with the finger root node as the origin, to obtain the finger node straight line corresponding to each finger in the virtual hand model; to preload the virtual scene based on the virtual scene information based on the data glove interaction command corresponding to the user data glove action; and to correct the position of the virtual interaction target object based on the positional relationship between the virtual interaction target object and the virtual hand model.

[0061] The information acquisition module is used to acquire data glove sensor information, sensor location information and sensor type information; based on the data glove initialization requirements, acquire data glove correction action information; based on the sensors integrated in the data glove, collect the data generated by the sensors when the user performs data glove correction actions, acquire correction action data, acquire data glove interaction command information, interaction command information, interaction action information, user data glove action information and virtual scene information;

[0062] The model adjustment module is used to classify virtual hand model nodes according to virtual hand model information, taking nodes located in the finger part of the virtual hand model as model finger nodes and nodes located in the palm part of the virtual hand model as model palm nodes, collecting sensor data when the user performs the initial action to obtain initial action data, collecting data on the virtual hand model when the user performs the first initialization action to obtain the first action model, adjusting the overall proportion of the virtual hand model until there are no abnormalities in the first action model, collecting data on the virtual hand model when the user performs the second initialization action to obtain the second action model, and adjusting the virtual hand model according to the initial action data and the second action model.

[0063] The display module interacts with the main control module and is used to output and display the virtual hand model, virtual scene information, data glove interaction command information, and user data glove action information.

[0064] Optionally, the main control module specifically includes:

[0065] The control unit is used to correct motion information based on the data glove. The initial motion is achieved with all fingers extended and together. Based on the initial motion data and using the model hand nodes as a reference, a model reference plane is constructed using two-dimensional plane fitting. The distance between each model finger node and the model reference plane is calculated based on the initial motion data corresponding to the model finger nodes and the model reference plane, obtaining the reference offset distance of the finger nodes. Using the finger root nodes as the origin, the model finger nodes are fitted based on a straight line equation to obtain the straight line corresponding to each finger node in the virtual hand model. Based on the data glove interaction commands corresponding to the user's data glove actions and virtual scene information, the virtual scene is preloaded. The position of the virtual interaction target object is corrected based on the positional relationship between the virtual interaction target object and the virtual hand model.

[0066] An information receiving unit interacts with an information acquisition module and a model adjustment module to receive data and transmit it to a judgment unit.

[0067] The judgment unit is used to determine whether the virtual hand model needs to be adjusted based on the first action model, to determine whether the model finger node is on the model reference plane based on the finger node reference offset distance and the reference plane node offset threshold, to determine whether the virtual hand model needs to be adjusted based on the coordinate axis intercept information, and to determine whether there is scene interaction based on the user data glove action information and virtual scene information and the data glove interaction instruction information.

[0068] Optionally, the information acquisition module specifically includes:

[0069] The first acquisition unit is used to acquire data glove sensor information, sensor location information and sensor type information, acquire data glove correction action information based on data glove initialization requirements, and collect data generated by the sensors when the user performs data glove correction actions based on the sensors integrated in the data glove to acquire correction action data.

[0070] The second acquisition unit is used to acquire data glove interaction instruction information, interaction instruction information, interaction action information, user data glove action information, and virtual scene information.

[0071] Optionally, the model adjustment module specifically includes:

[0072] The model initialization unit is used to classify virtual hand model nodes according to virtual hand model information, taking the nodes located in the finger part of the virtual hand model as model finger nodes, and the nodes located in the palm part of the virtual hand model as model palm nodes, and collecting sensor data when the user performs the initial action to obtain initial action data.

[0073] The model adjustment unit is used to collect the virtual hand model when the user performs the first initialization action, obtain the first action model, adjust the overall proportion of the virtual hand model until there are no abnormalities in the first action model, collect the virtual hand model when the user performs the second initialization action, obtain the second action model, and adjust the virtual hand model according to the initial action data and the second action model.

[0074] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0075] This invention proposes a data glove intelligent interaction method and system based on virtual scene correction. By analyzing the completion rate of the user's initial actions using initial motion data, the accuracy of subsequent model adjustments is improved. The virtual hand model is further adjusted using a first motion model and a second motion model, ensuring the realism of the virtual interaction. The virtual scene is adjusted and corrected using user data glove action information and data glove interaction command information, ensuring the visual effect of the virtual interaction, ensuring the consistency between the model and real hand actions, and reducing interaction latency. Attached Figure Description

[0076] Figure 1 This is a flowchart of a data glove intelligent interaction method based on virtual scene correction proposed in this invention;

[0077] Figure 2 This is a flowchart of the second action model acquisition process in this invention;

[0078] Figure 3 This is a flowchart of the initial motion data acquisition process in this invention;

[0079] Figure 4 This is a flowchart of the process for obtaining coordinate axis intercept information in this invention;

[0080] Figure 5 This is a block diagram of a data glove intelligent interaction system based on virtual scene correction proposed in this invention. Detailed Implementation

[0081] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0082] Reference Figure 1 - Figure 4 As shown in the figure, an intelligent interaction method for data gloves based on virtual scene correction in an embodiment of the present invention includes:

[0083] Acquire data glove sensor information, which includes sensor location information and sensor type information;

[0084] Based on the data glove initialization requirements, obtain data glove correction action information;

[0085] Based on the sensors integrated into the data glove, the data generated by the sensors when the user makes correction actions with the data glove is collected to obtain correction action data;

[0086] The virtual hand model is adjusted based on the corrected motion data and data glove sensor information;

[0087] Specifically, the virtual hand model is adjusted based on the corrected motion data and the data glove sensor information, including:

[0088] Based on the data glove, the motion information is corrected, with all fingers extended and fingers together as the initial motion;

[0089] Obtain virtual hand model information, which includes virtual hand model node information, wherein the nodes in the virtual hand model correspond to the data glove sensor;

[0090] Based on the virtual hand model information, the virtual hand model nodes are classified, with nodes located in the finger part of the virtual hand model being designated as model finger nodes, and nodes located in the palm part of the virtual hand model being designated as model palm nodes.

[0091] Among them, the node corresponding to the root of each finger in the palm node of the model is taken as the root node of the finger;

[0092] Sensor data is collected when the user performs the initial action to obtain initial action data. The initial action data includes action data corresponding to each node in the virtual hand model. The action data includes node position information and node curvature information.

[0093] Make a fist with all fingers as the first initial action;

[0094] Collect the virtual hand model when the user performs the first initialization action to obtain the first action model;

[0095] Based on the first action model, determine whether the virtual hand model needs to be adjusted. If so, adjust the overall proportion of the virtual hand model until there are no abnormalities in the first action model. If not, use the full extension of the fingers and the spread of the fingers as the second initialization action.

[0096] Among them, the first motion model anomalies include fingers passing through the palm, fingers crossing, and fingers dangling in the air;

[0097] Collect the virtual hand model when the user performs the second initialization action to obtain the second action model;

[0098] The virtual hand model is adjusted based on the initial motion data and the second motion model.

[0099] In this scheme, virtual hand model nodes are classified according to virtual hand model information. The first action model is obtained by collecting the virtual hand model when the user performs the first initialization action. Based on the first action model, it is determined whether the virtual hand model needs to be adjusted. The second action model is obtained by collecting the virtual hand model when the user performs the second initialization action. The virtual hand model is adjusted based on the initial action data and the second action model.

[0100] It is understandable that the sensors of the data glove correspond one-to-one with the nodes of the virtual hand model. Therefore, the data collected by the sensors is used as the node data corresponding to the nodes in the virtual hand model. For the virtual hand model, there are significant differences in the motion trajectories of finger nodes and palm nodes. Different people have different hand sizes and characteristics. However, when performing different specific actions, the motion trajectory of palm nodes often does not differ much, but the motion trajectory of finger nodes is highly correlated with hand size and characteristics. Therefore, classifying the nodes in the virtual hand model facilitates subsequent node analysis and improves the accuracy and efficiency of the analysis.

[0101] It's important to note that by clenching a fist, we can obtain data on the circumference of the palm when it's curled and the tightness of the fingers when they're bent. This data is crucial for building a model of the virtual hand in a clenched fist posture, ensuring that the virtual hand can accurately reflect the actual size and shape of different users' fists when simulating clenching operations such as grasping objects. Therefore, the overall proportions of the virtual hand model are adjusted through the first initialization action. However, the virtual hand model obtained at this point is only an adjustment of the size of a general model and cannot accurately reflect the user's hand characteristics. Therefore, a second initialization action is used to further adjust the virtual hand model.

[0102] Specifically, sensor data is collected when the user performs the initial action to obtain initial action data, which includes:

[0103] Collect sensor data when the user performs the initial action to obtain initial action data;

[0104] Based on the initial motion data, and taking the model's palm node as the reference, a model reference plane is constructed based on two-dimensional plane fitting.

[0105] Obtain the distance between each model hand node and the model reference plane, and use the maximum distance between the model hand node and the model reference plane as the reference plane node offset threshold;

[0106] Based on the initial motion data corresponding to the model finger nodes and the model reference plane, calculate the distance between each model finger node and the model reference plane, and obtain the reference offset distance of the finger node.

[0107] Based on the finger node reference offset distance and the reference plane node offset threshold, determine whether the model finger node is on the model reference plane;

[0108] If the distance of the finger node's reference offset exceeds the reference plane node offset threshold, the model's finger node is not on the model's reference plane, the initial action data is unavailable, and the user must re-execute the initial action.

[0109] If the distance of the finger node's reference offset does not exceed the reference plane node offset threshold, then the model's finger node is on the model's reference plane, and the initial motion data is available.

[0110] In this solution, initial action data is acquired by collecting sensor data when the user performs the initial action, providing a foundation for subsequent analysis. This data is processed to construct a model reference plane based on the model's hand nodes, which removes random deviations caused by individual hand differences and varying wearing positions. Subsequently, by comparing the distances between finger nodes and the reference plane with the reference plane node offset thresholds, abnormal data can be effectively filtered out, ensuring the accuracy and reliability of the initial action data used for interaction and improving the stability of the entire interaction system.

[0111] Understandably, virtual hand models need to match the user's real hand features and movement habits. Constructing a model reference plane based on the palm node helps determine the basic posture and positional relationships of the virtual hand. If the reference offset distance of the finger node is abnormal, it indicates that the initial motion data deviates significantly from the default posture of the virtual hand model, requiring the user to re-execute the action. This allows the virtual hand model to better adapt to the user during the initialization phase, enhancing the naturalness of the interaction.

[0112] Specifically, the virtual hand model is adjusted based on the initial motion data and the second motion model, including:

[0113] Based on the initial motion data, obtain the model finger node and finger root node corresponding to each finger in the virtual hand model;

[0114] Using the root node of the finger as the origin, the model finger nodes are fitted based on the linear equation to obtain the straight line corresponding to each finger in the virtual hand model;

[0115] Using the straight line of the finger node as the first coordinate axis, and taking the first coordinate axis as the reference and the root node of the finger as the origin, construct the second coordinate axis. The second coordinate axis intersects the first coordinate axis perpendicularly at the origin.

[0116] Construct a calibration plane coordinate system for each finger in the virtual hand model using the first coordinate axis, the second coordinate axis, and the origin;

[0117] Based on the second action model, obtain the second action data, which includes the model finger node data and the finger root node corresponding to each finger in the second action model;

[0118] Align the root node of the finger in the second motion data with the origin of the calibration plane coordinate system. Using the calibration plane coordinate system as a reference, fit the model finger node in the second motion data to obtain the expression of the second motion finger node.

[0119] Based on the calibration plane coordinate system and the expression of the second action finger node, obtain the coordinate axis intercept information, which represents the intercept of the expression of the second action finger node on the first coordinate axis and the second coordinate axis respectively;

[0120] Based on the coordinate axis intercept information, determine whether the virtual hand model needs adjustment;

[0121] If the coordinate axis intercept is not zero, the position of the root node of the finger in the virtual hand model is adjusted according to the coordinate axis intercept information until the coordinate axis intercept is zero.

[0122] In this scheme, by using the root node of the finger as the origin and fitting the finger node of the model based on the linear equation, the straight line corresponding to each finger in the virtual hand model is obtained. The straight line of the finger node is used as the first coordinate axis. With the first coordinate axis as the reference and the root node of the finger as the origin, the second coordinate axis is constructed, and a calibration plane coordinate system corresponding to each finger in the virtual hand model is established. According to the second action model, the second action data is obtained. The root node of the finger in the second action data is aligned with the origin of the calibration plane coordinate system. With the calibration plane coordinate system as the reference, the model finger node in the second action data is fitted to obtain the expression of the second action finger node. The coordinate axis intercept information is obtained. Based on the coordinate axis intercept information, it is determined whether the virtual hand model needs to be adjusted.

[0123] It is understandable that by constructing a calibration plane coordinate system with the root node of each finger as the origin, a unique and precise coordinate reference framework is established for each finger. This allows the position and posture of each finger in the virtual hand model to be described and analyzed in an independent and well-defined coordinate system, which helps to capture and analyze finger movement information in greater detail, laying the foundation for subsequent precise interaction. Aligning the root node of the finger in the second action data with the origin of the calibration plane coordinate system unifies the finger data under different actions into the previously established standard coordinate system. This eliminates interference caused by differences in the starting position or posture of different actions, making different action data comparable and consistent, facilitating unified analysis and processing of the model's finger nodes in subsequent steps. In this scheme, the initial action and the second initial action are selected for comparison. Both actions involve straightening the fingers, but the difference lies in whether the fingers are joined or spread apart. Therefore, the changes in the nodes during the action change are more clearly reflected. The intercept value intuitively reflects the degree of offset of the finger action in the two coordinate axes and other key information. This provides specific data basis for judging whether the virtual hand model needs to be adjusted and how to adjust it, thus transforming the analysis of virtual hand actions from qualitative description to quantitative analysis.

[0124] Acquire data glove interaction instruction information, which includes interaction instruction information and interaction action information;

[0125] Acquire user data, glove movement information, and virtual scene information;

[0126] Based on the user's data glove action information and virtual scene information, and based on the data glove interaction command information, it is determined whether there is scene interaction. If so, the virtual scene is adjusted according to the user's data glove action information.

[0127] Specifically, based on user data glove action information and virtual scene information, and based on data glove interaction command information, it is determined whether scene interaction exists, including:

[0128] Based on the user data glove action information, user action node information is obtained, wherein the user action node information represents the virtual hand model node information corresponding to the user data glove action;

[0129] Based on the user action node information, obtain the node feature position information and node feature curvature information of the virtual hand model corresponding to the user's data glove action;

[0130] Based on the data glove interaction command information, obtain the command node position information and command node curvature information of the virtual hand model corresponding to each data glove interaction command;

[0131] Based on the requirements of contour overlap analysis, recognition weights are set for node position and instruction node curvature.

[0132] The node feature location information and node feature curvature information are compared with the command node location information and command node curvature information. Based on the overlap degree judgment, the node location overlap degree and node curvature overlap degree are obtained.

[0133] Based on the set recognition weights, the overlap of node positions and the overlap of node bending are weighted and summed to obtain the instruction overlap index corresponding to the user's data glove action and each data glove interaction instruction. The instruction overlap index represents the completion degree of the data glove interaction instruction.

[0134] Based on the different interaction commands of the data glove, a threshold for command overlap index is set;

[0135] Based on the instruction overlap index and the instruction overlap index threshold, it is determined whether the instruction overlap index exceeds the instruction overlap index threshold. If so, the virtual scene is preloaded based on the virtual scene information according to the data glove interaction instructions corresponding to the user's data glove actions.

[0136] In this solution, by acquiring the node feature positions and curvature information of the virtual hand model corresponding to the user's actions, the representation of the user's hand movements on the virtual hand model can be accurately captured. Simultaneously, acquiring the instruction node positions and curvature information of the virtual hand model corresponding to each data glove interaction command provides a clear standard and basis for subsequent comparative analysis. This step ensures the accurate description of user actions and interaction commands, which is the foundation for achieving intelligent interaction. Setting recognition weights for node positions and instruction node curvature based on contour overlap analysis requirements reflects a differentiated consideration of the importance of different features in the interaction process. For example, in some interaction scenarios, node position may be more critical for accurate command execution, while in other scenarios, node curvature may play a decisive role. By setting weights, key features can be highlighted, making subsequent analysis more consistent with actual interaction needs. Comparing the relevant information of the user's actions with the relevant information of the commands to obtain the node position overlap and node curvature overlap quantifies the degree of matching between the user's actions and the interaction commands. This comparative analysis provides an objective indicator for evaluating whether the user's data glove actions conform to the interaction commands, helping to determine whether the user correctly executed the commands or to what extent they closely approximate the command requirements. The command coincidence index is obtained by weighted summing of node position coincidence and node curvature coincidence. This index can intuitively represent the completion degree of data glove interaction commands. It provides the interaction system with a clear numerical indicator, making it easier for the system to quickly determine the consistency between user actions and commands, thereby determining subsequent operations and feedback.

[0137] It is important to note that the gestures corresponding to different data glove interaction commands are also quite different, and the sensitivity of different types of gestures to virtual hand model nodes is also different. Therefore, in this embodiment, the command overlap index threshold for data glove interaction commands of fine operation type (such as grabbing, slapping, etc.) is 0.65, and the command overlap index threshold for data glove interaction commands of fast type (such as waving, clenching fist, pointing, etc.) is 0.85. This threshold is dynamically calculated by the sliding window algorithm and automatically adjusted based on the mean ± 1.5 times standard deviation of the historical 10 frames of data glove interaction command action.

[0138] Specifically, based on user data glove action information and virtual scene information, and based on data glove interaction command information, determining whether scene interaction exists also includes:

[0139] Based on the command overlap index, determine whether the user's data glove action has completed the data glove interaction command. If the command overlap index is 1, it means that the user's data glove action has completed the data glove interaction command, and obtain the user interaction command information.

[0140] Based on user interaction command information and virtual scene information, obtain virtual scene interaction target information;

[0141] Based on the virtual scene interaction target information, determine whether there is a virtual interaction target object. If not, adjust the virtual scene according to the virtual scene interaction target information. If yes, obtain the virtual interaction target object information, which includes the virtual object's position information.

[0142] Based on the user's interaction instructions, determine the positional relationship between the virtual interactive target object and the virtual hand model;

[0143] Based on user data and glove motion information, the position of the virtual interactive target object is corrected according to the positional relationship between the virtual interactive target object and the virtual hand model.

[0144] In this solution, obtaining virtual scene interaction target information through user interaction command information and virtual scene information helps the system understand the specific goals the user wants to achieve in the virtual scene. This step combines the user's commands with the actual situation of the virtual scene, enabling the system to accurately locate the core content of the interaction and provide clear goal guidance for subsequent operations. Based on the virtual scene interaction target information, the system determines whether a virtual interactive target object exists, obtains the information of the virtual interactive target object, and determines its positional relationship with the virtual hand model, providing the necessary spatial information foundation for realizing the interaction between the virtual hand and the virtual object. Clarifying the positional relationship between the two allows the system to better simulate the user's actual operations in the virtual scene, such as grasping, moving, and placing, thereby achieving a more natural and realistic interaction effect. Based on the user's data glove action information, the position of the virtual interactive target object is corrected according to the positional relationship between the virtual interactive target object and the virtual hand model, allowing for real-time adjustment of the virtual object's position to better match the user's hand movements. This process improves the accuracy of the interaction, reduces errors caused by discrepancies between the virtual object's position and the user's actual operations, enhances the realism and credibility of the virtual interaction, and makes the user's operations in the virtual scene smoother and more natural.

[0145] Reference Figure 5 As shown, further, combining the above-mentioned data glove intelligent interaction method based on virtual scene correction, a data glove intelligent interaction system based on virtual scene correction is proposed, including:

[0146] The main control module is used to determine whether the virtual hand model needs adjustment based on the first action model; to determine whether the model finger nodes are on the model reference plane based on the finger node reference offset distance and the reference plane node offset threshold; to determine whether the virtual hand model needs adjustment based on the coordinate axis intercept information; to determine whether there is scene interaction based on the user data glove action information and virtual scene information, and based on the data glove interaction command information; to correct the action information based on the data glove, taking full finger extension and finger folding as the initial action; to construct the model reference plane based on the model palm node as the reference and based on two-dimensional plane fitting based on the initial action data of the model finger nodes and the model reference plane, to calculate the distance between each model finger node and the model reference plane, to obtain the finger node reference offset distance; to fit the model finger nodes based on the linear equation with the finger root node as the origin, to obtain the finger node straight line corresponding to each finger in the virtual hand model; to preload the virtual scene based on the virtual scene information based on the data glove interaction command corresponding to the user data glove action; and to correct the position of the virtual interaction target object based on the positional relationship between the virtual interaction target object and the virtual hand model.

[0147] The information acquisition module is used to acquire data glove sensor information, sensor location information and sensor type information; based on the data glove initialization requirements, acquire data glove correction action information; based on the sensors integrated in the data glove, collect the data generated by the sensors when the user performs data glove correction actions, acquire correction action data, acquire data glove interaction command information, interaction command information, interaction action information, user data glove action information and virtual scene information;

[0148] The model adjustment module is used to classify virtual hand model nodes according to virtual hand model information, taking nodes located in the finger part of the virtual hand model as model finger nodes and nodes located in the palm part of the virtual hand model as model palm nodes, collecting sensor data when the user performs the initial action to obtain initial action data, collecting data on the virtual hand model when the user performs the first initialization action to obtain the first action model, adjusting the overall proportion of the virtual hand model until there are no abnormalities in the first action model, collecting data on the virtual hand model when the user performs the second initialization action to obtain the second action model, and adjusting the virtual hand model according to the initial action data and the second action model.

[0149] The display module interacts with the main control module and is used to output and display the virtual hand model, virtual scene information, data glove interaction command information, and user data glove action information.

[0150] The main control module specifically includes:

[0151] The control unit is used to correct motion information based on the data glove. The initial motion is achieved with all fingers extended and together. Based on the initial motion data and using the model hand nodes as a reference, a model reference plane is constructed using two-dimensional plane fitting. The distance between each model finger node and the model reference plane is calculated based on the initial motion data corresponding to the model finger nodes and the model reference plane, obtaining the reference offset distance of the finger nodes. Using the finger root nodes as the origin, the model finger nodes are fitted based on a straight line equation to obtain the straight line corresponding to each finger node in the virtual hand model. Based on the data glove interaction commands corresponding to the user's data glove actions and virtual scene information, the virtual scene is preloaded. The position of the virtual interaction target object is corrected based on the positional relationship between the virtual interaction target object and the virtual hand model.

[0152] An information receiving unit interacts with an information acquisition module and a model adjustment module to receive data and transmit it to a judgment unit.

[0153] The judgment unit is used to determine whether the virtual hand model needs to be adjusted based on the first action model, to determine whether the model finger node is on the model reference plane based on the finger node reference offset distance and the reference plane node offset threshold, to determine whether the virtual hand model needs to be adjusted based on the coordinate axis intercept information, and to determine whether there is scene interaction based on the user data glove action information and virtual scene information and the data glove interaction instruction information.

[0154] The information acquisition module specifically includes:

[0155] The first acquisition unit is used to acquire data glove sensor information, sensor location information and sensor type information, acquire data glove correction action information based on data glove initialization requirements, and collect data generated by the sensors when the user performs data glove correction actions based on the sensors integrated in the data glove to acquire correction action data.

[0156] The second acquisition unit is used to acquire data glove interaction instruction information, interaction instruction information, interaction action information, user data glove action information, and virtual scene information.

[0157] The model adjustment module specifically includes:

[0158] The model initialization unit is used to classify virtual hand model nodes according to virtual hand model information, taking the nodes located in the finger part of the virtual hand model as model finger nodes, and the nodes located in the palm part of the virtual hand model as model palm nodes, and collecting sensor data when the user performs the initial action to obtain initial action data.

[0159] The model adjustment unit is used to collect the virtual hand model when the user performs the first initialization action, obtain the first action model, adjust the overall proportion of the virtual hand model until there are no abnormalities in the first action model, collect the virtual hand model when the user performs the second initialization action, obtain the second action model, and adjust the virtual hand model according to the initial action data and the second action model.

[0160] In summary, the advantages of this invention are as follows: by collecting sensor data when the user performs the initial action, initial action data is obtained; by analyzing the completion rate of the user's initial action using the initial action data, the accuracy of subsequent model adjustments is improved; by further adjusting the virtual hand model using the first action model and the second action model, the realism of the virtual interaction is ensured; by using the user's data glove action information and virtual scene information, and based on the data glove interaction command information, it is determined whether scene interaction exists, and the virtual scene is adjusted and corrected, ensuring the visual effect of the virtual interaction, ensuring the consistency between the model and real hand actions, and reducing interaction latency.

[0161] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A data glove intelligent interaction method based on virtual scene correction, characterized in that, include: Acquire data glove sensor information, which includes sensor location information and sensor type information; Based on the data glove initialization requirements, obtain data glove correction action information; Based on the sensors integrated into the data glove, the data generated by the sensors when the user makes correction actions with the data glove is collected to obtain correction action data; The virtual hand model is adjusted based on the corrected motion data and data glove sensor information; Acquire data glove interaction instruction information, which includes interaction instruction information and interaction action information; Acquire user data, glove movement information, and virtual scene information; Based on user data glove action information and virtual scene information, and based on data glove interaction command information, determine whether there is scene interaction; if so, adjust the virtual scene according to user data glove action information. The adjustment of the virtual hand model based on the corrected motion data and data glove sensor information specifically includes: Based on the data glove, the motion information is corrected, with all fingers extended and fingers together as the initial motion; Obtain virtual hand model information, which includes virtual hand model node information, wherein the nodes in the virtual hand model correspond to the data glove sensor; Based on the virtual hand model information, the virtual hand model nodes are classified, with nodes located in the finger part of the virtual hand model being designated as model finger nodes, and nodes located in the palm part of the virtual hand model being designated as model palm nodes. Among them, the node corresponding to the root of each finger in the palm node of the model is taken as the root node of the finger; Sensor data is collected when the user performs the initial action to obtain initial action data. The initial action data includes action data corresponding to each node in the virtual hand model. The action data includes node position information and node curvature information. Make a fist with all fingers as the first initial action; Collect the virtual hand model when the user performs the first initialization action to obtain the first action model; Based on the first action model, determine whether the virtual hand model needs to be adjusted. If so, adjust the overall proportion of the virtual hand model until there are no abnormalities in the first action model. If not, use the full extension of the fingers and the spread of the fingers as the second initialization action. Among them, the first motion model anomalies include fingers passing through the palm, fingers crossing, and fingers dangling in the air; Collect the virtual hand model when the user performs the second initialization action to obtain the second action model; The virtual hand model is adjusted based on the initial motion data and the second motion model; The adjustment of the virtual hand model based on the initial motion data and the second motion model specifically includes: Based on the initial motion data, obtain the model finger node and finger root node corresponding to each finger in the virtual hand model; Using the root node of the finger as the origin, the model finger nodes are fitted based on the linear equation to obtain the straight line corresponding to each finger in the virtual hand model; Using the straight line of the finger node as the first coordinate axis, and taking the first coordinate axis as the reference and the root node of the finger as the origin, construct the second coordinate axis. The second coordinate axis intersects the first coordinate axis perpendicularly at the origin. Construct a calibration plane coordinate system for each finger in the virtual hand model using the first coordinate axis, the second coordinate axis, and the origin; Based on the second action model, obtain the second action data, which includes the model finger node data and the finger root node corresponding to each finger in the second action model; Align the root node of the finger in the second motion data with the origin of the calibration plane coordinate system. Using the calibration plane coordinate system as a reference, fit the model finger node in the second motion data to obtain the expression of the second motion finger node. Based on the calibration plane coordinate system and the expression of the second action finger node, obtain the coordinate axis intercept information, which represents the intercept of the expression of the second action finger node on the first coordinate axis and the second coordinate axis respectively; Based on the coordinate axis intercept information, determine whether the virtual hand model needs adjustment; If the coordinate axis intercept is not zero, the position of the root node of the finger in the virtual hand model is adjusted according to the coordinate axis intercept information until the coordinate axis intercept is zero.

2. The intelligent interaction method for data gloves based on virtual scene correction according to claim 1, characterized in that, The process of collecting sensor data when the user performs the initial action to obtain initial action data specifically includes: Collect sensor data when the user performs the initial action to obtain initial action data; Based on the initial motion data, and taking the model's palm node as the reference, a model reference plane is constructed based on two-dimensional plane fitting. Obtain the distance between each model hand node and the model reference plane, and use the maximum distance between the model hand node and the model reference plane as the reference plane node offset threshold; Based on the initial motion data corresponding to the model finger nodes and the model reference plane, calculate the distance between each model finger node and the model reference plane, and obtain the reference offset distance of the finger node. Based on the finger node reference offset distance and the reference plane node offset threshold, determine whether the model finger node is on the model reference plane; If the distance of the finger node's reference offset exceeds the reference plane node offset threshold, the model's finger node is not on the model's reference plane, the initial action data is unavailable, and the user must re-execute the initial action. If the distance of the finger node's reference offset does not exceed the reference plane node offset threshold, then the model's finger node is on the model's reference plane, and the initial motion data is available.

3. The intelligent interaction method for data gloves based on virtual scene correction according to claim 1, characterized in that, The step of determining whether scene interaction exists based on user data glove action information and virtual scene information, and on data glove interaction command information, specifically includes: Based on the user data glove action information, user action node information is obtained, wherein the user action node information represents the virtual hand model node information corresponding to the user data glove action; Based on the user action node information, obtain the node feature position information and node feature curvature information of the virtual hand model corresponding to the user's data glove action; Based on the data glove interaction command information, obtain the command node position information and command node curvature information of the virtual hand model corresponding to each data glove interaction command; Based on the requirements of contour overlap analysis, recognition weights are set for node position and instruction node curvature. The node feature location information and node feature curvature information are compared with the command node location information and command node curvature information. Based on the overlap degree judgment, the node location overlap degree and node curvature overlap degree are obtained. Based on the set recognition weights, the overlap of node positions and the overlap of node bending are weighted and summed to obtain the instruction overlap index corresponding to the user's data glove action and each data glove interaction instruction. The instruction overlap index represents the completion degree of the data glove interaction instruction. Based on the different interaction commands of the data glove, a threshold for command overlap index is set; Based on the instruction overlap index and the instruction overlap index threshold, it is determined whether the instruction overlap index exceeds the instruction overlap index threshold. If so, the virtual scene is preloaded based on the virtual scene information according to the data glove interaction instructions corresponding to the user's data glove actions.

4. The intelligent interaction method for data gloves based on virtual scene correction according to claim 3, characterized in that, The step of determining whether scene interaction exists based on user data glove action information and virtual scene information, and on data glove interaction command information, further includes: Based on the command overlap index, determine whether the user's data glove action has completed the data glove interaction command. If the command overlap index is 1, it means that the user's data glove action has completed the data glove interaction command, and obtain the user interaction command information. Based on user interaction command information and virtual scene information, obtain virtual scene interaction target information; Based on the virtual scene interaction target information, determine whether there is a virtual interaction target object. If not, adjust the virtual scene according to the virtual scene interaction target information. If yes, obtain the virtual interaction target object information, which includes the virtual object's position information. Based on the user's interaction instructions, determine the positional relationship between the virtual interactive target object and the virtual hand model; Based on user data and glove motion information, the position of the virtual interactive target object is corrected according to the positional relationship between the virtual interactive target object and the virtual hand model.

5. A data glove intelligent interaction system based on virtual scene correction, used to implement the interaction method as described in any one of claims 1-4, characterized in that, include: The main control module is used to determine whether the virtual hand model needs adjustment based on the first action model; to determine whether the model finger nodes are on the model reference plane based on the finger node reference offset distance and the reference plane node offset threshold; to determine whether the virtual hand model needs adjustment based on the coordinate axis intercept information; to determine whether there is scene interaction based on the user data glove action information and virtual scene information, and based on the data glove interaction command information; to correct the action information based on the data glove, taking full finger extension and finger folding as the initial action; to construct the model reference plane based on the model palm node as the reference and based on two-dimensional plane fitting based on the initial action data of the model finger nodes and the model reference plane, to calculate the distance between each model finger node and the model reference plane, to obtain the finger node reference offset distance; to fit the model finger nodes based on the linear equation with the finger root node as the origin, to obtain the finger node straight line corresponding to each finger in the virtual hand model; to preload the virtual scene based on the virtual scene information based on the data glove interaction command corresponding to the user data glove action; and to correct the position of the virtual interaction target object based on the positional relationship between the virtual interaction target object and the virtual hand model. The information acquisition module is used to acquire data glove sensor information, sensor location information and sensor type information; based on the data glove initialization requirements, acquire data glove correction action information; based on the sensors integrated in the data glove, collect the data generated by the sensors when the user performs data glove correction actions, acquire correction action data, acquire data glove interaction command information, interaction command information, interaction action information, user data glove action information and virtual scene information; The model adjustment module is used to classify virtual hand model nodes according to virtual hand model information, taking nodes located in the finger part of the virtual hand model as model finger nodes and nodes located in the palm part of the virtual hand model as model palm nodes, collecting sensor data when the user performs the initial action to obtain initial action data, collecting data on the virtual hand model when the user performs the first initialization action to obtain the first action model, adjusting the overall proportion of the virtual hand model until there are no abnormalities in the first action model, collecting data on the virtual hand model when the user performs the second initialization action to obtain the second action model, and adjusting the virtual hand model according to the initial action data and the second action model. The display module interacts with the main control module and is used to output and display the virtual hand model, virtual scene information, data glove interaction command information, and user data glove action information.

6. The intelligent interactive system for data gloves based on virtual scene correction according to claim 5, characterized in that, The main control module specifically includes: The control unit is used to correct motion information based on the data glove. The initial motion is achieved with all fingers extended and together. Based on the initial motion data and using the model hand nodes as a reference, a model reference plane is constructed using two-dimensional plane fitting. The distance between each model finger node and the model reference plane is calculated based on the initial motion data corresponding to the model finger nodes and the model reference plane, obtaining the reference offset distance of the finger nodes. Using the finger root nodes as the origin, the model finger nodes are fitted based on a straight line equation to obtain the straight line corresponding to each finger node in the virtual hand model. Based on the data glove interaction commands corresponding to the user's data glove actions and virtual scene information, the virtual scene is preloaded. The position of the virtual interaction target object is corrected based on the positional relationship between the virtual interaction target object and the virtual hand model. An information receiving unit interacts with an information acquisition module and a model adjustment module to receive data and transmit it to a judgment unit. The judgment unit is used to determine whether the virtual hand model needs to be adjusted based on the first action model, to determine whether the model finger node is on the model reference plane based on the finger node reference offset distance and the reference plane node offset threshold, to determine whether the virtual hand model needs to be adjusted based on the coordinate axis intercept information, and to determine whether there is scene interaction based on the user data glove action information and virtual scene information and the data glove interaction instruction information.

7. The intelligent interactive system for data gloves based on virtual scene correction according to claim 5, characterized in that, The information acquisition module specifically includes: The first acquisition unit is used to acquire data glove sensor information, sensor location information and sensor type information, acquire data glove correction action information based on data glove initialization requirements, and collect data generated by the sensors when the user performs data glove correction actions based on the sensors integrated in the data glove to acquire correction action data. The second acquisition unit is used to acquire data glove interaction instruction information, interaction instruction information, interaction action information, user data glove action information, and virtual scene information.

8. The intelligent interactive system for data gloves based on virtual scene correction according to claim 5, characterized in that, The model adjustment module specifically includes: The model initialization unit is used to classify virtual hand model nodes according to virtual hand model information, taking the nodes located in the finger part of the virtual hand model as model finger nodes, and the nodes located in the palm part of the virtual hand model as model palm nodes, and collecting sensor data when the user performs the initial action to obtain initial action data. The model adjustment unit is used to collect the virtual hand model when the user performs the first initialization action, obtain the first action model, adjust the overall proportion of the virtual hand model until there are no abnormalities in the first action model, collect the virtual hand model when the user performs the second initialization action, obtain the second action model, and adjust the virtual hand model according to the initial action data and the second action model.

Images