Data glove intelligent interaction method and system based on virtual scene correction

Abstract

The invention discloses a data glove intelligent interaction method and system based on virtual scene correction, and relates to the technical field of virtual reality, and the method comprises the steps: obtaining data glove sensor information, obtaining data glove correction action information based on a data glove initialization demand, and obtaining data glove correction action information based on a data glove integrated sensor. And collecting data generated by the sensor when the user performs the data glove correction action, and obtaining correction action data. According to the method, the accuracy of subsequent model adjustment is improved by analyzing the completion degree of the initial action of the user through the initial action data, the authenticity of virtual interaction is ensured by further adjusting the virtual hand model through the first action model and the second action model, and the user experience is improved through the action information of the user data glove and the interaction instruction information of the data glove. And the virtual scene is adjusted and corrected, so that the visual effect of virtual interaction is ensured, the consistency of the model and the real hand action is ensured, and the interaction delay is reduced.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and specifically relates to a data glove intelligent interaction method and system based on virtual scene correction. Background Art

[0002] With the continuous development of virtual reality (VR) and augmented reality (AR) technologies, natural interaction methods have become increasingly important. As a new type of human-computer interaction tool, data gloves are currently widely used in more and more virtual reality environments. Its purpose is to be able to obtain, through built-in sensors, data on the bending, abduction, and other angles of each effective part of the hand, including the palm, fingers, and wrist, in real time, and on this basis, invert gestures to achieve interaction with the virtual scene. The presence of soft tissues in the human hand determines that the human hand cannot be compared with the ordinary rigid rod hinges applied to robotic hands. The movement of a certain joint of the hand not only affects the readings of the corresponding sensors, but also, through the interaction of soft tissues, causes changes in the readings of other sensors. This requires that in order to ensure a certain accuracy, decoupling calculations must be performed on the obtained inverse mapping.

[0003] Currently, there are still problems in the interaction of data gloves, such as being unable to adjust the virtual hand model according to specific actions. Often, only feedback on the actions of the data gloves is based on a general virtual hand model, and there is a certain error between the actions and the real actions. If the user makes adjustments by themselves, they can often only adjust the overall proportion of the virtual hand model and cannot adjust the nodes of the virtual hand model according to the user's hand characteristics, which affects the authenticity of virtual interaction. Summary of the Invention

[0004] To solve the above technical problems, a data glove intelligent interaction method and system based on virtual scene correction are provided. The technical solution of the present invention solves the problems proposed in the above background art, such as being unable to adjust the virtual hand model according to specific actions, often only providing feedback on the actions of the data gloves based on a general virtual hand model, having a certain error between the actions and the real actions, and if the user makes adjustments by themselves, they can often only adjust the overall proportion of the virtual hand model and cannot adjust the nodes of the virtual hand model according to the user's hand characteristics, which affects the authenticity of virtual interaction.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A data glove intelligent interaction method based on virtual scene correction, comprising:

[0007] Obtaining data glove sensor information, where the data glove sensor information includes sensor position information and sensor type information;

[0008] Based on the data glove initialization requirements, obtain the data glove correction action information;

[0009] Based on the sensors integrated in the data gloves, the data generated by the sensors when the user performs correction actions on the data gloves are collected to obtain correction action data;

[0010] Adjust the virtual hand model according to the corrected motion data and the data glove sensor information;

[0011] Acquire data glove interaction instruction information, wherein the data glove interaction instruction information includes interaction instruction information and interaction action information;

[0012] Obtain user data glove motion information and virtual scene information;

[0013] According to the user data glove action information and the virtual scene information, based on the data glove interaction instruction information, it is determined whether there is scene interaction. If so, the virtual scene is adjusted according to the user data glove action information.

[0014] Preferably, the adjusting the virtual hand model according to the corrected motion data and the data glove sensor information specifically includes:

[0015] According to the corrected motion information of the data gloves, the initial motion is to straighten all fingers and close them together;

[0016] Acquire virtual hand model information, wherein the virtual hand model information includes virtual hand model node information, and the nodes in the virtual hand model correspond to the data glove sensors;

[0017] According to the virtual hand model information, the virtual hand model nodes are classified, and the nodes located in the finger part of the virtual hand model are used as model finger nodes, and the nodes located in the palm part of the virtual hand model are used as model palm nodes;

[0018] Among them, the node corresponding to the root of each finger in the palm node of the model is used as the finger root node;

[0019] Collecting sensor data when the user performs an initial action to obtain initial action data, wherein the initial action data includes action data corresponding to each node in the virtual hand model, and the action data includes node position information and node curvature information;

[0020] Make a full-fingered fist as the first initialization action;

[0021] Collecting a virtual hand model when the user performs a first initialization action to obtain a first action model;

[0022] According to the first motion model, determine whether the virtual hand model needs to be adjusted. If so, adjust the overall proportion of the virtual hand model until there is no abnormality in the first motion model. If not, take fully extending all fingers and spreading the fingers apart as the second initialization action;

[0023] Among them, the abnormalities of the first motion model include fingers passing through the palm, fingers crossing, and fingers hanging in the air;

[0024] Collect the virtual hand model when the user performs the second initialization action to obtain the second motion model;

[0025] Adjust the virtual hand model according to the initial motion data and the second motion model.

[0026] Preferably, the collecting the sensor data when the user performs the initial action to obtain the initial motion data specifically includes:

[0027] Collect the sensor data when the user performs the initial action to obtain the initial motion data;

[0028] According to the initial motion data, taking the model palm nodes as the benchmark, construct a model reference plane based on two-dimensional plane fitting;

[0029] Obtain the distance between each model palm node and the model reference plane, and take the maximum value of the distance between the model palm node and the model reference plane as the reference plane node offset threshold;

[0030] According to the initial motion data corresponding to the model finger nodes and the model reference plane, calculate the distance between each model finger node and the model reference plane to obtain the finger node reference offset distance;

[0031] According to the finger node reference offset distance and the reference plane node offset threshold, determine whether the model finger nodes are on the model reference plane;

[0032] Among them, if the finger node reference offset distance exceeds the reference plane node offset threshold, the model finger nodes are not on the model reference plane, the initial motion data is unavailable, and the user needs to perform the initial action again;

[0033] If the finger node reference offset distance does not exceed the reference plane node offset threshold, the model finger nodes are on the model reference plane, and the initial motion data is available.

[0034] Preferably, the adjusting the virtual hand model according to the initial motion data and the second motion model specifically includes:

[0035] According to the initial motion data, obtain the model finger nodes and finger root nodes corresponding to each finger in the virtual hand model;

[0036] Taking the finger root node as the origin, based on the straight-line equation, fitting the model finger nodes to obtain the straight lines of the finger nodes corresponding to each finger in the virtual hand model;

[0037] Taking the straight line of the finger node as the first coordinate axis, based on the first coordinate axis and with the finger root node as the origin, constructing a second coordinate axis, where the second coordinate axis is perpendicular to the first coordinate axis and intersects at the origin;

[0038] Using the first coordinate axis, the second coordinate axis and the origin to construct the calibration plane coordinate system corresponding to each finger in the virtual hand model;

[0039] According to the second action model, obtaining second action data, where the second action data includes the model finger node data in the second action model and the finger root node corresponding to each finger;

[0040] Aligning the finger root node in the second action data with the origin of the calibration plane coordinate system, and based on the calibration plane coordinate system, fitting the model finger nodes in the second action data to obtain the expression of the second action finger nodes;

[0041] According to the calibration plane coordinate system and the expression of the second action finger nodes, obtaining the axis intercept information, where the axis intercept information represents the intercepts of the expression of the second action finger nodes on the first coordinate axis and the second coordinate axis respectively;

[0042] Judging whether the virtual hand model needs to be adjusted according to the axis intercept information;

[0043] Among them, if the axis intercept is not zero, adjusting the position of the finger root node in the virtual hand model according to the axis intercept information until the axis intercept is zero.

[0044] Preferably, the judging whether there is a scene interaction based on the data glove interaction instruction information according to the user data glove action information and the virtual scene information specifically includes:

[0045] According to the user data glove action information, obtaining user action node information, where the user action node information represents the virtual hand model node information corresponding to the user data glove action;

[0046] According to the user action node information, obtaining the node feature position information and the node feature curvature information of the virtual hand model corresponding to the user data glove action;

[0047] According to the data glove interaction instruction information, obtaining the instruction node position information and the instruction node curvature information of the virtual hand model corresponding to each data glove interaction instruction;

[0048] Based on the contour coincidence analysis requirement, setting recognition weights for the node position and the instruction node curvature;

[0049] Compare the node feature position information and the node feature curvature information with the instruction node position information and the instruction node curvature information, and based on the coincidence degree judgment, obtain the node position coincidence degree and the node curvature coincidence degree;

[0050] Based on the set recognition weights, perform a weighted sum of the node position coincidence degree and the node curvature coincidence degree to obtain an instruction coincidence index corresponding to the user data glove action and each data glove interaction instruction, and the instruction coincidence index represents the completion degree of the data glove interaction instruction;

[0051] Based on different data glove interaction instructions, set an instruction coincidence index threshold;

[0052] According to the instruction coincidence index and the instruction coincidence index threshold, judge whether the instruction coincidence index exceeds the instruction coincidence index threshold. If so, based on the virtual scene information, preload the virtual scene according to the data glove interaction instruction corresponding to the user data glove action.

[0053] Preferably, the method for judging whether there is a scene interaction based on the user data glove action information and the virtual scene information and based on the data glove interaction instruction information further includes:

[0054] According to the instruction coincidence index, judge whether the user data glove action completes the data glove interaction instruction. If the instruction coincidence index is 1, it means that the user data glove action completes the data glove interaction instruction, and obtain the user interaction instruction information;

[0055] Based on the user interaction instruction information and the virtual scene information, obtain the virtual scene interaction target information;

[0056] According to the virtual scene interaction target information, judge whether there is a virtual interaction target object. If not, adjust the virtual scene according to the virtual scene interaction target information. If so, obtain the virtual interaction target object information, and the virtual interaction target object information includes the virtual object position information;

[0057] According to the user interaction instruction information, determine the position relationship between the virtual interaction target object and the virtual hand model;

[0058] Based on the user data glove action information, correct the position of the virtual interaction target object according to the position relationship between the virtual interaction target object and the virtual hand model.

[0059] Furthermore, a data glove intelligent interaction system based on virtual scene correction is proposed for implementing the interaction method as described above, including:

[0060] A main control module, the main control module is used to determine whether the virtual hand model needs to be adjusted according to the first action model, determine whether the model finger node is in the model reference plane according to the finger node reference offset distance and the reference plane node offset threshold, determine whether the virtual hand model needs to be adjusted according to the coordinate axis intercept information, determine whether there is scene interaction according to the user data glove action information and virtual scene information, based on the data glove interaction instruction information, correct the action information according to the data glove, take all fingers straightened and fingers together as the initial action, build the model reference plane based on two-dimensional plane fitting according to the initial action data and the model palm node as the reference, calculate the distance between each model finger node and the model reference plane according to the initial action data and the model reference plane corresponding to the model finger node, obtain the finger node reference offset distance, take the finger root node as the origin, based on the straight line equation, fit the model finger node, obtain the finger node straight line corresponding to each finger in the virtual hand model, preload the virtual scene according to the data glove interaction instruction corresponding to the user data glove action and the virtual scene information, and correct the position of the virtual interaction target object according to the position relationship between the virtual interaction target object and the virtual hand model;

[0061] An information acquisition module, the information acquisition module is used to acquire data glove sensor information, sensor location information and sensor type information, acquire data glove correction action information based on data glove initialization requirements, collect data generated by the sensor when the user performs data glove correction actions based on sensors integrated in the data glove, acquire correction action data, acquire data glove interaction instruction information, interaction instruction information, interaction action information, user data glove action information and virtual scene information;

[0062] A model adjustment module, wherein the model adjustment module is used to classify virtual hand model nodes according to virtual hand model information, use nodes located in the finger part of the virtual hand model as model finger nodes, use nodes located in the palm part of the virtual hand model as model palm nodes, collect sensor data when the user performs an initial action, obtain initial action data, collect the virtual hand model when the user performs a first initialization action, obtain a first action model, adjust the overall proportion of the virtual hand model until there is no abnormality in the first action model, collect the virtual hand model when the user performs a second initialization action, obtain a second action model, and adjust the virtual hand model according to the initial action data and the second action model;

[0063] The display module interacts with the main control module and is used to output and display the virtual hand model, virtual scene information, data glove interaction instruction information and user data glove action information.

[0064] Optionally, the main control module specifically includes:

[0065] A control unit, the control unit is used to correct the action information according to the data glove, take all fingers straightened and fingers together as the initial action, build a model reference plane based on the initial action data and the model palm node as the reference, based on two-dimensional plane fitting, calculate the distance between each model finger node and the model reference plane according to the initial action data and the model reference plane corresponding to the model finger node, obtain the finger node reference offset distance, take the finger root node as the origin, based on the straight line equation, fit the model finger node, obtain the finger node straight line corresponding to each finger in the virtual hand model, preload the virtual scene based on the data glove interaction instruction corresponding to the user data glove action and the virtual scene information, and correct the position of the virtual interaction target object according to the position relationship between the virtual interaction target object and the virtual hand model;

[0066] An information receiving unit, which interacts with the information acquisition module and the model adjustment module to receive data and transmit it to the judgment unit;

[0067] A judgment unit is used to judge whether the virtual hand model needs to be adjusted according to the first action model, judge whether the model finger nodes are on the model reference plane according to the finger node reference offset distance and the reference plane node offset threshold, judge whether the virtual hand model needs to be adjusted according to the coordinate axis intercept information, and judge whether there is scene interaction based on the user data glove action information and virtual scene information and the data glove interaction instruction information.

[0068] Optionally, the information acquisition module specifically includes:

[0069] a first acquisition unit, the first acquisition unit being used to acquire sensor information, sensor position information and sensor type information of the data glove, acquire correction action information of the data glove based on the initialization requirement of the data glove, and collect data generated by the sensor when the user performs the correction action of the data glove based on the sensor integrated in the data glove to acquire correction action data;

[0070] The second acquisition unit is used to acquire data glove interaction instruction information, interaction instruction information, interaction action information, user data glove action information and virtual scene information.

[0071] Optionally, the model adjustment module specifically includes:

[0072] The model initial unit is used to classify the nodes of the virtual hand model according to the virtual hand model information, take the nodes located in the finger part of the virtual hand model as model finger nodes, take the nodes located in the palm part of the virtual hand model as model palm nodes, collect the sensor data when the user performs the initial action, and obtain the initial action data;

[0073] The model adjustment unit is used to collect the virtual hand model when the user performs the first initialization action, obtain the first action model, adjust the overall proportion of the virtual hand model until there is no abnormality in the first action model, collect the virtual hand model when the user performs the second initialization action, obtain the second action model, and adjust the virtual hand model according to the initial action data and the second action model.

[0074] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0075] The present invention provides a data glove intelligent interaction method and system based on virtual scene correction. By analyzing the initial action data, the completion degree of the user's initial action is analyzed, which improves the accuracy of subsequent model adjustment. By using the first action model and the second action model, the virtual hand model is further adjusted to ensure the authenticity of virtual interaction. By using the user's data glove action information and data glove interaction instruction information, the virtual scene is adjusted and corrected to ensure the visual effect of virtual interaction, ensure the consistency between the model and the real hand movement, and reduce the interaction delay. Description of the Drawings

[0076] Figure 1 It is a flowchart of a data glove intelligent interaction method based on virtual scene correction proposed by the present invention;

[0077] Figure 2 It is a flowchart for obtaining the second action model in the present invention;

[0078] Figure 3 It is a flowchart for obtaining the initial action data in the present invention;

[0079] Figure 4 It is a flowchart for obtaining the coordinate axis intercept information in the present invention;

[0080] Figure 5 It is a structural block diagram of a data glove intelligent interaction system based on virtual scene correction proposed by the present invention. Detailed Embodiments

[0081] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations.

[0082] Reference Figure 1 - Figure 4 As shown, a data glove intelligent interaction method based on virtual scene correction in an embodiment of the present invention includes:

[0083] Acquire data glove sensor information, wherein the data glove sensor information includes sensor position information and sensor type information;

[0084] Based on the data glove initialization requirements, obtain the data glove correction action information;

[0085] Based on the sensors integrated in the data gloves, the data generated by the sensors when the user performs correction actions on the data gloves are collected to obtain correction action data;

[0086] Adjust the virtual hand model according to the corrected motion data and the data glove sensor information;

[0087] Specifically, the virtual hand model is adjusted according to the corrected motion data and the data glove sensor information, including:

[0088] According to the corrected motion information of the data gloves, the initial motion is to straighten all fingers and close them together;

[0089] Acquire virtual hand model information, wherein the virtual hand model information includes virtual hand model node information, and the nodes in the virtual hand model correspond to the data glove sensors;

[0090] According to the virtual hand model information, the virtual hand model nodes are classified, and the nodes located in the finger part of the virtual hand model are used as model finger nodes, and the nodes located in the palm part of the virtual hand model are used as model palm nodes;

[0091] Among them, the node corresponding to the root of each finger in the palm node of the model is used as the finger root node;

[0092] Collecting sensor data when the user performs an initial action to obtain initial action data, wherein the initial action data includes action data corresponding to each node in the virtual hand model, and the action data includes node position information and node curvature information;

[0093] Make a full-fingered fist as the first initialization action;

[0094] Collecting a virtual hand model when the user performs a first initialization action to obtain a first action model;

[0095] According to the first action model, determining whether the virtual hand model needs to be adjusted, if so, adjusting the overall proportion of the virtual hand model until there is no abnormality in the first action model, if not, straightening all fingers and opening the fingers as the second initialization action;

[0096] Among them, the abnormalities of the first motion model include fingers passing through the palm, fingers crossing, and fingers hanging in the air;

[0097] Collect the virtual hand model when the user performs the second initialization action to obtain the second motion model;

[0098] Adjust the virtual hand model according to the initial motion data and the second motion model.

[0099] In this solution, according to the virtual hand model information, the nodes of the virtual hand model are classified. By collecting the virtual hand model when the user performs the first initialization action, the first motion model is obtained. According to the first motion model, it is judged whether the virtual hand model needs to be adjusted. The virtual hand model when the user performs the second initialization action is collected to obtain the second motion model, and the virtual hand model is adjusted according to the initial motion data and the second motion model;

[0100] It can be understood that the sensors of the data glove are in one-to-one correspondence with the nodes of the virtual hand model. Therefore, the data collected by the sensors is used as the node data corresponding to the nodes in the virtual hand model. For the virtual hand model, there are great differences in the motion trajectories of the finger nodes and the palm nodes. Different people have different hand sizes and characteristics. However, when performing different specific actions, the motion trajectories of the palm nodes often do not vary much, but the motion trajectories of the finger nodes are highly correlated with the hand size and characteristics. Therefore, classifying the nodes in the virtual hand model facilitates subsequent node analysis and improves the accuracy and efficiency of the analysis.

[0101] It should be noted that through the fist clenching action, the circumference when the palm curls and the data on the compactness after the fingers bend can be obtained. These data are crucial for constructing the model of the virtual hand in the fist clenching posture, ensuring that when the virtual hand simulates operations related to grasping objects and other fist clenching actions, it can fit the actual size and shape of the fists of different users. Therefore, through the first initialization action, the overall proportion of the virtual hand model is adjusted. However, the virtual hand model obtained at this time only adjusts the size of the general model and cannot accurately reflect the user's hand characteristics. Therefore, the virtual hand model is further adjusted through the second initialization action.

[0102] Specifically, collect the sensor data when the user performs the initial action to obtain the initial action data, specifically including:

[0103] Collect the sensor data when the user performs the initial action to obtain the initial action data;

[0104] Based on the initial action data, with the model palm nodes as the reference, a model reference plane is constructed based on two-dimensional plane fitting;

[0105] Obtain the distance between each model palm node and the model reference plane, and use the maximum value of the distance between the model palm node and the model reference plane as the reference plane node offset threshold;

[0106] According to the initial action data corresponding to the model finger nodes and the model reference plane, calculate the distance between each model finger node and the model reference plane, and obtain the reference offset distance of the finger nodes;

[0107] Based on the reference offset distance of the finger nodes and the reference plane node offset threshold, determine whether the model finger nodes are on the model reference plane;

[0108] Among them, if the reference offset distance of the finger nodes exceeds the reference plane node offset threshold, the model finger nodes are not on the model reference plane, the initial action data is unavailable, and the user needs to re-perform the initial action;

[0109] If the reference offset distance of the finger nodes does not exceed the reference plane node offset threshold, the model finger nodes are on the model reference plane, and the initial action data is available.

[0110] In this solution, the initial action data is obtained by collecting the sensor data when the user performs the initial action, providing a basis for subsequent analysis. Processing these data and constructing a model reference plane based on the model palm nodes can remove some random deviations caused by individual hand differences, different wearing positions, etc. Subsequently, by comparing the distance between the finger nodes and the reference plane with the reference plane node offset threshold, abnormal data can be effectively screened out, ensuring the accuracy and reliability of the initial action data for interaction and improving the stability of the entire interaction system.

[0111] It can be understood that the virtual hand model needs to fit the user's real hand characteristics and action habits. Constructing a model reference plane based on the model palm nodes helps to determine the basic posture and position relationship of the virtual hand. If the reference offset distance of the finger nodes is abnormal, it indicates that the initial action data deviates greatly from the default posture of the virtual hand model, and the user needs to re-perform the action, which enables the virtual hand model to better adapt to the user at the initialization stage and enhances the naturalness of the interaction.

[0112] Specifically, according to the initial action data and the second action model, the virtual hand model is adjusted, specifically including:

[0113] According to the initial action data, obtain the model finger nodes and finger root nodes corresponding to each finger in the virtual hand model;

[0114] Taking the finger root node as the origin, based on the straight line equation, fit the model finger nodes to obtain the finger node straight line corresponding to each finger in the virtual hand model;

[0115] Take the finger joint line as the first coordinate axis. Based on the first coordinate axis, with the finger root joint as the origin, construct a second coordinate axis. The second coordinate axis is perpendicular to the first coordinate axis and intersects at the origin.

[0116] Using the first coordinate axis, the second coordinate axis, and the origin, construct a calibrated plane coordinate system for each finger in the virtual hand model.

[0117] According to the second motion model, obtain second motion data. The second motion data includes the model finger joint data in the second motion model and the finger root joint corresponding to each finger.

[0118] Align the finger root joint in the second motion data with the origin of the calibrated plane coordinate system. Based on the calibrated plane coordinate system, fit the model finger joints in the second motion data to obtain the second motion finger joint expression.

[0119] According to the calibrated plane coordinate system and the second motion finger joint expression, obtain the coordinate axis intercept information. The coordinate axis intercept information represents the intercepts of the second motion finger joint expression on the first coordinate axis and the second coordinate axis respectively.

[0120] According to the coordinate axis intercept information, determine whether the virtual hand model needs to be adjusted.

[0121] Among them, if the coordinate axis intercept is not zero, then adjust the position of the finger root joint in the virtual hand model according to the coordinate axis intercept information until the coordinate axis intercept is zero.

[0122] In this solution, by taking the finger root joint as the origin, based on the linear equation, fit the model finger joints to obtain the finger joint line corresponding to each finger in the virtual hand model. Take the finger joint line as the first coordinate axis. Based on the first coordinate axis, with the finger root joint as the origin, construct a second coordinate axis, and establish a calibrated plane coordinate system for each finger in the virtual hand model. According to the second motion model, obtain second motion data. Align the finger root joint in the second motion data with the origin of the calibrated plane coordinate system. Based on the calibrated plane coordinate system, fit the model finger joints in the second motion data to obtain the second motion finger joint expression. Obtain the coordinate axis intercept information according to the second motion finger joint expression. According to the coordinate axis intercept information, determine whether the virtual hand model needs to be adjusted.

[0123] It can be understood that by taking the root node of the finger as the origin and constructing a calibration plane coordinate system that corresponds to it one by one, an exclusive and accurate coordinate reference frame is established for each finger. This enables the position and posture of each finger in the virtual hand model to be described and analyzed in an independent and clear coordinate system, which helps to capture and analyze the finger motion information more carefully, and lays the foundation for subsequent precise interaction. Aligning the root node of the finger in the second motion data with the origin of the calibration plane coordinate system can unify the finger data under different actions into the standard coordinate system established previously. This can eliminate the interference caused by differences in the starting positions or postures of different actions, make different action data comparable and coherent, and facilitate the subsequent unified analysis and processing of the model finger nodes. In this scheme, the initial action is selected for comparison with the second initialization action. Compared with each other, these two actions are both about straightening the fingers, and the difference lies in the closing and opening of the fingers. Therefore, they can better reflect the node change status in the action change. The intercept value intuitively reflects the key information such as the degree of deviation of the finger action in the two coordinate axis directions, which provides specific data basis for judging whether the virtual hand model needs to be adjusted and how to adjust it, so that the analysis of the virtual hand action is transformed from qualitative description to quantitative analysis.

[0124] Acquire data glove interaction instruction information, wherein the data glove interaction instruction information includes interaction instruction information and interaction action information;

[0125] Obtain user data glove motion information and virtual scene information;

[0126] According to the user data glove action information and the virtual scene information, based on the data glove interaction instruction information, it is determined whether there is scene interaction. If so, the virtual scene is adjusted according to the user data glove action information.

[0127] Specifically, according to the user data glove action information and virtual scene information, based on the data glove interaction instruction information, it is determined whether there is scene interaction, specifically including:

[0128] According to the user data glove action information, user action node information is obtained, where the user action node information represents virtual hand model node information corresponding to the user data glove action;

[0129] According to the user action node information, the node feature position information and node feature curvature information of the virtual hand model corresponding to the user data glove action are obtained;

[0130] According to the data glove interaction instruction information, the instruction node position information and instruction node curvature information of the virtual hand model corresponding to each data glove interaction instruction are obtained;

[0131] Based on the needs of contour coincidence analysis, identification weights are set for node positions and instruction node curvature;

[0132] Compare the node feature position information and the node feature curvature information with the instruction node position information and the instruction node curvature information, and obtain the node position coincidence degree and the node curvature coincidence degree based on the coincidence degree judgment.

[0133] Based on the set recognition weights, perform weighted summation on the node position coincidence degree and the node curvature coincidence degree to obtain an instruction coincidence index corresponding to the user data glove action and each data glove interaction instruction, where the instruction coincidence index represents the completion degree of the data glove interaction instruction.

[0134] Set an instruction coincidence index threshold based on different data glove interaction instructions.

[0135] According to the instruction coincidence index and the instruction coincidence index threshold, determine whether the instruction coincidence index exceeds the instruction coincidence index threshold. If so, based on the data glove interaction instruction corresponding to the user data glove action and based on the virtual scene information, preload the virtual scene.

[0136] In this solution, by obtaining the node feature position and curvature information of the virtual hand model corresponding to the user's action, the manifestation of the user's hand action on the virtual hand model can be accurately captured. At the same time, obtaining the instruction node position and curvature information of the virtual hand model corresponding to each data glove interaction instruction provides a clear standard and basis for subsequent comparative analysis. This step ensures the accurate description of the user's action and interaction instruction, and is the basis for realizing intelligent interaction. Setting recognition weights for the node position and the instruction node curvature based on the contour coincidence analysis requirement reflects the differential consideration of the importance of different features in the interaction process. For example, in some interaction scenarios, the node position may be more critical for accurately executing the instruction, while in other scenarios, the node curvature may play a decisive role. By setting weights, key features can be highlighted, making the subsequent analysis more in line with the actual interaction requirements. Comparing the relevant information of the user's action with the relevant information of the instruction to obtain the node position coincidence degree and the node curvature coincidence degree can quantify the matching degree between the user's action and the interaction instruction. This comparative analysis provides an objective index for evaluating whether the user data glove action conforms to the interaction instruction, which helps to judge whether the user has correctly executed the instruction or to what extent the instruction requirements are approximated. Obtaining the instruction coincidence index by performing weighted summation on the node position coincidence degree and the node curvature coincidence degree, this index can intuitively represent the completion degree of the data glove interaction instruction. It provides a clear numerical index for the interaction system, facilitating the system to quickly judge the fit between the user's action and the instruction, and thus determining subsequent operations and feedback.

[0137] It should be noted that the gestures corresponding to different data glove interaction instructions are also completely different, and the sensitivity of different types of gestures to the virtual hand model nodes is also different. Therefore, in this embodiment, the instruction coincidence index threshold for the data glove interaction instructions of fine operation types (such as grasping, slapping, etc.) is 0.65, and the instruction coincidence index threshold for the data glove interaction instructions of fast types (such as waving, making a fist, pointing, etc.) is 0.85. This threshold is dynamically calculated by the sliding window algorithm and automatically adjusted based on the mean ± 1.5 times the standard deviation of the historical 10-frame data of the data glove interaction instruction actions.

[0138] Specifically, according to the user data glove action information and the virtual scene information, based on the data glove interaction instruction information, to determine whether there is a scene interaction, it further includes:

[0139] According to the instruction coincidence index, to determine whether the user data glove action completes the data glove interaction instruction. If the instruction coincidence index is 1, it means that the user data glove action completes the data glove interaction instruction, and obtain the user interaction instruction information;

[0140] Based on the user interaction instruction information and the virtual scene information, obtain the virtual scene interaction target information;

[0141] According to the virtual scene interaction target information, to determine whether there is a virtual interaction target object. If not, then according to the virtual scene interaction target information, adjust the virtual scene. If so, obtain the virtual interaction target object information, and the virtual interaction target object information includes virtual object position information;

[0142] According to the user interaction instruction information, the position relationship between the virtual interaction target object and the virtual hand model;

[0143] Based on the user data glove action information, according to the position relationship between the virtual interaction target object and the virtual hand model, correct the position of the virtual interaction target object.

[0144] In this solution, obtaining virtual scene interaction target information through user interaction instruction information and virtual scene information helps the system understand the specific goals that the user wants to achieve in the virtual scene. This step combines the user's instructions with the actual situation of the virtual scene, so that the system can accurately locate the core content of the interaction and provide a clear goal orientation for subsequent operations. According to the virtual scene interaction target information, it is determined whether there is a virtual interaction target object, and the virtual interaction target object information is obtained and its position relationship with the virtual hand model is determined, which provides the necessary spatial information basis for realizing the interaction between the virtual hand and the virtual object. Clarifying the position relationship between the two enables the system to better simulate the user's actual operation in the virtual scene, such as grabbing, moving, placing and other actions, so as to achieve a more natural and realistic interaction effect. Based on the user data glove action information, the position of the virtual interaction target object is corrected according to the position relationship between the virtual interaction target object and the virtual hand model, and the position of the virtual object can be adjusted in real time to make it more consistent with the user's hand movement. This process improves the accuracy of the interaction, reduces the error caused by the inconsistency between the position of the virtual object and the actual operation of the user, enhances the realism and credibility of the virtual interaction, and makes the user's operation in the virtual scene smoother and more natural.

[0145] Reference Figure 5 As shown, further, in combination with the above-mentioned data glove intelligent interaction method based on virtual scene correction, a data glove intelligent interaction system based on virtual scene correction is proposed, including:

[0146] A main control module, the main control module is used to determine whether the virtual hand model needs to be adjusted according to the first action model, determine whether the model finger node is in the model reference plane according to the finger node reference offset distance and the reference plane node offset threshold, determine whether the virtual hand model needs to be adjusted according to the coordinate axis intercept information, determine whether there is scene interaction according to the user data glove action information and virtual scene information, based on the data glove interaction instruction information, correct the action information according to the data glove, take all fingers straightened and fingers together as the initial action, build the model reference plane based on two-dimensional plane fitting according to the initial action data and the model palm node as the reference, calculate the distance between each model finger node and the model reference plane according to the initial action data and the model reference plane corresponding to the model finger node, obtain the finger node reference offset distance, take the finger root node as the origin, based on the straight line equation, fit the model finger node, obtain the finger node straight line corresponding to each finger in the virtual hand model, preload the virtual scene according to the data glove interaction instruction corresponding to the user data glove action and the virtual scene information, and correct the position of the virtual interaction target object according to the position relationship between the virtual interaction target object and the virtual hand model;

[0147] An information acquisition module, the information acquisition module is used to acquire data glove sensor information, sensor location information and sensor type information, acquire data glove correction action information based on data glove initialization requirements, collect data generated by the sensor when the user performs data glove correction actions based on sensors integrated in the data glove, acquire correction action data, acquire data glove interaction instruction information, interaction instruction information, interaction action information, user data glove action information and virtual scene information;

[0148] A model adjustment module, wherein the model adjustment module is used to classify virtual hand model nodes according to virtual hand model information, use nodes located in the finger part of the virtual hand model as model finger nodes, use nodes located in the palm part of the virtual hand model as model palm nodes, collect sensor data when the user performs an initial action, obtain initial action data, collect the virtual hand model when the user performs a first initialization action, obtain a first action model, adjust the overall proportion of the virtual hand model until there is no abnormality in the first action model, collect the virtual hand model when the user performs a second initialization action, obtain a second action model, and adjust the virtual hand model according to the initial action data and the second action model;

[0149] The display module interacts with the main control module and is used to output and display the virtual hand model, virtual scene information, data glove interaction instruction information and user data glove action information.

[0150] Main control module, including:

[0151] A control unit, the control unit is used to correct the action information according to the data glove, take all fingers straightened and fingers together as the initial action, build a model reference plane based on the initial action data and the model palm node as the reference, based on two-dimensional plane fitting, calculate the distance between each model finger node and the model reference plane according to the initial action data and the model reference plane corresponding to the model finger node, obtain the finger node reference offset distance, take the finger root node as the origin, based on the straight line equation, fit the model finger node, obtain the finger node straight line corresponding to each finger in the virtual hand model, preload the virtual scene based on the data glove interaction instruction corresponding to the user data glove action and the virtual scene information, and correct the position of the virtual interaction target object according to the position relationship between the virtual interaction target object and the virtual hand model;

[0152] An information receiving unit, which interacts with the information acquisition module and the model adjustment module to receive data and transmit it to the judgment unit;

[0153] A judgment unit, which is used to judge whether the virtual hand model needs to be adjusted according to the first action model, judge whether the model finger nodes are on the model reference plane according to the reference offset distance of the finger nodes and the offset threshold of the reference plane nodes, judge whether the virtual hand model needs to be adjusted according to the coordinate axis intercept information, and judge whether there is a scene interaction based on the user data glove action information and the virtual scene information and the data glove interaction instruction information.

[0154] An information acquisition module, specifically including:

[0155] A first acquisition unit, which is used to acquire data glove sensor information, sensor position information, and sensor type information, acquire data glove correction action information based on the initialization requirements of the data glove, and collect the data generated by the sensors when the user performs data glove correction actions based on the sensors integrated in the data glove to obtain correction action data;

[0156] A second acquisition unit, which is used to acquire data glove interaction instruction information, interaction instruction information, interaction action information, user data glove action information, and virtual scene information.

[0157] A model adjustment module, specifically including:

[0158] A model initial unit, which is used to classify the nodes of the virtual hand model according to the virtual hand model information, regard the nodes located in the finger part of the virtual hand model as model finger nodes, regard the nodes located in the palm part of the virtual hand model as model palm nodes, and collect the sensor data when the user performs the initial action to obtain initial action data;

[0159] A model adjustment unit, which is used to collect the virtual hand model when the user performs the first initialization action to obtain the first action model, adjust the overall proportion of the virtual hand model until there is no abnormality in the first action model, collect the virtual hand model when the user performs the second initialization action to obtain the second action model, and adjust the virtual hand model according to the initial action data and the second action model.

[0160] In summary, the advantages of the present invention are as follows: By collecting sensor data when the user performs an initial action to obtain initial action data, analyzing the completion degree of the user's initial action through the initial action data, the accuracy of subsequent model adjustment is improved. Through the first action model and the second action model, the virtual hand model is further adjusted to ensure the authenticity of virtual interaction. Based on the user data glove action information and virtual scene information, and according to the data glove interaction instruction information, it is judged whether there is scene interaction, and the virtual scene is adjusted and corrected to ensure the visual effect of virtual interaction, ensure the consistency between the model and the real hand movement, and reduce the interaction latency.

[0161] The foregoing has shown and described the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification is only the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A data glove intelligent interaction method based on virtual scene correction, characterized in that: include: Acquire data glove sensor information, wherein the data glove sensor information includes sensor position information and sensor type information; Based on the data glove initialization requirements, obtain the data glove correction action information; Based on the sensors integrated in the data gloves, the data generated by the sensors when the user performs correction actions on the data gloves are collected to obtain correction action data; Adjust the virtual hand model according to the corrected motion data and the data glove sensor information; Acquire data glove interaction instruction information, wherein the data glove interaction instruction information includes interaction instruction information and interaction action information; Obtain user data glove motion information and virtual scene information; According to the user data glove action information and the virtual scene information, based on the data glove interaction instruction information, it is determined whether there is scene interaction. If so, the virtual scene is adjusted according to the user data glove action information.

2. The data glove intelligent interaction method based on virtual scene correction according to claim 1 is characterized in that: The step of adjusting the virtual hand model according to the corrected motion data and the data glove sensor information specifically includes: According to the corrected motion information of the data gloves, the initial motion is to straighten all fingers and close them together; Acquire virtual hand model information, wherein the virtual hand model information includes virtual hand model node information, and the nodes in the virtual hand model correspond to the data glove sensors; According to the virtual hand model information, the virtual hand model nodes are classified, and the nodes located in the finger part of the virtual hand model are used as model finger nodes, and the nodes located in the palm part of the virtual hand model are used as model palm nodes; Among them, the node corresponding to the root of each finger in the palm node of the model is used as the finger root node; Collecting sensor data when the user performs an initial action to obtain initial action data, wherein the initial action data includes action data corresponding to each node in the virtual hand model, and the action data includes node position information and node curvature information; Make a full-fingered fist as the first initialization action; Collecting a virtual hand model when the user performs a first initialization action to obtain a first action model; According to the first action model, determining whether the virtual hand model needs to be adjusted, if so, adjusting the overall proportion of the virtual hand model until there is no abnormality in the first action model, if not, straightening all fingers and opening the fingers as the second initialization action; The first action model abnormality includes fingers passing through the palm, fingers crossing, and fingers hanging in the air; Collecting the virtual hand model of the user when performing the second initialization action to obtain a second action model; The virtual hand model is adjusted according to the initial motion data and the second motion model.

3. The data glove intelligent interaction method based on virtual scene correction according to claim 2 is characterized in that: The collecting of sensor data when the user performs the initial action to obtain the initial action data specifically includes: Collect sensor data when the user performs an initial action to obtain initial action data; According to the initial motion data, the model palm node is taken as the reference, and the model reference surface is constructed based on two-dimensional plane fitting; Obtain the distance between each model palm node and the model reference plane, and use the maximum value of the distance between the model palm node and the model reference plane as the reference plane node offset threshold; According to the initial motion data corresponding to the model finger nodes and the model reference plane, the distance between each model finger node and the model reference plane is calculated to obtain the finger node reference offset distance; According to the finger node reference offset distance and the reference plane node offset threshold, it is determined whether the model finger node is on the model reference plane; If the finger node reference offset distance exceeds the reference plane node offset threshold, the model finger node is not on the model reference plane, the initial action data is unavailable, and the user re-executes the initial action; If the finger node base offset distance does not exceed the base plane node offset threshold, the model finger node is on the model base plane and the initial motion data is available.

4. The data glove intelligent interaction method based on virtual scene correction according to claim 2 is characterized in that: The adjusting of the virtual hand model according to the initial motion data and the second motion model specifically includes: According to the initial action data, a model finger node and a finger root node corresponding to each finger in the virtual hand model are obtained; Taking the root node of the finger as the origin, the finger nodes of the model are fitted based on the straight line equation to obtain the finger node straight line corresponding to each finger in the virtual hand model; The finger node straight line is used as the first coordinate axis, and the first coordinate axis is used as the reference, and the finger root node is used as the origin to construct a second coordinate axis, wherein the second coordinate axis intersects the first coordinate axis perpendicularly at the origin; Constructing a calibration plane coordinate system corresponding to each finger in the virtual hand model using the first coordinate axis, the second coordinate axis and the origin; According to the second motion model, second motion data is acquired, wherein the second motion data includes model finger node data in the second motion model and a finger root node corresponding to each finger; Align the finger root node in the second motion data with the origin of the calibration plane coordinate system, and fit the model finger node in the second motion data based on the calibration plane coordinate system to obtain the second motion finger node expression; According to the calibration plane coordinate system and the second action finger node expression, coordinate axis intercept information is obtained, where the coordinate axis intercept information represents the intercepts of the second action finger node expression on the first coordinate axis and the second coordinate axis respectively; According to the coordinate axis intercept information, determine whether the virtual hand model needs to be adjusted; If the coordinate axis intercept is not zero, the position of the finger root node in the virtual hand model is adjusted according to the coordinate axis intercept information until the coordinate axis intercept is zero.

5. The data glove intelligent interaction method based on virtual scene correction according to claim 1 is characterized in that: The determining whether there is scene interaction based on the user data glove action information and the virtual scene information and the data glove interaction instruction information specifically includes: According to the user data glove action information, user action node information is obtained, where the user action node information represents virtual hand model node information corresponding to the user data glove action; According to the user action node information, the node feature position information and node feature curvature information of the virtual hand model corresponding to the user data glove action are obtained; According to the data glove interaction instruction information, the instruction node position information and instruction node curvature information of the virtual hand model corresponding to each data glove interaction instruction are obtained; Based on the needs of contour coincidence analysis, identification weights are set for node positions and instruction node curvature; Compare the node feature position information and the node feature curvature information with the instruction node position information and the instruction node curvature information, and obtain the node position coincidence and the node curvature coincidence based on the coincidence judgment; Based on the set recognition weights, the node position coincidence and the node bending coincidence are weighted and summed to obtain the instruction coincidence index corresponding to the user data glove action and each data glove interaction instruction, wherein the instruction coincidence index represents the completion degree of the data glove interaction instruction; Based on the different interactive commands of the data gloves, the command overlap index threshold is set; According to the instruction overlap index and the instruction overlap index threshold, it is determined whether the instruction overlap index exceeds the instruction overlap index threshold. If so, the virtual scene is preloaded based on the virtual scene information according to the data glove interaction instruction corresponding to the user data glove action.

6. The data glove intelligent interaction method based on virtual scene correction according to claim 5 is characterized in that: The step of judging whether there is scene interaction based on the user data glove action information and the virtual scene information and the data glove interaction instruction information also includes: According to the command overlap index, determine whether the user data glove action completes the data glove interaction command. If the command overlap index is 1, it means that the user data glove action completes the data glove interaction command, and obtain the user interaction command information; Based on the user interaction instruction information and the virtual scene information, obtaining virtual scene interaction target information; According to the virtual scene interaction target information, determine whether there is a virtual interaction target object, if not, adjust the virtual scene according to the virtual scene interaction target information, and if yes, obtain the virtual interaction target object information, wherein the virtual interaction target object information includes virtual object position information; According to the user interaction instruction information, the position relationship between the virtual interaction target object and the virtual hand model is established; Based on the user data glove motion information, the position of the virtual interaction target object is corrected according to the position relationship between the virtual interaction target object and the virtual hand model.

7. A data glove intelligent interaction system based on virtual scene correction, used to implement the interaction method according to any one of claims 1 to 6, characterized in that: include: A main control module, the main control module is used to determine whether the virtual hand model needs to be adjusted according to the first action model, determine whether the model finger node is in the model reference plane according to the finger node reference offset distance and the reference plane node offset threshold, determine whether the virtual hand model needs to be adjusted according to the coordinate axis intercept information, determine whether there is scene interaction according to the user data glove action information and virtual scene information, based on the data glove interaction instruction information, correct the action information according to the data glove, take all fingers straightened and fingers together as the initial action, build the model reference plane based on two-dimensional plane fitting according to the initial action data and the model palm node as the reference, calculate the distance between each model finger node and the model reference plane according to the initial action data and the model reference plane corresponding to the model finger node, obtain the finger node reference offset distance, take the finger root node as the origin, based on the straight line equation, fit the model finger node, obtain the finger node straight line corresponding to each finger in the virtual hand model, preload the virtual scene according to the data glove interaction instruction corresponding to the user data glove action and the virtual scene information, and correct the position of the virtual interaction target object according to the position relationship between the virtual interaction target object and the virtual hand model; An information acquisition module, the information acquisition module is used to acquire data glove sensor information, sensor location information and sensor type information, acquire data glove correction action information based on data glove initialization requirements, collect data generated by the sensor when the user performs data glove correction actions based on sensors integrated in the data glove, acquire correction action data, acquire data glove interaction instruction information, interaction instruction information, interaction action information, user data glove action information and virtual scene information; A model adjustment module, wherein the model adjustment module is used to classify virtual hand model nodes according to virtual hand model information, use nodes located in the finger part of the virtual hand model as model finger nodes, use nodes located in the palm part of the virtual hand model as model palm nodes, collect sensor data when the user performs an initial action, obtain initial action data, collect the virtual hand model when the user performs a first initialization action, obtain a first action model, adjust the overall proportion of the virtual hand model until there is no abnormality in the first action model, collect the virtual hand model when the user performs a second initialization action, obtain a second action model, and adjust the virtual hand model according to the initial action data and the second action model; The display module interacts with the main control module and is used to output and display the virtual hand model, virtual scene information, data glove interaction instruction information and user data glove action information.

8. The data glove intelligent interaction system based on virtual scene correction according to claim 7 is characterized in that: The main control module specifically includes: A control unit, the control unit is used to correct the action information according to the data glove, take all fingers straightened and fingers together as the initial action, build a model reference plane based on the initial action data and the model palm node as the reference, based on two-dimensional plane fitting, calculate the distance between each model finger node and the model reference plane according to the initial action data and the model reference plane corresponding to the model finger node, obtain the finger node reference offset distance, take the finger root node as the origin, based on the straight line equation, fit the model finger node, obtain the finger node straight line corresponding to each finger in the virtual hand model, preload the virtual scene based on the data glove interaction instruction corresponding to the user data glove action and the virtual scene information, and correct the position of the virtual interaction target object according to the position relationship between the virtual interaction target object and the virtual hand model; An information receiving unit, which interacts with the information acquisition module and the model adjustment module to receive data and transmit it to the judgment unit; A judgment unit is used to judge whether the virtual hand model needs to be adjusted according to the first action model, judge whether the model finger nodes are on the model reference plane according to the finger node reference offset distance and the reference plane node offset threshold, judge whether the virtual hand model needs to be adjusted according to the coordinate axis intercept information, and judge whether there is scene interaction based on the user data glove action information and virtual scene information and the data glove interaction instruction information.

9. The data glove intelligent interaction system based on virtual scene correction according to claim 7, characterized in that: The information acquisition module specifically includes: a first acquisition unit, the first acquisition unit being used to acquire sensor information, sensor position information and sensor type information of the data glove, acquire correction action information of the data glove based on the initialization requirement of the data glove, and collect data generated by the sensor when the user performs the correction action of the data glove based on the sensor integrated in the data glove to acquire correction action data; The second acquisition unit is used to acquire data glove interaction instruction information, interaction instruction information, interaction action information, user data glove action information and virtual scene information.

10. The data glove intelligent interaction system based on virtual scene correction according to claim 7, characterized in that: The model adjustment module specifically includes: A model initialization unit, wherein the model initialization unit is used to classify virtual hand model nodes according to virtual hand model information, use nodes located in the finger part of the virtual hand model as model finger nodes, use nodes located in the palm part of the virtual hand model as model palm nodes, collect sensor data when the user performs an initial action, and obtain initial action data; A model adjustment unit, wherein the model adjustment unit is used to collect the virtual hand model when the user performs a first initialization action, obtain the first action model, adjust the overall proportion of the virtual hand model until there is no abnormality in the first action model, collect the virtual hand model when the user performs a second initialization action, obtain the second action model, and adjust the virtual hand model according to the initial action data and the second action model.

Images