A contact processing method and system for virtual hand force tactile interaction

Abstract

The application relates to a contact processing method and system for virtual hand force interaction, which is based on a virtual hand structure and considers the different thicknesses of various joints of human fingers. The fingers are represented by circular truncated cones. On this basis, a ball-truncated cone contact processing system for virtual hand and virtual object force interaction is constructed by adopting a ball tree model. The system can realize stable, non-penetrating and relatively real virtual hand force interaction simulation and has a relatively high (up to 1 kHz) contact processing efficiency. A circular truncated cone hinge model is used to simulate a real hand, ensuring the visual reality of the virtual hand. The contact constraint between the ball and the circular truncated cone ensures that the side of the virtual finger does not embed into the virtual object, thereby effectively solving the problem that the graphic hand and the haptic hand are difficult to match in the interaction process due to the large shape gap between the simplified virtual hand model and the real hand in the prior art, and the visual and tactile feedback is inconsistent.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of haptic interaction, in particular to a contact processing method and system for virtual hand haptic interaction. BACKGROUND

[0002] In order to simulate the interactive experience of using tools to operate objects, researchers at home and abroad have developed a series of contact processing methods (3-DOF (Degree of Freedom) / 6-DOF) and systems for virtual tool haptic interaction, which have promoted the wide application of desktop haptic interaction devices, such as the use of Phantom Premium in dental surgery simulators. In daily life, people usually directly touch and operate objects with their hands. In order to simulate this intuitive interaction mode to enhance the immersion of interaction with the virtual world (such as virtual assembly, virtual surgery, high-risk work environment simulation, professional skill training, entertainment, games, etc.), in addition to the development of force feedback devices for virtual hand interaction, such as Dexmo force feedback gloves of Daisi Technology, high-fidelity contact processing methods and systems for virtual hand haptic interaction are also needed to support.

[0003] Compared with the contact processing method for virtual tool haptic interaction, the implementation of the contact processing method for virtual hand haptic interaction is more challenging, mainly because virtual hand haptic interaction involves more degrees of freedom of operation, more diversified operation modes, and more contact points / areas, making the calculation involved in the contact processing method for virtual hand haptic interaction more complex and time-consuming. The contact processing method for virtual hand haptic interaction mainly needs to solve the following two key problems: (1) quickly and accurately determine the contact state of the virtual hand and the virtual object; (2) quickly and accurately calculate the virtual hand avatar and the contact force without penetrating the virtual object.

[0004] The solution of the above two problems needs to involve the virtual hand model, which is used to describe the three-dimensional shape, the degree of freedom of motion, and the geometric constraints of the palm and multiple fingers of the virtual hand, to meet the needs of motion mapping, collision detection, collision response, contact force calculation, and graphics rendering. In virtual hand geometry modeling, simplified models (such as capsule models representing fingers or sampling part of points / areas on the surface of the virtual hand model) are usually used to cope with the demand for high update frequency of haptic interaction. These simplified models can also be combined with other geometric models to represent the virtual hand.

[0005] Although the contact handling performance of virtual hand force tactile interaction can be improved to some extent by the existing method, due to the large difference between the simplified virtual hand model and the shape of the real hand, the graphic hand and the tactile hand are difficult to match in the process of virtual hand force tactile interaction, resulting in inconsistent visual and tactile feedback. For example, when the graphic hand is embedded in the virtual object, the effective contact handling cannot be performed, and the user cannot feel the force feedback. In addition, due to the complexity of the contact constraint model, the existing contact handling method can only achieve an update frequency of tens of hertz, which cannot meet the demand of high update frequency (such as 1 kHz) for force tactile interaction.

[0006] Therefore, there is an urgent need in the art for a non-penetration simulation technical solution that ensures the visualization of the virtual hand, realizes fast and stable force feedback, and ensures the interaction between the virtual hand and the virtual object. SUMMARY

[0007] The purpose of the present application is to provide a contact handling method and system for virtual hand force tactile interaction, which simulates a real hand using a circular cone hinge model, ensures the visualization of the virtual hand, and ensures that the side of the virtual finger does not embed into the virtual object by using the contact constraint between the sphere and the circular cone, thereby effectively solving the problem of inconsistent visual and tactile feedback caused by the difficulty of matching the graphic hand and the tactile hand in the interaction process due to the large difference between the simplified virtual hand model and the shape of the real hand. In addition, the circular cone is used to simulate the knuckles of the real hand, reducing the number of contact constraints, and parallel processing can be used, thereby effectively solving the problem of contact handling efficiency in the prior art.

[0008] To achieve the above purpose, the present application provides the following solutions:

[0009] A contact handling method for virtual hand force tactile interaction, the method comprising:

[0010] Simulating a real hand using a circular cone hinge model to obtain a virtual hand; the circular cone hinge model comprises a plurality of circular cones, each of which simulates each knuckle of the real hand; the metacarpophalangeal joint and the joint between two knuckles are simulated by a sphere; the virtual hand further comprises a plurality of hemispheres, each of which is located on the circular cone simulating the end knuckle, and the hemispheres are used to simulate the fingertips of the real hand;

[0011] Simulating an object using a sphere tree model to obtain a virtual object; the virtual object comprises a plurality of numbered spheres;

[0012] Obtaining the pose of the real hand of the user;

[0013] Driving the virtual hand using the pose of the real hand, referred to as the tactile hand;

[0014] displaying the haptic hand in a virtual scene to obtain a graphic hand, and displaying the virtual object in the virtual scene;

[0015] constructing a contact judging condition between the sphere and the circular truncated cone: and wherein |OC| j is a distance from a sphere center of a sphere numbered j on the virtual object to a central axis l of a circular truncated cone in the graphic hand, R j is a radius of the sphere numbered j on the virtual object, and M is a number of spheres on the virtual object to be detected, C j represents a foot point of the sphere center of the sphere numbered j on the virtual object on the central axis l, E and D respectively represent circle centers of upper and lower bases of the circular truncated cone in the graphic hand, a represents an included angle between a side of the circular truncated cone in the graphic hand and the central axis thereof, r2 and r1 respectively represent radii of the upper and lower bases of the circular truncated cone in the graphic hand;

[0016] obtaining collision sphere information in contact with a side of a finger of the graphic hand according to graphic hand pose information, virtual object pose information, and the contact judging condition between the sphere and the circular truncated cone; the collision sphere information comprises a number of the collision sphere, a sphere center coordinate of the collision sphere, and a sphere radius of the collision sphere; the collision sphere is a sphere of the virtual object in contact with the graphic hand; the side of the finger of the graphic hand is a side of a knuckle of a finger of the circular truncated cone; and the graphic hand pose information is used for graphic display;

[0017] wherein and respectively represent a metacarpophalangeal joint angle, a proximal interphalangeal joint angle, and a distal interphalangeal joint angle of each finger of the graphic hand, and a non-embedded contact constraint between a side of a finger of the graphic hand and the virtual object is constructed according to the graphic hand pose information and the collision sphere information of the virtual object in contact with the graphic hand wherein N is a number of spheres of the virtual object in contact with the finger of the graphic hand; and for each finger of the graphic hand, three joint angles and are taken as a whole to be optimized under the non-embedded contact constraint set, and joint angle information of the finger of the graphic hand not embedded in the virtual object is obtained by solving;

[0018] the graphic hand finger joint angle information and a formula F i =G i ·n′·|P ig -P ih is used to calculate normal pressure F i of the ith collision sphere in contact with a corresponding virtual finger; wherein G i is a 3x3 stiffness matrix, n′ represents a unit vector of , and Oi represents the center of the i-th collision sphere, P i represents the contact point between the i-th collision sphere and the side of the virtual finger, |P ig -P ih | is the length of the line segment between the point P ig on the graphical hand and the corresponding position point P ih on the haptic hand;

[0019] In some embodiments, in the contact determination between the graphical hand and the virtual object:

[0020] the pose of the graphical hand is the pose of the graphical hand at the previous time;

[0021] the radius of each cylindrical section of the phalanx of the graphical hand is increased by Δr;

[0022] the radius of each hemispherical section of the phalanx of the graphical hand is increased by Δr;

[0023] the graphical hand obtained by the above changes is used for the contact determination between the graphical hand and the virtual object, and the collision spheres obtained according to the sphere-cylinder contact determination condition and the sphere-sphere contact determination condition are used for the construction of the non-embedded contact constraint set.

[0024] In some embodiments, for the distal phalanx of each finger, in the construction of the non-embedded contact constraint set, further comprising:

[0025] if the collision sphere is in contact with the sphere of the simulated fingertip, a non-embedded constraint between the sphere and the hemisphere is constructed: the contact constraint is added to the non-embedded contact constraint set for the optimization of the joint angle of the phalanx of the graphical hand.

[0026] In some embodiments, after the normal pressure F i = G i · n' · |P ig -P ih | of the i-th collision sphere in contact with the corresponding virtual phalanx is calculated according to the formula F i , further comprising:

[0027] the friction force in the sliding process of the virtual hand is calculated using the formula f i = u · n i · |F i |; wherein f i represents the friction force at the i-th collision sphere, u represents the dynamic friction coefficient, and n i is the normal vector of the relative motion between the i-th collision sphere and the virtual finger.

[0028] A contact processing system for virtual hand haptic interaction, the system comprising:

[0029] a virtual hand construction module, configured to:

[0030] simulate a real hand by using a cone hinge model to obtain a virtual hand; the virtual hand comprises a plurality of cones, each of the cones simulating each knuckle of the real hand; a metacarpophalangeal joint and two knuckles are simulated by using a sphere; the virtual hand further comprises a plurality of hemispheres, each of the hemispheres being located on a cone simulating a distal knuckle, and each of the hemispheres simulating each fingertip of the real hand;

[0031] simulate an object by using a sphere tree model to obtain a virtual object; the virtual object comprises a plurality of numbered spheres;

[0032] obtain a pose of a real hand of a user;

[0033] drive the virtual hand by using the pose of the real hand, referred to as a haptic hand;

[0034] display the haptic hand in a virtual scene to obtain a graphic hand, and display the virtual object in the virtual scene;

[0035] a sphere-cone collision detection module, configured to:

[0036] construct a contact judgment condition between the sphere and the cone: and wherein, |OC| j is a distance from a sphere center of a sphere numbered j on the virtual object to a central axis l of a cone in the graphic hand, R j is a radius of the sphere numbered j on the virtual object, and M is a number of spheres to be detected on the virtual object, C j is a foot of a perpendicular from the sphere center of the sphere numbered j on the virtual object on the central axis l, E and D respectively represent centers of upper and lower bases of the cone in the graphic hand, a represents an included angle between a side of the cone in the graphic hand and the central axis thereof, and r2 and r1 respectively represent radii of the upper and lower bases of the cone in the graphic hand;

[0037] obtain collision sphere information in contact with a side of a finger of the graphic hand according to graphic hand pose information, virtual object pose information, and the contact judgment condition between the sphere and the cone; the collision sphere information comprises a number of the collision sphere, a sphere center coordinate of the collision sphere, and a sphere radius of the collision sphere; the collision sphere is a sphere of the virtual object in contact with the graphic hand; the side of the finger of the graphic hand is a side of a knuckle of a cone of the finger; and the graphic hand pose information is used for graphic display;

[0038] a sphere-cone collision response module, configured to:

[0039] use and Let these represent the metacarpophalangeal joint angle, proximal interphalangeal joint angle, and distal interphalangeal joint angle of each finger of the graphical hand. Based on the graphical hand pose information and the collision ball information between the graphical hand and the virtual object, non-embedded contact constraints between the sides of the graphical hand's fingers and the virtual object are constructed. Where N is the number of spheres on the virtual object that are in contact with the virtual fingers of the graphical hand; according to the non-embedded contact constraint between the frustum and the spheres, for each finger of the graphical hand, its three joint angles are... and The optimization is performed as a whole under the non-embedded contact constraint set to obtain the finger joint angle information of the graphical hand that is not embedded in the virtual object;

[0040] The contact force calculation module is used for:

[0041] Based on the graphic information of the finger joint angles and formula F i =G i ·n′·|P ig -P ih Calculate the normal force F that the i-th colliding ball exerts upon contact with the corresponding virtual finger. i Among them, G i It is a 3×3 stiffness matrix, where n′ represents unit vector, O i Let P represent the center of the i-th colliding ball. i |P represents the contact point between the i-th colliding ball and the side of the virtual finger. ig -P ih |P the hand icon for the graphic. ig Corresponding location P on the tactile hand ih The length of the line segment between them;

[0042] In some embodiments, when determining contact between the graphical hand and the virtual object:

[0043] The sphere-frustum collision detection module also includes the following features:

[0044] The graphic hand pose is the graphic hand pose of the previous moment;

[0045] The radius of the frustum cross section of each finger joint of the graphic hand increases by Δr;

[0046] The radius of the hemisphere of each phalanx of the graphic hand increases by Δr;

[0047] The graphic hand obtained through the above modifications is used to determine the contact between the graphic hand and the virtual object. The collision ball obtained according to the ball-frustum contact determination condition and the ball-ball contact determination condition is used to construct the non-embedded contact constraint set.

[0048] In some embodiments, it further includes: an endpoint contact processing module, configured to:

[0049] For the distal phalanx of each finger, while constructing the non-embedded contact constraints ;

[0050] If the collision ball is in contact with the sphere of the simulated finger tip, construct the non-embedded constraints between the sphere and the hemisphere: Add the contact constraint to the set of non-embedded contact constraints for the optimization of the graphical finger joint angles;

[0051] If the collision ball is not in contact with the sphere of the simulated finger tip, only construct the contact constraint between the sphere and the truncated cone.

[0052] In some embodiments, after the formula F i =G i ·n′·|P ig -P ih |is used to calculate the normal pressure F i of the ith collision ball in contact with the corresponding virtual phalanx, further comprising:

[0053] Using the formula f i =u·n i ·|F i |to calculate the friction force during the sliding of the virtual hand; wherein f i represents the friction force at the ith collision ball, u represents the dynamic friction coefficient, and n i is the normal vector of the relative motion between the ith collision ball and the virtual finger.

[0054] In some embodiments, when palm interaction is involved, the palm pose information of the graphical hand is first optimized, the finger pose information of the graphical hand is obtained through matrix transformation, and then the graphical hand finger joint angle information of the non-embedded virtual object is optimized and solved according to the finger pose information of the graphical hand and the pose information of the virtual object, using the ball-truncated cone collision detection module, the ball-truncated cone collision response module, and the end-point contact processing module.

[0055] In some embodiments, when processing multi-finger interaction, a parallel processing method is used.

[0056] According to the specific embodiments provided by the present application, the following technical effects are disclosed:

[0057] The application provides a contact processing method and system for virtual hand force interaction, aiming at accurate and efficient contact processing, starting from a virtual hand structure, considering that the thicknesses of various joints of human fingers are different, and representing the fingers as circular truncated cones. On this basis, a ball-truncated cone contact processing system for virtual hand and virtual object force interaction is constructed by using a ball tree model, which can realize stable, non-penetrating and relatively realistic virtual hand force interaction simulation while achieving a high (higher than 1 kHz) contact processing efficiency. The circular truncated cone hinge model is used to simulate the real hand, ensuring the visualization of the virtual hand, and the small amount of contact constraints between the ball and the circular truncated cone ensure that the side of the virtual finger does not embed the virtual object while reducing the computational complexity, and the computational efficiency can be improved through parallel processing, thereby effectively solving the problem that the simplified virtual hand model in the prior art is greatly different from the shape of the real hand, which leads to the difficulty in matching the graphic hand and the haptic hand in the interaction process, resulting in inconsistent visual and tactile feedback, and the low contact processing efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed in the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0059] Figure 1 The flow chart of the contact processing method for virtual hand force interaction provided by the first embodiment of the present application.

[0060] Figure 2 The schematic diagram of the contact processing system for virtual hand force interaction provided by the second embodiment of the present application.

[0061] Figure 3 The hybrid virtual hand model based on the circular truncated cone hinge and the different level ball tree provided by the second embodiment of the present application.

[0062] Figure 4 The ball-truncated cone contact state schematic diagram for ball-truncated cone discrete collision detection provided by the second embodiment of the present application.

[0063] Figure 5 The endpoint optimization anomaly problem of the contact constraint between the ball tree model and the circular truncated cone model provided by the second embodiment of the present application.

[0064] Figure 6 The object ball-fingertip hemisphere (including the circular truncated cone of the knuckle) contact state schematic diagram for solving the endpoint optimization problem provided by the second embodiment of the present application.

[0065] Figure 7A contact processing process diagram of finger-object interaction based on contact constraint prediction is provided for the second embodiment of the present application.

[0066] Figure 8 A contact force model (partial contact) of virtual finger and virtual object is provided for the second embodiment of the present application.

[0067] Figure 9 A flowchart of a virtual hand force interaction system is provided for the second embodiment of the present application. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0069] The purpose of the present application is to provide a contact processing method and system for virtual hand force interaction, which simulates a real hand by using a circular cone hinge model to ensure the visual fidelity of the virtual hand, and ensures that the side of the virtual finger does not embed into the virtual object by using the contact constraint between the sphere and the circular cone, thereby effectively solving the problem of inconsistent visual and tactile feedback caused by the difficulty in matching the graphic hand and the tactile hand in the interaction process due to the large gap between the simplified virtual hand model and the shape of the real hand in the prior art.

[0070] The purpose of the present application is to overcome the defects of the prior art and provide a contact processing method and system for virtual hand force interaction, which aims to achieve efficient and non-penetrating contact processing, and starts from the structure of the virtual hand, considers that the thickness of each joint of the human finger is different, and represents the finger by a circular cone. On this basis, a sphere-tree model is used to construct a virtual object, and a sphere-circular cone contact processing system for virtual hand and virtual object force interaction is proposed. The system can achieve stable, non-penetrating and relatively realistic virtual hand force interaction simulation while achieving a relatively high (up to 1 kHz) contact processing efficiency.

[0071] In virtual hand tactile simulation, the avatar of the user's hand in the virtual environment is called a haptic hand, and its pose is the direct mapping of the pose of the user's real hand in the virtual world (OpenGL or existing engines such as Unity, Unreal, etc. C++, c#, etc.). In graphic display, the visual avatar of the haptic hand is defined as a graphic hand.

[0072] The technical scheme of the present application is: a contact processing system for virtual hand force sensation interaction, which is composed of a virtual hand construction module, a ball-cone collision detection module, a ball-cone collision response module, an end-point contact processing module and a contact force calculation module.

[0073] The virtual hand construction module establishes a cone hinge virtual hand model according to specific parameters of fingers (including the length of each finger joint and the cross-sectional diameter of both ends of each finger joint);

[0074] The ball-cone collision detection module is used to determine whether a virtual finger has collided with a virtual object and return collision information;

[0075] The ball-cone collision response module is used to quickly and accurately calculate the reasonable pose of the graphic hand to ensure that the graphic hand does not embed the virtual object;

[0076] The end-point contact processing module is used to process the problem of finger tip suspension or embedding that may occur when the virtual finger tip contacts the virtual object;

[0077] The contact force calculation module calculates the contact force between the virtual hand and the virtual object according to the collision information obtained by the collision detection module and the optimization result of the collision response module. The following will introduce the above modules in detail from the construction of the virtual hand model.

[0078] The following three problems are worth mentioning:

[0079] (1) The virtual hand force sensation interaction contact processing system proposed by the present application is applicable to different virtual hand interaction applications, such as virtual hand single-hand object grabbing, two-hand object operation, multi-person collaborative object operation, etc. The method proposed by the present application supports multi-object, multi-hand operation and multi-person collaboration.

[0080] (2) The present application focuses on the modeling and contact processing of virtual hand fingers. The virtual hand palm can have multiple modeling methods, such as a ball tree model and a triangular mesh model. The virtual hand palm of the present embodiment is constructed using a ball tree model.

[0081] (3) The contact force calculation module proposed by the present application can calculate the normal pressure at all contact points during the virtual object and virtual hand interaction. On this basis, different feedback force calculation models can be designed according to the force feedback form (such as finger tip force feedback) that can be provided by different force feedback devices, so that users can experience different feedback force modes through different force feedback devices.

[0082] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0083] Embodiment one:

[0084] AsFigure 1 The embodiment shown provides a contact processing method for virtual hand force interaction, which comprises the following steps:

[0085] S1, a real hand is simulated by a circular truncated cone hinge model to obtain a virtual hand; the circular truncated cone hinge model comprises a plurality of circular truncated cones, each of which simulates a knuckle of the real hand; the virtual hand further comprises a plurality of hemispheres, each of which is located on a circular truncated cone simulating a terminal knuckle, and the hemispheres are used to simulate the fingertips of the real hand;

[0086] S2, an object is simulated by a ball tree model to obtain a virtual object; the virtual object comprises a plurality of numbered balls;

[0087] S3, the pose of the real hand of a user is obtained;

[0088] S4, the virtual hand, referred to as a haptic hand, is driven by the pose of the real hand;

[0089] S5, the haptic hand is displayed in a virtual scene to obtain a graphic hand, and the virtual object is displayed in the virtual scene.

[0090] In free space (i.e. the virtual hand is not in contact with the virtual object), the haptic hand is displayed in a virtual scene to obtain a graphic hand, and the virtual object is displayed in the virtual scene; in the constrained space, if the pose of the haptic hand is directly used for the graphic hand, the graphic hand will be embedded in the virtual object, and the graphic hand kept on the surface of the virtual object is obtained through the following contact processing steps.

[0091] S6, a contact judgment condition between the balls and the circular truncated cones is constructed: and wherein, |OC| j is the distance from the center of the ball numbered j on the virtual object to the central axis l of the circular truncated cone in the graphic hand, R j is the radius of the ball numbered j on the virtual object, and M is the number of balls to be detected on the virtual object, C j represents the foot of the center of the ball numbered j on the virtual object on the central axis l, E and D respectively represent the centers of the upper and lower bases of the circular truncated cone in the graphic hand, α represents the included angle between the side of the circular truncated cone in the graphic hand and the central axis thereof, and r2 and r1 respectively represent the radii of the upper and lower bases of the circular truncated cone in the graphic hand;

[0092] S7. Based on the graphic hand pose information, the virtual object pose information, and the contact judgment conditions between the sphere and the frustum, obtain the collision ball information that contacts the side of the graphic hand's fingers; the collision ball information includes the collision ball's number, the collision ball's center coordinates, and the collision ball's radius; the collision ball is the sphere in the virtual object that contacts the graphic hand; the side of the graphic hand's fingers is the side of the frustum's knuckle; the graphic hand pose information is used for graphic display;

[0093] S8, Utilization and Let these represent the metacarpophalangeal joint angle, proximal interphalangeal joint angle, and distal interphalangeal joint angle of each finger of the graphical hand. Based on the graphical hand pose information and the collision ball information between the graphical hand and the virtual object, a non-embedded contact constraint is constructed between the sides of the graphical hand's fingers (i.e., frustums) and the virtual object (i.e., sphere). Where N is the number of spheres on the virtual object that are in contact with the fingers of the graphical hand; according to the non-embedded contact constraint between the frustum and the spheres, for each finger of the graphical hand, its three joint angles are... and The optimization is performed as a whole under the non-embedded contact constraint set to obtain the finger joint angle information of the graphical hand that is not embedded in the virtual object;

[0094] S9. Based on the graphic finger joint angle information and formula F i =G i ·n′·|P ig -P ih Calculate the normal force F that the i-th colliding ball exerts upon contact with the corresponding virtual finger. i Among them, G i It is a 3×3 stiffness matrix, where n′ represents unit vector, O i Let P represent the center of the i-th colliding ball. i |P represents the point of contact between the i-th colliding ball and the side of the virtual finger. ig -P ih |P for the graphic's hand. ig Corresponding location P on the tactile hand ih The length of the line segment between them.

[0095] The graphic hand and its pose used in the ball-frustum contact judgment condition and the ball-ball contact judgment condition also include the following characteristics:

[0096] The graphic hand pose is the graphic hand pose of the previous moment;

[0097] The radius of the frustum cross section of each finger joint of the graphic hand increases by Δr;

[0098] The radius of each knuckle hemisphere of the graphic hand is increased by Δr;

[0099] The graphic hand obtained by the above changes is used for contact judgment of the graphic hand and the virtual object, and the collision sphere obtained according to the ball-cone contact judgment condition and the ball-ball contact judgment condition is used for construction of a non-embedded contact constraint set.

[0100] For the distal knuckle of each finger, while constructing the non-embedded contact constraint set , the following is also included:

[0101] If the collision sphere is in contact with the ball of the simulated fingertip, a non-embedded constraint between the ball and the hemisphere is constructed: The contact constraint is added to the non-embedded contact constraint set for optimization of the knuckle joint angle of the graphic hand.

[0102] After the normal pressure F i i of the i-th collision sphere in contact with the corresponding virtual knuckle is calculated according to the formula F i =G ig ·n′·|P ih -P i , the following is also included:

[0103] Using a friction calculation formula, for example, when the virtual hand slides on the surface of the virtual object, the friction force during sliding of the virtual hand is calculated using the formula f i i = u · n i i · |F i i |; wherein f i i represents the friction force at the i-th collision sphere, u represents the dynamic friction coefficient, and n i i is the normal vector of the relative motion between the i-th collision sphere and the virtual finger.

[0104] The above describes a preferred embodiment of the present application, which can be implemented in many different forms, such as construction of a virtual hand model, a virtual hand pose optimization method, and a method for calculating interactive force, and is not limited to the embodiments described herein.

[0105] Embodiment Two:

[0106] As shown in the following, the present embodiment provides a contact processing system for virtual hand force interaction, which comprises: Figure 2 A virtual hand construction module, configured to:

[0107]

[0108] ​The virtual hand is obtained by using a circular truncated cone hinge model to simulate a real hand; the virtual hand comprises a plurality of circular truncated cones, each of the circular truncated cones simulating each knuckle of the real hand; a metacarpophalangeal joint and joints between two knuckles are simulated by a sphere; the virtual hand further comprises a plurality of hemispheres, each of the hemispheres being located on a circular truncated cone simulating a distal knuckle, and each of the hemispheres simulating each fingertip of the real hand.

[0109] The virtual object is obtained by using a ball tree model to simulate an object; the virtual object comprises a plurality of numbered spheres.

[0110] An attitude of a real hand of a user is acquired.

[0111] The virtual hand is driven by the attitude of the real hand, and is referred to as a haptic hand.

[0112] In a free space, the haptic hand is displayed in a virtual scene to obtain a graphic hand, and the virtual object is displayed in the virtual scene; in a constrained space, if a pose of the haptic hand is directly used for the graphic hand, the graphic hand will be embedded in the virtual object, and the graphic hand kept on a surface of the virtual object is obtained through contact processing of the following functional modules.

[0113] In the embodiment, a virtual hand construction module adopts a circular truncated cone hinge model to construct a virtual finger, that is, a circular truncated cone (with different diameters of upper and lower sections) is used to represent each joint of the virtual finger, a sphere is used to connect two adjacent knuckles to represent a joint, and a hemisphere is used to represent a fingertip. In order to better simulate a thumb, other models (for example, a ball tree model) can be used to construct a metacarpal joint of the thumb.

[0114] Notably, the present application focuses on describing a finger construction model of the virtual hand, and a triangular mesh model, a ball tree model, etc. can be used to construct a palm of the virtual hand, which are all within the protection scope of the present application.

[0115] Specifically, the ball tree model is used to construct a virtual tool and a virtual object, which is effective in improving the calculation efficiency of a tool-based six-degree-of-freedom force sensation synthesis method and non-penetration simulation. However, the octree ball tree is used to construct a virtual hand, and in the process of realizing interaction between the virtual hand and the virtual object, as the number of interactive / contact fingers and contacts increases, the time consumption of contact processing also increases. The knuckles of a real human hand can be approximated as circular truncated cones, and the circular truncated cone hinge model can be used to represent the fingers of the virtual hand, with one circular truncated cone representing one knuckle of the finger. Intuitively, the number of circular truncated cones used for virtual hand modeling is less than the number of spheres required by the ball tree model, and compared with the multi-layer ball tree model, the circular truncated cone hinge model used to represent the fingers is more conducive to efficiently performing contact processing of virtual hand force sensation interaction.

[0116] For each virtual knuckle built with a truncated cone, three key parameters are included, i.e. the length of the knuckle and the diameters of the upper and lower end surfaces of the knuckle. The three parameters can be obtained by measuring the length of the knuckles of a real hand of a user and the diameters of the cross sections at each joint. In addition, in order to achieve visual display and interactive realism, a sphere is used to represent the joint between adjacent knuckles, and a hemisphere is connected to the truncated cone of the distal knuckle to represent the fingertip.

[0117] The human palm is a complex object composed of five metacarpal bones. The metacarpal bone of the thumb has a rotational degree of freedom, and the movements of the other four metacarpal bones are very small. The metacarpal bone region of the thumb is regarded as a phalanx, and a truncated cone model is used to represent it like other phalanges. Therefore, each finger of the virtual hand is modeled using a hinge model including three truncated cones. Considering the small range of motion of the other four metacarpal bones, the palm except the metacarpal bone region of the thumb can be regarded as a rigid body, which can be further represented by an octree model. Figure 3 A virtual hand model based on a truncated cone hinge is given, in which the palm adopts an octree model of different levels.

[0118] A sphere-truncated cone collision response module is used to:

[0119] A contact judgment condition between the sphere and the truncated cone (here, a contact detection condition for the side of the finger and the virtual object) is constructed: And Where |OC| j is the distance from the center of the sphere numbered j on the virtual object to the central axis l of the truncated cone in the graphic hand, R j is the radius of the sphere numbered j on the virtual object, and M is the number of spheres to be detected on the virtual object, C j represents the foot of the center of the sphere numbered j on the virtual object on the central axis l, E and D represent the centers of the upper and lower bases of the truncated cone in the graphic hand, respectively, a represents the included angle between the side of the truncated cone in the graphic hand and the central axis, r2 and r1 represent the radii of the upper and lower bases of the truncated cone in the graphic hand, respectively;

[0120] The finger pose information and the collision sphere information of the graphic hand are obtained according to the contact constraint between the sphere and the truncated cone; the collision sphere information includes the number, the center coordinates and the radius of the collision sphere; the collision sphere is the sphere in contact with the finger of the graphic hand.

[0121] According to the graphic hand pose information, the virtual object pose information, and the contact judgment condition between the sphere and the circular truncated cone, collision sphere information in contact with a finger side edge of the graphic hand is obtained; the collision sphere information includes a number of the collision sphere, a sphere center coordinate of the collision sphere, and a sphere radius of the collision sphere; the collision sphere is a sphere in the virtual object in contact with the graphic hand; the finger side edge of the graphic hand is a side edge of a finger circular truncated cone joint; and the graphic hand pose information is used for graphic display.

[0122] The sphere-circular truncated cone collision detection module in the embodiment, on the basis of the virtual finger being represented by a circular truncated cone and the virtual object being represented by a sphere tree model, adopts a sphere-circular truncated cone collision detection algorithm to determine whether the virtual finger collides with the virtual object, and the algorithm fully considers the characteristics of different radii of the virtual finger at different sections and can accurately determine the contact state of the virtual finger at different positions and the object.

[0123] Specifically, the purpose of collision detection is to determine whether the virtual hand avatar collides with a virtual object in a virtual scene at a certain time and how the collision occurs. In order to support real-time contact processing, the collision detection algorithm for virtual hand interaction needs to have high computational efficiency. At the same time, relevant parameters of the contact state of the virtual hand and the virtual object need to be accurately calculated, so as to provide accurate contact information for collision response. The embodiment proposes a sphere-circular truncated cone collision detection algorithm to determine the contact state of the virtual finger and the virtual object.

[0124] As shown in Figure 4 , the circle is a sphere on the virtual object, and the trapezoid represents a circular truncated cone simulating a finger. Point C is the foot of the perpendicular from the sphere center O to the center axis l of the circular truncated cone. D and E represent the centers of the two circular bases of the circular truncated cone, and the coordinates are (x D ,y D ,z D ) and (x E ,y E ,z E ) respectively. The contact state between the virtual object and the finger can be determined by comparing the current |OC| (i.e. the distance from the center O of the object sphere to the center axis l of the virtual finger) and the threshold |OC| Γ ( Figure 4 ) when the sphere is tangent to the side of the circular truncated cone. If |OC|≤|OC| Γ , it can be determined that the circular truncated cone collides with the sphere, and |OC| can be calculated as follows:

[0125]

[0126] wherein l1, l2, l3 and l4 are calculated by the following formula:

[0127]

[0128] Assuming the radii of the upper and lower bases of the circular truncated cone are r2and r1(r2 Γ :

[0129]

[0130] where R is the radius of the sphere, a is the angle between the side of the circular truncated cone and the central axis, and k represents the position of the foot C of the center O of the virtual sphere on the central axis of the virtual finger (i.e. and the ratio of is as follows:

[0131]

[0132] where l5= (x O -x E )(x D -x E )+(y O -y E )(y D -y E )+(z O -z E )(z D -z E ).

[0133] As shown in Figure 4 (b), although the condition |OC|≤|OC| Γ is satisfied, the gray dashed sphere does not contact the circular truncated cone. More specifically, when k>1+R / |DE| or k<-R / |DE|, the sphere will not contact the circular truncated cone. When k=1+R / |DE|, the sphere is tangent to the lower base (extension) of the circular truncated cone (i.e. Figure 4 the black dashed sphere O1 in (c)); when k=-R / |DE|, the sphere is tangent to the upper base (extension) of the circular truncated cone (i.e. Figure 4 the black dashed sphere O2 in (c)); when k=0, the foot C coincides with the endpoint E; and when k=1, the foot C coincides with the endpoint D. In general, another condition, i.e., -R / |DE|<k<1+R / |DE|, needs to be satisfied for the sphere and the circular truncated cone to collide. Therefore, when the following condition is satisfied, it can be determined that the virtual finger collides with the virtual object.

[0134]

[0135] The sphere-circular truncated cone collision response module is configured to:

[0136] using and The metacarpophalangeal joint angle, the proximal phalangeal joint angle and the distal phalangeal joint angle of each finger of the graphic hand are respectively represented, and a non-embedded contact constraint of a side (i.e. a circular truncated cone) of a finger of the graphic hand and a virtual object (i.e. a sphere) is constructed according to the graphic hand pose information and the collision sphere information of the contact between the graphic hand and the virtual object where N is the number of spheres on the virtual object in contact with the virtual fingers of the graphic hand, and according to the non-embedded contact constraint of the circular truncated cone and the sphere, for each finger of the graphic hand, the three joint angles of the finger are solved and as a whole under the non-embedded contact constraint set, and the joint angle information of the finger of the graphic hand not embedded in the virtual object is obtained.

[0137] The sphere-circular truncated cone collision response module in the embodiment, on the basis of the collision information obtained by the sphere-circular truncated cone collision detection algorithm, adopts a sphere-circular truncated cone collision response algorithm to solve a reasonable virtual hand pose, the algorithm constructs a sphere-circular truncated cone contact constraint suitable for different radius sections of the virtual fingers, and can optimize and solve the virtual finger pose not embedded in the side of the virtual object.

[0138] Specifically, collision response is one of the core components of force feedback rendering, and the module needs the information obtained by the collision detection module as its input, and then performs collision response related calculations. In the early stage, the contact processing of virtual hand force feedback interaction depends on the prior information of pre-defined hand and / or object contact, and the collision response is executed according to the closeness between the actual operation behavior and the pre-defined operation behavior. This method performs well in pre-defined operations, but in actual applications, there is unrealistic contact (such as mutual penetration) between the virtual hand and the manipulated object. In order to realize realistic virtual hand interaction, the embodiment adopts a constraint-based method to perform contact processing in the virtual hand force feedback interaction process, and specifically, for the constructed virtual hand model, an optimization target is established and a non-penetration constraint of the virtual hand and the virtual object interaction is established according to the collision detection result, and finally a reasonable graphic pose of the virtual hand not penetrating the virtual object is obtained.

[0139] In virtual hand haptic interaction simulation, the direct mapping avatar of the user's hand in the virtual environment is called haptic hand, and the pose thereof is obtained by directly mapping the pose of the real hand (for example, the six-degree-of-freedom position information of the hand is obtained by an HTC tracker, and the bending angles of the knuckles of each finger are obtained by a Razer data glove) into the virtual world. The visual avatar of the haptic hand is defined as a graphic hand, which is used for graphic display. In order to realistically and naturally reproduce the virtual hand interaction, it is necessary to ensure that the graphic hand does not penetrate the virtual object during the interaction.

[0140] The virtual hand finger of this embodiment adopts a circular truncated cone hinge model, and a spherical-circular truncated cone contact constraint needs to be constructed to calculate the finger pose information (for display) of the graphic hand during interaction with virtual objects, wherein and represent the metacarpophalangeal joint angle, the proximal interphalangeal joint angle and the distal interphalangeal joint angle of each finger respectively. For each finger, the three joint angles are optimized as a whole, and the optimization objective can be written as:

[0141]

[0142] wherein, represents the finger joint angle information of the haptic hand (i.e. obtained by a hardware device such as a data glove), and are defined in the same way as and . G r is a 3x3 diagonal torsional stiffness matrix.

[0143] After the palm pose information (3DoF (degree of freedom) translation and 3oF rotation) of the graphic hand and the joint angle information and of the fingers are known, the coordinates (x D , y D , z D ) and (x E , y E , z E ) of the centers D and E of the two bottom surfaces of the circular truncated cone in the global coordinate system are obtained through the following matrix transformation:

[0144]

[0145] wherein, P D / E represents the coordinates of D or E in the global coordinate system, represents the local coordinates of D or E, which are determined by the virtual finger knuckle length. is a homogeneous translation rotation matrix containing the six-dimensional variables in . M s (θ s ) is a rotation matrix of the abduction movement of the metacarpal joint, and θ s is the abduction angle. is a homogeneous translation rotation matrix of the metacarpal joint relative to the carpometacarpal joint coordinate system, is a translation rotation matrix of the proximal interphalangeal joint relative to the metacarpal joint, is a translation rotation matrix of the distal interphalangeal joint relative to the proximal interphalangeal joint.

[0146] Therefore, the spherical-circular truncated cone contact constraint can be expressed as:

[0147]

[0148] Among them, C j Let J represent the j-th constraint, and M be the number of spheres on the virtual object that are in contact with the frustum.

[0149] According to the sphere-frustum collision detection conditions (Equation (5)) described in the sphere-frustum collision detection module, the contact constraint between the sphere and the frustum in Equation (8) can be expressed as:

[0150]

[0151] Among them, |OC| j Let O(x) be the center of the j-th sphere that collides with the frustum. Oj ,y Oj ,z Oj The distance R from the central axis l of the frustum j Let k be the radius of the sphere. j Represents the center of the sphere O(x) Oj ,y Oj ,z Oj The position of the foot of the perpendicular on the central axis l, i.e. Formula (8) is the unified way of writing constraints, and formula (9) is the method of calculating constraints.

[0152] For a specific finger, the parameters r1, r2, and α of the frustum are constant. and Substituting equations (2) and (4), the parameters to be optimized can be expressed as equation (9). After linearizing the contact constraints using Taylor expansion, the joint angle information of the five fingers of the graphic hand is obtained by iterative optimization using the effective set method. and ).

[0153] Endpoint contact processing module, used for:

[0154] This module addresses the issue of virtual fingertips being suspended or embedded when in contact with virtual objects. It determines the contact state between the virtual hand's finger and the virtual object sphere by identifying the position of the virtual object sphere on the central axis of the finger's frustum. Based on this state, it determines whether to use sphere-frustum constraints or sphere-sphere constraints for collision response constraint optimization.

[0155] Specifically, when using the sphere-cylinder collision response algorithm to perform virtual hand interaction, an issue arises where the fingertip pose information is abnormally optimized, such as... Figure 5 As shown, (a) is a case where the fingertips of the graphic hand are suspended in the air (i.e., the graphic hand is separated from the virtual object), and (b) is a case where the fingers of the graphic hand are embedded (i.e., the graphic hand penetrates the virtual object).

[0156] To solve the above-mentioned endpoint exception problem, the embodiment constructs an endpoint contact processing module. The virtual finger fingertip and the ball contact state on the object are as shown in Figure 6 When the hemisphere connected at the bottom of the circular table collides with the object ball, the ball-ball collision detection is used to determine the contact state of the virtual fingertip and the virtual object, that is, by comparing whether the distance between the centers of the fingertip and the virtual object on the two balls is less than the sum of the radii of the two balls, as follows:

[0157] |OC| 2 ≤(R+r2) 2 (10)

[0158] Wherein, |OC| 2 =(x T -x O ) 2 +(y T -y O ) 2 +(z T -z O ) 2 , (x T ,y T ,z T ) and (x O ,y O ,z O ) respectively represent the center coordinates of the virtual fingertip hemisphere and the virtual object ball in the global coordinate system, and R represents the radius of the virtual object ball.

[0159] Specifically, according to the ball-circular table collision detection condition, while saving the information of the collision ball itself (including the ball number, the center coordinates of the ball, and the radius of the ball), the state of the collision ball is also saved. When constructing the constraint, if the state of the collision ball is that the position of its center on the center axis of the circular table satisfies the condition of formula (11) while satisfying formula (10), the contact constraint of the ball is established as a ball-ball contact constraint, otherwise the contact constraint of the ball is established as a ball-circular table contact constraint represented by formula (8):

[0160]

[0161] Wherein, R represents the radius of the ball colliding with the circular table. The ball-ball contact constraint refers to the distance between the centers of the two balls being greater than or equal to the sum of the radii of the two balls, which is obtained according to the ball-ball collision detection condition of formula (10). Each pair of balls is processed in the same way, without distinguishing the number of balls.

[0162] Formula (11) is obtained according to the geometric relationship between the object ball and the fingertip hemisphere in Figure 6 , and the geometric relationship between the object ball and the finger circular table.

[0163] k = -R * sin a / |DE| is the object sphere center and the edge of the finger cone base perpendicular to the side of the cone.

[0164] When the object sphere is tangent to the finger hemisphere.

[0165] In addition, when the haptic hand is embedded in the virtual object, the graphic hand optimization abnormal problem occurs, that is, the phenomenon of the virtual finger embedded in the virtual object. This is mainly due to the use of the haptic hand as the collision body. When the haptic hand is embedded in the virtual object (even through), the result of the collision detection is the object sphere in contact with the haptic hand at the current time. On the one hand, these spheres may not be on the surface of the virtual object, resulting in the inability to establish a surface contact constraint; on the other hand, when the haptic hand penetrates the virtual object, the result of the collision detection is basically invalid, resulting in the inability to establish appropriate sphere-cone and sphere-sphere contact constraints. Thus, the appropriate graphic hand pose cannot be optimized.

[0166] Through the analysis of the above-mentioned graphic hand optimization abnormal problem when the haptic hand is embedded in the virtual object, the embodiment uses the graphic hand as the collision detection body to detect whether the cone and sphere formed by the finger elements of the graphic hand collide with the object sphere. The collision result is used for the construction of the sphere-cone and sphere-sphere contact constraints. In addition, the idea of contact prediction is used to alleviate the problem of missed detection, such as Figure 7 As shown in the figure, the black trapezoid represents the knuckle of the graphic hand at t-1 time, the radius of each section of the knuckle at t-1 time is increased by Δr to obtain the dashed trapezoid, then the cone-sphere contact pair is obtained by using the collision detection algorithm described in the second section, and then the contact constraint of the intersecting cone and sphere is established according to formula (8). Finally, under this constraint, the contact constraint is linearized by using Taylor expansion, and the effective set method is used to iteratively optimize the non-penetration joint angle of the virtual finger, Figure 7 The trapezoid t shown in the figure is the calculated virtual hand finger.

[0167] As an optional implementation, when the palm interaction is involved, the palm pose information of the graphic hand is first optimized, the finger pose information of the graphic hand is obtained through matrix transformation, and then the ball-cone collision detection module, the ball-cone collision response module, and the end contact processing module are used to optimize and solve the joint angle information of the graphic hand finger not embedded in the virtual object according to the finger pose information of the graphic hand and the pose information of the virtual object.

[0168] The contact force calculation module is used to:

[0169] This module calculates the contact force information between the virtual hand and the virtual object during interaction. It uses contact information obtained from the collision detection module and optimization results from the collision response module to calculate the contact force between the virtual finger and the virtual object during the interaction.

[0170] It is worth noting that, in addition to the contact force calculation method related to the contact point and contact area mentioned in this invention, calculating richer feedback forces using the contact information obtained by the collision detection module and the optimization results of the collision response module proposed in this invention, such as calculating the fingertip normal contact force for a force feedback glove that can only provide fingertip feedback force, is also within the scope of protection of this invention.

[0171] Specifically, in addition to efficient and accurate collision detection and collision response algorithms, the calculation of contact force during the interaction between virtual fingers and virtual objects is another key step in achieving continuous and stable force feedback. Figure 8 The diagram shows partial contact between a virtual finger and a virtual object. The gray trapezoid represents the cross-section of a virtual knuckle (frustum), and the black circle represents a portion of the sphere on the surface of the virtual object that is in contact with the virtual finger. Figure 8 The demonstrated contact state between the virtual hand and the virtual object is obtained through a series of contact processing methods, including collision detection and collision response algorithms, and is ultimately optimized. Specifically, during the interaction between the virtual hand and the virtual object, the virtual fingers do not embed themselves into the virtual object.

[0172] Assume that n balls on the surface of the virtual object come into contact with the virtual finger, and... Figure 8 For example, suppose the i-th ball (with center O) i The contact point (i.e., the tangent point) between the virtual finger and the side of the finger is P. i The positive pressure (gray arrow) applied to the virtual finger during the interaction is F. i Its direction is perpendicular to the side of the virtual finger, that is, perpendicular to the vector. Same direction. Center O i global coordinates Given, calculate F i The direction needs to be solved first for P. i global coordinates

[0173] Next, we will introduce P. i global coordinates A calculation method. Let The intersection point of the extension of and l is The calculation can be performed using equation (3). Furthermore, given α and C i The position on the central axis l characterizes the parameter k. i Under the premise of (4)), Q can be calculated i global coordinates and and The ratio of to is used to calculate P. i global coordinates Let it be denoted as P ig This refers to the coordinates of the contact position on the graphic hand. Similarly, the coordinates of the point of contact with P can be calculated using the local coordinates of the two ends of the corresponding phalanx on the haptic hand. i Corresponding position P i The global coordinates at point ' are denoted as P. ih The normal force F between the i-th ball and the corresponding virtual knuckle can be calculated using the following formula. i :

[0174] F i =G i ·n′·|P ig -P ih | (12)

[0175] Among them, G i It is a 3×3 stiffness matrix. express The unit vector of . |P ig -P ih |P is the point on the graphichand graphic. ig The corresponding point P on the haptic hand ih The length of the line segment between them.

[0176] Furthermore, if there is a tendency for relative motion between the i-th ball and the virtual finger, or if relative motion has already occurred, the frictional force f between the i-th ball and the virtual finger can be calculated using the contact normal force calculated by this invention. i If relative friction occurs between the two, and the coefficient of friction of the virtual object's surface is u, then one method for calculating f is... i The method is as follows:

[0177] f i =u·n i ·|F i | (13)

[0178] Where, n i Let be the normal vector of the relative motion between the i-th ball and the virtual finger.

[0179] The calculations above represent the normal force and friction generated when a ball on the surface of a virtual object comes into contact with a virtual finger. Considering the form of feedback force provided by the force feedback device and the position of the force applied on the finger, the required feedback force can be further calculated. The specific flowchart of this embodiment is shown below. Figure 9 As shown.

[0180] The contact processing method and system for virtual hand force tactile interaction can be used not only for single-finger single-joint interaction, but also for single-finger multi-joint interaction and multi-finger interaction. When multi-finger interaction is involved, parallel processing can be used to improve the efficiency of contact processing, which is also within the protection scope of the present application.

[0181] Notably, the interaction includes virtual hand touching, grabbing virtual objects, and virtual hand sliding on the surface of an object, and also includes single-person double-hand operation and multi-person cooperation.

[0182] The contact processing method and system for virtual hand force tactile interaction not only support virtual hand interaction with a virtual object in a virtual scene in the above-mentioned manner, but also support interaction between virtual hand and virtual objects of different shapes in a virtual scene, which is also within the protection scope of the present application.

[0183] Notably, the virtual objects include not only virtual objects with regular geometric shapes such as spheres, cylinders, and cubes, but also virtual objects with complex geometric features such as sharp features and concave shapes. In addition, the virtual objects can be rigid bodies or deformable bodies, which are also within the protection scope of the present application.

[0184] The contact processing system for virtual hand force tactile interaction has the following advantages:

[0185] 1. The cone model adopted by the virtual hand model in the system is conducive to more realistic virtual hand visualization and facilitates the implementation of virtual finger-object collision detection algorithm and collision response algorithm, thereby realizing efficient and stable contact processing.

[0186] 2. The sphere-cone collision detection algorithm can accurately and efficiently determine the contact state of the virtual finger side edge and the virtual object, and can adapt to complex contact types such as multi-point / multi-area sliding contact and concave contact.

[0187] 3. The sphere-cone collision response algorithm can accurately constrain the optimized graphical virtual finger side edge to the surface of the virtual object, effectively avoiding the virtual finger from being embedded in the virtual object, and can well adapt to the constraint optimization of complex contact (multi-point / multi-area sliding contact, concave contact, etc.).

[0188] 4. The end-point contact processing method can effectively avoid the virtual finger tip from being suspended in the air for a distance or embedded in the virtual object surface during interaction, and well supports fingertip operations such as fingertip sliding on the object surface and two-finger fingertip pinching.

[0189] 5. The contact processing method for virtual hand haptic interaction proposed in this paper can effectively calculate the contact forces of various contact types (single-point contact, multi-point contact, etc.) involved in virtual hand interaction, including normal contact force and tangential contact force;

[0190] 6. The contact processing method for virtual hand haptic interaction proposed in this paper can accelerate the methods of collision detection, collision response, end-point contact processing and contact force calculation involved in contact processing by using parallel processing according to the anatomical characteristics of human hand, thereby improving the efficiency of virtual hand haptic interaction;

[0191] 7. The contact processing method and system for virtual hand haptic interaction proposed in this paper can achieve a haptic interaction update frequency of 1 kHz without using graphics card acceleration, and can be used for low-cost development of virtual hand interaction application systems such as games and entertainment.

[0192] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be mutually referred to.

[0193] The principles and implementation modes of the present application are described by applying specific examples in this paper. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation modes and application ranges of the present application can be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A contact handling method for virtual hand haptics interaction, characterized in that, The method comprises: simulate a real hand by using a circular-truncated cone hinge model to obtain a virtual hand; the circular-truncated cone hinge model comprises a plurality of circular-truncated cones, each of which simulates each knuckle of the real hand; the metacarpophalangeal joint and the joint between two knuckles are simulated by using a sphere; the virtual hand further comprises a plurality of hemispheres, each of which is located on a circular-truncated cone simulating a distal knuckle, and each of which is used to simulate a fingertip of the real hand; simulate an object by using a sphere tree model to obtain a virtual object; the virtual object comprises a plurality of spheres with numbers; obtain a pose of a real hand of a user; drive the virtual hand by using the pose of the real hand, which is referred to as a haptic hand; display the haptic hand in a virtual scene to obtain a graphic hand, and display the virtual object in the virtual scene; Constructing the contact judging condition between the sphere and the circular truncated cone: And Wherein, |OC| j is the distance from the center of the sphere numbered j on the virtual object to the central axis l of the circular truncated cone in the graphic hand, R j is the radius of the sphere numbered j on the virtual object, M is the number of spheres to be detected on the virtual object, C j represents the foot of the center of the sphere numbered j on the virtual object on the central axis l, E and D respectively represent the centers of the upper and lower bases of the circular truncated cone in the graphic hand, α represents the included angle between the side of the circular truncated cone in the graphic hand and the central axis thereof, r2 and r1 respectively represent the radii of the upper and lower bases of the circular truncated cone in the graphic hand; obtain collision sphere information in contact with a side edge of a finger of the graphic hand according to graphic hand pose information, virtual object pose information, and a contact judgment condition between the spheres and the circular-truncated cones; the collision sphere information comprises a number of a collision sphere, a sphere center coordinate of the collision sphere, and a sphere radius of the collision sphere; the collision sphere is a sphere of the virtual object in contact with the graphic hand; the side edge of the finger of the graphic hand is a side edge of a knuckle of a finger circular-truncated cone; the graphic hand pose information is used for graphic display; Utilizing and respectively represent the metacarpophalangeal joint angle, the proximal interphalangeal joint angle and the distal interphalangeal joint angle of each finger of the graphic hand, according to the graphic hand pose information and the collision sphere information of the contact between the graphic hand and the virtual object, a non-embedded contact constraint between the lateral edge of the circular truncated cone of the finger of the graphic hand and the sphere of the virtual object is constructed wherein N is the number of spheres on the virtual object in contact with the virtual finger of the graphic hand; according to the non-embedded contact constraint of the circular truncated cone and the sphere, for each finger of the graphic hand, its three joint angles and are optimized as a whole under the non-embedded contact constraint set, and the joint angle information of the finger of the graphic hand which does not embed the virtual object is solved. According to the graphical hand finger joint angle information and formula F i = G i · n' · |P ig -P ih | the normal pressure F of the ith collision ball in contact with the corresponding virtual finger is calculated i ; wherein G i is a 3x3 stiffness matrix, n' represents a unit vector, O i represents the center of the ith collision ball, P i represents the contact point of the ith collision ball with the side of the virtual finger, |P ig -P ih | is the length of the line segment between the point P ig on the graphical hand and the corresponding position point P ih on the haptic hand.

2. The contact handling method for virtual hand force tactile interaction according to claim 1, wherein, when judging contact between the graphic hand and the virtual object: the pose of the graphic hand is a pose of the graphic hand at a previous time; a radius of a cross section of each knuckle circular-truncated cone of the graphic hand is increased by Δr; a radius of each hemisphere of the graphic hand is increased by Δr; the graphic hand obtained by the above changes is used for judging contact between the graphic hand and the virtual object, and the collision sphere obtained according to the sphere-circular-truncated cone contact judgment condition and the sphere-sphere contact judgment condition is used for constructing a non-embedded contact constraint set.

3. The contact handling method for virtual hand force tactile interaction of claim 1, wherein, For each distal phalangeal joint of a finger, while constructing the non-embedded contact constraint also includes: If the collision sphere has contact with the sphere of the simulated fingertip, then a non-embedding constraint between the sphere and the hemisphere is constructed: This contact constraint is added to the set of non-embedding contact constraints used for the optimization of the graphical finger joint angles.

4. The contact processing method for virtual hand force tactile interaction according to claim 1, wherein, In accordance with formula F i = G i · n' · |P ig - P ih | the positive pressure F calculated for the i-th colliding ball in contact with the corresponding virtual knuckle i Further comprising: Using the formula f i = u · n i · |F i | to calculate the friction during the virtual hand sliding process; where f i represents the friction at the i-th collision sphere, u represents the dynamic friction coefficient, and n i is the normal vector of the relative motion between the i-th collision sphere and the virtual finger.

5. A contact handling system for virtual hand force-impedance interaction, characterized by The system comprises: a virtual hand construction module, configured to: simulate a real hand by using a circular-truncated cone hinge model to obtain a virtual hand; the virtual hand comprises a plurality of circular-truncated cones, each of which simulates each knuckle of the real hand; the metacarpophalangeal joint and the joint between two knuckles are simulated by using a sphere; the virtual hand further comprises a plurality of hemispheres, each of which is located on a circular-truncated cone simulating a distal knuckle, and each of which simulates each fingertip of the real hand; simulate an object by using a sphere tree model to obtain a virtual object; the virtual object comprises a plurality of spheres with numbers; obtain a pose of a real hand of a user; drive the virtual hand by using the pose of the real hand, which is referred to as a haptic hand; display the haptic hand in a virtual scene to obtain a graphic hand, and display the virtual object in the virtual scene; a sphere-circular-truncated cone collision detection module, configured to: Constructing the contact judging condition between the sphere and the circular truncated cone: And Wherein, |OC| j is the distance from the sphere center of the sphere numbered j on the virtual object to the central axis l of the circular truncated cone in the graphic hand, R j is the radius of the sphere numbered j on the virtual object, M is the number of spheres to be detected on the virtual object, C j represents the foot of the sphere center of the sphere numbered j on the virtual object on the central axis l, E and D respectively represent the centers of the upper and lower bases of the circular truncated cone in the graphic hand, α represents the included angle between the side of the circular truncated cone in the graphic hand and the central axis thereof, r2 and r1 respectively represent the radii of the upper and lower bases of the circular truncated cone in the graphic hand; obtain collision sphere information in contact with a side edge of a finger of the graphic hand according to graphic hand pose information, virtual object pose information, and a contact judgment condition between the spheres and the circular-truncated cones; the collision sphere information comprises a number of a collision sphere, a sphere center coordinate of the collision sphere, and a sphere radius of the collision sphere; the collision sphere is a sphere of the virtual object in contact with the graphic hand; the side edge of the finger of the graphic hand is a side edge of a knuckle of a finger circular-truncated cone; the graphic hand pose information is used for graphic display; a sphere-circular-truncated cone collision response module, configured to: Utilizing and respectively represent the metacarpophalangeal joint angle, the proximal interphalangeal joint angle and the distal interphalangeal joint angle of each finger of the graphic hand, according to the graphic hand pose information and the collision sphere information of the contact between the graphic hand and the virtual object, a non-embedded contact constraint between the lateral edge of the circular truncated cone of the finger of the graphic hand and the sphere of the virtual object is constructed wherein N is the number of spheres on the virtual object in contact with the virtual finger of the graphic hand; according to the non-embedded contact constraint of the circular truncated cone and the sphere, for each finger of the graphic hand, the three joint angles and are optimized as a whole under the non-embedded contact constraint set, and the joint angle information of the finger of the graphic hand which does not embed the virtual object is solved. a contact force calculation module, configured to: According to the graphical hand finger joint angle information and formula F i = G i · n' · |P ig - P ih | calculate the normal pressure F of the ith collision ball in contact with the corresponding virtual finger i ; wherein G i is a 3x3 stiffness matrix, n' represents a unit vector, O i represents the center of the ith collision ball, P i represents the contact point of the ith collision ball with the virtual finger side, |P ig - P ih | is the length of the line segment between point P ig on the graphical hand and point P ih on the corresponding position of the haptic hand.

6. The contact processing system for virtual hand force tactile interaction of claim 5, wherein, In the contact judgment of the graphic hand and the virtual object: The ball-circular cone collision detection module further comprises the following features: The pose of the graphic hand is the pose of the graphic hand at the previous time; The radius of each knuckle cone section of the graphic hand is increased by Δr; The radius of each knuckle hemisphere of the graphic hand is increased by Δr; The graphic hand obtained by the above changes is used for the contact judgment of the graphic hand and the virtual object, and the collision ball obtained according to the ball-circular cone contact judgment condition and the ball-ball contact judgment condition is used for the construction of the non-embedded contact constraint set.

7. The contact processing system for virtual hand force tactile interaction of claim 5, wherein, Further comprising: An endpoint contact processing module, configured to: For each distal phalangeal joint of the fingers, while constructing the non-embedded contact constraints of the fingers; If the ball collides with the sphere of the simulated fingertip, a non-embedding constraint between the sphere and the hemisphere is constructed: This contact constraint is added to the set of non-embedding contact constraints used for the optimization of the graphical finger joint angles; If the collision ball is not in contact with the ball of the simulated fingertip, only the contact constraint between the ball and the circular cone is constructed.

8. The contact processing system for virtual hand force tactile interaction of claim 5, wherein, In accordance with formula F i = G i · n' · |P ig - P ih | the positive pressure F i After that, also comprising: Using the formula f i = u · n i · |F i | to calculate the friction during the virtual hand sliding process; wherein f i represents the friction at the i-th collision sphere, u represents the dynamic friction coefficient, and n i is the normal vector of the relative motion between the i-th collision sphere and the virtual finger.

9. The contact processing system for virtual hand force tactile interaction of claim 5, wherein, When the palm interaction is involved, the palm pose information of the graphic hand is first optimized, the finger pose information of the graphic hand is obtained through matrix transformation, and then the graphic hand finger joint angle information of the graphic hand not embedded in the virtual object is optimized and solved according to the finger pose information of the graphic hand and the pose information of the virtual object by using the ball-circular cone collision detection module, the ball-circular cone collision response module, and the endpoint contact processing module.

10. The contact processing system for virtual hand force tactile interaction of claim 5, wherein, When processing multi-finger interaction, a parallel processing mode is adopted.

Images