Grabbing operation method and system based on dynamic release distance adjustment and program product

Abstract

The invention belongs to the technical field of mixed reality, and particularly discloses a grabbing operation method and system based on dynamic release distance adjustment and a program product, collision condition analysis aiming at a target object is carried out by collecting spatial position parameters of fingers of a user in real time, so that grabbing operation is triggered when corresponding grabbing conditions are met, and the grabbing efficiency is improved. And then whether the grabbing operation fingers meet the releasing condition or not is judged according to the calculated time dynamic releasing distance and the real-time fingertip distance of the grabbing operation fingers, so that the releasing operation is triggered when the releasing condition is met, and the mixed reality interaction object is grabbed and released with high recognition precision and high reliability. The stability and reliability of the grabbing operation are ensured by monitoring the fingertip distance in real time and dynamically adjusting the release distance, the grabbing success rate of the mixed reality interaction object can be effectively increased, the operation failure rate is reduced, and the operation experience of a user is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of mixed reality, and particularly relates to a grasping operation method, system and program product based on dynamic release distance adjustment. Background Art

[0002] Mixed Reality (MR) is a technology that deeply integrates the virtual world with the real environment. It combines Virtual Reality (VR) and Augmented Reality (AR), and through computer-generated virtual objects interact with the real world in real time. It can superimpose virtual elements onto the real environment and endow these virtual objects with physical properties of the real world (such as real-time physical interaction and dynamic reaction).

[0003] Mixed reality technology can be applied to fields such as MR games, metaverse social interactions, virtual surgery training, virtual design, etc. In these application scenarios, sometimes users need to interact with virtual objects through gestures to perform operations such as grasping objects and throwing objects. However, in the existing MR gesture recognition technology, when users perform grasping operations, it is easy to have problems of inaccurate and unstable monitoring due to the shaking or collision of users' fingers, which may lead to the failure of object grasping or the dropping of the object after grasping. The instability of the grasping operation will directly affect the user's immersion and operation experience, especially in scenarios that require fine operations. Summary of the Invention

[0004] The purpose of the present invention is to provide a grasping operation method, system and program product based on dynamic release distance adjustment to solve the above problems existing in the prior art.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions: In the first aspect, a grasping operation method based on dynamic release distance adjustment is provided, including: Real-time collecting the spatial position parameters of each finger of the user; When it is determined according to the spatial position parameters of each finger that two of the fingers satisfy the collision condition with the target object, triggering the grasping operation condition, outputting a grasping operation instruction for the target object, and taking the time point when the grasping operation condition is triggered as the initial time point; Determining the two fingers of the user for grasping the target object, and taking the two fingers of the user for grasping the target object as the grasping operation fingers; Calculating the real-time fingertip distance between the two grasping fingers according to the spatial position parameters of the two grasping operation fingers, and determining the real-time dynamic release distance according to the initial time point; When it is determined according to the spatial position parameters of the two grasping fingers that the two grasping fingers do not meet the collision condition with the target object, and it is determined that the real-time fingertip distance between the two grasping fingers exceeds the real-time dynamic release distance, the release operation condition is triggered, and a release operation instruction for the target object is output.

[0006] In a possible design, the determining that two of the fingers meet the collision condition with the target object according to the spatial position parameters of each finger includes: When it is determined according to the spatial position parameters of each finger that the fingertips of two of the fingers both enter the set collision range of the target object, it is determined that the two fingers meet the collision condition with the target object, and the grasping operation condition is triggered.

[0007] In a possible design, the calculating the real-time fingertip distance between the two grasping fingers according to the spatial position parameters of the two grasping fingers includes: Determine the fingertip spatial position coordinates of the two grasping fingers according to the spatial position parameters of the two grasping fingers; Based on the fingertip spatial position coordinates of the two grasping fingers, calculate the real-time fingertip distance d between the two grasping fingers:

[0008] where (x1, y1, z1) and (x2, y2, z2) are the fingertip spatial position coordinates of the two grasping fingers respectively.

[0009] In a possible design, the determining the real-time dynamic release distance according to the initial time point includes: Obtain the set maximum distance value, and calculate the real-time dynamic release distance D based on the maximum distance value and the initial time point:

[0010] where D max is the set maximum distance value, v is the set change rate parameter, and t is the duration from the initial time point to the current time point.

[0011] In a possible design, the determining the real-time dynamic release distance according to the initial time point includes: Obtain the set maximum distance value, and calculate the real-time dynamic release distance D based on the maximum distance value and the initial time point:

[0012] where D max is the set maximum distance value, k is the set attenuation coefficient, and t is the duration from the initial time point to the current time point.

[0013] In a possible design, determining the real-time dynamic release distance according to the initial time point includes: Obtain the set maximum spacing value, and calculate the real-time dynamic release distance D based on the maximum spacing value and the initial time point:

[0014] where D max is the set maximum spacing value, T1 is the set first reference duration starting from the initial time point, T2 is the set second reference duration starting from the initial time point, T2 > T1, d is the real-time fingertip spacing between the two grasping fingers at the current time point, and t is the duration from the initial time point to the current time point.

[0015] In a possible design, determining that neither of the two grasping operation fingers satisfies the collision condition with the target object according to the spatial position parameters of the two grasping operation fingers includes: When it is determined according to the spatial position parameters of the two grasping operation fingers that the fingertips of the two grasping operation fingers have left the set collision range of the target object, it is determined that the two grasping operation fingers do not satisfy the collision condition with the target object.

[0016] In a second aspect, a grasping operation system for a grasping operation method based on dynamic release distance adjustment is provided, including a position acquisition unit, a grasping trigger unit, a finger determination unit, a distance calculation unit, and a release trigger unit, where: The position acquisition unit is configured to acquire the spatial position parameters of each finger of the user in real time; The grasping trigger unit is configured to trigger the grasping operation condition when it is determined according to the spatial position parameters of each finger that two of the fingers satisfy the collision condition with the target object, output a grasping operation instruction for the target object, and use the time point when the grasping operation condition is triggered as the initial time point; The finger determination unit is configured to determine the two fingers by which the user grasps the target object, and use the two fingers by which the user grasps the target object as the grasping operation fingers; The distance calculation unit is configured to calculate the real-time fingertip spacing between the two grasping fingers according to the spatial position parameters of the two grasping operation fingers, and determine the real-time dynamic release distance according to the initial time point; The release trigger unit is configured to trigger the release operation condition when it is determined according to the spatial position parameters of the two grasping operation fingers that the two grasping operation fingers do not satisfy the collision condition with the target object and it is determined that the real-time fingertip spacing between the two grasping operation fingers exceeds the real-time dynamic release distance, and output a release operation instruction for the target object.

[0017] In a third aspect, a grasping operation system based on dynamic release distance adjustment is provided, including: A memory for storing instructions; A processor, configured to read instructions stored in the memory and execute any one of the methods described in the first aspect according to the instructions.

[0018] In a fourth aspect, a computer-readable storage medium is provided, on which instructions are stored. When the instructions are run on a computer, the computer is caused to execute any one of the methods described in the first aspect. Meanwhile, a computer program product is also provided, which, when run on a computer, executes any one of the methods described in the first aspect.

[0019] Advantageous effects: By collecting the spatial position parameters of the user's finger in real time, the present invention analyzes the collision conditions for a target object, triggers a grasping operation when corresponding grasping conditions are met, and then determines whether the grasping operation fingers meet the release conditions based on the calculated dynamic release distance and the real-time fingertip distance between the fingers during the grasping operation, and triggers a release operation when the release conditions are met, so as to achieve high-accuracy and high-reliability grasping and releasing of mixed reality interactive objects. By monitoring the fingertip distance in real time and dynamically adjusting the release distance, the present invention ensures the stability and reliability of the grasping operation, can effectively improve the success rate of grasping mixed reality interactive objects, reduce the operation failure rate, and enhance the user's operation experience. Description of the Drawings

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts.

[0021] Figure 1 It is a flowchart of the method in Embodiment 1 of the present invention; Figure 2 It is a schematic diagram of the composition of the system in Embodiment 2 of the present invention; Figure 3 It is a schematic diagram of the composition of the system in Embodiment 3 of the present invention. Detailed Embodiments

[0022] It should be noted here that the description of these embodiments is for helping to understand the present invention, but does not constitute a limitation to the present invention. The specific structures and functional details disclosed herein are only used to describe the exemplary embodiments of the present invention. However, the present invention can be embodied in many alternative forms and should not be construed as limited to the embodiments described herein.

[0023] It should be understood that, unless otherwise clearly specified and limited, the corresponding terms should be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be directly connected or indirectly connected through an intermediate medium, and can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments can be understood according to specific situations.

[0024] Specific details are provided in the following description to facilitate a complete understanding of the exemplary embodiments. However, those of ordinary skill in the art should understand that the exemplary embodiments can be implemented without these specific details. For example, the device can be shown in a block diagram to avoid obscuring the example with unnecessary details. In other embodiments, well-known processes, structures, and technologies can be shown without unnecessary details to avoid obscuring the embodiments.

[0025] Embodiment 1: This embodiment provides a grasping operation method based on dynamic release distance adjustment, which can be applied to a corresponding MR (Mixed Reality) terminal, such as Figure 1 As shown, the method includes the following steps: S1. Real-time collect the spatial position parameters of each finger of the user.

[0026] In specific implementation, in the MR interaction scenario, the spatial position parameters of each finger of the user can be real-time collected through a corresponding position detection sensor or gesture recognition device, and the spatial position parameters of each finger of the user are transmitted to the MR terminal. The spatial position parameters of each finger are detailed data such as the position, direction, and bending degree of the finger joints / parts. These data can be used to accurately simulate the actions and postures of the fingers.

[0027] S2. When it is determined according to the spatial position parameters of each finger that two of the fingers meet the collision condition with the target object, trigger the grasping operation condition, output a grasping operation instruction for the target object, and use the time point when the grasping operation condition is triggered as the initial time point.

[0028] In specific implementation, the MR terminal can determine whether the user's fingers collide with a virtual target object according to the spatial position parameters of each finger of the user. When the fingertips of two fingers (such as the thumb plus one of the index finger, middle finger, ring finger, and little finger) both enter the set collision range of the target object, it is determined that the two fingers meet the collision condition with the target object, and the grasping operation condition is triggered, and a grasping operation instruction for the target object is output, and the grasping of the target object in the MR scenario is started, and the time point when the grasping operation condition is triggered is used as the initial time point. At the same time, appropriate collision bodies can be set for the fingertips and the target object respectively, and corresponding physical properties such as mass, friction, and elasticity are given to them. When the spatial position parameters reflect that the fingertips are approaching the object, the MR terminal can automatically detect the possibility of collision. Once it is determined that a collision occurs, the MR terminal can calculate the reaction after the collision according to the preset physical rules, such as generating resistance, rebound, or adhesion effects, so as to simulate a real grasping effect. The combination of these effects can make the grasping feel more real and natural, bringing an immersive interaction experience to the user.

[0029] S3. Determine the two fingers of the user for grasping the target object, and use the two fingers of the user for grasping the target object as the grasping operation fingers.

[0030] In specific implementation, the MR terminal determines the two fingers of the user for grasping the target object (such as the thumb plus one of the index finger, middle finger, ring finger, and little finger), and uses the two fingers for grasping the target object as the grasping operation fingers.

[0031] S4. Calculate the real-time fingertip distance between the two grasping operation fingers according to the spatial position parameters of the two grasping operation fingers, and determine the real-time dynamic release distance according to the initial time point.

[0032] In specific implementation, the MR terminal can determine the fingertip spatial position coordinates of the two grasping operation fingers according to the spatial position parameters of the two grasping operation fingers, and then calculate the real-time fingertip distance d between the two grasping fingers based on the fingertip spatial position coordinates of the two grasping operation fingers:

[0033] where (x1, y1, z1) and (x2, y2, z2) are the fingertip spatial position coordinates of the two grasping operation fingers respectively. The real-time fingertip distance d will be used as the basis for subsequent judgment of whether to perform the release operation. However, the traditional judgment method has certain limitations in accuracy and reliability, resulting in the jitter of the user's fingers during the grasping operation may mis-trigger the release operation. To solve this problem, this method introduces a dynamic release distance mechanism: First, an initial maximum spacing value can be set according to specific application scenarios and requirements. The setting of this maximum spacing value can comprehensively consider various factors such as the size of the target object, the difficulty of grasping, and the user's operating habits. For example, if the target object is large or the grasping operation is relatively difficult, the maximum spacing value may be set larger to give the user a certain margin of error. Conversely, if the target object is small or the grasping is relatively easy, the maximum spacing value can be set relatively small to ensure the sensitivity and accuracy of the operation.

[0034] Then, based on the initial time point and the set maximum spacing value, the real-time dynamic release distance can be determined. This dynamic release distance mechanism sets three calculation methods to determine the real-time dynamic release distance: The first is the linear change calculation method:

[0035] Among them, D is the real-time dynamic release distance, D max is the set maximum spacing value, v is the set change rate parameter, and t is the duration from the initial time point to the current time point.

[0036] The second is the exponential decay calculation method:

[0037] Among them, D is the real-time dynamic release distance, D max is the set maximum spacing value, k is the set decay coefficient, and t is the duration from the initial time point to the current time point.

[0038] The third is the piecewise function calculation method:

[0039] Among them, D is the real-time dynamic release distance, D max is the set maximum spacing value, T1 is the set first reference duration from the initial time point, T2 is the set second reference duration from the initial time point, T2 > T1, d is the real-time fingertip spacing of the two grasping fingers at the current time point, and t is the duration from the initial time point to the current time point.

[0040] Finally, after selecting the corresponding calculation method according to actual needs, according to the changes of time and conditions, calculate the real-time dynamic release distance according to the selected calculation method. S5. When it is determined according to the spatial position parameters of the two grasping operation fingers that the two grasping operation fingers do not meet the collision condition with the target object, and it is determined that the real-time fingertip spacing of the two grasping operation fingers exceeds the real-time dynamic release distance, the release operation condition is triggered, and a release operation instruction for the target object is output.

[0041] During specific implementation, when the MR terminal determines that the fingertips of the two grasping operation fingers both leave the set collision range of the target object based on the spatial position parameters of the two grasping operation fingers, it determines that the two grasping operation fingers do not meet the collision condition with the target object. Moreover, when it determines that the real-time fingertip distance between the two grasping operation fingers exceeds the real-time dynamic release distance, it triggers the release operation condition and outputs a release operation instruction for the target object, thereby realizing the release operation during the grasping process of the target object.

[0042] This method can effectively improve the grasping success rate of mixed reality interaction objects, reduce the operation failure rate, and enhance the user's operation experience by real-time monitoring the fingertip distance and dynamically adjusting the release distance to ensure the stability and reliability of the grasping operation.

[0043] Embodiment 2: This embodiment provides a grasping operation system for the grasping operation method based on dynamic release distance adjustment, as Figure 2 shown, including a position acquisition unit, a grasping trigger unit, a finger determination unit, a distance calculation unit, and a release trigger unit, where: The position acquisition unit is used to acquire the spatial position parameters of each finger of the user in real time; The grasping trigger unit is used to trigger the grasping operation condition, output a grasping operation instruction for the target object, and use the time point when the grasping operation condition is triggered as the initial time point when it determines that two of the fingers meet the collision condition with the target object according to the spatial position parameters of each finger; The finger determination unit is used to determine the two fingers by which the user grasps the target object, and use the two fingers by which the user grasps the target object as the grasping operation fingers; The distance calculation unit is used to calculate the real-time fingertip distance between the two grasping fingers according to the spatial position parameters of the two grasping operation fingers, and determine the real-time dynamic release distance according to the initial time point; The release trigger unit is used to trigger the release operation condition and output a release operation instruction for the target object when it determines that the two grasping operation fingers do not meet the collision condition with the target object according to the spatial position parameters of the two grasping operation fingers, and determines that the real-time fingertip distance between the two grasping operation fingers exceeds the real-time dynamic release distance.

[0044] Embodiment 3: This embodiment provides a grasping operation system based on dynamic release distance adjustment, as Figure 3 shown. At the hardware level, it includes: A data interface for establishing data docking between the processor and an external data terminal; A memory for storing instructions; A processor for reading instructions stored in the memory and performing the grasping operation method based on dynamic release distance adjustment in Embodiment 1 according to the instructions.

[0045] Optionally, the system further includes an internal bus through which the processor, the memory, and the data interface can be interconnected. The internal bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0046] The memory can include, but is not limited to, a random access memory (RAM), a read only memory (ROM), a flash memory, a first input first output (FIFO) memory, and / or a first in last out (FILO) memory, etc. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0047] Embodiment 4: This embodiment provides a computer-readable storage medium with instructions stored thereon. When the instructions are run on a computer, the computer is caused to perform the grasping operation method based on dynamic release distance adjustment in Embodiment 1. Among them, the computer-readable storage medium refers to a carrier for storing data, and can include, but is not limited to, a floppy disk, an optical disc, a hard disk, a flash memory, a USB flash drive, and / or a memory stick, etc. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0048] This embodiment also provides a computer program product. When the computer program product runs on a computer, it executes the grasping operation method based on dynamic release distance adjustment in Embodiment 1. Among them, the computer can be a general computer, a special computer, a computer network or other programmable devices.

[0049] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A grasping operation method based on adjusting the dynamic release distance, characterized in that Including: Real-time collect the spatial position parameters of each finger of the user; When it is determined according to the spatial position parameters of each finger that two of the fingers meet the collision condition with the target object, trigger the grasping operation condition, output the grasping operation instruction for the target object, and use the time point when the grasping operation condition is triggered as the initial time point; Determine the two fingers of the user grasping the target object, and use the two fingers of the user grasping the target object as the grasping operation fingers; Calculate the real-time fingertip distance between the two grasping fingers according to the spatial position parameters of the two grasping operation fingers, and determine the real-time dynamic release distance according to the initial time point; When it is determined according to the spatial position parameters of the two grasping operation fingers that the two grasping operation fingers do not meet the collision condition with the target object, and it is determined that the real-time fingertip distance between the two grasping operation fingers exceeds the real-time dynamic release distance, trigger the release operation condition and output the release operation instruction for the target object.

2. The grasping operation method based on dynamic release distance adjustment according to claim 1, characterized in that The determination that two of the fingers meet the collision condition with the target object according to the spatial position parameters of each finger includes: When it is determined according to the spatial position parameters of each finger that the fingertips of two of the fingers both enter the set collision range of the target object, determine that the two fingers meet the collision condition with the target object and trigger the grasping operation condition.

3. The grasping operation method based on dynamic release distance adjustment according to claim 1, characterized in that The calculation of the real-time fingertip distance between the two grasping fingers according to the spatial position parameters of the two grasping operation fingers includes: Determine the fingertip spatial position coordinates of the two grasping operation fingers according to the spatial position parameters of the two grasping operation fingers; Calculate the real-time fingertip distance d between the two grasping fingers based on the fingertip spatial position coordinates of the two grasping operation fingers: where (x1, y1, z1) and (x2, y2, z2) are the fingertip spatial position coordinates of the two grasping operation fingers respectively.

4. The grasping operation method based on dynamic release distance adjustment according to claim 1, characterized in that The determination of the real-time dynamic release distance according to the initial time point includes: Obtain the set maximum distance value, and calculate the real-time dynamic release distance D based on the maximum distance value and the initial time point; Among them, D max is the set maximum value of the spacing, v is the set change rate parameter, and t is the duration from the initial time point to the current time point.

5. The grasping operation method based on dynamic release distance adjustment according to claim 1, wherein, The determination of the real-time dynamic release distance according to the initial time point includes: Obtain the set maximum distance value, and calculate the real-time dynamic release distance D based on the maximum distance value and the initial time point; Among them, D max is the set maximum spacing value, k is the set attenuation coefficient, and t is the duration from the initial time point to the current time point.

6. The grasping operation method based on dynamic release distance adjustment according to claim 1, characterized in that The determination of the real-time dynamic release distance according to the initial time point includes: Obtain the set maximum distance value, and calculate the real-time dynamic release distance D based on the maximum distance value and the initial time point; Among them, D max is the set maximum spacing value, T1 is the set first reference duration starting from the initial time point, T2 is the set second reference duration starting from the initial time point, T2 > T1, d is the real-time fingertip spacing between the two grasping fingers at the current time point, and t is the duration from the initial time point to the current time point.

7. The grasping operation method based on dynamic release distance adjustment according to claim 1, wherein, The determination that the two grasping operation fingers do not meet the collision condition with the target object according to the spatial position parameters of the two grasping operation fingers includes: When it is determined according to the spatial position parameters of the two grasping operation fingers that the fingertips of the two grasping operation fingers both leave the set collision range of the target object, determine that the two grasping operation fingers do not meet the collision condition with the target object.

8. The grasping operation system of the grasping operation method based on dynamic release distance adjustment according to any one of claims 1-7, characterized in that, Including a position acquisition unit, a grasping trigger unit, a finger determination unit, a distance calculation unit, and a release trigger unit, where: The position acquisition unit is used to real-time collect the spatial position parameters of each finger of the user; A grasping trigger unit, configured to trigger a grasping operation condition when it is determined according to the spatial position parameters of each finger that two fingers satisfy the collision condition with the target object, output a grasping operation instruction for the target object, and use the time point when the grasping operation condition is triggered as the initial time point; A finger determination unit, configured to determine two fingers by which the user grasps the target object, and use the two fingers by which the user grasps the target object as the grasping operation fingers; A distance calculation unit, configured to calculate the real-time fingertip distance between the two grasping fingers according to the spatial position parameters of the two grasping operation fingers, and determine the real-time dynamic release distance according to the initial time point; A release trigger unit, configured to trigger a release operation condition when it is determined according to the spatial position parameters of the two grasping operation fingers that the two grasping operation fingers do not satisfy the collision condition with the target object and it is determined that the real-time fingertip distance between the two grasping operation fingers exceeds the real-time dynamic release distance, and output a release operation instruction for the target object.

9. A grasping operation system based on dynamic release distance adjustment, characterized in that, Comprising: A memory, configured to store instructions; A processor, configured to read the instructions stored in the memory and execute the grasping operation method based on dynamic release distance adjustment according to any one of claims 1-7.

10. A computer program product, characterized in that, When the computer program product runs on a computer, it executes the grasping operation method based on dynamic release distance adjustment according to any one of claims 1-7.

Images