Grasping operation method, system and program product based on dynamic release distance adjustment

Abstract

The present invention belongs to the field of mixed reality technology and specifically discloses a grabbing operation method, system, and program product based on dynamic release distance adjustment. The method analyzes the collision conditions of a target object by collecting the spatial position parameters of a user's finger in real time, triggering a grabbing operation when the corresponding grabbing conditions are met. The method then determines whether the grabbing operation finger meets the release conditions based on the calculated time-dynamic release distance and the real-time fingertip spacing of the grabbing operation finger. When the release conditions are met, the release operation is triggered, achieving high recognition accuracy and high reliability in grabbing and releasing mixed reality interactive objects. By monitoring the fingertip distance in real time and dynamically adjusting the release distance to ensure the stability and reliability of the grabbing operation, the present invention can effectively improve the success rate of grabbing mixed reality interactive objects, reduce the failure rate of operations, and enhance the user's operating experience.

Description

Technical Field

[0001] The present invention belongs to the field of mixed reality technology, and in particular relates to a grabbing operation method, system and program product based on dynamic release distance adjustment. Background Art

[0002] Mixed Reality (MR) is a technology that deeply integrates the virtual world with the real environment. It combines Virtual Reality (VR) and Augmented Reality (AR), and allows computer-generated virtual targets to interact with the real world in real time. It can overlay virtual elements into the real environment and enable these virtual objects to have the physical characteristics of the real world (such as real-time physical interaction and dynamic response).

[0003] Mixed reality technology can be applied to areas such as MR gaming, social interaction in the metaverse, virtual surgical training, and virtual design. In these application scenarios, users sometimes need to interact with virtual objects through gestures to perform operations such as grabbing and throwing objects. However, existing MR gesture recognition technology is prone to inaccurate and unstable monitoring due to finger shaking or collisions when users perform grabbing operations, which can lead to object grabbing failures or objects being dropped after grabbing. The instability of the grabbing operation will directly affect the user's immersion and operating experience, especially in scenarios that require delicate operations. Summary of the Invention

[0004] The purpose of the present invention is to provide a grasping operation method, system and program product based on dynamic release distance adjustment to solve the above-mentioned problems existing in the prior art.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, a grasping operation method based on dynamic release distance adjustment is provided, including:

[0007] Collect the spatial position parameters of each finger of the user in real time;

[0008] When it is determined based on the spatial position parameters of each finger that two of the fingers meet the collision condition with the target object, a grasping operation condition is triggered, a grasping operation instruction for the target object is output, and the time point when the grasping operation condition is triggered is used as the initial time point;

[0009] Determine two fingers of the user grasping the target object, and use the two fingers of the user grasping the target object as grasping operation fingers;

[0010] The real-time fingertip distance between the two grasping fingers is calculated based on the spatial position parameters of the two grasping fingers, and the real-time dynamic release distance is determined based on the initial time point;

[0011] When it is determined based on the spatial position parameters of the two grasping operation fingers that the two grasping operation fingers do not meet the collision condition with the target object, and when it is determined that the real-time fingertip distance between the two grasping operation fingers exceeds the real-time dynamic release distance, the release operation condition is triggered and a release operation instruction for the target object is output.

[0012] In one possible design, determining, based on the spatial position parameters of each finger, whether two fingers meet the collision condition with the target object includes:

[0013] When it is determined according to the spatial position parameters of each finger that the fingertips of two fingers enter the set collision range of the target object, it is determined that the two fingers meet the collision condition with the target object, and the grasping operation condition is triggered.

[0014] In one possible design, the calculating of the real-time fingertip distance between the two grasping fingers according to the spatial position parameters of the two grasping fingers includes:

[0015] Determining the fingertip spatial position coordinates of the two grasping operation fingers according to the spatial position parameters of the two grasping operation fingers;

[0016] The real-time fingertip distance d between the two grasping fingers is calculated based on the spatial position coordinates of the fingertips of the two grasping fingers:

[0017]

[0018] Among them, (x1, y1, z1) and (x2, y2, z2) are the spatial position coordinates of the fingertips of the two grasping fingers.

[0019] In one possible design, determining the real-time dynamic release distance according to the initial time point includes:

[0020] Get the set maximum spacing value and calculate the real-time dynamic release distance D based on the maximum spacing value and the initial time point:

[0021]

[0022] Among them, D max is the set maximum interval value, v is the set change rate parameter, and t is the duration from the initial time point to the current time point.

[0023] In one possible design, determining the real-time dynamic release distance according to the initial time point includes:

[0024] Get the set maximum spacing value and calculate the real-time dynamic release distance D based on the maximum spacing value and the initial time point:

[0025]

[0026] Among them, D max is the set maximum interval, k is the set attenuation coefficient, and t is the duration from the initial time point to the current time point.

[0027] In one possible design, determining the real-time dynamic release distance according to the initial time point includes:

[0028] Get the set maximum spacing value and calculate the real-time dynamic release distance D based on the maximum spacing value and the initial time point:

[0029]

[0030] Among them, D max is the set maximum distance, T1 is the set first reference time length starting from the initial time point, T2 is the set second reference time length starting from the initial time point, T2>T1, d is the real-time fingertip distance between the two grasping fingers at the current time point, and t is the duration from the initial time point to the current time point.

[0031] In one possible design, determining, based on the spatial position parameters of the two grasping operation fingers, that both grasping operation fingers do not meet the collision condition with the target object includes:

[0032] When it is determined according to the spatial position parameters of the two grasping operation fingers that the fingertips of the two grasping operation fingers are out of the set collision range of the target object, it is determined that the two grasping operation fingers do not meet the collision condition with the target object.

[0033] In a second aspect, a grasping operation system is provided based on a grasping operation method of dynamic release distance adjustment, including a position acquisition unit, a grasping trigger unit, a finger determination unit, a distance calculation unit, and a release trigger unit, wherein:

[0034] A position acquisition unit, used to acquire the spatial position parameters of each finger of the user in real time;

[0035] a grasping trigger unit, configured to trigger a grasping operation condition when it is determined, based on the spatial position parameters of each finger, that two of the fingers meet a collision condition with a target object, output a grasping operation instruction for the target object, and use the time point when the grasping operation condition is triggered as the initial time point;

[0036] a finger determination unit, configured to determine the two fingers of the user grasping the target object, and use the two fingers of the user grasping the target object as grasping operation fingers;

[0037] a distance calculation unit, configured to calculate the real-time fingertip distance between the two grasping fingers based on the spatial position parameters of the two grasping fingers, and determine the real-time dynamic release distance based on the initial time point;

[0038] The release trigger unit is used to trigger the release operation condition and output the release operation instruction to the target object when it is determined that the two grasping operation fingers do not meet the collision condition with the target object based on the spatial position parameters of the two grasping operation fingers, and when it is determined that the real-time fingertip distance between the two grasping operation fingers exceeds the real-time dynamic release distance.

[0039] Thirdly, a gripping operating system based on dynamic release distance adjustment is provided, including:

[0040] a memory for storing instructions;

[0041] A processor is used to read the instructions stored in the memory and execute any one of the methods described in the first aspect according to the instructions.

[0042] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform any one of the methods described in the first aspect. Furthermore, a computer program product is provided that, when executed on a computer, performs any one of the methods described in the first aspect.

[0043] Beneficial Effects: The present invention analyzes the collision conditions of target objects by collecting the spatial position parameters of the user's fingers in real time, triggering a grabbing operation when the corresponding grabbing conditions are met. It then determines whether the grabbing finger meets the release conditions based on the calculated time-dynamic release distance and the real-time fingertip spacing of the grabbing fingers. When the release conditions are met, the release operation is triggered, achieving high recognition accuracy and high reliability for grabbing and releasing mixed reality interactive objects. By monitoring the fingertip distance in real time and dynamically adjusting the release distance to ensure the stability and reliability of the grabbing operation, the present invention can effectively improve the success rate of grabbing mixed reality interactive objects, reduce the failure rate of operations, and enhance the user's operational experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 is a flow chart of the method in Example 1 of the present invention;

[0046] Figure 2 Schematic diagram of the system structure in Example 2 of the present invention;

[0047] Figure 3 This is a schematic diagram of the system structure in Example 3 of the present invention. DETAILED DESCRIPTION

[0048] It should be noted that the description of these embodiments is intended to help understand the present invention, but does not constitute a limitation of the present invention. The specific structural and functional details disclosed herein are merely intended to describe exemplary embodiments of the present invention. However, the present invention may be embodied in a variety of alternative forms, and should not be construed as being limited to the embodiments set forth herein.

[0049] It should be understood that, unless otherwise expressly specified or limited, the corresponding terms should be understood in a broad sense. For example, "connection" can refer to a fixed connection, a detachable connection, or an integral connection; it can refer to a direct connection, an indirect connection through an intermediate medium, or internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the embodiments based on specific circumstances.

[0050] In the following description, certain details are provided to facilitate a thorough understanding of the example embodiments. However, one skilled in the art will appreciate that the example embodiments may be practiced without these specific details. For example, devices may be shown in block diagrams to avoid obscuring the examples with unnecessary detail. In other embodiments, well-known processes, structures, and techniques may be shown without unnecessary detail to avoid obscuring the embodiments.

[0051] Example 1:

[0052] This embodiment provides a grabbing operation method based on dynamic release distance adjustment, which can be applied to corresponding MR (mixed reality) terminals, such as Figure 1 As shown, the method includes the following steps:

[0053] S1. Collect the spatial position parameters of each finger of the user in real time.

[0054] In specific implementation, in the MR interaction scenario, the spatial position parameters of each finger of the user can be collected in real time through the corresponding position detection sensor or gesture recognition device, and the spatial position parameters of each finger of the user can be transmitted to the MR terminal. The spatial position parameters of each finger, such as the position, direction, bending degree and other detailed data of the finger joints / parts, can be used to accurately simulate the movements and postures of the fingers.

[0055] S2. When it is determined that two of the fingers meet the collision condition with the target object based on the spatial position parameters of each finger, the grasping operation condition is triggered, the grasping operation instruction for the target object is output, and the time point when the grasping operation condition is triggered is used as the initial time point.

[0056] In specific implementations, the MR terminal can determine whether the user's fingers collide with a virtual target object based on the spatial position parameters of each finger. When the tips of two fingers (e.g., the thumb plus one of the index, middle, ring, or pinky fingers) enter the target object's set collision range, the terminal determines that the two fingers meet the collision condition with the target object, triggers a grasping operation condition, outputs a grasping operation instruction for the target object, and begins grasping the target object in the MR scene. The time when the grasping operation condition is triggered is used as the initial time point. Furthermore, appropriate collision bodies can be set for the fingertips and the target object, respectively, and assigned corresponding physical properties such as mass, friction, and elasticity. When the spatial position parameters indicate that the fingertips are close to the object, the MR terminal automatically detects the possibility of a collision. Once a collision is determined, the terminal calculates the post-collision reaction based on preset physical rules, such as generating resistance, rebound, or adhesion effects, thereby simulating a real grasping effect. This combination of effects makes the grasping feel more realistic and natural, providing the user with an immersive interactive experience.

[0057] S3. Determine the two fingers of the user grasping the target object, and use the two fingers of the user grasping the target object as grasping operation fingers.

[0058] During specific implementation, the MR terminal determines the two fingers of the user grasping the target object (such as the thumb plus one of the index finger, middle finger, ring finger, and little finger), and uses the two fingers grasping the target object as the grasping operation fingers.

[0059] S4. Calculate the real-time fingertip distance between the two grasping fingers based on the spatial position parameters of the two grasping operation fingers, and determine the real-time dynamic release distance based on the initial time point.

[0060] In specific implementation, the MR terminal can determine the spatial position coordinates of the fingertips of the two grasping operation fingers according to the spatial position parameters of the two grasping operation fingers, and then calculate the real-time fingertip distance d between the two grasping operation fingers based on the spatial position coordinates of the fingertips of the two grasping operation fingers:

[0061]

[0062] Here, (x1, y1, z1) and (x2, y2, z2) are the spatial coordinates of the fingertips of the two grasping fingers. The real-time fingertip distance d is used as the basis for determining whether to release the finger. However, traditional judgment methods have certain limitations in accuracy and reliability. As a result, even the shaking of the user's finger during the grasping operation may mistakenly trigger the release operation. To solve this problem, this method introduces a dynamic release distance mechanism:

[0063] First, an initial maximum distance can be set based on the specific application scenario and needs. The setting of this maximum distance can take into account multiple factors such as the size of the target object, the difficulty of grasping, and the user's operating habits. For example, if the target object is large or the grasping operation is relatively difficult, then the maximum distance may be set larger to give the user a certain amount of tolerance. On the contrary, if the target object is small or the grasping operation is relatively easy, the maximum distance can be set relatively smaller to ensure the sensitivity and accuracy of the operation.

[0064] Then, the real-time dynamic release distance can be determined based on the initial time point and the set maximum spacing. The dynamic release distance mechanism sets three calculation methods to determine the real-time dynamic release distance:

[0065] The first is the linear change calculation method:

[0066]

[0067] Among them, D is the real-time dynamic release distance, D max is the set maximum interval value, v is the set change rate parameter, and t is the duration from the initial time point to the current time point.

[0068] The second is the exponential decay calculation method:

[0069]

[0070] Among them, D is the real-time dynamic release distance, D max is the set maximum interval, k is the set attenuation coefficient, and t is the duration from the initial time point to the current time point.

[0071] The third method is piecewise function calculation:

[0072]

[0073] Among them, D is the real-time dynamic release distance, D maxis the set maximum distance, T1 is the set first reference time length starting from the initial time point, T2 is the set second reference time length starting from the initial time point, T2>T1, d is the real-time fingertip distance between the two grasping fingers at the current time point, and t is the duration from the initial time point to the current time point.

[0074] Finally, after selecting the corresponding calculation method according to actual needs, the real-time dynamic release distance is calculated according to the selected calculation method based on changes in time and conditions.

[0075] S5. When it is determined that the two grasping operation fingers do not meet the collision condition with the target object based on the spatial position parameters of the two grasping operation fingers, and when it is determined that the real-time fingertip distance between the two grasping operation fingers exceeds the real-time dynamic release distance, the release operation condition is triggered and a release operation instruction for the target object is output.

[0076] During specific implementation, when the MR terminal determines that the fingertips of the two grasping operation fingers have left the set collision range of the target object based on the spatial position parameters of the two grasping operation fingers, it determines that the two grasping operation fingers do not meet the collision conditions with the target object. Moreover, when it is determined that the real-time fingertip distance between the two grasping operation fingers exceeds the real-time dynamic release distance, the release operation condition is triggered, and a release operation instruction for the target object is output to realize the release operation during the grasping process of the target object.

[0077] This method monitors the fingertip distance in real time and dynamically adjusts the release distance to ensure the stability and reliability of the grasping operation, which can effectively improve the success rate of grasping mixed reality interactive objects, reduce the operation failure rate, and enhance the user's operation experience.

[0078] Example 2:

[0079] This embodiment provides a grabbing operation system based on a grabbing operation method with dynamic release distance adjustment, such as Figure 2 As shown, it includes a position acquisition unit, a grasping trigger unit, a finger determination unit, a distance calculation unit and a release trigger unit, wherein:

[0080] A position acquisition unit, used to acquire the spatial position parameters of each finger of the user in real time;

[0081] a grasping trigger unit, configured to trigger a grasping operation condition when it is determined, based on the spatial position parameters of each finger, that two of the fingers meet a collision condition with a target object, output a grasping operation instruction for the target object, and use the time point when the grasping operation condition is triggered as the initial time point;

[0082] a finger determination unit, configured to determine the two fingers of the user grasping the target object, and use the two fingers of the user grasping the target object as grasping operation fingers;

[0083] a distance calculation unit, configured to calculate the real-time fingertip distance between the two grasping fingers based on the spatial position parameters of the two grasping fingers, and determine the real-time dynamic release distance based on the initial time point;

[0084] The release trigger unit is used to trigger the release operation condition and output the release operation instruction to the target object when it is determined that the two grasping operation fingers do not meet the collision condition with the target object based on the spatial position parameters of the two grasping operation fingers, and when it is determined that the real-time fingertip distance between the two grasping operation fingers exceeds the real-time dynamic release distance.

[0085] Example 3:

[0086] This embodiment provides a gripping operation system based on dynamic release distance adjustment, such as Figure 3 As shown, at the hardware level, it includes:

[0087] Data interface, used to establish data connection between the processor and the external data terminal;

[0088] a memory for storing instructions;

[0089] The processor is configured to read the instructions stored in the memory and execute the grasping operation method based on dynamic release distance adjustment in Example 1 according to the instructions.

[0090] Optionally, the system further includes an internal bus, through which the processor, memory, and data interface can be interconnected. The internal bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0091] The memory may include, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, first-in-first-out memory (FIFO) and / or first-in-last-out memory (FILO), etc. The processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0092] Example 4:

[0093] This embodiment provides a computer-readable storage medium having instructions stored thereon. When the instructions are executed on a computer, the computer is caused to execute the grasping operation method based on dynamic release distance adjustment in Embodiment 1. The computer-readable storage medium refers to a data storage medium, which may include, but is not limited to, a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash drive, and / or a memory stick. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable device.

[0094] This embodiment further provides a computer program product, which, when executed on a computer, executes the grasping operation method based on dynamic release distance adjustment in embodiment 1. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0095] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A grasping operation method based on dynamic release distance adjustment, characterized in that: include: Collect the spatial position parameters of each finger of the user in real time; When it is determined based on the spatial position parameters of each finger that two of the fingers meet the collision condition with the target object, a grasping operation condition is triggered, a grasping operation instruction for the target object is output, and the time point when the grasping operation condition is triggered is used as the initial time point; Determine two fingers of the user grasping the target object, and use the two fingers of the user grasping the target object as grasping operation fingers; The real-time fingertip spacing between the two grasping fingers is calculated based on the spatial position parameters of the two grasping fingers, and the real-time dynamic release distance is determined based on the initial time point, including: Get the set maximum spacing value and calculate the real-time dynamic release distance D based on the maximum spacing value and the initial time point: Among them, D max is the set maximum interval value, v is the set change rate parameter, and t is the duration from the initial time point to the current time point; or, Among them, D max is the set maximum interval, k is the set attenuation coefficient, and t is the duration from the initial time point to the current time point; or, Among them, D max is the set maximum distance, T1 is the set first reference duration from the initial time point, T2 is the set second reference duration from the initial time point, T2>T1, d is the real-time fingertip distance between the two grasping fingers at the current time point, and t is the duration from the initial time point to the current time point; When it is determined based on the spatial position parameters of the two grasping operation fingers that the two grasping operation fingers do not meet the collision condition with the target object, and when it is determined that the real-time fingertip distance between the two grasping operation fingers exceeds the real-time dynamic release distance, the release operation condition is triggered and a release operation instruction for the target object is output.

2. The grasping operation method based on dynamic release distance adjustment according to claim 1, characterized in that: The determining, based on the spatial position parameters of the fingers, whether two of the fingers meet the collision condition with the target object includes: When it is determined according to the spatial position parameters of each finger that the fingertips of two fingers enter the set collision range of the target object, it is determined that the two fingers meet the collision condition with the target object, and the grasping operation condition is triggered.

3. The grasping operation method based on dynamic release distance adjustment according to claim 1, characterized in that: The calculating of the real-time fingertip distance between the two grasping fingers according to the spatial position parameters of the two grasping operation fingers includes: Determining the fingertip spatial position coordinates of the two grasping operation fingers according to the spatial position parameters of the two grasping operation fingers; The real-time fingertip distance d between the two grasping fingers is calculated based on the spatial position coordinates of the fingertips of the two grasping fingers: Among them, (x1, y1, z1) and (x2, y2, z2) are the spatial position coordinates of the fingertips of the two grasping fingers.

4. The grasping operation method based on dynamic release distance adjustment according to claim 1, characterized in that: The determining, based on the spatial position parameters of the two grasping operation fingers, that both grasping operation fingers do not meet the collision condition with the target object includes: When it is determined according to the spatial position parameters of the two grasping operation fingers that the fingertips of the two grasping operation fingers are out of the set collision range of the target object, it is determined that the two grasping operation fingers do not meet the collision condition with the target object.

5. The grasping operation system according to any one of claims 1 to 4, characterized in that: It includes a position acquisition unit, a grasping trigger unit, a finger determination unit, a distance calculation unit and a release trigger unit, wherein: A position acquisition unit, used to acquire the spatial position parameters of each finger of the user in real time; a grasping trigger unit, configured to trigger a grasping operation condition when it is determined, based on the spatial position parameters of each finger, that two of the fingers meet a collision condition with a target object, output a grasping operation instruction for the target object, and use the time point when the grasping operation condition is triggered as the initial time point; a finger determination unit, configured to determine the two fingers of the user grasping the target object, and use the two fingers of the user grasping the target object as grasping operation fingers; The distance calculation unit is used to calculate the real-time fingertip distance between the two grasping fingers based on the spatial position parameters of the two grasping fingers, and determine the real-time dynamic release distance based on the initial time point, including: Get the set maximum spacing value and calculate the real-time dynamic release distance D based on the maximum spacing value and the initial time point: Among them, D max is the set maximum interval value, v is the set change rate parameter, and t is the duration from the initial time point to the current time point; or, Among them, D max is the set maximum interval, k is the set attenuation coefficient, and t is the duration from the initial time point to the current time point; or, Among them, D max is the set maximum distance, T1 is the set first reference duration from the initial time point, T2 is the set second reference duration from the initial time point, T2>T1, d is the real-time fingertip distance between the two grasping fingers at the current time point, and t is the duration from the initial time point to the current time point; The release trigger unit is used to trigger the release operation condition and output the release operation instruction to the target object when it is determined that the two grasping operation fingers do not meet the collision condition with the target object based on the spatial position parameters of the two grasping operation fingers, and when it is determined that the real-time fingertip distance between the two grasping operation fingers exceeds the real-time dynamic release distance.

6. A gripping operating system based on dynamic release distance adjustment, characterized in that: include: a memory for storing instructions; A processor is configured to read the instructions stored in the memory and execute the grasping operation method based on dynamic release distance adjustment according to any one of claims 1 to 4 according to the instructions.

7. A computer program product, characterized in that When the computer program product is run on a computer, the grasping operation method based on dynamic release distance adjustment according to any one of claims 1 to 4 is executed.

Images