Virtual Reality Controller

Abstract

The present concept relates to a virtual reality controller that enables fine control of virtual objects using natural movements involving the dexterity of a user's fingers and provides realistic tactile sensations to the user's fingers. The controller can have a rigid structural design with no moving parts. Force sensors under the finger rests can detect the forces applied by the user's fingers. Actuators can render tactile feedback from the virtual reality world to the user's fingers. The controller can include one or more trackpads on which the user can slide her fingers. The controller can be used to explore and manipulate virtual objects, for example, by grasping, releasing, rotating, and feeling the surface of the virtual objects.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

[0001] The accompanying drawings illustrate implementations of the present concepts. The features of the illustrated implementations may be more readily understood by reference to the following description in conjunction with the accompanying drawings. Where feasible, the same reference numerals are used in the various drawings to indicate the same elements. In some cases, parentheses are used after the reference numerals to distinguish similar elements. The use of reference numerals without associated parentheses is common to the elements. The accompanying drawings are not necessarily drawn to scale. In the drawings, the leftmost digit of a reference numeral identifies the figure in which the reference numeral first appears. The use of similar reference numerals in different instances in the specification and drawings may identify similar or identical items.

[0002] Figure 1 A virtual reality system consistent with some implementations of the present concepts is shown.

[0003] Figure 2A and Figure 2B Different perspectives of a controller consistent with some implementations of the present concepts are shown.

[0004] FIG. 3A to FIG. 3E A schematic diagram showing components of a controller consistent with some implementations of the present concepts is shown.

[0005] Figure 4A and Figure 4B A block diagram showing components of a controller consistent with some implementations of the present concepts is shown.

[0006] Figure 5 A block diagram illustrating an example configuration of a virtual reality system consistent with some implementations of the present concepts.

[0007] FIG. 6A to FIG. 6E Virtual rendering of a manipulated virtual object is shown consistent with some implementations of the present concepts.

[0008] Figure 7 A flow chart illustrating a process involving grasping a virtual object using a controller consistent with some implementations of the present concepts is shown.

[0009] Figure 8 A flow chart is shown that involves a process of releasing a virtual object using a controller consistent with some implementations of the present concepts.

[0010] Fig. 9 A flow chart is shown of a process involving sliding a virtual finger across a virtual object surface using a controller consistent with some implementations of the present concepts.

[0011] Fig.10 A flow chart is shown that involves a process of rotating a virtual object using a controller consistent with some implementations of the present concepts. DETAILED DESCRIPTION

[0012] The present concept relates to a controller that can be used to simulate natural motion in a virtual reality environment (including augmented reality and mixed reality). The controller can be a handheld rigid natural user interface (NUI) that can be used to manipulate virtual objects using fine-grained dexterity manipulation. The controller can also provide a versatile tactile experience.

[0013] Traditional virtual reality controllers vary greatly in design, but have several disadvantages. Typical commercial controllers commonly used with video game systems are bulky handheld controllers with joysticks, buttons, and / or triggers. However, these controllers do not mimic natural motion. In addition, the tactile sensation created by the vibrotactile buzz of the entire controller is not exciting. These controllers fail to generate a compelling form of tactile feedback including grip, compliance, and tactile sensation.

[0014] Other conventional controllers that mimic natural hand movements may have a relatively simple design and may be held in the user's hand. Such controllers may be held using the user's palm (often referred to as a power grip) and only allow manipulation of virtual objects through arm and wrist movements. They may not allow the user to use the dexterity of her fingers for fine manipulation of virtual objects as in real life.

[0015] Other conventional controllers include finger-worn tactile interfaces. Such controllers can enable precise touch manipulation and provide a realistic dexterity experience derived from pressure-sensitive stimulation of the skin at the fingertips. However, they have limited forceful input because they do not have a structure that enables the user's hand to hold firmly.

[0016] Conventional glove-type controllers can enable both strong grip and high-precision manipulation. Such controllers require tactile rigidity to simulate a strong grip or touch of a virtual object. Therefore, they may include many electromagnetic motors and brakes to provide a realistic feel. Therefore, such controllers are very complex, which increases cost and reduces reliability. These controllers may have a series of flexible and active parts, as well as many expensive components such as electromagnetic motors and gears. Compared with other controllers that can be easily and quickly picked up, putting them on is also difficult and time-consuming.

[0017] To address the issues associated with conventional virtual reality controllers, the present concept relates to a multi-purpose haptic controller that can be stably held using a power grip while enabling fine control of virtual objects using natural movements. The controller can also provide convincingly realistic skin-level sensations. The controller can have a rigid design that enables efficient and cost-effective mass manufacturing.

[0018] The controller can sense finger motion and / or force, and thereby enable the user to perform natural and dexterous finger-based interactions and exploration of virtual objects. In some implementations, the controller may include a touchpad (also referred to as a touchpad), and the user can explore and interact with virtual objects by freely sliding her thumb or other fingers on a plane. Compared to traditional virtual reality controllers, controllers according to the present concept can enable more precise manipulation of virtual objects with human-scale forces. For example, a user can use a tripod handle with a thumb and two other fingers to grasp a virtual object. The user can manipulate the virtual object deftly, such as by feeling the surface of the virtual object or rotating the virtual object. The user can also release her fingers and let go of the virtual object.

[0019] The controller can also output compelling tactile sensations to the user's fingers, providing a realistic virtual reality experience. For example, the controller can generate tactile sensations that represent squeezing, shearing, or rotating an object. Despite its rigid design, the controller can render a wide range of compelling tactile feedback, including the compliance of the virtual object material and the texture of the virtual object surface. The controller can generate compelling tactile sensations without relying on human-scale forces actuated by multiple motors along multiple degrees of freedom.

[0020] Figure 1 A virtual reality system 100 consistent with some implementations of the present concepts is shown. Virtual reality system 100 may include base station 102. Base station 102 may include hardware and / or software for generating and executing a virtual reality world, including receiving and processing input from user 104, and generating and outputting feedback to user 104. Base station 102 may be any computing device, including a personal computer (PC), server, game console, smart phone, tablet, laptop, car, simulator, etc.

[0021] In some implementations, the virtual reality system 100 may include a headset 106. The headset 106 may be, for example, a head mounted display (HMD) that receives visual information from the virtual reality world executed by the base station 102 and includes a display for displaying the visual information to the user 104. The user 104 may wear the headset 106 on her head. The headset 106 may also receive auditory information associated with the virtual reality world from the base station 102 and include a speaker to render the auditory information to the user 104. In some implementations, the headset 106 may include one or more sensors ( Figure 1The head mounted device 106 may be configured to provide input to the base station 102. The sensors may include, for example, buttons, accelerometers, gyroscopes, cameras, microphones, etc. Thus, the head mounted device 106 may be capable of detecting objects around the user, the position of the user's head, the direction the user's head is facing, whether the user's eyes are open or closed, which direction the user's eyes are looking at, etc.

[0022] Virtual reality system 100 may also include controller 108. Consistent with the present concept, controller 108 may be a handheld device that user 104 may hold in her hand 110 and manipulate with her fingers to provide input to base station 102. Controller 108 may include sensor 112 capable of detecting finger motion and / or force. Example sensor 112 incorporating FIG. 3A to FIG. 3E In addition, the controller 108 may receive haptic information from the virtual reality world executed by the base station 102 and may render the haptic information to the user's hand 110 via the vibrator 114. Example vibrator 114 in combination with Figure 3A-3E Be introduced.

[0023] Combination Figure 1 The described virtual reality system 100 is only one example. Various configurations of the virtual reality system 100 are possible. The base station 102 can communicate with the head-mounted device 106 and / or the controller 108 via wired and / or wireless means. In some implementations, the base station 102 can be included in the head-mounted device 106 or the controller 108. The display can be a television, a monitor, a screen, a projector, goggles, glasses, lenses, etc. The speakers can be external independent speakers, built-in speakers of a computing device, headphones, earphones, etc. In addition, although the controller 108 is Figure 1 It is shown as a standalone device, but it can be an addition to another peripheral device.

[0024] Figure 2A and Figure 2B Different perspective views of the controller 108 are shown consistent with some implementations of the present concepts. Figure 2A A front view (i.e., from a fingertip perspective) of the controller 108 is shown, and Figure 2B A left side view (ie, from the thumb side) of the controller 108 is shown. These figures will be described together.

[0025] The controller 108 may include a body 202 that provides a structure to which the various parts of the controller 108 may be attached and mounted. The body 202 may also store the various parts of the controller 108 therein. Optionally, the controller 108 may include a handle 204 that allows the user 104 to hold the controller 108 in the user's hand 110. The example controller 108 shown and described herein may be designed to be held by the user's right hand 110, but alternative controllers may be designed to be held by the user's left hand, or ambidextrous controllers may be designed. Although Figure 2A and Figure 2B The controller 108 shown in FIG. 1 has a bar-shaped body 202 and a handle 204 that can be grasped by a ring finger 206 (4) and a pinky finger 206 (5), but other design configurations are possible. For example, the controller 108 may include a body with a convex shape similar to the surface of a ball that can fit in the palm of the hand, and may include a handle in the form of finger holes similar to a bowling ball. In some implementations, the body 202 and the handle 204 may be one and the same, or have an overlapping structure.

[0026] The controller 108 may include one or more finger rests 208 on which the user 104 may place her fingers 206. For example, the controller 108 may be operated to provide input using only one finger 206 or up to all five fingers 206. Figure 2A and Figure 2B In the example implementation shown, the controller 108 can be operated to provide input using the thumb 206(1), the index finger 206(2), and / or the middle finger 206(3). Accordingly, the controller 108 includes a thumb rest 208(1), an index finger rest 208(2), and a middle finger rest 208(3) rigidly attached to the body 202.

[0027] According to some implementations, such as Figure 2A (front view), the finger rests 208 can be positioned on the left and right sides, facing away from each other, so that the user 104 can place her fingers 206 on the outside of the finger rests 208 and can squeeze the finger rests 208 inwardly together (e.g., squeeze the thumb and other fingers together). Figure 3A-3E As will be described in greater detail, the finger rest 208 may include a sensor 112 ( Figure 2A and Figure 2B ), and includes a vibrator 114 (not shown) for providing sensation to a user's finger 206. Figure 2A and Figure 2B1 ). The finger rests 208 on opposite sides may be parallel to each other or at an angle (e.g., 30 degrees apart but still substantially facing away from each other). The finger rests 208 on the controller 108 may be used by the user 104 to provide input to the base station 102 to close the virtual fingers, e.g., to grasp a virtual object.

[0028] Optionally, in some implementations, one or more finger rests 208 may be attached to a limiter 210. The limiter 210 attached to the finger rest 208 may form a ring (a fully or partially closed ring) around the finger 206 of the user 104, so that the user 104 can pull her finger 206 away from the finger rest 208 and apply a negative force away from the finger rest 208. The limiter 210 on the controller 108 may be used to provide input by the user 104 to the base station 102 to cause the virtual fingers to open, for example, to release a virtual object. Each finger rest 208 may include a limiter 210. Or a subset of the finger rests 208 may include a limiter 210. Two or more finger rests 208 may share a limiter 210.

[0029] Consistent with the present concepts, the finger rests 208 may be rigid, non-compliant structures that do not substantially move relative to the body 202 and each other when the user 104 operates the controller 108 (e.g., by squeezing the finger rests 208 together or pulling the finger rests 208 apart using the limiters 210). Although the finger rests 208 may move a very small amount (e.g., less than 50 microns) detectable by the sensor 112, the finger rests 208 may be perceived as rigid by the user 104. Alternatively, the finger rests 208 may be movable components, but their movement may not necessarily provide input to the base station 102. For example, the position and / or angle of the finger rests 208 may be adjustable to comfortably fit the user's individual hands 110 and fingers 206.

[0030] FIG. 3A to FIG. 3E Schematic diagrams showing components of an example controller 108 according to various implementations of the present concepts. These figures will be described together. FIG. 3A to FIG. 3E The controller 108 shown in FIG. 1 may be configured to be operated by the thumb 206 ( 1 ), the index finger 206 ( 2 ), and / or the middle finger 206 ( 3 ). Therefore, the corresponding description will be consistent with the present implementation. However, many variations of the configuration of the controller 108 are possible. FIG. 3A to FIG. 3E The controller 108 is shown from different perspectives and various configurations of the controller 108 are also shown. Figure 3A A rear view of controller 108, ie, from the wrist end, is shown. Figure 3B A left side view of the controller 108, ie, from the thumb side, is shown. Figure 3C and Figure 3EA right side view of the controller 108 is shown, ie, from the index and middle finger side. Figure 3D A front view of controller 108 is shown, ie, as viewed from a fingertip.

[0031] The controller 108 includes a body 202, a portion of which is Figure 3A-3E In some implementations, the body 202 can include a support structure portion that protrudes from a main portion of the body 202, such that the support structure portion provides a framework by which other portions of the controller 108 can be attached to the body 202 of the controller 108.

[0032] exist Figure 3A-3C In the illustrated configuration, the controller 108 may include a thumb rest 208(1) on one side of the body 202, and may include an index finger rest 208(2) and a middle finger rest 208(3) on the other side of the body 202. The thumb rest 208(1) may face away from the index finger rest 208(2) and the middle finger rest 208(3) so that the user 104 may use her thumb 206(1), index finger 206(2), and middle finger 206(3) to squeeze the finger rests 208 toward each other (i.e., toward the body 202 of the controller 108).

[0033] exist Figure 3D and Figure 3E In the illustrated alternative configuration, rather than having a separate index finger rest 208(2) and a separate middle finger rest 208(3), the controller 108 may include a shared finger rest 208(4) shared by (i.e., operated by) the index finger 206(2) and / or the middle finger 206(3).

[0034] Finger rest 208 may be rigidly attached to body 202, either directly or indirectly through other portions of controller 108. Thus, finger rest 208 may not substantially move when finger rest 208 is squeezed toward body 202 by a user's finger 206. In other words, the controller does not rely on movement of portions of controller 108 to achieve the described functionality.

[0035] In some configurations, the limiter 210 can be attached to the finger rest 208 so that the user 104 can apply a negative force on the finger rest 208 away from the body 202. Figure 3DAs shown, the index finger limiter 210(2) and the middle finger limiter 210(3) can form two separate loops for the index finger 206(2) and the middle finger 206(3), respectively. Alternatively, one limiter (not shown) can form a larger loop for both the index finger 206(2) and the middle finger 206(3). The limiter 210 can be elastic (e.g., rubber) and / or removable (e.g., Velcro strips). Alternatively, the limiter 210 can be rigid (e.g., plastic). The limiter 210 can be firmly formed so that the user's finger 206 is always in contact with the finger rest 208, or the limiter 210 can be loosely formed so that the user's finger 206 can be lifted off the finger rest 208 even when the user's finger 206 is surrounded by the limiter 210.

[0036] Consistent with some implementations of the present concepts, the controller 108 may include various sensors 112 capable of sensing information about the user's finger 206, including, for example, the position of the user's finger 206 and the amount of force (or pressure) applied by the user's finger 206. For example, the controller 108 may include one or more force sensors 302. The force sensor 302 may be positioned below the finger rest 208 to detect and measure the amount of force applied by the user's finger 206 on the finger rest 208. In one implementation, the force sensor 302 is capable of sensing up to 1.5 kilograms of force (kgf). The force sensor 302 may be configured to push the finger rest 208, thereby increasing the stiffness of the finger rest 208. Thus, for the entire range of motion of the force sensor 302 (e.g., 30 microns), the force sensing range may be approximately doubled to 3 kgf. In some configurations, one force sensor 302 may detect the force applied by one finger 206. In an alternative configuration, multiple force sensors 302 can be mounted below each finger rest 208, which can enable additional ways to manipulate virtual objects, such as using a virtual thumb and two other virtual fingers to shake a virtual pencil up and down. In some other configurations, one force sensor 302 can detect the force applied by multiple fingers 206.

[0037] like Figure 3A-3C As shown, the controller 108 may include a thumb force sensor 302(1) below the thumb rest 208(1), an index finger force sensor 302(2) below the index finger rest 208(2), and a middle finger force sensor 302(3) below the middle finger rest 208(3). Figure 3D and Figure 3E As shown, a shared force sensor 302 ( 4 ) may be mounted beneath the shared finger rest 208 ( 4 ) to measure the level of force applied by the index finger 206 ( 2 ) and / or the middle finger 206 ( 3 ).

[0038] In some implementations, the force sensor 302 can be biased toward the body 202, i.e., in the direction in which the user's finger 206 will exert force on the finger rest 208. For example, when no pressure is applied to the finger rest 208, the force sensor 302 can be adjusted using one or more set screws 304 (e.g., Figure 3A and Figure 3D The force sensors 302 may be mechanically biased to be placed approximately 10% into their force sensing range (as shown). In one implementation, the force sensor 302 may have a force sensing range from 0 Newtons (N) to 15N, and when no force is applied to it, it may be biased to approximately 1.5N. With the additional stiffness of the finger rest 208 pushing the force sensor 302, the force sensing range may be 0N to 30N, and the force sensor 302 may be biased to approximately 3N. Thus, the force sensor 302 is not only able to detect the force applied by squeezing the finger rest 208 toward the body 202, but also the negative force applied when the finger rest 208 is pulled away from the body 202 by extending the user's finger 206 using the limiter 210.

[0039] In accordance with some implementations, the controller 108 may include a position sensor capable of detecting the position of the user's finger 206. For example, the controller 108 may include a touch pad 308 (or touch pad). The touch pad 308 may be mounted on top of the finger rest 208 or may be mounted in place of the finger rest 208. The touch pad 308 may detect the position of the finger 206 and thereby track the sliding movement of the finger 206 on the touch pad 308. For example, the touch pad 308 may be able to detect approximately 130×130 different positions. The touch pad 308 on the controller 108 may be used by the user 104 to provide input to the base station 102 to move the virtual finger. In some implementations, the touch pad 308 may be a copper pad based on a two-dimensional capacitor. In addition, the touch pad 308 may be composed of multiple (e.g., 3×3) pads assembled together. In some implementations, a thin acetal sheet may be added on top of the touch pad 308 to minimize friction with the user's finger 206.

[0040] exist Figure 3D In the illustrated configuration, the controller 108 may include a thumb trackpad 308(1) for detecting a touch by a user's thumb 206(1). The trackpad 308 may be rigidly attached to the body 202, either directly or indirectly through other portions of the controller 108. For example, the thumb trackpad 308(1) may be mounted on the thumb rest 208(1). In some other implementations, the thumb trackpad 308(1) may be mounted in place of the thumb rest 208(1), and thus function as the thumb rest 208(1).

[0041] Furthermore, in addition to detecting the position of the finger 206 on the touch pad 308, the touch pad 308 can also measure the amount of force applied by the finger 206 on the touch pad 308. In the case where the touch pad 308 has this additional capability, the force sensor 302 can be omitted below the touch pad 308. For example, a thumb force sensor ( Figure 3D A thumb force sensor (not shown) may be mounted below the thumb trackpad 308(1). However, a thumb force sensor may be unnecessary and may be omitted, where the thumb trackpad 308(1) is capable of measuring not only the position of the user's thumb 206(1) on the thumb trackpad 308(1) but also the force applied by the user's thumb 206(1) on the thumb trackpad 308(1).

[0042] The controller 108 may include various other sensors 112 for detecting the user's finger 206. For example, in an alternative implementation, a mechanical sensor (such as a joystick) may be installed in place of a capacitive sensor (such as a touch pad 308). In other implementations, the limiter 210 may include a pressure sensor, such as a capacitive sensor (not shown), facing the back of the user's finger 206 so that the pressure sensor can detect when the finger 206 is lifted off the finger rest 208 and is touching the pressure sensor.

[0043] Consistent with some implementations of the present concept, the controller 108 may include one or more vibrators 114, such as an actuator 306. For example, the actuator 306 may be a voice coil actuator (VCA) that can provide broadband vibrotactile stimulation relative to the inertial mass of the VCA. (Although the controller 108 has been described as not including moving parts, the actuator 306 moves or vibrates very slightly.) The actuator 306 can play sounds that provide tactile feedback (not sounds that humans can hear, but vibrations within the audio spectrum). For example, the VCA may include a voice coil with a diameter of 9 mm and be capable of generating a sound pressure level (dbSPL) of 55 to 75 decibels. The actuator 306 can therefore provide kinesthetic perceptions including force and proprioception. The vibrator 114 can be any other kind of tactile output mechanism.

[0044] The actuator 306 may be positioned below the finger rest 208 and / or the touch pad 308 to provide a tactile sensation to the user's finger 206 touching the finger rest 208 and / or the touch pad 308. Figure 3A-3C In the illustrated configuration, the controller 108 may include a thumb actuator 306(1) below the thumb rest 208(1), an index finger actuator 306(2) below the index finger rest 208(2), and a middle finger actuator 306(3) below the middle finger rest 208(3) to provide tactile sensations to the user's fingers 206. Figure 3D and Figure 3E In the configuration shown, thumb actuator 306(1) can be positioned below thumb trackpad 308(1) to provide tactile sensations to the user's thumb 206(1), and shared finger actuator 306(4) can be positioned below shared finger rest 208(4) to provide tactile sensations to index finger 206(2) and middle finger 206(3). In some implementations, a combination of multiple actuators 306 can be used to create an illusion of inertia and virtual motion between actuators 306. Actuators 306 can provide a variety of tactile feedback while keeping the cost of controller 108 relatively inexpensive.

[0045] Figure 4A and Figure 4B A block diagram of components of the controller 108 consistent with some implementations of the present concepts is shown. Figure 4A The block diagram in Figure 3A-3C The configuration of the controller 108 shown in FIG. Figure 4B The block diagram in Figure 3D and Figure 3E The configuration of the controller 108 shown in FIG. 1 corresponds to that in FIG.

[0046] Consistent with some implementations of the present concepts, the controller 108 may include a processor, such as a microcontroller 402. The microcontroller 402 may have one or more pins 404 for receiving input signals and sending output signals. Consistent with the present concepts, the microcontroller 402 may receive signals from the force sensor 302. The signal from the force sensor 302 may include a measurement of the level of force applied to the corresponding finger rest 208. In some implementations, the signal from the force sensor 302 may be amplified by an amplifier 406 and then routed to an analog-to-digital conversion (ADC) pin 404 of the microcontroller 402.

[0047] In some implementations, the microcontroller 402 can receive signals from one or more trackpads 308. For example, where the trackpad 308 consists of nine (3x3) pads, the nine pads can be wired to nine pins 404 of the microcontroller 402. The signals from the trackpad 308 can include the location of the finger 206 touching the trackpad 308. The signals can also include a measurement of the force applied by the finger 206 on the trackpad 308.

[0048] Consistent with the present concepts, the microcontroller 402 may generate input signals based on signals received from one or more force sensors 302 (or from the amplifier 406) and / or signals received from one or more touch pads 308. The microcontroller 402 may send the input signals to the base station 102. In some implementations, the input signals may include raw signal outputs from sensors (e.g., force sensors 302 and touch pads 308), such as x- and y-coordinate positions provided by the touch pad 308 indicating the sensed position of the user's finger 206, capacitance levels provided by the touch pad 308 depending on how hard the user's finger 206 presses on the touch pad 308, and / or force levels provided by the force sensors 302 depending on how hard the user's finger 206 presses or pulls off the finger rest 208. In alternative implementations, the input signals may include interpreted data signals that conform to a protocol understood by the controller 108 and the base station 102, and convey logical representations of the motion and force of the user's finger.

[0049] Consistent with the present concept, the microcontroller 402 can receive an output signal from the base station 102. The microcontroller 402 can generate a control signal based on the output signal received from the base station 102, and can send the control signal via a pulse width modulation (PWM) pin 404 to drive the actuator 306. The control signal from the microcontroller 402 can be amplified by the motor driver 408, for example, using a full bridge with an external drive voltage. In some implementations, the output signal can include signal parameters (e.g., amplitude, frequency, and duration) of a tactile vibration to be rendered by one or more actuators 306 in the controller 108. Alternatively, the output signal can include a logical command understood by the controller 108 (e.g., via a lookup table stored in the microcontroller 402 that maps the command to a certain way of driving the actuator 306).

[0050] Combination Figure 4A and Figure 4B The components of the controller 108 described are merely examples and are not exhaustive. The controller 108 may include many other components. For example, the controller 108 may include an internal power source (such as a battery) or an external power source. The controller 108 may include a wireless transceiver, buttons, display lights, etc.

[0051] Figure 5 A block diagram of an example configuration of a virtual reality system 100 consistent with some implementations of the present concepts is shown. The virtual reality system 100 may include a base station 102 that communicates with one or more peripheral devices 502 via wired and / or wireless means. For example, the base station 102 may communicate with the peripheral devices 502 via a network 504 via a wired and / or wireless protocol. Figure 5In the example shown, the base station 102 can be a server computing device. The number and device types described and depicted are intended to be illustrative and not restrictive. The base station 102 may include other types of computing devices, such as personal computers, desktop computers, notebook computers, mobile phones, smart phones, personal digital assistants, tablet computers, mobile computers, wearable devices, cameras, electrical appliances, smart devices, Internet of Things devices, vehicles, etc., and / or any of countless types of computing devices that are constantly developing or have not yet been developed. The terms "device", "computer" or "computing device" used herein may mean any type of device with a certain amount of processing power and / or storage capacity. Processing power can be provided by one or more hardware processors 508, which can execute data in the form of computer-readable instructions to provide functions. Data, such as computer-readable instructions and / or user-related data, can be stored on a storage device 510, such as a storage device that can be inside or outside the device. The storage device 510 may include any one or more of volatile or non-volatile memory, a hard drive, a flash memory device and / or an optical storage device (e.g., CD, DVD, etc.), a remote storage device (e.g., a cloud-based storage device), and the like. As used herein, the term "computer-readable medium" may include transient propagating signals. In contrast, the term "computer-readable storage medium" does not include transient propagating signals. Computer-readable storage media include "computer-readable storage devices." Examples of computer-readable storage devices include volatile storage media, such as RAM, and non-volatile storage media, such as hard drives, optical disks, and flash memory, among others.

[0052] Consistent with the present concepts, the base station 102 may include a virtual reality module 512 that provides a virtual reality (including augmented reality or mixed reality) experience to the user 104. For example, the virtual reality module 512 may include software and use hardware resources to perform computer processing for creating and maintaining a simulated environment, including a virtual world and virtual objects; generating visual, auditory, and / or tactile feedback; and processing input signals from the user 104.

[0053] Consistent with the present concepts, the base station 102 may operate in conjunction with one or more peripheral devices 502 to provide a virtual reality experience. For example, the peripheral device 502 may include a display 514 that may render visual feedback to the user 104 based on signals from the base station 102. The peripheral device 502 may also include a speaker 516 that renders auditory feedback to the user 104 based on signals from the base station 102. The peripheral device may include a controller 108 that renders tactile feedback (such as kinesthetic and skin feedback) to the user 104 based on signals from the base station 102.

[0054] In addition, the controller 108 can provide finger movement information to the base station 102. The peripheral device 502 can include a camera 518 that can provide visual information to the base station 102. The peripheral device 502 can also include a microphone 520 that can provide auditory information to the base station 102.

[0055] The peripheral devices 502 may include wearable devices, such as glasses 522 and head mounted devices 106. These wearable devices may include one or more of a display, a speaker, a microphone, and a camera.

[0056] Figure 5 Two example device configurations 524(1) and 524(2) are shown that may be employed by base station 102. Base station 102 may employ either configuration 524(1) or 524(2), or an alternative configuration. An example of each configuration 524 is Figure 5 As shown. Device configuration 524(1) may represent an operating system (OS)-centric configuration. Device configuration 524(1) may be organized into one or more applications 526, an operating system 528, and hardware 530. Device configuration 524(2) may represent a system on a chip (SOC) configuration. Device configuration 524(2) may be organized into shared resources 532, dedicated resources 534, and interfaces 536 therebetween.

[0057] In either configuration 524(1) or 524(2), the base station 102 may include storage / memory 510 and a processor 508. As used herein, the term "processor" may also refer to a hardware processor, such as a central processing unit (CPU), a graphics processing unit (GPU), a controller, a microcontroller, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a processor core, or other types of processing devices. The base station 102 may also include Figure 5 Other components not shown, such as a battery (or other power source), network communication components, and / or input-output components.

[0058] In the case of device configuration 524(2), certain functions provided by base station 102 may be integrated on a single SOC or multiple coupled SOCs. One or more processors 508 may be configured to coordinate with shared resources 532, such as storage / memory 510, etc., and / or with one or more dedicated resources 534, such as hardware blocks configured to perform specific functions. For example, certain functions of virtual reality module 512 may be optimized and implemented on an FPGA.

[0059] FIG. 6A to FIG. 6EA virtual rendering of a manipulated virtual object 602 is shown consistent with some implementations of the present concepts. For example, these virtual scenes can be generated by the base station 102 based on input signals from the controller 108, using inverse kinematics, and sent by the base station 102 to the display 514 for display to the user 104.

[0060] Fig. 6A A virtual finger 604 is shown grasping a virtual object 602. This visual information may have been generated by the base station 102 based on input signals received from the controller 108 in response to the user 104 squeezing the finger rest 208 and / or the trackpad 308 using her finger 206. The base station 102 may use collision dynamics to calculate the contact point between the virtual finger 604 and the virtual object 602, for example, by sending out a ray at the tip of the virtual finger 604 in a direction orthogonal to the corresponding finger rest 208 or trackpad 308 and determining the intersection point between the ray and the virtual object 602. These calculations may be performed for each virtual finger 604 that has a force applied to its corresponding finger rest 208 or trackpad 308.

[0061] When, for example, at least two rays emitted from the virtual finger 604 in substantially opposite directions intersect the virtual object 602 from opposite sides of the virtual object 602 within a certain threshold distance tolerance (e.g., 1 cm) from the surface of the virtual object 602, the base station 102 can declare the acquisition of the virtual object 602 by the virtual finger 604. In addition, using inverse kinematics, the positions and angles of the joints of the virtual finger 604 can be estimated. Once the virtual object 602 has been grasped and acquired, the virtual object 602 can follow the hand movement in the virtual world.

[0062] The base station 102 may generate a simulated virtual finger 604 at Fig. 6A 606 and grasping the virtual object 602. The visual information can be sent by the base station 102 to the display 514 and rendered to the user 104. The rigidity of the finger rest 208 combined with the contemporaneous visual feedback showing the virtual finger 604 grasping the virtual object 602 can provide a convincing experience for the user 104, while the rigidity can provide a resistive pressure to the user's finger 206 squeezing the finger rest 208.

[0063] Optionally, the base station 102 may also generate tactile information that simulates the virtual finger 604 grasping the virtual object 602. In some implementations, the characteristics of the tactile information, including amplitude, frequency, and / or duration, may depend on several factors, such as the level of force applied by the finger 206 on the finger rest 208, and the virtual size, shape, weight, texture, structure, material, and / or orientation of the virtual object 602. For example, in a scenario where the virtual object 602 is a potato chip, if the user's finger 206 applies gentle force on the finger rest 208 when handling the virtual potato chip, the base station 102 may generate little or no tactile feedback. However, if the user's finger 206 squeezes the finger rest 208 hard, the base station 102 may simulate the virtual potato chip breaking or shattering, and generate high-frequency vibrations as tactile feedback, thereby rendering a crisp feeling to the user's finger 206. The tactile information may be sent by the base station 102 to the controller 108 and rendered to the user's finger 206 via the actuator 306. Although the user's finger 206 does not substantially move the rigid finger rest 208, the combination of contemporaneous visual feedback and tactile feedback can create a compelling, realistic experience for the user 104. For example, even though the finger rest 208 is rigid and does not move, the user 104 can sense the compression of the virtual object 602. Furthermore, the user's 104 mind can interpret the vibrations produced by the actuator 306 as the virtual object material fracturing under pressure.

[0064] Figure 6B A virtual finger 604 is shown releasing a virtual object 602. This visual information may be generated by the base station 102 based on input signals received from the controller 108 in response to the user 104 releasing her finger 206 from the finger rest 208 and / or the trackpad 308. The base station 102 may declare a release of the virtual object 602 by the virtual finger 604 based on a number of factors. For example, the base station 102 may declare a release of the virtual object 602 when the force applied by the user's finger 206 on the finger rest 208 and / or the trackpad 308 is negative, i.e., the user's finger 206 is applying a negative force to the finger rest 208 and / or the trackpad 308 using the limiter 210. Alternatively, a release may be declared when the user's finger 206 applies zero force on the finger rest 208 and / or the trackpad 308. Alternatively, a release may be declared when the user's finger 206 applies a small positive force on the finger rest 208 and / or the trackpad 308. Different scenarios for declaring a release may depend on, for example, the shape of the virtual object 602, the virtual weight of the virtual object 602, the angle at which the virtual finger 604 is grasping the virtual object 602, the virtual texture of the virtual object 602 (smooth or rough), whether the virtual object 602 is resting on a virtual surface or being held in the air when released by the virtual finger 604, etc. In some implementations, such as Figure 6CAs shown, while the virtual hand (and virtual finger 604) lifts the virtual object 602 from the bottom, the base station 102 may not declare the release of the virtual object 602.

[0065] The base station 102 may generate a simulated virtual finger 604 at Figure 6B 608 and releases the virtual object 602. If the virtual object 602 has not yet rested on a virtual surface, the base station 102 can simulate the virtual object 602 falling, rotating, hitting a virtual surface, rolling, etc. according to the virtual world scene. This visual information can be sent by the base station 102 to the display 514 and rendered to the user 104. By simultaneously showing the virtual finger 604 opening and thereby releasing the virtual object 602, a real experience can be provided to the user 104 who pulls her finger 206 away from the finger rest 208.

[0066] Optionally, the base station 102 may also generate tactile information that simulates the virtual finger 604 releasing the virtual object 602. For example, in a scenario where the virtual object 602 is a rough piece of wood, the base station 102 may simulate the rough texture of the virtual object 602 sliding off the grip of the user's finger 206 by generating corresponding tactile vibrations at an appropriate frequency to provide a realistic and natural experience to the user 104. The tactile information may be sent by the base station 102 to the controller 108 and rendered to the user's finger 206 via the actuator 306.

[0067] Fig.6D A virtual thumb 604(1) is shown sliding across the surface of the virtual object 602. The visual information may have been generated by the base station 102 in response to the user 104 moving her thumb 206(1) across the thumb trackpad 308(1) based on input signals received from the controller 108. The base station 102 may calculate the direction and speed at which the virtual thumb 604(1) is moving across the surface of the virtual object 602 based on the input signals from the controller 108.

[0068] In some implementations, the base station 102 may compare the force applied by the user's thumb 206(1) on the thumb trackpad 308(1) to a threshold value to determine whether the user 104 wishes to slide the virtual thumb 604(1) along the surface of the virtual object 602 (e.g., to feel the surface texture of the virtual object 602) or to rotate the virtual object 602. That is, the user 104 may slide the virtual thumb 604(1) on the surface of the virtual object 602 by sliding her thumb 206(1) thereon while applying a relatively small force on the trackpad 308. For example, in one implementation, the threshold value may be set to 15N, which may be the maximum force that the trackpad 308 can detect (or the force sensor 302 below the trackpad 308, or to 13.5N, where the force sensor 302 is offset by 1.5N). In this example, if less than 15N is detected while the user's thumb 206(1) slides across the trackpad 308, the base station 102 may slide the virtual thumb 604(1) across the surface of the virtual object 602, or if at least 15N is detected while the user's thumb 206(1) slides across the trackpad 308, the base station 102 may rotate the virtual object 602. Other thresholds are possible.

[0069] The base station 102 may generate a simulated virtual thumb 604(1) at Fig.6D 610 along the surface of the virtual object 602. The visual information can be sent by the base station 102 to the display 514 and rendered to the user 104.

[0070] Base station 102 may also generate tactile information simulating a virtual thumb 604(1) sliding along a surface of virtual object 602. For example, the tactile information may depend on material and surface information of virtual object 602, including surface texture. The magnitude of the texture feedback may be proportional to the speed at which thumb 206(1) moves on thumb trackpad 308(1). Thus, when thumb 206(1) stops moving on thumb trackpad 308(1), the magnitude of the texture feedback may be zero.

[0071] For example, if the virtual object 602 is a virtual television remote control, and the user 104 uses the controller 108 to simulate sliding the virtual thumb 604(1) across the virtual television remote control by sliding her thumb 206(1) across the thumb trackpad 308(1), then the base station 102 can generate appropriate tactile information to simulate the virtual thumb 604(1) sliding between the bumps between the buttons of the virtual television remote control, and to simulate the texture of the soft rubber buttons and hard smooth plastic body of the virtual television remote control. For example, when the virtual thumb 604(1) passes over the edge of the virtual button, the base station 102 can generate tactile feedback with a high amplitude pulse simulating an impact. The base station 102 can also generate tactile feedback that simulates the force-displacement profile of the rubber texture and soft button using known techniques. The tactile information can be sent by the base station 102 to the controller 108 and rendered to the user's thumb 206(1) via the thumb actuator 306(1) below the thumb trackpad 308(1).

[0072] Fig. 6E A virtual finger 604 is shown rotating a virtual object 602. This visual information may be provided by the base station 102 in response to the user 104 moving her thumb 206(1) on the thumb trackpad 308(1) while applying a motion greater than that described above with respect to FIG. Fig.6D The force of the threshold discussed is generated based on the input signal received from the controller 108. This action may be inconsistent with using natural movements to control the virtual fingers 604 and the virtual object 602. For example, the user's index finger 206(2) and middle finger 206(3) may not move in reality, but the corresponding virtual index finger 604(2) and virtual middle finger 604(3) may be visually rendered as rotating. The base station 102 can calculate the rotation direction and speed of the virtual object 602 based on the input signal from the controller 108. In some implementations, for example, when the user 104 rotates a virtual screwdriver or a virtual key in a virtual lock, the rotation of the virtual object 602 can be constrained around a given axis.

[0073] In some implementations, the user 104 may be allowed to lift her finger 206 off the trackpad 308 without releasing the corresponding virtual finger 604 from the grasped virtual object 602. For example, the user 104 may slide her finger 206 on the trackpad 308, lift her finger 206 off the trackpad 308, place her finger 206 back on the trackpad 308 at a different location, and slide her finger 206 on the trackpad 308 again, and so on, in order to continue rotating the virtual object 602. (These actions may be similar to a user moving a mouse up to the edge of a mouse pad, lifting the mouse off the mouse pad, placing the mouse back on the mouse pad at a different location, and then moving the mouse on the mouse pad again, and so on, in order to move a cursor on a display.) To enable this feature, the base station 102 may receive an input signal from the controller 108 indicating that the user 104 has lifted her finger 206 off the trackpad 308, but may not register that the corresponding virtual finger 604 has been lifted off the virtual object 602, such as Figure 6C This may be a scenario that the base station 102 considers when determining whether to release the virtual object 602, as described above in conjunction with Figure 6C For example, the base station 102 may determine that lifting a finger 206 off the trackpad 308 immediately or shortly after sliding the finger 206 along the trackpad 308 may indicate that the user 104 wishes to continue rotating the virtual object 602 rather than lifting the corresponding virtual finger 604 off the virtual object 602. In such an implementation, the user 104 may release the virtual object 602 by lifting her other fingers 206 (except the finger 206 used to slide on the trackpad 308) off their corresponding finger rests 208.

[0074] The base station 102 may generate a simulated virtual finger 604 at Fig. 6E 612 (ie, changing the orientation of the virtual object 602). The visual information may be sent by the base station 102 to the display 514 and rendered to the user 104.

[0075] The base station 102 may also generate tactile information that simulates the virtual finger 604 rotating the virtual object 602. For example, the tactile information may include a vibration pulse (e.g., 6 milliseconds) for every 5 degrees of rotation (similar to an old-fashioned radio dial that clicks when turned). The tactile information may be sent by the base station 102 to the controller 108 and rendered to the user's finger 206.

[0076] Figure 7-10 A flowchart illustrating a method for manipulating a virtual object using a controller and providing a haptic sensation via the controller consistent with some implementations of the present concept is shown. The actions on the left side of the flowchart may be performed by a base station, and the actions on the right side of the flowchart may be performed by a controller.

[0077] Figure 7 A flow chart illustrating a grasping method 700 is shown. In action 702, the controller detects a force applied by one or more fingers of a user on one or more finger rests and / or a touch pad on the controller. For example, the user may use her fingers to squeeze together two or more finger rests that are positioned on opposite sides of the controller and facing away from each other. The user may perform this action in order to grasp a virtual object that is positioned between virtual fingers corresponding to the squeezed finger rests.

[0078] In act 704, the controller may send an input signal indicating that the finger rest is being squeezed. For example, the input signal may include information from one or more force sensors, including an identification of the finger and how much force is applied to the force sensor under the finger rest. In some implementations, the input signal may include information from one or more trackpads indicating the force (or pressure) with which the user's finger is pressing on the trackpad. In act 706, the base station may receive the input signal from the controller.

[0079] In action 708, the base station may interpret the input signals and apply them to the virtual world. That is, the base station may convert the input signals from the controller (indicating the position of the user's fingers and the forces they apply) into the movement of the corresponding virtual fingers in the virtual world. In this example, the base station may simulate the virtual fingers moving toward each other, and if there is a virtual object between the virtual fingers, the base station may declare that the virtual fingers are grasping the virtual object.

[0080] In addition, the base station can generate visual information corresponding to the virtual fingers closing and grasping the virtual object. The visual information can be sent by the base station to a display for rendering to the user.

[0081] The base station may also generate tactile information corresponding to the virtual fingers grasping the virtual object. For example, for each 0.49N change in force detected by the corresponding force sensor, the actuator may render a 6ms vibration pulse at a frequency of 170Hz to the appropriate finger rest. In action 710, the base station may send an output signal corresponding to the tactile information to the controller.

[0082] In act 712, the controller may receive the output signal from the base station. In act 714, the controller may drive an actuator in the controller to provide a tactile sensation on the finger rest and / or the trackpad, and thus on the user's finger.

[0083] Figure 8A flow chart illustrating a release method 800 is shown. In action 802, the controller detects a force applied by one or more fingers of a user on one or more finger rests and / or a touch pad on the controller. The detected force can be a negative force, a zero force, or a small positive force on the finger rest and / or the touch pad. For example, the user can pull her finger away from the finger rest. In some implementations, the back of the user's finger can push against a limiter attached to the finger rest and / or the touch pad. The user can perform this action to release a virtual object that is being grasped by the virtual finger.

[0084] In act 804, the controller may send an input signal indicating that the finger rest is no longer squeezed. For example, the input signal may include information from one or more force sensors, including the identity of the finger and how much force is applied to the force sensor below the finger rest. In some implementations, the input signal may include information from one or more trackpads indicating the force (or pressure) with which the user's finger is pressing on the trackpad, if any. In act 806, the base station may receive the input signal from the controller.

[0085] In action 808, the base station may interpret the input signals and apply them to the virtual world. In this example, the base station may move the virtual fingers away from each other and release the virtual object being grasped by the virtual fingers. The base station may perform further processing on the virtual world, such as the virtual object being dropped, etc.

[0086] The base station may generate visual information corresponding to a virtual finger releasing a virtual object. The visual information may be sent by the base station to a display for presentation to a user. The base station may also generate auditory information, such as corresponding to a virtual object being dropped and landing on a surface. The auditory information may be sent by the base station to a speaker for presentation to a user.

[0087] The base station may also generate tactile information corresponding to the virtual finger releasing the virtual object. In action 810, the base station may send an output signal corresponding to the tactile information to the controller.

[0088] In act 812, the controller may receive the output signal from the base station. In act 814, the controller may drive an actuator in the controller to provide a tactile sensation on the finger rest and / or the touch pad, and thus on the user's finger.

[0089] Fig. 9 A flow chart illustrating a sliding method 900 is shown. In action 902, the controller detects a finger sliding on a trackpad on the controller. For example, a user may perform this action to move a virtual finger along the surface of a virtual object.

[0090] In action 904, the controller may send an input signal indicating the position of the finger on the touch pad and / or the level of force applied by the finger on the touch pad. Both the position information and the level of force information may be provided from the touch pad. Alternatively, the position information may be provided from the touch pad, and the level of force information may be provided from a force sensor below the touch pad. In action 906, the base station may receive the input signal from the controller.

[0091] In action 908, the base station may interpret the input signals and apply them to the virtual world. In this example, if the base station determines that the level of force applied to the trackpad is less than a certain threshold, the base station may slide a virtual finger over the virtual object along its surface. The virtual finger may be moved in a direction corresponding to the direction in which the user's finger is moved on the trackpad.

[0092] In addition, the base station can generate visual information corresponding to the virtual finger sliding along the surface of the virtual object. The visual information can be sent by the base station to a display for rendering to the user.

[0093] The base station may also generate tactile information corresponding to the sliding of a virtual finger on the surface of a virtual object. For example, the tactile information may depend on the surface texture of the virtual object. In action 910, the base station may send an output signal corresponding to the tactile information to the controller.

[0094] In act 912, the controller may receive the output signal from the base station. In act 914, the controller may drive an actuator in the controller to provide a tactile sensation on the touch pad, and thus on the user's finger.

[0095] Fig.10 A flow chart illustrating a rotation method 1000 is shown. In action 1002, the controller detects a finger sliding on a trackpad on the controller. For example, a user may perform this action to rotate a virtual object using a virtual finger.

[0096] In action 1004, the controller may send an input signal indicating the position of the finger on the touch pad and / or the level of force applied by the finger on the touch pad. Both the position information and the level of force information may be provided from the touch pad. Alternatively, the position information may be provided from the touch pad, and the level of force information may be provided from a force sensor under the touch pad. In action 1006, the base station may receive the input signal from the controller.

[0097] In action 1008, the base station may interpret the input signals and apply them to the virtual world. In this example, if the base station determines that the level of force applied to the trackpad is greater than a certain threshold, the base station may rotate the virtual object being grasped by the virtual finger. The virtual object may be rotated in a direction corresponding to the direction in which the user's finger is moving on the trackpad.

[0098] In addition, the base station can generate visual information corresponding to the virtual object being rotated by the virtual finger. The visual information can be sent by the base station to a display for rendering to the user.

[0099] The base station may also generate haptic information corresponding to the rotation of the virtual object. In action 1010, the base station may send an output signal corresponding to the haptic information to the controller.

[0100] In act 1012, the controller may receive the output signal from the base station. In act 1014, the controller may drive the actuator in the controller to provide a tactile sensation on the touch pad and the finger rest, and thus provide tactile feedback to the user's fingers.

[0101] Various device examples are described above. Additional examples are described below. One example includes a system including a controller and a base station, the base station including a base station processor configured to: execute a virtual reality world including a virtual object and a virtual finger, receive an input signal from the controller, manipulate the virtual finger and / or the virtual object in response to the input signal, and generate an output signal based at least on the manipulation, and send the output signal to the controller. The controller includes a thumb rest and at least one finger rest facing away from each other, the thumb rest and the at least one finger rest being substantially rigid, a thumb force sensor positioned below the thumb rest and configured to sense a force applied to the thumb rest, at least one finger force sensor positioned below the at least one finger rest and configured to sense a force applied to the at least one finger rest, a thumb actuator positioned below the thumb rest and configured to provide a tactile sensation on the thumb rest, and at least one finger actuator positioned below the at least one finger rest and configured to provide a tactile sensation on the at least one finger rest. The controller also includes a controller processor, which is configured to generate an input signal based at least on the force sensed by the thumb force sensor and / or at least one finger force sensor, the input signal is not based on the movement of the thumb rest and at least one finger rest, send the input signal to the base station, receive an output signal from the base station, and drive the thumb actuator and / or at least one finger actuator based at least on the output signal.

[0102] Another example may include any of the above and / or following examples, wherein the controller further comprises a touch pad positioned on the thumb rest, and the controller processor is configured to generate the input signal based at least also on the position of the thumb on the touch pad.

[0103] Another example includes an apparatus comprising: finger rests substantially facing away from each other, the finger rests being substantially rigid, one or more force sensors positioned below the finger rests, the one or more force sensors being configured to sense a force applied by the finger to the finger rests, a touch pad configured to sense a position of one of the fingers touching the touch pad, and one or more actuators positioned and configured to provide a tactile sensation on the finger rest and / or the touch pad. The apparatus also includes a processor configured to generate an input signal based at least on the force applied by the finger on the finger rest and / or the position of one of the fingers on the touch pad, and to drive the actuator based at least on the output signal. The apparatus also includes a transceiver configured to send the input signal and receive the output signal.

[0104] Another example may include any of the above and / or following examples, wherein the device further comprises a restraint attached to the finger rest, the restraint and the finger rest being configured to form a ring around the finger, wherein the one or more force sensors are mechanically biased, and wherein the one or more force sensors are capable of sensing a finger pulling the restraint away from the finger rest.

[0105] Another example may include any of the above and / or following examples, wherein the device further comprises a handle for holding the device using at least a palm of a hand.

[0106] Another example can include any of the above and / or following examples, wherein at least two of the force sensors are positioned below one of the finger rests.

[0107] Another example may include any of the above and / or following examples, wherein one of the actuators provides tactile sensations to at least two fingers.

[0108] Another example can include any of the above and / or following examples, wherein at least two of the actuators are positioned below one of the finger rests.

[0109] Another example may include any of the above and / or following examples, wherein the trackpad is a capacitive sensor.

[0110] Another example includes a method comprising executing a virtual reality world including a virtual object and virtual fingers of a virtual hand, receiving a closure input signal from a controller including finger rests facing away from each other, the closure input signal indicating that the fingers are applying force to the finger rests, generating a grasp tactile output signal simulating the virtual fingers grasping the virtual object in response to the closure input signal, and sending the grasp tactile output signal to the controller, the grasp tactile output signal causing the controller to drive one or more actuators in the controller that generate a tactile sensation on the finger rests.

[0111] Another example may include any of the above and / or following examples, wherein the method further includes determining a virtual contact point between one of the virtual fingers and the virtual object by emitting a ray at one of the virtual fingers in a direction orthogonal to a corresponding one of the finger rests and calculating an intersection point between the ray and the virtual object.

[0112] Another example may include any of the above and / or following examples, wherein the method further comprises declaring a grasp of the virtual object acquired by the virtual finger when two or more rays emitted from the virtual finger intersect the virtual object from opposite sides.

[0113] Another example may include any of the above and / or following examples, wherein the method further comprises estimating positions of virtual joints of the virtual hand.

[0114] Another example may include any of the above and / or following examples, wherein the method further includes receiving an open input signal from a controller while the virtual finger is grasping the virtual object, the controller including a limiter attached to the finger rest, the open input signal indicating that the finger is applying a negative force away from the finger rest, generating a release visual output signal simulating the virtual finger releasing the virtual object in response to the open input signal, and sending the release visual output signal to the display.

[0115] Another example may include any of the above and / or following examples, wherein the method further includes receiving a rotational input signal from a controller while the virtual fingers are grasping a virtual object, the controller including a touch pad, the rotational input signal indicating that one of the fingers slides on the touch pad with a force greater than a threshold, generating a rotational tactile output signal simulating the virtual fingers rotating the virtual object in response to the rotational input signal, and sending the rotational tactile output signal to the controller to produce a tactile sensation.

[0116] Another example may include any of the above and / or following examples, wherein the method further comprises, in response to the rotation input signal, generating a rotation visual output signal simulating a virtual finger rotating the virtual object, and sending the rotation visual output signal to a display.

[0117] Another example may include any of the above and / or following examples, wherein the virtual object rotates in a direction corresponding to a direction in which one of the fingers on the trackpad slides.

[0118] Another example may include any of the above and / or following examples, wherein the method further includes receiving a lift input signal from a controller while the virtual fingers are grasping a virtual object, the lift input signal indicating that one of the fingers has been lifted off the touchpad, and in response to the lift input signal, maintaining one of the virtual fingers corresponding to one of the fingers grasping the virtual object.

[0119] Another example may include any of the above and / or following examples, wherein the method further includes receiving a sliding input signal from a controller including a touch pad, the sliding input signal indicating that one of the fingers is sliding on the touch pad with a force below a threshold, generating a sliding visual output signal and a sliding tactile output signal in response to the sliding input signal, the sliding visual output signal and the sliding tactile output signal simulating a virtual finger corresponding to one of the fingers sliding on a virtual object, sending the sliding visual output signal to a display, and sending the sliding tactile output signal to the controller to produce a tactile sensation.

[0120] Another example may include any of the above and / or following examples, wherein the haptic sensation simulates a surface texture of the virtual object.

[0121] Various examples are described above. Although the subject matter has been described in language specific to structural features and / or methodological acts, the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are presented as example forms of implementing the claims, and other features and acts that those skilled in the art will recognize are intended to fall within the scope of the claims.

Claims

1. A device for virtual reality, comprising: finger rests substantially facing away from each other, the finger rests being substantially rigid and not substantially moving relative to each other when a force is applied to the finger rests to generate an input signal; one or more force sensors positioned below the finger rest, the one or more force sensors configured to sense a force applied by a finger to the finger rest; a touch panel configured to sense a position of one of the fingers touching the touch panel; one or more actuators positioned and configured to provide a tactile sensation on the finger rest and / or the touch pad; The processor is configured as: generating the input signal based at least on a force exerted by the finger on the finger rest and / or the position of the one of the fingers on the touch pad, the input signal not based on substantial movement of the finger rests relative to each other; as well as driving the actuator based at least on the output signal; a transceiver configured to transmit the input signal and receive the output signal; as well as a restraint attached to the finger rest, the restraint and the finger rest being configured to form a ring around the finger, wherein the one or more force sensors are mechanically biased, and The one or more force sensors are capable of sensing the finger pulling the restraint away from the finger rest.

2. The device of claim 1, further comprising a handle for holding the device using at least a palm of a hand.

3. The apparatus of claim 1, wherein at least two of the force sensors are positioned below one of the finger rests.

4. The device of claim 1, wherein one of the actuators provides tactile sensations to at least two fingers.

5. The apparatus of claim 1, wherein at least two of the actuators are positioned below one of the finger rests. The device of claim 1 , wherein the touch pad is a capacitive sensor.

7. A method for virtual reality, comprising: executing a virtual reality world including virtual objects and virtual fingers of a virtual hand; receiving a closure input signal from a controller including finger rests facing away from each other, the closure input signal being based on forces applied by fingers to the finger rests and not based on substantial movement of the finger rests relative to each other in response to the forces applied by the fingers; In response to the closing input signal, generating a grasping tactile output signal simulating the virtual finger grasping the virtual object; sending the grip tactile output signal to the controller, the grip tactile output signal causing the controller to drive one or more actuators in the controller that generate a tactile sensation on the finger rest; receiving an open input signal from the controller while the virtual finger is grasping the virtual object, the controller including a limiter attached to the finger rest, the open input signal indicating that the finger is applying a negative force away from the finger rest; In response to the open input signal, generating a release visual output signal simulating the virtual finger releasing the virtual object; as well as The release visual output signal is sent to a display.

8. The method according to claim 7, further comprising: A virtual contact point between the one of the virtual fingers and the virtual object is determined by emitting a ray at one of the virtual fingers in a direction orthogonal to a corresponding one of the finger rests and calculating an intersection point between the ray and the virtual object.

9. The method according to claim 8, further comprising: When two or more rays emitted from the virtual finger intersect the virtual object from opposite sides, grasping acquisition of the virtual object by the virtual finger is declared.

10. The method according to claim 7, further comprising: Positions of virtual joints of the virtual hand are estimated.

11. The method according to claim 7, further comprising: receiving a rotation input signal from the controller while the virtual fingers are grasping the virtual object, the controller comprising a touch pad, the rotation input signal indicating that one of the fingers is sliding on the touch pad with a force above a threshold; In response to the rotation input signal, generating a rotation tactile output signal simulating the virtual finger rotating the virtual object; as well as The rotational tactile output signal is sent to the controller to generate a tactile sensation.

12. The method according to claim 11, further comprising: In response to the rotation input signal, generating a rotation visual output signal simulating the virtual finger rotating the virtual object; as well as The rotational visual output signal is sent to a display. 13 . The method of claim 12 , wherein the virtual object rotates in a direction corresponding to a direction of the sliding of the one of the fingers on the trackpad.

14. The method according to claim 11, further comprising: receiving a lift input signal from the controller while the virtual finger is grasping the virtual object, the lift input signal indicating that the one of the fingers has been lifted off the touchpad; as well as In response to the lift-off input signal, one of the virtual fingers corresponding to the one of the fingers grasping the virtual object is held.

15. The method according to claim 7, further comprising: receiving a slide input signal from the controller including a touch pad, the slide input signal indicating that one of the fingers slides on the touch pad with a force below a threshold; generating, in response to the sliding input signal, a sliding visual output signal and a sliding tactile output signal, the sliding visual output signal and the sliding tactile output signal simulating one of the virtual fingers corresponding to the one of the fingers sliding on the virtual object; sending the sliding visual output signal to a display; as well as The sliding tactile output signal is sent to a controller to generate a tactile sensation. The method of claim 15 , wherein the haptic sensation simulates a surface texture of the virtual object.

17. A system for virtual reality, comprising: Controller; as well as A base station comprising a base station processor, the base station processor being configured to: implementing a virtual reality world including a virtual object and a virtual finger; receiving an input signal from the controller; In response to the input signal, manipulating the virtual finger and / or the virtual object, and generating an output signal based at least on the manipulation; as well as sending an output signal to the controller; The controller comprises: main body; a thumb rest and at least one finger rest rigidly attached to the body, the thumb rest and the at least one finger rest facing away from each other, the thumb rest and the at least one finger rest being substantially rigid and not substantially moving relative to the body and relative to each other when a force is applied to the thumb rest and / or the at least one finger rest to generate the input signal; a thumb force sensor positioned below the thumb rest and configured to sense a force applied to the thumb rest; at least one finger force sensor positioned below at least one finger rest and configured to sense a force applied to the at least one finger rest; a thumb actuator positioned below the thumb rest and configured to provide a tactile sensation on the thumb rest; at least one finger actuator positioned below the at least one finger rest and configured to provide a tactile sensation on the at least one finger rest; and The controller processor is configured to: generating the input signal based at least on the force sensed by the thumb force sensor and / or the at least one finger force sensor, the input signal not based on substantial movement of the thumb rest and the at least one finger rest; sending the input signal to the base station; receiving the output signal from the base station; and The thumb actuator and / or the at least one finger actuator are driven based at least on the output signal.

18. The system of claim 17, wherein: The controller also includes a touch pad positioned on the thumb rest; and The controller processor is configured to generate the input signal based at least also on a position of the thumb on the trackpad.

Images