Input device

Abstract

An input device 20 is worn on a finger of a user and used for the operation of an application. The input device 20 includes a housing 40 having a through hole 62 for inserting the finger of the user. The housing 40 has a front surface 50, an upper surface 52 adjacent to the front surface 50, and a back surface 54 adjacent to the upper surface 52. An operation member operated by the user is provided on each of the front surface 50, the upper surface 52, and the back surface 54.

Description

Input Devices

[0001] The present disclosure relates to an input device used to operate an application or an accessory device that is detachable from an input device.

[0002] Patent Literature 1 discloses an information processing system including an operation device that is held by a user's hand and provides a tactile sensation to the user's hand. In the information processing system disclosed in Patent Literature 1, operation information indicating the movement of the operation device is used to control the movement of a virtual device corresponding to the operation device.

[0003] International Publication No. 2023 / 286191

[0004] A head-mounted display (HMD) is worn on the user's head and provides the user with a visual world of virtual reality (VR) or mixed reality (MR).Recently, research has been conducted into operating applications using hand gestures, and it is believed that combining hand interaction with the visual world provided by an HMD will provide users with a new experience.

[0005] Increasing the variety of inputs to an application allows the application to provide a variety of experiences to the user. The present disclosure has been made in light of these circumstances, and its purpose is to provide an input device used to operate an application and / or an accessory device that is detachable from the input device.

[0006] One aspect of the present disclosure is an input device used to operate an application, comprising a housing having a through hole for inserting a user's finger, the housing having a first surface, a second surface adjacent to the first surface, and a third surface adjacent to the second surface, and an operating member operated by the user is provided on each of the first surface, the second surface, and the third surface.

[0007] Any combination of the above components and conversion of the expressions of the present disclosure into methods, devices, systems, etc. are also valid aspects of the present disclosure.

[0008] 5 is a diagram illustrating an example of the configuration of an information processing system in an embodiment. FIG. 6 is a diagram illustrating an example of the external shape of an HMD. FIG. 7 is a diagram illustrating functional blocks of an HMD. FIG. 8 is a diagram illustrating an example of an input device worn on both hands of a user. FIG. 9 is a side view of the input device. (a) and (b) are perspective views of the input device. FIG. 9 is a partial cross-sectional view of the area surrounded by the dotted line in FIG. 5. FIG. 10 is a diagram illustrating a state in which a movable member protrudes from the inner circumferential surface of a through-hole. (a) and (b) are diagrams illustrating an example in which the length of the band has been adjusted. FIG. 11 is a diagram illustrating a modified example of a structure for holding a user's fingers. FIG. 12 is a diagram illustrating functional blocks of an input device. FIG. 13 is a diagram illustrating functional blocks of an information processing device. (a) and (b) are diagrams illustrating an accessory device that is detachable from the input device. FIG. 14 is a diagram illustrating a state in which an accessory device connected to the input device is being held. FIG. 15 is a diagram illustrating functional blocks of the accessory device.

[0009] 1 shows an example of the configuration of an information processing system 1 according to an embodiment. The information processing system 1 includes an information processing device 10 that executes an application, a recording device 11, a head-mounted display (HMD) 100 worn on a user's head, an input device 20 attached to the user's fingers, and an output device 15 that outputs images and sounds. The output device 15 may be a television. The information processing device 10 is connected to an external network 2, such as the Internet, via an access point (AP) 17. The AP 17 has the functions of a wireless access point and a router, and the information processing device 10 and AP 17 may be connected by a cable or via a known wireless communication protocol.

[0010] The recording device 11 records system software and application software such as games. The information processing device 10 may download application software from a content server (not shown) via the network 2. The information processing device 10 executes the application based on a user's operation input, generates image data and sound data of the application, and supplies them to the HMD 100 and the output device 15. The information processing device 10 and the HMD 100 may be connected using a known wireless communication protocol or by a cable.

[0011] The HMD 100 is a display device worn by a user on the head that displays images on a display panel located in front of the user's eyes, and is also called a VR headset. The HMD 100 displays an image for the left eye on a left-eye display panel and an image for the right eye on a right-eye display panel. These images form parallax images seen from left and right viewpoints, realizing stereoscopic vision. Because the user views the display panel through optical lenses, the information processing device 10 supplies the HMD 100 with parallax image data that has been corrected for optical distortion caused by the lenses. The HMD 100 provides the user with a virtual reality (VR) or mixed reality (MR) video world. The information processing device 10 is equipped with a head tracking function that updates the displayed image in conjunction with the user's head movement, thereby enhancing the sense of immersion in the video world.

[0012] Although the output device 15 is not necessary for a user wearing the HMD 100, providing the output device 15 allows another user to view the image displayed on the output device 15. In the embodiment, the information processing device 10 displays on the output device 15 the same image as the image viewed by the user wearing the HMD 100, but a different image may also be displayed.

[0013] The input device 20 is powered by an internal battery and is used to operate applications. The input device 20 may be a wireless controller equipped with multiple operation members, such as operation buttons and operation sticks. When a user operates the operation members of the input device 20, operation data of the operation members is transmitted to the information processing device 10. The input device 20 may transmit the operation data of the operation members to the information processing device 10 at a predetermined interval. A wireless connection may be established between the input device 20 and the information processing device 10, for example, using the Bluetooth (registered trademark) protocol. The input device 20 may also be a wired controller connected to the information processing device 10 via a cable. Upon receiving operation data from the input device 20, the information processing device 10 controls the progress of the application based on the operation data and generates image data and sound data for the application. While FIG. 1 shows a user wearing a right-hand input device 20 on the fingers of their right hand, a left-hand input device 20 may also be worn on the fingers of their left hand.

[0014] In the application of the embodiment, a virtual hand (virtual hand) 22 is moved in a virtual space in accordance with the movement of a user's hand in real space, and the virtual hand 22 acts on a virtual object arranged in the virtual space. In the example shown in Fig. 1, a virtual button (NO button) 24 and a virtual button (YES button) 26 are arranged in the virtual three-dimensional space, and the user moves their hand in real space to move the virtual hand 22 in the virtual three-dimensional space and select either the virtual button 24 or 26. Because the input device 20 is attached to the user's finger, the user can move their hand freely in real space without the input device 20 interfering with their hand movement.

[0015] The input device 20 of the embodiment may include a tactile sensation providing component that provides a tactile sensation to the user's hand. Here, tactile sensation refers to a skin sensation that occurs when the surface of the hand is subjected to pressure or vibration. The tactile sensation providing component of the embodiment provides haptic feedback to the user, enhancing the user's sense of immersion in the virtual world. In the example shown in FIG. 1 , when the virtual hand 22 touches the virtual button 24 or 26, the tactile sensation providing component may be activated to provide a tactile sensation to the user's fingers. The tactile sensation allows the user to recognize that the virtual hand 22 has touched the virtual button.

[0016] The HMD 100 is equipped with multiple image capture devices 14. The multiple image capture devices 14 are attached to the front of the HMD 100 in different positions and orientations so that the combined overall capture range of each captures the entire user's field of view. The image capture devices 14 may have a visible light sensor used in general digital video cameras, such as a charge-coupled device (CCD) sensor or a complementary metal oxide semiconductor (CMOS) sensor. The multiple image capture devices 14 capture images of the area in front of the user at a predetermined frequency (e.g., 120 frames per second) in a synchronized manner and transmit image data of the real space to the information processing device 10.

[0017] 2 shows an example of the external shape of the HMD 100. The HMD 100 is composed of an output mechanism unit 102 and a wearing mechanism unit 104. The wearing mechanism unit 104 includes a wearing band 106 that, when worn by a user, goes around the head and secures the HMD 100 to the head. The wearing band 106 is made of a material or has a structure that allows the length to be adjusted to fit the user's head circumference.

[0018] The output mechanism unit 102 includes a housing 108 shaped to cover the left and right eyes when the HMD 100 is worn by the user, and includes a display panel inside that faces the eyes when worn. The display panel may be a liquid crystal panel, an organic EL panel, or the like. The housing 108 also includes a pair of optical lenses, one on each side, that are positioned between the display panel and the user's eyes and expand the user's field of view. The HMD 100 may also include speakers or earphones at positions corresponding to the user's ears, and may be configured to allow external headphones to be connected.

[0019] Multiple image capture devices 14a, 14b, 14c, and 14d are provided on the front outer surface of the housing 108. With respect to the direction of the user's face as a reference, image capture device 14a is attached to the upper right corner of the front outer surface so that its optical axis faces diagonally upward to the right, image capture device 14b is attached to the upper left corner of the front outer surface so that its optical axis faces diagonally upward to the left, image capture device 14c is attached to the lower right corner of the front outer surface so that its optical axis faces diagonally downward to the right, and image capture device 14d is attached to the lower left corner of the front outer surface so that its optical axis faces diagonally downward to the left. By installing multiple image capture devices 14a, 14b, 14c, and 14d (hereinafter referred to as "image capture devices 14" unless otherwise specified), the total image capture range obtained by adding together the image capture ranges of each device encompasses the entire field of view of the user.

[0020] 3 shows functional blocks of the HMD 100. The control unit 120 is a main processor that processes and outputs various data such as image data, sound data, and sensor data, as well as commands. The storage unit 122 temporarily stores the data and commands processed by the control unit 120. The IMU (Inertial Measurement Unit) 124 acquires data related to the movement of the HMD 100. The IMU 124 may include at least a three-axis acceleration sensor and a three-axis angular velocity sensor. The IMU 124 detects the values ​​of each axial component (sensor data) at a predetermined cycle (e.g., 800 Hz).

[0021] The communication control unit 128 transmits data output from the control unit 120 to the external information processing device 10 via a network adapter or an antenna, by wired or wireless communication. The communication control unit 128 also receives data from the information processing device 10 and outputs it to the control unit 120.

[0022] When the control unit 120 receives image data and sound data from the information processing device 10, it supplies the data to the display panel 130 for display and to the sound output unit 132 for sound output. The display panel 130 is composed of a left-eye display panel 130a and a right-eye display panel 130b, and a pair of parallax images is displayed on each display panel. The control unit 120 also causes the communication control unit 128 to transmit IMU data detected by the IMU 124, sound data acquired by the microphone 126, and image data captured by the imaging device 14 to the information processing device 10.

[0023] 4 shows an example of the input device 20 worn on both hands of a user. In the information processing system 1 of the embodiment, an input device 20 for a left hand and an input device 20 for a right hand are prepared, and the user wears the input device 20 for a left hand on the fingers of his left hand and the input device 20 for a right hand on the fingers of his right hand. The housing 40 of the input device 20 has a ring shape with a through-hole formed therein, and the user wears the input device 20 by inserting one finger into the through-hole. The user may wear the input device 20 on his index finger.

[0024] The index finger is composed of, from the tip, a distal phalanx with a nail, a DIP joint (first joint), a middle phalanx, a PIP joint (second joint), a proximal phalanx, and an MP joint (third joint). The input device 20 is preferably attached to the proximal phalanx of the user's index finger. By attaching the input device 20 to the proximal phalanx, the user can freely use the tip of the index finger. Therefore, with the input device 20 attached to the index finger, the user can also touch real objects in real space with the index finger. The input device 20 may also be attached to the middle finger.

[0025] The input device 20 of the embodiment is preferably compact in structure because it is worn on the index finger or middle finger. The input device 20 for the left hand and the input device 20 for the right hand may be formed symmetrically and have substantially the same structure. Therefore, the structure of the input device 20 will be described below without making any particular distinction between the left-handed and right-handed input devices.

[0026] Fig. 5 shows a side view of the input device 20. Fig. 6(a) shows a front perspective view of the input device 20, and Fig. 6(b) shows a rear perspective view of the input device 20. The input device 20 includes a housing 40 having a through-hole 62 for inserting a user's finger, and the user wears the input device 20 by inserting the finger into the through-hole 62. As will be described later, the finger inserted into the through-hole 62 is pressed against the inner circumferential surface of the housing 40 by a band 80 made of an elastic material and fixed in place.

[0027] The outer peripheral surface of the housing 40 is composed of a front surface 50, an upper surface 52 adjacent to the front surface 50, a rear surface 54 adjacent to the upper surface 52, and a lower surface 56 adjacent to the rear surface 54 and the front surface 50. In the side view shown in FIG. 5 , the front surface 50, the upper surface 52, the rear surface 54, and the lower surface 56 constitute the outer peripheral surface of the housing 40 in this order, clockwise. The front surface 50, the upper surface 52, the rear surface 54, and the lower surface 56 may each have a flat surface. In the example shown in FIG. 5 , the rear surface 54 has a flat surface and a curved surface, and the curved surface of the rear surface 54 is connected to the lower surface 56. Note that the front surface 50 may also have a flat surface and a curved surface, and the curved surface of the front surface 50 is connected to the lower surface 56.

[0028] Since the lower surface 56 has a flat surface, the input device 20 can be placed upright on a table, floor, or the like, with the flat surface of the lower surface 56 serving as the base. In this embodiment, the flat surface of the front surface 50 and the flat surface of the upper surface 52 are connected at a predetermined first angle θ1, and the flat surface of the upper surface 52 and the flat surface of the rear surface 54 are connected at a predetermined second angle θ2. Here, the first angle θ1 is preferably set larger than the second angle θ2, and the first angle θ1 may be approximately 135 degrees, and the second angle θ2 may be approximately 90 degrees. The angle formed between the flat surface of the front surface 50 and the flat surface of the lower surface 56 may also be approximately 90 degrees.

[0029] As shown in the figure, the flat surface of the upper surface 52 and the flat surface of the lower surface 56 are not parallel. In the embodiment, the positional relationship between the flat surface of the upper surface 52 and the flat surface of the lower surface 56 is set so that the angle between the axial direction of the directional control 70, which is perpendicular to the flat surface of the upper surface 52, and the flat surface of the lower surface 56 is α. Here, the angle α may be between 30 degrees and 60 degrees, and in the embodiment, the angle α is set to approximately 45 degrees. By setting the angle α in this manner, when the input device 20 is worn on a hand, a finger other than the finger inserted in the through-hole 62 abuts on the lower surface 56, thereby preventing rotation of the housing 40. For example, when an index finger is inserted into the through-hole 62, the middle finger abuts on the lower surface 56, preventing the housing 40 from rotating around the index finger.

[0030] The front surface 50, the top surface 52, and the back surface 54 are each provided with an operation member operated by the user. The operation members may be provided on a flat surface. In this example, the front surface 50 is provided with two press-type operation buttons 72, 74, the top surface 52 is provided with a directional control 70 for inputting directions, and the back surface 54 is provided with two press-type operation buttons 76, 78. When a user inserts their index finger into the through-hole 62 (see FIG. 4 ), the front surface 50 is located below the index finger, the top surface 52 is located to the side of the index finger, and the back surface 54 is located above the index finger. However, the user can use their thumb to operate the operation members provided on the front surface 50, the top surface 52, and the back surface 54. Providing operation members on three surfaces of the housing 40 of the input device 20 allows the user to perform various operation inputs to applications.

[0031] The number of push-buttons arranged on the front surface 50 and the back surface 54 is not limited to two, and may be one, or three or more. Operation buttons may also be provided on the side surface of the housing 40, and a distance measuring sensor for measuring distance may also be provided.

[0032] The directional control device 70 may be a control stick called an analog stick or joystick, in which the user inputs a direction by tilting the stick. Alternatively, the directional control device 70 may be a pressure-sensitive sensor having multiple pressure-sensitive elements arranged in a matrix. The user can input a direction by moving their fingertip on the pressure sensor or by changing the direction of pressure with their fingertip. Using a pressure-sensitive sensor as the directional control device 70 has the advantage of being smaller in structure than using a control stick. The operability of the directional control device 70 may be improved by closely adhering a dome-shaped cover part made of a gel material to the pressure sensor.

[0033] Furthermore, the operation buttons 72, 74, 76, and 78 may be configured as pressure sensors with multiple pressure-sensitive portions instead of depressible buttons (push buttons). By providing pressure sensors continuously on the continuous surfaces of the front surface 50, top surface 52, and back surface 54, and dividing and allocating the pressure sensor area clockwise to the operation button 74, operation button 72, directional control 70, operation button 76, and operation button 78, the pressure sensors provided across three surfaces may be used as five different operation members. When pressure sensors are used, the areas can be divided by software and used as different operation members, making it possible to change the operation member used for each application.

[0034] Figure 7 shows a partial cross-sectional view of the area surrounded by the dotted line in Figure 5. Figure 7 shows an example of a means for fixing the band 80 to the housing 40. In this example, the housing 40 is configured by fitting an exterior part 84 into a main part 82, and both ends of the band 80 are sandwiched and fixed between the outer peripheral surface of the main part 82 and the inner peripheral surface of the exterior part 84.

[0035] As shown in the figure, convex portions 86a and concave portions 86b are formed alternately and periodically in the circumferential direction on the outer peripheral surface of the main body component 82, and convex portions 88a and concave portions 88b are formed alternately and periodically in the circumferential direction on the inner peripheral surface of the exterior component 84. When the exterior component 84 is fitted into the main body component 82, the convex portions 86a and concave portions 88b, and the concave portions 86b and convex portions 88a, are positioned opposite each other. Therefore, by closely contacting the outer peripheral surface of the main body component 82 with the inner peripheral surface of the exterior component 84, the band 80 can be firmly fixed between the outer peripheral surface of the main body component 82 and the inner peripheral surface of the exterior component 84. Note that while FIG. 7 shows a structure in which one end of the band 80 is fixed to the housing 40, the same structure is formed on the other end of the band 80, and both ends of the band 80 are fixed to the housing 40. Note that the fixing structure of the band 80 shown in FIG. 7 is merely an example, and other fixing structures may be used.

[0036] The band 80 is formed from an elastic material, and the user's fingers inserted between the band 80 and the inner circumferential surface of the through-hole 62 are pressed against the inner circumferential surface of the through-hole 62 to form a tight seal. For example, the band 80 may be formed from rubber, and the user's fingers are pressed against the inner circumferential surface of the through-hole 62 by elastic force. The band 80 may also be formed from a material that softens and becomes easily deformed at temperatures near human body temperature. When the band 80 is formed from this material, the band 80 can deform to a shape that fits the user's fingers and hold them. Regardless of the elastic material used for the band 80, the band 80 presses and holds the user's fingers against the inner circumferential surface of the through-hole 62.

[0037] 6( b), a tactile sense providing component 64 that provides a tactile sensation to the user's fingers is provided inside the housing 40. The tactile sense providing component 64 has a function of contacting the fingers to provide a pressure stimulus and / or a vibration stimulus. The tactile sense providing component 64 has a movable member 64a that moves, and the movable member 64a is arranged so as to be able to protrude outside the housing 40 from a cutout portion in the inner circumferential surface of the through-hole 62. The movable member 64a comes into contact with the fingers that are pressed against the inner circumferential surface of the through-hole 62 by the band 80.

[0038] FIG. 8 shows a state in which the tactile sensation providing component 64 is driven, causing the movable member 64a to protrude from the inner circumferential surface of the through-hole 62. For example, the tactile sensation providing component 64 may have a pneumatically driven balloon as the movable member 64a, and the balloon may be inflated or deflated to provide a tactile sensation to the user's fingers. The tactile sensation providing component 64 may also be a vibrating component such as a voice coil motor (VCM). The tactile sensation providing component 64 is driven by an application, and the user can be provided with a tactile sensation from the movable member 64a, thereby enhancing the user's sense of immersion in the virtual world. Note that the movable member 64a may not contact the user's fingers, but may instead vibrate the inner circumferential surface of the through-hole 62 in the radial direction to provide a tactile stimulus to the fingers.

[0039] As a result of various experiments and investigations, the present inventors have found that setting the diameter of the circular through-hole 62 within the range of 16 mm to 25 mm allows the fingers of many users to be held in a suitable position. In particular, they have found that setting the diameter of the through-hole 62 to 24 mm to 25 mm allows the input device 20 to be formed compactly while being able to hold the fingers of almost all users in a suitable position. They have also found that providing options for the band length of the through-hole 62 allows the user to insert their fingers into the through-hole 62 comfortably.

[0040] 9(a) and 9(b) show examples of adjusting the length of the band 80. Compared to the state shown in Fig. 5, Fig. 9(a) shows a state in which the band length at the through-hole 62 is longer, and Fig. 9(b) shows a state in which the band length at the through-hole 62 is further longer. Since finger thickness varies from person to person, providing several band length options allows users to use the input device 20 with a band length that suits them.

[0041] Fig. 10 shows a modified example of the structure for holding the user's fingers. Fig. 10 is an explanatory diagram for explaining the modified example of the holding structure, and does not show the operation members or the back surface 54. However, when the input device 20 employs this holding structure, the housing 40 does not need to have a curved surface of the back surface 54 or a lower surface 56, and therefore the input device 20 can be made more compact.

[0042] In this example, one end of the band 80 is fixed to the fixing portion 42 of the housing 40, and the other end passes through a temporary fastening portion 44 of the housing 40, leaving it open. This holding structure makes it possible to adjust the length of the band between the fixing portion 42 and the temporary fastening portion 44. A user can insert their fingers into the loop formed by the band 80 and pull the other end of the band 80 to bring their fingers into close contact with the surface of the housing 40. The temporary fastening portion 44 may have a fastening structure that fixes the position of the band 80, or the position of the band 80 may be fixed by friction between the band 80 and the temporary fastening portion 44.

[0043] 11 shows functional blocks of the input device 20. The input device 20 includes a control device 90, a directional operator 70, operation buttons 72, 74, 76, and 78, a tactile sense providing component 64, and a communication device 92. The directional operator 70 and operation buttons 72, 74, 76, and 78 are operation members that can be operated by the user. When the user operates the operation members, the control device 90 acquires operation data of the operation members, and the communication device 92 transmits the operation data to the information processing device 10. While an application is running, the user wears the two input devices 20 on the index fingers of both hands to operate the application. It is preferable that different functions be assigned to each of the multiple operation members provided on the two input devices 20.

[0044] 12 shows functional blocks of the information processing device 10. The information processing device 10 includes a processing unit 200 and a communication unit 202. The processing unit 200 includes an acquisition unit 210, a control unit 220, a gaze direction determination unit 230, an image generation unit 232, a sound generation unit 234, and a provision unit 236. The communication unit 202 has a function of transmitting and receiving data between the HMD 100 and the input device 20. The acquisition unit 210 includes an IMU data acquisition unit 212, an image data acquisition unit 214, and an operation data acquisition unit 216. The control unit 220 has a function of executing an application, and includes a virtual object control unit 222 and a drive data generation unit 224.

[0045] The functions of the components in the information processing device 10 shown in Figure 12 may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), a CPU (a Central Processing Unit), conventional circuits, and / or combinations thereof, configured or programmed to perform the functions described herein. A processor is considered to be a circuit or processing circuitry including transistors and other circuits. A processor may also be a programmed processor that executes a program stored in a memory.

[0046] In this specification, a circuit, unit, or means is hardware that is programmed to realize or performs the described functions, which may be any hardware disclosed herein or any hardware known to be programmed to realize or perform the described functions.

[0047] When the hardware is a processor that is considered to be a type of circuit, the circuit, means, or unit may be a combination of hardware and software used to configure the hardware and / or processor. In an embodiment, the control unit 220 executes an application that moves the virtual hand 22 in the virtual space in accordance with the movement of the user's hand in the real space and causes the virtual hand 22 to act on a virtual object placed in the virtual space.

[0048] The IMU data acquisition unit 212 acquires sensor data from the IMU 124 of the HMD 100 worn by the user at a predetermined cycle and supplies the data to the gaze direction determination unit 230. For example, the cycle at which the HMD 100 transmits IMU data may be set to 11.25 ms. The gaze direction determination unit 230 determines the gaze direction and position of the user in the virtual three-dimensional space based on the IMU data of the HMD 100.

[0049] The gaze direction determination unit 230 acquires posture information of the HMD 100 using sensor data from the triaxial angular velocity sensor. The gaze direction determination unit 230 also calculates the amount of movement from a reference position using sensor data from the triaxial acceleration sensor to acquire position information of the HMD 100. The gaze direction determination unit 230 may also acquire posture information and position information of the HMD 100 using image data captured by the imaging device 14. The gaze direction determination unit 230 converts the posture information of the HMD 100 into the user's gaze direction in virtual three-dimensional space and converts the position information of the HMD 100 into the user's current position in virtual three-dimensional space. When the gaze direction determination unit 230 supplies the gaze direction and current position to the image generation unit 232, the image generation unit 232 determines the orientation and position of a virtual camera in the virtual three-dimensional space based on the gaze direction and current position, and generates an application image. At this time, the sound generation unit 234 generates an application sound synchronized with the application image. The providing unit 236 transmits the application image and application sound to the HMD 100 and the output device 15 via the communication unit 202 .

[0050] The image data acquisition unit 214 acquires image data obtained by capturing images of the real space by the imaging device 14 at a predetermined period and supplies the image data to the virtual object control unit 222. For example, the imaging device 14 may capture images of the real space every 1 / 60 seconds, and the image data acquisition unit 214 may receive the image data every 1 / 60 seconds. The virtual object control unit 222 has a function of estimating the posture of the user's fingers included in the captured images.

[0051] In recent years, AI engines capable of analyzing human images and estimating bones have been developed, and the virtual object control unit 222 may estimate the posture of the user's fingers by using an existing AI engine. The virtual object control unit 222 has a function of estimating the posture of the user's hand using image data captured in real space, and reflecting the estimated posture of the user's hand in the posture of a virtual hand (virtual hand 22 in FIG. 1 ), which is a virtual object.

[0052] The operation data acquisition unit 216 acquires operation data transmitted from the input device 20 and supplies it to the virtual object control unit 222. For example, functions for controlling the virtual hand 22 or other virtual objects are assigned to the operation buttons 72, 74, 76, and 78, and the virtual object control unit 222 may proceed with the application based on the operation data. For example, in the example shown in FIG. 1 , if the user presses a predetermined operation member (e.g., the operation button 72 shown in FIG. 6( a)) when the virtual hand 22 touches the virtual button 26, the virtual object control unit 222 may recognize that the user has indicated "Yes" and proceed with the application.

[0053] For example, in an application in which a user experiences a virtual space, the user can grab and lift a virtual object placed in the virtual space by moving the virtual hand 22. For example, if a cup is placed in the virtual space, the user brings the virtual hand 22 close to the virtual cup, fits the fingers of the hand to the shape of the virtual cup, and lifts the virtual cup.

[0054] When the virtual hand 22 touches the virtual cup, the drive data generation unit 224 generates drive data for driving the tactile sense providing component 64 in the input device 20, and the providing unit 236 transmits the drive data to the input device 20 via the communication unit 202. When the communication device 92 in the input device 20 receives the drive data, the control device 90 drives the tactile sense providing component 64 to provide a tactile sensation to the user's fingers. This allows the user to recognize that the virtual hand 22 is touching the virtual cup.

[0055] The present disclosure has been described above based on the embodiments. These embodiments are merely examples, and those skilled in the art will understand that various modifications are possible in the combination of each component and each processing process, and that such modifications are also within the scope of the present disclosure. The input device 20 may be used as an input tool for hand interaction, but it can also be used as an input tool for other applications. In addition, although the embodiment has been described in which the control unit 220 executes a VR application, it may also execute applications other than VR applications.

[0056] 13( a) and 13(b) show an example of an accessory device 300 that is detachable from the input device 20. The accessory device 300 includes a connection part 302 that connects to the input device 20, and a housing 304. The connection part 302 is connected to the input device 20 in a manner that prevents it from rotating relative to the input device 20. In the embodiment, the connection part 302 is connected to a connection port 66 formed in the housing 40 of the input device 20 in a manner that prevents it from rotating relative to the input device 20. The connection port 66 is a USB female connector formed on the bottom surface 56 of the housing 40, and the connection part 302, which is a USB male connector, is inserted into the connection port 66. Note that a magnet or the like may be used to connect the housing 304 to the input device 20, but the housing 304 must be connected to the input device 20 in a manner that prevents it from rotating relative to the input device 20.

[0057] Fig. 14 shows a state in which the accessory device 300 connected to the input device 20 is being held. The housing 304 is held by fingers other than those wearing the input device 20. In the example shown in Fig. 14, the index finger is inserted into the through-hole 62 of the input device 20, and the middle finger and ring finger are holding the housing 304. By holding the housing 304 with the user's fingers, it is possible to prevent the input device 20 from rotating around the index finger wearing the input device 20.

[0058] 15 shows functional blocks of the accessory device 300. The accessory device 300 includes a control unit 310, a tactile sense presentation component 312, and a battery 314 inside a housing 304, and an operation member 316 on the surface of the housing 304. The battery 314 not only supplies power to the components inside the accessory device 300, but also to the input device 20. When the accessory device 300 is connected to the input device 20, the control unit 310 may use the battery 314 to charge an internal battery (not shown) of the input device 20. By enabling charging from the accessory device 300, the capacity of the internal battery of the input device 20 can be reduced, thereby realizing a more compact input device 20.

[0059] The tactile sensation providing component 312 may be a vibrating component such as a voice coil motor (VCM), and vibrates the surface of the housing 304 from inside the housing 304. When the communication device 92 in the input device 20 receives drive data, the control unit 310 may drive the tactile sensation providing component 312 to provide a tactile sensation to the fingers of the user holding the housing 304.

[0060] The operation member 316 may be, for example, a push-down operation button. By providing the operation member 316 on the accessory device 300, it is possible to increase the variety of operation inputs.

[0061] The present disclosure can be applied to the technical field of input devices used to operate applications and / or accessory devices connected to input devices.

[0062] 1...information processing system, 10...information processing device, 20...input device, 40...housing, 42...fixing portion, 44...temporary fastening portion, 50...front surface, 52...upper surface, 54...rear surface, 56...lower surface, 62...through hole, 64...tactile presentation component, 64a...movable member, 66...connection port, 70...direction operator, 72, 74, 76, 78...operation button, 80...band, 82...main body component, 84...exterior component, 86a...convex portion, 86b...concave portion, 88a...convex portion, 88b...concave portion, 90...control device, 92...communication device, 100...HMD, 102...output mechanism unit, 104...wearing mechanism unit, 106...wearing band, 108...housing, 120...control unit, 122...storage unit, 1 24: IMU, 126: microphone, 128: communication control unit, 130: display panel, 130a: left eye display panel, 130b: right eye display panel, 132: sound output unit, 200: processing unit, 202: communication unit, 210: acquisition unit, 212: IMU data acquisition unit, 214: image data acquisition unit, 216: operation data acquisition unit, 220: control unit, 222: virtual object control unit, 224: drive data generation unit, 230: gaze direction determination unit, 232: image generation unit, 234: sound generation unit, 236: provision unit, 300: accessory device, 302: connection component, 304: housing, 310: control unit, 312: tactile presentation component, 314: battery, 316: operation member.

Claims

1. An input device used to operate an application, comprising a housing having a through-hole for inserting a user's finger, the housing having a first surface, a second surface adjacent to the first surface, and a third surface adjacent to the second surface, and an operating member to be operated by the user is provided on each of the first surface, the second surface, and the third surface.

2. The input device according to claim 1, wherein the first surface, the second surface, and the third surface each have a flat surface.

3. The input device according to claim 1, wherein the second surface is provided with a direction operator for inputting a direction.

4. The input device according to claim 3, wherein operation buttons are provided on the first surface and the third surface.

5. An input device as described in claim 1, characterized in that it has a fourth surface adjacent to the first surface, and a finger other than the finger inserted into the through hole abuts against the fourth surface to suppress rotation of the housing.

6. The input device according to claim 5, wherein the fourth surface has a flat surface, and the finger of the other hand abuts against the flat surface of the fourth surface.

Images