Information processing system and information processing method

Abstract

This disclosure relates to an information processing system and an information processing method with which it is possible to issue a command as intended by a user. This information processing system is provided with a command issuing unit that, when a first operation and a second operation detected at different timings are detected by an input device attached to a part further to the tip side than the wrist of a user, controls issuance of a command corresponding to the first operation and a command corresponding to the second operation on the basis of the priority between the first operation and the second operation. This disclosure can be applied to, for example, a ring-type input device used for operation of an HMD.

Description

Information processing system and information processing method

[0001] The present technology relates to an information processing system and an information processing method, and more particularly to an information processing system and an information processing method that enable a user to issue commands as intended.

[0002] Conventionally, a ring-shaped input device that is worn on a user's index finger has been proposed (see, for example, Patent Document 1). In a small input device such as a ring-shaped input device, on which it is difficult to arrange multiple buttons, a touch operation on a touchpad (for example, a swipe operation), a click operation on a touchpad, a motion operation of moving the finger or the entire hand wearing the ring-shaped input device, etc. are detected, and a command is issued based on the detected operation.

[0003] International Publication No. 2023 / 286316

[0004] For example, when a user clicks on the top side of the touchpad and then moves their finger to click on the bottom side of the touchpad, a swipe operation is detected. In this case, different types of operations may conflict with each other, resulting in the issuance of a command that the user did not intend.

[0005] The present technology has been developed in light of such circumstances, and enables a user to issue commands as intended.

[0006] An information processing system according to one aspect of the present technology includes a command issuing unit that, when a first operation and a second operation having different detection timings are detected by an input device worn before a user's wrist, controls the issuance of commands corresponding to the first operation and the second operation, respectively, based on the priority between the first operation and the second operation.

[0007] An information processing method according to one aspect of the present technology includes, when a first operation and a second operation having different detection timings are detected by an input device worn before a user's wrist, controlling the issuance of commands corresponding to the first operation and the second operation, respectively, based on the priority between the first operation and the second operation.

[0008] In one aspect of the present technology, when a first operation and a second operation having different detection timings are detected by an input device worn before a user's wrist, issuance of commands corresponding to the first operation and the second operation, respectively, is controlled based on the priority between the first operation and the second operation.

[0009] 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment of the present technology. FIG. 1 is a diagram illustrating details of an input device. FIG. 2 is a diagram illustrating an example of a touch operation. FIG. 3 is a diagram illustrating an example of a click operation. FIG. 4 is a first diagram illustrating an example of a motion operation. FIG. 5 is a second diagram illustrating an example of a motion operation. FIG. 6 is a third diagram illustrating an example of a motion operation. FIG. 7 is a diagram illustrating an example of assignment of commands to various operations. FIG. 8 is a block diagram illustrating example configurations of an input device and an information processing device. FIG. 9 is a diagram illustrating an example of conflicting operations. FIG. 10 is a flowchart illustrating processing performed by the input device to avoid conflict between a click operation and a swipe operation in an edge area. FIG. 11 is a diagram illustrating details of an input device according to a second embodiment. FIG. 12 is a diagram illustrating an example of a pressure distribution detected by a pressure sensor. FIG. 13 is a diagram illustrating an example of a directional operation. FIG. 14 is a diagram illustrating an example of a pressure distribution actually detected when a directional operation of up, down, left, or right is performed. FIG. 15 is a diagram illustrating a method of identifying the direction of a directional operation. FIG. 16 is a diagram illustrating an example of a confirm operation. FIG. 17 is a diagram illustrating an example of an insensitive period. FIG. 18 is a flowchart illustrating processing performed by the input device to avoid conflict between a confirm operation and a directional operation. FIG. 19 is a diagram illustrating a method of applying pressure to an edge area of ​​a pressure sensor. FIG. 19 is a diagram illustrating a modified configuration of a pressure sensor. FIG. 19 is a diagram illustrating an example of a motion operation in which the direction of movement of the operating hand is different. FIG. 10 is a diagram illustrating a first example of controlling the timing of issuing a command based on the amount of movement of an operating hand. FIG. 11 is a diagram illustrating a second example of controlling the timing of issuing a command based on the amount of movement of an operating hand. FIG. 12 is a flowchart illustrating a process in which an input device detects a motion operation. FIG. 13 is a flowchart illustrating another process in which an input device detects a motion operation. FIG. 14 is a flowchart illustrating a process performed by an information processing system. FIG. 15 is a block diagram illustrating an example of the hardware configuration of a computer.

[0010] Hereinafter, embodiments of the present technology will be described in the following order: 1. First embodiment 2. Second embodiment 3. Details of motion operation 4. Other

[0011] 1. First Embodiment FIG. 1 is a diagram illustrating an example of the configuration of an information processing system 1 according to an embodiment of the present technology.

[0012] The information processing system 1 includes an input device 11 and an information processing device 12 .

[0013] The input device 11 is worn by the user on a part of the wrist (for example, on a finger) and is used to operate the information processing device 12. The input device 11 detects an operation performed by the user and transmits a command based on the detected operation to the information processing device 12.

[0014] The information processing device 12 is a device that presents information to a user. The information processing device 12 presents visual information, auditory information, vibration information, etc. to the user. Hereinafter, information presented to the user will also be referred to as presented information.

[0015] The information processing device 12 is configured, for example, as an optically transmissive or non-transmissive head-mounted display (HMD) worn on the user's head. For example, the information processing device 12 is realized by a glasses-type device having optically transmissive display units 57L and 57R in the lens unit. The display units 57L and 57R display information of a virtual space (hereinafter referred to as a virtual object) superimposed on the real space in a state where the real space is directly visible to the user's eyes. The information processing device 12 provides the user with so-called augmented reality (AR).

[0016] Hereinafter, when there is no need to distinguish between the display unit 57L and the display unit 57R, they will be referred to as the display unit 57.

[0017] The information processing device 12 that provides AR is not limited to optically transparent devices, but may also be a non-transparent device using a video see-through display. In a video see-through display, an image captured in the direction directly in front of the user (image of real space) is displayed in real time, and virtual objects are superimposed on the image. Furthermore, by providing the video see-through display with a configuration that dynamically blocks light from the real space, it is possible to switch between a transparent display (optical see-through display) and a non-transparent display.

[0018] 1, the display unit 57 of the information processing device 12 displays, for example, virtual objects VO1 to VO3 for presenting various types of information. The virtual objects may be two-dimensional content, three-dimensional objects, or UIs (User Interfaces). Examples of two-dimensional content include websites, social networking service (SNS) screens, photos, videos, and e-books.

[0019] The input device 11 may be used not only to operate an HMD, but also to operate home appliances such as a TV (television receiver), a refrigerator, and an air conditioner.

[0020] FIG. 2 is a diagram illustrating the details of the input device 11. As shown in FIG.

[0021] 2A, the input device 11 has a shape that can be worn on a user's finger. For example, the input device 11 has a ring shape that is intended to be worn on the proximal or middle joint of the index finger.

[0022] The ring shape may be a closed loop shape or a partially open, non-closed loop shape (C-ring). By using a non-closed loop shape, it is possible to accommodate slight differences in finger sizes. To accommodate different finger sizes, a deformable portion may be provided in part of the input device 11.

[0023] A touchpad 103 is provided on the side surface of the housing of the input device 11 at a position facing a finger other than the finger on which the input device 11 is worn when the input device 11 is worn. For example, when the input device 11 is worn on the index finger, the touchpad 103 is located at a position facing the thumb, so that the user can operate the touchpad 103 with the thumb.

[0024] The touchpad 103 has a function of detecting a touch operation. A touch operation is an operation performed by contacting the operation surface of the touchpad 103, and includes, for example, a tap operation, a double tap operation, a flick operation, a swipe operation, and a hold operation (long press).

[0025] The touchpad 103 may be implemented using a resistive film, capacitive, ultrasonic, or optical system, and any of these systems may be used in this embodiment. A plurality of nodes are arranged on the detection surface of the touchpad 103, and the touchpad 103 detects contact with the user for each node. The touchpad 103 outputs touch information indicating the detection result of the user's contact for each node. Alternatively, the detection surface of the touchpad 103 is configured to detect touch coordinates on the X-axis (horizontal direction) and Y-axis (vertical direction), and the input device 11 can divide the touchpad into a plurality of touch areas based on the touch coordinates on the X-axis and Y-axis, as shown below, and process the touchpad. The touchpad 103 can be considered a contact sensor that detects the position of contact made by the user on the surface of the housing of the input device 11.

[0026] FIG. 3 is a diagram showing an example of a touch operation.

[0027] As shown in the balloons in FIG. 3, the operation surface of the touchpad 103 is divided into five areas: a central area 103C, an upper edge area 103U, a left edge area 103L, a right edge area 103R, and a lower edge area 103B.

[0028] The upper edge area 103U is an area that includes the upper edge of the operation surface of the touchpad 103, and the lower edge area 103B is an area that includes the lower edge of the operation surface of the touchpad 103. The left edge area 103L is an area that includes the left edge of the operation surface of the touchpad 103, excluding the upper edge area 103U and the lower edge area 103B. The right edge area 103R is an area that includes the right edge of the operation surface of the touchpad 103, excluding the upper edge area 103U and the lower edge area 103B. The central area 103C is an area that includes the center of the operation surface of the touchpad 103, excluding the upper edge area 103U, the left edge area 103L, the right edge area 103R, and the lower edge area 103B.

[0029] In the above example, a configuration in which the operation surface of the touchpad 103 is divided into multiple touch areas has been described. However, as another example, touchpads for the edge areas may be provided in addition to the touchpad 103. In this case, the touchpad 103 is dedicated to the central area 103C. Touchpads for the upper edge area 103U, the lower edge area 103B, the left edge area 103L, and the right edge area 103R are provided on each side of the touchpad 103. The operation surfaces of the touchpad 103 for each edge area are, for example, provided at an angle relative to the operation surface of the touchpad 103.

[0030] A tap operation is, for example, an operation of touching the operation surface of the touchpad 103 and then immediately releasing it, and a double tap operation is, for example, an operation of touching the operation surface of the touchpad 103 and then immediately releasing it twice in succession.

[0031] A flick operation is, for example, an operation in which a finger is swung in one of the up, down, left, and right directions while touching the operation surface of the touchpad 103 (a finger is removed from the touchpad surface at a certain speed or more in one of the up, down, left, and right directions). For example, FIG. 3A shows a flick operation in which a finger is swung downward while touching the upper edge area 103U. A swipe operation is, for example, an operation in which a finger is slid in one of the up, down, left, and right directions for a certain distance or more, as if tracing the operation surface of the touchpad 103, while keeping the finger in contact with the operation surface of the touchpad 103. For example, FIG. 3B shows a swipe operation in which a finger is slid from the upper edge area 103U to the lower edge area 103B while keeping the finger in contact with the operation surface of the touchpad 103.

[0032] The hold operation is an operation of touching the operation surface of the touchpad 103 for a predetermined period of time or longer.

[0033] 2B, a tactile switch 104 is provided in the approximate center of the back side (finger side) of the touchpad 103. The tactile switch 104 has a function of detecting a click operation. The tactile switch 104 outputs switch information indicating whether the switch is on or off (whether or not a click operation has been performed).

[0034] FIG. 4 is a diagram showing an example of a click operation.

[0035] A click operation is, for example, an operation of pressing the operation surface of the touchpad 103. In the example of Fig. 4, the user clicks the upper edge area 103U. In this case, the input device 11 can detect that the user has clicked the upper edge area 103U based on touch information indicating that the user is in contact with the upper edge area 103U and switch information indicating that the user has pressed the touchpad 103.

[0036] 2B, a motion sensor 105 is provided, for example, at the upper end of the housing of the input device 11. The motion sensor 105 has a function of detecting a motion operation of moving a finger or the entire hand wearing the ring-shaped input device.

[0037] The motion sensor 105 is configured by an IMU (Inertial Measurement Unit) including a three-axis acceleration sensor, a three-axis gyro sensor, a geomagnetic sensor, etc. The motion sensor 105 outputs motion information indicating the detection result of the movement (posture) of the input device 11, that is, the movement (posture) of the finger or the entire hand wearing the input device 11.

[0038] 5 to 7 are diagrams showing examples of motion operations.

[0039] For example, as shown in Figure 5, when a user holds the touchpad 103 with their thumb and performs a motion operation of twisting the hand wearing the input device 11 (hereinafter referred to as the operating hand) clockwise, a command to change the volume of the content is issued in the input device 11.

[0040] For example, as shown in the upper part of Figure 6, when a user taps and holds the touchpad 103 with their thumb (tap and hold operation) and then performs a stroke operation by moving their operating hand as if drawing a character in the air, a command to launch an app (application) corresponding to the character is issued in the input device 11.

[0041] For example, as shown in the lower part of Figure 6, when a stroke operation to draw the letter "-" is performed, a command to display a list of apps is issued, and when a stroke operation to draw the letter "U" is performed, a command to launch the YouTube (registered trademark) app is issued.

[0042] For example, as shown in FIG. 7, when a user holds the touchpad 103 with his / her thumb and performs a motion operation of pointing the upper end of the input device 11 up, down, left, or right, a command to move the cursor up, down, left, or right is issued in the input device 11.

[0043] If motion operations are detected constantly, commands unintended by the user may be issued. Therefore, it is desirable that the input device 11 start detecting motion operations using a touch operation or a click operation as a trigger, as described with reference to Figures 5 to 7.

[0044] FIG. 8 is a diagram showing an example of assignment of commands to various operations.

[0045] For various operations using the input device 11, commands corresponding to the buttons provided on a TV remote controller, for example, are assigned as shown in FIG.

[0046] For example, a "power" command that controls power on / off of the information processing device 12 is assigned to a double tap operation on the upper edge area.

[0047] A motion operation of twisting the operating hand while holding the upper edge area 103U is assigned a "TV button" command for displaying a TV broadcast on the display unit 57. A motion operation of twisting the operating hand while holding the left edge area 103L is assigned an "input switch" command for switching the display content of the display unit 57.

[0048] A "setting button" command that displays a setting menu on the display unit 57 is assigned to a motion operation of twisting the operating hand while holding the right edge area 103R. A "number key appearance" command that displays number keys on the display unit 57 is assigned to a motion operation of twisting the operating hand while holding the bottom edge area 103B.

[0049] Clicking on the top, bottom, left, and right edge areas is assigned a "cross key" command for moving the cursor displayed on the display unit 57 up, down, left, and right. Clicking on the center area 103C is assigned a "confirm" command for confirming (selecting) the object indicated by the cursor.

[0050] A double tap on the central area 103C is assigned a "Back" command for executing a "Back" function in a TV broadcast, a website, etc. A double tap on the bottom edge area 103B is assigned a "Home" command for displaying a home screen on the display unit 57.

[0051] A "volume" command for changing the volume of the content is assigned to a motion operation of twisting the operating hand while holding the central area 103C. For example, when the motion operation of twisting the operating hand clockwise is performed, the volume of the content increases, and when the motion operation of twisting the operating hand counterclockwise is performed, the volume of the content decreases.

[0052] A "mute" command for muting the volume of content is assigned to a snap operation in which the wrist of the operating hand is quickly bent left and right while holding the central area 103C.

[0053] A "channel up" command and a "channel down" command for changing the channel of a TV broadcast are assigned to an up-down swipe operation. For example, when an up-down swipe operation is performed, a TV broadcast with a channel number higher than the currently displayed TV broadcast is displayed on the display unit 57, and when a down-down swipe operation is performed, a TV broadcast with a channel number lower than the currently displayed TV broadcast is displayed on the display unit 57. Furthermore, if a "channel up" command is issued once by a swipe operation and the user continues to touch the touchpad 103 with their finger in the same position, the "channel up" command may be issued continuously every time a certain time (e.g., 0.5 seconds) has elapsed until the finger is released.

[0054] When you tap and hold in the central area, the stroke operation of drawing letters in the air is assigned the "Launch App" command, which launches the app corresponding to the letters.

[0055] In the following, the command groups assigned to various touch operations will be referred to as touch-related commands, the command groups assigned to various click operations will be referred to as click-related commands, and the command groups assigned to various motion operations will be referred to as motion-related commands.

[0056] In this way, the user can indirectly perform various operations on the virtual objects displayed on the display unit 57 and the information processing device 12 itself via the input device 11 .

[0057] In the information processing system 1 of the present technology, the ring-shaped input device 11 worn on a finger can be operated with other fingers, allowing for more natural, low-stress, and less cumbersome operations to be performed in daily life.

[0058] For example, with methods that require users to raise their arms in front of them and directly manipulate virtual objects with their hands, users may feel uncomfortable because they stand out in crowded streets. Also, with methods that require users to directly tap on glasses-type devices, their arms can easily get tired because they have to bring their hands up to their face. Furthermore, with methods that use handheld remote controls, carrying and removing the remote control is a hassle.

[0059] In contrast, the information processing system 1 of the present technology uses an input device 11 that is attached to a finger and can be operated at hand even with the hand down, thereby reducing the inconvenience described above and enabling more natural and low-stress operations to be performed in daily life.

[0060] FIG. 9 is a block diagram showing an example of the configuration of the input device 11 and the information processing device 12.

[0061] As shown in FIG. 9, the input device 11 includes a communication unit 101 , a control unit 102 , a touchpad 103 , a tactile switch 104 , a motion sensor 105 , a vibration unit 106 , and a storage unit 107 .

[0062] The communication unit 101 is connected to an external device via wired or wireless communication to transmit and receive data. For example, the communication unit 101 can be connected to the information processing device 12 via wireless communication such as Wi-Fi (registered trademark) or Bluetooth (registered trademark).

[0063] The control unit 102 functions as an arithmetic processing unit and a control unit, and controls the overall operation of the input device 11 in accordance with various programs. The control unit 102 is realized by electronic circuits such as a CPU (Central Processing Unit) or a microprocessor. The control unit 102 may include a ROM (Read Only Memory) that stores programs to be used, arithmetic parameters, etc., and a RAM (Random Access Memory) that temporarily stores parameters that change as appropriate.

[0064] The control unit 102 includes a user operation detection unit 111 and a vibration control unit 112 .

[0065] The user operation detection unit 111 detects an operation performed by the user on the input device 11 based on touch information supplied from the touchpad 103, switch information supplied from the tactile switch 104, and motion information supplied from the motion sensor 105. The user operation detection unit 111 functions as a command issuing unit that issues a command based on the detected operation, and controls transmission of the command to the information processing device 12 via the communication unit 101.

[0066] The vibration control unit 112 controls the vibration of the vibration unit 106 in accordance with the vibration control signal transmitted from the information processing device 12 via the communication unit 51. The vibration caused by the vibration unit 106 is presented to the user as feedback for an operation performed by the user, for example.

[0067] The touchpad 103 is a sensor that detects the user's hand movements related to touch operations (contact with the touchpad 103), and supplies the control unit 102 with touch coordinates or touch information indicating the detection results of contact with the user for each of the multiple installed nodes.

[0068] The tactile switch 104 is a sensor that detects the user's hand movement (pressing on the touchpad 103) for a click operation, and supplies the control unit 102 with switch information indicating whether the switch is on or off.

[0069] The motion sensor 105 is a sensor that detects the user's hand movements related to motion operations (movements of the fingers and entire hand of the user wearing the input device 11), and supplies motion information indicating the detection results of the movement of the input device 11 to the control unit 102.

[0070] The vibration unit 106 is configured as, for example, a vibration actuator, and presents tactile information (tactile stimulation to the skin) to the user. Note that the vibration method is not particularly limited. There may be a single vibration unit 106, or multiple vibration units 106 may be provided.

[0071] The storage unit 107 includes a ROM for storing programs and calculation parameters used in the processing of the control unit 102, and a RAM for temporarily storing parameters that change as needed.

[0072] The configuration of the input device 11 is not limited to the example shown in Fig. 9. The input device 11 does not need to include all of the components shown in Fig. 9, and may include other components. For example, the input device 11 may include one or more other sensors. For example, the input device 11 may further include a distance sensor. The distance sensor may be an optical sensor such as a ToF (Time of Flight) sensor.

[0073] The input device 11 may further include a microphone. The microphone collects sounds around the input device 11 and supplies audio data representing the collected sounds to the control unit 52. This allows the user to perform audio input using the input device 11. For example, when the user continues to click on the central area of ​​the touchpad 103 for a certain period of time or more, the input device 11 enters audio input mode, and the user can bring the input device 11 close to their mouth to perform audio input. The user's action of bringing the input device 11 close to their mouth may be detected by a motion sensor, an optical sensor, or the like, and the audio input mode may be activated when the user brings the input device 11 close to their mouth. To improve the accuracy of detecting the action of bringing the input device 11 close to their mouth, a touch operation or a click operation may be used as a trigger to start detecting the action of bringing the input device 11 close to their mouth.

[0074] The information processing device 12 is composed of, for example, a communication unit 51 , a control unit 52 , a gaze detection unit 53 , an outward-facing camera 54 , an operation input unit 55 , a microphone 56 , a display unit 57 , an audio output unit 58 , and a memory unit 59 .

[0075] The communication unit 51 is connected to an external device via a wired or wireless connection to transmit and receive data. For example, the communication unit 51 can be connected to the input device 11 via wireless communication such as Wi-Fi or Bluetooth. The communication unit 51 can also be connected to the Internet via a wired / wireless local area network (LAN), Wi-Fi, Bluetooth, or a mobile communication network (such as LTE (Long Term Evolution), 3G (third-generation mobile communication system), 4G (fourth-generation mobile communication system), or 5G (fifth-generation mobile communication system)), and can transmit and receive data to and from a server on the network.

[0076] The control unit 52 functions as an arithmetic processing unit and a control device, and controls the overall operation of the information processing device 12 in accordance with various programs. The control unit 52 is realized by electronic circuits such as a CPU or a microprocessor. The control unit 52 may include a ROM for storing the programs to be used, arithmetic parameters, etc., and a RAM for temporarily storing parameters that change as appropriate.

[0077] The control unit 52 includes a user operation detection unit 61 and a presentation control unit 62 .

[0078] The user operation detection unit 61 detects an operation performed by the user on the information processing device 12 based on the gaze information supplied from the gaze detection unit 53, the captured image supplied from the outward camera 54, and the operation input information supplied from the operation input unit 55. The user operation detection unit 61 issues a command based on the detected operation and supplies it to the presentation control unit 62.

[0079] The presentation control unit 62 controls the presentation of presentation information in accordance with commands transmitted from the input device 11 via the communication unit 51 or commands supplied from the user operation detection unit 61. The presentation control unit 62 functions as a processing execution unit that executes processing in accordance with commands issued by the input device 11 or the user operation detection unit 61. For example, the presentation control unit 62 controls the display unit 57 to display virtual objects on the display unit 57, and further controls selection, confirmation, scrolling, zooming, and the like for the virtual objects.

[0080] The presentation control unit 62 can control the display position and orientation of a virtual object superimposed on real space based on the analysis result of an image captured by the outward-facing camera 54 (an image that includes at least the user's field of view). The presentation control unit 62 can also control the display position and orientation of a virtual object in consideration of the user's own position and orientation.

[0081] For example, analysis of the captured image (object recognition, etc.) is performed by the presentation control unit 62. Detection of the user's own position and posture is performed by various sensors provided in the information processing device 12. Examples of the various sensors include the outward-facing camera 54, a gyro sensor, an acceleration sensor, a geomagnetic sensor, and a position measurement unit. The position measurement unit may be a measurement unit that measures an absolute position (for example, position measurement using a Global Navigation Satellite System (GNSS)), or a measurement unit that measures a relative position (for example, position measurement using a Wi-Fi or Bluetooth signal).

[0082] The data of the virtual object to be displayed may be data stored in the storage unit 59, may be data received from an external device via the communication unit 51, or may be data generated by the presentation control unit 62. Examples of the external device include a server on the Internet and a communication terminal owned by the user (for example, a smartphone, a tablet terminal, a wearable device, etc.).

[0083] The presentation control unit 62 may control power on / off, adjust the volume, and control display on / off in accordance with commands based on user operations. The presentation control unit 62 may also control the sound output from the sound output unit 58.

[0084] The presentation control unit 62 may control vibration information presented by the vibration unit 106 of the input device 11. For example, the presentation control unit 62 controls transmission of a vibration control signal to the input device 11 via the communication unit 51.

[0085] The gaze detection unit 53 detects information related to the user's gaze (gaze information). The gaze detection unit 53 may include an inward-facing camera that captures an image of the user's eyes when the information processing device 12 is worn, or may include an electromyographic sensor that detects the electrooculography. The gaze detection unit 53 detects the direction of the user's gaze based on information acquired by various sensors, and supplies gaze information indicating the detection result of the user's gaze direction to the control unit 52. There are no particular limitations on the method for detecting the gaze direction.

[0086] Outward-facing camera 54 is a camera provided in information processing device 12 to capture an image in the direction directly in front of the user when information processing device 12 is worn. The angle of view of outward-facing camera 54 includes at least the user's field of view. Outward-facing camera 54 includes a lens system composed of an imaging lens, an aperture, a zoom lens, a focus lens, etc., a drive system that performs focusing and zooming operations on the lens system, and a solid-state image sensor array that photoelectrically converts the imaging light obtained by the lens system to generate an image signal. The solid-state image sensor array is realized by a CCD (Charge Coupled Device) sensor array, a CMOS (Complementary Metal Oxide Semiconductor) sensor array, etc.

[0087] The operation input unit 55 accepts operation input by the user and supplies operation input information indicating the input operation to the control unit 52. The operation input unit 55 is realized by, for example, a push button, a touch sensor, an optical sensor, or the like.

[0088] The microphone 56 collects sounds around the information processing device 12 and supplies audio data representing the collected sounds to the control unit 52 .

[0089] The display unit 57 is realized by, for example, a transmissive display. A transmissive display is a display that can deliver light from real space directly to the user's eyes. The user can directly view the real space through the transmissive display. The transmissive display may be, for example, an optical see-through display. Known types of optical see-through displays, including a half mirror type, a light guide plate type, and a retinal direct imaging type, may be adopted.

[0090] The audio output unit 58 has a function of outputting audio, and is configured by headphones, earphones, a bone conduction speaker, or the like.

[0091] The memory unit 59 includes a ROM for storing programs and calculation parameters used in the processing of the control unit 52, an EEPROM (Electrically Erasable and Programmable Read Only Memory) for storing image data acquired by the outward-facing camera 54 and audio data acquired by the microphone 56, and a RAM (Random Access Memory) for temporarily storing parameters that change as appropriate.

[0092] The configuration of the information processing device 12 is not limited to the example shown in Fig. 9. For example, the information processing device 12 does not need to include all of the components shown in Fig. 9, or may include other components. Furthermore, the information processing device 12 may be composed of multiple devices.

[0093] However, different types of operations may conflict with each other in the input device 11, resulting in the issuance of a command not intended by the user. The types of operations include, for example, a touch operation, a click operation, and a motion operation.

[0094] FIG. 10 is a diagram showing an example of conflicting operations.

[0095] For example, a click operation and a swipe operation on an edge area may conflict with each other, as indicated by the white arrow A1 in Fig. 10. Specifically, when a user clicks on the upper edge area 103U and then moves his / her finger to click on the lower edge area 103B, a downward swipe operation may be detected, and a "channel" command may be issued against the user's intention.

[0096] 10, a hold operation on the central area 103C, a motion operation of twisting the operating hand, and a click operation on the central area 103C may conflict with each other. Specifically, when the user attempts to perform a motion operation while holding the central area 103C, a click operation on the central area 103C may be detected, and a "Confirm" command may be issued against the user's intention.

[0097] 10, a conflict may occur between a tap-and-hold operation in central area 103C and a stroke operation for drawing a character in the air and a double-tap operation in central area 103C. Specifically, when a user attempts to perform a stroke operation while tapping and holding central area 103C, the user may accidentally double-tap and then hold, which may result in the double-tap operation in central area 103C being detected and a "back" command being issued against the user's intention.

[0098] Therefore, when a first operation and a second operation having different detection timings are detected, the user operation detection unit 111 of the input device 11 of the present technology controls the issuance of commands corresponding to the first operation and the second operation, respectively, based on the priority between the first operation and the second operation. Here, the operation detection timing can be any timing between the start and end of the user's operation. This allows the input device 11 to avoid conflicting operations and issue commands as intended by the user.

[0099] Specifically, the user operation detection unit 111 sets an insensitive period for low-priority operations based on the detection timing of a high-priority operation, and if the detection timing of a low-priority operation is included in the insensitive period, it does not issue a command corresponding to the low-priority operation.

[0100] For example, when the user operation detection unit 111 detects a click operation in the edge area and issues a command corresponding to the click operation, the user operation detection unit 111 does not issue a command corresponding to a swipe operation until the user's finger is removed from the touchpad 103. In other words, the period from when the click operation in the edge area is detected until the user's finger is removed from the touchpad 103 is set as an insensitive period for the swipe operation.

[0101] This prevents a conflict between a click operation and a swipe operation in the edge area. In this example, it is predetermined that the priority of a click operation in the edge area is higher than the priority of a swipe operation.

[0102] Furthermore, the user operation detection unit 111 issues a command corresponding to a click operation in the central area 103C not when pressing the central area 103C starts but when pressing the central area 103C ends. If the user operation detection unit 111 detects a motion operation after pressing the central area 103C starts, the user operation detection unit 111 does not issue a command corresponding to a click operation in the central area 103C even if pressing the central area 103C ends. In other words, the period from when the motion operation is detected to when the end of pressing (release of the click operation) is detected is set as an insensitive period for click operations in the central area 103C.

[0103] This avoids conflict between the hold operation and the motion operation of twisting the operating hand in the central area 103C and the click operation in the central area 103C. In this example, it is predetermined that the priority of the hold operation and the motion operation in the central area 103C is higher than the priority of the click operation in the central area 103C.

[0104] If the user operation detection unit 111 does not detect a motion operation within a predetermined period (e.g., one second) from the start of pressing the central area 103C, it considers the timing one second after the start of pressing to be the start of a click operation. If pressing the central area 103C continues for a predetermined period (e.g., 1.5 seconds) from the start of the click operation, the user operation detection unit 111 detects a long click operation and issues a command corresponding to the long click operation.

[0105] Furthermore, if the user operation detection unit 111 does not detect a hold operation on the central area 103C within a predetermined period (e.g., 500 milliseconds) from the timing of detecting the double tap operation on the central area 103C, the user operation detection unit 111 confirms the detection of the double tap operation on the central area 103C and issues a command corresponding to the double tap operation on the central area 103C. On the other hand, if the user operation detection unit 111 detects a hold operation on the central area 103C within that period, the user operation detection unit 111 cancels the detection of the double tap operation and accepts the input of a stroke operation.

[0106] In other words, the period from the detection timing of a double tap operation in the central area 103C to the detection timing of a hold operation is set as an insensitive period to the double tap operation in the central area 103C.

[0107] This avoids conflict between the tap-and-hold operation in the central area 103C and the stroke operation of drawing a character in the air and the double-tap operation in the central area 103C. In this example, it is predetermined that the priority of the tap-and-hold operation and the motion operation in the central area 103C is higher than the priority of the double-tap operation in the central area 103C.

[0108] Here, with reference to the flowchart of FIG. 11, a process performed by the input device 11 to avoid conflict between a click operation and a swipe operation in an edge area will be described.

[0109] In step S1 , the user operation detection unit 111 acquires touch information and switch information from the touchpad 103 and the tactile switch 104 .

[0110] In step S2, the user operation detection unit 111 determines whether or not the user is touching the touchpad 103 based on the touch information.

[0111] If it is determined in step S2 that a touch operation has been detected, then in step S3, the user operation detection unit 111 determines whether or not a click operation has been detected based on the touch information and the click information.

[0112] If it is determined in step S3 that a click operation has been detected, in step S4, the user operation detection unit 111 issues an "up / down key" command or a "confirm" command depending on the area on the touchpad 103 that the user's finger is touching, and sets a click-related command issued flag to true. Thereafter, the process returns to step S1, and the subsequent processes are repeated.

[0113] On the other hand, if it is determined in step S3 that a click operation has not been detected, the user operation detection unit 111 determines whether the amount of change in the touch position on the touchpad 103 is greater than or equal to a threshold value and whether the click-related command issued flag is False.

[0114] If it is determined in step S5 that the amount of change in the touch position on the touchpad 103 is equal to or greater than the threshold and the click-related command issued flag is False, the user operation detection unit 111 issues a "channel" command corresponding to the swipe operation in step S6. Thereafter, the process returns to step S1, and the subsequent processes are performed.

[0115] On the other hand, if it is determined in step S5 that the amount of change in the touch position on the touchpad 103 is less than the threshold value, or if it is determined in step S5 that the click-related command issued flag is true, the process returns to step S1 and subsequent processes are performed.

[0116] If it is determined in step S2 that the user is not touching the touchpad 103, the user operation detection unit 111 sets the click-related command issued flag to false in step S7. Thereafter, the process returns to step S1, and the subsequent processes are repeated.

[0117] In the above process, if the amount of change in the touch position on the touchpad 103 is equal to or greater than the threshold, a swipe operation is detected and a "channel" command is issued. However, if a click operation is detected (a click-related command is issued), a click-related command issued flag is set to true, and while the click-related command issued flag is true, the "channel" command is not issued even if the amount of change in the touch position on the touchpad 103 is equal to or greater than the threshold.

[0118] This avoids conflict between the click operation and the swipe operation, and enables the input device 11 to issue a command as intended by the user.

[0119] 2. Second Embodiment FIG. 12 is a diagram illustrating details of an input device 11 according to a second embodiment.

[0120] 12A, a pressure sensor 201 for detecting pressure distribution is provided on the side surface of the housing of the input device 11 at a position facing a finger other than the finger on which the input device 11 is worn when the input device 11 is worn. A tactile switch 104 is provided in approximately the center of the back side (finger side) of the pressure sensor 201.

[0121] The pressure sensor 201 has a function of detecting a touch operation, similar to the touch pad 103 (FIG. 2). For example, as shown in B of FIG. 12, when the input device 11 is worn on the index finger of the user, the pressure sensor 201 can detect how the user touches the pressure sensor 201 with the thumb.

[0122] FIG. 13 is a diagram showing an example of a pressure distribution detected by the pressure sensor 201. In FIG.

[0123] A plurality of nodes are arranged on the detection surface of the pressure sensor 201 (the surface of the housing of the input device 11), and the pressure sensor 201 can be said to be a pressure distribution sensor that detects the pressure applied by the user for each node, as shown in Fig. 13. In the example of Fig. 13, the detection values ​​of the pressure detected at each of the 5 x 5 nodes are shown as a bar graph.

[0124] The pressure sensor 201 outputs touch information indicating the detection result of the pressure distribution as shown in FIG.

[0125] 14 , the user can perform a directional operation (touch operation) to indicate one of the four directions (up, down, left, right) by tilting the finger in one of four directions (up, down, left, right) while the finger is touching the pressure-sensitive sensor 201. The user can indicate any direction by simply changing the way the finger is tilted without removing the finger from the pressure-sensitive sensor 201 each time the direction to be indicated is changed.

[0126] As shown on the right side of FIG. 14, when a finger is tilted in a certain direction, ideally, pressure should be applied only to the area of ​​the pressure sensor 201 on the side of the finger's tilt direction.

[0127] FIG. 15 is a diagram showing an example of pressure distribution that is actually detected when up, down, left, or right directional operations are performed.

[0128] In Fig. 15, when a directional operation is performed to indicate one of the four directions (up, down, left, and right), the pressure detected by each of the 5 x 6 nodes is represented by a shade of color. In the example of Fig. 15, nodes to which weak pressure is being applied are represented by dark-colored squares, and nodes to which strong pressure is being applied are represented by light-colored squares.

[0129] As shown in the upper left side of FIG. 15, when an upward directional operation is performed, pressure is detected not only in the area at the top end of the pressure sensor 201 but also in the central and left and right end areas.

[0130] 15, when a downward directional operation is performed, pressure is detected not only in the bottom area of ​​the pressure sensor 201, but also in the central and left and right end areas. When the input device 11 is worn on the index finger of the right hand, the base of the thumb that touches the pressure sensor 201 is located lower and to the right of the input device 11, so that the pressure applied to the right node in the bottom area of ​​the pressure sensor 201 is stronger than the pressure applied to the left node.

[0131] As shown in the lower left side of FIG. 15, when a leftward directional operation is performed, pressure is detected not only in the leftmost area of ​​the pressure sensor 201 but also in the central and upper and lower end areas.

[0132] 15, when a directional operation is performed in the right direction, pressure is detected not only in the rightmost area of ​​the pressure sensor 201 but also in the central and upper and lower end areas. When the input device 11 is worn on the index finger of the right hand, the pressure applied to the lower node in the rightmost area of ​​the pressure sensor 201 is stronger than the pressure applied to the upper node.

[0133] In this way, because the directional operation is performed with the thumb of the right hand, a bias toward the lower right occurs in the pressure distribution detected by the pressure sensor 201. Therefore, if the direction of the directional operation is identified based on the center of gravity of the pressure distribution detected by the pressure sensor 201, there is a possibility that a command unintended by the user may be issued.

[0134] Therefore, the user operation detection unit 111 of the input device 11 of the present technology determines the direction of the directional operation based on the difference (ratio) between the detection values ​​of the nodes in the top row of the pressure sensor 201 and the detection values ​​of the nodes in the bottom row, and the difference (ratio) between the detection values ​​of the nodes in the leftmost column and the detection values ​​of the nodes in the rightmost column.

[0135] FIG. 16 is a diagram illustrating a method for identifying the direction of a directional operation.

[0136] First, the user operation detection unit 111 calculates the ratio between the sum of the detection values ​​of each node in the leftmost column of the pressure sensor 201 (detection value of the left edge area 201L) and the sum of the detection values ​​of each node in the rightmost column of the pressure sensor 201 (detection value of the right edge area 201R), as shown on the left side of Figure 16.

[0137] In addition, the user operation detection unit 111 calculates the ratio between the sum of the detection values ​​of each node in the top row of the pressure sensor 201 (detection value of the upper edge area 201U) and the sum of the detection values ​​of each node in the bottom column of the pressure sensor 201 (detection value of the lower edge area 201B), as shown on the right side of Figure 16.

[0138] Next, the user operation detection unit 111 compares the ratio of the detection values ​​of the left edge area 201L and the right edge area 201R with the ratio of the detection values ​​of the top edge area 201U and the bottom edge area 201B, and determines the axis corresponding to the larger ratio (the vertical axis or the horizontal axis) as the axis of the directional operation. In the example of Fig. 16, the vertical axis is determined as the axis of the directional operation.

[0139] Finally, the user operation detection unit 111 compares the ratios of the detection values ​​of the upper edge area 201U and the lower edge area 201B, and identifies the direction corresponding to the edge area with the larger detection value as the direction of the directional operation. In the example of Fig. 16, the downward direction is identified as the direction of the directional operation.

[0140] As described above, the input device 11 can identify the direction of the directional operation intended by the user based on the pressure distribution detected by the pressure sensor 201, and issue the command as intended by the user.

[0141] The user can move a cursor or the like up, down, left, or right by touching the finger to the pressure sensor 201 and tilting the finger in four directions (up, down, left, right), and can confirm the object indicated by the cursor by pressing the center of the pressure sensor 201 (click operation).

[0142] A pressing operation of pressing the pressure sensor 201 is detected by the tactile switch 104 provided on the back side of the pressure sensor 201, as shown on the left side of Fig. 17. As shown on the right side of Fig. 17, when a user presses the pressure sensor 201 (tactile switch 104), pressure is applied to each node of the pressure sensor 201, and therefore, the input device 11 may detect a directional operation along with the pressing operation. In other words, there is a possibility that the pressing operation and the directional operation may conflict with each other.

[0143] Therefore, the user operation detection unit 111 of the input device 11 sets the period before and after the timing of detecting a pressing operation as an insensitive period, and cancels detection of a directional operation during the insensitive period. Here, the timing of detecting a pressing operation can be any timing between the timing when the user starts pressing the pressure sensor 201 and the timing when the user stops pressing. This allows the input device 11 to avoid conflict between a pressing operation and a directional operation, and to issue a command as intended by the user.

[0144] FIG. 18 is a diagram illustrating an example of an insensitive period.

[0145] As shown in A of Figure 18, if no pressing operation is detected by the tactile switch 104 during the waiting period t0 from the detection timing T1 of a leftward directional operation to the issuance timing T2 of the command, the user operation detection unit 111 issues a directional command corresponding to the leftward directional operation at timing T2.

[0146] 18B, if the tactile switch 104 detects that the user has started pressing the pressure sensor 201 at timing T3 within the waiting period t0 from timing T1 when a leftward directional operation is detected to timing T2 when the command is issued, the user operation detection unit 111 cancels the detection of the leftward directional operation and does not issue a leftward directional command at timing T2. Here, the period from timing T1 to timing T3 can be considered the dead period before the detection of the pressing operation.

[0147] Furthermore, as shown in C of Figure 18, if a leftward directional operation is detected by the pressure sensor 201 at timing T12 within the insensitive period t10 after timing T11 when the tactile switch 104 detects that the user has stopped pressing the pressure sensor 201, the user operation detection unit 111 cancels the detection of the leftward directional operation and does not issue a leftward directional command.

[0148] In this way, in the input device 11, if the timing of detecting a directional operation falls within a dead period set before or after the timing of detecting a confirmation operation (for example, before the user starts pressing the pressure sensor 201 or after the user stops pressing the pressure sensor 201), the detection of the directional operation is canceled. Also, if a confirmation operation is not detected in the waiting period after the timing of detecting a directional operation, the detection of the directional operation is confirmed.

[0149] This allows the input device 11 to avoid conflict between the decision operation and the direction operation, and to issue the command as intended by the user. Note that the decision operation may be detected by the pressure sensor 201 instead of the tactile switch 104.

[0150] The process performed by the input device 11 to avoid conflict between the decision operation and the direction operation will be described with reference to the flowchart of FIG.

[0151] In step S21, the user operation detection unit 111 determines whether or not a directional operation has been detected based on the touch information supplied from the pressure-sensitive sensor 201, and waits until it is determined that a directional operation has been detected.

[0152] If it is determined in step S21 that a directional operation has been detected, in step S22, the user operation detection unit 111 determines, based on the switch information supplied from the tactile switch 104, whether or not a decision operation has been detected in the waiting period t0 following the detection timing T1 of the directional operation.

[0153] If it is determined in step S22 that a confirmation operation has been detected during the waiting period t0, in step S23, the user operation detection unit 111 cancels the detection of the directional operation and issues a "confirm" command corresponding to the confirmation operation.

[0154] On the other hand, if it is determined in step S22 that the confirmation operation has not been detected during the waiting period t0, then in step S24, the user operation detection unit 111 issues a "cross key" command corresponding to the directional operation.

[0155] After the process of step S23 or step S24 is performed, the process returns to step S21, and the subsequent processes are repeated.

[0156] In the input device 11 according to the second embodiment, when a user applies pressure to the edge area of ​​the pressure-sensitive sensor 201 by tilting their finger as shown on the left side of Fig. 20, the directional operation intended by the user is likely to be detected. On the other hand, when a user applies pressure to the edge area of ​​the pressure-sensitive sensor 201 by sliding their finger over the pressure-sensitive sensor 201 as shown on the right side of Fig. 20, the directional operation intended by the user is unlikely to be detected.

[0157] FIG. 21 is a diagram showing a modified example of the configuration of the pressure sensor 201.

[0158] In the example on the left side of FIG. 21 , the back surface (finger side) of the pressure-sensitive sensor 201 is joined onto a base member 211 , and a cap member 212 is provided on the surface of the pressure-sensitive sensor 201 .

[0159] The cap member 212 has, for example, a frustum shape, and is provided to reduce the contact area between the pressure sensor 201 and the user's finger.

[0160] By providing such a cap member 212, it is possible to apply pressure to the edge area of ​​the pressure sensor 201 in an appropriate manner, even if the user moves their finger in a sliding manner over the pressure sensor 201, as shown on the right side of Figure 21.

[0161] 3. Details of Motion Operation Both the input device 11 according to the first embodiment and the input device 11 according to the second embodiment are provided with a motion sensor 105, and motion operations are detected by the motion sensor 105. Since it is difficult to mount multiple buttons or the like on the ring-shaped input device 11, it is required to be able to detect motion operations.

[0162] The method by which the input device 11 detects a motion operation will be described in detail below.

[0163] In the input device 11, motion operations are distinguished according to the direction of movement of the operating hand, for example.

[0164] FIG. 22 is a diagram showing examples of motion operations in which the direction of the operating hand movement is different.

[0165] 22A shows a left-right motion operation in which the operating hand is moved left and right (the wrist is bent left and right) with the thumb facing up. For example, a "mute" command is assigned to the left-right motion operation.

[0166] In addition, a "cross key" command for moving the cursor displayed on the display unit 57 left and right may be assigned to a left and right motion operation.

[0167] 22B shows an up-and-down motion operation in which the operating hand is moved up and down with the thumb facing upward. For example, a "cross key" command for moving the cursor up and down is assigned to the up-and-down motion operation.

[0168] 22C shows a rotational motion operation in which the operating hand is rotated clockwise or counterclockwise (twisting the wrist). For example, a "volume" command is assigned to the rotational motion operation.

[0169] In the input device 11, the timing of issuing a command corresponding to a motion operation is controlled based on, for example, the amount of movement of the operating hand.

[0170] 23 is a diagram illustrating a first example of controlling the timing of issuing a command based on the amount of movement of the operating hand. In the example of FIG. 23, a case where a motion operation is performed in the left and right directions will be described.

[0171] 23, the user operation detection unit 111 determines the reference posture as the posture of the operating hand (input device 11) when the user's thumb touches the touchpad 103. As shown in #1 of Fig. 23, the state in which the posture of the operating hand is the reference posture is also referred to as an idle state.

[0172] In the input device 11, the amount of movement of the operating hand (amount of change in posture) is indicated by the angle of the input device 11 relative to the center position of the user's wrist. Here, the angle of the input device 11 in the idle state is set to 0°.

[0173] As shown in #2 of FIG. 23, when the angle indicating the amount of movement of the operating hand is less than a threshold angle (for example, 20°), the user operation detection unit 111 does not issue a command.

[0174] 23, when the angle indicating the amount of movement of the operating hand becomes equal to or greater than the threshold angle, the user operation detection unit 111 issues a single command corresponding to a directional operation in the left-right direction. If the state in which the angle indicating the amount of movement of the operating hand is equal to or greater than the threshold angle is maintained for a certain period of time, the user operation detection unit 111 continuously issues commands corresponding to motion operations in the left-right direction.

[0175] When the angle indicating the amount of movement of the operating hand becomes less than the threshold angle, or when the operating hand becomes idle, the user operation detection unit 111 does not issue a command until the angle indicating the amount of movement of the operating hand becomes equal to or greater than the threshold angle again.

[0176] 24 is a diagram illustrating a second example of controlling the timing of issuing a command based on the amount of movement of the operating hand. In the example of FIG. 24, a case where a motion operation is performed in the left and right directions will be described.

[0177] As shown in FIG. 24, the user operation detection unit 111 determines the posture of the operating hand (input device 11) when the user's thumb touches the touch pad 103 as the reference posture.

[0178] If the angle indicating the amount of movement of the operating hand is less than a threshold angle (for example, 20°), the user operation detection unit 111 does not issue a command.

[0179] Each time the angle indicating the amount of movement of the operating hand becomes a multiple of the threshold angle, the user operation detection unit 111 issues a single command corresponding to a motion operation in the left-right direction. For example, the user operation detection unit 111 issues a command when the angle indicating the amount of movement of the operating hand becomes 20°, and issues a command again when the angle indicating the amount of movement of the operating hand becomes 40°.

[0180] The user operation detection unit 111 controls the issuance of commands so as to prevent duplicate commands from being issued when the user repeatedly assumes a certain posture (for example, a posture in which the angle indicating the amount of movement of the operating hand is 20°). Specifically, when the user operation detection unit 111 issues a command at the timing when the angle indicating the amount of movement of the operating hand reaches a certain angle, the user operation detection unit 111 does not issue a command until the angle indicating the amount of movement of the operating hand reaches another angle at which a command is issued.

[0181] For example, the user operation detection unit 111 issues a command when the angle indicating the amount of movement of the operating hand reaches 20°, and then does not issue a command even if the angle indicating the amount of movement of the operating hand reaches 20° again. The user operation detection unit 111 issues a command when the angle indicating the amount of movement of the operating hand reaches 40°, and then issues a command when the angle indicating the amount of movement of the operating hand reaches 20° again.

[0182] The process of detecting a motion operation by the input device 11 will be described with reference to the flowchart of FIG.

[0183] In step S41, the user operation detection unit 111 breaks down the movement amount and speed of the operating hand into each axis (horizontal direction, depth direction, and vertical direction) based on the motion information supplied from the motion sensor 105.

[0184] In step S42, the user operation detection unit 111 determines whether or not the user has performed a motion operation based on the amount and speed of movement of the operating hand for each axis.

[0185] If it is determined in step S42 that the user is not performing a motion operation, the process returns to step S41, and the subsequent processes are performed.

[0186] On the other hand, if it is determined in step S42 that the user has performed a motion operation, in step S43, the user operation detection unit 111 distinguishes the motion operation performed by the user based on the direction of the movement of the operating hand.

[0187] In step S44, the user operation detection unit 111 controls the timing of issuing a command corresponding to the motion operation performed by the user based on the amount of movement of the operating hand. After that, the process returns to step S41, and the subsequent processes are repeated.

[0188] Another process in which the input device 11 detects a motion operation will be described with reference to the flowchart of FIG.

[0189] In step S61, the user operation detection unit 111 determines whether or not the user has performed a motion operation based on the motion information supplied from the motion sensor 105, and waits until the user has performed a motion operation.

[0190] In step S62, the user operation detection unit 111 calculates the direction and amount of movement of the operating hand based on the motion information.

[0191] In step S63, the user operation detection unit 111 distinguishes the motion operation performed by the user based on the direction of the movement of the operating hand.

[0192] In step S64, the user operation detection unit 111 controls the timing of issuing a command corresponding to the motion operation based on the amount of movement of the operating hand. After that, the process returns to step S61, and the subsequent processes are repeated.

[0193] In the processing of Figure 25, the movement amount and speed of the operating hand are broken down by axis, and then a determination is made as to whether or not the user has performed a motion operation and a distinction is made between motion operations. However, in the processing of Figure 26, the movement amount and speed of the operating hand are not broken down by axis, and a determination is made as to whether or not the user has performed a motion operation and a distinction is made between motion operations.

[0194] The user operation detection unit 111 can more accurately determine whether the user has performed a motion operation and calculate the direction and amount of movement of the operating hand by not breaking down the amount and speed of the movement of the operating hand for each axis. The user operation detection unit 111 distinguishes between motion operations and controls the timing of issuing commands based on the accurately calculated direction and amount of movement of the operating hand, thereby making the content and timing of issuing commands more consistent with the user's intention.

[0195] As described above, in the input device 11 of the present technology, a motion operation is distinguished (detected) based on the direction of movement of the operating hand, and the timing of issuing a command is controlled based on the amount of movement of the operating hand. The input device 11 can provide a user with a system that enables intuitive operation of a virtual object or the information processing device 12 by moving their hand or fingers.

[0196] <4. Others> - Operation of the Overall Information Processing System The processing performed by the information processing system 1 will be described with reference to the flowchart of FIG.

[0197] In step S101, the gaze detection unit 53 of the information processing device 12 detects the gaze of the user and acquires gaze information.

[0198] In step S102, the user operation detection unit 111 of the input device 11 detects an operation performed by the user and issues a command based on the detected operation. In step S102, the processing described above with reference to Figures 11, 19, 25, and 26 is performed, for example.

[0199] In step S103 , the communication unit 51 of the information processing device 12 receives the command issued by the input device 11 .

[0200] In step S104, the presentation control unit 62 of the information processing device 12 controls the presentation of the presentation information in accordance with the command issued by the input device 11. The presentation of the presentation information is also controlled based on the line-of-sight information acquired in step S101.

[0201] In step S105, the presentation control unit 62 feeds back vibration information to the input device 11 as necessary. Specifically, the presentation control unit 62 generates a vibration control signal for presenting vibration corresponding to the operation content or the presentation information, and transmits the signal to the input device 11 via the communication unit 51. The vibration control unit 112 of the input device 11 vibrates the vibration unit 106 based on the vibration control signal transmitted from the information processing device 12 via the communication unit 101, thereby presenting the vibration information to the user.

[0202] Thereafter, the process returns to step S101, and the subsequent processes are repeated.

[0203] Regarding the division of processing between the input device 11 and the information processing device 12, the division of processing between the input device 11 and the information processing device 12 can be changed as appropriate.

[0204] For example, a function equivalent to the user operation detection unit 111 of the input device 11 may be provided in the information processing device 12, and the information processing device 12 may detect operations performed by the user based on touch information acquired by the touchpad 103, switch information acquired by the tactile switch 104, and motion information acquired by the motion sensor 105.

[0205] In this case, for example, the communication unit 101 of the input device 11 transmits touch information, switch information, and motion information to the information processing device 12 .

[0206] The input device 11 and the information processing device 12 may share the task of detecting user operations.

[0207] Regarding the computer, the above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware or a general-purpose personal computer.

[0208] FIG. 28 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0209] A CPU (Central Processing Unit) 501 , a ROM (Read Only Memory) 502 , and a RAM (Random Access Memory) 503 are interconnected by a bus 504 .

[0210] An input / output interface 505 is also connected to the bus 504. An input unit 506 including a keyboard, a mouse, etc., and an output unit 507 including a display, a speaker, etc. are connected to the input / output interface 505. Also connected to the input / output interface 505 are a storage unit 508 including a hard disk, a nonvolatile memory, etc., a communication unit 509 including a network interface, etc., and a drive 510 that drives removable media 511.

[0211] In a computer configured as described above, the CPU 501 performs the above-described series of processes by, for example, loading a program stored in the storage unit 508 into the RAM 503 via the input / output interface 505 and the bus 504 and executing it.

[0212] The program executed by the CPU 501 is installed in the storage unit 508 by being recorded on, for example, a removable medium 511 or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting.

[0213] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0214] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are housed in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0215] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0216] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0217] Each step described in the above flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

[0218] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0219] <Examples of Combinations of Configurations> The present technology can also have the following configurations.

[0220] (1) An information processing system comprising: a command issuing unit that, when a first operation and a second operation having different detection timings are detected by an input device worn before a user's wrist, controls the issuance of commands corresponding to the first operation and the second operation based on the priority between the first operation and the second operation. (2) The information processing system described in (1), wherein the command issuing unit sets an insensitive period for a lower priority operation based on the detection timing of the higher priority operation, and does not issue a command corresponding to the lower priority operation when the detection timing of the lower priority operation falls within the insensitive period. (3) The information processing system described in (1) or (2), wherein the first operation and the second operation are different types of operations. (4) The information processing system described in (3), wherein the types of operations include a touch operation, a click operation, and a motion operation. (5) The information processing system according to any one of (1) to (4), further comprising: a first sensor provided on a housing of the input device and detecting a hand movement of the user related to the first operation; and a second sensor provided on a housing of the input device and detecting a hand movement of the user related to the second operation. (6) The information processing system according to (5), further comprising an operation detection unit that detects the first operation based on a detection result by the first sensor and detects the second operation based on a detection result by the second sensor. (7) The information processing system according to (6), wherein the first sensor is a contact sensor that detects a contact position by the user on a surface of the housing of the input device. (8) The information processing system according to (6), wherein the first sensor is a pressure distribution sensor that detects pressure applied to each node arranged on the surface of the housing of the input device. (9) The information processing system according to (8), wherein the first operation is a directional operation indicating one of four directions: up, down, left, and right, and the operation detection unit identifies the direction of the directional operation based on a difference in detection values ​​between up and down and a difference in detection values ​​between left and right in the pressure distribution sensor. (10) The information processing system according to (8) or (9), wherein a cap member having a frustum shape is provided on a surface of the pressure distribution sensor.(11) The information processing system according to any one of (6) to (10), wherein the operation detection unit detects the second operation based on a direction of movement of the user's hand, and the command issuing unit controls timing of issuing a command corresponding to the second operation based on an amount of movement of the user's hand. (12) The information processing system according to any one of (1) to (11), wherein the input device has a ring-shaped housing worn on the user's finger. (13) The information processing system according to any one of (1) to (12), further comprising a processing execution unit that executes processing in accordance with the command issued by the command issuing unit. (14) An information processing method including: when a first operation and a second operation having different detection timings are detected by an input device worn before a user's wrist, controlling issuance of commands corresponding to the first operation and the second operation, respectively, based on priority between the first operation and the second operation. (15) The information processing method according to (14), wherein the control of issuing commands corresponding to the first operation and the second operation includes: setting an insensitive period for the low-priority operation based on the detection timing of the high-priority operation; and not issuing a command corresponding to the low-priority operation if the detection timing of the low-priority operation falls within the insensitive period. (16) The information processing method according to (14) or (15), wherein the first operation and the second operation are different types of operations. (17) The information processing method according to (16), wherein the types of operations include a touch operation, a click operation, and a motion operation. (18) The information processing method according to any of (14) to (17), further includes: detecting the first operation based on a detection result by a first sensor provided on a housing of the input device and detecting a hand movement of the user related to the first operation; and detecting the second operation based on a detection result by a second sensor provided on a housing of the input device and detecting a hand movement of the user related to the second operation. (19) The information processing method according to any one of (14) to (18), wherein the input device has a ring-shaped housing that is worn on the finger of the user.(20) The information processing method according to any one of (14) to (19), further comprising: executing a process in accordance with the issued command.

[0221] REFERENCE SIGNS LIST 1 Information processing system, 11 Input device, 12 Information processing device, 51 Communication unit, 52 Control unit, 53 Gaze detection unit, 54 Outward camera, 55 Operation input unit, 56 Microphone, 57 Display unit, 58 Audio output unit, 59 Memory unit, 61 User operation detection unit, 62 Presentation control unit, 101 Communication unit, 102 Control unit, 103 Touch pad, 104 Tactile switch, 105 Motion sensor, 106 Vibration unit, 107 Memory unit, 111 User operation detection unit, 112 Vibration control unit, 201 Pressure sensor, 211 Base member, 212 Cap member

Claims

1. An information processing system comprising a command issuing unit that, when a first operation and a second operation having different detection timings are detected by an input device worn before the user's wrist, controls the issuance of commands corresponding to the first operation and the second operation, respectively, based on the priority between the first operation and the second operation.

2. The information processing system of claim 1, wherein the command issuing unit sets an insensitive period for the low priority operation based on the detection timing of the high priority operation, and does not issue a command corresponding to the low priority operation if the detection timing of the low priority operation is included in the insensitive period.

3. The information processing system according to claim 1, wherein the first operation and the second operation are different types of operations.

4. The information processing system according to claim 3, wherein the types of operations include a touch operation, a click operation, and a motion operation.

5. The information processing system according to claim 1, further comprising: a first sensor provided on the housing of the input device for detecting the user's hand movement relating to the first operation; and a second sensor provided on the housing of the input device for detecting the user's hand movement relating to the second operation.

6. The information processing system according to claim 5, further comprising an operation detection unit that detects the first operation based on a detection result by the first sensor and detects the second operation based on a detection result by the second sensor.

7. The information processing system according to claim 6, wherein the first sensor is a contact sensor that detects a position on the surface of the housing of the input device that is touched by the user.

8. The information processing system according to claim 6, wherein the first sensor is a pressure distribution sensor that detects pressure applied to each node arranged on the surface of the housing of the input device.

9. The information processing system of claim 8, wherein the first operation is a directional operation indicating one of four directions: up, down, left, and right, and the operation detection unit identifies the direction of the directional operation based on the difference in detection values ​​between up and down and the difference in detection values ​​between left and right in the pressure distribution sensor.

10. The information processing system according to claim 8, wherein a cap member having a frustum shape is provided on the surface of the pressure distribution sensor.

11. The information processing system described in claim 6, wherein the operation detection unit detects the second operation based on the direction of the user's hand movement, and the command issuing unit controls the timing of issuing a command corresponding to the second operation based on the amount of the user's hand movement.

12. The information processing system according to claim 1, wherein the input device has a ring-shaped housing that is worn on the user's finger.

13. The information processing system according to claim 1, further comprising a process execution unit that executes a process in accordance with the command issued by said command issuing unit.

14. An information processing method including, when a first operation and a second operation having different detection timings are detected by an input device worn before the user's wrist, controlling the issuance of commands corresponding to the first operation and the second operation, respectively, based on the priority between the first operation and the second operation.

15. The information processing method of claim 14, wherein the control of issuing commands corresponding to the first operation and the second operation includes: setting an insensitive period for the low priority operation based on the detection timing of the high priority operation; and not issuing a command corresponding to the low priority operation if the detection timing of the low priority operation is included in the insensitive period.

16. The information processing method according to claim 14, wherein the first operation and the second operation are different types of operations.

17. The information processing method according to claim 16, wherein the types of operations include a touch operation, a click operation, and a motion operation.

18. The information processing method according to claim 14, further comprising: detecting the first operation based on a detection result by a first sensor provided on the housing of the input device and detecting the user's hand movement related to the first operation; and detecting the second operation based on a detection result by a second sensor provided on the housing of the input device and detecting the user's hand movement related to the second operation.

19. The information processing method according to claim 14, wherein the input device has a ring-shaped housing that is worn on the user's finger.

20. The information processing method according to claim 14, further comprising: executing processing in accordance with the issued command.

Images