Information processing apparatus, method for controlling information processing apparatus, computer program product, and computer-readable storage medium

Abstract

The invention relates to an information processing apparatus, a control method of the information processing apparatus, a computer program product, and a computer readable storage medium. An information processing apparatus according to the present invention is an information processing apparatus that receives an operation of an operation interface by a user, and includes: an acquisition unit configured to acquire first information related to a contact between a real object and a hand of the user, the real object being different from the operation interface and the information processing apparatus; and a control unit configured to disable a function associated with the operation interface in a case where the first information acquired by the acquisition unit indicates that the hand is in contact with the real object.

Description

Technical Field

[0001] The present invention relates to an information processing device, a control method of the information processing device, a computer program product, and a computer-readable storage medium. Background Art

[0002] In a cross-reality (XR) system that allows users to experience virtual reality, a head-mounted display (HMD) is often used. In addition, a controller can be used to convert the movement of the user's hands into movement in a virtual space (virtual reality space). HMD stands for a glasses-shaped device that includes a small display worn on the user's head.

[0003] Japanese Patent Laid-Open No. 2015-152973 discloses a method in which, when a hand is in contact with a real object, the movement of the real object and the movement of the hand relative to the real object are detected, thereby receiving an input operation that regards the real object as a pointing device such as a mouse, etc. In addition, Japanese Patent Laid-Open No. 2021-8794 discloses a method in which, when a real object exists in a specified area, an operation switch is disabled, thereby preventing a door from opening or closing.

[0004] However, in the technology disclosed in Japanese Patent Laid-Open No. 2015-152973, when the hand is in contact with a real object, the input operation must be enabled. Therefore, even if the movement of the hand is not intended to perform an input operation, the movement of the hand may still be regarded as an input operation. In addition, in the technology disclosed in Japanese Patent Laid-Open No. 2021-8794, when there is no real object in the specified area, the operation switch must be enabled, which may lead to misoperation of the operation switch. Therefore, even if the technology disclosed in Japanese Patent Laid-Open No. 2015-152973 and Japanese Patent Laid-Open No. 2021-8794 is applied to the XR system, misoperation of the controller may occur. Summary of the invention

[0005] The present invention provides a technology capable of preventing erroneous operation of a controller with high accuracy.

[0006] In its first aspect, the present invention provides an information processing device configured to receive a user's operation on an operation interface, the information processing device comprising: an acquisition unit configured to acquire first information related to contact between a real object and the user's hand, the real object being different from the operation interface and the information processing device; and a control unit configured to disable a function associated with the operation interface when the first information acquired by the acquisition unit indicates that the hand is in contact with the real object.

[0007] In its second aspect, the present invention provides a control method for an information processing device, which receives an operation of an operation interface by a user, the control method comprising: acquiring first information related to contact between a real object and a hand of the user, the real object being different from the operation interface and the information processing device; and disabling a function associated with the operation interface when the first information acquired by the acquisition unit indicates that the hand is in contact with the real object.

[0008] In a third aspect, the present invention provides a computer program product, which includes a program for causing a computer to execute the steps of the above control method. In a fourth aspect, the present invention provides a computer readable storage medium, which stores a program for causing a computer to execute the steps of the above control method.

[0009] Further features of the present invention will become apparent from the following description of embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a block diagram showing the configuration of an information processing system according to a first embodiment;

[0011] Figure 2 is a schematic diagram showing a state in which a user wears an HMD and a controller is attached;

[0012] Figure 3 is a schematic diagram showing a display example of a ray;

[0013] Figure 4 is a schematic diagram showing a state where another real object is held by a hand to which a controller is attached;

[0014] Figure 5 is a flowchart showing a disabling process according to the first embodiment;

[0015] Figure 6 is a flowchart showing an activation process according to the first embodiment;

[0016] Figure 7 is a sequence diagram showing a disabling process according to the second embodiment;

[0017] Figure 8 is a flowchart showing a disabling process according to the third embodiment;

[0018] Fig. 9 is a schematic diagram showing a state where a real object comes into contact with a CG object;

[0019] Fig.10 is a flowchart showing an activation process according to a fourth embodiment;

[0020] Fig.11 is a schematic diagram showing a warning screen according to a fifth embodiment; and

[0021] Fig.12 is a flowchart showing a disabling process according to the sixth embodiment. DETAILED DESCRIPTION

[0022] Hereinafter, embodiments of the present invention are described in detail based on the accompanying drawings. Note that the following embodiments are provided as examples of means for implementing the present invention, and can be appropriately deformed or changed according to the configuration of the device or various conditions to which the present invention is applied. In addition, the various embodiments can also be appropriately combined together.

[0023] First embodiment

[0024] System Configuration

[0025] Figure 1 1 is a block diagram showing a configuration of an information processing system according to the first embodiment. The information processing system includes a head mounted display (HMD) 100 as an example of an information processing apparatus and a controller 200 .

[0026] HMD 100 configuration

[0027] The HMD 100 includes a display device 101 and a control device 102 .

[0028] Configuration of the display device 101

[0029] The display device 101 includes a camera unit 111, a display unit 112, and a position and posture detection unit 113. The display device 101 constitutes, for example, a glasses-shaped display portion of the HMD 100. The display device 101 displays a composite image in which a real space image representing a user's field of view (an image obtained by photographing the front range of the HMD 100) is combined with a computer graphic (CG), which is a virtual object in a form corresponding to the user's position and posture. Therefore, a user wearing the HMD 100 can experience various virtual realities (mixed realities), such as virtual realities in which CG is merged with a real space. Note that the display device 101 may also have two display units, one for the right eye and the other for the left eye, to generate a stereoscopic image. In the following description, it is assumed that the display device 101 constitutes the display portion of the HMD 100. However, the display portion may include a tablet terminal or other display device. That is, any display device that is transportable and capable of displaying an image corresponding to the user's field of view may be used as the display device 101.

[0030] The imaging unit 111 is composed of an objective optical system that receives external light (light in real space), an image sensor that converts an optical signal into an electrical signal, etc. The imaging unit 111 captures an image representing a user's field of view (an image in real space).

[0031] The display unit 112 displays an image generated by the control device 102 .

[0032] The position and posture detection unit 113 detects the position and posture of the user in the real space. For example, a position and posture sensor using an acceleration sensor and an angular velocity sensor, a direction sensor using geomagnetism, and a posture sensor using GPS can be used as the position and posture detection unit 113.

[0033] Configuration of the control device 102

[0034] The control device 102 includes a CPU 120, a ROM 130, a RAM 140, and a communication unit 150. As the control device 102, it is assumed that a device having a high-performance computing processing function or a graphic display function is used. The connection between the control device 102 and the display device 101 enables a user wearing the HMD 100 to watch a video of a virtual space. The video of the virtual space may also include a video of a mixed reality world in which the real world and the virtual world are seamlessly merged in real time.

[0035] The image acquisition unit 121 , the position and orientation acquisition unit 122 , and the control unit 123 are control blocks operating within the CPU 120 .

[0036] The image acquisition unit 121 acquires a real-space image captured by the imaging unit 111 of the display device 101 .

[0037] The position and posture acquisition unit 122 acquires information indicating the position and posture of the user detected by the position and posture detection unit 113 of the display device 101. The position and posture acquisition unit 122 may also detect a marker arranged in the real space from the image of the real space acquired by the image acquisition unit 121 to acquire information indicating the position and posture.

[0038] The control unit 123 generates a composite image by combining the image of the real space acquired by the image acquisition unit 121 with the CG, and transmits the generated composite image to the display unit 112 of the display device 101. In addition, the control unit 123 acquires information output from the controller 200 via the communication unit 150, and controls the display device 101 based on the acquired information.

[0039] The ROM 130 is a storage unit that stores information such as CG, etc. Note that the control unit 123 is capable of switching the CG read from the ROM 130 (ie, the CG used to generate a synthesized image).

[0040] The RAM 140 is a working memory. In the RAM 140, constants and variables for operating the control unit 123, or programs read from the ROM 130, etc. are stored.

[0041] The communication unit 150 is an interface for connecting to an external device. The communication unit 150 includes an interface for communicating with an external device through a so-called wireless LAN that complies with the IEEE802.11 standard. In addition, the communication unit 150 includes an interface for communicating with an external device through a so-called Bluetooth (registered trademark) that complies with the IEEE802.15.1 standard. Note that the communication system is not limited to a wireless LAN or Bluetooth (registered trademark). A wired or wireless communication system can be used as long as it can communicate, and any type of communication protocol can be used. The HMD 100 communicates with the controller 200 via the communication unit 150. In addition, the communication unit 150 can communicate with devices other than the controller 200 (such as a smartphone, etc.).

[0042] The display device 101 and the control device 102 are connected so that data communication can be performed with each other. Therefore, the display device 101 and the control device 102 can be connected to each other in a wired or wireless manner. Note that the display device 101 and the control device 102 can be separate devices, or can be integrated into a single unit so that the user can carry it. In addition, some of the processing performed by the display device 101, the control device 102, and the controller 200 can be implemented by an external device (such as a smartphone, a personal computer, a workstation, a video game console, and a cloud server device, etc.).

[0043] Configuration of the controller 200

[0044] The controller 200 includes a CPU 201, a motion detection unit 202, an operation unit 203, a vibration unit 204, a communication unit 205, and a contact detection unit 206. The controller 200 is a device for operating the HMD 100. The CPU 201 controls the respective units of the controller 200.

[0045] The motion detection unit 202 has an inertial measurement unit (IMU; inertial sensor) capable of detecting inertial information (spatial motion amount or angle) of the controller 200. Note that the motion detection unit 202 may also have any device as long as the device does not hinder the miniaturization of the controller 200 and can detect inertial information (information such as a difference in position, velocity, and acceleration).

[0046] The operation unit 203 may include any one of a button, a touch pad, a touch panel, a directional cross key, a joystick, and a trackpad device. For example, the user can display a menu including a pointer on the display device 101 by pressing and holding a button. Thereafter, the user can move the pointer to a desired item by pressing the directional cross key in any direction. Then, the user can perform an action to confirm the selection of the item by pressing a button. For example, the user can switch Figure 3 The display and hiding of the line 300 shown in FIG.

[0047] The vibration unit 204 causes the controller 200 to vibrate. Figure 3 When the wire 300 shown is in contact with the CG 301, the CPU 201 receives a vibration instruction from the HMD 100 via the communication unit 205. In response to the vibration instruction, the CPU 201 controls the vibration unit 204 to vibrate the controller 200. Through the vibration of the controller 200, the user can recognize that the wire 300 has been in contact with the CG 301.

[0048] The communication unit 205 is an interface for connecting with an external device. The controller 200 communicates with the HMD 100 via the communication unit 205. The communication unit 205 has the same configuration as the communication unit 150, for example.

[0049] The contact detection unit 206 is a functional unit for detecting whether the controller 200 is in contact with a real object. The contact detection unit 206 detects whether the controller 200 is in contact with a real object using, for example, a pressure sensor or a contact sensor.

[0050] Method of wearing HMD 100 and attaching controller 200

[0051] Figure 2 is a schematic diagram showing a state in which the user wears the HMD 100 and attaches the controller 200. Figure 2 As shown, the user wears the HMD 100 on the head and attaches the ring controller 200 to the hand 210. The controller 200 has a shape like a ring, so that the user can freely move the hand with the controller 200 attached to the finger. Note that the shape of the controller 200 is a ring, but is not limited to this shape. For example, the controller 200 may also have a shape like a glove that can be attached to the hand. As described above, for the convenience of the user, the controller 200 may be designed to be holdable by the user's hand or attachable to the user's hand. In addition, in the first embodiment, the operation interface for operating the HMD 100 is the controller 200, but is not limited to this. The operation interface may also be the user's hand or finger.

[0052] Line Display

[0053] Figure 3 is a schematic diagram showing an example of a line display. A line is a beam-shaped display item (graphic) indicating a position pointed to by a user. Figure 3 As shown, a line 300 is displayed from the controller 200 attached to the user's hand 210 to the CG 301. The front end position and angle of the line 300 are determined based on information related to the position and posture of the controller 200 attached to the user's hand 210. The user can use the line 300 to operate items in the virtual space (such as CG 301, etc.). Note that in the first embodiment, the line 300 is displayed based on both the position and posture of the controller 200. However, the line 300 may also be displayed based on one of the position and posture of the controller 200.

[0054] Disable the function of controller 200

[0055] Figure 4 2 is a schematic diagram showing a state where another real object (here, the smartphone 400) is held by the hand 210 to which the controller 200 is attached. Figure 3 Similarly, the line 300 is displayed from the controller 200 to the CG 301. However, when the line 300 is displayed while holding the smartphone 400 with the hand 210, the user may feel that the line 300 is troublesome. In addition, when the user operates the smartphone 400 or when the user just wants to touch the smartphone 400, an erroneous operation of the controller 200 may occur. Therefore, in the first embodiment, when the hand to which the controller 200 is attached is in contact with a specified real object, the function of the controller 200 is disabled.

[0056] Figure 5 1 is a flowchart showing the disabling process according to the first embodiment. In step S501 , the control unit 123 acquires a real image (an image of a real space) captured by the imaging unit 111 through the image acquisition unit 121 .

[0057] In step S502, the control unit 123 determines whether the hand to which the controller 200 is attached is in contact with the real object based on the acquired real image. When the hand to which the controller 200 is attached is in contact with the real object, the process proceeds to step S503. Otherwise, the process proceeds to step S501. For example, the control unit 123 may detect the area of ​​the hand and the area of ​​the real object from the real image, and determine whether the hand is in contact with the real object based on the positional relationship between these areas. The control unit 123 may determine that the hand is in contact with the real object when the distance between the area of ​​the hand and the area of ​​the real object is less than a threshold value, and determine that the hand is not in contact with the real object when the distance is at least a threshold value. In addition, the control unit 123 may determine that the hand is in contact with the real object when the area of ​​the hand overlaps with the area of ​​the real object, and determine that the hand is not in contact with the real object when the area of ​​the hand does not overlap with the area of ​​the real object. Note that in step S502, the area of ​​the controller 200 is not determined as the area of ​​the real object.

[0058] In ROM 130, information related to the specified real object is stored. In step S503, the control unit 123 obtains the information related to the specified real object from ROM 130. In the first embodiment, the control unit 123 obtains a disabled image reflecting the specified real object as the information related to the specified real object. The specified real object refers to a real object that disables the function of the controller 200 when the hand to which the controller 200 is attached contacts the real object.

[0059] In step S504, the control unit 123 determines whether the real object reflected in the real image is consistent with the real object reflected in the prohibited image. When the real object reflected in the real image is consistent with the real object reflected in the prohibited image, the process proceeds to step S505. Otherwise, the process proceeds to step S501.

[0060] In step S505, the control unit 123 disables the functions of the controller 200 and ends the disable process. The disabled functions include, for example, a function of displaying lines, a function of receiving an operation on the controller 200, and a function of vibrating the controller 200.

[0061] In the first embodiment, the specified real object refers to a smartphone. The control unit 123 determines whether the real object in contact with the hand is consistent with the smartphone based on the real image captured by the camera unit 111 and the image of the smartphone stored in the ROM 130. When the hand is in contact with the smartphone, the function of the controller 200 is disabled. Figure 4As shown, when the smartphone 400 is held by the hand 210 to which the controller 200 is attached, the user can be prevented from feeling blocked by the wire 300 or from making erroneous operations using the controller 200 with high accuracy. Note that the designated real object is not limited to the smartphone, but may also include objects operated by the user (such as a tablet, keyboard, mouse, car steering wheel and drive, etc.) and objects not operated by the user (such as a mug, etc.). In addition, the designated real object is not limited to one type, but may also include multiple types of real objects.

[0062] Furthermore, in the first embodiment, when the real object in contact with the hand does not coincide with the designated real object, the function of the controller 200 is not disabled. However, when the hand is in contact with a real object other than the controller 200, the function of the controller 200 may be disabled.

[0063] Enable the function of controller 200

[0064] When the hand to which the controller 200 is attached is separated from the real object, the function of the controller 200 is enabled (the disabled state of the function of the controller 200 is canceled). In addition, when the position of the hand to which the controller 200 is attached is outside the camera range of the camera unit 111, making it impossible to determine whether the hand is in contact with the real object, the function of the controller 200 is also enabled.

[0065] Figure 6 1 is a flowchart showing the activation process according to the first embodiment. In step S601 , the control unit 123 acquires a real image captured by the imaging unit 111 through the image acquisition unit 121 .

[0066] In step S602, the control unit 123 determines whether the hand to which the controller 200 is attached is reflected in the real image (whether the area of ​​the hand is included in the real image). When the hand to which the controller 200 is attached is reflected in the real image, the process proceeds to step S604. Otherwise, the process proceeds to step S603.

[0067] In step S603, the control unit 123 enables Figure 5 The functions of the controller 200 are disabled in step S505.

[0068] In step S604, the control unit 123 determines whether the hand to which the controller 200 is attached is in contact with the real object based on the acquired real image. The method for determining whether the hand is in contact with the real object may be the same as the method performed in step S502. When the hand is in contact with the real object, the process proceeds to step S601. When the hand is not in contact with the real object (the hand is separated from the real object), the process proceeds to step S603, thereby enabling the function of the controller 200.

[0069] As described above, when the hand to which the controller 200 is attached is separated from the smartphone after contacting with the smartphone, or when the hand is not reflected in the real image, the once disabled function of the controller 200 is enabled. Therefore, the user can resume various operations using the controller 200.

[0070] Disable some functions of the controller 200

[0071] exist Figure 5 In step S505, the control unit 123 disables all three functions, namely, the function of displaying a line, the function of receiving an operation on the controller 200, and the function of vibrating the controller 200. However, any one of these functions may be disabled, or two or three of these functions may be disabled. In addition, functions associated with the controller 200 other than these functions may also be disabled.

[0072] Second embodiment

[0073] In the first embodiment, it is determined whether the hand to which the controller 200 is attached is in contact with the real object based on the real image captured by the camera unit 111, thereby acquiring information indicating whether the hand is in contact with the real object (determination result). Then, based on the acquired determination result and the information (disable image) related to the specified real object stored in the ROM 130, it is determined whether to disable the function of the controller 200. In the second embodiment, the real object detects the user operation and outputs information (detection result) indicating whether the user is operating the real object. Then, the HMD 100 determines whether to disable the function of the controller 200 based on the information output from the real object. The HMD 100 regards the state in which the user is operating the real object as the state in which the hand is in contact with the real object, and regards the state in which the user is not operating the real object as the state in which the hand is not in contact with the real object. Therefore, even if the information related to the specified real object is not stored in the ROM 130, it is possible to appropriately determine whether to disable the function of the controller 200.

[0074] Figure 7 1 is a sequence diagram showing a disabling process according to the second embodiment. In the second embodiment, the real object refers to an object operable by a user, such as a smartphone 400 or the like. The HMD 100 communicates with the smartphone 400 via the communication unit 150 .

[0075] In step S701, the smartphone 400 detects a user operation on the smartphone 400 (determines whether a user operation has been performed). The smartphone 400 waits for detection of a user operation. After detecting a user operation, the process proceeds to step S702.

[0076] In step S702 , the smartphone 400 notifies (outputs) the HMD 100 of the detection of the user operation.

[0077] In step S703 , the control unit 123 of the HMD 100 , upon receiving a notification (a notification indicating that the smartphone 400 has detected a user operation) from the smartphone 400 via the communication unit 150 , disables the function of the controller 200 .

[0078] Third embodiment

[0079] In the second embodiment, the real object determines whether the user is operating the real object. In the third embodiment, the HMD 100 acquires information indicating whether the hand to which the controller 200 is attached is in contact with the real object and information related to the movement of the hand, and determines whether to disable the function of the controller 200 based on these acquired information. Therefore, even if the real object does not have a function of determining whether the user is operating the real object, it is possible to appropriately determine whether to disable the function of the controller 200.

[0080] Figure 8 is a flowchart showing the disabling process according to the third embodiment. In step S801, the control unit 123 acquires the real image captured by the camera unit 111 through the image acquisition unit 121. In step S802, the control unit 123 determines whether the hand to which the controller 200 is attached is in contact with the real object based on the acquired real image. When the hand to which the controller 200 is attached is in contact with the real object, the process proceeds to step S803. Otherwise, the process proceeds to S801.

[0081] In step S803, the control unit 123 determines whether the amount of movement of the hand in contact with the real object is at least a specified threshold. When the amount of movement of the hand in contact with the real object is at least the specified threshold, the process proceeds to S804. Otherwise, the process proceeds to step S801. The specified threshold may be, for example, an amount of movement that enables determination of whether the real object is being operated (whether the hand is moving), such as a representative amount of movement (average value, minimum value, or median value) observed when the real object is being operated.

[0082] In step S804 , the control unit 123 disables the function of the controller 200 and ends the disabling process.

[0083] Fourth embodiment

[0084] In the fourth embodiment, during the contact of a real object with the CG in a virtual space (a three-dimensional space where the real world and the virtual world are merged), the function of the controller 200 is enabled. Therefore, it is possible to prevent a decrease in convenience due to disabling the function of the controller 200, such as a situation where it becomes difficult to determine whether the real object is in contact with the CG when the controller 200 stops vibrating.

[0085] Fig. 92 is a schematic diagram showing a state where a real object is in contact with a CG in a virtual space. The user holds a driver 900 serving as a real object with a hand 210 to which a controller 200 is attached. A tip tool 901 of the driver 900 is in contact with the CG 301.

[0086] In the first embodiment, when it is assumed that the driver 900 is used as a designated real object, the vibration function of the controller 200 is disabled during the period when the hand to which the controller 200 is attached is in contact with the driver 900. In addition, in the third embodiment, when the hand to which the controller 200 is attached is in contact with the driver 900 and is moving, the vibration function of the controller 200 is disabled. In contrast, in the fourth embodiment, when the front end tool 901 (real object) of the driver 900 is in contact with the CG 301, the vibration function of the controller 200 can be enabled regardless of whether the hand is in contact with the real object. As a result, the control unit 123 of the HMD 100 is able to send a vibration instruction to the controller 200 to vibrate it, and is also able to notify the user of the contact between the front end tool 901 and the CG 301.

[0087] Fig.10 is a flowchart showing the activation processing according to the fourth embodiment. In step S1001, the control unit 123 determines whether the real object is in contact with the CG. When the real object is in contact with the CG, the processing proceeds to step S1002. Otherwise, the processing proceeds to step S1001. In step S1002, the control unit 123 activates the function of the controller 200 and ends the activation processing.

[0088] Note that an example in which the front end tool 901 is used as a real object is described. However, the front end tool 901 may also be a CG. In this case, the front end tool 901 (CG) may be superimposed on the driver 900 (real object), and the controller 200 may be configured to vibrate when the front end tool 901 comes into contact with the CG 301. As described above, when the CG superimposed on the real object held by the user comes into contact with a different CG, the function of the controller 200 may be enabled.

[0089] Fifth embodiment

[0090] In the fifth embodiment, when the hand to which the controller 200 is attached comes into contact with the real object, a specified notification is made. Therefore, the user can easily understand the reason why the function of the controller 200 is disabled.

[0091] Fig.11 2 is a schematic diagram showing a warning screen. The warning screen is a screen that warns (notifies) the user that the user is holding a real object with a different hand when the user is holding another real object with the hand to which the controller 200 is attached. Figure 5When it is determined in step S502 that the hand to which the controller 200 is attached is in contact with the real object, the control unit 123 displays on the display unit 112 Fig.11 Note that notification can be made not only by displaying a warning screen but also by outputting a sound, for example.

[0092] Note that in the fifth embodiment, notification is performed after disabling the controller 200, but disabling may be omitted. Even with such a configuration, erroneous operation of the controller 200 can be prevented. Furthermore, the processing load can be reduced by omitting disabling.

[0093] Sixth embodiment

[0094] In the first embodiment, the HMD 100 determines whether the hand is in contact with the real object based on the real image. In the sixth embodiment, the controller 200 regards the state in which the controller 200 is in contact with the real object as the state in which the hand is in contact with the real object, and regards the state in which the controller 200 is not in contact with the real object as the state in which the hand is not in contact with the real object. In this way, the controller 200 determines whether the hand is in contact with the real object. Therefore, for example, even in the case where the real image is not acquired, it is possible to determine whether to disable the function of the controller 200. Note that in the sixth embodiment, the real object refers to a real object other than the controller 200 (and the hand).

[0095] Fig.12 200 is a flowchart showing the disabling process according to the sixth embodiment. In step S1201, the CPU 201 of the controller 200 detects the contact between the controller 200 and the real object (determines whether the controller 200 is in contact with the real object) through the contact detection unit 206. The CPU 201 waits for the detection of the contact between the controller 200 and the real object. After the contact is detected, the process proceeds to step S1202.

[0096] In step S1202, the CPU 201 disables the function of the controller 200 and ends the disabling process.

[0097] Note that the controller 200 may determine whether the controller 200 is in contact with the real object and output information indicating whether the controller 200 is in contact with the real object (determination result) to the outside. Then, the HMD 100 may determine whether the hand is in contact with the real object based on the information output from the controller 200 and switch between enabling and disabling the function of the controller 200.

[0098] Note that the various types of control described above may be processes that are performed by only one hardware (e.g., a processor or a circuit) or are otherwise performed. Processing may be shared between multiple hardware (e.g., multiple processors, multiple circuits, or a combination of one or more processors and one or more circuits) to perform control of the entire device.

[0099] In addition, the above-mentioned processor is a processor in a broad sense, and includes a general-purpose processor and a special-purpose processor. Examples of general-purpose processors include a central processing unit (CPU), a microprocessing unit (MPU), and a digital signal processor (DSP). Examples of special-purpose processors include a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), and a programmable logic device (PLD). Examples of PLDs include a field programmable gate array (FPGA) and a complex programmable logic device (CPLD).

[0100] The above-mentioned embodiments (including variations) are merely examples. Any configuration obtained by appropriately deforming or changing some configurations of the embodiments within the scope of the subject matter of the present invention is also included in the present invention. The present invention also includes other configurations obtained by appropriately combining various features of the embodiments.

[0101] According to the present invention, erroneous operation of the controller can be prevented with high accuracy.

[0102] Other embodiments

[0103] The embodiments of the present invention may also be implemented by providing software (program) for performing the functions of the above-described embodiments to a system or device via a network or various storage media, and a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0104] While the present invention has been described with reference to embodiments, it is to be understood that the invention is not limited to the disclosed embodiments but is instead defined by the scope of the following claims.

Claims

1. An information processing device, configured to receive an operation of an operation interface by a user, the information processing device comprising: an acquisition unit configured to acquire first information related to contact between a real object and a hand of the user, the real object being different from the operation interface and the information processing device; as well as A control unit is configured to disable a function associated with the operation interface when the first information acquired by the acquisition unit indicates that the hand is in contact with the real object.

2. The information processing device according to claim 1, wherein: The control unit is configured not to disable the function if the real object in contact with the hand does not coincide with a designated real object.

3. The information processing device according to claim 2, further comprising: a storage unit configured to store second information related to the specified real object, The control unit is configured to determine whether the real object in contact with the hand is consistent with the designated real object based on the first information and the second information.

4. The information processing device according to any one of claims 1 to 3, wherein: The control unit is configured to enable the function if the hand is separated from the real object after coming into contact with the real object.

5. The information processing device according to any one of claims 1 to 3, wherein: The real object is an object that the user can operate. The acquisition unit is configured to acquire, from the real object, information related to whether the user is operating the real object as the first information, and The first information is configured to indicate that the hand is in contact with the real object if the user is operating the real object, and to indicate that the hand is not in contact with the real object if the user is not operating the real object.

6. The information processing device according to any one of claims 1 to 3, wherein: The acquisition unit is configured to acquire, from the operation interface, information related to whether the operation interface is in contact with the real object as the first information, and The first information is configured to indicate that the hand is in contact with the real object when the operation interface is in contact with the real object, and to indicate that the hand is not in contact with the real object when the operation interface is not in contact with the real object.

7. The information processing device according to any one of claims 1 to 3, wherein: The acquisition unit is configured to acquire the first information based on an image representing a field of view of the user, and The control unit is configured not to disable the function if the area of ​​the hand is not included in the image.

8. The information processing device according to any one of claims 1 to 3, wherein: The acquisition unit is further configured to acquire third information related to the movement of the hand, and The control unit is configured to not disable the function if the hand in contact with the real object is not moving, based on the first information and the third information.

9. The information processing device according to any one of claims 1 to 3, wherein: In a case where the real object is in contact with a virtual object in a three-dimensional space where the real world and the virtual world are merged, the control unit is configured to enable the function regardless of whether the hand is in contact with the real object.

10. The information processing device according to any one of claims 1 to 3, wherein: The function associated with the operation interface includes at least one of a function of displaying a graphic based on a position or a posture of the operation interface, a function of receiving an operation on the operation interface, and a function of vibrating the operation interface.

11. The information processing device according to any one of claims 1 to 3, wherein: The control unit is further configured to perform a designated notification when the hand is in contact with the real object.

12. The information processing device according to any one of claims 1 to 3, wherein: The information processing device is accommodated in the display device or the operation interface.

13. A control method for an information processing device, wherein the information processing device receives an operation of a user on an operation interface, the control method comprising: Acquiring first information related to contact between a real object and a hand of the user, the real object being different from the operation interface and the information processing device; as well as In a case where the acquired first information indicates that the hand is in contact with the real object, a function associated with the operation interface is disabled.

14. A computer program product comprising a program for causing a computer to execute each step of the control method according to claim 13. 15 . A computer-readable storage medium storing a program for causing a computer to execute each step of the control method according to claim 13 .

Images