Information processing device, information processing system, and program

Abstract

To achieve authentication for unlocking at the start of the use of a terminal used by a user without adding a special configuration to the terminal.SOLUTION: A user terminal 10 as an information processing apparatus includes a control unit 11. In the control unit 11, an operation receiving unit 102 receives a predetermined input operation to the user terminal 10. When the input operation is received, a switching control unit 105 performs control for switching from a restricted state in which an input operation to an HMD (head-mounted display), which is another information processing apparatus, is restricted to a released state in which the input operation restriction is released.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing system, and a program. [Background technology]

[0002] For example, wearable devices such as HMDs (head-mounted displays), AR (augmented reality) glasses, and smartwatches may require authentication to unlock them when they are first used. Conventional authentication methods include passwords, personal identification numbers, biometric information (such as irises or fingerprints), and gesture patterns (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-99702 Summary of the Invention [Problem to be solved by the invention]

[0004] However, authentication using passwords or biometric information requires that a device for acquiring authentication information be installed on the wearable device, which poses problems in terms of cost, size, etc. Furthermore, authentication using gesture patterns is not practical because it requires unnatural movements (gestures) by the user.

[0005] An object of the present invention is to realize authentication for unlocking a terminal used by a user when the terminal is first used, without adding any special configuration to the terminal. [Means for solving the problem]

[0006] The invention described in claim 1 includes a processor, and the processor receives a predetermined input operation for the own device, and when the input operation is received, performs control to switch from a restricted state in which input operations for other information processing devices are restricted to a released state in which the restriction on the input operation is released. When the other information processing device is in a predetermined state and also in the restricted state, upon receiving the predetermined input operation on the own device, control is performed to switch the other information processing device from the restricted state to the released state; when the predetermined state is a state in which a positional relationship between the other information processing device and the own device satisfies a predetermined condition, upon receiving the predetermined input operation on the own device, control is performed to switch the other information processing device from the restricted state to the released state; when the state in which the predetermined condition is a state in which the other information processing device and the own device are in a predetermined positional relationship in a vertical direction, upon receiving the predetermined input operation on the own device, control is performed to switch the other information processing device from the restricted state to the released state The information processing device is characterized by the above. The invention described in claim 2 is an information processing device comprising a processor, wherein the processor receives a predetermined input operation on the information processing device itself, and upon receiving the input operation, performs control to switch from a restricted state in which input operations on other information processing devices are restricted to an unlocked state in which the restriction on the input operation is unlocked; when the other information processing device is in a predetermined state and is also in the restricted state, upon receiving the predetermined input operation on the information processing device itself, performs control to switch from the restricted state to the unlocked state; when information indicating that the other information processing device is in the predetermined state is acquired from the other information processing device, upon receiving the predetermined input operation on the information processing device itself, controls to switch the other information processing device from the restricted state to the unlocked state; and when information of an image captured by the other information processing device is acquired from the other information processing device as information indicating that the other information processing device is in the predetermined state, upon receiving the predetermined input operation on the information processing device itself, controls to switch the other information processing device from the restricted state to the unlocked state. The invention described in claim 3 is an information processing device comprising a processor, wherein the processor receives a predetermined input operation on the information processing device itself, and upon receiving the input operation, performs control to switch from a restricted state in which input operations on other information processing devices are restricted to an unlocked state in which the restriction on the input operation is unlocked; when the other information processing device is in a predetermined state and is also in the restricted state, upon receiving the predetermined input operation on the information processing device itself, performs control to switch from the restricted state to the unlocked state; detects by the information processing device itself that the other information processing device is in the predetermined state, and upon receiving the predetermined input operation on the information processing device itself, performs control to switch the other information processing device from the restricted state to the unlocked state; detects that the other information processing device is in the predetermined state based on information of an image of the other information processing device captured by the information processing device itself, and upon receiving the predetermined input operation on the information processing device itself, performs control to switch the other information processing device from the restricted state to the unlocked state. The invention described in claim 4 is an information processing device comprising a processor, which receives a predetermined input operation on the device itself, and upon receiving the input operation, performs control to switch from a restricted state in which input operations on other information processing devices are restricted to an unlocked state in which the restriction on the input operation is unlocked, and when the other information processing device is in a predetermined state and is also in the restricted state, upon receiving the predetermined input operation on the device itself, performs control to switch from the restricted state to the unlocked state, and when the other information processing device is no longer in the predetermined state after switching from the restricted state to the unlocked state, performs control to switch from the unlocked state to the restricted state. Claim 5 The invention described in claim 1 is characterized in that, after the other information processing device has switched from the restricted state to the released state, when the other information processing device is no longer in the predetermined state and the state continues for a predetermined time, the processor performs control to switch from the released state to the restricted state. 4 The information processing device is described in Claim 6 The invention described in the above is provided with a processor, and when the device itself enters a predetermined state, the processor controls to transmit information indicating that to another information processing device, and based on the information from the other information processing device, switches from a restricted state in which input operations on the device itself are restricted to a released state in which the restrictions on the input operations are released. When the positional relationship between the own device and the other information processing device satisfies a predetermined condition as the predetermined state, control is performed to transmit information indicating that to the other information processing device, and when the positional relationship between the own device and the other information processing device satisfies a predetermined vertical positional relationship as the state satisfying the predetermined condition, control is performed to transmit information indicating that to the other information processing device. The information processing device is characterized in that: Claim 7 The invention described in the above includes a receiving means for receiving a predetermined input operation for a first information processing device, and a switching control means for controlling switching from a restricted state in which an input operation for a second information processing device is restricted to a released state in which the restriction on the input operation is released when the input operation is received by the first information processing device. When the second information processing device is in a predetermined state and also in the restricted state, upon receiving the predetermined input operation on the first information processing device, the switching control means controls the second information processing device to switch from the restricted state to the released state; when the predetermined state is a state in which a positional relationship between the second information processing device and the first information processing device satisfies a predetermined condition, upon receiving the predetermined input operation on the first information processing device, the switching control means controls the second information processing device to switch from the restricted state to the released state; when the second information processing device and the first information processing device are in a predetermined vertical positional relationship, upon receiving the predetermined input operation on the first information processing device, the switching control means controls the second information processing device to switch from the restricted state to the released state Characterized by 、 It is an information processing system. The invention described in claim 8 is an information processing system comprising: a receiving means for receiving a predetermined input operation for a first information processing device; and a switching control means, when the first information processing device receives the input operation, for controlling a switch from a restricted state in which input operations for a second information processing device are restricted to an unlocked state in which the restriction on the input operation is unlocked, wherein the switching control means, when the second information processing device is in a predetermined state and in the restricted state, controls a switch from the restricted state to the unlocked state when the predetermined input operation for the first information processing device is received; when information indicating that the second information processing device is in the predetermined state is acquired from the second information processing device, controls a switch of the second information processing device from the restricted state to the unlocked state when the predetermined input operation for the first information processing device is received; and when information of an image captured by the second information processing device is acquired from the second information processing device as information indicating that the second information processing device is in the predetermined state, controls a switch of the second information processing device from the restricted state to the unlocked state when the predetermined input operation for the first information processing device is received. The invention described in claim 9 is an information processing system comprising: a receiving means for receiving a predetermined input operation for a first information processing device; and a switching control means for, when the first information processing device receives the input operation, controlling a switch from a restricted state in which input operations for a second information processing device are restricted to an unlocked state in which the restriction on the input operation is unlocked, wherein the switching control means, when the second information processing device is in a predetermined state and in the restricted state, receives the predetermined input operation for the first information processing device, controls a switch from the restricted state to the unlocked state; detects by the first information processing device that the second information processing device is in the predetermined state and receives the predetermined input operation for the first information processing device, controls a switch of the second information processing device from the restricted state to the unlocked state; detects that the second information processing device is in the predetermined state based on information of an image of the second information processing device captured by the first information processing device, and receives the predetermined input operation for the first information processing device, controls a switch of the second information processing device from the restricted state to the unlocked state. The invention described in claim 10 is an information processing system comprising: a receiving means for receiving a predetermined input operation for a first information processing device; and a switching control means for, when the input operation is received by the first information processing device, controlling a switch from a restricted state in which input operations for a second information processing device are restricted to an unlocked state in which the restriction on the input operation is unlocked, wherein the switching control means, when the second information processing device is in a predetermined state and in the restricted state, controls a switch from the restricted state to the unlocked state when it receives the predetermined input operation for the first information processing device, and when the second information processing device is no longer in the predetermined state after switching from the restricted state to the unlocked state, controls a switch from the unlocked state to the restricted state. Claim 11The invention described in the above provides a computer with a function of receiving a predetermined input operation for the computer itself, and a control for switching, when the input operation for the computer itself is received, from a restricted state in which input operations for other information processing devices are restricted to a released state in which the restriction on the input operation is released. and a control for switching the other information processing device from the restricted state to the released state when the predetermined input operation on the own device is received when the other information processing device is in a predetermined state and also in the restricted state; a control for switching the other information processing device from the restricted state to the released state when the predetermined input operation on the own device is received when the positional relationship between the other information processing device and the own device satisfies a predetermined condition as the predetermined state; and a control for switching the other information processing device from the restricted state to the released state when the predetermined input operation on the own device is received when the other information processing device and the own device are in a predetermined positional relationship in a vertical direction as the state satisfying the predetermined condition. It is a function that performs the above and a program to realize the above. The invention described in claim 12 is a program for causing a computer to realize the following functions: a function of accepting a predetermined input operation for the computer's own device; and a function of, when the input operation for the computer is accepted, performing control to switch from a restricted state in which input operations for other information processing devices are restricted to an released state in which the restriction on the input operation is released, wherein, when the other information processing device is in a predetermined state and is in the restricted state, when the predetermined input operation for the computer is accepted, control to switch the other information processing device from the restricted state to the released state when the predetermined input operation for the computer is accepted when information indicating that the other information processing device is in the predetermined state is obtained from the other information processing device; and control to switch the other information processing device from the restricted state to the released state when information of an image captured by the other information processing device is obtained from the other information processing device as information indicating that the other information processing device is in the predetermined state when the predetermined input operation for the computer is accepted. The invention described in claim 13 is a program for causing a computer to realize the following functions: a function of receiving a predetermined input operation for the computer's own device; and a function of, upon receiving the input operation for the computer's own device, performing control to switch from a restricted state in which input operations for other information processing devices are restricted to an unlocked state in which the restriction on the input operation is unlocked, wherein, when the other information processing device is in a predetermined state and in the restricted state, upon receiving the predetermined input operation for the computer's own device, control to switch from the restricted state to the unlocked state; control to detect by the computer that the other information processing device is in the predetermined state and upon receiving the predetermined input operation for the computer's own device, switch the other information processing device from the restricted state to the unlocked state; and control to detect that the other information processing device is in the predetermined state based on information of an image of the other information processing device captured by the computer's own device, and upon receiving the predetermined input operation for the computer's own device, switch the other information processing device from the restricted state to the unlocked state. The invention described in claim 14 is a program for causing a computer to realize the following functions: a function of accepting a predetermined input operation for the computer's own device; and a function of, when the input operation for the computer is accepted, controlling a switch from a restricted state in which input operations for other information processing devices are restricted to an released state in which the restriction on the input operation is released, wherein, when the other information processing device is in a predetermined state and is in the restricted state, when the predetermined input operation for the computer is accepted, the control of switching from the restricted state to the released state; and, when the other information processing device is switched from the restricted state to the released state and then is no longer in the predetermined state, the control of switching from the released state to the restricted state. Claim 15 The invention described in the above includes a function of controlling a computer to transmit information indicating that its own device has entered a predetermined state to another information processing device, and a function of switching from a restricted state in which input operations on the own device are restricted to a released state in which the restrictions on the input operations are released based on information from the other information processing device; a function for controlling, when a positional relationship between the own device and the other information processing device satisfies a predetermined condition as the predetermined state, transmitting information indicating that to the other information processing device; and a function for controlling, when a positional relationship between the own device and the other information processing device satisfies a predetermined vertical positional relationship as the state satisfying the predetermined condition, transmitting information indicating that to the other information processing device; This is a program to achieve this. [Effects of the Invention]

[0007] According to the present invention of claim 1, it is possible to provide an information processing device that realizes authentication for unlocking another information processing device when the other information processing device begins to be used without adding any special configuration to the other information processing device. Furthermore, it is possible to control unlocking of the other information processing device depending on the state of the other information processing device. Furthermore, it is possible to control unlocking of the other information processing device depending on the positional relationship between the other information processing device and the own device (information processing device). Furthermore, it is possible to control unlocking of the other information processing device depending on the positional relationship between the other information processing device and the own device (information processing device) in the up-down direction. According to the present invention of claim 2, it is possible to provide an information processing device that realizes authentication for unlocking another information processing device when the other information processing device starts to be used without adding any special configuration to the other information processing device. Furthermore, it is possible to control unlocking of the other information processing device depending on the state of the other information processing device. Furthermore, it is possible to obtain information indicating that the other information processing device is in a predetermined state from the other information processing device and control unlocking of the other information processing device. Furthermore, it is possible to determine whether the other information processing device is in a predetermined state from information on an image captured by the other information processing device. According to the present invention of claim 3, it is possible to provide an information processing device that realizes authentication for unlocking another information processing device when the other information processing device starts to be used, without adding any special configuration to the other information processing device. Furthermore, it is possible to control unlocking of the other information processing device depending on the state of the other information processing device. Furthermore, it is possible for the information processing device itself (information processing device) to detect that the other information processing device is in a predetermined state and control unlocking of the other information processing device. Furthermore, it is possible to detect that the other information processing device is in a predetermined state based on information on an image of the other information processing device captured by the information processing device itself. According to the present invention of claim 4, it is possible to provide an information processing device that realizes authentication for unlocking another information processing device when the other information processing device starts to be used without adding any special configuration to the other information processing device. Also, it is possible to control unlocking of the other information processing device depending on the state of the other information processing device. Furthermore, when the other information processing device is no longer in a predetermined state, it is possible to return the unlocked state to the restricted state again. Claim 5 According to the present invention, even if the other information processing device is no longer in the predetermined state, the other information processing device does not immediately return to the restricted state, thereby improving convenience for the user. Claim 6 According to the present invention, it is possible to provide an information processing device that realizes authentication for unlocking the information processing device when the information processing device is first used, without adding any special configuration to the information processing device. Furthermore, the lock of the information processing device can be released depending on the positional relationship between the information processing device and other information processing devices. Furthermore, the lock of the information processing device can be released depending on the positional relationship between the information processing device and other information processing devices in the vertical direction. Claim 7According to the present invention, it is possible to provide an information processing system that realizes authentication for unlocking another information processing device when the other information processing device starts to be used, without adding any special configuration to the other information processing device. Furthermore, it is possible to control unlocking of the other information processing device depending on the state of the other information processing device. Furthermore, it is possible to control unlocking of the other information processing device depending on the positional relationship between the other information processing device and the own device (information processing device). Furthermore, it is possible to control unlocking of the other information processing device depending on the positional relationship between the other information processing device and the own device (information processing device) in the up-down direction. According to the present invention of claim 8, it is possible to provide an information processing system that realizes authentication for unlocking another information processing device when the other information processing device is first used, without adding any special configuration to the other information processing device. Furthermore, it is possible to control unlocking of the other information processing device depending on the state of the other information processing device. Furthermore, it is possible to obtain information indicating that the other information processing device is in a predetermined state from the other information processing device and control unlocking of the other information processing device. Furthermore, it is possible to determine whether the other information processing device is in a predetermined state from information on an image captured by the other information processing device. According to the present invention of claim 9, it is possible to provide an information processing system that realizes authentication for unlocking another information processing device when starting to use the other information processing device without adding any special configuration to the other information processing device. Furthermore, it is possible to control unlocking of the other information processing device depending on the state of the other information processing device. Furthermore, it is possible for the information processing device itself (information processing device) to detect that the other information processing device is in a predetermined state and control unlocking of the other information processing device. Furthermore, it is possible to detect that the other information processing device is in a predetermined state based on information of an image of the other information processing device captured by the information processing device itself. According to the present invention of claim 10, it is possible to provide an information processing system that realizes authentication for unlocking another information processing device when the other information processing device starts to be used without adding any special configuration to the other information processing device. Also, it is possible to control unlocking of the other information processing device depending on the state of the other information processing device. Furthermore, when the other information processing device is no longer in a predetermined state, it is possible to return the unlocked state to the restricted state again. Claim 11 According to the present invention, it is possible to provide a program that realizes authentication for unlocking another information processing device when the other information processing device starts to be used, without adding any special configuration to the other information processing device. Furthermore, it is possible to control unlocking of the other information processing device depending on the state of the other information processing device. Furthermore, it is possible to control unlocking of the other information processing device depending on the positional relationship between the other information processing device and the own device (information processing device). Furthermore, it is possible to control unlocking of the other information processing device depending on the positional relationship between the other information processing device and the own device (information processing device) in the up-down direction. According to the present invention of claim 12, it is possible to provide a program that realizes authentication for unlocking another information processing device when the other information processing device is first used, without adding any special configuration to the other information processing device. Furthermore, it is possible to control unlocking of the other information processing device depending on the state of the other information processing device. Furthermore, it is possible to obtain information indicating that the other information processing device is in a predetermined state from the other information processing device and control unlocking of the other information processing device. Furthermore, it is possible to determine whether the other information processing device is in a predetermined state from information on an image captured by the other information processing device. According to the present invention of claim 13, it is possible to provide a program that realizes authentication for unlocking another information processing device when the other information processing device is first used, without adding any special configuration to the other information processing device. Furthermore, it is possible to control unlocking of the other information processing device depending on the state of the other information processing device. Furthermore, it is possible for the information processing device itself to detect that the other information processing device is in a predetermined state and control unlocking of the other information processing device. Furthermore, it is possible to detect that the other information processing device is in a predetermined state based on information about an image of the other information processing device captured by the information processing device itself. According to the present invention of claim 14, it is possible to provide a program that realizes authentication for unlocking another information processing device when the other information processing device starts to be used without adding any special configuration to the other information processing device. Also, it is possible to control unlocking of the other information processing device depending on the state of the other information processing device. Furthermore, when the other information processing device is no longer in a predetermined state, it is possible to return the unlocked state to the restricted state again. Claim 15 According to the present invention, it is possible to provide a program that realizes authentication for unlocking an information processing device when the device is first used, without adding any special configuration to the information processing device. Furthermore, the lock of the information processing device can be released depending on the positional relationship between the information processing device and other information processing devices. Furthermore, the lock of the information processing device can be released depending on the positional relationship between the information processing device and other information processing devices in the vertical direction. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an overall configuration of an information processing system to which a first embodiment is applied. [Figure 2] FIG. 2 is a diagram illustrating a hardware configuration of a user terminal to which the first embodiment is applied. [Figure 3] FIG. 1 is a diagram illustrating a hardware configuration of an HMD to which a first embodiment is applied. [Figure 4] FIG. 2 is a diagram illustrating a functional configuration of a control unit of a user terminal to which the first embodiment is applied. [Figure 5] FIG. 2 is a diagram illustrating a functional configuration of a control unit of an HMD to which the first embodiment is applied. [Figure 6] 10 is a flowchart showing a flow of processing up to transmission of an unlock instruction to an HMD, among processing of a user terminal to which the first embodiment is applied. [Figure 7] 10 is a flowchart showing the flow of processing in an HMD to which the first embodiment is applied. [Figure 8] FIG. 2 is a diagram showing a specific example of the positional relationship between a user terminal, an HMD, and a user. [Figure 9] FIG. 10 is a diagram illustrating a hardware configuration of a user terminal to which the second embodiment is applied. [Figure 10] FIG. 10 is a diagram illustrating a functional configuration of a control unit of a user terminal to which the second embodiment is applied. [Figure 11] FIG. 10 is a diagram illustrating a functional configuration of a control unit of an HMD to which a second embodiment is applied. [Figure 12] 10 is a flowchart showing a flow of processing up to transmission of an unlock instruction to an HMD, among processing of a user terminal to which the second embodiment is applied. [Figure 13] 10 is a flowchart showing the flow of processing in an HMD to which the second embodiment is applied. [Figure 14] FIG. 10 is a diagram illustrating an overall configuration of an information processing system to which a third embodiment is applied. [Figure 15] FIG. 10 is a diagram illustrating a hardware configuration of an image processing apparatus to which a third embodiment is applied. [Figure 16] FIG. 10 is a diagram illustrating a functional configuration of a control unit of an image processing apparatus to which a third embodiment is applied. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. First Embodiment (Configuration of information processing system) FIG. 1 is a diagram showing the overall configuration of an information processing system 1 to which the first embodiment is applied. An information processing system 1 to which the first embodiment is applied is configured by connecting a user terminal 10 and an HMD (head mounted display) 30 via a network 90 or a communication method such as infrared communication, visible light communication, close proximity wireless communication, Bluetooth (registered trademark), RFID (registered trademark), UWB (Ultra Wide Band), etc. The network 90 is, for example, a LAN (Local Area Network), the Internet, etc.

[0010] The user terminal 10 is an information processing device such as a smartphone, a personal computer, or a tablet terminal used by a user U. The user terminal 10 is connected to the HMD 30 in a locked state, and when the HMD 30 is worn on the head of the user U or when it is possible to estimate that the HMD 30 is worn on the head of the user U, the user terminal 10 performs control to unlock the HMD 30 based on an input operation by the user U.

[0011] A "locked state" refers to a state in which input operations are restricted, and an "unlocked state" refers to a state in which input operations are not restricted. Furthermore, an "input operation by user U" refers to an input operation by user U on the user terminal 10. For example, an operation using dedicated application software pre-installed on the user terminal 10 and an operation using a dedicated website accessible via the browser function of the user terminal 10 are both examples of "input operations by user U."

[0012] The HMD 30 is a head-mounted information processing device having a display for displaying image information. The HMD 30 may be of a binocular type, a monocular type, a see-through type, or a non-see-through type, but is not limited thereto. For example, a see-through HMD 30 allows the user terminal 10 to be viewed through the HMD 30. The HMD 30 controls transmission of information indicating its own status to the user terminal 10, and then unlocks its own device based on an unlock instruction from the user terminal 10. This allows the user U to start using the HMD 30. The HMD 30 is a device used while worn on the head of the user U. For this reason, it is desirable to unlock the HMD 30 after the user U wears it, as in this embodiment, in order to prevent unauthorized use by others and to save power consumption when the HMD 30 is not being worn.

[0013] Examples of information indicating the status of the HMD transmitted from the HMD30 include information indicating the position of the HMD (HMD30), the position of the user terminal 10, the position of the user U, the positional relationship between the HMD (HMD30) and the user terminal 10, and the positional relationship between the HMD (HMD30) and the user U, as well as information on still or moving images of the user terminal 10 or the user U.

[0014] (Hardware configuration of user terminal) FIG. 2 is a diagram showing the hardware configuration of the user terminal 10 to which the first embodiment is applied. A user terminal 10 to which the first embodiment is applied includes a control unit 11, a memory 12, a storage unit 13, a communication unit 14, an operation unit 15, a display unit 16, a sensor unit 17, and an imaging unit 18. These units are connected to each other via a data bus, an address bus, a PCI (Peripheral Component Interconnect) bus, etc.

[0015] The control unit 11 is a processor that controls the operation of the user terminal 10 through the execution of various software such as an OS (operating system) and application software. The control unit 11 is configured, for example, by a CPU (Central Processing Unit). The memory 12 is a storage area that stores various software and data used for executing the software, and is used as a working area for calculations. The memory 12 is configured, for example, by a RAM (Random Access Memory).

[0016] The storage unit 13 is a storage area that stores input data for various software programs and output data from various software programs, and stores a database that stores various information. The storage unit 13 is composed of, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a semiconductor memory that is used to store programs and various setting data. The communication unit 14 transmits and receives data via the network 90 or by a communication method such as infrared communication. The communication unit 14 transmits and receives data between the HMD 30 and the outside.

[0017] The operation unit 15 is configured with, for example, a keyboard, a mouse, mechanical buttons, and switches, and accepts input operations. The operation unit 15 also includes a touch sensor that forms a touch panel integrally with the display unit 16. The display unit 16 displays images, text information, and the like. The display unit 16 is configured with, for example, a liquid crystal display or an organic EL (Electro Luminescence) display used to display information.

[0018] The sensor unit 17 is composed of various sensors such as an optical sensor and an acceleration sensor, and detects, for example, the position of the device itself, the position of the HMD 30, the position of the user U, the positional relationship between the device itself and the HMD 30, and the positional relationship between the HMD 30 and the user U. The imaging unit 18 is composed of a camera or the like, and captures images of the HMD 30 and the user U.

[0019] (HMD hardware configuration) FIG. 3 is a diagram showing the hardware configuration of the HMD 30 to which the first embodiment is applied. The HMD 30 to which the first embodiment is applied has a configuration similar to the hardware configuration of the user terminal 10 shown in FIG. 2 , except for the operation unit 15. Specifically, the HMD 30 has a control unit 31 configured with a processor such as a CPU, a memory 32 configured with a storage area such as RAM, and a storage unit 33 configured with a storage area such as a HDD, an SSD, or a semiconductor memory. The HMD 30 also has a communication unit 34 that transmits and receives data to and from the user terminal 10 via a network 90 or by a communication method such as infrared communication, and a display unit 35 configured with a liquid crystal display, an organic EL display, or the like. The HMD 30 also has a sensor unit 36 ​​configured with various sensors such as an optical sensor such as a LiDAR (Laser Imaging Detection and Ranging) sensor, an acceleration sensor, and a pressure sensor, and an imaging unit 37 configured with a camera, or the like. These units are connected via a data bus, an address bus, a PCI bus, or the like.

[0020] (Functional configuration of the control unit of the user terminal) FIG. 4 is a diagram showing the functional configuration of the control unit 11 of the user terminal 10 to which the first embodiment is applied. In the control unit 11 of the user terminal 10 to which the first embodiment is applied, a connection control unit 101, an operation reception unit 102, a status acquisition unit 103, a status determination unit 104, a switching control unit 105, a designation reception unit 106, an output control unit 107, and a learning unit 108 function.

[0021] The connection control unit 101 controls the connection between the user's own device and the HMD 30. Specifically, the connection control unit 101 controls the combination of the information processing device, including the user's own device and the HMD 30, to perform mutual registration and authentication, enabling communication and function calling. The connection control by the connection control unit 101 is the control of a process known as "pairing." For example, when a user U wearing the HMD 30 on his / her head recognizes a QR code (registered trademark) displayed on the user terminal 10 through the HMD 30, pairing may be automatically established. This configuration makes it possible to ensure that only the authentic user U can unlock the HMD 30. This prevents, for example, a situation in which the user terminal 10 is mistakenly paired with another HMD 30 when a user U2 wearing another HMD 30 is nearby when the user U1 performs an operation to unlock the user terminal 10, resulting in unintended information leakage.

[0022] The operation reception unit 102 receives input operations. This "input operation" includes an operation to unlock the user terminal 10 that is in a locked state, and other operations. Among these, operations to unlock the user terminal 10 include, for example, an operation to swipe the screen, an operation to perform authentication using biometric information such as face authentication or fingerprint authentication, and an operation to input a lock pattern or a PIN number.

[0023] Furthermore, examples of operations other than the operation for unlocking the user terminal 10 include operations that are predetermined as operations for unlocking the HMD 30 after the user terminal 10 has been unlocked. Examples of operations that are predetermined as operations for unlocking the HMD 30 after the user terminal 10 has been unlocked include an operation of pressing a button for unlocking the HMD 30 that is displayed on the display unit 16, an operation of capturing an image of an unlock symbol formed on a part of the surface of the HMD 30, and an operation of shaking the user terminal 10 itself from side to side in the vicinity of the HMD 30.

[0024] When an operation involving a relatively simple process, such as swiping the screen of the user terminal 10, is performed to unlock the user terminal 10, only the HMD 30 may be unlocked. Furthermore, when an operation involving a relatively complicated process, such as an operation for performing authentication using biometric information, is performed, both the user terminal 10 and the HMD 30 may be unlocked. Whether both the user terminal 10 and the HMD 30 are unlocked or only the HMD 30 is unlocked may be determined based on the state of the user terminal 10. For example, when a notification is received from the user terminal 10, both the user terminal 10 and the HMD 30 may be unlocked, and when no notification is received from the user terminal 10, only the HMD 30 may be unlocked.

[0025] Similarly, for operations other than the operation to unlock the user terminal 10, when an operation involving relatively simple processing is performed, only the user terminal 10 may be unlocked, and when an operation involving relatively complex processing is performed, both the user terminal 10 and the HMD 30 may be unlocked.

[0026] The state acquisition unit 103 acquires information indicating the state of the HMD 30. Specifically, the state acquisition unit 103 acquires, as information indicating the state of the HMD 30, the result of detection by the sensor unit 17 and information on a still image or a moving image of the HMD 30 and the user U captured by the imaging unit 18. The state acquisition unit 103 also acquires, as information indicating the state of the HMD 30, the result of detection by the sensor unit 36 ​​of the HMD 30 and information on a still image or a moving image of the user terminal 10 and the user U captured by the imaging unit 37 of the HMD 30.

[0027] The state determination unit 104 determines the state of the HMD 30 based on the information indicating the state of the HMD 30 acquired by the state acquisition unit 103 and the results of learning by the learning unit 108. Specifically, the state determination unit 104 determines whether the HMD 30 is in a locked state or an unlocked state. The state determination unit 104 also determines whether the HMD 30 is worn on the head of the user U or whether it is in a state in which it can be estimated that the HMD 30 is worn on the head of the user U.

[0028] The state determination unit 104 determines whether the HMD 30 is worn on the head of the user U or whether it is possible to estimate that the HMD 30 is worn on the head of the user U, for example, as follows. That is, when the HMD 30 and the user U are in a predetermined positional relationship, the state determination unit 104 determines that the HMD 30 is worn on the head of the user U or that it is possible to estimate that the HMD 30 is worn on the head of the user U. On the other hand, when the HMD 30 and the user U are not in a predetermined positional relationship, the state determination unit 104 determines that the HMD 30 is not worn on the head of the user U or that it is not possible to estimate that the HMD 30 is worn on the head of the user U. A specific example of when the HMD 30 and the user U are in a predetermined positional relationship will be described later with reference to FIG. 8.

[0029] Furthermore, for example, when the HMD 30 and the user terminal 10 are in a predetermined positional relationship, the state determination unit 104 determines that the HMD 30 is worn on the head of the user U or that it is possible to estimate that the HMD 30 is worn on the head of the user U. On the other hand, when the HMD 30 and the user terminal 10 are not in a predetermined positional relationship, the state determination unit 104 determines that the HMD 30 is not worn on the head of the user U or that it is not possible to estimate that the HMD 30 is worn on the head of the user U.

[0030] The state determination unit 104 can determine whether the HMD 30 and the user terminal 10 are in a predetermined positional relationship by combining, for example, a LiDAR sensor in the sensor unit 36 ​​of the HMD 30 and positioning using UWB (Ultra Wide Band) adopted in the communication unit 14 of the user terminal 10. Specifically, the position of the user terminal 10 is identified based on a combination of an estimation result that a rectangular object detected by the LiDAR sensor is the user terminal 10 (e.g., a smartphone) and a positioning result using UWB. Furthermore, if the user terminal 10 appears in a still image or a moving image captured by the imaging unit 37 of the HMD 30, the positional relationship between the HMD 30 and the user terminal 10 can be identified from the position and size of the user terminal 10. A specific example of a case where the HMD 30 and the user terminal 10 are in a predetermined positional relationship will be described later with reference to FIG. 8.

[0031] The switching control unit 105 performs control to switch the HMD 30 from a locked state to an unlocked state based on the result of the determination by the state determination unit 104. Specifically, the switching control unit 105 transmits an unlock instruction to the HMD 30 as control to switch the HMD 30 to an unlocked state. Furthermore, the switching control unit 105 performs control to switch the HMD 30 from an unlocked state to a locked state again based on the result of the determination by the state determination unit 104. Specifically, the switching control unit 105 transmits a re-lock instruction to the HMD 30 as control to switch the HMD 30 to a locked state.

[0032] The designation receiving unit 106 receives designation of information to be output from the HMD 30 after the HMD 30 is unlocked. The output control unit 107 controls the HMD 30 to output the information designated as the output target by the designation receiving unit 106. "Control to output" includes control to display text information or image information on the display unit 35, control to output audio information from a speaker, etc. Note that the designation of information to be output by the HMD 30 is performed by an input operation by the user U.

[0033] For example, when information from the user terminal 10 (e.g., notification information) is designated as information to be output from the HMD 30, the HMD 30 can be used as a sub-display of the user terminal 10. Furthermore, for example, when there is information (e.g., confidential information) that cannot be output from the HMD 30 among the information that can be output from the user terminal 10, the information that cannot be output from the HMD 30 can be excluded from the designation of the output target.

[0034] The learning unit 108 performs machine learning on the history of determinations made by the state determination unit 104, the history of switching control by the switching control unit 105, and information about the user U's usual usage patterns. The machine learning is performed, for example, by AI (artificial intelligence). Examples of the "information about usual usage patterns" include information indicating the positional relationship between the user terminal 10 and the HMD 30, which is periodically detected by the sensor unit 17 of the user terminal 10 and the sensor unit 36 ​​of the HMD 30, information indicating the positional relationship between the HMD 30 and the user U, which is periodically captured by the imaging unit 18 of the user terminal 10 and the imaging unit 37 of the HMD 30, and the average value of the time required to put on and take off the HMD 30 and the usage time, which are detected by the sensor unit 36. The results of learning by the learning unit 108 are used for determination by the state determination unit 104.

[0035] (HMD functional configuration) FIG. 5 is a diagram showing the functional configuration of the control unit 31 of the HMD 30 to which the first embodiment is applied. In the control unit 31 of the HMD 30 to which the first embodiment is applied, a state acquisition unit 301, a transmission control unit 302, an instruction reception unit 303, and a switching control unit 304 function.

[0036] The status acquisition unit 301 acquires information indicating the status of the own device. Specifically, the status acquisition unit 301 acquires, as information indicating the status of the own device, the results of detection by the sensor unit 36 ​​and information on still images or moving images of the user terminal 10 and the user U captured by the imaging unit 37. The transmission control unit 302 controls transmission of the information indicating the status of the own device acquired by the status acquisition unit 301 to the user terminal 10.

[0037] The instruction receiving unit 303 receives an unlock instruction transmitted from the user terminal 10. The instruction receiving unit 303 also receives a re-lock instruction transmitted from the user terminal 10. When the instruction receiving unit 303 receives an unlock instruction, the switching control unit 304 performs control to switch the own device from a locked state to an unlocked state. When the instruction receiving unit 303 receives a re-lock instruction, the switching control unit 304 performs control to switch the own device from an unlocked state to a locked state again.

[0038] (User terminal processing) FIG. 6 is a flowchart showing the flow of processing up to transmission of an unlock instruction to the HMD 30, among the processing of the user terminal 10 to which the first embodiment is applied. The user terminal 10 to which the first embodiment is applied controls the connection between itself and the HMD 30 (step 401). Specifically, the user terminal 10 controls the combination of the information processing devices, consisting of itself and the HMD 30, to perform mutual registration and authentication, etc., and to bring them into a state where communication and function calling are possible (i.e., a paired state).

[0039] If an input operation is performed on the user terminal 10 (YES in step 402), the user terminal 10 accepts the input operation (step 403). On the other hand, if an input operation is not performed on the user terminal 10 (NO in step 402), the user terminal 10 repeats the processing of step 402 until an input operation is performed on the user terminal 10.

[0040] When the information indicating the state of the HMD 30 is transmitted (YES in step 404), the user terminal 10 acquires the information indicating the state of the HMD 30 (step 405). On the other hand, when the information indicating the state of the HMD 30 is not transmitted (NO in step 404), the user terminal 10 repeats the process of step 404 until the information indicating the state of the HMD 30 is transmitted.

[0041] If the user terminal 10 determines the state of the HMD 30 based on the acquired information indicating the state of the HMD 30 and finds that the HMD 30 is in a locked state (YES in step 406) and that the HMD 30 is worn on the head of the user U or that it can be estimated that the HMD 30 is worn on the head of the user U (YES in step 407), the user terminal 10 transmits an unlock instruction to switch the locked HMD 30 to an unlocked state (step 408). On the other hand, if the HMD 30 is not in an unlocked state (NO in step 406), the processing by the user terminal 10 ends.

[0042] Furthermore, if the HMD30 is in a locked state (YES in step 406), but the HMD30 is not in a state where it is worn on the head of the user U, or it is not in a state where it can be estimated that the HMD30 is worn on the head of the user U (NO in step 407), the user terminal 10 repeats the processing of step 407 until the HMD30 is in a state where it is worn on the head of the user U, or it is in a state where it can be estimated that the HMD30 is worn on the head of the user U.

[0043] (HMD processing) FIG. 7 is a flowchart showing the flow of processing by the HMD 30 to which the first embodiment is applied. The HMD 30 to which the first embodiment is applied acquires information indicating its own state (step 601) and transmits the information to the user terminal 10 (step 602). Thereafter, when an unlock instruction is transmitted from the user terminal 10 (YES in step 603), the HMD 30 accepts the unlock instruction (step 604) and switches its own state from locked to unlocked (step 605). On the other hand, when an unlock instruction is not transmitted from the user terminal 10 (NO in step 603), the HMD 30 repeats the process of step 603 until an unlock instruction is transmitted from the user terminal 10.

[0044] (Example) FIG. 8 is a diagram showing a specific example of the positional relationship between the user terminal 10, the HMD 30, and the user U. As described above, when it is determined that the HMD 30 is worn on the head of the user U or that it is possible to estimate that the HMD 30 is worn on the head of the user U, an unlock instruction is transmitted from the user terminal 10 to the HMD 30. Whether the HMD 30 is worn on the head of the user U or that it is possible to estimate that the HMD 30 is worn on the head of the user U is determined based on whether the HMD 30 and the user U are in a predetermined positional relationship, or whether the HMD 30 and the user terminal 10 are in a predetermined positional relationship, etc.

[0045] 8, when the distance L between the HMD 30 and the user U is equal to or less than a predetermined value, it is determined that the HMD 30 is worn on the head of the user U or that it is possible to estimate that the HMD 30 is worn on the head of the user U. On the other hand, when the distance L between the HMD 30 and the user U exceeds a predetermined value, it is determined that the HMD 30 is not worn on the head of the user U or that it is not possible to estimate that the HMD 30 is worn on the head of the user U.

[0046] Furthermore, for example, when the positional relationship between the HMD 30 and the user terminal 10 is a predetermined vertical positional relationship, it is determined that the HMD 30 is worn on the head of the user U or that it is possible to estimate that the HMD 30 is worn on the head of the user U. On the other hand, when the positional relationship between the HMD 30 and the user terminal 10 is not a predetermined vertical positional relationship, it is determined that the HMD 30 is not worn on the head of the user U or that it is not possible to estimate that the HMD 30 is worn on the head of the user U.

[0047] Here, "predetermined vertical direction" means that the direction of the user U's head is "up" and the direction of the user U's feet is "down." Therefore, even if the user U is lying down on a bed, for example, the meaning remains the same: the user U's head is "up" and the user U's feet are "down." This determination method based on "predetermined vertical positional relationship" is based on the idea that the HMD 30 is worn on the user U's head, whereas the user terminal 10 is generally operated with a hand positioned below the user U's head.

[0048] As described above, when it is determined that the HMD 30 in a locked state is neither worn on the head of the user U nor in a state in which it can be estimated that it is worn on the head of the user U, a relock instruction is transmitted from the user terminal 10 to the HMD 30. The HMD 30 then relocks based on the transmitted relock instruction. However, if the HMD 30 is immediately relocked when it is no longer worn on the head of the user U nor in a state in which it can be estimated that it is worn on the head of the user U, this would be inconvenient for the user U. For example, if the user U wears the HMD 30 and then unlocks the HMD 30, and then suddenly experiences itchy eyes, the user may temporarily remove the HMD 30 and rub their eyes with their hands. It is not appropriate to immediately relock the HMD 30 in such a case.

[0049] Therefore, the user terminal 10 determines that the HMD 30 is neither worn on the head of the user U nor in a state in which it can be estimated that the HMD 30 is worn on the head of the user U, and if this state continues for a predetermined time, it re-locks the HMD 30. Note that the "predetermined time" is not particularly limited, and may be a predetermined time such as 5 seconds or 10 seconds, or may be set arbitrarily by the user U.

[0050] Furthermore, even if the user terminal 10 determines that the HMD 30 is not being worn on the head of the user U, or that it is not possible to estimate that the HMD 30 is being worn on the head of the user U, the user terminal 10 suspends re-locking as long as the position of the HMD 30 does not exceed a predetermined area. For example, as shown in FIG. 8, re-locking is suspended when the position of the HMD 30 remains within area D near the head of the user U. Note that the "predetermined area" is not particularly limited, and may be a predetermined distance from the outer edge of the head of the user U (e.g., 30 cm, 50 cm, etc.), or may be arbitrarily set by the user U.

[0051] <Second embodiment> In the first embodiment described above, the state of the HMD 30 is determined on the user terminal 10 side, but in the second embodiment, the state of the HMD 30 is determined on the HMD 30 side. (Configuration of information processing system) The configuration of the information processing system 1 to which the second embodiment is applied and the hardware configuration of the HMD 30 that constitutes the information processing system 1 are similar to the configurations of the first embodiment, and therefore a description thereof will be omitted.

[0052] (Hardware configuration of user terminal) FIG. 9 is a diagram showing the hardware configuration of a user terminal 10 to which the second embodiment is applied. A user terminal 10 to which the second embodiment is applied has a configuration similar to the hardware configuration of the user terminal 10 shown in Fig. 2, except for the sensor unit 17 and the imaging unit 18. That is, the user terminal 10 has a control unit 11 configured with a processor such as a CPU, a memory 12 configured with a storage area such as RAM, and a storage unit 13 configured with a storage area such as an HDD, an SSD, or a semiconductor memory. The user terminal 10 also has a communication unit 14 that transmits and receives data to and from the HMD 30 via a network 90 or by a communication method such as infrared communication, an operation unit 15 configured with a keyboard, a mouse, mechanical buttons, and switches, and a display unit 16 configured with a liquid crystal display, an organic EL display, or the like. These units are connected by a data bus, an address bus, a PCI bus, or the like.

[0053] (Functional configuration of the control unit of the user terminal) FIG. 10 is a diagram showing the functional configuration of the control unit 11 of the user terminal 10 to which the second embodiment is applied. 4, except for the state determination unit 104 and the learning unit 108. That is, the control unit 11 of the user terminal 10 functions as a connection control unit 101 that controls the connection between the user terminal 10 and the HMD 30, an operation reception unit 102 that receives input operations, a state acquisition unit 103, a switching control unit 105, a designation reception unit 106 that receives designation of information to be output from the HMD 30, and an output control unit 107 that controls the output of information from the HMD 30.

[0054] The state acquisition unit 103 acquires information when the HMD 30 transmits information indicating that the HMD 30 is worn on the head of the user U or that a state has been established in which it can be estimated that the HMD 30 is worn on the head of the user U. Furthermore, the state acquisition unit 103 acquires information when the HMD 30 transmits information indicating that the HMD 30 is no longer worn on the head of the user U or that it is no longer possible to estimate that the HMD 30 is worn on the head of the user U.

[0055] When the state acquisition unit 103 acquires information indicating that the HMD 30 is no longer worn on the head of the user U or that it is no longer possible to estimate that the HMD 30 is worn on the head of the user U, the switching control unit 105 performs control to switch the HMD 30 from the unlocked state to the locked state again. Specifically, the switching control unit 105 transmits a re-lock instruction to the HMD 30 as control to switch the HMD 30 to the locked state.

[0056] (HMD functional configuration) FIG. 11 is a diagram showing the functional configuration of the control unit 31 of the HMD 30 to which the second embodiment is applied. In the control unit 31 of the HMD 30 to which the second embodiment is applied, a state determination unit 305 functions in addition to the functional configuration of the HMD 30 shown in Fig. 5. That is, in the control unit 31 of the HMD 30, a state acquisition unit 301 that acquires information indicating the state of the HMD 30, a transmission control unit 302 that controls transmission of information indicating the state of the HMD 30 to the user terminal 10, an instruction receiving unit 303 that receives an unlock instruction and a re-lock instruction, a switching control unit 304 that controls switching between unlocking and re-locking the HMD 30, and the state determination unit 305 function.

[0057] The state determination unit 305 determines the state of the own device (HMD 30) based on the information acquired by the state acquisition unit 301. Specifically, the state determination unit 305 determines whether the own device is in a locked state or an unlocked state. The state determination unit 305 also determines whether the own device is in a state where it is worn on the head of the user U or in a state where it can be estimated that the own device is worn on the head of the user U. Specifically, based on, for example, the detection results of various sensors such as an optical sensor, an acceleration sensor, and a pressure sensor of the sensor unit 36, and information on still or moving images of the user terminal 10 and the user U captured by the imaging unit 37, it is determined whether the own device is in a state where it can be estimated that the own device is worn on the head of the user U.

[0058] The transmission control unit 302 controls the transmission of information indicating whether the device itself is in a locked state or an unlocked state to the user terminal 10 based on the result of the determination by the state determination unit 305. The transmission control unit 302 also controls the transmission of information indicating that the device itself is worn on the head of the user U or that it is possible to estimate that the device itself is worn on the head of the user U to the user terminal 10.

[0059] (User terminal processing) Fig. 12 is a flowchart showing the flow of processing of the user terminal 10 to which the second embodiment is applied, up to the time when an unlock instruction is transmitted to the HMD 30. Note that the contents of steps 801 to 804 in Fig. 12 are the same as the contents of steps 401 to 404 in Fig. 6, and therefore description thereof will be omitted.

[0060] When the user terminal 10 to which the second embodiment is applied receives information indicating that the HMD 30 in a locked state is worn on the head of the user U or is in a state in which it can be presumed that it is worn on the head of the user U (YES in step 805), it transmits an unlock instruction (step 806). On the other hand, if the user terminal 10 has not received information indicating that the HMD 30 in a locked state is worn on the head of the user U or is in a state in which it can be presumed that it is worn on the head of the user U (NO in step 805), it repeats the processing of step 805 until it receives information indicating that the HMD 30 in a locked state is worn on the head of the user U or is in a state in which it can be presumed that it is worn.

[0061] (HMD processing) FIG. 13 is a flowchart showing the flow of processing by the HMD 30 to which the second embodiment is applied. The HMD 30 to which the second embodiment is applied acquires information indicating its own state (step 901), and if the result of determining its own state based on the acquired information indicating its own state shows that it is in a locked state (YES in step 902) and that it is worn on the head of the user U or that it can be estimated that it is worn on the head of the user U (YES in step 903), it performs control to transmit information indicating this to the user terminal 10 as information indicating the state of the HMD 30 (step 904). On the other hand, if the result of determining its own state based on the acquired information indicating its own state shows that it is unlocked (NO in step 902), the processing by the HMD 30 ends.

[0062] Furthermore, if the device is locked (YES in step 902) but is not in a state where it is worn on the head of the user U or where it is not possible to estimate that the device is worn on the head of the user U (NO in step 903), the HMD30 repeats the processing of step 903 until it is in a state where it is worn on the head of the user U or where it is possible to estimate that the device is worn on the head of the user U.

[0063] When an unlock instruction is transmitted from the user terminal 10 (YES in step 905), the HMD 30 accepts the unlock instruction (step 906) and performs control to switch its own device, which is in a locked state, to an unlocked state (step 907). On the other hand, if an unlock instruction is not transmitted from the user terminal 10 (NO in step 905), the HMD 30 repeats the processing of step 905 until an unlock instruction is transmitted from the user terminal 10.

[0064] <Third embodiment> When the third embodiment is compared with the first and second embodiments described above, the configuration of the information processing system is different. (Configuration of information processing system) FIG. 14 is a diagram showing the overall configuration of an information processing system 1 to which the third embodiment is applied. An information processing system 1 to which the third embodiment is applied has an image processing device 50 in addition to the configurations of the first and second embodiments. The image processing device 50 is a device that has, for example, a function to form an image on a recording medium, a function to read the image formed on the recording medium, a function to send and receive image information via communication, and the like.

[0065] (Hardware configuration of user terminal, HMD, and image processing device) 15 is a diagram showing the hardware configuration of an image processing device 50 to which the third embodiment is applied. Note that the hardware configurations of the user terminal 10 and HMD 30 to which the third embodiment is applied are the same as those of the first and second embodiments, and therefore a description thereof will be omitted. An image processing device 50 to which the third embodiment is applied has the same hardware configuration as the user terminal 10 shown in Fig. 2 except for the sensor unit 17 and the imaging unit 18. In addition to the hardware configuration of the user terminal 10 shown in Fig. 2, the image processing device 50 also has an image forming unit 57.

[0066] That is, the image processing device 50 has a control unit 51 configured with a processor such as a CPU, a memory 52 configured with a storage area such as RAM, and a storage unit 53 configured with a storage area such as an HDD, an SDD, or a semiconductor memory. It also has a communication unit 54 that transmits and receives data to and from the user terminal 10 and the HMD 30 via the network 90 or by a communication method such as infrared communication, an operation unit 55 that accepts input operations, a display unit 56 configured with a liquid crystal display, an organic EL display, or the like, and an image forming unit 57. These units are connected by a data bus, an address bus, a PCI bus, or the like.

[0067] The image forming unit 57 forms an image on a storage medium. Specifically, for example, the image forming unit 57 forms and outputs an image based on image information on a sheet of paper as a recording medium by a so-called electrophotographic method in which a toner image is formed on the sheet of paper, or a so-called inkjet method in which ink is ejected onto the sheet of paper.

[0068] (Functional configuration of the control units of the user terminal, HMD, and image processing device) 16 is a diagram showing the functional configuration of a control unit 51 of an image processing device 50 to which the third embodiment is applied. Note that the functional configurations of the control units of the user terminal 10 and the HMD 30 to which the third embodiment is applied are the same as those of the first and second embodiments, and therefore a description thereof will be omitted. In a control unit 51 of an image processing device 50 to which the third embodiment is applied, a switching detection unit 501 and a switching control unit 502 function.

[0069] The switching detection unit 501 detects the unlocking and relocking of the HMD 30. As described above, in the first and second embodiments, the HMD 30 is unlocked as a result of an unlock instruction being transmitted from the user terminal 10 to the HMD 30. Also, the HMD 30 is relocked as a result of a relock instruction being transmitted from the user terminal 10 to the HMD 30. In contrast, in the third embodiment, the switching detection unit 501 detects the unlocking and relocking of the HMD 30. The detection result is used as information necessary for unlocking and relocking the HMD 30 itself.

[0070] The switching control unit 502 switches between unlocking and re-locking the own device based on the detection result by the switching detection unit 501. Specifically, when the detection result by the switching detection unit 501 indicates that the HMD 30 has been unlocked, the switching control unit 502 unlocks the own device that is in a locked state. When the detection result by the switching detection unit 501 indicates that the HMD 30 has been re-locked, the switching control unit 502 re-locks the own device that is in an unlocked state.

[0071] The control unit 51 of the image processing device 50 has the above-described functional configuration, which makes it possible to realize, for example, the following use case: That is, a user U approaches the image processing device 50 while holding a locked user terminal 10 in his / her hand and wearing a locked HMD 30 on his / her head, and performs an operation to unlock the user terminal 10 and the HMD 30. Here, it is assumed that the user U unlocks the user terminal 10 and the HMD 30 by performing an operation to unlock the user terminal 10 while viewing the user terminal 10 through the HMD 30, for example.

[0072] In this case, first, when a user U holding the user terminal 10 approaches the image processing device 50, the user terminal 10 and the image processing device 50 are connected. This connection may be automatic or manual. Examples of connection methods include communication methods such as infrared communication, visible light communication, close proximity wireless communication, Bluetooth (registered trademark), and RFID (registered trademark). Thereafter, when the user U performs an input operation to unlock the user terminal 10 and the HMD 30, the image processing device 50 is also unlocked in conjunction with this.

[0073] As another similar use case, for example, assume that the HMD 30 is unlocked by an operation other than the operation for unlocking the user terminal 10. In this case, when the HMD 30 is unlocked, information required to unlock the image processing device 50 is displayed on the display unit 56 of the image processing device 50. For example, information such as an unlock button, a PIN number, and a QR code (registered trademark) for unlocking the image processing device 50 is displayed. The user U unlocks the image processing device 50 by pressing the unlock button, entering a PIN number using the user terminal 10 connected to the image processing device 50, or performing an operation to read the QR code (registered trademark). In the above use case, only the genuine user U operating the user terminal 10 connected to the image processing device 50 can unlock the image processing device 50.

[0074] As described above, various methods can be employed as an operation performed on the user terminal 10 to unlock the image processing device 50. The degree to which the image processing device 50 is unlocked can be varied depending on the operation method. For example, when the user unlocks the user terminal 10 and the HMD 30 by unlocking the user terminal 10 while viewing the user terminal 10 through the HMD 30, all functions of the image processing device 50 may be unlocked. However, when the HMD 30 is unlocked by any other operation, the functions of the image processing device 50 may be restricted. In this case, for example, the content of information displayed on the display unit 56 of the image processing device 50 may be varied between displaying the entire contents of a file and displaying only the cover of the file.

[0075] Although the present embodiment has been described above, the present invention is not limited to the above-described embodiment. Furthermore, the effects of the present invention are not limited to those described in the above-described embodiment. For example, the system configurations shown in FIGS. 1 and 14 and the hardware configurations shown in FIGS. 2, 3, 9, and 15 are merely examples for achieving the object of the present invention and are not particularly limited. Furthermore, the functional configurations shown in FIGS. 4, 5, 10, 11, and 16 are also merely examples and are not particularly limited. It is sufficient for the information processing system 1 in FIGS. 1 and 14 to have the functionality to execute the above-described processes as a whole, and the functional configurations used to realize these functions are not limited to the examples in FIGS. 4, 5, 10, 11, and 16.

[0076] 6, 7, 12, and 13 are merely examples and are not particularly limited. The processes may not necessarily be performed in chronological order according to the illustrated order of steps, but may be performed in parallel or individually. Furthermore, the specific example of the positional relationship between the user terminal 10, the HMD 30, and the user U shown in FIG. 8 is also merely an example and is not particularly limited.

[0077] Furthermore, in the above-described embodiment, an example has been described in which there is one HMD as the second information processing device, but there may be a plurality of second information processing devices for one first information processing device.

[0078] Furthermore, the image processing device 50 to which the above-described third embodiment is applied is configured not to have a function for acquiring the state of its own device, but may be configured to acquire the state of its own device by including a sensor unit or an imaging unit, for example, like the HMD 30. In this case, like the HMD 30, it is possible to control switching between unlocking and re-locking the image processing device 50.

[0079] Furthermore, in the above-described embodiment, when information to be output from the HMD 30 is specified, a difference can be provided between the information to be output from the user terminal 10 and the information to be output from the HMD 30. However, it is also possible to provide a difference in the content of the information to be output from the HMD 30 depending on the content of the input operation by the user U. For example, when the HMD 30 is unlocked by performing an operation to unlock the user terminal 10 while visually recognizing the user terminal 10 through the HMD 30, it is possible to output from the HMD 30 the same information as the information to be output from the user terminal 10. On the other hand, when the HMD 30 is unlocked by an operation other than the operation to unlock the user terminal 10 while visually recognizing the user terminal 10 through the HMD 30, it is possible to output from the HMD 30 only notification information in which detailed information is omitted.

[0080] Furthermore, in the above-described embodiment, the operation to unlock the HMD 30, which is an eyeglass-type wearable terminal, is performed on the user terminal 10. However, the operation to unlock the HMD 30 may be performed on another wearable terminal. Examples of wearable terminals include wristwatch-type wearable terminals known as smart watches, as well as various types such as ring-type, shoe-type, pocket-type, and pendant-type terminals. Among these types, methods for unlocking the HMD 30 by inputting an input operation on a wristwatch-type wearable terminal include, for example, a method of unlocking the HMD 30 by entering a PIN into the wristwatch-type wearable terminal, a method of unlocking the HMD 30 by authenticating the biometric information of the user U, and a method of forming a symbol on a part of the surface of the wristwatch-type wearable terminal and having the HMD 30 read the symbol to unlock the HMD 30.

[0081] In addition to the above-mentioned methods, there is also a method in which the HMD 30 and the wristwatch-type wearable device are in a predetermined positional relationship and unlock when that state continues for a predetermined time. In this case, the location information of the wristwatch-type wearable device can be used. Generally, the time that the user U wears the wristwatch-type wearable device on their wrist is longer than the time that the user U carries, for example, a user terminal 10 such as a smartphone. Naturally, the location relationship between the wristwatch-type wearable device and the user U is also close. Furthermore, since the wristwatch-type wearable device is in close contact with the user U's wrist, it is easy to obtain biometric information that can also be used to confirm the location information. For this reason, the location information of the wristwatch-type wearable device can be treated as more reliable information than location information from a smartphone (user terminal 10) or the like.

[0082] In this way, the HMD30 can be unlocked by inputting an input operation on the wristwatch-type wearable terminal. However, for example, if a user U has a user terminal 10 such as a smartphone and wears a wristwatch-type wearable terminal on his / her wrist, the HMD30 can be unlocked by inputting an input operation on the user terminal 10 or an input operation on the wristwatch-type wearable terminal.

[0083] In this case, the type and content of information displayed on the HMD 30 when the HMD 30 is unlocked may vary depending on the type of terminal that accepted the input operation to unlock the HMD 30 and the type of input operation. For example, when the HMD 30 is unlocked by entering a PIN into the user terminal 10, it is not possible to ensure that the person who performed the input operation is the genuine user U, so only notification information may be displayed on the HMD 30. Furthermore, when the user unlocks the user terminal 10 and the HMD 30 by unlocking the user terminal 10 while viewing the user terminal 10 through the HMD 30, it is possible to ensure that the person who performed the input operation is the genuine user U, so all information may be displayed on the HMD 30. However, if it is more natural to display information only on the user terminal 10, it may be possible to display nothing on the HMD 30. Furthermore, when the HMD 30 is unlocked by an input operation on a wristwatch-type wearable terminal, it is possible to ensure that the person who performed the input operation is the genuine user U, so all information may be displayed on the HMD 30. [Explanation of symbols]

[0084] 1...information processing system, 10...user terminal, 11...control unit, 30...HMD, 31...control unit, 50...image processing device, 51...control unit, 90...network, 101...connection control unit, 102...operation reception unit, 103...status acquisition unit, 104...status determination unit, 105...switching control unit, 106...designation reception unit, 107...output control unit, 108...learning unit, 301...status acquisition unit, 302...transmission control unit, 303...instruction reception unit, 304...switching control unit, 305...status determination unit, 501...switching detection unit, 502...switching control unit

Claims

1. a processor; The processor: Accepting a predetermined input operation for the player's own aircraft; When the input operation is received, control is performed to switch from a restricted state in which input operations to other information processing devices are restricted to a released state in which the restriction on the input operations is released; when the other information processing device is in a predetermined state and also in the restricted state, and the other information processing device receives the predetermined input operation on the other information processing device, the other information processing device performs control to switch from the restricted state to the released state; when the predetermined input operation is received on the own device in a state where a positional relationship between the other information processing device and the own device satisfies a predetermined condition as the predetermined state, control is performed to switch the other information processing device from the restricted state to the released state; When the other information processing device and the own device are in a predetermined vertical positional relationship as a state that satisfies the predetermined condition, upon receiving the predetermined input operation on the own device, control is performed to switch the other information processing device from the restricted state to the released state. Information processing device.

2. A processor is provided, The processor: Accepting a predetermined input operation for the player's own aircraft; When the input operation is received, control is performed to switch from a restricted state in which input operations to other information processing devices are restricted to a released state in which the restriction on the input operations is released; when the other information processing device is in a predetermined state and also in the restricted state, and the other information processing device receives the predetermined input operation on the other information processing device, the other information processing device performs control to switch from the restricted state to the released state; when receiving the predetermined input operation on the information processing device itself in a case where information indicating that the other information processing device is in the predetermined state is acquired from the other information processing device, performing control to switch the other information processing device from the restricted state to the released state; When information of an image captured by the other information processing device is acquired from the other information processing device as information indicating that the device is in the predetermined state, upon receiving the predetermined input operation on the information processing device itself, control is performed to switch the other information processing device from the restricted state to the released state. Information processing device.

3. A processor is provided, The processor: Accepting a predetermined input operation for the player's own aircraft; When the input operation is received, control is performed to switch from a restricted state in which input operations to other information processing devices are restricted to a released state in which the restriction on the input operations is released; when the other information processing device is in a predetermined state and also in the restricted state, and the other information processing device receives the predetermined input operation on the other information processing device, the other information processing device performs control to switch from the restricted state to the released state; detecting, by the own device, that the other information processing device is in the predetermined state, and when the own device receives the predetermined input operation for the own device, performing control to switch the other information processing device from the restricted state to the released state; The device detects that the other information processing device is in the predetermined state based on information of an image of the other information processing device captured by the device itself, and when the device itself receives the predetermined input operation, controls to switch the other information processing device from the restricted state to the released state. Information processing device.

4. A processor is provided, The processor: Accepting a predetermined input operation for the player's own aircraft; When the input operation is received, control is performed to switch from a restricted state in which input operations to other information processing devices are restricted to a released state in which the restriction on the input operations is released; when the other information processing device is in a predetermined state and also in the restricted state, and the other information processing device receives the predetermined input operation on the other information processing device, the other information processing device performs control to switch from the restricted state to the released state; When the other information processing device is no longer in the predetermined state after switching from the restricted state to the released state, the other information processing device performs control to switch from the released state to the restricted state. Information processing device.

5. the processor controls the switching from the release state to the restricted state when the other information processing device is no longer in the predetermined state and the state continues for a predetermined time after the other information processing device has switched from the restricted state to the release state. The information processing device according to claim 4 .

6. a processor; The processor: When the device itself reaches a predetermined state, the device performs control to transmit information indicating that state to another information processing device; switching from a restricted state in which an input operation on the information processing device itself is restricted to a released state in which the restriction on the input operation is released based on information from the other information processing device; When a positional relationship between the information processing device itself and the other information processing device satisfies a predetermined condition as the predetermined state, control is performed to transmit information indicating that the positional relationship between the information processing device itself and the other information processing device satisfies a predetermined condition, When the predetermined condition is satisfied, and the positional relationship between the information processing device itself and the other information processing device is in a predetermined vertical direction, control is performed to transmit information indicating this to the other information processing device. Information processing device.

7. a receiving unit for receiving a predetermined input operation for the first information processing device; a switching control means for controlling, when the first information processing device receives the input operation, switching from a restricted state in which input operations to the second information processing device are restricted to a released state in which the restriction on the input operations is released; and The switching control means When the second information processing device is in a predetermined state and also in the restricted state, if the second information processing device receives the predetermined input operation on the first information processing device, the second information processing device performs control to switch from the restricted state to the released state; When the predetermined input operation is received on the first information processing device in a state where a positional relationship between the second information processing device and the first information processing device satisfies a predetermined condition as the predetermined state, control is performed to switch the second information processing device from the restricted state to the released state; When the second information processing device and the first information processing device are in a predetermined vertical positional relationship as a state that satisfies the predetermined condition, upon receiving the predetermined input operation on the first information processing device, control is performed to switch the second information processing device from the restricted state to the released state. Information processing system.

8. A receiving means for receiving a predetermined input operation for a first information processing device; a switching control means for controlling, when the first information processing device receives the input operation, switching from a restricted state in which input operations to the second information processing device are restricted to a released state in which the restriction on the input operations is released; and The switching control means When the second information processing device is in a predetermined state and also in the restricted state, if the second information processing device receives the predetermined input operation on the first information processing device, the second information processing device performs control to switch from the restricted state to the released state; when the information indicating that the second information processing device is in the predetermined state is acquired from the second information processing device, and when the predetermined input operation to the first information processing device is accepted, control is performed to switch the second information processing device from the restricted state to the released state; When information of an image captured by the second information processing device is acquired from the second information processing device as information indicating that the first information processing device is in the predetermined state, if the predetermined input operation on the first information processing device is accepted, control is performed to switch the second information processing device from the restricted state to the released state. Information processing system.

9. A receiving means for receiving a predetermined input operation for a first information processing device; a switching control means for controlling, when the first information processing device receives the input operation, switching from a restricted state in which input operations to the second information processing device are restricted to a released state in which the restriction on the input operations is released; and The switching control means When the second information processing device is in a predetermined state and also in the restricted state, if the second information processing device receives the predetermined input operation on the first information processing device, the second information processing device performs control to switch from the restricted state to the released state; the first information processing device detects that the second information processing device is in the predetermined state, and when the first information processing device receives the predetermined input operation on the first information processing device, controls the second information processing device to switch from the restricted state to the released state; The second information processing device is characterized in that it detects that the second information processing device is in the predetermined state based on information of an image of the second information processing device captured by the first information processing device, and when the predetermined input operation on the first information processing device is accepted, it performs control to switch the second information processing device from the restricted state to the released state. Information processing system.

10. A receiving means for receiving a predetermined input operation for a first information processing device; a switching control means for controlling, when the first information processing device receives the input operation, switching from a restricted state in which input operations to the second information processing device are restricted to a released state in which the restriction on the input operations is released; and The switching control means When the second information processing device is in a predetermined state and also in the restricted state, if the second information processing device receives the predetermined input operation on the first information processing device, the second information processing device performs control to switch from the restricted state to the released state; When the second information processing device is no longer in the predetermined state after switching from the restricted state to the released state, the second information processing device performs control to switch from the released state to the restricted state. Information processing system.

11. On the computer, a function of accepting a predetermined input operation for the player's own device; When the input operation on the information processing device itself is received, the information processing device performs control to switch from a restricted state in which the input operation on the information processing device itself is restricted to a released state in which the restriction on the input operation is released, as follows: a control for switching the other information processing device from the restricted state to the released state when the other information processing device is in a predetermined state and also in the restricted state and receives the predetermined input operation on the own device; control to switch the other information processing device from the restricted state to the released state when the predetermined input operation is received on the own device in a state where a positional relationship between the other information processing device and the own device satisfies a predetermined condition as the predetermined state; a function of performing control to switch the other information processing device from the restricted state to the released state when the other information processing device and the own device are in a predetermined vertical positional relationship as a state that satisfies the predetermined condition and the predetermined input operation is received on the own device; A program to achieve this.

12. A computer comprising: a function of accepting a predetermined input operation for the player's own device; When the input operation on the information processing device itself is received, the information processing device performs control to switch from a restricted state in which the input operation on the information processing device itself is restricted to a released state in which the restriction on the input operation is released, as follows: a control for switching the other information processing device from the restricted state to the released state when the other information processing device is in a predetermined state and also in the restricted state and receives the predetermined input operation on the own device; a control for switching the other information processing device from the restricted state to the released state when the predetermined input operation is received on the information processing device when the information indicating that the other information processing device is in the predetermined state is acquired from the other information processing device; control to switch the other information processing device from the restricted state to the released state when the predetermined input operation is received on the information processing device itself in a case where information of an image captured by the other information processing device is acquired from the other information processing device as information indicating that the information processing device is in the predetermined state; A program to achieve this.

13. A computer comprising: a function of accepting a predetermined input operation for the player's own device; When the input operation on the information processing device itself is received, the information processing device performs control to switch from a restricted state in which the input operation on the information processing device itself is restricted to a released state in which the restriction on the input operation is released, as follows: a control for switching the other information processing device from the restricted state to the released state when the other information processing device is in a predetermined state and also in the restricted state and receives the predetermined input operation on the own device; detecting, by the own device, that the other information processing device is in the predetermined state and, when the own device receives the predetermined input operation for the own device, controlling to switch the other information processing device from the restricted state to the released state; detecting that the other information processing device is in the predetermined state based on information on an image of the other information processing device captured by the own device, and switching the other information processing device from the restricted state to the released state when the own device receives the predetermined input operation; A program to achieve this.

14. A computer comprising: a function of accepting a predetermined input operation for the player's own device; When the input operation on the information processing device itself is received, the information processing device performs control to switch from a restricted state in which the input operation on the information processing device itself is restricted to a released state in which the restriction on the input operation is released, as follows: a control for switching the other information processing device from the restricted state to the released state when the other information processing device is in a predetermined state and also in the restricted state and receives the predetermined input operation on the own device; a control for switching the other information processing device from the release state to the restricted state when the other information processing device is no longer in the predetermined state after switching from the restricted state to the release state; A program to achieve this.

15. On the computer, a function of controlling the transmission of information indicating that the device itself has entered a predetermined state to other information processing devices when the device itself has entered a predetermined state; a function of switching from a restricted state in which an input operation on the information processing device itself is restricted to a released state in which the restriction on the input operation is released based on information from the other information processing device; a function of performing control such that, when a positional relationship between the own device and the other information processing device satisfies a predetermined condition as the predetermined state, information indicating that the positional relationship between the own device and the other information processing device satisfies a predetermined condition is transmitted to the other information processing device; a function of performing control to transmit information indicating that the predetermined condition is satisfied when the positional relationship between the information processing device itself and the other information processing device in a predetermined vertical direction is satisfied, to the other information processing device; A program to achieve this.

Images