Information processing device and control method

Abstract

To provide an information processing device, a control method and the like for facilitating operation input using an input interface.SOLUTION: An information processing device is a head-mounted type information processing device. The information processing device includes a communication unit which communicates with an external device, a capturing unit which creates a captured image, a display unit which displays the image, and a control unit. The capturing unit creates the captured image on which an input interface connected to the external device is captured. The control unit performs control to display the image acquired from the external device in a first display region which is a part of a display region of the display unit, and display the captured image including the input interface in a second display region which is another part of the display region.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an information processing device and a control method. [Background technology]

[0002] Conventionally, a head-mounted display (HMD), which is a head-mounted information processing device, has been widely known. For example, by covering a user's field of vision with an HMD, it is possible to provide a service using immersive virtual reality (VR) technology. Patent Document 1 discloses a method for displaying an image of a desk top on a desk object in a virtual space when an HMD is used. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2021-125834 A Summary of the Invention [Problem to be solved by the invention]

[0004] There is a method of using a keyboard or mouse formed in a virtual space to input operations while wearing an HMD, but this cannot be said to be highly efficient. Also, the method in Patent Document 1 only displays an actual image of the user's hands and is not specialized for input using a keyboard, etc.

[0005] According to some aspects of the present disclosure, it is possible to provide an information processing device and a control method that facilitate operation input using an input interface. [Means for solving the problem]

[0006] One aspect of the present disclosure relates to an information processing device that is a head-mounted type, including a communication unit that communicates with an external device, an imaging unit that generates an image by imaging a given imaging range, a display unit that displays an image, and a control unit, wherein the imaging unit generates the image in which the input interface is imaged by imaging the imaging range including an input interface connected to the external device, and the control unit controls the information processing device to display the image obtained from the external device in a first display area that is a part of the display area of ​​the display unit, and to display the image including the input interface in a second display area that is another part of the display area.

[0007] Another aspect of the present disclosure relates to a control method for a head-mounted information processing device having a communication unit that communicates with an external device, an imaging unit that generates an image by imaging a given imaging range, and a display unit that displays an image, the control method controlling the control to generate the image captured by the input interface by imaging the imaging range including an input interface connected to the external device, display an image obtained from the external device in a first display area that is a part of the display area of ​​the display unit, and display the image including the input interface in a second display area that is another part of the display area. [Brief description of the drawings]

[0008] [Figure 1A] 1 is a configuration example of an information processing system including an information processing device. [Figure 1B] 1 is a configuration example of an information processing system including an information processing device. [Diagram 2] FIG. 1 is a perspective view showing a configuration example of an information processing device. [Diagram 3] 1 is a configuration example of an information processing device. [Figure 4] 11 is a flowchart illustrating a process in an information processing device. [Diagram 5] 4 is an example of a screen displayed on a display unit of an information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the present embodiment will be described with reference to the drawings. In the drawings, the same or equivalent elements are given the same reference numerals, and duplicated explanations are omitted. Note that the present embodiment described below does not unduly limit the contents described in the claims. In addition, all of the configurations described in the present embodiment are not necessarily essential configurations of the present disclosure.

[0010] 1. System configuration example 1A and 1B are diagrams showing an example of the configuration of an information processing system 10 including an information processing device 100 according to this embodiment. The information processing system 10 includes the information processing device 100, an external device 200, and an input interface 300.

[0011] The information processing device 100 is a device that can be worn by a user and allows the user to experience a virtual reality space. For example, the information processing device 100 is a head-mounted display that can be worn on the user's head. Note that the information processing device 100 is not limited to a device that only performs processing that allows the user to experience a virtual reality space, and may be capable of presenting a flat image to the user.

[0012] The external device 200 is a device configured separately from the information processing device 100. The external device 200 may be, for example, a smartphone as shown in Fig. 1A, a PC (Personal Computer) as shown in Fig. 1B, or another device connectable to the information processing device 100.

[0013] The information processing device 100 is connected to an external device 200. The information processing device 100 and the external device 200 may be connected by wire using a cable conforming to a method such as USB (Universal Serial Bus), DisplayPort, or HDMI (High-Definition Multimedia Interface), or may be connected wirelessly using a communication method conforming to a standard such as Bluetooth (registered trademark) or IEEE802.11. The method used to connect the information processing device 100 and the external device 200 is not limited to the above example, and various modifications are possible.

[0014] The input interface 300 is a device that is connected to the external device 200 and is used as a user interface for the external device 200. The external device 200 and the input interface 300 may be connected in a wired manner using a USB cable or the like, or may be connected wirelessly using a communication method conforming to Bluetooth or the like.

[0015] The input interface 300 is a tool that exists in the real space. In other words, the input interface 300 is not a tool displayed on a display or a tool formed in a virtual space. The input interface 300 may be a physical keyboard or a pointing device such as a mouse. An example in which the input interface 300 is a physical keyboard will be described below.

[0016] 1B shows an example in which a display 400, which is a display unit, is connected to the external device 200, which is a PC. However, in the method of the present embodiment, since an image output from the external device 200 can be displayed on the information processing device 100 as described later, the display 400 of the external device 200 may be omitted. Furthermore, devices other than the input interface 300 and the display 400 may be connected to the external device 200.

[0017] 2 is a diagram showing an example of the configuration of the information processing device 100. The information processing device 100 includes a display unit 140, an imaging unit 150, and an operation unit 160.

[0018] The display unit 140 includes a right-eye display disposed in front of the user's right eye, and a left-eye display disposed in front of the user's left eye. The display unit 140 performs stereoscopic display by displaying different images on the left and right. For example, the display unit 140 is configured with a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like. Alternatively, the display unit 140 may be configured with a single display, with the display divided into regions and images for the right eye and the left eye displayed in each region.

[0019] The imaging unit 150 generates a captured image by capturing an image of a given imaging range in real space. The imaging unit 150 may be, for example, a camera that captures an image in the front direction of the information processing device 100. By displaying the captured image on the display unit 140, information of the real space can be presented to the user.

[0020] The number, arrangement, imaging direction (direction of the optical axis), and angle of view of the imaging units 150 can be modified in various ways. For example, the imaging unit 150 is a camera capable of capturing an image in the front direction, and the captured range may be changed by changing the direction of the face of the user wearing the information processing device 100. For example, the imaging unit 150 captures an image of the user's hands by turning the face downward. The imaging unit 150 may include multiple imaging units each having a different captureable range. For example, the imaging unit 150 may include a first imaging unit that captures an image in the front direction and a second imaging unit that captures an image downward (the user's hands).

[0021] The operation unit 160 is an interface used to operate the information processing device 100. In FIG. 2, an operation unit 160a for a left hand and an operation unit 160b for a right hand are illustrated as examples of the operation unit 160. For example, a user holds the grip portion of the operation unit 160a and presses a button provided at the upper end of the grip, or changes the position or angle of the operation unit 160a to perform an operation. The same applies to the operation unit 160b for a right hand. The operation units 160a and 160b are connected to the main body (goggle unit) of the information processing device 100, for example, by wire or wirelessly. Although not illustrated in FIG. 2, the operation unit 160 may include a button or the like provided on the main body of the information processing device 100.

[0022] 3 is a diagram showing an example of the configuration of the information processing device 100. The information processing device 100 includes a control unit 110, a storage unit 120, a communication unit 130, a display unit 140, an imaging unit 150, and an operation unit 160.

[0023] The control unit 110 is connected to each unit of the information processing device 100 and controls each unit. The control unit 110 is configured by the following hardware. The hardware can include at least one of a circuit for processing digital signals and a circuit for processing analog signals. For example, the hardware can be configured by one or more circuit devices or one or more circuit elements mounted on a circuit board. The one or more circuit devices are, for example, an IC (Integrated Circuit), an FPGA (Field-programmable gate array), etc. The one or more circuit elements are, for example, a resistor, a capacitor, etc.

[0024] The control unit 110 may be realized by the following processor. The information processing device 100 of this embodiment includes a memory that stores information and a processor that operates based on the information stored in the memory. The information is, for example, a program and various data. The processor includes hardware. The processor can use various processors such as a central processing unit (CPU), a graphics processing unit (GPU), and a digital signal processor (DSP). The memory may be a semiconductor memory such as a static random access memory (SRAM) or a dynamic random access memory (DRAM), a register, a magnetic storage device such as a hard disk drive (HDD), or an optical storage device such as an optical disk device. For example, the memory stores computer-readable instructions, and the function of the control unit 110 is realized as a process by the processor executing the instructions. The instructions here may be instructions of an instruction set constituting a program, or instructions that instruct a hardware circuit of the processor to operate.

[0025] The storage unit 120 is a work area for the control unit 110, and stores various information. The storage unit 120 can be realized by various types of memory, and the memory may be a semiconductor memory, a register, a magnetic storage device, or an optical storage device.

[0026] The communication unit 130 is an interface for performing communication via a network. For example, when performing wireless communication, the communication unit 130 may include an antenna, an RF (radio frequency) circuit, and a baseband circuit. However, the communication unit 130 may perform wired communication, and in that case, the communication unit 130 may include a communication interface such as an Ethernet connector and a control circuit for the communication interface. The communication unit 130 may operate according to the control by the control unit 110, or may include a processor for communication control different from the control unit 110.

[0027] The communication unit 130 of the present embodiment communicates with, for example, the external device 200. As described above, various modifications are possible for the communication method between the information processing device 100 and the external device 200, and the communication unit 130 may be a connector such as a USB, DisplayPort, or HDMI, a communication chip for Bluetooth, a communication chip for IEEE802.11, or another communication interface.

[0028] The display unit 140 is a display for the HMD as described above. The imaging unit 150 is a camera or the like that captures an image of the real space. The operation unit 160 is an operation interface such as the operation units 160a and 160b shown in FIG.

[0029] In the information processing system 10 shown in FIG. 1A and FIG. 1B, for example, the control unit 110 of the information processing device 100 acquires an image from the external device 200 via the communication unit 130, and controls the display unit 140 to display the image. The image here may be an image acquired by the external device 200 from another device. For example, the external device 200, which is a smartphone, may communicate with a server system that provides a cloud service. The external device 200 acquires an image (for example, a service use screen) from the server system and transmits the image to the information processing device 100. In this way, the information processing device 100 can use a cloud service via the external device 200. The cloud service here may be a service that uses various applications such as document creation, table creation, and image editing, or may be a distribution service of video content such as movies. In addition, the use of the cloud service is not essential, and the information processing device 100 may acquire an image generated by the external device 200.

[0030] However, when using an HMD as a display device for using an application, depending on the type of application, it is possible that the operation unit 160 of the HMD may not be sufficiently easy to operate. For example, various applications provided by an OS such as Windows (registered trademark) include applications that are intended to be operated using a mouse or keyboard. Also, virtual input interfaces have issues in terms of usability, such as the lack of physical feedback (e.g., key press response).

[0031] Therefore, in the information processing device 100 of the present embodiment, a physical interface is utilized by displaying an image of an input interface 300 connected to an external device 200. Specifically, the information processing device 100 of the present embodiment is a head-mounted information processing device, and includes a communication unit 130, an imaging unit 150, a display unit 140, and a control unit 110. The imaging unit 150 generates an image of the input interface 300 by imaging an imaging range including the input interface 300 connected to the external device 200. This may be a case where the imaging unit 150 that images downward images an image of a physical keyboard at hand when the user faces forward, or a case where the imaging unit 150 that images the front direction images an image of a physical keyboard at hand when the user faces downward. The control unit 110 controls the display unit 140 to display an image acquired from the external device 200 in a first display area RE1, which is a part of the display area, and to display an image including the input interface 300 in a second display area RE2, which is another part of the display area. Hereinafter, the image displayed by this control will also be referred to as a display image IM. A specific example of the display image IM will be described later with reference to FIG.

[0032] Since a captured image of the input interface 300 such as a physical keyboard is displayed in a part of the display area, the user can perform operations while visually checking the state of the physical keyboard existing in the real space and the positional relationship with the user's hands. In this embodiment, the device that provides the image displayed in the first display area RE1 to the information processing device 100 and the device to which the input interface 300 is connected are a common device (external device 200). Therefore, it becomes possible to use the operation result of the input interface 300 for the operation of the application being displayed, etc., without performing complex linkage control.

[0033] In addition, a part or all of the processing performed by the information processing device 100 of the present embodiment may be realized by a program. The processing performed by the information processing device 100 is, in the narrow sense, the processing performed by the control unit 110.

[0034] The program according to this embodiment can be stored in a non-transitory information storage medium (information storage device), which is a medium readable by a computer, for example. The information storage medium can be realized by, for example, an optical disk, a memory card, a HDD, or a semiconductor memory. The semiconductor memory is, for example, a ROM. The control unit 110 and the like perform various processes of this embodiment based on the program stored in the information storage medium. That is, the information storage medium stores a program for making a computer function as the control unit 110 and the like. A computer is a device equipped with an input device, a processing unit, a storage unit, and an output unit. Specifically, the program according to this embodiment is a program for making a computer (specifically, an HMD) execute each step described later with reference to FIG. 4 and the like.

[0035] The technique of the present embodiment can also be applied to a control method for a head-mounted information processing device having a communication unit that communicates with an external device, an imaging unit that generates an image by imaging a given imaging range, and a display unit that displays the image. The control method includes a step of generating an image in which the input interface is imaged by imaging an imaging range including the input interface connected to the external device, and a step of controlling to display the image acquired from the external device in a first display area that is a part of the display area of ​​the display unit, and to display the image including the input interface in a second display area that is another part of the display area.

[0036] 2. Processing Details 2.1 Processing flow 4 is a flowchart for explaining the processing of the information processing device 100. When this processing starts, in step S101, the control unit 110 performs control to display an image acquired from the external device 200 on the display unit 140. In step S101, the control unit 110 does not perform control to display a captured image. For example, the control unit 110 may display the image acquired from the external device 200 using the entire display area of ​​the display unit 140.

[0037] In step S102, the control unit 110 determines whether or not to start imaging by the imaging unit 150. Details of the determination process in step S102 will be described later. If it is determined not to start imaging (step S102: No), the control unit 110 returns to step S102 and periodically executes the same determination. In other words, the control unit 110 continues to display only the image acquired from the external device 200 while keeping the imaging unit 150 off (or in a standby state).

[0038] When it is determined that imaging should be started (step S102: Yes), in step S103, the control unit 110 performs control to turn on the imaging unit 150. As a result, acquisition of an image captured by the imaging unit 150 is started.

[0039] In step S104, the control unit 110 determines whether to start a through display. The through display refers to a display control that presents information of the real space to the user wearing the HMD by displaying a captured image. The determination process in step S104 will be described later in detail.

[0040] In addition, in a broad sense, through display may include control of displaying an image of a range equivalent to the user's field of view across the entire display area of ​​the display unit 140 (showing information of the real space as if the HMD was not being worn), but in the following it will be taken to mean control of simultaneously displaying an image from the external device 200 and the captured image.

[0041] If it is determined that the through display is not to be performed (step S104: No), the control unit 110 returns to step S104 and periodically executes the same determination. In other words, the control unit 110 continues to control the display of only the image acquired from the external device 200 while acquiring the captured image.

[0042] If it is determined that a through display should be started (step S104: Yes), in step S105, the control unit 110 performs a through display by continuing to display an image from the external device 200 in the first display area RE1 of the display unit 140 while displaying a captured image in the second display area RE2 (displaying the display image IM).

[0043] FIG. 5 is an example of a display image IM when a through display is performed. For example, the first display area RE1 may be an upper area of ​​the display area of ​​the display unit 140, and the second display area RE2 may be a lower area of ​​the display area of ​​the display unit 140. In this way, the positional relationship between the display screen (first display area RE1) of an application or the like and the keyboard (second display area RE2) can be made the same as the positional relationship when using a general PC, etc., so that an interface that is easy for the user to use can be realized. Note that FIG. 5 shows an example in which a window corresponding to a given application is displayed in the first display area RE1, but the image displayed in the first display area RE1 is not limited to this, and various images output from the external device 200 can be displayed.

[0044] As shown in Fig. 5, the first display area RE1 and the second display area RE2 may each be a rectangular area. However, the positions, sizes, and shapes of the first display area RE1 and the second display area RE2 are not limited to the example in Fig. 5, and various modifications are possible. Furthermore, the first display area RE1 and the second display area RE2 are not limited to exclusive areas (non-overlapping areas), and may partially overlap. In this case, in the overlapping area, one image may be subjected to a transparency process and then superimposed on the other image.

[0045] As shown in Fig. 5, the captured image displayed in the second display area RE2 includes the physical keyboard, so if the user's hands are within the captured image (e.g., on the physical keyboard), the hands are also included in the captured image. This makes it possible to present the user with the positional relationship between the hands and the physical keyboard, making operation easier. The image displayed in the second display area RE2 may be the entire captured image, or a portion of the captured image that includes the physical keyboard.

[0046] After the through display starts, in step S106, the control unit 110 determines whether or not to end the through display. The control unit 110 determines to end the through display when, for example, a predetermined operation is performed on the operation unit 160 of the information processing device 100. Alternatively, the control unit 110 may determine to end the through display when a predetermined operation is performed on a physical keyboard. However, when a physical keyboard is used, it is necessary to distinguish between an operation to end the through display and a normal operation for an application or the like, so an operation that is sufficiently unlikely to be performed as a normal operation may be set as the operation to end the through display.

[0047] If it is determined that the through display should not be ended (step S106: No), the control unit 110 returns to step S105 and continues the process. Specifically, the through display (display of the display image IM) is continued. If it is determined that the through display should be ended (step S106: Yes), the control unit 110 returns to step S101 and continues the process. Specifically, the control unit 110 performs control to turn off the imaging unit 150 and control to display only the image from the external device 200 on the display unit 140.

[0048] If it is determined that the through display is not to be ended (step S106: No), the control unit 110 may return to step S104 and continue the process. Specifically, the control unit 110 may transition to a state in which the through display is not performed while keeping the imaging unit 150 on. In addition, the flow of the process in the information processing device 100 is not limited to that in FIG. 4, and various modifications are possible.

[0049] 2.2 First judgment processing example Next, some specific examples of the imaging start determination (step S102) and the through-display start determination (step S104) in FIG. 4 will be described.

[0050] The communication unit 130 may acquire operation information indicating that a user operation on the input interface 300 has been detected from the external device 200. As described above, since the input interface 300 is an interface connected to the external device 200, the external device 200 can easily acquire the operation result of the input interface 300. The external device 200 outputs information indicating the operation result to the information processing device 100 as operation information. Then, when the operation information is acquired, the control unit 110 may start control to display the captured image in the second display area RE2. In other words, the determination of the start of the through display shown in step S104 of FIG. 4 may be a process of determining whether or not a user operation has been performed on the input interface 300. In this way, when the user is likely to use the input interface 300, it is possible to present the state of the input interface 300 using the through display.

[0051] Note that the user operation here may represent an operation on any key, not an operation limited to a specific key. For example, even when the user is wearing the HMD, it is considered that the user remembers the approximate location of the physical keyboard based on the positional relationship before wearing it. Therefore, when the user wants to operate the physical keyboard, it is relatively easy to find the physical keyboard by touch and operate some key even if the physical keyboard is not visible. In other words, by not excessively limiting the content of the user operation, the user can appropriately perform the operation to start the through display. However, the user operation here is not prevented from being an operation on a specific key of the physical keyboard (for example, a relatively large Enter key or an Escape key located at the upper left corner, etc.).

[0052] Furthermore, when the operation information is acquired, the control unit 110 may control the imaging unit 150 to start imaging. In other words, the imaging start determination shown in step S102 in Fig. 4 may be a process of determining whether or not a user operation has been performed on the input interface 300, similar to the determination of the start of the through display. In this way, imaging is started when the need for the through display is sufficiently high. When the need for the through display is low, the imaging unit 150 is not started, which makes it possible to suppress power consumption by the imaging unit 150, etc.

[0053] 4, an example is shown in which the determination of the start of imaging (step S102) and the determination of the start of through display (step S104) are different steps, but these may be combined. For example, in the above example, the control unit 110 may determine whether or not a user operation on the input interface 300 has been detected, and if detected, may cause the imaging unit 150 to start imaging and the display unit 140 to start through display.

[0054] However, when the operation of the input interface 300 is used as the start trigger for the through display, the start trigger for the imaging may be different. For example, the control unit 110 may perform control to start imaging by the imaging unit 150 when an operation input to the operation unit 160 of the information processing device 100 is accepted. In this way, the user can instruct the start of imaging using an interface (operation unit 160) that is easy to use even while wearing the HMD.

[0055] 2.3 Second judgment processing example The start trigger of the through display may be based on image processing. For example, on the premise that imaging by the imaging unit 150 has started, the control unit 110 may perform image processing to detect the input interface 300 from the captured image generated by the imaging unit 150. Note that various methods are known for object detection processing to detect a specific object from an image, and detailed description thereof will be omitted since they can be widely applied in this embodiment.

[0056] Then, when the control unit 110 determines that the captured image includes the input interface 300, it starts control to display the captured image in the second display area RE2. For example, when the image processing is a process for determining the presence or absence of an object in a binary manner, the control unit 110 starts a through display when the input interface 300 is detected. Also, when the image processing is a process for outputting a score representing the likelihood of an object, the control unit 110 may start a through display when the score related to the input interface 300 is equal to or greater than a predetermined threshold value.

[0057] If the captured image does not include the input interface 300, displaying the captured image may not contribute to improving operability. In this regard, by using the detection result of the input interface 300, it becomes possible to start displaying the captured image when the through display is enabled. Note that the control unit 110 may make a determination based on the position or size of the input interface 300 in the captured image. For example, the control unit 110 may start through display when the input interface 300 is captured at a certain size or larger near the center of the captured image.

[0058] Furthermore, the control unit 110 may perform image processing to detect the user's hand from the captured image generated by the imaging unit 150. Then, when the control unit 110 determines that the captured image includes the input interface 300 and that the user's hand has been detected in a position overlapping the input interface 300, the control unit 110 may start control to display the captured image in the second display area RE2.

[0059] When a user performs an operation using the input interface 300, it is assumed that the user's hand is located near the input interface 300 (for example, the state shown in the second display area RE2 in FIG. 5). Therefore, by making a determination regarding the user's hand in addition to the presence or absence of the input interface 300, it becomes possible to start through display when it is estimated that the user is highly likely to operate the input interface 300.

[0060] When image processing is used as a trigger for starting the through display, it is necessary to acquire a captured image beforehand. Therefore, the control unit 110 determines whether or not to start capturing images using a method other than image processing. For example, the control unit 110 may control the image capturing unit 150 to start capturing images when an operation input is received on the operation unit 160 of the information processing device 100. In this way, the user can instruct the start of capturing images using an interface that is easy to use even while wearing the HMD.

[0061] 2.4 Third judgment processing example Furthermore, the control unit 110 may start control to display the captured image in the second display area RE2 when an operation input is received to the operation unit 160 of the information processing device 100. In this way, the user can instruct the start of through display using an interface that is easy to use even while wearing the HMD.

[0062] In this case, if the input interface 300 is included in the displayed captured image, the user can immediately start operations using the input interface 300. Even if the captured image does not include the input interface 300, the through display has started, so the user can search for the input interface 300 based on the information of the real space displayed in the second display area RE2.

[0063] Although the present embodiment has been described in detail as above, it will be easily understood by those skilled in the art that many modifications are possible that do not substantially deviate from the novel matters and effects of the present embodiment. Therefore, all such modifications are intended to be included in the scope of the present disclosure. For example, a term described at least once in the specification or drawings together with a different term having a broader meaning or synonymy can be replaced with that different term anywhere in the specification or drawings. In addition, all combinations of the present embodiment and modifications are also included in the scope of the present disclosure. In addition, the configurations and operations of the information processing device, external device, input interface, etc. are not limited to those described in the present embodiment, and various modifications are possible. [Explanation of symbols]

[0064] 10...information processing system, 100...information processing device, 110...control unit, 120...storage unit, 130...communication unit, 140...display unit, 150...imaging unit, 160, 160a, 160b...operation unit, 200...external device, 300...input interface, 400...display, IM...display image, RE1...first display area, RE2...second display area

Claims

1. A head-mounted information processing device, A communication unit that communicates with an external device; an imaging unit that captures an image of a given imaging range to generate a captured image; A display unit for displaying an image; A control unit; Including, The imaging unit includes: capturing an image of the imaging range including an input interface connected to the external device to generate the captured image of the input interface; The control unit is An information processing device that controls so that an image obtained from the external device is displayed in a first display area that is a part of the display area of ​​the display unit, and so that the captured image including the input interface is displayed in a second display area that is another part of the display area.

2. In claim 1, The communication unit is acquiring operation information from the external device, the operation information indicating that a user operation on the input interface has been detected; The control unit is When the operation information is acquired, the information processing device starts control to display the captured image in the second display area.

3. In claim 2, The control unit is When the operation information is acquired, the information processing device performs control to start imaging by the imaging unit.

4. In claim 2, Further comprising an operation unit through which a user inputs an operation, The control unit is An information processing apparatus that performs control to start imaging by the imaging unit when the operation input to the operation unit is received.

5. In claim 1, The control unit is performing image processing to detect the input interface from the captured image generated by the imaging unit; When it is determined that the captured image includes the input interface, the information processing device starts control to display the captured image in the second display area.

6. In claim 5, performing image processing to detect a user's hand from the captured image generated by the imaging unit; An information processing device that starts control to display the captured image in the second display area when it is determined that the captured image includes the input interface and that the user's hand has been detected in a position overlapping the input interface.

7. In claim 5, Further comprising an operation unit through which a user inputs an operation, The control unit is An information processing apparatus that performs control to start imaging by the imaging unit when the operation input to the operation unit is received.

8. In claim 1, Further comprising an operation unit through which a user inputs an operation, The control unit is When the operation input to the operation unit is received, the information processing device starts control to display the captured image in the second display area.

9. In any one of claims 1 to 8, the first display area is an upper area of ​​the display area of ​​the display unit, The information processing device, wherein the second display area is a lower area of ​​the display area of ​​the display unit.

10. A method for controlling a head-mounted information processing device having a communication unit that communicates with an external device, an imaging unit that generates an image by imaging a given imaging range, and a display unit that displays the image, comprising: capturing an image of the imaging range including an input interface connected to the external device to generate the captured image of the input interface; and performing control to display the image acquired from the external device in a first display area that is a part of a display area of ​​the display unit, and to display the captured image including the input interface in a second display area that is another part of the display area. Control methods.

Images