Control device
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- CANON KK
- Filing Date
- 2023-07-26
- Publication Date
- 2026-07-29
AI Technical Summary
In the prior art, when using a virtual keyboard to input characters in virtual reality, tactile perception cannot be provided, input speed and accuracy are poor, and physical keyboard display may interfere with user immersion or cannot be displayed in time.
By controlling the image of the virtual space, the image of the physical keyboard is displayed to facilitate the user to enter characters, the image capturing unit of the head-mounted display detects the position of the physical keyboard, and displays it correspondingly in the virtual space, or displays the physical space image when it is too far away or is not connected.
Realizes visualization of physical keyboards in virtual reality, improving input speed and accuracy while maintaining user immersion.
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a control device, and more particularly to a character input method when wearing a head mounted display. [Background technology]
[0002] In recent years, a technology called virtual reality (VR) has become widespread, which allows users to experience a computer-generated virtual space as if it were real. For example, a user can wear a head-mounted display (HMD) on their head to experience (view) virtual reality images. When the HMD is worn on the user's head, the user's field of vision is covered by the HMD. This makes it possible to provide the user with a highly immersive visual experience.
[0003] While experiencing virtual reality images using an HMD, a user can perform various tasks, such as typing. In a conventional method of typing in a virtual space, a virtual keyboard generated by CG is displayed in the virtual space, and the user types characters by operating the virtual keyboard. However, since the virtual keyboard is CG without a physical substance and the user cannot obtain a tactile sensation when typing characters, typing using a virtual keyboard is inferior in typing speed and accuracy to typing using a physical keyboard in the real world. Therefore, it is desirable to be able to use a real-world keyboard when typing characters in a virtual space.
[0004] Methods for visually recognizing real-world objects in a virtual space are disclosed in Patent Documents 1 and 2. Patent Document 1 discloses that when external information that matches preregistered information is detected, the information is displayed on a head-mounted display. Patent Document 2 discloses that an image of the area around the user's hand in the real world is superimposed on the virtual space and displayed. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] JP 2005-086328 A [Patent Document 2] JP 2020-042206 A Summary of the Invention [Problem to be solved by the invention]
[0006] However, in the method disclosed in Patent Document 1, the real-world keyboard is unnecessarily displayed, which may hinder the user's immersion. In the method disclosed in Patent Document 2, if the real-world keyboard is not located near the user's hands, the user cannot see the real-world keyboard.
[0007] An object of the present invention is to enable a user experiencing a virtual reality image to view a character input device (eg, a keyboard) in the real world at a desired timing. [Means for solving the problem]
[0008] The control device of the present invention has a control means capable of controlling so that an image of a virtual space is displayed on a head mounted display and so that a text field for character input is displayed in the virtual space, and a detection means for detecting a character input device from an image of a real space captured by an imaging means, and the control means displays an image representing the detected character input device when the text field is not displayed and character input is possible. The present invention is characterized in that, when the character input state is present, an image representing the detected character input device from the image of the real space is controlled to be displayed in the virtual space rather than being controlled to be displayed in the virtual space. Effect of the Invention
[0009] According to the present invention, a user experiencing a virtual reality image can view a character input device (eg, a keyboard) in the real world at a desired timing. [Brief description of the drawings]
[0010] [Figure 1] FIG. 2 is a block diagram showing a configuration of a head mounted display. [Diagram 2] 13 is a flowchart of processing of the head mounted display. [Diagram 3] 13 is a flowchart of a display control process. [Figure 4] FIG. 2 is a schematic diagram showing a virtual space image. [Diagram 5] 13 is a modified example of the flowchart of the display control process. [Figure 6] 13 is another modified example of the flowchart of the display control process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Although an example in which the present invention is applied to a head mounted display will be described, the control device of the present invention may be a device separate from the head mounted display. For example, the control device of the present invention may be a PC (personal computer) connected to the head mounted display.
[0012] 1 is a block diagram showing the configuration of a head mounted display 100 according to this embodiment. The head mounted display 100 includes a control unit 101, a memory 102, a non-volatile memory 103, an imaging unit 104, a posture detection unit 105, an operation unit 106, a display unit 107, an audio output unit 108, an external I / F 109, and a communication I / F 110. These are connected to an internal bus 150, and can exchange data with each other via the internal bus 150.
[0013] The control unit 101 is composed of, for example, at least one processor or circuit, and controls the entire head mounted display 100. The memory 102 is composed of, for example, a RAM (a volatile memory using a semiconductor element, etc.). The non-volatile memory 103 stores various data (information) such as image data, audio data, other data, and various programs for the operation of the control unit 101. The non-volatile memory 103 is composed of, for example, a flash memory, a ROM, a HDD, or an SSD. The control unit 101 controls each part of the head mounted display 100 according to, for example, a program stored in the non-volatile memory 103, using the memory 102 as a work memory.
[0014] The imaging unit 104 captures an image of real space under the control of the control unit 101. An imaging element such as a CCD or CMOS element that converts an optical image into an electrical signal can be used as the imaging unit 104. Alternatively, the imaging unit 104 may be a combination of an imaging element and a depth sensor configured to detect the physical distance of an object.
[0015] The attitude detection unit 105 detects the attitude of the head mounted display 100 with respect to the direction of gravity, and detects the inclination of the attitude with respect to each axis of yaw, roll, and pitch. Based on the attitude detected by the attitude detection unit 105, it is possible to determine whether the head mounted display 100 is facing up, facing down, or at an angle. As the attitude detection unit 105, at least one of an acceleration sensor, a gyro sensor, a geomagnetic sensor, a direction sensor, and an altitude sensor can be used, and two or more of them can also be used in combination.
[0016] The operation unit 106 is an input device for receiving user operations, and includes at least one of a pointing device such as a mouse or a touch panel, a button, a dial, a joystick, a touch sensor, and a touch pad. The operation unit 106 may be a device (for example, a controller or a keyboard) separate from the head mounted display 100. In that case, communication with the operation unit 106 is performed using the communication I / F 110.
[0017] The display unit 107 displays various images such as images and GUI screens constituting a GUI (Graphical User Interface) under the control of the control unit 101. The control unit 101 generates a display control signal according to a program, and controls each unit of the head mounted display 100 to generate a video signal for display on the display unit 107 and output the video signal to the display unit 107. The display unit 107 displays an image based on the output video signal.
[0018] The audio output unit 108 outputs various sounds such as the sound of video or music data, operation sounds, ringtones, and various notification sounds. The audio output unit 108 may be a speaker or an audio output terminal for connecting earphones or the like. The audio output unit 108 may be a communication unit that performs wireless communication with an external speaker or earphones.
[0019] The external I / F 109 is an interface for connecting to an external device via a wired cable or wirelessly and inputting / outputting at least one of a video signal and an audio signal between the external device and the external device. The communication I / F 110 is an interface for communicating with an external device or the Internet and transmitting / receiving various data such as files and commands.
[0020] The head mounted display 100 may be separated into a part to be worn on the head (head mounted display) and an external PC. In that case, for example, the display unit 107 is provided in the head mounted display, and the control unit 101 is provided in the external PC.
[0021] Fig. 2 is a flowchart of the process of the head mounted display 100. This process is realized by the control unit 101 expanding a program stored in the non-volatile memory 103 into the memory 102 and executing it. For example, when the head mounted display 100 is started, the process of Fig. 2 is started.
[0022] In S201, the control unit 101 displays an image of a virtual space on the display unit 107.
[0023] In S202, the control unit 101 determines whether or not a predetermined start operation for starting a character input state in which character input is possible has been performed. The control unit 101 waits until the start operation is performed, and proceeds to S203 when the start operation is performed. For example, when a button for starting a character input state (a button for displaying a text field, which will be described later) is displayed in the virtual space and the button is operated using the operation unit 106, the control unit 101 proceeds to S203. The start operation is not limited to this, and may be an operation on a button, dial, touch pad, or mouse of the operation unit 106.
[0024] In S203, the control unit 101 displays a text field for character input in the virtual space displayed on the display unit 107.
[0025] In S204, the control unit 101 starts accepting character input into the displayed text field, thereby entering a character input state.
[0026] In S205, the control unit 101 captures an image of the real space using the imaging unit 104, and detects a character input device from the captured real space (image of the real space). The character input device can be detected using various known techniques. The character input device is a device for inputting characters, such as a keyboard, a laptop, a smartphone, a tablet PC, or a touch pen. In the present embodiment, the detection of the character input device is started in response to the character input state being entered, but this is not limited to this. For example, the detection of the character input device may be performed regardless of whether the character input state is entered or not. The detection of the character input device may be started in response to the start of the head mounted display 100.
[0027] In S206, the control unit 101 controls the display of the display unit 107 (display control process). Details of the display control process will be described later with reference to Fig. 3. In the display control process, for example, an image (input device image) representing the detected character input device is displayed in the virtual space.
[0028] In S207, the control unit 101 determines whether or not a predetermined end operation for ending the character input state has been performed. If the end operation has been performed, the process proceeds to S208, and if not, the process proceeds to S205. For example, when a button for ending the character input state (a button for hiding the text field) is displayed in the virtual space and the button is operated using the operation unit 106, the process proceeds to S208. The end operation is not limited to this, and may be an operation on a button, dial, touch pad, or mouse of the operation unit 106, or an operation on an object other than a text field.
[0029] In S208, the control unit 101 hides the text field. The user may use the operation unit 106 to perform an operation to hide the input device image (a hide operation), and the control unit 101 may hide the input device image in response to the hide operation while maintaining the character input state. This enables usage such as temporarily hiding the input device image in the middle of character input, improving convenience.
[0030] FIG. 3 is a flowchart of the display control process performed in S206 of FIG.
[0031] In S301, the control unit 101 acquires the detection result of the character input device (the processing result of S205).
[0032] In S302, the control unit 101 determines whether or not a character input device has been detected based on the detection result of the character input device. If a character input device has been detected, the process proceeds to S303, and if not, the process proceeds to S304.
[0033] In S303, the control unit 101 calculates the position (coordinates) of the detected character input device in the real space (image of the real space) captured by the imaging unit 104. Then, the control unit 101 displays an image (input device image) representing the detected character input device at a position in the virtual space corresponding to the calculated position. Here, the input device image is an image obtained by cutting out an area corresponding to the position (coordinates) of the detected character input device from the image of the real space captured by the imaging unit 104. By doing so, the user can visually recognize the input device image (character input device in the real world) and preferably input characters, and the operability of character input is improved while obtaining a sense of immersion in the virtual space. The position of the character input device in the real space (real world) can be detected by using various known techniques. The control unit 101 may change the display position of the text field to a position near the displayed input device image. By doing so, the amount of movement of the line of sight between the input device image and the text field is shortened, so that the user can more preferably input characters.
[0034] In S304, the control unit 101 displays a virtual space image on the display unit 107 instead of the virtual space image. The image of the virtual space is hidden, and an image of the real space captured by the imaging unit 104 is displayed. This allows the user to easily find the character input device and input characters. The control unit 101 may display both the image of the virtual space and the image of the real space.
[0035] 4 is a schematic diagram showing a virtual space image 400 (image of a virtual space) displayed on the head mounted display 100. The control unit 101 displays the virtual space image 400 on the display unit 107 (S201 in FIG. 2). When the user performs a predetermined operation while viewing the virtual space image 400, the control unit 101 displays a text field 401 in the virtual space (S203). For example, when a button for displaying the text field 401 is displayed in the virtual space and the button is operated using the operation unit 106, the text field 401 is displayed. The predetermined operation for displaying the text field 401 is not limited to this, and may be an operation on a button, a dial, a touch pad, or a mouse of the operation unit 106.
[0036] After displaying the text field 401, the control unit 101 starts accepting character input to the text field 401 (S204). The acceptance of character input to the text field 401 may start automatically at the same time as the display of the text field 401, or may start in response to a user performing an operation on the text field 401 using the operation unit 106.
[0037] After starting to accept character input to the text field 401, the control unit 101 starts to detect the character input device. The control unit 101 captures an image of the real space using the imaging unit 104, and detects the character input device from the captured real space (image of the real space) (S205). When the control unit 101 detects the character input device, it calculates the position of the detected character input device in the real space captured using the imaging unit 104. Then, the control unit 101 displays an image (input device image) 402 representing the detected character input device at a position in the virtual space corresponding to the calculated position (S303 in FIG. 3). The input device image 402 is an image of the detected character input device. The control unit 101 extracts a rectangular area including the detected character input device from the image of the real space captured using the imaging unit 104, and displays the extracted rectangular area as the input device image 402. Note that the control unit 101 may detect the outline of the character input device, extract an area surrounded by the outline, and display it as the input device image. The control unit 101 may replace the input device image with a three-dimensional model (3DCG) corresponding to the detected character input device and display it.
[0038] As described above, according to this embodiment, a user experiencing a virtual reality image can visually recognize a real-world character input device (e.g., a keyboard) at a desired timing. For example, when the character input state is not in which character input into a text field is possible, an input device image representing the detected character input device is not displayed in the virtual space, and when the character input state is in, the input device image is displayed in the virtual space. By doing so, when character input is not performed, the input device image is not displayed, so that the user's immersion is not hindered by the input device image. On the other hand, when character input is performed, the input device image is displayed, so that the user can visually recognize the input device image (a real-world character input device) and perform character input suitably.
[0039] (Variation 1) Fig. 5 is a modified example of the flowchart of the display control process performed in S206 in Fig. 2. In the display control process in Fig. 5, the distance from the head mounted display 100 to the detected character input device is taken into consideration.
[0040] In S501, the control unit 101 obtains the detection result of the character input device (the processing result of S205). You will benefit.
[0041] In S502, the control unit 101 determines whether or not a character input device has been detected based on the detection result of the character input device. If a character input device has been detected, the process proceeds to S503, and if not, the process proceeds to S506.
[0042] In S503, the control unit 101 acquires information on the distance to the detected character input device (distance information) from the head mounted display 100. The distance information can be acquired using various known techniques, for example, the imaging unit 104.
[0043] In S504, the control unit 101 determines whether or not the distance from the head mounted display 100 to the character input device is shorter than a threshold (predetermined distance) according to the acquired distance information. If the distance is shorter than the threshold, the process proceeds to S505, and if not, the process proceeds to S506. Although the process proceeds to S506 if the distance is equal to the threshold, the process may proceed to S505.
[0044] In S505, the control unit 101 calculates the position (coordinates) of the detected character input device in real space (image of real space) captured by the imaging unit 104. Then, the control unit 101 displays an image (input device image) representing the detected character input device at a position in the virtual space corresponding to the calculated position. The position of the character input device in the real space (real world) can be detected using various known techniques.
[0045] In S506, control unit 101 displays an image of real space captured by imaging unit 104 in place of the image of virtual space displayed on display unit 107 (by hiding the image of virtual space). Control unit 101 may display an item indicating the position of the detected character input device by superimposing it on the image of real space. This allows the user to easily find the character input device from the image of real space and approach the character input device when inputting characters.
[0046] When the distance from the head mounted display 100 to the character input device is long, the user needs to approach the character input device when inputting characters. Therefore, it is preferable for the user to be able to grasp the surroundings. Therefore, in the above-mentioned modification example 1, when the distance from the head mounted display 100 to the character input device is longer than a threshold, the character input image is not displayed, and an image of the real space is displayed. In this way, the user can easily approach the character input device while avoiding surrounding obstacles.
[0047] (Variation 2) Fig. 6 is another modified example of the flowchart of the display control process performed in S206 in Fig. 2. In the display control process in Fig. 6, the detected connection state of the character input device is taken into consideration.
[0048] In S601, the control unit 101 acquires the detection result of the character input device (the processing result of S205).
[0049] In S602, the control unit 101 determines whether or not a character input device has been detected based on the detection result of the character input device. If a character input device has been detected, the process proceeds to S603, and if not, the process proceeds to S606.
[0050] In S603, the control unit 101 acquires information (connection information) on the connection state of the detected character input device. For example, if the detected character input device is not connected to the head mounted display 100, the connection information is not acquired, and if the detected character input device is connected to the head mounted display 100, the connection information is acquired using the communication I / F 110.
[0051] In S604, the control unit 101 determines, based on the acquisition result of the connection information, whether or not the detected character input device is connected (wired or wirelessly) to the head mounted display 100. If the detected character input device is connected to the head mounted display 100, the process proceeds to S605, and if not, the process proceeds to S606.
[0052] In S605, the control unit 101 calculates the position (coordinates) of the detected character input device in real space (image of real space) captured by the imaging unit 104. Then, the control unit 101 displays an image (input device image) representing the detected character input device at a position in the virtual space corresponding to the calculated position. The position of the character input device in real space (real world) can be detected using various known techniques.
[0053] In S606, the control unit 101 displays an image of the real space captured by the imaging unit 104 instead of the image of the virtual space displayed on the display unit 107 (by hiding the image of the virtual space). The control unit 101 may display guidance for prompting the user to connect the character input device to the head mounted display 100 by superimposing it on the image of the real space. This allows the user to easily understand that it is necessary to connect the character input device. The control unit 101 may display a list of character input devices (paired character input devices) that can be connected to the head mounted display 100 by superimposing it on the image of the real space. This allows the user to easily select a paired character input device and connect it to the head mounted display 100.
[0054] When a character input device is not connected to the head mounted display 100, the user needs to connect the character input device to the head mounted display 100 when inputting characters. Therefore, it is preferable for the user to be able to grasp the surroundings. Therefore, in the above-mentioned modified example 2, when the detected character input device is not connected to the head mounted display 100, the character input image is not displayed, and an image of the real space is displayed. In this way, the user can easily perform the task (operation) of connecting the character input device to the head mounted display 100 while visually checking it.
[0055] The above-described embodiment (including the modified examples) is merely an example, and the present invention also includes configurations obtained by appropriately modifying or changing the configurations of the above-described embodiment within the scope of the gist of the present invention. The present invention also includes configurations obtained by appropriately combining the configurations of the above-described embodiment.
[0056] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.
[0057] The disclosure of the present embodiment includes the following configuration, method, program, and medium. (Configuration 1) A control means capable of controlling so that an image of a virtual space is displayed on a head mounted display and controlling so that a text field for character input is displayed in the virtual space; A detection means for detecting a character input device from an image of real space captured by the imaging means; having The control means When the text field is displayed and the text input is not possible, does not control so as to display an image representing the detected character input device in the virtual space; When the character input state is reached, an image representing the detected character input device from among the image of the real space is displayed in the virtual space. A control device comprising: (Configuration 2) The control means controls so as to display an image representing the detected character input device at a position in the virtual space corresponding to a position of the character input device in the real space. 2. The control device according to configuration 1, (Configuration 3) The image representing the detected character input device is an image of a region of the image of the real space corresponding to the position of the detected character input device. 3. The control device according to configuration 1 or 2. (Configuration 4) The image representing the detected character input device is a three-dimensional model corresponding to the detected character input device. 4. The control device according to any one of configurations 1 to 3. (Configuration 5) The control means performs control so as to make the image representing the detected character input device invisible in response to a predetermined operation for terminating the character input state. 5. The control device according to any one of configurations 1 to 4. (Configuration 6) The control device according to configuration 5, wherein the predetermined operation is an operation on an object other than the text field. (Configuration 7) When an image representing the detected character input device is displayed, the control means changes the display position of the text field to a position near the image. 7. The control device according to any one of configurations 1 to 6. (Configuration 8) The control means performs control so as to hide the image representing the detected character input device while maintaining the character input state in response to an operation of hiding the image representing the detected character input device. 8. The control device according to any one of configurations 1 to 7. (Configuration 9) When the character input state is reached and a character input device is not detected, the control means controls the head mounted display to display an image of the real space. 9. The control device according to any one of configurations 1 to 8. (Configuration 10) When the character input state is reached and a character input device is not detected, the control means controls the head mounted display to hide the image of the virtual space and to display the image of the real space on the head mounted display. 10. The control device according to configuration 9. (Configuration 11) Even in the case of the character input state, when a distance from the head mounted display to the detected character input device is longer than a threshold value, the control means does not control so as to display an image representing the detected character input device in the virtual space, but controls so as to display an image of the real space on the head mounted display. 11. The control device according to any one of configurations 1 to 10. (Configuration 12) When the character input state is in effect and the distance is greater than the threshold value, the control means and controlling the display of an item indicating the position of the character input device so as to be superimposed on the image of the real space. 12. The control device according to claim 11, (Configuration 13) Even in the case where the character input state is established, when the detected character input device is not connected to the head mounted display, the control means does not control so as to display an image representing the detected character input device in the virtual space, but controls so as to display an image of the real space on the head mounted display. 13. The control device according to any one of configurations 1 to 12. (Configuration 14) When the character input state is established and the detected character input device is not connected to the head mounted display, the control means performs control to display a guidance for prompting connection of the character input device to the head mounted display in a superimposed manner on the image of the real space. 14. The control device according to claim 13, (Configuration 15) When the character input state is established and the detected character input device is not connected to the head mounted display, the control means performs control to display a list of character input devices connectable to the head mounted display in a superimposed manner on the image of the real space. 15. The control device according to configuration 13 or 14. (Configuration 16) The detection means starts detecting a character input device in response to the character input state being entered. 16. The control device according to any one of configurations 1 to 15. (Configuration 17) The imaging means is further provided. 17. The control device according to any one of configurations 1 to 16. (method) A step of controlling so that an image of the virtual space is displayed on a head mounted display; a step of controlling so as to display a text field for character input in the virtual space; a detection step of detecting a character input device from an image of a real space captured by an imaging means; when the text field is displayed and the device is not in a character input state in which character input is possible, controlling not to display an image representing the detected character input device in the virtual space, and when the device is in the character input state, controlling to display an image representing the detected character input device in the image of the real space in the virtual space; A control method comprising the steps of: (program) A program for causing a computer to function as each of the means of the control device according to any one of configurations 1 to 17. (medium) A computer-readable storage medium storing a program for causing a computer to function as each of the means of the control device according to any one of configurations 1 to 17. [Explanation of symbols]
[0058] 100: Head mounted display 101: Control unit
Claims
1. A control means capable of controlling the display of images from a virtual space on a head-mounted display, and controlling the display of a text field for text input in the virtual space, A detection means for detecting a text input device from an image of the real space captured by an imaging means. It has, The control means controls the virtual space to display an image representing the detected character input device from the real-world video when the text field is displayed and character input is possible, and controls the head-mounted display to display the real-world video when the character input state is in place and character input is possible. A control device characterized by the following features.
2. The control means controls the display of the image representing the detected character input device at a position in the virtual space corresponding to the position of the character input device in the real space. The control device according to feature 1.
3. The image representing the detected character input device is an image of the region in the real-world video that corresponds to the location of the detected character input device. The control device according to feature 1.
4. The image representing the detected character input device is a three-dimensional model corresponding to the detected character input device. The control device according to feature 1.
5. The control means controls the image representing the detected character input device to be hidden in response to a predetermined operation to terminate the character input state. The control device according to feature 1.
6. The aforementioned predetermined operation is an operation on an object other than the text field. The control device according to claim 5.
7. When the control means displays an image representing the detected character input device, it changes the display position of the text field to a position near the image. The control device according to feature 1.
8. The control means, in response to an operation to hide the image representing the detected character input device, controls the system to hide the image representing the detected character input device while maintaining the character input state. The control device according to feature 1.
9. If the character input state is not present, the control means controls the display of the image representing the detected character input device in the virtual space. The control device according to feature 1.
10. If the text input state is in place and no text input device is detected, the control means controls the system to hide the image of the virtual space and display the image of the real space on the head-mounted display. The control device according to feature 1.
11. Even when the character input state is in effect, if the distance from the head-mounted display to the detected character input device is greater than the threshold, the control means will not control the display of an image representing the detected character input device in the virtual space, but will instead control the display of an image from the real space on the head-mounted display. The control device according to feature 1.
12. If the character input state is in place and the distance is greater than the threshold, the control means controls the display of an item indicating the position of the character input device superimposed on the image of the real space. The control device according to feature 11.
13. Even if the character input state is in effect, if the detected character input device is not connected to the head-mounted display, the control means does not control the display of an image representing the detected character input device in the virtual space, but instead controls the display of an image from the real space on the head-mounted display. The control device according to feature 1.
14. If the character input state is in place and the detected character input device is not connected to the head-mounted display, the control means controls the display to superimpose guidance prompting the connection of the character input device to the head-mounted display onto the image of the real space. The control device according to claim 13.
15. If the character input state is active and the detected character input device is not connected to the head-mounted display, the control means controls the display to superimpose a list of character input devices that can be connected to the head-mounted display onto the image of the real space. The control device according to claim 13.
16. The detection means, in response to the character input state, opens the detection of the character input device. Start The control device according to feature 1.
17. The imaging means further comprises The control device according to feature 1.
18. A step to control the display of images in a virtual space on a head-mounted display, A step of controlling the display of a text field for character input in the virtual space, A detection step for detecting a text input device from an image of the real space captured by an imaging means, If the text field is displayed and text input is possible, the system controls the display of an image representing the detected text input device from the real-world video in the virtual space; if the text input state is present but the text input device is not detected, the system controls the display of the real-world video on the head-mounted display. A control method characterized by having the following features.
19. A program for causing a computer to function as one of the means of the control device described in any one of claims 1 to 17.
20. A computer-readable storage medium storing a program for causing a computer to function as one of the means of the control device described in any one of claims 1 to 17.