Head-mounted display system

Abstract

To provide a head-mounted display system in which one user can easily confirm an image displayed to the other user.SOLUTION: The head-mounted display system 100 includes a first U1 worn by a first user HMD10 and a second U2 worn by a second user HMD20. The first HMD10 includes a display unit 11, a communication unit 12 that acquires, from the second HMD20, a second image U2 to be displayed in the second user IM2 by the second HMD20, a position detection unit 13 that detects a positional relationship between the first HMD10 and the second HMD20, an orientation detection unit 15 that detects an orientation of the first HMD10, and a control unit 14.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a head-mounted display system. [Background technology]

[0002] The system disclosed in Patent Document 1 processes the operation of a head-mounted display (HMD) user to participate in a virtual reality (VR) scene. The system provides a first view of the VR scene to a first HMD of a first user and receives a request from a second user requesting to participate in the VR scene provided to the first HMD. The system further acquires real-world position and orientation data of a second HMD relative to the first HMD, and then provides a second view of the VR scene based on this data. The system also controls the first and second views according to changes in their respective positions and orientations while viewing the VR scene. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2019-522856 Summary of the Invention [Problem to be solved by the invention]

[0004] The inventors of the present invention have found the following problems. The system disclosed in Patent Document 1 aims to allow one user (host user) to participate in a VR scene with another user. In such a system, a first and a second user can participate in a common VR scene by performing multiple processes by the user, such as receiving a request and responding to the request. Therefore, one user cannot easily check the image displayed to the other user.

[0005] The present disclosure has been made in consideration of the above-mentioned problems, and provides a head-mounted display system that allows one user to easily check an image displayed to another user. [Means for solving the problem]

[0006] The head-mounted display system according to the present disclosure comprises: a first HMD (head mounted display) worn by a first user; a second HMD worn by a second user; The first HMD includes: a display unit that displays a first video to the first user; a communication unit that acquires, from the second HMD, a second image that the second HMD displays to the second user; a position detection unit that detects a positional relationship between the first HMD and the second HMD; an orientation detection unit that detects an orientation of the first HMD; a control unit, The display unit is configured to enable the first user to view the real space within the field of view, and when the control unit determines, based on the positional relationship and the orientation of the first HMD, that the second user is located within the field of view of the first user, the control unit displays the second image around the real image of the second user viewed by the first user on the display unit. [Effects of the Invention]

[0007] According to the present disclosure, it is possible to provide a head-mounted display system that allows one user to easily check an image displayed to another user. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of the configuration of a head-mounted display system according to a first embodiment. [Figure 2]1 is a block diagram showing a specific example of a head-mounted display system according to a first embodiment. FIG. [Figure 3] FIG. 10 is a block diagram showing another specific example of the head-mounted display system according to the first embodiment. [Figure 4] 5 is a flowchart showing an example of a control method for the head mounted display system according to the first embodiment. [Figure 5] 10 is a flowchart showing a modified example of the control method for the head mounted display system according to the first embodiment. [Figure 6] 3 is a schematic diagram showing an example of an image that can be viewed through the display units of the first and second HMDs. FIG. [Figure 7] 10 is a schematic diagram showing a specific example of an image that can be viewed through the display unit of the first HMD. FIG. [Figure 8] 10 is a schematic diagram showing a specific example of an image that can be viewed through the display unit of the first HMD. FIG. [Figure 9] FIG. 2 is a schematic diagram showing the relationship between the first and second HMDs and the first and second images. [Figure 10] 3A and 3B are schematic diagrams showing examples of the positions and orientations of first and second HMDs. [Figure 11] 10 is a schematic diagram showing another example of the positions and orientations of the first and second HMDs. FIG. [Figure 12] FIG. 2 is a schematic diagram showing a specific example of a first image. [Figure 13] 3A and 3B are schematic diagrams showing examples of images displayed by the first and second HMDs to the first and second users, respectively. [Figure 14] FIG. 2 is a schematic diagram showing a specific example of a first image. [Figure 15] FIG. 2 is a schematic diagram showing a specific example of a first image. [Figure 16] FIG. 2 is a schematic diagram showing a specific example of a first image. [Figure 17] FIG. 2 is a schematic diagram showing a specific example of a first image. [Figure 18] FIG. 2 is a schematic diagram showing a specific example of a first image. [Figure 19]FIG. 2 is a schematic diagram showing a specific example of a first image. [Figure 20] FIG. 2 is a schematic diagram showing a specific example of a first image. [Figure 21] FIG. 2 is a schematic diagram showing a specific example of a first image. [Figure 22] 3A and 3B are schematic diagrams showing examples of images displayed by the first and second HMDs to the first and second users, respectively. [Figure 23] FIG. 2 is a schematic diagram showing a specific example of a first image. [Figure 24] FIG. 2 is a schematic diagram showing a specific example of a first image. [Figure 25] FIG. 2 is a schematic diagram showing a specific example of a first image. [Figure 26] FIG. 2 is a schematic diagram showing a specific example of a first image. [Figure 27] FIG. 2 is a schematic diagram showing a specific example of a first image. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. However, the present invention is not limited to the following embodiments. In addition, the following description and drawings have been simplified as appropriate for clarity of explanation.

[0010] Embodiment 1 <Example of a head-mounted display system configuration> An example of the configuration of the head mounted display system according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of the head mounted display system according to the first embodiment.

[0011] As shown in FIG. 1, the head-mounted display system 100 includes a first HMD (head-mounted display) 10 and a second HMD 20. The first HMD 10 and the second HMD 20 may be wearable devices capable of displaying images. The shapes of the first HMD 10 and the second HMD 20 are not limited, and may be, for example, a head-mounted type, a goggle type, or a pair of glasses that can be worn by a user on their head. The head-mounted type may have the same configuration as, for example, a helmet or a hat. The first HMD 10 and the second HMD 20 may be, for example, smart glasses or AR (Augmented Reality) glasses. The first HMD 10 may be worn on the head of, for example, a first user U1 shown in FIG. 6. The second HMD 20 may be worn on the head of, for example, a second user U2 shown in FIG. 6. The first HMD 10 and the second HMD 20 are, for example, a fully immersive HMD (non-transparent HMD) or a transparent HMD, and are preferably transparent HMDs.

[0012] Examples of see-through HMDs include video see-through HMDs and optical see-through HMDs. Video see-through HMDs display an image of the real space in the user's field of view captured in real time using an imaging device, and then superimpose that image onto another image on the display unit. Optical see-through HMDs use an optical system to display an image of the real space in the user's field of view, which the user views through the display unit. The optical system is, for example, a prism or a half mirror.

[0013] The first HMD 10 includes a display unit 11, a communication unit 12, a position detection unit 13, a control unit 14, and an orientation detection unit 15.

[0014] The communication unit 12 acquires a second image from the second HMD 20 that the second HMD 20 displays to the second user U2. The position detection unit 13 detects the positional relationship between the first HMD 10 and the second HMD 20. The orientation detection unit 15 detects the orientation of the first HMD 10. The control unit 14 determines whether the second user U2 is located within the field of view of the first user U1 based on the positional relationship between the first HMD 10 and the second HMD 20 and the orientation of the first HMD 10.

[0015] The control unit 14 may display an image showing the real space within the field of view of the first user U1 on the display unit 11. In this case, when the control unit 14 determines that the second user U2 is located within the field of view of the first user U1, the control unit 14 causes the display unit 11 to display the second image around the real image of the second user U2 within the image showing the real space within the field of view of the first user U1.

[0016] Furthermore, the display unit 11 may use an optical system (not shown) to allow the first user U1 to view the real space within the field of view of the first user U1 through the display unit 11. In such a case, when the control unit 14 determines that the second user U2 is located within the field of view of the first user U1, the control unit 14 causes the display unit 11 to display the second video around the real image of the second user U2 that can be viewed through the display unit 11.

[0017] As a result, the first user U1 can easily check the second video displayed to the second user U2.

[0018] The first HMD 10 may further include a gaze detection unit that detects the gaze of the first user U1. The gaze detection unit includes, for example, a light source and an optical sensor. The light source supplies light other than visible light, such as infrared light or ultraviolet light, to the pupil of the first user U1. The optical sensor captures an image of the light reflected from the pupil of the first user U1. The gaze detection unit can detect the gaze of the first user U1 based on the captured light. If the gaze of the first user U1 is directed toward a predetermined target, it can be determined that the first user U1 is gazing at the predetermined target. Based on this detected gaze of the first user U1, a gaze level, which will be described later, can be calculated.

[0019] The first HMD 10 may further include a biometric information acquisition unit that acquires biometric information of the first user U1. The biometric information is, for example, information whose value changes depending on the psychological state. The gaze level, which will be described later, can be calculated using such information whose value changes. Examples of such information whose value changes include heart rate, pulse wave, and brain wave. The biometric information acquisition unit may include, for example, a heart rate sensor, a pulse wave sensor, and an electroencephalograph.

[0020] <Video transparent type> A specific example of the head mounted display system according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the specific example of the head mounted display system according to the first embodiment.

[0021] The head-mounted display system 100A shown in Fig. 2 is a specific example of the head-mounted display system 100 shown in Fig. 1. The head-mounted display system 100A includes a first HMD 10A and a second HMD 20A. The first HMD 10A and the second HMD 20A are video-transmitting HMDs.

[0022] The first HMD 10A includes a display unit 11A, a communication unit 12A, a position detection unit 13A, a control unit 14A, a direction detection unit 15A, and an imaging unit 16.

[0023] The second HMD 20A may have the same configuration as the first HMD 10A. Specifically, the second HMD 20A includes a display unit 21, a communication unit 22, a position detection unit 23, a control unit 24, an orientation detection unit 25, and an imaging unit 26. In the head-mounted display system 100A, the configuration of the first HMD 10A is the same as the configuration of the second HMD 20A. The configuration of the first HMD 10A will be described in detail below, but the display unit 21, the communication unit 22, the position detection unit 23, the control unit 24, the orientation detection unit 25, and the imaging unit 26 have the same configuration as the display unit 11A, the communication unit 12A, the position detection unit 13A, the control unit 14A, the orientation detection unit 15A, and the imaging unit 16, respectively.

[0024] When the first user U1 wears the first HMD 10A, the imaging unit 16 captures an image of the real space within the field of view of the first user U1 through the display unit 11A. The imaging unit 16 generates a real-space image that represents the captured real space. The imaging unit 16 includes an imaging element such as a CMOS (Complementary Metal-Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor.

[0025] The display unit 11A is a specific example of the display unit 11 shown in FIG. 1 and is, for example, a flat panel display such as a liquid crystal monitor or an organic EL (Electro-Luminescence) monitor. When the first HMD 10A is worn by, for example, a first user U1 shown in FIG. 6, the display unit 11A is disposed in front of the head of the first user U1. The display unit 11A displays a first image, and the first image includes a real space image generated by the imaging unit 16. The first image is generated by superimposing another image on the real space image generated by the imaging unit 16.

[0026] The communication unit 12A is a specific example of the communication unit 12 shown in FIG. 1 and includes an interface capable of communicating with other devices, including the second HMD 20A. This interface may be a known interface, such as a wireless LAN or Bluetooth (registered trademark). The communication unit 12A can receive content from other devices (not shown). The communication unit 12A can also acquire visual information of the second user U2 from the second HMD 20A by communicating with the communication unit 22 of the second HMD 20A. This visual information of the second user U2 is information about an image displayed on the second HMD 20A and viewed by the second user U2. In other words, the visual information is information about a second image displayed on the display unit 21 or second content included in the second image. By communicating with the communication unit 22 of the second HMD 20A, the communication unit 12A can preferably share visual information and, further, auditory information between the first HMD 10A and the second HMD 20A. The communication unit 12A acquires, from the second HMD 20A, a second video image that the second HMD 20A displays to the second user U2.

[0027] The position detection unit 13A is a specific example of the position detection unit 13 shown in FIG. 1 and detects the positional relationship between the first HMD 10A and the second HMD 20A. The position detection unit 13A may detect the positional relationship between the first HMD 10A and the second HMD 20A using a known position detection means. Examples of the position detection means include "outside-in tracking," in which the first HMD 10A acquires position information of the first HMD 10A and the second HMD 20A via the communication unit 12A, the position information being detected by a device other than the first HMD 10A or the second HMD 20A using a camera sensor or the like, and "inside-out tracking," in which the first HMD 10A acquires position information of the other HMD (the second HMD 20A) using a camera sensor (not shown) mounted on the first HMD 10A. The positional relationship between the first HMD 10A and the second HMD 20A includes the distance between the first HMD 10A and the second HMD 20A, and the like.

[0028] 1, and detects the orientation of the first HMD 10A. The orientation detection unit 15A may detect the orientation of the first HMD 10A using, for example, a gyro, an acceleration sensor, a magnetic sensor, or the like.

[0029] The control unit 14A is a specific example of the control unit 14 shown in FIG. 1 and can be configured by a processor such as a CPU (Central Processing Unit). In other words, the control unit 14A can have the functions of a computer. The control unit 14A can execute programs stored in a storage device such as an internal memory (not shown) and perform various processes. The control unit 14A can, for example, obtain input from a first user U1 or the like and display any video or content on the display unit 11A.

[0030] The control unit 14A of the first HMD 10A causes the display unit 11A to display a first image. The first image includes the real space image and the first content. The first image is generated by superimposing the first content on the real space image. The control unit 14A acquires the first content from another device (not shown) via the communication unit 12A and generates the first image. The first image may include multiple contents. In this case, the multiple displayed contents are referred to as a first content group. The first content may also be stored in a storage unit (not shown) within the first HMD 10A. Similarly, the control unit 24 of the second HMD 20A causes the display unit 21 to display a second image. The second image includes the second content. The second image, like the first image, may be generated by superimposing the second content on a real space image showing the real space within the field of view of the second user U2 through the display unit 21 when the second user U2 wears the second HMD 20A. The first content and the second content are diverse and include, for example, movies, music, photos, cartoons, animations, computer games, videos, websites, etc. The first video and the second video may also include icon images for operation. Furthermore, a background image may be displayed as a common background for the first content or the second content and the icon images, etc.

[0031] The control unit 14A determines whether the second user U2 is located within the field of view of the first user U1 based on the positional relationship between the first HMD 10A and the second HMD 20A detected by the position detection unit 13A and the orientation of the first HMD 10A detected by the orientation detection unit 15A. If the second user U2 is located within the field of view of the first user U1, the first user U1 can visually recognize the real image of the second user U2 through the display unit 11A. The control unit 14A estimates the position of the real image of the second user U2 on the display unit 11A based on the positional relationship and the orientation of the first HMD 10. If the control unit 14A determines that the second user U2 is located within the field of view of the first user U1, it displays the second video around the real image of the second user U2 in the first video on the display unit 11A. Furthermore, the control unit 14A may change at least one of the size, resolution, and transparency of the second image based on the distance between the first HMD 10A and the second HMD 20A detected by the position detection unit 13A. When the control unit 14A determines that the second user U2 is located within the field of view of the first user U1, the control unit 14A may cause the display unit 11A to display the first content at a position separated from the real image of the second user U2 in the first image. Furthermore, the control unit 14A may cause the display unit 11A to display only the second content included in the second image around the real image of the second user U2 in the first image.

[0032] The control unit 14A may also acquire a first gaze degree indicating the degree to which the first user U1 is gazing at the first content. The control unit 14A can calculate and acquire the first gaze degree based on, for example, the result of detecting the gaze of the first user U1 or biometric information of the first user U1. The first HMD 10 may include the gaze detection unit and biometric information acquisition unit described above. The gaze detection unit may detect the gaze of the first user U1. The biometric information acquisition unit may acquire the biometric information of the first user U1. The first gaze degree may be calculated based on, for example, the length of time the gaze of the first user U1 is directed toward the first content. The first gaze degree may be calculated based on, for example, whether the biometric information of the first user U1 is a value indicating a psychological state corresponding to the first user U1 gazing at the first content.

[0033] Similarly, the control unit 14A may acquire a second gaze degree indicating the degree to which the second user U2 is gazing at the second content. The control unit 14A may acquire the second gaze degree from the second HMD 20A via the communication unit 12A, for example. The control unit 24 of the second HMD 20A may calculate and acquire the second gaze degree based on the result of detecting the line of sight of the second user U2 or biometric information of the second user U2. The second HMD 20A may include the above-mentioned gaze detection unit and biometric information acquisition unit. The gaze detection unit may detect the line of sight of the second user U2. The biometric information acquisition unit may acquire the biometric information of the second user U2. The second gaze degree may be calculated based on, for example, the length of time the line of sight of the second user U2 is directed at the second content. The second gaze degree can be calculated, for example, depending on whether or not the biological information of the second user U2 has a value indicating a psychological state corresponding to the second user U2 gazing at the second content.

[0034] The control unit 14A may change at least one of the size, resolution, and transparency of the first and second contents based on the first or second gaze degree. The control unit 14A may change at least one of the size, resolution, and transparency of the first and second contents, for example, by comparing the first or second gaze degree with a predetermined value. For example, when the first gaze degree is higher than the predetermined value, it indicates that the first user U1 is gazing at the first content. In such a case, the control unit 14A may display the second video by lowering at least one of the resolution and transparency of the second video so as not to interfere with viewing. Furthermore, for example, when the second gaze degree is higher than the predetermined value, it indicates that the second user U2 is concentrating on viewing the second content. In such a case, the control unit 14A may increase at least one of the resolution and transparency of the second image and display the second image so that the first user U1 can understand the content that the second user U2 is gazing at.

[0035] Furthermore, when the control unit 14A determines that the real image of the second user U2 is located within the central region of the first video, the control unit 14A may change at least one of the size, resolution, and transparency of the first and second content. The central region of the first video may be a predetermined region that encompasses the center of the first video. When the first video is displayed on the entire surface of the display unit 11A, the central region of the first video becomes the central region of the display unit 11A. Furthermore, since the first video is generated by superimposing another video on the aforementioned real space image, the central region of the first video becomes a predetermined region at the center of the aforementioned real space image. Furthermore, when the control unit 14A determines that the real image of the second user U2 is located outside the central region of the first video, the control unit 14A may change at least one of the size, resolution, and transparency of the first and second content as the real image of the second user U2 in the first video approaches the central region of the first video. For example, if the control unit 14A determines that the real image of the second user U2 is located within the central region of the first image, the first user U1 is gazing at the second user U2. In such a case, the control unit 14A may increase the resolution of the second image, decrease the transparency, or both, in order to grasp the situation of the second user U2 with higher accuracy, before displaying the second image. Furthermore, for example, if the control unit 14A determines that the real image of the second user U2 is located outside the central region of the first image, the first user U1 is not gazing at the second user U2. In such a case, the control unit 14A does not need to grasp the situation of the second user U2 with high accuracy, and may decrease the resolution of the second image, increase the transparency, or both, before displaying the second image. In this embodiment, the control unit 14A determines whether the second user U2 is located within the field of view of the first user U1 based on the positional relationship between the first HMD 10A and the second HMD 20A detected by the position detection unit 13A and the orientation of the first HMD 10A detected by the orientation detection unit 15A. However, the control unit 14A may also determine whether the second user U2 is located within the field of view of the first user U1 based on the real space image generated by the imaging unit 16.

[0036] <Optical transmission type> Another specific example of the head mounted display system according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing another specific example of the head mounted display system according to the first embodiment.

[0037] The head-mounted display system 100B shown in Fig. 3 is a specific example of the head-mounted display system 100 shown in Fig. 1. The head-mounted display system 100B includes a first HMD 10B and a second HMD 20B. The first HMD 10B and the second HMD 20B are optically transmissive HMDs.

[0038] The first HMD 10B includes a display unit 11B, a communication unit 12A, a position detection unit 13A, a control unit 14B, and an orientation detection unit 15A.

[0039] The second HMD 20B may have the same configuration as the first HMD 10B. Specifically, the second HMD 20B includes a display unit 21B, a communication unit 22, a position detection unit 23, a control unit 24B, and an orientation detection unit 25. In the head-mounted display system 100B, the configuration of the first HMD 10B is the same as the configuration of the second HMD 20B. The configuration of the first HMD 10B will be described in detail below, but the display unit 21B and the control unit 24B have the same configurations as the display unit 11B and the control unit 14B, respectively.

[0040] The display unit 11B is a specific example of the display unit 11 shown in FIG. 1. The display unit 11B guides at least a portion of visible light from a real space within the field of view of the first user U1 to the eyes of the first user U1. This allows the first user U1 to view the real space within the field of view of the first user U1 through the display unit 11B. The display unit 11B uses an optical system to superimpose a third image, which will be described later, on the real space within the field of view of the first user U1, as viewed through the display unit 11B. The optical system is a prism, a half mirror, or the like.

[0041] Control unit 14B is a specific example of control unit 14 shown in Fig. 1. Control unit 14B has the same configuration as control unit 14A, except that control unit 14B displays a third image in place of the first image.

[0042] The control unit 14B causes the display unit 11B to display the third image. The control unit 14B may acquire the third image from another device (not shown) via the communication unit 12A. The third image, like the first image, includes the first content. Unlike the first image, the third image does not include the real space image generated by the imaging unit 16. When the control unit 14B determines that the second user U2 is located within the field of view of the first user U1, the control unit 14B causes the display unit 11B to display the second image around the real image of the second user U2 visible through the display unit 11B. The control unit 14B may also change at least one of the size, resolution, and transparency of the second image based on the distance between the first HMD 10A and the second HMD 20A detected by the position detection unit 13A. When the control unit 14B determines that the second user U2 is located within the field of view of the first user U1, the control unit 14B may cause the display unit 11B to display the first content at a position away from the real image of the second user U2 visible through the display unit 11B. Furthermore, the control unit 14B may cause the display unit 11B to display only the second content included in the second video around the real image of the second user U2 visible through the display unit 11B.

[0043] Furthermore, when the control unit 14B determines that the real image of the second user U2 visible through the display unit 11B is located within the central region of the display unit 11B, it may change at least one of the size, resolution, and transparency of the first and second contents. The central region of the display unit 11B may be a predetermined region that encompasses the center of the display unit 11B. Because the first user U1 views the real space within the field of view through the display unit 11B, the central region of the display unit 11B is the same as the predetermined region at the center of the real space image. Furthermore, when the control unit 14B determines that the real image of the second user U2 is located outside the central region of the real space image, it may change at least one of the size, resolution, and transparency of the first and second contents as the real image of the second user U2 visible through the display unit 11B approaches the central region of the real space image.

[0044] <An example of a method for controlling a head-mounted display system> Next, an example of a control method for the video see-through head mounted display system 100A shown in Fig. 2 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing an example of a control method for the head mounted display system according to the first embodiment.

[0045] The control unit 14A causes the display unit 11A of the first HMD 10A to display the first image (step ST11). The display unit 11A displays the first image, and the first user U1 can view the first image. The position detection unit 13A also starts detecting the positional relationship between the first HMD 10A and the second HMD 20A. The position detection unit 13A may continue to detect the positional relationship between the first HMD 10A and the second HMD 20A from step ST11 to step ST14, which will be described later. The orientation detection unit 15A also starts detecting the orientation of the first HMD 10A. The orientation detection unit 15A may continue to detect the orientation of the first HMD 10A from step ST11 to step ST14, which will be described later.

[0046] Next, the control unit 14A determines whether the second user U2 is located within the field of view of the first user U1 (step ST12). Specifically, the control unit 14A determines whether the second user U2 is located within the field of view of the first user U1 based on the positional relationship between the first HMD 10A and the second HMD 20A detected by the position detection unit 13A and the orientation of the first HMD 10A detected by the orientation detection unit 15A. More specifically, the control unit 14A estimates whether the second HMD 20A is located in the direction in which the first HMD 10A faces from the first HMD 10A based on the positional relationship between the first HMD 10A and the second HMD 20A and the orientation of the first HMD 10A. When the control unit 14A estimates that the second HMD 20A is located in the direction in which the first HMD 10A faces from the first HMD 10A, the control unit 14A determines that the second user U2 is located within the field of view of the first user U1. When the control unit 14A determines that the second user U2 is not located within the field of view of the first user U1 (step ST12: NO), the control unit 14A repeats step ST12.

[0047] When the control unit 14A determines that the second user U2 is located within the field of view of the first user U1 (step ST12: YES), the control unit 14A further causes the display unit 11A of the first HMD 10A to display a second image (step ST13). Specifically, the communication unit 12A acquires the second image as visual information from the second HMD 20A, and the control unit 14A displays the acquired second image around the real image of the second user U2 in the first image on the display unit 11A.

[0048] Next, the control unit 14A determines whether the second user U2 is located outside the field of view of the first user U1 (step ST14). Specifically, when the control unit 14A estimates that the second HMD 20A is located in the direction in which the first HMD 10A faces from the first HMD 10A based on the positional relationship between the first HMD 10A and the second HMD 20A and the orientation of the first HMD 10A, it determines that the second user U2 is not located outside the field of view of the first user U1 (step ST14: NO), and repeats steps ST13 and ST14.

[0049] When the control unit 14A estimates that the second HMD 20A is not located in the direction of the first HMD 10A from the first HMD 10A and determines that the second user U2 is located outside the field of view of the first user U1 (step ST14: YES), the control unit 14A stops displaying the second image on the display unit 11A of the first HMD 10A (step ST15). In other words, the control unit 14A displays only the first image on the display unit 11A of the first HMD 10A. The communication unit 12A may stop acquiring the second image as visual information from the second HMD 20A.

[0050] As described above, according to the above-described control method, the first user U1 can check the second image displayed around the real image of the second user U2 in the first image. This allows the first user U1 to easily check the second image.

[0051] Furthermore, in the above steps ST12 and ST14, the control unit 14 may make the above-mentioned determinations based on the distance between the first HMD 10 and the second HMD 20 in addition to the position of the second user relative to the field of view of the first user. A control method for making such determinations will be described with reference to Fig. 5. Fig. 5 is a flowchart showing a modified example of the control method shown in Fig. 4. In the modified example of the control method shown in Fig. 5, steps ST11, ST22, ST13, ST24, and ST15 are executed in this order.

[0052] In step ST22, the control unit 14A determines whether the second user U2 is located within the field of view of the first user U1 and whether the distance between the first HMD 10A and the second HMD 20A is equal to or less than a predetermined distance. The positional relationship between the first HMD 10A and the second HMD 20A detected by the position detection unit 13A includes the distance between the first HMD 10A and the second HMD 20A. The control unit 14A can calculate the distance between the first HMD 10A and the second HMD 20A from the positional relationship between the first HMD 10A and the second HMD 20A detected by the position detection unit 13A. The predetermined distance may be set in advance. For example, the predetermined distance may be set in advance depending on whether the usage environment of the first HMD 10A and the second HMD 20A is indoors or outdoors. The control unit 14A determines whether the second user U2 is located within the field of view of the first user U1 and whether the distance between the first HMD 10A and the second HMD 20A is equal to or less than a predetermined distance. If the control unit 14A determines that the second user U2 is located within the field of view of the first user U1 and that the distance between the first HMD 10A and the second HMD 20A is equal to or less than the predetermined distance (step ST22: YES), the control unit 14A proceeds to step ST13. On the other hand, if the control unit 14A determines that the second user U2 is not located within the field of view of the first user U1 or that the distance between the first HMD 10A and the second HMD 20A is not equal to or less than the predetermined distance (step ST22: NO), the control unit 14A repeats step ST22.

[0053] In step ST24, the control unit 14A determines whether the second user U2 is located outside the field of view of the first user U1 or whether the distance between the first HMD 10A and the second HMD 20A exceeds a predetermined distance. If the control unit 14A determines that the second user U2 is located outside the field of view of the first user U1 or that the distance between the first HMD 10A and the second HMD 20A exceeds a predetermined distance (step ST24: YES), the control unit 14A proceeds to step ST15. On the other hand, if the control unit 14A determines that the second user U2 is not located outside the field of view of the first user U1 or that the distance between the first HMD 10A and the second HMD 20A does not exceed a predetermined distance (step ST24: NO), steps ST13 and ST24 are repeated.

[0054] 4, the control unit 14A of the first HMD 10A and the like executes steps ST11 to ST15 described above, but the control unit 24 of the second HMD 20A may also execute steps ST11 to ST15 in parallel with steps ST11 to ST15 described above. That is, the control unit 24 of the second HMD 20A may cause the display unit 21 to display the first video image and the like, or may stop the display, in parallel with steps ST11 to ST15 described above. Similarly, in the control method shown in FIG. 5, the control unit 24 of the second HMD 20A may also execute steps ST11, ST22, ST13, ST24, and ST15 described above.

[0055] 4 and 5 can be applied to the optically transmissive head-mounted display system 100B shown in Fig. 3. In the control method applied to the head-mounted display system 100B shown in Fig. 3, the first user U1 views the real space within the field of view through the display unit 11B, and in each step, the control unit 14B appropriately displays a third image on the display unit 11B. Also, in step ST13, the control unit 14B displays a second image around the real image of the second user viewable through the display unit 11B.

[0056] (Example of processing) <Processing using optically transparent HMD> Next, a specific example of steps ST11 to ST13 of the processing by the head mounted display system 100B shown in Fig. 3 will be described with reference to Figs. 6 to 8. As described above, the head mounted display system 100B includes a first HMD 10B and a second HMD 20B that are optically transmissive HMDs. Note that in the following specific example, the first HMD 10B is an optically transmissive HMD, and the first user U1 can view the real space in front of them through the display unit 11B of the first HMD 10B.

[0057] In step ST11, the first user U1 views the real space image R1 through the display unit 11B. The control unit 14B shown in FIG. 3 causes the display unit 11B to display the third image IM31 shown in FIG. 6 superimposed on the real space image R1. The first user U1 views the real space image R1 through the display unit 11B and the third image IM31 superimposed on the display surface 11c of the display unit 11B. The third image IM31 includes a first content C1. The third image IM31 may include multiple copies of the first content C1. When the third image IM31 includes only one copy of the first content C1, the first content C1 may be displayed over the entire third image IM31 or may be positioned near the center of the third image IM31. A first user U1 is viewing a third video IM31, and a second user U2 is viewing a second video M21.

[0058] In step ST12, the control unit 14B determines that the second user U2 is located within the field of view of the first user U1 (ST12: YES), and proceeds to step ST13.

[0059] Next, in step ST13, as shown in FIG. 8, the control unit 14B causes the display unit 11B to display the second image IM21 in the vicinity of the real image of the second user U2 visible through the display unit 11B. The control unit 14B may also cause the third image IM31 to be displayed in the display unit 11B at a position spaced apart from the real image of the second user U2 and the second image IM21. Note that the control unit 14B may also display the second content C2, instead of the second image IM21, in the vicinity of the real image of the second user U2 visible through the display unit 11B. This allows the first user U1 to easily check the first content C1 and the second content C2.

[0060] In this example, the third image IM31 is spaced apart when the second image IM21 is displayed in step ST13, but the second image IM21 may not be displayed and the third image IM31 may be spaced apart from the second user U2 in the real space image R1 as shown in Fig. 7. This allows the first user U1 to easily check the real image of the second user U2 and the first content C1 on the display surface 11c.

[0061] <Processing using a video-transparent HMD> Next, specific examples of steps ST11 to ST13 of the processing by the head mounted display system 100A shown in Fig. 2 will be described. As described above, the head mounted display system 100A includes a first HMD 10A and a second HMD 20A that are video see-through HMDs. Note that in the following specific examples, the first HMD 10A is a video see-through HMD, and the display unit 11A of the first HMD 10A displays a real space image R1 that indicates at least the real space in front of the first user U1.

[0062] <First specific example> A first specific example of steps ST11 to ST13 will be described with reference to FIGS.

[0063] In step ST11, the control unit 14A shown in FIG. 2 causes the display unit 11A of the first HMD 10A to display the first image IM1 shown in FIG. 9. Similarly, the control unit 24 shown in FIG. 2 causes the display unit 21 of the second HMD 20A to display the second image IM2 shown in FIG. 9. The first image IM1 includes a real space image R1 generated by the imaging unit 16 and representing a field of view of the first user U1 through the display unit 11A when the first user U1 wears the first HMD 10A. The first image IM1 may further include first content. The first image IM1 is displayed with the first content C1 superimposed on the real space image R1. The first image IM1 may be displayed with a background image (not shown) superimposed on the real space image R1, with the first content C1 displayed within the background image. A first user U1 can view a first image IM1, and a second user U2 can view a second image IM2.

[0064] 10, the first user U1 may face a direction in which the second user U2 cannot be seen. In such a case, in step ST12, the control unit 14A determines that the second user U2 is not located within the field of view of the first user U1 based on the positional relationship between the first HMD 10A and the second HMD 20A and the orientation of the first HMD 10A (ST12: NO). Therefore, the control unit 14A continues to display the first image IM1 on the display unit 11A.

[0065] As shown in FIG. 11, the first user U1 may turn in a direction that allows the second user U2 to be visually recognized. In such a case, in step ST12, the control unit 14A determines that the second user U2 is located within the field of view of the first user U1 based on the positional relationship between the first HMD 10A and the second HMD 20A and the orientation of the first HMD 10A (ST12: YES). When the second user U2 is located within the field of view of the first user U1, the first user U1 can visually recognize the second user U2 within the real space image R1 in the first image IM1a. In step ST13, the control unit 14A displays the second image IM2, which the communication unit 12A has acquired as visual information from the second HMD 20A, around the real image of the second user U2 in the first image IM1a. As shown in Fig. 12, the first image IM1a is displayed with the second image IM2 superimposed around the real image of the second user U2 in the first image IM1 shown in Fig. 9. The first user U1 can check the second image IM2 displayed around the real image of the second user U2 in the first image IM1a. This allows the first user U1 to easily check the second image IM2.

[0066] <Second specific example> Next, a second specific example of steps ST11 to ST13 will be described with reference to FIGS.

[0067] In step ST11, the control unit 14A shown in FIG. 2 causes the display unit 11A of the first HMD 10A to display the first image IM1b shown in FIG. 13. Similarly, the control unit 24 shown in FIG. 2 causes the display unit 21 of the second HMD 20A to display the second image IM2b shown in FIG. 13. The first image IM1b shows the real space image R1 and the first content C1 generated by the imaging unit 16. The second image IM2b shows the real space image R2 and the second content C2 generated by the imaging unit 26. The first user U1 is viewing the first image IM1b, and the second user U2 is viewing the second image IM2b.

[0068] In step ST12, the control unit 14A determines that the second user U2 is located within the field of view of the first user U1 (ST12: YES), and proceeds to step ST13.

[0069] In this embodiment, in step ST13, the second image IM21 is not displayed, and the first content C1 is displayed in the first image IM1c at a position separated from the real image of the second user U2. In step ST13, the control unit 14A displays the first image IM1c shown in FIG. 14 on the display unit 11A. When the second user U2 is located within the field of view of the first user U1, the first image IM1c shows the real space image R1 and the first content C1 in front of the first user U1, and the second user U2 is displayed in the real space image R1. In other words, the first user U1 can view the second user U2 present in the real space displayed on the display unit 11A. In the first image IM1c, the first content C1 is displayed at a position separated from the real image of the second user U2. This allows the first user U1 to easily check the real image of the second user U2 and the first content C1 in the first image IM1c.

[0070] In step ST13 of this example, the control unit 14A causes the display unit 11A to display the first content C1 at a position distant from the real image of the second user U2 in the first image IM1c, but the control unit 14A may also stop the display of the first content C1 by the display unit 11A, which allows the first user U1 to easily confirm the real image of the second user U2 in the first image IM1c.

[0071] <Third specific example> Next, a third specific example of steps ST11 to ST13 will be described with reference to FIGS.

[0072] Steps ST11 and ST12 are executed in the same manner as in the second specific example described above. Subsequently, in step ST13, the control unit 14A causes the display unit 11A to display the first image IM1d shown in FIG. 15. When the second user U2 is located within the field of view of the first user U1, the first image IM1d shows a real space image R1, a first content C1, and a second content C2 in front of the first user U1. The second user U2 is displayed in the real space image R1. That is, the first user U1 can visually recognize the second user U2 existing in the real space displayed on the display unit 11A. The control unit 14A causes the display unit 11A to display the first content C1 at a position separated from the real image of the second user U2 in the first image IM1d, while displaying the second content C2 in the periphery of the real image of the second user U2 in the first image IM1d. This allows the first user U1 to easily check the first content C1 that he or she is viewing and the second content C2 that the second user U2 is viewing in the first video IM1d.

[0073] <Fourth Specific Example> Next, a fourth specific example of steps ST11 to ST13 will be described with reference to FIGS.

[0074] Steps ST11 and ST12 are executed in the same manner as in the third specific example described above. Subsequently, in step ST13, the control unit 14A displays the second content C2 in the periphery of the real image of the second user U2 in the first image IM1, and displays the first content C1 at a position apart from the real image of the second user U2 in the first image IM1, as in the third specific example. Also, in this step ST13, the control unit 14A changes the size of the first content C1 based on the first gaze degree of the first user U1.

[0075] For example, when a first gaze level indicating the degree to which the first user U1 gazes at the first content C1 is higher than a predetermined value, the control unit 14A increases the size of the first content C1 shown in FIG. 15. The control unit 14A causes the display unit 11A to display a first image IM1e shown in FIG. 16. When the second user U2 is located within the field of view of the first user U1, the first image IM1e shows a real space image R1 in front of the first user U1, the first content C1e, and the second content C2. The second user U2 is displayed in the real space image R1. The first user U1 can visually recognize the second user U2 present in the real space displayed on the display unit 11A. The first content C1e is the same as the first content C1 shown in FIG. 15 except for its size. The size of the first content C1e is larger than the size of the first content C1. This allows the first user U1 to visually recognize the first content C1e and easily confirm the first content C1 that he or she is viewing.

[0076] For example, when the first gaze level, which indicates the degree to which the first user U1 is gazing at the first content C1, is lower than a predetermined value, the control unit 14A reduces the size of the first content C1 shown in FIG. 15. The control unit 14A displays the first content C1f shown in FIG. 17. The first content C1f is the same as the first content C1 shown in FIG. 15 except for the size. The size of the first content C1f is smaller than the size of the first content C1. This allows the first user U1 to easily check parts of the first image IM1f other than the first content C1f, such as the second user U2 and the second content C2 that the second user U2 is viewing. In this embodiment, the size of the first content C1 is enlarged or reduced when it is determined that the second user U2 is located within the field of view of the first user U1. However, if the first gaze level of the first user U1 is higher or lower than a predetermined value, the size of the first content C1 displayed in the first image IM1 in step ST11 may be enlarged or reduced before the first image IM1 is displayed, regardless of whether the second user U2 is located within the field of view of the first user U1.

[0077] <Fifth Specific Example> Next, a fifth specific example of steps ST11 to ST13 will be described with reference to FIGS.

[0078] Steps ST11 and ST12 are executed in the same manner as in the third specific example described above. Subsequently, in step ST13, the control unit 14A displays the second content C2 in the periphery of the real image of the second user U2 in the first image IM1, and displays the first content C1 at a position apart from the real image of the second user U2 in the first image IM1, as in the third specific example. Also, in this step ST13, the control unit 14A changes the resolution of the first content C1 based on the first gaze degree of the first user U1.

[0079] For example, the control unit 14A may increase the resolution of the first content C1 shown in FIG. 15 when a first gaze level indicating the degree to which the first user U1 gazes at the first content C1 is higher than a predetermined value. The control unit 14A causes the display unit 11A to display a first image IM1g shown in FIG. 18. When the second user U2 is located within the field of view of the first user U1, the first image IM1g shows a real space image R1, the first content C1g, and the second content C2 in front of the first user U1. The second user U2 is displayed in the real space image R1. That is, the first user U1 can view the second user U2 present in the real space displayed on the display unit 11A. The first content C1g is the same as the first content C1 shown in FIG. 15 except for the resolution. The resolution of the first content C1g is higher than the resolution of the first content C1. This allows the first user U1 to visually recognize the first content C1g and easily confirm the first content C1 that he or she is viewing.

[0080] For example, the control unit 14A may reduce the resolution of the first content C1 shown in FIG. 15 when the first gaze level, which indicates the degree to which the first user U1 gazes at the first content C1, is lower than a predetermined value. The control unit 14A causes the display unit 11A to display the first image IM1h shown in FIG. 19. When the second user U2 is located within the field of view of the first user U1, the first image IM1h shows the real space image R1, the first content C1h, and the second content C2 in front of the first user U1. The second user U2 is displayed in the real space image R1. That is, the first user U1 can view the second user U2 present in the real space displayed on the display unit 11A. The first content C1h is the same as the first content C1 shown in FIG. 15 except for the resolution. The resolution of the first content C1h is lower than the resolution of the first content C1. This allows the first user U1 to lower the resolution of the first content C1, which he or she does not pay much attention to, making it easier for the first user U1 to see the surrounding areas of the first content C1h in the first image IM1h, such as the second user U2 and the second content C2 that the second user U2 is viewing.

[0081] In step ST13 of this example, the control unit 14A may change the transparency of the first content C1 based on the first gaze degree of the first user U1. Specifically, in step ST13, the control unit 14A may decrease the transparency of the first content C1 shown in Fig. 15 when the first gaze degree of the first user U1 is higher than a predetermined value. Furthermore, the control unit 14A may increase the transparency of the first content C1 shown in Fig. 15 when the first gaze degree of the first user U1 is lower than a predetermined value. In this embodiment, when it is determined that the second user U2 is located within the field of view of the first user U1, the resolution or transparency of the first content C1 is changed. However, when the first gaze level of the first user U1 is higher or lower than a predetermined value, the resolution or transparency of the first content C1 displayed in the first image IM1 in step ST11 may be changed to display the first image IM1, regardless of whether the second user U2 is located within the field of view of the first user U1.

[0082] <Sixth Specific Example> Next, a sixth specific example of steps ST11 to ST13 will be described with reference to FIGS.

[0083] Steps ST11 and ST12 are executed in the same manner as in the third specific example described above. Subsequently, in step ST13, the control unit 14A displays the second content C2 in the periphery of the real image of the second user U2 in the first image IM1, and displays the first content C1 at a position apart from the real image of the second user U2 in the first image IM1, as in the third specific example. Also, in this step ST13, the control unit 14A changes at least one of the size, resolution, and transparency of the second content C2 based on a second gaze degree indicating the degree to which the second user U2 is gazing at the second content C2.

[0084] For example, when the second gaze level, which indicates the degree to which the second user U2 gazes at the second content C2, is higher than a predetermined value, the control unit 14A reduces the transparency of the second content C2 shown in FIG. 15. The control unit 14A causes the display unit 11A to display the first image IM1i shown in FIG. 20. When the second user U2 is located within the field of view of the first user U1, the first image IM1i shows the real space image R1, the first content C1, and the second content C2i in front of the first user U1. The second user U2 is displayed in the real space image R1. That is, the first user U1 can view the second user U2 present in the real space displayed on the display unit 11A. The second content C2i is the same as the second content C2 shown in FIG. 15 except for the transparency. The transparency of the second content C2i is lower than the transparency of the second content C2. This allows the first user U1 to visually recognize the second content C2i and easily check the second content C2 that the second user U2 is viewing.

[0085] For example, when the second gaze level, which indicates the degree to which the second user U2 gazes at the second content C2, is lower than a predetermined value, the control unit 14A increases the transparency of the second content C2 shown in FIG. 15. The control unit 14A causes the display unit 11 to display the first image IM1j shown in FIG. 21. When the second user U2 is located within the field of view of the first user U1, the first image IM1j shows the real space image R1, the first content C1, and the second content C2j in front of the first user U1. The second user U2 is displayed in the real space image R1. That is, the first user U1 can visually recognize the second user U2 present in the real space displayed on the display unit 11A. The second content C2j is the same as the second content C2 shown in FIG. 15 except for the transparency. The transparency of the second content C2j is higher than the transparency of the second content C2. This allows the first user U1 to view the second content C2j, easily confirm the second content C2 that the second user U2 is viewing, and understand that the second user U2 is not paying attention to the second content C2j.

[0086] <Seventh Specific Example> Next, a seventh specific example of steps ST11 to ST13 will be described with reference to FIGS. 22 to 24 and 27.

[0087] In step ST11, the control unit 14A shown in FIG. 2 causes the display unit 11A of the first HMD 10A to display a first image IM1k shown in FIG. 22. Similarly, the control unit 24A shown in FIG. 2 causes the display unit 21 of the second HMD 20A to display a second image IM2b shown in FIG. 22. The first image IM1k shows a real space image R1 in front of the first user U1, a first content C11, and a third content C13. The second image IM2b shows a real space image R2 in front of the second user U2, and a second content C2. The first user U1 can view the first image IM1k, and the second user U2 can view the second image IM2b.

[0088] In step ST12, the control unit 14A determines that the second user U2 is located within the field of view of the first user U1 (ST12: YES), and proceeds to step ST13.

[0089] In step ST13, the control unit 14A changes at least one of the size, resolution, and transparency of the first content C11 and the third content C13 based on the gaze degree of the first user U1. The gaze degree of the first user U1 described above includes two gaze degrees that indicate the degree to which the first user U1 gazes at the first content C11 and the third content C13, respectively.

[0090] For example, when the first user U1's degree of attention to the first content C11 shown in FIG. 22 is higher than the degree of attention to the third content C13, the control unit 14A increases the size of the first content C1 relative to the third content C13. The control unit 14A causes the display unit 11 to display the first image IM1m shown in FIG. 23. When the second user U2 is located within the field of view of the first user U1, the first image IM1m shows the real space image R1, the first content C11m, and the third content C13m in front of the first user U1. The second user U2 is displayed in the real space image R1. That is, the first user U1 can view the second user U2 present in the real space displayed on the display unit 11A. The first content C11m is the same as the first content C11 shown in FIG. 22, except for the size. The third content C13m is the same as the third content C13 shown in Fig. 22 except for the size. The size of the first content C11m relative to the third content C13m is larger than the size of the first content C11 relative to the third content C13 shown in Fig. 22. This makes it possible to display the first content C11m, which is larger in size relative to the third content C13, for the first content C11 that has attracted a high level of attention. The first user U1 may easily view the first content C11m and easily confirm the first content C1 that he or she is viewing.

[0091] For example, when the first user U1's gaze level on the first content C11 shown in FIG. 22 is higher than that on the third content C13, the control unit 14A may display the first content C11 so that the third content C13 is transparent. The control unit 14A displays the first image IM1n shown in FIG. 24 on the display unit 11. When the second user U2 is located within the field of view of the first user U1, the first image IM1n shows the real space image R1, the first content C11m, and the third content C13m in front of the first user U1. The second user U2 is displayed in the real space image R1. That is, the first user U1 can view the second user U2 present in the real space displayed on the display unit 11A. The third content C13n is the same as the third content C13m shown in FIG. 23 except for the transparency. The transparency of the third content C13n may be set so that the third content C13n transmits the first content C11m. The third content C13n is placed closer to the front of the paper than the first content C11m, and a portion of the third content C13n overlaps a portion of the first content C11m. This overlapping portion of the first content C11m transmits a portion of the third content C13n. This makes it possible to display the first content C11m so that the first content C11m transmits the third content C13n, making it easier for the first user U1 to view the first content C11m and easily confirm the first content C1 that they are viewing.

[0092] In step ST13, the control unit 14A may display at least one of the first content C11 and the third content C13 and stop displaying the remaining content based on the gaze degree of the first user U1. Furthermore, the control unit 14A may display the second content C2 around the real image of the second user U2 in the first image IM1k.

[0093] For example, when the degree of gaze of the first user U1 on the first content C11 shown in FIG. 22 is higher than the degree of gaze of the first user U1 on the third content C13, the control unit 14A displays the first content C11 while stopping the display of the third content C13. The control unit 14A causes the display unit 11 to display the first image IM1r shown in FIG. 27. When the second user U2 is located within the field of view of the first user U1, the first image IM1r shows the real space image R1, the first content C11, and the second content C2 in front of the first user U1. The second user U2 is displayed in the real space image R1. The first user U1 can visually recognize the second user U2 existing in the real space displayed on the display unit 11A. The control unit 14A displays the first content C11 while stopping the display of the third content C13. Furthermore, the control unit 14A displays the second content C2 in the vicinity of the real image of the second user U2 in the first image IM1r. This causes only the first content C11 that the first user U1 has a high degree of gaze at to be displayed, and the second content C2 to be displayed in the vicinity of the real image of the second user U2. This allows the second user U2 and the second content C2 that the second user U2 is viewing to be easily confirmed, along with the first content C11 that the first user U1 has been gazing at.

[0094] <Eighth Specific Example> Next, an eighth specific example of steps ST11 to ST13 will be described with reference to FIGS.

[0095] Steps ST11 and ST12 are executed in the same manner as in the third specific example described above. Subsequently, in step ST13, the control unit 14A displays the second content C2 in the periphery of the real image of the second user U2 in the first image IM1, and displays the first content C1 at a position apart from the real image of the second user U2 in the first image IM1, as in the third specific example. Also, in this step ST13, if the control unit 14A determines that the real image of the second user U2 in the first image IM1b is located within the central region of the first image IM1b, it changes at least one of the size, resolution, and transparency of the first content C1 and the second content C2.

[0096] For example, when the control unit 14A determines that the real image of the second user U2 in the first image IM1p shown in FIG. 25 is not located within the central region AC of the first image IM1p, the control unit 14A increases the transparency of the second content C2p. The central region AC may be, for example, an area including the center point PC of the first image IM1p, and may be arbitrarily determined in advance. The control unit 14A causes the display unit 11A to display the first image IM1p shown in FIG. 25. When the second user U2 is located within the field of view of the first user U1, the first image IM1p displays the real space image R1, the first content C1, and the second content C2p in front of the first user U1. The second user U2 is displayed in the real space image R1. The first user U1 can view the second user U2 present in the real space displayed on the display unit 11A. The second content C2p is the same as the second content C2 shown in FIG. 15, except for the transparency. The transparency of the second content C2p is higher than that of the second content C2, which allows the first user U1 to view the second content C2p and easily confirm the second content C2 that the second user U2 is viewing.

[0097] For example, when the control unit 14A determines that the real image of the second user U2 in the first image IM1q shown in FIG. 26 is located within a central region AC of the first image IM1q, the control unit 14A reduces the transparency of the second content C2q. The central region AC may be any region located at a predetermined center point PC of the first image IM1q. The control unit 14A causes the display unit 11A to display the first image IM1q shown in FIG. 26. When the second user U2 is located within the field of view of the first user U1, the first image IM1p shows a real space image R1, the first content C1, and the second content C2q in front of the first user U1. The second user U2 is displayed in the real space image R1. That is, the first user U1 can view the second user U2 present in the real space of the display unit 11A. The second content C2q is the same as the second content C2 shown in FIG. 15, except for the transparency. The transparency of the second content C2q is lower than the transparency of the second content C2. When the real image of the second user U2 is located within the central region AC of the first image IM1q, it is highly likely that the first user U1 is gazing at the second user U2. Therefore, the first user U1 can easily confirm the second content C2 viewed by the second user U2 by viewing the second content C2q, which has low transparency. Furthermore, the control unit 14A may increase the size of the second content C2, increase the resolution of the second content C2, or decrease the transparency of the second content C2 as the real image of the second user U2 approaches the central region AC of the first image IM1q. This makes it easier for the second user U2 to confirm the second content C2 viewed by the second user U2 as the second user U2 approaches the center of the field of view of the first user U1. As described above, in the third to eighth specific examples, the control unit 14A displays the second contents C2, C2i, C2j, C2p, and C2q in the vicinity of the real image of the second user U2 in the first images IM1d-IM1j and IM1p-IM1r. Simultaneously, the control unit 14A displays the first contents C1, C1e-C1h, and C11 at positions spaced apart from the real image of the second user U2 in the first images IM1d-IM1j and IM1p-IM1r. Simultaneously, the control unit 14A changes the size, resolution, and transparency of the first content C1, etc. or the second content C2, etc.On the other hand, the control unit 14A may change the size, resolution, and transparency of the first content C1, etc. or the second content C2, etc. without moving the position of the first content C1, etc. within the first image IM1, etc.

[0098] (Other embodiments, etc.) The head mounted display systems 100, 100A, and 100B according to the above-described embodiments can be expressed as programs executed by the control units 14, 14A, 14B, 24, and 24B. The above-described control units 14, 14A, 14B, 24, and 24B include a processor and a memory (not shown). The configuration of the control units 14, 14A, 14B, 24, and 24B described in the above-described embodiments is realized by the processor reading and executing a program stored in the memory. In other words, this program is a control program that causes the processor to function as the head mounted display system 100, 100A, and 100B, or as a part thereof.

[0099] The above-described program includes a set of instructions (or software code) that, when loaded into a computer, causes the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, optical disk storage, and magnetic storage devices. The program may be transmitted on a transitory computer-readable medium or a communication medium. By way of example and not limitation, transitory computer-readable media or communication media include electrical, optical, or other forms of propagated signals.

[0100] Furthermore, as described in the various embodiments above regarding the processing procedures in the head mounted display systems 100, 100A, and 100B, the present disclosure may also be embodied as a control method for the head mounted display systems 100, 100A, and 100B. Also, the above-described program can be said to be a control program for causing the head mounted display systems 100, 100A, and 100B to execute such a control method.

[0101] The present invention is not limited to the above-described embodiments, and can be modified as appropriate without departing from the spirit of the present invention. The present invention may also be implemented by appropriately combining the above-described embodiments and examples thereof. For example, the head-mounted display system 100 shown in FIG. 1 includes a first HMD 10 and a second HMD 20, but may also include one or more additional HMDs. Similarly, the head-mounted display systems 100A and 100B may also include one or more additional HMDs.

[0102] Furthermore, in the head-mounted display system 100, the configuration of the second HMD 20 may be different from the configuration of the first HMD 10. If at least one of the first HMD 10 and the second HMD 20 has the same configuration as the first HMD 10 shown in FIG. 1, a user wearing the other of the first HMD 10 and the second HMD 20 can check the displayed image. The same applies to the head-mounted display systems 100A and 100B. For example, the head-mounted display system 100A may include one of the first HMD 10B and the second HMD 20B. Similarly, the head-mounted display system 100B may include one of the first HMD 10A and the second HMD 20A.

[0103] Furthermore, if the first HMD 10 and the second HMD 20 are fully immersive head-mounted displays, the present invention can be implemented by switching them to the above-mentioned video-transparent head-mounted displays. For example, when the control unit 14 determines that the second user is located within the field of view of the first user, it causes the display unit 11 to display the real space image R1 (switching to video-transparent type) and causes the real image of the second user and the second video to be displayed around the second user. Furthermore, when the control unit 14 detects the second user within a predetermined distance range of the first user, it causes the display unit 11 to display the peripheral video (switching to video-transparent type), and when it determines that the second user is located within the field of view, it causes the second video to be displayed around the second user. [Explanation of symbols]

[0104] 100, 100A, 100B Head Mounted Display System 10, 10A, 10B First HMD 20, 20A, 20B Second HMD 11, 11A, 11B, 21 Display section 11c, 21c display surface 12, 12A, 22 Communications Department 13, 13A, 23 Position detection unit 14, 14A, 14B, 24 Control section 15, 15A, 25 Orientation detection unit 16 Imaging unit U1 First User U2 Second User IM1, IM1a~IM1k, IM1m~IM1q First video IM2, IM2b, second video IM31 Third Image R1, R2 Real space images C1, C1e~C1h, C11, C11m First content C2, C2i, C2j, C2p, C2q Secondary Content C13, C13m, C13n Third Content AC central area PC center point

Claims

1. a first HMD (head mounted display) worn by a first user; a second HMD worn by a second user; The first HMD includes: a display unit that displays a first video to the first user; a communication unit that acquires, from the second HMD, a second image that the second HMD displays to the second user; a position detection unit that detects a positional relationship between the first HMD and the second HMD; an orientation detection unit that detects an orientation of the first HMD; a control unit, the display unit is configured to be able to view a real space within a visual field of the first user, and when the control unit determines that the second user is located within the visual field of the first user based on the positional relationship and the orientation of the first HMD, the control unit displays the second video around a real image of the second user viewed by the first user on the display unit. Head-mounted display system.

2. the control unit changes at least one of a size, a resolution, and a transparency of the second image based on a distance between the first HMD and the second HMD. The head-mounted display system according to claim 1 .

3. the first video includes a first content; the second video includes second content; When the control unit determines that the second user is located within the field of view of the first user based on the positional relationship and the orientation of the first HMD, displaying the first content at a position separated from the real image of the second user on the display unit, and displaying the second content around the real image of the second user on the display unit, by the display unit; The head-mounted display system according to claim 1 or 2.

4. the first video includes a first content; the second video includes second content; the first HMD acquires a first gaze degree indicating a degree to which the first user gazes at the first content; the second HMD acquires a second gaze degree indicating a degree to which the second user gazes at the second content; When the control unit determines that the second user is located within the field of view of the first user based on the positional relationship and the orientation of the first HMD, the control unit changes at least one of a size, a resolution, and a transparency of the first and second contents based on the first gaze degree or the second gaze degree. The head-mounted display system according to claim 1 or 2.

5. the first video includes a first content; the second video includes second content; the control unit changes at least one of a size, a resolution, and a transparency of the first and second contents as the real image of the second user on the display unit approaches a central region of the display unit. The head-mounted display system according to claim 1 or 2.

Images