Information processing apparatus, head-mounted display device, information processing method, and program

Abstract

To reduce discomfort of a user who feels from gaze of a person who is wearing HMD around the user.SOLUTION: An information processing apparatus includes: control means for displaying a virtual object on a first display device serving as a head-mounted display device, so as to synthesize the virtual object with a real space; and a first acquisition means which acquires gaze information about whether a user wearing a second display device, serving as a head-mounted display, different from the first display device is gazing at a virtual object displayed by the second display device. The control means is configured to display a specific virtual object on the first display device on the basis of the gaze information acquired by the acquisition means.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, a head-mounted display device, an information processing method, and a program.

Background Art

[0002] A head-mounted display device, a so-called head-mounted display (HMD), having a shape similar to glasses or goggles and projecting an image for augmented reality or mixed reality on a portion corresponding to a transmissive lens has been developed. In this HMD, virtual objects are displayed (projected) so as to be synthesized (overlaid) in the real space, and a user wearing the HMD can view the virtual objects as if they actually exist.

[0003] When such an HMD becomes widely popular, it is conceivable to use the HMD in an environment where there are people around, like a smartphone. Further, when the HMD is made smaller and lighter and it becomes indistinguishable from glasses, it becomes difficult for people around to easily determine whether the HMD is being used or not.

[0004] Consider the case where the HMD wearer A is gazing at a virtual object (content). In this case, if the virtual object is displayed so as to overlap the person B who is in front of the HMD wearer A, the line-of-sight direction of the HMD wearer A may inadvertently face the person B. Then, the person B cannot determine whether the HMD wearer A is using the HMD or not, misidentifies that he / she is being stared at by the HMD wearer A, and may become concerned about or feel uncomfortable with the line-of-sight of the HMD wearer A.

[0005] Patent Document 1 discloses a method of specifying based on information on the transition of the line of sight for determining the degree of attention of a user to a virtual object based on the line of sight of the user.

Prior Art Documents

Patent Documents

[0006] [Patent Document 1] Japanese Patent Publication No. 2018-195319 [Overview of the project] [Problems that the invention aims to solve]

[0007] However, even if the technology disclosed in Patent Document 1 is used, it is not possible to alleviate discomfort caused by the gaze of other HMD wearers around you.

[0008] The present invention aims to provide a technology that can reduce discomfort caused by the gaze of other HMD wearers in the vicinity. [Means for solving the problem]

[0009] A first aspect of the present invention is an information processing device comprising: control means for causing a virtual object to be displayed on a first display device, which is a head-mounted display device, so as to be synthesized into real space; and first acquisition means for acquiring gaze information regarding whether a user wearing a second display device, which is a head-mounted display device different from the first display device, is gazing at a virtual object displayed on the second display device, wherein the control means causes a specific virtual object to be displayed on the first display device based on the gaze information acquired by the acquisition means.

[0010] A second aspect of the present invention is a method for compositing a virtual object into real space, the virtual object The information processing device is characterized by comprising: control means for displaying an object on a head-mounted display device; acquisition means for acquiring gaze information relating to the gaze direction of a user wearing the head-mounted display device; determination means for determining whether or not the user is gazing at the virtual object based on the display position of the virtual object and the gaze information acquired by the acquisition means; and output means for outputting gaze information relating to the determination result of the determination means to an external source.

[0011] A third aspect of the present invention is a head-mounted display device characterized by comprising the above-described information processing device and a display means for displaying virtual objects in a manner that they are synthesized into real space.

[0012] A fourth aspect of the present invention is an information processing method comprising: a control step of causing a virtual object to be displayed on a first display device, which is a head-mounted display device, so as to be synthesized into real space; and an acquisition step of acquiring gaze information regarding whether a user wearing a second display device, which is a head-mounted display device different from the first display device, is gazing at a virtual object displayed on the second display device, wherein the control step causes a specific virtual object to be displayed on the first display device based on the gaze information acquired in the acquisition step.

[0013] A fifth aspect of the present invention is an information processing method characterized by comprising: a control step of displaying a virtual object on a head-mounted display device so as to synthesize the virtual object into real space; an acquisition step of acquiring gaze information relating to the gaze direction of a user wearing the head-mounted display device; a determination step of determining whether or not the user is gazing at the virtual object based on the display position of the virtual object and the gaze information acquired in the acquisition step; and an output step of outputting gaze information relating to the determination result of the determination step to an external source.

[0014] A sixth aspect of the present invention is a program for causing a computer to function as one of the means of the information processing apparatus described above. [Effects of the Invention]

[0015] According to the present invention, it becomes possible to reduce discomfort caused by the gaze of other HMD wearers around the user. [Brief explanation of the drawing]

[0016] [Figure 1] This is a block diagram showing an example of an HMD configuration. [Figure 2] It is a schematic diagram showing an example of the usage status of the HMD. [Figure 3] It is a diagram related to the field of view of a user wearing an HMD. [Figure 4] Diagram related to the field of view of a user wearing an HMD [Figure 5] It is a flowchart showing an example of the processing of the HMD according to Embodiment 1. [Figure 6] It is a flowchart showing an example of the fixation information acquisition processing according to Embodiment 1. [Figure 7] It is a flowchart showing an example of the processing of the HMD according to Embodiment 2. [Figure 8] It is a flowchart showing an example of the fixation information acquisition processing according to Embodiment 2. [Figure 9] It is a schematic diagram showing an example of a virtual object according to Embodiment 2.

Mode for Carrying Out the Invention

[0017] (Embodiment 1) Hereinafter, Embodiment 1 of the present invention will be described. FIG. 1 is a block diagram showing an example of the configuration of a head-mounted display (HMD) 100 which is a head-mounted display device according to Embodiment 1. The HMD 100 includes image input units 101, 102, a gaze detection unit 103, a position information acquisition unit 104, an image processing unit 105, a control unit 106, a ROM 107, a RAM 10 8, a communication unit 109, image projection units 110, 111, and lens units 112, 113. These are connected to each other via a bus 114. Note that the image processing unit 105, the control unit 106, the ROM 107, the RAM 108, etc. may be provided in an information processing device (for example, a dedicated controller or a smartphone) separate from the HMD 100.

[0018] The image input unit 101 has a lens and an imaging device, and captures an image (moving image) by imaging the visual field direction of the left eye of the user wearing the HMD 100 (on the head). The image input unit 102 has a lens and an imaging device, and captures an image (moving image) by imaging the visual field direction of the right eye of the user wearing the HMD 100. Although an example of a binocular HMD that captures left and right images with two image input units 101 and 102 is shown, the present invention is also applicable to a monocular HMD that captures one image.

[0019] The gaze detection unit 103 has a lens, an imaging device, and an image processing circuit, and detects the gaze direction of each of the right and left eyes of the user wearing the HMD 100. Then, the gaze detection unit 103 outputs gaze information regarding the detected gaze direction. The gaze information is, for example, information indicating the gaze direction of the user wearing the HMD 100.

[0020] The position information acquisition unit 104 measures the distance and position of surrounding HMDs and acquires the position information of the surrounding HMDs. For example, the position information acquisition unit 104 acquires position information by UWB (Ultra Wide Band) wireless communication. The position information acquisition unit 104 receives a wireless signal for position detection, and identifies surrounding HMDs using distances, azimuths, identifiers, etc. measured from the time difference of arrival or the difference in radio wave intensity of the wireless signal.

[0021] The image processing unit 105 performs various image processing operations on the images (image signals, image data) captured by the image input units 101 and 102, such as compression and decompression, white balance adjustment, interpolation, edge enhancement, gamma correction, and gradation conversion. The image processing unit 102 detects people (faces) from the images captured by the image input units 101 and 102. Furthermore, the image processing unit 102 detects the gaze direction of the detected person (face) from the images captured by the image input units 101 and 102. Since gaze detection (detection of gaze direction) is a well-known technique, a detailed explanation of it will be omitted. With gaze detection using a visible light camera, it is possible to detect the gaze direction of a person wearing glasses, as well as the gaze direction of a person wearing an optical see-through HMD. An optical see-through HMD, for example, has lenses similar to those of glasses (see-through lenses) and projects virtual objects onto the lenses. A user wearing an optical see-through HMD can directly see the real world through the lenses. Furthermore, the user can also view the virtual object projected onto the lens. The image processing unit 105 also generates a virtual object (image) for the left eye and a virtual object (image) for the right eye. The image processing unit 105 may be implemented as a processing circuit or as a computer program executed by the control unit 106.

[0022] The control unit 106 acquires gaze information from the gaze detection unit 103. Based on the acquired gaze information (the direction of the gaze of the user wearing the HMD 100) and the display position of the virtual object generated by the image processing unit 105, the control unit 106 determines whether the user wearing the HMD 100 is looking at the virtual object. The control unit 106 also comprehensively controls each device connected to the bus 114. The ROM 107 stores various programs and parameters. The RAM 108 is used by the control unit 106 as main memory and work area. The control unit 106 loads programs from the ROM 107 into the RAM 108 and executes the programs loaded into the RAM 108 to perform various operations.

[0023] The communication unit 109 connects to and communicates with external devices via a network. For example, communication Unit 109 handles internet communication using TCP / IP, etc.

[0024] The image projection unit 110 projects (displays) the virtual object for the left eye, generated by the image processing unit 105, onto the lens unit 112. The image projection unit 111 projects (displays) the virtual object for the right eye, generated by the image processing unit 105, onto the lens unit 113.

[0025] Lens unit 112 is mounted on the HMD 100 so as to be positioned in front of the left eye of the user wearing the HMD 100, and has a see-through lens. By projecting a virtual object from the image projection unit 110 onto this see-through lens, the virtual object is displayed in a way that it appears to be superimposed (composited) onto the real space. Lens unit 113 is mounted on the HMD 100 so as to be positioned in front of the left eye of the user wearing the HMD 100, and has a see-through lens. By projecting a virtual object from the image projection unit 111 onto this see-through lens, the virtual object is displayed in a way that it appears to be superimposed (composited) onto the real space. Although an example of a binocular HMD has been shown in which left and right images are displayed on the left and right lens units 112 and 113 using two image projection units 110 and 111, the present invention is also applicable to a monocular HMD that captures a single image (as described above).

[0026] Figure 2 is a schematic diagram showing an example of how HMD100 is used. In the situation shown in Figure 2, user 201 wearing HMD100 and user 202 wearing HMD100' are facing each other. HMD100' is assumed to have the same configuration as HMD100.

[0027] Figures 3(A) and 3(B) show the field of view of user 201 wearing the HMD 100. Image 301 shown in Figure 3(A) is an image captured by the image input unit 101 of the HMD 100. As shown in Figure 2, user 201 and user 202 are facing each other, so user 202 is visible in image 301. The image input unit 102 of the HMD 100 also captures an image similar to image 301. Figure 3(B) will be discussed later.

[0028] Figures 4(A) and 4(B) show the field of view of user 202 wearing the HMD 100'. Image 401 shown in Figure 4(A) is an image captured by the image input unit 101 of the HMD 100'. As shown in Figure 2, user 201 and user 202 are facing each other, so user 201 is visible in image 401. The image input unit 102 of the HMD 100' also captures an image similar to image 401.

[0029] Figure 4(B) shows the range that user 202 can see through the lens portion 112 of HMD 100' (user 202's field of view). A virtual object 402 is displayed on the lens portion 112. Since the virtual object 402 is displayed so as to overlap with user 201's face, user 202 can see the virtual object 402 but cannot see user 201's face. The range that user 202 can see through the lens portion 112 of HMD 100' is the same as in Figure 4(B).

[0030] When user 202 is looking at virtual object 402, even if user 202 is not looking at user 201, user 202's line of sight will be directed towards user 201. Therefore, if user 202's face is included in user 201's field of view, as in image 401 in Figure 3(A), user 201 may mistakenly perceive that user 202 is looking at them, causing them to become self-conscious or uncomfortable with user 202's gaze.

[0031] Therefore, in Embodiment 1, in the situation shown in Figures 3(A), 4(A), and 4(B), the HMD 100 displays a specific virtual object 302 as shown in Figure 3(B). Figure 3(B) shows the range that the user 201 can see through the lens portion 112 of the HMD 100 (the user 201's field of view). The virtual object 302 is displayed in the lens portion 112. The virtual object 302 is displayed so as to overlap with the face of user 202, so even though user 201 can see the virtual object 302, they cannot see user 202's face. This helps to suppress the aforementioned misidentification and reduce the aforementioned discomfort.

[0032] Figure 5 is a flowchart showing an example of the processing performed by the HMD100. For example, the control unit 106 of the HMD100 loads a program from the ROM 107 into the RAM 108, and then executes the program loaded into the RAM 108 to perform the processing shown in Figure 5.

[0033] In step S501, the control unit 106 controls the position information acquisition unit 104 to identify HMDs in the vicinity of HMD 100 and acquire the position information of the identified HMDs. For example, the position information of HMD 100' is acquired.

[0034] In step S502, the control unit 106 controls the image input units 101 and 102 to acquire an image of the field of view of the user 201 wearing the HMD 100. The control unit 106 also controls the image processing unit 105 to perform various image processing on the acquired image. For example, image 301 in Figure 3(A) is acquired.

[0035] In step S503, the control unit 106 controls the image processing unit 105 to detect a person (face) from the image acquired in step S502. For example, the face of user 202 wearing the HMD 100' is detected from image 301 in Figure 3(A).

[0036] In step S504, the control unit 106 controls the image processing unit 105 to detect the gaze direction of the person (face) detected in step S503 from the image acquired in step S502. For example, the gaze direction of user 202 wearing the HMD 100' is detected from image 301 in Figure 3(A).

[0037] In step S505, the control unit 106 determines whether the gaze direction detected in step S504 is directed toward user 201 wearing the HMD 100. If the control unit 106 determines that the gaze direction detected in step S504 is directed toward user 201, for example, that the gaze direction of user 202 wearing the HMD 100' is directed toward user 201, it proceeds to step S506. If the control unit 106 determines that the gaze direction detected in step S504 is not directed toward user 201, for example, that the gaze direction of user 202 is not directed toward user 201, it terminates the process shown in Figure 5. In addition, it is possible that multiple people (multiple faces) are detected in step S503, and the gaze direction of each of the multiple people (multiple faces) is detected in step S504. In that case, if the control unit 106 determines that the gaze direction of at least one of the multiple people (multiple faces) is directed toward user 201, it proceeds to step S506. If the control unit 106 determines that the gaze directions of all the multiple people (multiple faces) are not directed towards the user 201, it terminates the process shown in Figure 5.

[0038] In step S506, the control unit 106 obtains gaze information of the person identified in step S505 (the person whose gaze direction is determined to be directed toward user 201) from the HMD worn by that person. The HMD worn by the person identified in step S505 is, for example, the HMD whose position information was obtained in step S501, and is identified based on the processing results of step S501 and step S505. In step S506, for example, gaze information of user 202 is obtained from HMD 100'. The gaze information is information regarding whether the person wearing the HMD is gazing at a virtual object, for example, information indicating whether user 202 is gazing at a virtual object displayed by HMD 100'.

[0039] Thus, in Embodiment 1, gaze information is acquired that corresponds to a person who is in the user 201's field of view and is facing the user 201.

[0040] Figure 6 is a flowchart showing an example of the process in step S506 (gaze information acquisition process). Figure 6 shows an example where the person identified in step S505 is user 202, and user 202's gaze information is acquired from HMD 100'. Figure 6 shows the processing of HMD 100 and the processing of HMD 100'.

[0041] In step S601, the control unit 106 of the HMD 100' controls the image processing unit 105 and image projection units 110, 111 of the HMD 100' to generate and display virtual objects. For example, the virtual object 402 in Figure 4(B) is displayed in the lens units 112, 113.

[0042] In step S602, the control unit 106 of the HMD 100 controls the communication unit 109 of the HMD 100 to send a request to the HMD 100' for gaze information of the person (user 202) wearing the HMD 100'.

[0043] In step S603, the control unit 106 of the HMD 100' controls the communication unit 109 of the HMD 100' to receive the request (a request for gaze information) sent from the HMD 100 in step S602.

[0044] In step S604, the control unit 106 of the HMD 100' controls the gaze detection unit 103 of the HMD 100' to detect the direction of the user 202's gaze.

[0045] In step S605, the control unit 106 of the HMD 100' determines whether the user 202 is gazing at the virtual object displayed in step S601, based on the gaze direction detected in step S604. For example, if the gaze direction is directed towards the virtual object, it is determined that the user is gazing at the virtual object, and if the gaze direction is not directed towards the virtual object, it is determined that the user is not gazing at the virtual object.

[0046] In step S606, the control unit 106 of the HMD 100' controls the communication unit 109 of the HMD 100' to transmit gaze information regarding the determination result of step S605 to the HMD 100 (external output).

[0047] In step S607, the control unit 106 of the HMD 100 controls the communication unit 109 of the HMD 100 to receive the gaze information transmitted from the HMD 100' in step S606.

[0048] Returning to the explanation of Figure 5, in step S507, the control unit 106 (of the HMD 100) determines, based on the gaze information acquired (received) in step S506, whether the person identified in step S505 is gazing at the virtual object. If the control unit 106 determines that the person identified in step S505 is gazing at the virtual object, for example, that user 202 is gazing at the virtual object displayed by HMD 100', the process proceeds to step S508. If the control unit 106 determines that the person identified in step S505 is not gazing at the virtual object, for example, that user 202 is not gazing at the virtual object displayed by HMD 100', the process in Figure 5 ends. In the situation shown in Figure 4(B), user 202's gaze is directed towards user 201, but user 202 is looking at the virtual object 402 and not at user 201. In this situation, the process proceeds to step S508.

[0049] In step S508, the control unit 106 controls the image processing unit 105 and the image projection units 110, 111 to generate a specific virtual object and display it at a position based on the location of the person identified in step S505. For example, as shown in Figure 3(B), the virtual object 302 is displayed at a position that overlaps with the eyes of user 202. This prevents user 201 from seeing user 202's eyes.

[0050] In addition, it may be determined in step S505 that the gaze direction of each of the multiple people (multiple faces) is directed towards user 201. In that case, the processing in steps S506 to S508 is performed for each of the multiple people (multiple faces).

[0051] Furthermore, while an example was described in which a specific virtual object is not displayed if the person identified in step S505 is not looking at the virtual object, and is displayed if the person is looking at the virtual object, the system is not limited to this. For example, the specific virtual object may be displayed in both of these cases. In that case, the type, form, and display pattern of the specific virtual object should be different between the two cases to make it possible to distinguish whether or not the person identified in step S505 is looking at the virtual object.

[0052] Furthermore, while we have described an example where the gaze information of individuals is acquired after narrowing down the individuals (step S505) (step S506), this is not the only example. For instance, the gaze information of all individuals detected in step S503 may be acquired from an external source, and the gaze information may be narrowed down based on the processing result (gaze information) of step S504.

[0053] Furthermore, while we have described an example in which gaze information is sent and received in both cases—when the person identified in step S507 is gazing at the virtual object and when they are not—the system is not limited to this. For example, gaze information may be sent and received in only one of these two cases.

[0054] Furthermore, while an example was described in which a specific virtual object is displayed at a position that overlaps with the eyes of the person identified in step S505 from the perspective of user 201, this is not the only example. For instance, a specific virtual object may be displayed at a position near (around) the person identified in step S505 from the perspective of user 201, so as to be associated with that person.

[0055] As described above, according to Embodiment 1, a specific virtual object is displayed based on the gaze information of surrounding HMD wearers (such as whether or not they are gazing at the virtual object). This reduces discomfort caused by the gaze of surrounding HMD wearers.

[0056] (Embodiment 2) The following describes Embodiment 2 of the present invention. Embodiment 1 described an example in which gaze information is obtained by determining whether or not the gaze direction is directed toward user 201. Embodiment 2 describes an example in which gaze information is obtained without performing such a determination. Note that explanations of points similar to Embodiment 1 (such as configuration and processing) will be omitted as appropriate, and the points that differ from Embodiment 1 will be explained in detail.

[0057] Figure 7 is a flowchart showing an example of the processing of the HMD100 according to Embodiment 2. For example, the control unit 106 of the HMD100 loads a program from the ROM 107 into the RAM 108 and executes the program loaded into the RAM 108 to realize the processing shown in Figure 7.

[0058] The processing in step S701 is the same as the processing in step S501 in Embodiment 1 (Figure 5). In step S702, the control unit 106 processes the position information acquired in step S701. Gaze information is acquired from the corresponding HMD (HMD from which position information has been acquired by HMD100). This allows, for example, gaze information corresponding to a person in the user 201's field of view to be acquired. Alternatively, the same processing as in steps S502 and S503 in Figure 5 may be performed to acquire gaze information for the detected person.

[0059] Figure 8 is a flowchart of an example of the process in step S702 (gaze information acquisition process). Figure 8 shows an example where the HMD from which position information was acquired in step S701 is HMD100'', and gaze information is acquired from HMD100''. Figure 8 shows the processing of HMD100 and the processing of HMD100''. It is assumed that HMD100'' has the same configuration as HMD100.

[0060] The processing in steps S801 to S804 is the same as the processing in steps S601 to S604 in Embodiment 1 (Figure 6).

[0061] In step S805, similar to step S605, the control unit 106 of the HMD 100'' determines whether the virtual object displayed in step S801 is being gazed upon, based on the gaze direction detected in step S804. If the control unit 106 of the HMD 100'' determines that the virtual object is being gazed upon, it proceeds to step S806; if it determines that the virtual object is not being gazed upon, it proceeds to step S807.

[0062] In step S806, the control unit 106 of the HMD 100'' generates (acquires) type information regarding the type of virtual object (e.g., video, still image, text, etc.) that was determined to be viewed in step S805. The type information is, for example, information indicating the type of virtual object. The type information may also be mode information regarding the operation mode (application) corresponding to the virtual object. The mode information corresponding to the virtual object is, for example, information indicating the operation mode. Examples of operation modes include a video mode that plays a video in the virtual object's window, a text mode that displays text in the virtual object's window, and a telephone mode that displays the image of the person being talked to as a virtual object.

[0063] The processing in steps S807 and S808 is the same as the processing in steps S606 and S607 in Embodiment 1 (Figure 6). However, if the processing in step S806 is performed, the attention information includes virtual object type information.

[0064] Returning to the explanation of Figure 7, the processing in steps S703 and S704 is the same as the processing in steps S507 and S508 of Embodiment 1 (Figure 5). The specific virtual object is displayed, for example, at a position that overlaps with the eyes of the person wearing the HMD (another HMD wearer in the vicinity) whose position information was acquired in step S701. At this time, the control unit 106 (of the HMD 100) performs processing similar to that in steps S502 and S503 of Figure 5, for example, and determines the display position of the specific virtual object based on the processing result.

[0065] In step S704, the control unit 106 displays virtual objects that can identify the type of virtual object that the surrounding HMD wearer is looking at, based on the received type information (type information of virtual objects), as specific virtual objects. As shown in Figure 9, virtual objects 901 to 903 are displayed. Figure 9 shows the range that user 201 can see through the lens section 112 of the HMD 100. Virtual object 901 indicates video mode and is displayed so as to overlap with the user who is watching a video. Virtual object 902 indicates text mode and is displayed so as to overlap with the user who is reading text. Virtual object 903 indicates telephone mode and is displayed so as to overlap with the user who is making a call.

[0066] As described above, in Embodiment 2 as well, a specific virtual object is displayed based on the gaze information of surrounding HMD wearers, thereby reducing discomfort caused by the gaze of surrounding HMD wearers. Furthermore, according to Embodiment 2, the gaze information of surrounding HMD wearers includes information on the type of virtual object being gazed upon, and different virtual objects are displayed as specific virtual objects depending on the type information. As a result, the user can easily understand which virtual objects surrounding HMD wearers are gazing upon.

[0067] It should be noted that Embodiments 1 and 2 are merely examples, and configurations obtained by appropriately modifying or changing the configurations of Embodiments 1 and 2 within the scope of the gist of the present invention are also included in the present invention. Configurations obtained by appropriately combining the configurations of Embodiments 1 and 2 are also included in the present invention.

[0068] For example, while an example of an optical see-through HMD has been described, the present invention is also applicable to a video see-through HMD. A video see-through HMD displays an image (virtual space) of the real world, with virtual objects composited as needed. A user wearing a video see-through HMD cannot directly see the real world, but can indirectly see the real world by viewing the displayed image.

[0069] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0070] This embodiment includes the following configurations, methods, and programs. (Composition 1) Control means for causing the virtual object to be displayed on a first display device, which is a head-mounted display device, so as to synthesize the virtual object into the real space, A first acquisition means for acquiring gaze information regarding whether a user wearing a second display device, which is a head-mounted display device different from the first display device, is gazing at a virtual object displayed by the second display device. It has, The control means causes a specific virtual object to be displayed on the first display device based on the gaze information acquired by the acquisition means. An information processing device characterized by the following: (Configuration 2) The first acquisition means acquires gaze information corresponding to a person in the field of view of the user wearing the first display device. The information processing device according to configuration 1, characterized by the above. (Composition 3) The first acquisition means acquires gaze information corresponding to a person who is in the field of view of the user wearing the first display device and is facing the user wearing the first display device. An information processing device according to configuration 1 or 2, characterized by the above. (Composition 4) A second acquisition means for acquiring position information of a head-mounted display device different from the first display device, A first detection means for detecting a person from an image captured in the field of view of a user wearing the first display device, and It further possesses, The information processing apparatus according to Configuration 1, characterized in that the first acquisition means is equipped with a head-mounted display device from which location information has been acquired by the second acquisition means, and the first detection means acquires gaze information corresponding to a person detected. (Composition 5) The system further includes a second detection means for detecting the gaze direction of a person detected by the first detection means from the aforementioned image, The first acquisition means acquires gaze information corresponding to a person whose gaze direction, as detected by the second detection means, is directed toward the user wearing the first display device, and whose positional information has been acquired by the second acquisition means. The information processing apparatus according to configuration 4, characterized by the features described above. (Composition 6) The control means causes the first display device to display the specific virtual object at a position based on the position of the user wearing the second display device. An information processing device according to any one of configurations 1 to 5, characterized by the above. (Composition 7) The control means causes the first display device to display the specific virtual object in such a position that it is in line with the eyes of the user wearing the second display device, as seen from the perspective of the user wearing the first display device. An information processing device according to any one of configurations 1 to 6, characterized by the above. (Composition 8) The control means is If the user wearing the second display device is not looking at the virtual object displayed by the second display device, the specific virtual object will not be displayed on the first display device. When a user wearing the second display device is looking at a virtual object displayed on the second display device, the specific virtual object is displayed on the first display device. An information processing device according to any one of configurations 1 to 7, characterized by the above. (Composition 9) When a user wearing the second display device is looking at a virtual object displayed by the second display device, The gaze information includes information about the type of virtual object that the user wearing the second display device is gazing at, The control means causes the first display device to display a virtual object on the first display device that can identify the type of virtual object that the user wearing the second display device is looking at, as the specific virtual object. An information processing device according to any one of configurations 1 to 8, characterized by the above. (Composition 10) Control means for displaying the virtual object on a head-mounted display device so as to synthesize the virtual object into the real space, An acquisition means for acquiring gaze information relating to the gaze direction of a user wearing the head-mounted display device, A determination means for determining whether the user is looking at the virtual object based on the display position of the virtual object and the gaze information acquired by the acquisition means, An output means for outputting attention information related to the determination result of the determination means to an external source. An information processing device characterized by having the following features. (Composition 11) The aforementioned attention information includes information about the type of virtual object. The information processing apparatus according to configuration 10, characterized by the above. (Composition 12) An information processing device described in any one of items 1 to 11, A display means for displaying virtual objects as if they were being synthesized into real space. A head-mounted display device characterized by having the following features. (method) A control step of causing the virtual object to be displayed on a first display device, which is a head-mounted display device, so as to synthesize the virtual object into the real space, An acquisition step to acquire gaze information regarding whether a user wearing a second head-mounted display device, which is different from the first display device, is gazing at a virtual object displayed by the second display device. It has, In the control step, a specific virtual object is displayed on the first display device based on the gaze information acquired in the acquisition step. An information processing method characterized by the following: (method) A control step of displaying the virtual object on a head-mounted display device so as to synthesize the virtual object into the real space, The acquisition step involves acquiring gaze information regarding the direction of gaze of a user wearing the head-mounted display device, A determination step that determines whether the user is looking at the virtual object based on the display position of the virtual object and the gaze information acquired in the acquisition step, An output step that outputs attention information regarding the determination result of the aforementioned determination step to an external source. An information processing method characterized by having the following features. (program) A program for causing a computer to function as one of the means of the information processing device described in any one of the configurations 1 to 11. [Explanation of symbols]

[0071] 100: HMD 106: Control Unit

Claims

1. a control means for displaying the virtual object on a first display device, the first display device being a head-mounted display device, so as to synthesize the virtual object in a real space; a first acquisition means for acquiring information about a virtual object displayed on a second display device which is a head-mounted display device different from the first display device; having The control means when a second user wearing the second display device faces a first user wearing the first display device, a specific virtual object is displayed on the first display device based on the information acquired by the first acquisition means; When the second user is not facing the first user, the specific virtual object is not displayed on the first display device.

23. An information processing apparatus comprising:

2. A first detection means for detecting a person from an image captured in the field of view of the first user; a second detection means for detecting a gaze direction of the person detected by the first detection means from the image; Further having 2. The information processing apparatus according to claim 1,

3. Further comprising a second acquisition means for acquiring position information of a head mounted display device different from the first display device, The second display device is a head-mounted display device whose position information is acquired by the second acquisition means.

2. The information processing apparatus according to claim 1,

4. The control means causes the first display device to display the specific virtual object at a position based on a position of the second user.

2. The information processing apparatus according to claim 1,

5. The control means causes the first display device to display the specific virtual object at a position overlapping an eye of the second user as seen by the first user.

2. The information processing apparatus according to claim 1,

6. The control means When the second user is gazing at a virtual object displayed on the second display device, the specific virtual object is displayed on the first display device; When the second user does not gaze at the virtual object displayed on the second display device, the specific virtual object is not displayed on the first display device.

2. The information processing apparatus according to claim 1,

7. The information includes gaze information regarding whether the second user is gazing at a virtual object displayed by the second display device.

2. The information processing apparatus according to claim 1,

8. When the second user is gazing at a virtual object, the information includes information regarding a type of the virtual object, The control means causes the first display device to display, as the specific virtual object, a virtual object that can identify the type of the virtual object that the second user is gazing at.

2. The information processing apparatus according to claim 1,

9. An information processing device according to any one of claims 1 to 8; A display means for displaying a virtual object so as to be composited into a real space; A head-mounted display device comprising:

10. a control step of displaying the virtual object on a first display device that is a head-mounted display device so as to synthesize the virtual object in a real space; an acquisition step of acquiring information about a virtual object displayed on a second display device which is a head-mounted display device different from the first display device; having In the control step, When a second user wearing the second display device faces a first user wearing the first display device, a specific virtual object is displayed on the first display device based on the information acquired by the acquiring step; When the second user is not facing the first user, the specific virtual object is not displayed on the first display device.

23. An information processing method comprising:

11. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 8.

Images