Information processing apparatus, method, and program

Abstract

The present technology pertains to an information processing device, method, and program that enable quick response of a user to be supported and risk to be reduced. An information processing device detects an intervening person from the outside with respect to a user wearing a device that blocks an external field of view and sound, and performs control to present information related to the detected intervening person to the user. The present technology can be applied to an environmental computing system.

Description

TECHNICAL FIELD

[0001] The present technology relates to an information processing apparatus, method, and program, and more particularly, to an information processing apparatus, method, and program capable of supporting a user to respond quickly and reducing risks. BACKGROUND

[0002] With the improvement of performance of a head-mounted display (HMD) and the progress of virtual reality (VR) content generation technology, the experience (viewing) of a VR space and the viewing of VR content are becoming common. At this time, although a user generally wears an HMD, since the visibility, sound, and the like of the surroundings are blocked when the user wears the HMD, the user generally does not notice the intervention (call, and the like) of an obstacle or a person in the surroundings.

[0003] Regarding the obstacle in the surroundings, for example, Patent Literature 1 proposes a technology for detecting a real obstacle around a user, replacing the obstacle with a virtual object, and displaying the virtual object on a screen of an HMD. However, Patent Literature 1 does not describe the detection of the intervention of a person.

[0004] LIST OF CITATIONS

[0005] PATENT LITERATURE

[0006] Patent Literature 1: Japanese Patent Application Laid-Open No. 2013-257716 SUMMARY

[0007] PROBLEMS TO BE SOLVED BY THE INVENTION

[0008] The above-described risk of not noticing the intervention of a person during the viewing of VR content in the case of wearing an HMD is expected to increase in the future. In addition, even in the case of noticing the intervention of a person, it takes time and effort to detach and attach the HMD or to leave the VR space to communicate with the person, for example.

[0009] The present technology is developed in view of such a situation, and aims to support a user to respond quickly and reduce risks.

[0010] SOLUTION TO PROBLEM

[0011] An information processing apparatus according to one aspect of the present technology includes a detection unit that detects an external intervenor with respect to a user of a user apparatus that wears a blockage of a field of view and sound of the outside, and a presentation control unit that performs control for presenting information about the detected intervenor to the user.

[0012] In an aspect of the present technology, an external intervenor with respect to a user of a user apparatus that wears a blockage of a field of view and sound of the outside is detected, and control for presenting information about the detected intervenor to the user is performed. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is a block diagram illustrating a configuration example of an environmental computing system according to an embodiment of the present technology.

[0014] Figure 2 is a diagram illustrating an outline of an HMD system according to a first embodiment of the present technology.

[0015] Figure 3 is a block diagram illustrating a configuration example of an HMD system in Figure 2 .

[0016] Figure 4 is a block diagram illustrating a configuration example of a detection unit in Figure 3 .

[0017] Figure 5 is a flowchart illustrating a process of an HMD system in Figure 2 .

[0018] Figure 6 is a flowchart illustrating a detection process in step S11 in Figure 5 .

[0019] Figure 7 is a diagram illustrating a modified example of an HMD system in Figure 2 .

[0020] Figure 8 is a flowchart illustrating another process of an HMD system in Figure 2 .

[0021] Figure 9 is a flowchart illustrating a detection process in step S51 in Figure 8 .

[0022] Figure 10 is a flowchart illustrating another example of a detection process in step S51 in Figure 8 .

[0023] Figure 11 is a diagram illustrating an outline of an HMD system according to a second embodiment of the present technology.

[0024] Figure 12 is a block diagram illustrating a configuration example of an HMD system in Figure 11 .

[0025] Figure 13 is a flowchart illustrating a process of an HMD system in Figure 11 .

[0026] Figure 14 is a block diagram illustrating a configuration example of an HMD system according to a third embodiment of the present technology.

[0027] Figure 15 is a diagram showing an outline of an HMD system according to a fourth embodiment of the present technology.

[0028] Figure 16 is a block diagram showing a configuration example of an HMD system in Figure 15 .

[0029] Figure 17 is a block diagram showing a configuration example of an HMD system according to a fifth embodiment of the present technology.

[0030] Figure 18 is a flowchart showing a process of an HMD system in Figure 17 .

[0031] Figure 19 is a diagram showing communication between a user and an intervener.

[0032] Figure 20 is a diagram showing a display method in a case where a video of an intervener is presented to a user.

[0033] Figure 21 is a flowchart showing another process of an HMD system in Figure 17 .

[0034] Figure 22 is a block diagram showing a configuration example of an HMD system according to a sixth embodiment of the present technology.

[0035] Figure 23 is a flowchart showing a process of an HMD system in Figure 22 .

[0036] Figure 24 is a diagram showing a method for presenting a message to a person who speaks to a user.

[0037] Figure 25 is a diagram showing an outline of an HMD system according to a seventh embodiment of the present technology.

[0038] Figure 26 is a diagram showing an outline of an HMD system according to an eighth embodiment of the present technology.

[0039] Figure 27 is a flowchart showing a process of an HMD system in Figure 26 .

[0040] Figure 28 is a flowchart showing a first control process performed by an HMD system according to an intervention level according to a ninth embodiment of the present technology.

[0041] Figure 29is a diagram showing a control method based on whether a user and an intervenor are an adult or a child.

[0042] Figure 30 is a flowchart showing a second control process performed by an HMD system according to an intervention level according to the ninth embodiment of the present technology.

[0043] Figure 31 is a diagram showing an example of a control method determination table.

[0044] Figure 32 is a diagram showing an example of a VR space viewed or experienced by a user.

[0045] Figure 33 is a block diagram showing an example of a configuration of a computer. DETAILED DESCRIPTION

[0046] A mode for implementing the present technology will be described below. The description will be given in the following order.

[0047] 1. Overall system

[0048] 2. First embodiment of the present technology (notification of intervention detection)

[0049] 3. Second embodiment of the present technology (superimposed presentation on a VR space)

[0050] 4. Third embodiment of the present technology (notification based on person recognition)

[0051] 5. Fourth embodiment of the present technology (superimposed presentation of an avatar in a VR space)

[0052] 6. Fifth embodiment of the present technology (determination of a response according to user selection)

[0053] 7. Sixth embodiment of the present technology (determination of a response by a system)

[0054] 8. Seventh embodiment of the present technology (intervention and obstacle detection)

[0055] 9. Eighth embodiment of the present technology (sharing of a VR space)

[0056] 10. Ninth embodiment of the present technology (control according to intervention level)

[0057] 11. Others

[0058] <1. Overall system>

[0059] <Configuration of system>

[0060] Figure 1 is a diagram showing an example of a configuration of an environment computing system according to an embodiment of the present technology.

[0061] Figure 1 The environment computing system 1 is a system that detects information about a situation or a peripheral environment in which a user is located, controls each of the devices according to the detected information and previous information, and provides a service desired by the user.

[0062] While providing such a service, as a function according to the present technology, the environment computing system 1 provides, for example, a service that detects a human intervention for a user who is watching VR content (hereinafter, referred to as a VR watching period) and controls an HMD and presents information according to the detected human intervention.

[0063] The environment computing system 1 includes a user interface 21, an environment previous information obtaining unit 22, a user previous information obtaining unit 23, a collaborative sensing unit 24, a peripheral environment detecting unit 25, a user state detecting unit 26, a user response detecting unit 27, a context management unit 28, a device control unit 29, an environment context model storage unit 30, a user context model storage unit 31, and a service list storage unit 32.

[0064] Here, in the present specification, a context is information including user information, environment information, and a relationship between these information. Further, a context model is a model used when the context management unit 28 is trained, and is also a model (i.e., an algorithm or a model of machine learning) in which the context management unit 28 constructs a service personalized for a user.

[0065] The user interface 21 has an interface function for a user such as providing a notification to the user and receiving an instruction from the user.

[0066] The environment computing system 1 has two paths for obtaining information.

[0067] The first path is a path for obtaining previous information in advance based on a user input. The first path is implemented by the environment previous information obtaining unit 22 and the user previous information obtaining unit 23.

[0068] The environment previous information obtaining unit 22 obtains previous information about an environment based on a user input, and outputs the obtained environment previous information as environment information to the context management unit 28.

[0069] As a function according to the present technology, in particular, the environment previous information obtaining unit 22 outputs, for example, information about an identification database for identifying a face and a voice of an intervening person (hereinafter, also referred to as an intervener) as environment information to the context management unit 28.

[0070] The user previous information obtaining unit 23 obtains previous information about the user based on user input, and outputs the obtained user previous information as user information to the context management unit 28.

[0071] As a function according to the present technology, in particular, the user previous information obtaining unit 23 outputs, for example, information indicating a type of viewing content viewed by the user as user information to the context management unit 28.

[0072] The second path is a path for detecting current state information about the environment and the user (in the case of the present technology, a state in which the user is viewing VR) by an identification device that cooperates with a plurality of sensing devices such as the Internet of Things (IoT) to perform sensing, process sensing information, and is also capable of multi-modal processing. Multi-modal processing refers to processing a plurality of types of information at a time. The second path is implemented by the cooperative sensing unit 24 as the identification device, the surrounding environment detection unit 25, the user state detection unit 26, and the user response detection unit 27.

[0073] The cooperative sensing unit 24 obtains sensing information from a plurality of sensing devices, and performs predetermined signal processing for converting the obtained information into a format that is easy to process in subsequent processing (for example, removing noise components or performing data cleaning for synchronizing data). The cooperative sensing unit 24 outputs the information after signal processing to the detection units corresponding to the information among the surrounding environment detection unit 25, the user state detection unit 26, and the user response detection unit 27.

[0074] The surrounding environment detection unit 25 detects surrounding information such as images, temperatures, and sounds from the information supplied from the cooperative sensing unit 24, and outputs information about the environment obtained as a result (hereinafter referred to as environment information) to the context management unit 28.

[0075] As a function according to the present technology, the surrounding environment detection unit 25 detects human intervention such as viewing, approaching, speaking, and the like of a person, and outputs environment information obtained as a result to the context management unit 28.

[0076] The user state detection unit 26 detects information about the user from the information supplied from the cooperative sensing unit 24. For example, the user state detection unit 26 performs personal identification of the user by using face recognition of an RGB image, detects the user's action based on information about skeleton detection and an environmental map, and processes life information such as expressions obtained from an RGB camera and body temperature obtained from a wearable terminal in a multi-modal manner to detect emotions such as the user's comfort and discomfort.

[0077] The user state detection unit 26 outputs user-related information (hereinafter referred to as user information) obtained as a result of detection to the context management unit 28.

[0078] As a function according to the present technology, in particular, the user state detection unit 26 detects, for example, that the user is watching VR, and outputs user information obtained as a result to the context management unit 28.

[0079] The user response detection unit 27 detects a user's response to a service provided by the context management unit 28 from information supplied from the cooperative sensing unit 24, and outputs response information and feedback information related to the user obtained as a result of detection to the context management unit 28.

[0080] As a function according to the present technology, in particular, the user response detection unit 27 detects the user's permission, selection, and the like of a service (such as a notification) provided by the context management unit 28, and outputs response information and feedback information related to the user obtained as a result of detection to the context management unit 28.

[0081] The context management unit 28 accumulates information supplied from the surrounding environment detection unit 25, the user state detection unit 26, and the user response detection unit 27, thereby learning a model related to an environment as an environmental context model, which indicates a specific environment in which a specific region is likely to be in a specific time zone. The context management unit 28 stores the learned environmental context model in the environmental context model storage unit 30.

[0082] The context management unit 28 accumulates information supplied from the surrounding environment detection unit 25, the user state detection unit 26, and the user response detection unit 27, thereby learning a model related to a user's habitual behavior pattern as a user context model, which indicates a specific situation in which the user performs a specific action. The context management unit 28 stores the learned user context model in the user context model storage unit 31.

[0083] In a case where information indicating a similar situation is obtained after each context model is learned, the context management unit 28 controls the device control unit 29 to provide a personalized service preempting a user's action using the user context model from a device that cooperates with the environmental computing system 1.

[0084] The context management unit 28 receives a user's response when providing a personalized service via the user response detection unit 27 to obtain feedback information indicating whether the service is actually appropriate or not, and relearns each context model to improve the accuracy of the context model.

[0085] Further, by accumulating learning of the context model of habitual behavior patterns, the context management unit 28 can model the behavioral criteria of the user to tend to select a particular option in a particular situation, and be able to implement a personalized service in advance for the user even if the user does not habitually perform some action.

[0086] As a function according to the present technology, particularly, for example, the context management unit 28 controls the device control unit 29 in such a manner as to provide a personalized service for notification, presentation, inquiry, etc. to the user from the HMD that is one of the devices that cooperate with the environment computing system 1.

[0087] Under the control of the context management unit 28, the device control unit 29 causes the device (HMD in the case of the present technology) that cooperates with the environment computing system 1 to provide a personalized service.

[0088] The environment context model storage unit 30 stores the environment context model. In addition, the environment context model storage unit 30 stores information used when learning the environment context model and information including relationships as the environment context.

[0089] The user context model storage unit 31 stores the user context model. In addition, the user context model storage unit 31 stores information used when learning the user context model and information including relationships as the user context.

[0090] The service list storage unit 32 stores a list of devices that execute a service that can be provided to the user.

[0091] Details of the function in the present technology in the environment computing system 1 will be described below. Figure 1

[0092] <2. First embodiment of the present technology (notification of intervention detection)>

[0093]

[0094] Figure 2 is a diagram that shows an outline of the HMD system according to the first embodiment of the present technology.

[0095] Figure 2 shows the state of the user of the HMD system 51 having the function in the present technology in the environment computing system 1 of Figure 1

[0096] For example, another person (intervener; Figure 2 a child in) says "Hey, hey, dad" from the outside to the user who wears the HMD 61 and watches VR.

[0097] ​​The HMD system 51 detects that the child is speaking to the user using the camera 62 and the microphone 71 (mentioned later) provided in the HMD 61, presents the detection result to the HMD 61, and notifies the user of the detection result. Figure 3

[0098] Accordingly, even if the user is immersed in the VR space, the user can immediately notice that the child is speaking to the user.

[0099] <Configuration of HMD system>

[0100] Figure 3 is a block diagram showing a configuration example of the HMD system 51 in Figure 2

[0101] The HMD system 51 includes the camera 62, the microphone 71, the detection unit 72, the notification information generation unit 73, and the information presentation control unit 74 shown in Figure 2

[0102] In addition, the camera 62, the microphone 71, and the detection unit 72 correspond to the cooperative sensing unit 24, the surrounding environment detection unit 25, the user state detection unit 26, and the user response detection unit 27 in Figure 1 Figure 1 The notification information generation unit 73 corresponds to the context management unit 28 in Figure 1 The information presentation control unit 74 corresponds to the device control unit 29 in

[0103] The camera 62 is an imaging unit that images the surroundings, and is provided in the HMD 61 worn by the user. The camera 62 outputs video data generated by imaging the surroundings of the user to the detection unit 72.

[0104] The microphone 71 is a sound collecting unit, and is provided in the HMD 61 worn by the user. The microphone 71 outputs sound data generated by collecting the sound of the surroundings of the user to the detection unit 72.

[0105] The detection unit 72 detects an external person intervention (presence of an intervener) from the outside based on the video data supplied from the camera 62 and the sound data supplied from the microphone 71, and supplies the detection result to the notification information generation unit 73. The person intervention includes, for example, intervention by sound when spoken to by a person, intervention by a line of sight when looked at by a person, intervention by a motion when approached by a person, and the like.

[0106] The notification information generation unit 73 generates notification information for notifying the detection result supplied from the detection unit 72, and outputs the generated notification information to the information presentation control unit 74. ​​​​

[0107] The information presentation control unit 74 causes a presentation unit (not shown) included in the HMD 61 to present the notification information supplied from the notification information generation unit 73.

[0108] At this time, as a means for notification, it is possible to envisage "display on a video being watched by the user", "notification by sound", and the like, but any method can be used. Therefore, the notification information generated by the notification information generation unit 73 is video data in the case of video notification, and sound data in the case of sound notification.

[0109] [Configuration of detection unit]

[0110] Figure 4 is a block diagram showing a configuration example of the detection unit 72 in Figure 3 .

[0111] In Figure 4 , the detection unit 72 includes a person detection section 81, a motion determination section 82, an utterance portion detection section 83, a speaker recognition section 84, a speaker data database (DB) 85, and an intervention detection section 86.

[0112] The person detection section 81 detects a person on the basis of video data supplied from the imaging device 62. The person detection section 81 outputs information about the detected person to the motion determination section 82.

[0113] The motion determination section 82 determines a face direction on the basis of the information about the person supplied from the person detection section 81, and outputs a determination result of the face direction to the intervention detection section 86. The motion determination section 82 also determines whether or not the size of the person in the video is becoming larger on the basis of the information about the person supplied from the person detection section 81, and outputs a result of determining whether or not the size of the person is becoming larger to the intervention detection section 86.

[0114] The utterance portion detection section 83 detects an utterance portion on the basis of sound data supplied from the microphone 71. The utterance portion detection section 83 outputs information indicating the detected utterance portion to the speaker recognition section 84.

[0115] The speaker recognition section 84 determines whether or not the person (hereinafter also referred to as a speaker) who is speaking is a person other than the user, on the basis of the sound data in the utterance portion indicated by the information supplied from the utterance portion detection section 83 and the speaker data registered in the speaker data DB 85. The speaker recognition section 84 outputs information about the determined speaker to the intervention detection section 86.

[0116] Based on the determination result of the face direction supplied from the movement determination section 82, the speaker information supplied from the speaker recognition section 84, and the like, the intervention detection section 86 detects a person intervention such as a person talking to the user, a person looking at the user, or a person approaching the user. The intervention detection section 86 outputs the detection result of the person intervention to the notification information generation unit 73.

[0117] <Process of HMD system>

[0118] Figure 5 is a flowchart showing the process of the HMD system 51 in Figure 2 . Figure 5 The process of the HMD system 51 is executed for each frame or for each frame at a predetermined interval. Note that other processes of the HMD system described later are also executed for each frame or for each frame at a predetermined interval.

[0119] For example, a child says "Hey, hey, dad" to a user who wears the HMD 61 and watches VR.

[0120] In step S11, the camera 62, the microphone 71, and the detection unit 72 detect that a person talks to the user.

[0121] That is, the camera 62 and the microphone 71 provided in the HMD 61 sense the situation around the user, and output video data and sound data to the detection unit 72, respectively. The detection unit 72 detects that a person talks to the user based on the video data supplied from the camera 62 and the sound data supplied from the microphone 71, and supplies the detection result to the notification information generation unit 73. Note that details of the detection process in step S11 will be described later with reference to Figure 6 .

[0122] In step S12, the notification information generation unit 73 generates notification information for notifying the user that a person talks to the user as the detection result supplied from the detection unit 72, and outputs the generated notification information to the information presentation control unit 74.

[0123] In step S13, the information presentation control unit 74 causes the presentation unit included in the HMD 61 to display the notification information supplied from the notification information generation unit 73, thereby notifying the user that a person talks to the user. Then, the process ends.

[0124] <Detection process>

[0125] Figure 6 is a flowchart showing details of the detection process in step S11 in Figure 5 .

[0126] In step S31, the imaging device 52 and the microphone 71 sense the surrounding situation. The imaging device 62 outputs video data generated by imaging the surroundings of the user to the detection unit 72. The microphone 71 outputs sound data generated by collecting the sound around the user to the detection unit 72.

[0127] In step S32, the person detection section 81 determines whether a person is detected in the video indicated by the video data supplied from the imaging device 62. In a case where it is determined in step S32 that no person is detected in the video, the processing returns to step S31, and the subsequent processing is repeated.

[0128] In a case where it is determined in step S32 that a person is detected in the video, the processing proceeds to step S33.

[0129] In step S33, the motion determination section 82 determines whether the face of the person in the video is oriented toward the user, based on the information on the person supplied from the person detection section 81. In a case where it is determined in step S33 that the face of the person is not oriented toward the user, the processing returns to step S31, and the subsequent processing is repeated.

[0130] In a case where it is determined in step S33 that the face of the person is oriented toward the user, the processing proceeds to step S34.

[0131] The utterance section detection section 83 detects an utterance section from the collected sound, based on the sound data supplied from the microphone 71.

[0132] In step S34, the utterance section detection section 83 determines whether a person's voice is detected from the collected sound, based on the sound data supplied from the microphone 71. In a case where it is determined in step S34 that no person's voice is detected from the collected sound, the processing returns to step S31, and the subsequent processing is repeated.

[0133] In a case where it is determined in step S34 that a person's voice is detected from the collected sound, the processing proceeds to step S35.

[0134] In step S35, the speaker identification section 84 determines whether the speaker is a person other than the user, based on the sound data in the utterance section indicated by the information supplied from the utterance section detection section 83 and the speaker data registered in the speaker data DB 85. In a case where it is determined that the speaker is the user, the processing returns to step S31, and the subsequent processing is repeated.

[0135] In a case where it is determined in step S35 that the speaker is a person other than the user, the processing proceeds to step S36.

[0136] In step S36, the intervention detection section 86 determines "person speaking", and outputs a detection result indicating that person speaking is detected. Then, the processing returns toFigure 5 Step S11 in FIG. 10.

[0137] Note that, although the example in which sensing is performed by the imaging device 62 and the microphone 71 provided in the HMD 61 is described in the above description, in the case of viewing VR at a specific place, sensing can also be performed by an external imaging device and an external microphone.

[0138] [Modified Example of HMD System]

[0139] Figure 7 is a diagram illustrating a modified example of the HMD system in FIG. 10. Figure 2

[0140] Figure 7 An example in which a child speaks to a user who is viewing VR using the HMD system 51 at a specific place such as a room is illustrated.

[0141] Figure 7 The HMD system 51 in FIG. 11 is different from the HMD system 51 in FIG. 10 in that the imaging device 62 and the microphone 71 are replaced with an imaging device 91 and a microphone 92. Figure 2

[0142] The imaging device 91 and the microphone 92 are not provided in the HMD 61 but are provided externally, for example, in a room or the like.

[0143] That is, the imaging device 91 is configured similarly to the imaging device 62 except for the aspect of the installation position. The microphone 92 is configured similarly to the microphone 71 except for the aspect of the installation position.

[0144] As described above, in the case where the user is always viewing VR at a specific place, the external imaging device 91 and the microphone 92 can perform sensing.

[0145] Note that, although the example in which a person speaking to the user is detected has been described with reference to Figure 5 , a person looking at or approaching the user can also be detected.

[0146] [Another Process of HMD System]

[0147] Figure 8 is a flowchart illustrating another process of the HMD system 51 in FIG. 10. Figure 2 For example, a person approaches or looks at a user who wears the HMD 61 and views VR.

[0148] In step S51, the imaging device 62, the microphone 71, and the detection unit 72 detect that a person approaches or looks at the user.

[0149] In step S51, the imaging device 62, the microphone 71, and the detection unit 72 detect that a person approaches or looks at the user.​​

[0150] That is, the imaging device 62 and the microphone 71 provided in the HMD 61 sense the situation around the user, and respectively output video data and sound data to the detection unit 72. The detection unit 72 detects that a person is approaching or looking at the user on the basis of the video data supplied from the imaging device 62 and the sound data supplied from the microphone 71, and supplies the detection result to the notification information generation unit 73. Note that details of the detection processing in step S51 will be described later with reference to Figure 9 and Figure 10 Details of the detection processing in step S51 will be described.

[0151] In step S52, the notification information generation unit 73 generates notification information for notifying the user that a person is approaching or looking at the user as the detection result supplied from the detection unit 72, and outputs the generated notification information to the information presentation control unit 74.

[0152] In step S53, the information presentation control unit 74 causes the presentation unit included in the HMD 61 to display the notification information supplied from the notification information generation unit 73, thereby notifying the user that a person is approaching or looking at the user. Then, the processing ends.

[0153] <Detection Processing>

[0154] Figure 9 is a flowchart showing details of the detection processing in step S51 in Figure 8

[0155] Figure 9 A process for detecting that a person is approaching the user is shown.

[0156] In step S71, the imaging device 62 and the microphone 71 sense the situation around. The imaging device 62 outputs video data generated by imaging the surroundings of the user to the detection unit 72. The microphone 71 outputs sound data generated by collecting the sound around the user to the detection unit 72.

[0157] In step S72, the person detection section 81 determines whether a person is detected in the video indicated by the video data supplied from the imaging device 62. In a case where it is determined in step S72 that no person is detected in the video, the processing returns to step S71, and the subsequent processing is repeated.

[0158] In a case where it is determined in step S72 that a person is detected in the video, the processing proceeds to step S73.

[0159] ​In step S73, the motion determination section 82 determines whether the face of the person is oriented toward the user on the basis of the information about the person supplied from the person detection section 81. In a case where it is determined in step S73 on the basis of the information about the person that the face of the person is not oriented toward the user, the process returns to step S71, and the subsequent process is repeated.

[0160] In a case where it is determined in step S73 that the face of the person is oriented toward the user, the process proceeds to step S74.

[0161] In step S74, the motion determination section 82 determines whether the size of the person in the video is becoming larger on the basis of the information about the person supplied from the person detection section 81. In a case where it is determined in step S74 that the size of the person in the video is not becoming larger, the process returns to step S71, and the subsequent process is repeated.

[0162] In a case where it is determined in step S74 that the size of the person in the video is becoming larger, the process proceeds to step S75.

[0163] In step S75, the intervention detection section 86 determines that "a person is approaching" and outputs a detection result indicating that a person is detected to be approaching. Then, the process returns to step S51 in Figure 8

[0164] <DETECTION PROCESS>

[0165] Figure 10 is a flowchart showing another example of the detection process in step S51 in Figure 8

[0166] Figure 10 shows a process for detecting that a person is looking at the user.

[0167] In step S91, the imaging device 52 and the microphone 71 sense the surrounding situation. The imaging device 62 outputs video data generated by imaging the surroundings of the user to the detection unit 72. The microphone 71 outputs sound data generated by collecting the sound around the user to the detection unit 72.

[0168] In step S92, the person detection section 81 determines whether a person is detected in the video indicated by the video data supplied from the imaging device 62. In a case where it is determined in step S92 that a person is not detected in the video, the process returns to step S91, and the subsequent process is repeated.

[0169] In a case where it is determined in step S92 that a person is detected in the video, the process proceeds to step S93.

[0170] ​​In step S93, the motion determination section 82 determines whether the face of the person is oriented toward the user on the basis of the information about the person supplied from the person detection section 81. In the case where it is determined in step S93 on the basis of the information about the person that the face of the person is not oriented toward the user, the process returns to step S91, and the subsequent process is repeated.

[0171] In the case where it is determined in step S93 that the face of the person is oriented toward the user, the process proceeds to step S94.

[0172] In step S94, the intervention detection section 86 determines that "the person is looking in this direction", and outputs a detection result indicating that the person is detected to be looking in this direction. Then, the process returns to step S51 in Figure 8 .

[0173] Note that, since the sound data is not used in Figure 9 and Figure 10 , the HMD system 51 can have a configuration without the microphone 71.

[0174] <3. Second embodiment of the present technology (superimposition presentation on VR space)>

[0175] <Outline of HMD system>

[0176] Figure 11 is a diagram showing an outline of an HMD system according to the second embodiment of the present technology.

[0177] Figure 11 shows the state of a user of an HMD system 101 having the functions in the present technology in the environment computing system 1 in Figure 1 .

[0178] For example, another person (an intervener; Figure 11 a child) says "Hey, hey, daddy" to the user wearing the HMD 61 and watching VR from the outside.

[0179] The HMD system 101 not only detects that the child speaks to the user and notifies the user of the detection result by using the camera 62 and the microphone 71 Figure 12 ) provided in the HMD 61, but also projects the obtained video data and sound data to the VR space 111 displayed on the presentation unit of the HMD 61 to present the video data and the sound data in the VR space 111.

[0180] As described above, in the HMD system 101 in Figure 11 , the video and the sound of the person speaking to the user are superimposed and presented on the data about the VR space 111 watched by the user. Therefore, the user can continuously interact with the person speaking to him or her without leaving the VR space 111.

[0181] <Configuration of HMD system>

[0182] Figure 12 is a block diagram showing a configuration example of the HMD system 101 in Figure 11

[0183] The HMD system 101 differs from the HMD system 51 in Figure 3 by replacing the notification information generating unit 73 with a notification information superimposing unit 131. Note that the notification information superimposing unit 131 corresponds to the context management unit 28 in Figure 1

[0184] The detection unit 72 supplies the video data supplied from the imaging device 62, the sound data supplied from the microphone 71, and the detection result of the intervention to the notification information superimposing unit 131.

[0185] The notification information superimposing unit 131 superimposes the video data related to the intervener on the video data related to the VR space 111, superimposes (synthesizes) the sound data on the sound data related to the VR space 111, and outputs the superimposed data (referred to as superimposed data) to the information presentation control unit 74, on the basis of the video data, the sound data, and the detection result supplied from the detection unit 72. Note that the notification information superimposing unit 131 can generate notification information for notifying the detection result of the intervention supplied from the detection unit 72, and can also superimpose the generated notification information on the data related to the VR space 111.

[0186] The information presentation control unit 74 displays the superimposed data supplied from the notification information superimposing unit 131 in the VR space 111 of the presentation unit included in the HMD 61.

[0187] <Processing of HMD system>

[0188] Figure 13 is a flowchart showing the processing of the HMD system 101 in Figure 11 For example, a child says "Hey, hey, dad" to a user who wears the HMD 61 and watches VR.

[0189] In step S111, the imaging device 62, the microphone 71, and the detection unit 72 detect that a person speaks to the user, as described above with reference to

[0190] Figure 6

[0191] ​​​​That is, the imaging device 62 and the microphone 71 provided in the HMD 61 sense the situation around the user, and output video data and sound data, respectively, to the detection unit 72. The detection unit 72 detects that a person is speaking to the user on the basis of the video data supplied from the imaging device 62 and the sound data supplied from the microphone 71, and supplies the video data, the sound data, and the detection result to the notification information superimposition unit 131.

[0192] In step S112, the notification information superimposition unit 131 superimposes the video data and the sound data related to the intervenor on the data related to the VR space 111 on the basis of the detection result supplied from the detection unit 72, and outputs the superimposed data to the information presentation control unit 74.

[0193] In step S113, the information presentation control unit 74 causes the presentation unit included in the HMD 61 to present the superimposed data supplied from the notification information superimposition unit 131 in the VR space 111, thereby notifying the user that a person is speaking to the user. Then, the process ends.

[0194] <4. Third embodiment of the present technology (notification based on person recognition)>

[0195] <Configuration of HMD system>

[0196] Figure 14 is a block diagram illustrating a configuration example of an HMD system according to the third embodiment of the present technology.

[0197] Figure 14 A configuration example of the HMD system 151 having the functions in the present technology in the environment computing system 1 of Figure 1 is shown.

[0198] Figure 14 The HMD system 151 of Figure 3 differs from the HMD system 51 of Figure 1 in that a recognition unit 161 and a recognition database 162 are added. Note that the recognition unit 161 and the recognition database 162 correspond to the context management unit 28 in

[0199] The detection unit 72 supplies the video data supplied from the imaging device 62, the sound data supplied from the microphone 71, and the detection result of the intervention to the recognition unit 161.

[0200] The recognition unit 161 refers to the detection result of the intervention supplied from the detection unit 72 by using the information registered in the recognition database 162, and recognizes who the person speaking to the user is from the video data.

[0201] Note that the method for identifying who the person is can be face recognition using video data or voice recognition using sound. In either case, the recognition unit 161 performs recognition by, for example, referring to an identification database 162 in which user information for identifying the face and voice of the person is registered in advance, and returning the ID of the person if the person is included in the identification database, or returning a non-conclusive result if the person is not included.

[0202] The recognition unit 161 outputs the video data and sound data supplied from the detection unit 72 and the recognition result of the person to the notification information generation unit 73.

[0203] In the identification database 162, user information for identifying the face and voice of the person is registered in advance.

[0204] The notification information generation unit 73 generates notification information for notifying of the recognition result supplied from the recognition unit 161, and outputs the generated notification information to the information presentation control unit 74.

[0205] <5. Fourth Embodiment of the Present Technology (Superimposed Presentation of Avatar in VR Space)>

[0206] <Outline of HMD System>

[0207] Figure 15 is a diagram showing an outline of an HMD system according to the fourth embodiment of the present technology.

[0208] Figure 15 shows the state of the user of the HMD system 181 having the functions in the present technology in the environment computing system 1 using in Figure 1

[0209] For example, the other person (intervener; Figure 15 a child in the house) speaks "Hey, hey, daddy" from the outside to the user wearing the HMD 61 and watching VR.

[0210] The HMD system 181 not only detects that the child speaks to the user and notifies the user of the detection result using the camera 62 and the microphone 71 Figure 16 ) provided in the HMD 61, but also generates an avatar 190 from the obtained video data, and presents the generated avatar 190 in the VR space 111.

[0211] Unlike the HMD system 101 in Figure 11 in which the video and sound of the person speaking to the user are superimposed and presented on the data related to the VR space 111 watched by the user, in Figure 15 ​In the HMD system 181, the avatar 190 generated from the video of the person who spoke to the user is superimposed and presented on the data related to the VR space 111 that the user views.

[0212] <Configuration of HMD system>

[0213] Figure 16 is a block diagram illustrating a configuration example of the HMD system 181 in Figure 15

[0214] The HMD system 181 differs from the HMD system 51 in Figure 3 Figure 1 in that a skeleton data extraction unit 191 is added, and the notification information generation unit 73 is replaced with an avatar generation and three-dimensional information reconstruction unit 192. Note that the skeleton data extraction unit 191 and the avatar generation and three-dimensional information reconstruction unit 192 correspond to the context management unit 28 in

[0215] The detection unit 72 supplies the video data supplied from the imaging device 62, the sound data supplied from the microphone 71, and the detection result of the intervention to the skeleton data extraction unit 191.

[0216] The skeleton data extraction unit 191 extracts human skeleton data from the video data supplied from the detection unit 72, and outputs the video data, the sound data, and the skeleton data to the avatar generation and three-dimensional information reconstruction unit 192.

[0217] The avatar generation and three-dimensional information reconstruction unit 192 generates the avatar 190 based on the skeleton data from the skeleton data extraction unit 191, reconstructs the avatar 190 into three-dimensional information, superimposes the avatar 190 on the data related to the VR space 111, and outputs the resulting superimposed data to the information presentation control unit 74.

[0218] The information presentation control unit 74 causes the presentation unit included in the HMD 61 to display the superimposed data supplied from the avatar generation and three-dimensional information reconstruction unit 192.

[0219] <6. Fifth embodiment of the present technology (determination of response according to user selection)>

[0220] <Configuration of HMD system>

[0221] Figure 17 is a block diagram illustrating a configuration example of the HMD system according to the fifth embodiment of the present technology.

[0222] Figure 17 A configuration example of the HMD system 201 having the functions in the present technology in the environment computing system 1 in Figure 1 is shown. ​​

[0223] Figure 17 The HMD system 201 of Figure 14 differs from the HMD system 151 of Figure 1 in that a detection result presentation and inquiry unit 211 is added, and the notification information generation unit 73 is replaced with an extracted information superimposition unit 212. Note that the detection result presentation and inquiry unit 211 and the extracted information superimposition unit 212 correspond to the context management unit 28 in

[0224] That is, the HMD system 201 of Figure 17 notifies the user that an intervenor is detected, inquires the user about a response to the intervenor, and determines whether or not to respond to the intervenor in accordance with the user's selection.

[0225] The recognition unit 161 outputs the video data and the sound data supplied from the detection unit 72 and the recognition result of the person to the detection result presentation and inquiry unit 211. Note that in the case where recognition is not performed, the recognition unit 161 outputs the video data and the sound data supplied from the detection unit 72 and the detection result of the intervention to the detection result presentation and inquiry unit 211.

[0226] The detection result presentation and inquiry unit 211 generates notification information for notifying the recognition result of the person supplied from the recognition unit 161, and causes a presentation unit included in the HMD 61 to present the generated notification information and an inquiry about whether or not to display a person who speaks to the user in the VR space.

[0227] The detection result presentation and inquiry unit 211 receives the user's selection (agreement or disagreement) via an operation unit (not shown) or the like. In the case where the user agrees, the detection result presentation and inquiry unit 211 outputs the video data and the sound data supplied from the detection unit 72 and the recognition result of the person (or the detection result of the intervention) to the extracted information superimposition unit 212.

[0228] The extracted information superimposition unit 212 extracts video and sound data related to the intervenor from the video data and the sound data supplied from the detection result presentation and inquiry unit 211. The extracted information superimposition unit 212 superimposes the extracted video and sound data related to the intervenor on data related to the VR space 111, and outputs the superimposed data to the information presentation control unit 74.

[0229] <Processing of HMD system>

[0230] Figure 18 is a flowchart showing the processing of the HMD system 201 in Figure 17

[0231] For example, a child says "Hey, hey, daddy" to a user who wears the HMD 61 and watches VR.​

[0232] In step S201, the imaging device 62, the microphone 71, and the detection unit 72 detect that a person is talking to the user, as described above with reference to Figure 6 .

[0233] In step S202, the recognition unit 161 refers to the detection result of the intervention supplied from the detection unit 72 by using the information registered in the recognition database 162, and recognizes who the person talking to the user is from the video data.

[0234] In step S203, the detection result presentation and inquiry unit 211 generates notification information for notifying the recognition result of the person supplied from the recognition unit 161, and causes the presentation unit included in the HMD 61 to present the generated notification information and an inquiry about whether or not to display the person talking to the user in the VR space.

[0235] For example, the question "Q is talking to you. Display in the VR space?" is presented, and the user selects yes or no.

[0236] In step S204, the detection result presentation and inquiry unit 211 determines whether or not to display the intervener in the VR space according to the user's selection. In a case where it is determined in step S204 not to display the intervener in the VR space, Figure 17 the processing of the HMD system 201 ends.

[0237] In a case where it is determined in step S204 to display the intervener in the VR space, the processing proceeds to step S205. At this time, the detection result presentation and inquiry unit 211 outputs the video data, the sound data, and the recognition result of the person supplied from the recognition unit 161 to the extracted information superimposition unit 212.

[0238] In step S205, the extracted information superimposition unit 212 extracts the video and sound data related to the intervener from the video data and the sound data supplied from the detection result presentation and inquiry unit 211, superimposes the extracted video and sound data related to the intervener on the data related to the VR space 111, and outputs the superimposed data to the information presentation control unit 74.

[0239] In step S206, the information presentation control unit 74 causes the presentation unit included in the HMD 61 to present the superimposed data supplied from the extracted information superimposition unit 212 in the VR space 111, thereby presenting the video and the sound of the intervener to the user. Then, the processing ends.

[0240] Accordingly, the user can continuously interact with the person talking to him or her without leaving the VR space.

[0241] <Example of communication>

[0242] Figure 19 is a diagram showing communication between a user and an intervener.

[0243] Figure 19 A of shows "real" communication from the perspective of an intervener who speaks to a user.

[0244] The intervener who speaks to the user has a conversation in real space with the user who wears the HMD 61.

[0245] Figure 19 B of shows "virtual" communication from the perspective of a user who is spoken to.

[0246] The user who is spoken to has a conversation with the intervener projected in the VR space 111.

[0247] <Example of presentation method>

[0248] Figure 20 is a diagram showing a display method in a case where a video of an intervener is presented to a user.

[0249] In Figure 20 , two display methods are shown.

[0250] As shown in A of Figure 20 , the HMD system 201 reconstructs a virtual image (avatar) 232 of an intervener captured by an omnidirectional camera 231 provided in the HMD 61 into three-dimensional information, and superimposes and displays the virtual image 232 in the VR space 111 where the user is located.

[0251] In this case, the virtual image 232 of the intervener can be provided in the VR space 111 regardless of the position of the intervener.

[0252] As shown in B of Figure 20 , the HMD system 201 performs perspective display of a video of an intervener captured by a front-facing camera 241 provided in the HMD 61. In the perspective display, for example, the VR space 111 is displayed with a transmittance of 50%, and the video of the intervener is displayed with a transmittance of 50%.

[0253] In this case, the user can see the intervener in the real space at the actual position in the real space while being in the VR space 111.

[0254] <Another process of HMD system>

[0255] Figure 21 is a flowchart showing another process of the HMD system 201 in Figure 17

[0256] ​In step S221, the imaging device 52 and the microphone 71 sense the surrounding situation.

[0257] In step S222, the detection unit 72 determines whether or not the intervention (speaking, looking, or approaching) of a person in the vicinity has been detected on the basis of the video data supplied from the imaging device 62. In the case where it is determined in step S222 that the intervention of a person in the vicinity has not been detected, the processing returns to step S221, and the subsequent processing is repeated.

[0258] In the case where it is determined in step S222 that the intervention of a person in the vicinity has been detected, the processing proceeds to step S223.

[0259] In step S223, the detection unit 72 determines whether or not the person is speaking to the user. In the case where it is determined in step S223 that the person is speaking to the user, the processing proceeds to step S224.

[0260] In step S224, the detection result presentation and inquiry unit 211 generates notification information for notifying of the detection result (the person speaking to the user) supplied from the detection unit 72 via the recognition unit 161 and an inquiry about whether or not to respond to the intervenor (the person speaking to the user), and causes the presentation unit of the HMD 61 to present the inquiry.

[0261] In step S225, the detection result presentation and inquiry unit 211 determines whether or not to respond to the intervenor in accordance with the selection of the user about the presentation. In the case where it is determined in step S225 not to respond to the intervenor, the processing returns to step S221, and the subsequent processing is repeated.

[0262] In the case where it is determined in step S225 to respond to the intervenor, the processing proceeds to step S226.

[0263] In step S226, the extracted information superimposition unit 212 extracts the voice of the intervenor from the sound data, and the information presentation unit 74, for example, presents the voice to the presentation unit (of the VR space). Note that the video of the intervenor can also be presented as needed. Then, the processing ends.

[0264] On the other hand, in the case where it is determined in step S223 that the person is not speaking to the user, that is, in the case where it is determined in step S223 that the person is looking at or approaching the user, the processing proceeds to step S227.

[0265] In step S227, the detection result presentation and inquiry unit 211 generates notification information for notifying of the detection result (the person looking at or approaching the user) supplied from the detection unit 72 via the recognition unit 161 and an inquiry about whether or not to respond to the intervenor (the person looking at or approaching the user), and causes the presentation unit of the HMD 61 to present the notification information and the inquiry.

[0266] In step S228, the result of the detection is presented, and the inquiry unit 211 determines whether to respond to the intervenor according to the user's selection regarding the presentation. In a case where it is determined in step S228 not to respond to the intervenor, the processing returns to step S221, and the subsequent processing is repeated.

[0267] In a case where it is determined in step S228 to respond to the intervenor, the processing proceeds to step S229.

[0268] In step S229, the information superimposition unit 212 extracts the video of the intervenor from the video data, and the information presentation unit 74 presents the video to the presentation unit (of the VR space), for example. Note that the sound of the intervenor can also be presented as needed. Then, the processing ends.

[0269] Note that the timing at which the user is notified can be changed according to the state of the intervenor (speaking, looking, or approaching).

[0270] For example, in a case where a person speaks to the user, the user is notified at the timing at which the person speaks to the user.

[0271] In this case, for example, the sound obtained by the microphone 71 is reproduced as is in the HMD 61. In this way, the user can continue the conversation with the person who spoke to him or her without removing the HMD 61. Further, in some cases, the video of the person who spoke to the user can be presented.

[0272] On the other hand, in a case where a person is approaching or looking at the user, the user is notified at the timing of the approach or the looking.

[0273] In this case, for example, the video obtained by the imaging device 62 is reproduced as is in the HMD 61. In this way, the user can recognize the intervenor (the person who is approaching or looking at the user) without removing the HMD 61. In addition, in a case where the intervenor speaks, the sound of the intervenor can be reproduced in the HMD 61.

[0274] <7. Sixth Embodiment of the Present Technology (Response Determination of System)>

[0275] <Configuration of HMD System>

[0276] Figure 22 is a block diagram illustrating a configuration example of an HMD system according to the sixth embodiment of the present technology.

[0277] Figure 22 A configuration example of an HMD system 251 having the functions in the present technology in an environment computing system 1 of Figure 1 is illustrated.

[0278] Figure 22 The HMD system 251 and Figure 17 The difference between the HMD system 201 is that the detection result presentation and inquiry unit 211 is replaced by the registration data verification unit 261, and the registration data storage unit 262 is added. Note that the registration data verification unit 261 and the registration data storage unit 262 correspond to Figure 1 The context management unit 28 in

[0279] That is to say, Figure 22 The HMD system 251 and Figure 17 The difference between the HMD system 201 is that the HMD system 251 determines whether to respond to an intervener, rather than determining whether to respond to an intervener based on the user's selection.

[0280] The recognition unit 161 outputs the video data, sound data, and recognition results supplied from the detection unit 72 to the registration data verification unit 261.

[0281] Based on the recognition results supplied from the recognition unit 161 and the data registered in the registration data storage unit 262, the registration data verification unit 261 determines whether the person who speaks to the user while the user is watching VR is the person to respond to.

[0282] When the person who speaks to the user while the user is watching VR is the person to respond to, the registration data verification unit 261 outputs the video data, sound data, and recognition results supplied from the recognition unit 161 to the extraction information overlay unit 212 to present the video data, sound data, and recognition results in the VR space.

[0283] If the person who speaks to the user while the user is watching VR is the person not to respond to, the registration data verification unit 261 refuses to present in the VR space. At this time, a message indicating that a response cannot be made and its reason (for example, "I am doing something") or the user's situation can be presented to the person who speaks to the user.

[0284] For example, in the registration data storage unit 262, data on the persons to respond to and not to respond to when a person speaks to the user during VR viewing is pre-registered.

[0285] <Processing of the HMD system>

[0286] Figure 23 Shows Figure 22 The flowchart of the processing of the HMD system 251 in

[0287] For example, a child says "Hey, hey, Dad" to a user wearing the HMD 61 and watching VR.

[0288] In step S241, the imaging device 62, the microphone 71, and the detection unit 72 detect that a person is speaking to the user, as described above with reference to Figure 6 .

[0289] In step S242, the recognition unit 161 refers to the detection result of the intervention supplied from the detection unit 72, and recognizes who the person speaking to the user is from the video data, by using the information registered in the recognition database 162.

[0290] In step S243, the registration data collating unit 261 determines whether or not to display the intervener in the VR space, based on the recognition result of the person supplied from the recognition unit 161 and the data registered in the registration data storage unit 262.

[0291] In the case where the intervener is a person to be responded to, it is determined in step S243 to display the intervener in the VR space, and the process proceeds to step S244. At this time, the registration data collating unit 261 outputs the video data, the sound data, and the recognition result of the person supplied from the recognition unit 161 to the extraction information superimposition unit 212.

[0292] In step S244, the extraction information superimposition unit 212 extracts the video and sound data related to the intervener from the video data and the sound data supplied from the registration data collating unit 261, superimposes the extracted video and sound data related to the intervener on the data related to the VR space 111, and outputs the superimposed data to the information presentation control unit 74.

[0293] In step S245, the information presentation control unit 74 causes the presentation unit included in the HMD 61 to present the superimposed data supplied from the extraction information superimposition unit 212 in the VR space 111, thereby presenting the video and the sound of the intervener to the user. Then, the process ends.

[0294] On the other hand, in the case where the intervener is a person not to be responded to, it is determined in step S243 not to display the intervener in the VR space, and the process proceeds to step S246.

[0295] In step S246, the registration data collating unit 261 presents the fact (message) that the user is busy to the intervener as sound, text, or video.

[0296] With the above configuration, unlike the HMD system 201 of Figure 17 , Figure 22 the HMD system 251 of

[0297] <Example of Message>

[0298] Figure 24is a diagram showing a method for presenting a message to an intervenor.

[0299] In Figure 24 , two presentation methods are shown.

[0300] In Figure 24 A of

[0301] By listening to the message, the intervenor can understand that the user is in a situation where he / she is busy and cannot respond.

[0302] Note that in this case, the user does not need to hear the sound message.

[0303] In Figure 24 B of

[0304] By viewing the message, the intervenor can understand that the user is in a situation where he / she is busy and cannot respond.

[0305] <8. Seventh embodiment of the present technology (intervention and obstacle detection)>

[0306] <Summary of HMD system>

[0307] Figure 25 is a diagram showing an overview of an HMD system according to the seventh embodiment of the present technology.

[0308] Figure 25 shows the state of a user of an HMD system 301 having the functions in the present technology in the environment computing system 1 in Figure 1

[0309] The HMD system 301 is, for example, a system that combines the HMD system 181 according to the fourth embodiment described above with reference to Figure 15 and the reproduction and object recognition in the VR space described in Patent Literature 1.

[0310] For example, a child in Figure 25 says "Hey, hey, Daddy" to a user who wears the HMD 61 and watches VR from the outside.

[0311] In the HMD system 301, the detection unit 72 detects an object Ob that can be an obstacle in addition to detecting the fact that the child speaks to the user by using the camera 62 and the microphone 71 provided in the HMD 61.

[0312] ​Then, the HMD system 301 superimposes the avatar 190 generated from the video of the child and the virtual object Obr of the detected object Ob on the data related to the VR space 111 viewed by the user to present the data.

[0313] As Figure 25 shown in FIG. 1, by detecting not only a person but also an object Ob such as an obstacle, and reproducing a virtual object Obr together with the avatar 190 in the VR space 111, it is possible to perform an action such as "moving an obstacle" on a real space, and it is also possible to reduce the risk of colliding with the object Ob without noticing the object Ob.

[0314] Note that, although an example of combining the HMD system 181 according to the fourth embodiment with the patent literature 1 has been described, the present technology is not limited to the fourth embodiment, and can be combined with other embodiments as long as superimposition presentation on a VR space is performed.

[0315] <9. Eighth embodiment of the present technology (sharing of VR space)>

[0316] <Summary of HMD system>

[0317] Figure 26 is a diagram showing a summary of an HMD system according to the eighth embodiment of the present technology.

[0318] Figure 26 shows a state of a user of an HMD system 351 having a function in the present technology in the environment computing system 1 in Figure 1

[0319] In the HMD system 201 according to the fifth embodiment described above with reference to Figure 17 , communication between the user wearing the HMD 61 and viewing VR and an intervenor in a real space continues, but in the HMD system 351, communication between the user wearing the HMD 61-1 and viewing VR and the intervenor wearing the HMD 61-2 and viewing VR continues. Figure 26

[0320] For example, the intervenor wearing the HMD 61-2 and viewing VR says to the user wearing the HMD 61-1 and viewing VR, "Can I join you?"

[0321] The HMD system 351 notifies the user that the intervenor is detected, asks the user for a response to the intervenor, and in the case where the intervenor is to be responded to according to the user's selection, invites the avatar 361 of the intervenor to enter the VR space 111.

[0322] As described above, in the HMD system 351 Figure 26 ​​In the HMD system 351, the user who wears the HMD 61-1 and watches the VR continues to communicate with the intervenor who wears the HMD 61-2 and watches the VR.

[0323] [Process of HMD system]

[0324] Figure 27 is a flowchart showing the process of the HMD system 351 in Figure 26

[0325] Note that the configuration of the HMD system 351 is basically similar to that of the HMD system 201 described above in Figure 17 Figure 27 Figure 17 Therefore, in the configuration of the HMD system 201 described above will be used to describe the configuration of the HMD system 351.

[0326] For example, the intervenor who wears the HMD 61-2 and watches the VR says to the user who wears the HMD 61-1 and watches the VR, "Can I join you?"

[0327] Figure 6 In step S261, the camera 62, the microphone 71, and the detection unit 72 detect that a person speaks to the user, as described above with reference to

[0328] In step S262, the recognition unit 161 refers to the detection result supplied from the detection unit 72 by using the information registered in the recognition database 162, and recognizes who the person who speaks to the user is from the video data.

[0329] In step S263, the detection result presents the notification information generated by the notification unit 211 for notifying the recognition result supplied from the recognition unit 161, and causes the presentation unit included in the HMD 61-1 to present the generated notification information and the inquiry about whether to invite the intervenor into the same VR space.

[0330] For example, the question "Q speaks to you. Invite Q into the VR space?" is presented, and the user selects Yes or No.

[0331] In step S264, the detection result presents the inquiry unit 211 determines whether to invite the intervenor into the VR space according to the user's selection. In the case where it is determined not to invite the intervenor into the VR space in step S264, Figure 27 the process of the HMD system 351 in

[0332] ​​​When it is determined in step S264 that the interlocutor is invited to enter the VR space, the process proceeds to step S265. At this time, the detection result presents the same VR space 111 to the interlocutor by the inquiry unit 211, for example, by communicating with the HMD 61-2 of the interlocutor, inviting the interlocutor to enter the same VR space 111.

[0333] In step S265, the extraction information superimposition unit 212 and the information presentation control unit 74 share the VR space 111 with the HMD 61-2 of the interlocutor, and present avatars and the like to each other in the HMD 61. That is, the extraction information superimposition unit 212 shares data related to the VR space 111, and presents the shared data related to the VR space 111 to the information presentation control unit 74 so that the interlocutor is presented as an avatar 361 in the VR space 111 of the HMD 61-1 of the user. The VR space 111 is also presented on the presentation unit of the HMD 61-2 of the interlocutor together with the avatar of the user. Then, the process ends.

[0334] As described above, the user can continuously interact with the interlocutor by inviting the interlocutor to enter the VR space 111.

[0335] <10. Ninth embodiment of the present technology (control according to interlocutor level)

[0336] <First control process according to interlocutor level of HMD system>

[0337] Figure 28 is a flowchart showing a first control process performed by an HMD system according to an interlocutor level according to the ninth embodiment of the present technology.

[0338] Figure 28 The control process of Figure 17 is, for example, a first control process for changing control according to a condition in the presence of an external person interlocutor in the HMD system 201 according to the fifth embodiment described above with reference to

[0339] Note that, in Figure 28 , the interlocutor level is defined (set) based on, for example, three person interlocutor conditions (types) of looking at the user, approaching the user, and speaking to the user, and control is performed.

[0340] In step S271, the imaging device 62 and the microphone 71 sense the surrounding situation.

[0341] In step S272, the detection unit 72 determines the state recognized as a result of detecting a person and a state in the vicinity.

[0342] When it is determined in step S272 that the person in the vicinity is looking at the user, the process proceeds to step S273.

[0343] In step S273, the detection result presentation and inquiry unit 211 determines that the intervention level is low, and notifies the user that the person is looking at him / her. That is, the detection result presentation and inquiry unit 211 generates notification information for notifying the detection result (the person is looking at the user) supplied from the detection unit 72 via the recognition unit 161, and causes the presentation unit included in the HMD 61 to present the notification information. Then, Figure 28 The first control processing ends.

[0344] In a case where it is determined in step S272 that the person is speaking to the user, the processing proceeds to step S274.

[0345] In step S274, the detection result presentation and inquiry unit 211 determines that the intervention level is high, and notifies the user that the person is speaking to him / her. That is, the detection result presentation and inquiry unit 211 generates notification information for notifying the detection result (the person is speaking to the user) supplied from the detection unit 72 via the recognition unit 161, and causes the presentation unit included in the HMD 61 to present the notification information. Then, the processing proceeds to step S277.

[0346] In addition, in a case where it is determined in step S272 that the person in the vicinity is speaking to the user, the processing proceeds to step S275.

[0347] In step S275, the detection result presentation and inquiry unit 211 determines that the intervention level is medium, notifies the user that the person is approaching him / her, and presents an inquiry as to whether or not to respond to the approaching person.

[0348] That is, the detection result presentation and inquiry unit 211 generates notification information for notifying the detection result (the person is approaching the user) supplied from the detection unit 72 via the recognition unit 161, and an inquiry as to whether or not to respond to the approaching person, and causes the presentation unit included in the HMD 61 to present the notification information.

[0349] In step S276, the detection result presentation and inquiry unit 211 determines whether or not to respond in accordance with the user's selection with respect to the presentation. In a case where it is determined in step S276 not to respond to the approaching person, Figure 28 The first control processing ends.

[0350] In a case where it is determined in step S276 to respond to the approaching person, the processing proceeds to step S277.

[0351] In step S277, the extraction information superimposition unit 212 extracts the video and the sound of the intervener from the video data and the sound data, and, for example, the information presentation unit 74 presents the video and the sound of the intervener to the presentation unit (of the VR space) to initiate the interaction. Then, Figure 28 The first control processing ends.

[0352] Note that although examples of defining intervention levels and implementing controls based on the conditions of intervention have been described above, intervention levels can also be defined based on the type of intervention and its conditions (degree of intervention) as follows.

[0353] For example, for distances of long < medium < short, the intervention level can be defined as low < medium < high. For approach speeds of low < medium < high, the intervention level can be defined as low < medium < high. For speaking volume of low < medium < high, the intervention level can be defined as low < medium < high.

[0354] Furthermore, as mentioned above, there are three types of control methods: informing the user of the intervener's status, informing the user of the intervener's status and querying for a response method, and informing the user of the intervener's status and automatically determining and initiating a response method, with one control method selected based on the intervention level.

[0355] Finally, there are three types of response methods: not responding to the intervener (doing nothing or notifying the intervener that the user cannot respond), presenting the intervener's status to the user via video or sound, and initiating an interaction mode between the intervener and the user. The response method is selected based on the intervention level.

[0356] Figure 29 This diagram illustrates control methods based on whether the user and the intervener are adults or children.

[0357] Depending on whether the user and the intervener are adults or children, there are five types of control methods.

[0358] (1) Without stopping the viewing of content, activate the interactive mode that only presents sound in the VR space.

[0359] (2) Without stopping the viewing of content, activate the interactive mode that presents images and sounds in the VR space.

[0360] (3) When the content is not viewed, activate the interactive mode that only presents sound in the VR space.

[0361] (4) When the content is stopped, activate the interactive mode that presents images and sounds in the VR space.

[0362] (5) Not responding to the intervention.

[0363] For example, such as Figure 29 As shown, the settings are pre-configured so that when the user is an adult and the intervener is an adult, control method (1) in the above control methods (1) to (5) is used, that is, an interactive mode in which only sound is presented in the HMD is established without stopping the viewing of the content.

[0364] Pre-arranged so that in the case where the user is an adult and the intervenor is a child, control method (2) among the above control methods (1) to (5) is established, that is, an interactive mode in which images and sound are presented in the HMD is established without stopping the viewing content.

[0365] Pre-arranged so that in the case where the user is a child and the intervenor is an adult, control method (3) among the above control methods (1) to (5) is established, that is, an interactive mode in which only sound is presented in the HMD is established with stopping the viewing content.

[0366] Pre-arranged so that in the case where the user is a child and the intervenor is a child, control method (4) among the above control methods (1) to (5) is established, that is, an interactive mode in which images and sound are presented in the HMD is established with stopping the viewing content.

[0367] Note that the optimal mode can vary depending on the use case, and the allocation of the control method for each condition is preferably changeable.

[0368] Furthermore, the determination as to whether the intervenor is a child or an adult can be made in accordance with the video, and the determination as to whether the user is a child or an adult can be pre-arranged.

[0369] Furthermore, the optimal control method for each condition of an adult or a child can be changed depending on the use case.

[0370] For example, in a mode where an adult speaks to a child who is watching VR, the setting can be changed so that the adult can forcibly stop the child's VR viewing.

[0371] For example, the setting can be changed so that control method (4) is automatically selected when an adult speaks to a child, so that the adult can interact with the child without disturbing the child's VR viewing.

[0372] For example, the setting can be changed so that control method (1) is automatically selected when an adult speaks to a child.

[0373] Although the example of an adult speaking to a child is described above, when a child speaks to an adult, the setting is not made as described above.

[0374] Note that although the attributes of the user and the intervenor are described as being an adult or a child in the above description, the attributes can also be an elderly person or a young person, or can be gender.

[0375] Here, the above control methods (1) to (5) can be automatically changed depending on the type of content being viewed.

[0376] For example, the type of the viewing content and the corresponding control method can be as follows.

[0377] In a case where the viewing content is a game, since it is not desirable that the game is stopped, the control method (1) is selected.

[0378] In a case where the viewing content is dynamic content such as a movie, since it is no problem that the content is stopped and there is a possibility that a video and a sound in the vicinity cannot be presented if the viewing content is not stopped, the control method (4) is selected.

[0379] In a case where the viewing content is static content such as picture viewing, research, or work, since it is possible to grasp an external situation without stopping the content, the control method (2) is selected.

[0380] In a case where the viewing content is a conference (online conference, etc.), since it is desirable to restrict external intervention, the control method (5) is selected.

[0381] [Second control processing by HMD system according to intervention level]

[0382] Figure 30 is a flowchart showing a second control processing by an HMD system according to an intervention level according to the ninth embodiment of the present technology.

[0383] Figure 30 The control processing of Figure 17 is a second control processing for changing control according to a condition in the presence of external person intervention in the HMD system 201 according to the fifth embodiment described above with reference to

[0384] Note that in the second control processing, the following are set in advance: [automatic or manual], [type of control method], [calculation of intervention level], and [control method corresponding to intervention level].

[0385] [Automatic or manual]

[0386] It is set in advance whether the system performs automatic control (automatic) or the user selects a control method (manual).

[0387] [Type of control method]

[0388] The following control methods 1 to 5 are set in advance.

[0389] Control method 1: Do nothing

[0390] Control method 2: Notify the user of the presence of an intervener and the situation

[0391] Control method 3: Perform control method 2 and start an interactive mode in which only sound is presented in the HMD without stopping viewing the content

[0392] Control method 4: Perform control method 2 and start an interactive mode in which both image and sound are presented in the HMD without stopping viewing the content

[0393] Control method 5: Perform control method 2 and start an interactive mode in which both image and sound are presented in the HMD with stopping viewing the content

[0394] Note that the intervention level is control method 1 < control method 2 < control method 3 < control method 4 < control method 5.

[0395] [Calculation of intervention level]

[0396] The weight (value of 0.0 to 1.0) of the intervention level is set in advance for each of the following elements.

[0397] State of person (looking, approaching, or speaking): Ws

[0398] Type of viewing content (static content, dynamic content, game, or meeting): Wc

[0399] User attribute (child, adult, etc.): Wu

[0400] Intervener attribute (child, adult, or pre-registered person): Wo

[0401] Then, the intervention level (Li) is calculated according to the weights as the following equation (1).

[0402] Li = Ws*Wc*Wu*Wo... (1)

[0403] [Calculation of intervention level]

[0404] In order to determine the control method corresponding to the intervention level, a control method determination table indicating the control method corresponding to the intervention level is set in advance.

[0405] [Example of control method determination table]

[0406] Figure 31 This is a diagram showing an example of the control method determination table.

[0407] In the case where the intervention level is 0 to 0.2, control method 1 is performed.

[0408] In the case where the intervention level is 0.2 to 0.4, control method 2 is performed.

[0409] In a case where the intervention level is 0.4 to 0.6, the control method 3 is executed after the control method 2 is executed.

[0410] In a case where the intervention level is 0.6 to 0.8, the control method 4 is executed after the control method 2 is executed.

[0411] In a case where the intervention level is 0.8 to 1.0, the control method 5 is executed after the control method 2 is executed.

[0412] Next, referring back to Figure 30 , the second control processing will be described.

[0413] In step S291, the imaging device 62 and the microphone 71 sense the surrounding situation.

[0414] In step S292, the detection unit 72 detects human intervention (looking, approaching, or speaking).

[0415] In step S293, the detection result presents and the inquiry unit 211 determines whether to adopt automatic or manual. In a case where it is determined in step S273 to specify the automatic setting (that is, to adopt automatic), the processing proceeds to step S294.

[0416] In step S294, the detection result presents and the inquiry unit 211 calculates the intervention level from the detection information, the content information, and the user information.

[0417] In step S295, the detection result presents and the inquiry unit 211 determines which control method the calculated intervention level corresponds to. In a case where it is determined in step S295 that the calculated intervention level corresponds to the control method 1, the second control processing of Figure 30 ends.

[0418] In a case where it is determined in step S295 that the calculated intervention level corresponds to the control method 2, the processing proceeds to step S296. In step S296, the detection result presents and the inquiry unit 211 notifies the user of the detection result. Then, Figure 30 the second control processing of ends.

[0419] In a case where it is determined in step S295 that the calculated intervention level corresponds to the control methods 3 to 5, the processing proceeds to step S299.

[0420] In a case where it is determined in step S293 to consult the user to set the control method (that is, to adopt manual), the processing proceeds to step S297.

[0421] In step S297, the detection result presents and the inquiry unit 211 notifies the user of the detection result, and asks the user for the control method.

[0422] In step S298, the detection result presents which control method the user selected, determined by the inquiry unit 211. Note that, in the example of Figure 30 , the notification is necessary, and control method 1 is not included in the options. In the case where it is determined in step S298 that the user selected control method 2, the second control processing of Figure 30 ends.

[0423] In the case where it is determined in step S298 that the user selected control method 3 to 5, the processing proceeds to step S299.

[0424] In step S299, the information superimposition unit 212 extracts the video and sound of the intervenor from the video data and sound data, and the information presentation unit 74 presents the video and sound of the intervenor to the presentation unit or the VR space to start the interactive mode according to the selected content (the content of the control method). Then, Figure 30 the second control processing of ends.

[0425] <11. Other>

[0426] <Examples of Application of the Present Technology>

[0427] Figure 32 is a diagram showing an example of a VR space viewed or experienced by a user.

[0428] In Figure 32 , a VR space 381 viewed or experienced by a user is shown.

[0429] The VR space 381 viewed or experienced by the user wearing the HMD 61-1 can be a locally deployed space.

[0430] The VR space 381 can be a meta-universe-like space deployed on a network and in which a plurality of users (for example, the user wearing the HMD 61-1 and the user wearing the HMD 61-2) participate.

[0431] Furthermore, the present technology aims at: a user wearing an HMD operated for a dedicated application, a game, or the like, and an intervention from another person in a real space in which the user is located.

[0432] Here, the device for viewing the VR space is not limited to the HMD, and can be a user device that blocks external vision and sound when viewing the VR space, such as a VR headset.

[0433] Note that the VR headset is a headset for VR operated in a dedicated application, VR operated in a dedicated game machine, or the like, for example.

[0434] <Effects of the Present Technology>

[0435] In the present technology, an external intruder with respect to a user of a user device that blocks an external view and sound is detected, and a control for presenting information about the detected intruder to the user is executed.

[0436] Accordingly, it is possible to support the user to respond quickly and reduce the risk.

[0437] <Configuration example of computer>

[0438] The series of processing steps described above can be executed by hardware, or can be executed by software. In the case where the series of processing steps is executed by software, a program included in the software is installed on a computer included in a dedicated hardware, a general-purpose personal computer, or the like from a program recording medium.

[0439] Figure 33 is a block diagram showing a configuration example of hardware of a computer 900 that executes the series of processing by a program.

[0440] A central processing unit (CPU) 901, a read only memory (ROM) 902, and a random access memory (RAM) 903 are connected to each other by a bus 904.

[0441] The bus 904 is also connected to an input / output interface 910. An input unit 911 including a keyboard, a mouse, and the like, and an output unit 912 including a display, a speaker, and the like are connected to the input / output interface 910. Further, a storage unit 913 including a hard disk, a non-volatile memory, and the like, a communication unit 914 including a network interface, and the like, and a drive 915 that drives a removable storage medium 921 are connected to the input / output interface 910.

[0442] In the computer 900 configured as described above, for example, the CPU 901 loads a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executes the program, thereby executing the series of processing described above.

[0443] For example, the program executed by the CPU 901 is stored in the removable storage medium 921, or is provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting, and then installed in the storage unit 913.

[0444] Note that the program to be executed by the computer can be a program in which processing is executed in a time series in the order described herein, or can be a program in which processing is executed in parallel or at a necessary timing such as timing at which retrieval is executed.

[0445] Note that, in this specification, a system refers to a collection of a plurality of components (devices, modules (components), and the like), and it does not matter whether all components are in the same housing. Therefore, both a plurality of devices housed in separate housings and connected to each other via a network and one device including a plurality of modules housed in one housing are systems.

[0446] Furthermore, the effects described in this specification are merely illustrative or illustrative, and there can be other effects as well.

[0447] Embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present technology.

[0448] For example, the present technology can be configured as cloud computing in which one function is shared and cooperatively processed by a plurality of devices via a network.

[0449] Furthermore, each step described in the above-described flowcharts can be executed by a single device, or can be executed in a shared manner by a plurality of devices.

[0450] In addition, in a case where a single step includes a plurality of processes, the plurality of processes included in the single step can be executed by a single device or in a shared manner by a plurality of devices.

[0451] <Example of a combination of configurations>

[0452] The present technology can also be configured as follows: (1)

[0454] An information processing device including:

[0455] a detection unit that detects an external intruder with respect to a user of a user device that blocks external views and sounds of the outside; and

[0456] a presentation control unit that performs control for presenting information related to the detected intruder to the user. (2)

[0458] The information processing device according to (1), wherein

[0459] the presentation control unit performs control for presenting notification information as information related to the intruder, the notification information being for notifying the presence of the intruder. (3)

[0461] The information processing device according to (1), wherein

[0462] The presentation control unit performs control for presenting at least one of a video of the intervenor, an avatar of the intervenor, and a sound of the intervenor as information related to the intervenor. (4)

[0464] The information processing apparatus according to any one of (1) to (3), wherein

[0465] The presentation control unit performs control for presenting information related to the intervenor in a VR space viewed through the apparatus. (5)

[0467] The information processing apparatus according to any one of (1) to (4), further comprising:

[0468] A response determination unit that determines whether or not to respond to the intervenor, wherein

[0469] In a case where it is determined to respond to the intervenor, the presentation control unit performs control for presenting information related to the intervenor. (6)

[0471] The information processing apparatus according to (5), wherein

[0472] In a case where it is determined not to respond to the intervenor, the presentation control unit performs control for presenting, to the intervenor, information indicating that the user is unable to respond. (7)

[0474] The information processing apparatus according to (5), wherein

[0475] The response determination unit determines whether or not to respond to the intervenor in accordance with a selection of the user. (8)

[0477] The information processing apparatus according to (5), further comprising:

[0478] A response storage unit that pre-stores information indicating whether or not to respond to each of the intervenors, wherein

[0479] The response determination unit determines whether or not to respond to the intervenor in accordance with the information indicating whether or not to respond to each of the intervenors. (9)

[0481] The information processing apparatus according to (5), wherein

[0482] The response determination unit determines whether or not to respond to the intervenor in accordance with an intervention level. (10)

[0484] The information processing apparatus according to (9), wherein

[0485] The intervention level is set in accordance with an intervention type or an intervention degree of the intervenor. (11)

[0487] The information processing apparatus according to (10), wherein

[0488] The intervention type includes at least one of an action of speaking to the user, an action of approaching the user, or an action of looking at the user. (12)

[0490] The information processing apparatus according to (10), wherein

[0491] The intervention degree of the intervenor includes at least one of a distance when approaching the user, a speed of approaching the user, or a size of a voice speaking to the user. (13)

[0493] The information processing apparatus according to (9), wherein

[0494] The intervention level is set in accordance with at least one of an attribute of the user and an attribute of the intervenor. (14)

[0496] The information processing apparatus according to (13), wherein

[0497] The attribute is an adult or a child. (15)

[0499] The information processing apparatus according to (9), wherein

[0500] The intervention level is set in accordance with a type of content being viewed by the user. (16)

[0502] The information processing apparatus according to any one of (1) to (15), wherein

[0503] The detection unit detects the intervenor using at least one of video data captured and generated by an imaging unit and sound data collected and generated by a sound collection unit. (17)

[0505] The information processing apparatus according to (16), wherein

[0506] The imaging unit and the sound collection unit are provided in the user apparatus. (18)

[0508] The information processing apparatus according to any one of (1) to (17), wherein

[0509] The user device is a head-mounted display. (19)

[0511] An information processing method comprising:

[0512] by an information processing apparatus,

[0513] detecting an external intervenor with respect to a user wearing a device that blocks external visual fields and sound, and performing control for presenting information related to the detected intervenor to the user.

[0514] (20) A program causing a computer to function as:

[0515] a detection unit that detects an external intervenor with respect to a user wearing a device that blocks external visual fields and sound; and

[0516] a presentation control unit that performs control for presenting information related to the detected intervenor to the user.

[0517] List of reference numerals

[0518] 1 environmental computing system

[0519] 21 user interface

[0520] 22 environmental previous information obtaining unit

[0521] 23 user previous information obtaining unit

[0522] 24 collaboration sensing unit

[0523] 25 surrounding environment detecting unit

[0524] 26 user state detecting unit

[0525] 27 user response detecting unit

[0526] 28 context management unit

[0527] 29 device control unit

[0528] 30 environmental context model storage unit

[0529] 31 user context model storage unit

[0530] 32 service list storage unit

[0531] 51 HMD system

[0532] 61, 61-1, 61-2 HMD

[0533] 62 camera

[0534] 71 microphone

[0535] 72 detection unit

[0536] 73 notification information generation unit

[0537] 74 information presentation control unit

[0538] 81 person detection section

[0539] 82 motion determination section

[0540] 83 utterance portion detection section

[0541] 84 speaker identification section

[0542] 85 speaker data DB

[0543] 86 intervention detection section

[0544] 91 camera

[0545] 92 microphone

[0546] 101 HMD system

[0547] 111 VR space

[0548] 131 notification information superimposition unit

[0549] 151 HMD system

[0550] 161 identification unit

[0551] 162 identification database

[0552] 181 HMD system

[0553] 190 avatar

[0554] 191 skeletal data extraction unit

[0555] 192 three-dimensional information reconstruction unit

[0556] 201 HMD system

[0557] 211 detection result presentation and interrogation unit

[0558] 212 extracted information superimposition unit

[0559] 231 omnidirectional camera

[0560] 232 virtual image

[0561] 241 forward-facing camera

[0562] 251 HMD system

[0563] 261 registration data collating unit

[0564] 262 registration data storing unit

[0565] 271 external presentation unit

[0566] 301 HMD system

[0567] 351 HMD system

[0568] 361 avatar

[0569] 381 VR space

Claims

1.An information processing apparatus comprising: a detection unit that detects an external intervenor with respect to a user of a user apparatus that blocks an external field of view and sound; and a presentation control unit that performs control for presenting information related to the detected intervenor to the user. 2.The information processing apparatus according to claim 1, wherein the presentation control unit performs control for presenting notification information for notifying of the presence of the intervenor. 3.The information processing apparatus according to claim 1, wherein the presentation control unit performs control for presenting at least one of a video of the intervenor, an avatar of the intervenor, and a sound of the intervenor. 4.The information processing apparatus according to claim 1, wherein the presentation control unit performs control for presenting information related to the intervenor in a VR space viewed through the apparatus. 5.The information processing apparatus according to claim 1, further comprising: a response determination unit that determines whether to respond to the intervenor, wherein in a case where it is determined to respond to the intervenor, the presentation control unit performs control for presenting information related to the intervenor. 6.The information processing apparatus according to claim 5, wherein in a case where it is determined not to respond to the intervenor, the presentation control unit performs control for presenting information indicating that the user cannot respond to the intervenor to the intervenor. 7.The information processing apparatus according to claim 5, wherein the response determination unit determines whether to respond to the intervenor in accordance with a selection of the user. 8.The information processing apparatus according to claim 5, further comprising: a response storage unit that pre-stores information indicating whether to respond to each of the intervenors, wherein the response determination unit determines whether to respond to the intervenor in accordance with the information indicating whether to respond to each of the intervenors. 9.The information processing apparatus according to claim 5, wherein the response determination unit determines whether to respond to the intervenor in accordance with an intervention level. 10.The information processing apparatus according to claim 9, wherein the intervention level is set in accordance with an intervention type or an intervention degree of the intervenor. 11.The information processing apparatus according to claim 10, wherein the intervention type includes at least one of an action of speaking to the user, an action of approaching the user, or an action of looking at the user. 12.The information processing apparatus according to claim 10, wherein the intervention degree of the intervenor includes at least one of a distance when approaching the user, a speed of approaching the user, or a size of a voice of speaking to the user. 13.The information processing apparatus according to claim 9, wherein the intervention level is set in accordance with at least one of an attribute of the user and an attribute of the intervenor. 14.The information processing apparatus according to claim 13, wherein the attribute is an adult or a child. 15.The information processing apparatus according to claim 9, wherein The intervention level is set according to a type of content being viewed by the user. 16.The information processing apparatus according to claim 1, wherein The detection unit detects the intervenor using at least one of video data captured and generated by an imaging unit and sound data collected and generated by a sound collecting unit. 17.The information processing apparatus according to claim 16, wherein The imaging unit and the sound collecting unit are provided in the user apparatus. 18.The information processing apparatus according to claim 1, wherein The user apparatus is a head-mounted display. 19.An information processing method comprising: detecting, by an information processing apparatus, an external intervenor with respect to a user wearing an apparatus that blocks external visual field and sound, and performing control for presenting information related to the detected intervenor to the user. 20.A program causing a computer to function as: a detection unit that detects an external intervenor with respect to a user wearing an apparatus that blocks external visual field and sound; and a presentation control unit that performs control for presenting information related to the detected intervenor to the user.

Images