Information processing apparatus
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- CANON KK
- Filing Date
- 2023-05-29
- Publication Date
- 2026-06-16
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device, and more particularly to a technique for supporting communication between users in a virtual space or a mixed reality space. [Background technology]
[0002] Virtual reality (hereinafter, VR) technology is known, which uses a computer to generate an environment in which a user can interact with virtual objects. Mixed reality (hereinafter, MR) technology and augmented reality (hereinafter, AR) technology are also known as technologies that fuse images from the real world and the virtual world in real time. Normally, a goggle- or eyeglass-type display device called a head-mounted display (hereinafter, HMD) is used to watch images using these technologies (hereinafter, XR technology). An HMD has, for example, two displays arranged to match the positions of the left and right eyes. HMDs with a camera on the front that captures external images have also been proposed.
[0003] A technology has been proposed that enables communication between users using characters (hereafter, avatars) that represent users in virtual space (hereafter, VR space) or mixed reality space (hereafter, MR space). Avatars can be operated using a controller or by motion tracking.
[0004] Patent Document 1 discloses a technology that gives a user feedback that he or she has touched a virtual object. Patent Document 2 discloses a technology that starts a chat between users when avatars touch each other in a VR space. Patent Document 3 discloses a technology that notifies a terminal of a user who is not wearing an HMD in response to the avatar of the user who is not wearing an HMD being shaken by another avatar in a VR space. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] JP 2020-80152 A [Patent Document 2] JP 2001-236291 A [Patent Document 3] JP 2019-61434 A Summary of the Invention [Problem to be solved by the invention]
[0006] However, the techniques disclosed in Patent Documents 1 to 3 cannot fully utilize the contact between avatars as a communication tool. The technique disclosed in Patent Document 1 provides feedback to a user that the user has actively contacted a virtual object. Even if the technique disclosed in Patent Document 1 is used, feedback can be provided to a user (contactor) of an avatar that actively contacts another avatar, but no notification is given to a user (contactee) of an avatar that has been contacted by another avatar. Therefore, the contactee cannot recognize the contact. In the technique disclosed in Patent Document 2, the contact between avatars is merely used to start a chat. In the technique disclosed in Patent Document 3, the contact between avatars (the shaking of another avatar against an avatar) is merely used to notify a terminal of a user who is not wearing an HMD. Thus, in the techniques disclosed in Patent Documents 2 and 3, the contact between avatars is not used in the same way as contact in the real world, and is insufficient as a communication tool.
[0007] The present invention aims to realize a virtual contact between users, such as contact between avatars, as a means of sufficient communication. The purpose of this project is to make it possible to use the system as a collaborative tool. [Means for solving the problem]
[0008] A first aspect of the present invention is an information processing device characterized by having a detection means for detecting virtual contact of a second user with a first user in a three-dimensional space which is a virtual space or a mixed reality space, and a control means for controlling the device to notify the first user of information relating to the position of the virtual contact when the virtual contact occurs.
[0009] A second aspect of the present invention is a control method for an information processing device, characterized by having a step of detecting a virtual contact of a second user with a first user in a three-dimensional space which is a virtual space or a mixed reality space, and a step of controlling the device so that, when the virtual contact occurs, information regarding the position of the contact is notified to the first user.
[0010] A third aspect of the present invention is a program for causing a computer to function as each of the means of the information processing device described above. Effect of the Invention
[0011] According to the present invention, virtual contact between users, such as contact between avatars, can be used as a sufficient communication tool. [Brief description of the drawings]
[0012] [Figure 1] FIG. 1 is an external view of a head mounted display (hereinafter, HMD). [Diagram 2] FIG. 2 is a block diagram of an HMD. [Diagram 3] FIG. [Figure 4] 13 is a flowchart of a contact notification process. [Diagram 5] 13 is a flowchart of a contact detection process. [Figure 6] 13 is a flowchart of a notification determination process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
[0014] FIG. 1 is an external view showing the external appearance of a head mounted display (hereinafter, HMD) 100, which is an example of an information processing device to which the present invention can be applied. The HMD 100 is a goggle-type display device worn on the head of a user. The main body 101 includes various hardware components, which will be described later. For example, the main body 101 includes a display on the rear side of the main body 101. A user wearing the HMD 100 can view various contents displayed on the display. The strap 102 is used to fix the main body 101 to the user's head. The camera 103 captures an image of the surroundings of the HMD 100 (for example, the front side).
[0015] The HMD 100 can provide a mixed reality space (hereinafter, MR space) to the user by displaying an image captured by the camera 103 or an image based on the captured image on the display. The HMD 100 can also provide a virtual space (hereinafter, VR space) to the user by displaying an image unrelated to the image captured by the camera 103 on the display. The HMD 100 can display a character (hereinafter, avatar) representing a user (a user of the HMD 100 or a user of an HMD different from the HMD 100) in the VR space and the MR space. The HMD 100 may display the user himself captured by the camera 103 as the user of the HMD 100 instead of an avatar.
[0016] There are two methods for providing MR space to users: video see-through and optical see-through. In the video see-through method, an image of a captured real space is displayed, and the user can confirm the real space through the image. A virtual object such as an avatar is displayed, for example, by being synthesized (superimposed) on the image of the real space. In the optical see-through method, the user can confirm the real space that is not an image through a lens arranged to face the user's eyes. The virtual object is displayed, for example, on the lens. The method of the HMD 100 is assumed to be the video see-through method, but it may also be the optical see-through method. A user of an HMD using the optical see-through method may be able to confirm a displayed avatar as the user (the user's hands, arms, torso, legs, etc.), or may be able to confirm the user himself / herself that is not an image (the user's avatar may not be displayed). The present invention is also applicable to a personal computer, a game machine, a server, and the like connected to an HMD.
[0017] Instead of a display (a display having a display surface arranged to face the eyes of the user wearing the HMD 100) provided on the rear side of the main body 101, a retinal projection display that projects an image onto the user's retina may be used. In the case of an optical see-through method, a projection display that projects an image onto a lens arranged to face the user's eyes may be used.
[0018] 2 is a block diagram showing the hardware configuration of the HMD 100. A control unit 201, a memory 202, a non-volatile memory 203, an image / audio processing unit 204, a display 205, an operation unit 206, a storage medium I / F 207, an external I / F 209, and a communication I / F 210 are connected to an internal bus 250. A sound output unit 212 and a posture detection unit 213 are also connected to the internal bus 250. These are provided in the main body unit 101 shown in FIG. 1. The units connected to the internal bus 250 can exchange data with each other via the internal bus 250.
[0019] The control unit 201 is composed of at least one processor or circuit, and controls the entire HMD 100. The control unit 201 is, for example, a CPU. The memory 202 is, for example, a RAM (a volatile memory using semiconductor elements, etc.). The control unit 201 controls each part of the HMD 100 using the memory 202 as a work memory according to a program stored in, for example, a non-volatile memory 203. The non-volatile memory 203 stores image data, audio data, other data, various programs for the control unit 201 to operate, etc. The non-volatile memory 203 is, for example, a flash memory, a ROM, etc.
[0020] The image / audio processing unit 204 performs various image processing on images stored in the non-volatile memory 203 or the storage medium 208, images acquired via the external I / F 209, images acquired via the communication I / F 210, etc., based on the control of the control unit 201. Similarly, the image / audio processing unit 204 performs various audio processing on audio stored in the non-volatile memory 203 or the storage medium 208, audio acquired via the external I / F 209, audio acquired via the communication I / F 210, etc., based on the control of the control unit 201. The image processing and audio processing performed by the image / audio processing unit 204 include A / D conversion processing, D / A conversion processing, encoding processing, compression processing, decoding processing, noise reduction processing, etc. The image processing performed by the image / audio processing unit 204 also includes enlargement / reduction processing (resizing) and color conversion processing, etc. The image processing performed by the image / audio processing unit 204 includes panoramic development, mapping processing, conversion processing, and the like of omnidirectional images or VR images, which are wide-range images having wide-range data even if they are not omnidirectional. The image / audio processing unit 204 can also perform various image processing on images captured by the camera 103 shown in FIG. 1. If the HMD 100 is equipped with a microphone or a microphone is connected to the HMD 100, the image / audio processing unit 204 can also perform various audio processing on audio acquired by the microphone. The image / audio processing unit 204 is, for example, a dedicated circuit for performing various image processing and various audio processing. Image The control unit 201 may perform various image processing and various audio processing in accordance with a program without using the audio processing unit 204.
[0021] 1, and displays images, GUI (Graphical User Interface) screens, and the like based on the control of the control unit 201. The control unit 201 generates a display control signal according to a program, and controls each unit of the HMD 100 to generate a video signal for display on the display 205 and input the video signal to the display 205. The display 205 displays an image based on the video signal input to the display 205.
[0022] The operation unit 206 is an input device for receiving user operations, and includes a character information input device such as a keyboard, a pointing device such as a mouse, a button, a dial, a joystick, a touch sensor, a touch pad, and the like.
[0023] The storage medium I / F 207 is configured so that a storage medium 208 such as a memory card, CD, or DVD can be attached / detached, and reads data from / writes data to the attached storage medium 208 based on the control of the control unit 201. The external I / F 209 is an interface that is connected to an external device via a wired or wireless connection and is used to input / output video signals and audio signals to / from the HMD 100. The communication I / F 210 is an interface that communicates with an external device, the Internet 211, etc., and transmits / receives various types of data such as files and commands.
[0024] The audio output unit 212 outputs audio of video data and music data, operation sounds, ringtones, various notification sounds, etc. The audio output unit 212 includes an audio output terminal 212a for connecting earphones or the like, and a speaker 212b, but audio may also be output from an external device via wireless communication or the like.
[0025] The attitude detection unit 213 detects the attitude (tilt) of the HMD 100 with respect to the direction of gravity, and the attitude with respect to each axis of yaw, roll, and pitch. Based on the attitude detected by the attitude detection unit 213, it is possible to determine whether the HMD 100 is held horizontally, held vertically, pointed upward, pointed downward, or at an angle. As the attitude detection unit 213, at least one of an acceleration sensor, a gyro sensor, a geomagnetic sensor, a direction sensor, an altitude sensor, etc. can be used, and a combination of two or more sensors can also be used.
[0026] In this embodiment, when there is (virtual) contact between avatars in a three-dimensional space (hereinafter, XR space) that is a VR space or an MR space, information (contact information) regarding the position and strength of the contact is notified to the contacted user (contactee). As described above, the HMD 100 may display the user himself / herself captured by the camera 103 as the user of the HMD 100, instead of an avatar. Therefore, the above contact may be contact of the avatar of the contacting user (contactor) with the contacted user when viewed from the contacted user. Contact of the avatar of the contacting user with the contacted user when viewed from the contacted user can be interpreted as contact of the contacting user with the avatar of the contacted user when viewed from the contacted user.
[0027] 3(a) and 3(b) are schematic diagrams showing an example of contact information notified to the contacted user. The contacted user is assumed to be the user of the HMD 100. In FIGS. 3(a) and 3(b), an image of the XR space is displayed on a display surface 300 of the display 205. In FIG. 3(a), a notification item 301 is displayed superimposed on the image of the XR space, and in FIG. 3(b), a notification item 302 is displayed superimposed on the image of the XR space. Notification item 301 , 302 are displayed when the avatar of the contacted user comes into contact with the avatar of the contacted user. The display positions of the notification items 301, 302 indicate the position of contact between the avatars, and the display sizes of the notification items 301, 302 indicate the strength of the contact between the avatars. FIG. 3(a) corresponds to a case where the avatar of the contacting user lightly (weakly) comes into contact with the left shoulder of the avatar of the contacted user. Therefore, the notification item 301 is displayed small on the left side. FIG. 3(b) corresponds to a case where the avatar of the contacting user comes into strong contact with the head of the avatar of the contacted user (for example, when the avatar of the contacting user hits the head of the avatar of the contacted user strongly). Therefore, the notification item 302 is displayed large on the upper side.
[0028] Note that the intensity of the contact does not have to be notified. Furthermore, the contact information may be notified by a sound output using the audio output unit 212, instead of a display on the display 205. In this case, for example, the direction of the sound indicates the position of the contact, and the loudness (volume) of the sound indicates the intensity of the contact. The contact information may be notified by vibration of a device connected to the external I / F 209 (for example, a notification by haptics). In this case, for example, the position of the vibration indicates the position of the contact, and the intensity of the vibration indicates the intensity of the contact. The contact information may be notified by a combination of two or three of the display, the sound, and the vibration.
[0029] 4 is a flowchart of the contact notification process performed by the HMD 100. This contact notification process is realized by the control unit 201 expanding a program stored in the non-volatile memory 203 into the memory 202 and executing it. For example, when a user wears the HMD 100, starts it up, and executes a predetermined application for experiencing an XR space (communication between users in the XR space), the contact notification process is started and is repeatedly performed at a predetermined cycle.
[0030] In step S401, the control unit 201 performs a contact detection process to detect contact between avatars (contact between an avatar of a user of the HMD 100 and an avatar of another user), and stores a result of the contact detection process in the memory 202. Details of the contact detection process will be described later with reference to FIG.
[0031] In step S402, the control unit 201 checks the result of the contact detection process stored in the memory 202 (the result of the contact detection process in step S401), and determines whether or not contact was detected in step S401. If contact was detected, the process proceeds to step S403, and if not, the contact notification process ends.
[0032] In step S403, the control unit 201 performs a notification determination process to determine whether or not to notify the contact information, and stores the result of the notification determination process in the memory 202. The details of the notification determination process will be described later with reference to FIG.
[0033] In step S404, the control unit 201 checks the result of the notification determination process stored in the memory 202 (the result of the notification determination process in step S403), and determines whether or not to notify the contact information according to the result. If the contact information is to be notified, the process proceeds to step S405, and if not, the contact notification process ends.
[0034] In step S405, the control unit 201 calculates the direction of the position of the contact detected in step S401 relative to the field of view of the contacted user, and stores it in the memory 202. There are no particular limitations on the method of calculating the position (direction) of the contact, but for example, the position (direction) of the contact is calculated based on the position and orientation of the avatar of the contacted user (user of the HMD 100) and the position and orientation of the avatar of the contacting user. Information on the position and orientation of the avatar of the contacted user is managed, for example, by the HMD 100 (memory 202) or a server, and is obtained from the memory 202 or from the server via the Internet 211. The position and posture of the user's avatar are updated, for example, by the movement (movement and posture change) of the contacted user, or by a user operation using a controller (not shown) of the HMD 100. The position and posture of the contacted user's avatar are determined, for example, based on the imaging result of the camera 103, the detection result of the posture detection unit 213, or a user operation. Information on the position and posture of the contacting user's avatar is acquired, for example, from a server or the HMD of the contacting user via the Internet 211.
[0035] In step S406, the control unit 201 calculates the intensity of the contact detected in step S401 and stores it in the memory 202. The method of calculating the intensity of the contact is not particularly limited, but for example, the intensity of the contact is calculated based on the position and posture of the avatar of the contacted user (user of the HMD 100) and the position and posture of the avatar of the contacting user. When calculating the intensity of the contact, information such as the movement speed of the avatar, the mass (virtual mass) of the avatar, and the mass (virtual mass) of a virtual object held by the avatar may be used. Information on the mass of the avatar of the contacted user and information on the mass of the virtual object are managed, for example, by the HMD 100 (memory) or a server, and are obtained from the memory 202 or from the server via the Internet 211. The mass of the avatar of the contacted user is, for example, registered in advance in the HMD 100 or the server by the contacted user. The mass of the virtual object is, for example, determined in advance when an application for experiencing an XR space is created. The information on the mass of the contacting user is acquired via the Internet 211 from, for example, a server or the contacting user's HMD.
[0036] In step S407, the control unit 201 notifies the contacted user (the user of the HMD 100) of information (contact information) on the position and strength of the contact detected in step S401 based on the calculation results in steps S405 and S406 stored in the memory 202. The notification method is as described above.
[0037] At least one of steps S403, S404, and S406 may be omitted. For example, steps S403 and S404 may be omitted, and if contact is detected, the process may proceed from step S402 to step S405. Step S406 may be omitted, and the process may proceed from step S405 to step S407.
[0038] 5 is a flowchart of the contact detection process (step S401 in FIG. 4). This contact detection process is realized by the control unit 201 loading a program stored in the non-volatile memory 203 into the memory 202 and executing it.
[0039] In step S501, the control unit 201 performs a detection process to detect contact of an avatar of another user with an avatar of the user of the HMD 100. The method of the detection process is not particularly limited, and for example, the contact is detected based on the position and posture of the avatar of the user of the HMD 100 and the position and posture of the avatar of the other user. The detection process may be performed by a server or the like. The control unit 201 may obtain the result of the detection process from the server or the like via the Internet 211.
[0040] In step S502, the control unit 201 acquires contact area information on a part (contact area) of another avatar that has come into contact with the avatar of the user of the HMD 100, and stores the contact area information in the memory 202. The contact area is identified (detected), for example, in the above-mentioned detection process. In this embodiment, the contact area information indicates the contact area and whether or not the contact area is a tracking area (described later). The tracking area information is managed, for example, in the HMD 100 (storage medium 208) or a server, and is acquired from the storage medium 208 or from the server via the Internet 211. If no contact is detected in step S501, the contact area information is not acquired.
[0041] The tracking part is a part that is tracked, and the position is determined based on the position information of the controller or HMD. For example, when the position of the hand of the avatar is determined based on the position information of the controller, and the position of the elbow is determined based on the position of the hand, the hand corresponds to the tracking part, and the elbow does not correspond to the tracking part. It should be noted that the tracking device is not limited to the controller or HMD. For example, the tracking device may be a device attached to the elbow, shoulder, knee, foot, waist, etc., and the position of the elbow, shoulder, knee, foot, waist, etc. of the avatar may be determined based on the position of the device. Therefore, the elbow, shoulder, knee, foot, waist, etc. of the avatar may also correspond to the tracking part.
[0042] If the avatar is capable of holding a virtual object in the hand, the held virtual object may be regarded as part of the hand.Similarly, if the avatar is capable of holding a virtual object in the tracking part, the held virtual object may be regarded as part of the tracking part.
[0043] In step S503, the control unit 201 determines whether or not the contact site is a tracking site based on the contact site information (contact site information acquired in step S502) stored in the memory 202. If the contact site is a tracking site, the process proceeds to step S505; otherwise, the process proceeds to step S504. If contact site information was not acquired in step S502 (if contact was not detected in step S501), the process proceeds to step S504.
[0044] In step S504, the control unit 201 determines whether or not the contact part is a hand based on the contact part information (contact part information acquired in step S502) stored in the memory 202. If the contact part is a hand, the process proceeds to step S505, and if not, the process proceeds to step S506. If contact part information was not acquired in step S502 (if contact was not detected in step S501), the process proceeds to step S506.
[0045] In step S505, the control unit 201 stores in the memory 202 a processing result that contact of the avatar of the user of the HMD 100 with the avatar of another user has been detected.
[0046] In step S506, the control unit 201 stores in the memory 202 a processing result that contact of the avatar of the user of the HMD 100 with the avatar of another user has not been detected.
[0047] At least one of steps S502 to S504 and S506 may be omitted. For example, step S503 may be omitted and the process may proceed from step S502 to step S504. Step S504 may be omitted and the process may proceed from step S503 to step S506 if the contact part is not a tracking part. Steps S502 to S504 and S506 may be omitted and the process may proceed from step S501 to step S505.
[0048] 6 is a flowchart of the notification determination process (step S403 in FIG. 4). This notification determination process is realized by the control unit 201 loading a program stored in the non-volatile memory 203 into the memory 202 and executing it.
[0049] In step S601, the control unit 201 determines whether filtering using the notification target user list is enabled. If it is enabled, the process proceeds to step S602, and if it is not enabled, the process proceeds to step S605. The notification target user list is a list indicating users (notification target users) to be notified of contact information. Information on the setting (enabled / disabled) of filtering using the notification target user list is stored in, for example, the HMD 100 (memory 202 or storage medium 208). ) or a server. Therefore, information on the settings of filtering using the notification target user list is obtained from memory 202, from storage medium 208, or from the server via Internet 211.
[0050] In step S602, the control unit 201 acquires the user information of the contact user from a server, the contact user's HMD, or the like via the Internet 211, and stores the user information in the memory 202. The user information of the contact user includes, for example, information indicating the contact user (such as ID information for identifying the contact user).
[0051] In step S603, the control unit 201 acquires the notification target user list and compares it with the user information of the contact user. The notification target user list is managed, for example, by the HMD 100 (storage medium 208) or a server, and is acquired from the storage medium 208 or from the server via the Internet 211.
[0052] In step S604, the control unit 201 determines whether or not the contacted user corresponds to the notification target user. If the contacted user corresponds to the notification target user, the process proceeds to step S609, and if not, the process proceeds to step S611.
[0053] In step S605, the control unit 201 determines whether filtering using the non-notification user list is enabled. If it is enabled, the process proceeds to step S606, and if not, the process proceeds to step S609. The non-notification user list is a list indicating users (non-notification users) to whom contact information is not to be notified. Information on the settings (enabled / disabled) of filtering using the non-notification user list is managed, for example, by the HMD 100 (memory 202 or storage medium 208) or a server. Therefore, information on the settings of filtering using the non-notification user list is acquired from the memory 202, acquired from the storage medium 208, or acquired from the server via the Internet 211.
[0054] In step S606 , the control unit 201 acquires the user information of the contact user from a server, the contact user's HMD, or the like via the Internet 211 , and stores the user information in the memory 202 .
[0055] In step S607, the control unit 201 acquires the list of users not to be notified and compares it with the user information of the contact user. The list of users not to be notified is managed, for example, by the HMD 100 (storage medium 208) or a server, and is acquired from the storage medium 208 or from the server via the Internet 211.
[0056] In step S608, the control unit 201 determines whether or not the contacted user corresponds to a user not to be notified. If the contacted user corresponds to a user not to be notified, the process proceeds to step S611, and if not, the process proceeds to step S609.
[0057] In step S609, the control unit 201 determines whether or not the avatar of the contacting user is present in the field of view of the contacted user (user of the HMD 100). If the avatar of the contacting user is present in the field of view, the process proceeds to step S611, and if not, the process proceeds to step S610. The method of determination in step S609 is not particularly limited. For example, the control unit 201 may determine whether or not the avatar of the contacting user is present in the field of view based on a video signal (information used to generate the video signal) to be displayed on the display 205. The control unit 201 may determine whether or not the avatar of the contacting user is present in the field of view based on the position and posture of the avatar of the contacted user, the position and posture of the avatar of the contacting user, and the like.
[0058] In step S610, the control unit 201 stores in the memory 202 the determination result that the contact information is to be notified.
[0059] In step S611, the control unit 201 stores in the memory 202 the determination result that the contact information is not to be notified.
[0060] At least one of steps S601 to S609 may be omitted. For example, steps S601 to S604 may be omitted, and step S605 may be the first step. Steps S605 to S608 may be omitted, and if filtering using the notification target user list is invalid, the process may proceed from step S601 to step S609. Steps S601 to S608 may be omitted, and step S609 may be the first step. Step S609 may be omitted, and if the user corresponds to a notification target user, if filtering using the notification target user list is invalid, or if the user corresponds to a non-notification target user, the process of step S610 may be performed. If filtering using the notification target user list is invalid, the process of step S610 may be performed.
[0061] According to the embodiment described above, virtual contact by another user with a user in a VR space or MR space is detected, and when virtual contact occurs, information on the position of the contact is notified to the contacted user. This makes it possible to use virtual contact between users, such as contact between avatars, as a sufficient communication tool. For example, the contacted person can understand where the contacted person has been touched and can naturally react by looking in the direction of the contact, such as turning to the right if the right shoulder is touched (from behind), or turning to the left if the left shoulder is touched.
[0062] Furthermore, according to this embodiment, not only the position of the contact but also the strength of the contact is notified to the contactee, which allows the contactee to react more naturally, such as not noticing a weak contact and being surprised by a strong contact.
[0063] The above-described embodiment (including the modified examples) is merely an example, and the present invention also includes configurations obtained by appropriately modifying or changing the configurations of the above-described embodiment within the scope of the gist of the present invention. The present invention also includes configurations obtained by appropriately combining the configurations of the above-described embodiment.
[0064] For example, an example in which virtual contact is detected on the contactee side (contactee's HMD) has been described, but the present invention is not limited to this. For example, virtual contact may be detected on the contactor side (contactor's HMD). Then, the detection result of the virtual contact may be notified from the contactor side to the contactee side. In this case, the notification of the detection result (information) that virtual contact has occurred on the contactor side can be interpreted as control for notifying the contactee of information regarding the position of the virtual contact. In this way, the information processing device to which the present invention can be applied is not limited to the device on the contactee side, but may be the device on the contactor side. The information processing device to which the present invention can be applied may be a server. Note that the acquisition of the detection result (information) that virtual contact has occurred on the contactee side can also be interpreted as detection of virtual contact.
[0065] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.
[0066] The disclosure of the present embodiment includes the following configuration, method, program, and medium. (Configuration 1) A detection means for detecting a virtual contact of a second user with a first user in a three-dimensional space which is a virtual space or a mixed reality space; a control means for controlling, when the virtual contact has occurred, to notify the first user of information relating to the position of the virtual contact; 13. An information processing device comprising: (Configuration 2) The detection means detects contact of an avatar of the second user with the first user in a three-dimensional space that is a mixed reality space. 2. The information processing device according to configuration 1. (Configuration 3) The detection means detects contact of the avatar of the second user with the avatar of the first user in a three-dimensional space that is a virtual space. 2. The information processing device according to configuration 1. (Configuration 4) The control means controls to notify the information regarding the position and intensity of the contact. 4. The information processing device according to any one of configurations 1 to 3. (Configuration 5) The control means controls to notify the information by display or sound. 5. The information processing device according to any one of configurations 1 to 4. (Configuration 6) The control means controls so as to notify the information by vibration. 6. The information processing device according to any one of configurations 1 to 5. (Configuration 7) If the second user is not included in the first list, the information is not notified. 7. The information processing device according to any one of configurations 1 to 6. (Configuration 8) If the second user is included in the second list, the information is not notified. 8. The information processing device according to any one of configurations 1 to 7. (Configuration 9) When the second user is present in the field of view of the first user in the three-dimensional space, the information is not notified. 9. The information processing device according to any one of configurations 1 to 8. (Configuration 10) The detection means detects a virtual contact of the hand of the second user with the first user. 10. The information processing device according to any one of configurations 1 to 9. (Configuration 11) At least a portion of a plurality of body parts of the second user are tracked; The detection means detects a virtual contact of a tracked part of a plurality of parts of the second user with the first user. 11. The information processing device according to any one of configurations 1 to 10. (Configuration 12) The detection means detects contact of a virtual object held by the second user with the first user. 12. The information processing device according to any one of configurations 1 to 11. (method) Detecting a virtual contact of a second user with a first user in a three-dimensional space, which is a virtual space or a mixed reality space; a step of controlling, when the virtual contact occurs, to notify the first user of information relating to the position of the virtual contact; 13. A method for controlling an information processing apparatus comprising the steps of: (program) 13. A program for causing a computer to function as each of the means of the information processing device according to any one of configurations 1 to 12. [Explanation of symbols]
[0067] 100: Head mounted display 201: Control unit
Claims
1. A detection means for detecting virtual contact between a second user and a first user in a three-dimensional space which is a virtual space or augmented reality space, Control means that, in the event of the aforementioned virtual contact, notifies the first user of information regarding the location of said contact. An information processing device characterized by having the following features.
2. The detection means detects contact between the second user's avatar and the first user in a three-dimensional space which is a mixed reality space. The information processing apparatus according to feature 1.
3. The detection means detects contact between the avatar of the second user and the avatar of the first user in a three-dimensional virtual space. The information processing apparatus according to feature 1.
4. The control means controls to notify the information regarding the position and intensity of the contact. The information processing apparatus according to feature 1.
5. The control means controls the notification of the information by display or sound. The information processing apparatus according to feature 1.
6. The control means controls the system to notify the information by vibration. The information processing apparatus according to feature 1.
7. If the second user is not included in the first list, the information will not be notified. The information processing apparatus according to feature 1.
8. If the second user is included in the second list, the information will not be notified. The information processing apparatus according to feature 1.
9. If the second user is within the first user's field of view in the aforementioned three-dimensional space, the information will not be communicated. The information processing apparatus according to feature 1.
10. The detection means detects a virtual contact between the first user and the second user's hand. The information processing apparatus according to feature 1.
11. At least a portion of multiple body parts of the second user is being tracked. The detection means detects virtual contact between the first user and a tracking body part among a plurality of body parts of the second user. The information processing apparatus according to feature 1.
12. The detection means detects contact between the first user and the virtual object held by the second user. The information processing apparatus according to feature 1.
13. The control means controls to calculate the direction of the position of the contact with respect to the field of view of the first user, and controls to notify the first user of the information based on the calculated direction. The information processing apparatus according to feature 1.
14. The control means controls the notification to notify the first user of the information by superimposing the notification item onto the image in the three-dimensional space. The information processing apparatus according to feature 1.
15. The display position of the notification item indicates the position of the contact, The display size of the notification item indicates the intensity of the contact. The information processing apparatus according to feature 14.
16. The detection means is The contact area information is obtained regarding the contact area, which is the part of the second user where the contact occurred. Based on the contact area information, it is determined whether the contact area is a tracking area or a hand. If the contact area is a tracking area or a hand, it is determined that the contact has been detected. The information processing apparatus according to feature 1.
17. A step of detecting a virtual contact between a second user and a first user in a three-dimensional space which is a virtual space or augmented reality space, The steps include: when the aforementioned virtual contact occurs, controlling the system to notify the first user of information regarding the location of the contact; A control method for an information processing device, characterized by having the following features.
18. A program for causing a computer to function as one of the means of an information processing apparatus described in any one of claims 1 to 16.