Information processing device, information processing method, and program

Abstract

An information processing device according to an embodiment of the present technology comprises an acquisition unit and a playback control unit. The acquisition unit acquires avoidance region information pertaining to an avoidance region into which entry by a target to whom virtual content is played back and / or entry by a virtual object for the target that is displayed to the target is to be avoided, the avoidance region information being generated on the basis of the position of an other object which differs from the target and / or the position of a virtual object for said other object that is displayed to said other object. The playback control unit controls playback of virtual content for the target on the basis of the acquired avoidance region information.

Description

[Technical Field]

[0001] The present technology relates to an information processing device, an information processing method, and a program that can be applied to constructing an AR (Augmented Reality) space. [Background technology]

[0002] Patent Document 1 discloses a technique that enables a user to operate a virtual object in AR without losing the user's sense of immersion in the AR space. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-212345 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a demand for technology that can deliver such high-quality AR experiences.

[0005] In view of the above circumstances, an object of the present technology is to provide an information processing device, an information processing method, and a program that can realize a high-quality AR experience. [Means for solving the problem]

[0006] In order to achieve the above object, an information processing device according to an embodiment of the present technology includes an acquisition unit and a playback control unit. The acquisition unit acquires avoidance area information regarding an avoidance area in which at least one of the entry of the subject or the entry of a virtual object for the subject displayed for the subject is to be avoided, the avoidance area being generated based on at least one of the position of another object different from the subject for whom virtual content is to be played or the position of a virtual object for the other object displayed for the other object. The playback control unit controls playback of virtual content for the target person based on the acquired avoidance area information.

[0007] In this information processing device, the playback of virtual content for a target person is controlled based on avoidance area information regarding an avoidance area in which at least one of the entry of the target person for whom virtual content is played back or the entry of a virtual object for the target person displayed for the target person is avoided, thereby realizing a high-quality AR experience.

[0008] The playback control unit may control at least one of displaying a virtual object for the subject and outputting a virtual sound to the subject.

[0009] The playback control unit may perform at least one of playback of the virtual content to prevent the target person from entering the avoidance area, or playback of the virtual content to prevent a virtual object for the target person from entering the avoidance area.

[0010] The playback control unit may move a position of a virtual object for the target person to outside the avoidance area.

[0011] The playback control unit may play back the virtual content for guiding the subject's line of sight so that a virtual object for the subject included in the avoidance area is outside the subject's effective field of view.

[0012] The information processing device may further include a generation unit that generates the avoidance area information based on surrounding information related to the surroundings of the target person.

[0013] The generation unit may generate the avoidance area information based on at least one of a movement of the other object and a movement of a virtual object for the other object.

[0014] The generation unit may generate the avoidance area information, which sets an area of ​​a predetermined range based on the position of the other object as the avoidance area.

[0015] The other object may be another target person who is a target of playback of virtual content by another information processing device.

[0016] The generation unit may generate avoidance area information for the other target person based on at least one of a position of the target person or a position of a virtual object for the target person.

[0017] The information processing device may further include a transmission control unit that transmits avoidance area information for the other target person to the other information processing device.

[0018] The information processing device may further include a priority calculation unit that calculates a priority regarding playback of the virtual content based on the avoidance area information.

[0019] The transmission control unit may transmit the calculated priority to the other information processing device.

[0020] The playback control unit may compare the priority calculated by the priority calculation unit with the priority for the other target person transmitted from the other information processing device, and if the priority for the other target person is higher, control the playback of the virtual content based on the avoidance area information transmitted from the other information processing device.

[0021] The playback control unit compares the priority calculated by the priority calculation unit with the priority for the other target person transmitted from the other information processing device, and if the priority for the other target person is lower, it does not need to control the playback of the virtual content based on the avoidance area information transmitted from the other information processing device.

[0022] The information processing device may further include an avoidance request level setting unit that sets an avoidance request level indicating a level of avoidance required for the avoidance area included in the avoidance area information. In this case, the playback control unit may control playback of the virtual content based on the avoidance request level.

[0023] The avoidance request degree setting unit may set the avoidance request degree for an avoidance area included in the avoidance area information for the other target user. In this case, the transmission control unit may transmit the set avoidance request degree to the other information processing device.

[0024] An information processing method according to one embodiment of the present technology is an information processing method executed by a computer system, The method includes acquiring avoidance area information regarding an avoidance area in which at least one of the entry of the subject or the entry of a virtual object for the subject that is displayed for the subject is to be avoided, the avoidance area information being generated based on at least one of the position of another object different from the subject for whom virtual content is to be played or the position of a virtual object for the other object that is displayed for the other object. Based on the obtained avoidance area information, playback of virtual content for the target person is controlled.

[0025] A program according to one embodiment of the present technology causes a computer system to execute the following steps. A step of acquiring avoidance area information regarding an avoidance area in which at least one of the entry of the subject or the entry of a virtual object for the subject displayed for the subject is to be avoided, the avoidance area being generated based on at least one of the position of another object different from the subject for whom virtual content is to be played, or the position of a virtual object for the other object displayed for the other object. Controlling the playback of virtual content for the target person based on the obtained avoidance area information. [Brief explanation of the drawings]

[0026] [Figure 1] 1 is a schematic diagram for explaining an overview of an AR provision system according to a first embodiment of the present technology. FIG. [Figure 2] 10 is a flowchart illustrating an example of a basic operation performed by the HMD. [Figure 3] FIG. 1 is a perspective view showing an example of the appearance of an HMD. [Figure 4] FIG. 2 is a block diagram showing an example of the functional configuration of an HMD. [Figure 5] 10 is a flowchart showing an example of an operation of playing back virtual content using an HMD. [Figure 6] FIG. 10 is a schematic diagram illustrating a display example of a virtual object. [Figure 7] FIG. 10 is a schematic diagram showing an example of setting an avoidance area. [Figure 8] FIG. 10 is a schematic diagram showing an example of setting an avoidance area. [Figure 9] FIG. 10 is a schematic diagram showing an example of setting an avoidance area. [Figure 10] FIG. 10 is a schematic diagram showing an example of playback control of virtual content based on an avoidance area. [Figure 11] FIG. 10 is a schematic diagram showing an example of playback control of virtual content based on an avoidance area. [Figure 12] FIG. 10 is a schematic diagram showing an example of playback control of virtual content based on an avoidance area. [Figure 13] FIG. 10 is a schematic diagram showing an example of playback control of virtual content based on an avoidance area. [Figure 14] FIG. 10 is a schematic diagram showing an example of playback control of virtual content based on an avoidance area. [Figure 15] FIG. 10 is a schematic diagram illustrating an overview of an AR provision system according to a second embodiment. [Figure 16] FIG. 2 is a block diagram showing an example of the functional configuration of an HMD. [Figure 17] 10 is a flowchart showing an example of an operation of playing back virtual content using an HMD. [Figure 18] FIG. 10 is a schematic diagram for explaining an example of determining another nearby user. [Figure 19]FIG. 10 is a schematic diagram showing an example of setting an avoidance area for other users. [Figure 20] FIG. 10 is a schematic diagram showing an example of setting an avoidance area for other users. [Figure 21] FIG. 10 is a schematic diagram showing an example of setting an avoidance area for other users. [Figure 22] FIG. 10 is a schematic diagram showing an example of setting an avoidance area for another user. [Figure 23] FIG. 10 is a schematic diagram showing an example of setting an avoidance area for other users. [Figure 24] FIG. 10 is a schematic diagram showing an example of setting an avoidance area for another user. [Figure 25] FIG. 10 is a schematic diagram showing an example of playback control of virtual content. [Figure 26] FIG. 10 is a schematic diagram showing an example of playback control of virtual content. [Figure 27] FIG. 10 is a schematic diagram showing an example of playback control of virtual content. [Figure 28] FIG. 10 is a schematic diagram showing an example of playback control of virtual content. [Figure 29] FIG. 10 is a schematic diagram for explaining the setting of an avoidance request level. [Figure 30] FIG. 10 is a schematic diagram for explaining the setting of an avoidance request level. [Figure 31] FIG. 10 is a schematic diagram for explaining the setting of an avoidance request level. [Figure 32] FIG. 10 is a schematic diagram for explaining the setting of an avoidance request level. [Figure 33] FIG. 10 is a schematic diagram for explaining the setting of an avoidance request level. [Figure 34] FIG. 10 is a schematic diagram showing an example of regeneration control based on an avoidance request degree. [Figure 35] FIG. 10 is a block diagram showing an example of the functional configuration of an HMD according to another embodiment. [Figure 36] FIG. 1 is a block diagram illustrating an example of the hardware configuration of a computer to which the present technology can be applied. DETAILED DESCRIPTION OF THE INVENTION

[0027] Hereinafter, embodiments of the present technology will be described with reference to the drawings.

[0028] First Embodiment [AR provision system] 1 is a schematic diagram illustrating an overview of an AR provision system according to a first embodiment of the present technology. The AR provision system 1 corresponds to an embodiment of an information processing system according to the present technology.

[0029] The AR providing system 1 includes an HMD (Head Mount Display) 2. As shown in FIG. 1, the HMD 2 is worn on the head of a user 3 when in use. The HMD2 is a glasses-type device equipped with a see-through display, and is also known as AR glasses. In this embodiment, the HMD 2 plays back virtual content for the user 3. This makes it possible to provide an AR space to the user 3 who uses the HMD 2. By using the HMD 2, the user 3 can experience various AR worlds.

[0030] The reproduction of virtual content includes displaying virtual objects so as to be superimposed on the real world. The reproduction of virtual content also includes outputting virtual sounds (virtual voices). In addition, smells, tactile sensations, etc. may be virtually provided to the user 3.

[0031] As shown in Figure 1, an AR space (World A) is constructed for a user 3 based on the real world (World R). For example, an AR world in which cute monsters play an active role as virtual objects 4 is constructed in a city in the real world. Of course, not only characters such as monsters but also various other virtual objects that constitute the AR space, such as buildings and signs, may be displayed as the virtual object 4. That is, any image, such as CG (Computer Graphics) of a character, a photo, or text, may be displayed as the virtual object. As the virtual sound, any sound may be output, such as the cry of a character such as a monster, a siren sound, or the sound effect of a door closing.

[0032] In this embodiment, the HMD 2 functions as an embodiment of an information processing device according to the present technology. Also, a user 3 wearing the HMD 2 is an embodiment of a target person for whom virtual content is to be reproduced.

[0033] [Basic operation] FIG. 2 is a flowchart showing an example of a basic operation executed by the HMD 2. The HMD 2 acquires avoidance area information (step 101). The avoidance area information is information generated based on at least one of the position of another object different from the user 3 who is the target of playing the virtual content, or the position of a virtual object for that other object that is displayed relative to that other object. For example, avoidance area information may be generated for any object, such as a person, a moving object such as a vehicle, or the like, that is present around the user 3. Alternatively, the type of object for which avoidance area information is to be generated may be specified as appropriate. Furthermore, if there is another user using the HMD 2, the avoidance area information may be generated based on the position of a virtual object for that other user that is displayed to that other user.

[0034] The avoidance area information is information about an avoidance area where at least one of the entry of the user 3 and the entry of the virtual object 4 for the user 3 that is displayed to the user 3 is to be avoided. For example, the avoidance area may be defined as an area in which both the entry of the user 3 and the entry of the virtual object 4 for the user 3 are avoided. Alternatively, the avoidance area may be defined as an area in which only the entry of either the user 3 or the entry of the virtual object 4 for the user 3 is avoided. In either case, the present technology is applicable. In addition, in the present disclosure, entering the avoidance area includes not only the action of entering the avoidance area from outside the avoidance area, but also the state of already entering the avoidance area. Therefore, avoiding entry into the avoidance area includes not only avoiding the action of entering the avoidance area, but also avoiding a state in which the vehicle has already entered the avoidance area. Avoiding a state in which the vehicle has entered the avoidance area can also be said to be canceling the state in which the vehicle has entered the avoidance area. For example, moving a user 3 or a virtual object 4 that is present within the avoidance area outside the avoidance area corresponds to canceling the state of entry into the avoidance area, and is included in avoiding entry into the avoidance area.

[0035] Furthermore, the manner in which the avoidance area information is acquired is not limited. For example, the avoidance area information may be acquired by generating the avoidance area information by the HMD 2. Alternatively, the avoidance area information may be acquired by receiving the avoidance area information generated by another device or by reading out the avoidance area information stored in a recording medium or the like. Any other acquisition mode that makes the avoidance area information available is also included. Therefore, in step 101, the avoidance area information may be generated by the HMD 2, or the HMD 2 may receive avoidance area information generated by another device.

[0036] The HMD 2 controls the reproduction of the virtual content for the user 3 based on the avoidance area information (step 102). For example, a virtual object is displayed, a virtual sound is output, or the like is executed to prevent the user 3 from entering the avoidance area included in the avoidance area information. Alternatively, a virtual object is displayed, a virtual sound is output, or the like is executed to prevent the virtual object 4 for the user 3 from entering the avoidance area included in the avoidance area information. This makes it possible to avoid situations where interference with surrounding people or other people disrupts the worldview of the AR space you are enjoying. Furthermore, this avoidance can be achieved without disrupting the worldview of the AR space. As a result, it is possible to achieve a high-quality AR experience.

[0037] For example, when you are having a face-to-face conversation with the virtual monster object 4, if a person running crosses your path, the virtual monster object 4 may be displayed superimposed on the person. In this case, the worldview of the AR world may be disrupted. Also, suppose that a map and an arrow indicating a running route are displayed as virtual objects 4, and while the user 3 is enjoying a run along the arrow, the arrow is superimposed on an approaching person. In this case, the user 3 will naturally have to deviate from the route indicated by the arrow to avoid the person, which may prevent the user 3 from enjoying the run. Furthermore, the above-described interference may occur between a user 3 enjoying the AR world of the monster virtual object 4 and a user 3 enjoying the AR world for running. In this case, the worldview of the AR world may be disrupted for both users 3. Furthermore, if the positions of the virtual monster object 4 and the virtual arrow object 4 indicating the running route overlap, there is a high possibility that interference (such as a collision) will occur between the users 3. Therefore, the proximity of the virtual objects 4 to each other may also be a problem.

[0038] In this embodiment, playback control of virtual content is executed based on avoidance area information, which makes it possible to sufficiently mitigate the above-mentioned problems and realize a high-quality AR experience. Note that the playback control of the virtual content based on the avoidance area information is not limited to the playback control that prevents the user 3 or the virtual object 4 for the user 3 from entering the avoidance area. Various playback controls may be executed based on the avoidance area to avoid situations in which the worldview of the AR space is disrupted. 2 is repeated at a predetermined frame rate, but is not limited to being executed for each frame.

[0039] [HMD] FIG. 3 is a perspective view showing an example of the appearance of the HMD 2. As shown in FIG. The HMD 2 has a frame 6, a left-eye lens 7a and a right-eye lens 7b, a left-eye display 8a and a right-eye display 8b, a left-eye camera 9a and a right-eye camera 9b, and an outward-facing camera 10.

[0040] The frame 6 is shaped like glasses and has a rim portion 11 and temple portions 12 . The rim portions 11 are portions that are placed in front of the left and right eyes of the user 3, and support the left eye lens 7a and the right eye lens 7b, respectively. The temples 12 extend rearward from both ends of the rim 11 toward the ears of the user 3, and their tips are worn on the ears. The rim 11 and temples 12 are formed from a material such as synthetic resin or metal.

[0041] The left eye lens 7a and the right eye lens 7b are disposed in front of the left and right eyes of the user 3, respectively, so as to cover at least a part of the visual field of the user 3. Typically, each lens is designed to correct the vision of the user 3. Of course, this is not limiting, and so-called non-prescription lenses may also be used.

[0042] The left-eye display 8a and the right-eye display 8b are transmissive displays and are arranged so as to cover part of the areas of the left-eye and right-eye lenses 7a and 7b, respectively. That is, the left-eye and right-eye displays 8a and 8b are arranged in front of the left and right eyes of the user 3, respectively. The left-eye and right-eye displays 8a and 8b display images for the left and right eyes, respectively. A user 3 wearing the HMD 2 can view the real scenery and, at the same time, view the images displayed on the displays 8a and 8b. This allows the user 3 to experience augmented reality (AR) and the like.

[0043] For example, a virtual object 4 is displayed on each of the displays 8a and 8b. As the left-eye and right-eye displays 8a and 8b, for example, a transmissive organic EL display or an LCD (Liquid Crystal Display) display or the like is used. In addition, the specific configuration of the left-eye and right-eye displays 8a and 8b is not limited, and any type of transmissive display may be used as appropriate, such as a type that displays an image by projecting it onto a transparent screen or a type that displays an image using a prism or the like.

[0044] The left eye camera 9a and the right eye camera 9b are appropriately installed on the frame 6 so as to be able to capture images of the left eye and right eye of the user 3. For example, it is possible to detect line-of-sight information and the like relating to the line of sight of the user 3 based on the images of the left eye and right eye captured by the left eye and right eye cameras 9a and 9b. The left-eye and right-eye cameras 9a and 9b may be digital cameras equipped with image sensors such as a CMOS (Complementary Metal-Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor. Alternatively, an infrared camera equipped with infrared illumination such as an infrared LED may be used.

[0045] Hereinafter, the left-eye lens 7a and the right-eye lens 7b may both be referred to as lenses 7, and the left-eye display 8a and the right-eye display 8b may both be referred to as transmissive displays 8. Furthermore, the left-eye camera 9a and the right-eye camera 9b may both be referred to as inward-facing cameras 9. In this embodiment, the transmissive displays 8 correspond to the display unit.

[0046] Outward-facing camera 10 is positioned in the center of frame 6 (rim portion 11) facing outward (the opposite side from user 3). Outward-facing camera 10 is capable of capturing images of real space included in the field of view of user 3. Therefore, outward-facing camera 10 is capable of generating captured images of real space.

[0047] In this embodiment, the outward-facing camera 10 captures an area in front of the user 3, including the display area of ​​the transmissive display 8. In other words, the real space is captured so as to include the area that can be seen through the display area as viewed from the user 3. As the outward-facing camera 10, for example, a digital camera equipped with an image sensor such as a CMOS sensor or a CCD sensor is used. The range that can be seen through the display area from the user 3 is the range in which the virtual object 4 can be superimposed on the real world. In the present disclosure, this range is referred to as the effective field of view of the user 3. The effective field of view can also be referred to as the angle of view in which the virtual object 4 can be displayed.

[0048] FIG. 4 is a block diagram showing an example of the functional configuration of the HMD 2. As shown in FIG. 4, the HMD 2 further includes a speaker 14, a vibration unit 15, a communication unit 16, a connector 17, an operation button 18, a sensor unit 19, a storage unit 20, and a controller 21.

[0049] The speaker 14 is provided at a predetermined position on the frame 6. The configuration of the speaker 14 is not limited, and for example, a speaker 14 capable of outputting stereo sound, monaural sound, or the like may be used as appropriate. The vibration unit 15 is provided inside the frame 6 and generates vibrations. For example, any vibration motor or the like that can generate vibrations for notifications or the like is used as the vibration unit 15. The communication unit 16 is a module for executing network communication, short-distance wireless communication, etc. with other devices. For example, a wireless LAN module such as WiFi, or a communication module such as Bluetooth (registered trademark) is provided.

[0050] Connector 17 is a terminal for connecting to other devices. For example, a terminal such as a USB (Universal Serial Bus) or an HDMI (High-Definition Multimedia Interface) (registered trademark) is provided. During charging, a charging terminal of a charging dock (cradle) is connected to connector 17 to perform charging. The operation button 18 is provided, for example, at a predetermined position on the frame 6. The operation button 18 can be used to perform operations related to various functions of the HMD 2, such as power on / off operations, image display and audio output functions, and network communication functions.

[0051] The sensor unit 19 includes a nine-axis sensor 22 , a GPS 23 , a distance measurement sensor 24 , and a microphone 25 . The nine-axis sensor 22 includes a three-axis acceleration sensor, a three-axis gyro sensor, and a three-axis compass sensor. The nine-axis sensor 22 can detect the three-axis acceleration, angular velocity, and orientation of the HMD 2. Alternatively, an IMU (Inertial Measurement Unit) sensor having any configuration may be used. The GPS 23 acquires information about the current position of the HMD 2. The detection results of the 9-axis sensor 22 and the GPS 23 are used to detect, for example, the posture and position of the user 3 (HMD 2) and the movement (motion) of the user 3. These sensors are provided at predetermined positions on the frame 6, for example.

[0052] The distance measurement sensor 24 is capable of acquiring three-dimensional information (distance to a detection target). Examples of such sensors include LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), laser distance measurement sensors, stereo cameras, ToF (Time of Flight) sensors, ultrasonic sensors, and structured light type distance measurement sensors. A sensor having the functions of both an image sensor and a distance measurement sensor may also be used. The distance measurement sensor 24 is installed, for example, with its detection direction facing forward of the user 3. That is, the distance measurement sensor 24 is installed so as to be able to measure distances to the real space included in the field of view of the user 3. However, the present invention is not limited to this, and the distance measurement sensor 24 may be installed so as to be able to measure distances in a 360-degree periphery around the user 3. The microphone 25 detects sound information around the user 3. For example, the microphone 25 appropriately detects the voice uttered by the user 3. This allows the user 3 to, for example, enjoy an AR experience while having a voice call or to perform operation input for the HMD 2 using voice input.

[0053] The type of sensor provided as the sensor unit 19 is not limited, and any sensor may be provided. For example, a temperature sensor or a humidity sensor capable of measuring the temperature, humidity, etc. of the environment in which the HMD 2 is used may be provided. Also, a biosensor capable of detecting bioinformation of the user 3 may be provided. For example, a brain wave sensor, an electromyographic sensor, a pulse sensor, a sweat sensor, a temperature sensor, a blood flow sensor, a body movement sensor, etc. may be provided as the biosensor. The inward-facing camera 9 and the outward-facing camera 10 can also be considered as part of the sensor unit 19 .

[0054] The storage unit 20 is a storage device such as a nonvolatile memory, and may be, for example, a hard disk drive (HDD) or a solid state drive (SSD), etc. Alternatively, any other computer-readable non-transitory storage medium may be used. The storage unit 20 stores, for example, map data. The map data is data that functions as a map of real space. For example, map data related to the real world in which the user 3 is experiencing the AR world is stored. The storage unit 20 also stores a control program for controlling the overall operation of the HMD 2. The storage unit 20 also stores various information related to the AR application that provides the AR space. For example, content data such as the virtual object 4 and virtual sound, and various information and data related to the present technology, such as the avoidance area information, are stored. The method for installing map data, control programs, content data, etc. into the HMD 2 is not limited. For example, an AR application according to the present technology can be provided as a cloud service. In this case, an application program for constructing an AR space is installed on the HMD 2 from a content provider server on the network. However, the present technology is not limited to this configuration.

[0055] The controller 21 controls the operation of each block of the HMD 2. The controller 21 has hardware circuits necessary for a computer, such as a CPU and memory (RAM, ROM), etc. The CPU executes programs according to the present technology stored in the storage unit 20 and the memory, thereby performing various processes. The controller 21 may be, for example, a programmable logic device (PLD) such as a field programmable gate array (FPGA), or another device such as an application specific integrated circuit (ASIC).

[0056] In this embodiment, the CPU of the controller 21 executes a program (e.g., an application program) related to the present technology, thereby realizing functional blocks including an information acquisition unit 26, a surrounding environment determination unit 27, an avoidance area information generation unit 28, an AR playback control unit 29, and a transmission control unit 30. These functional blocks execute the information processing method according to this embodiment. Note that dedicated hardware such as an IC (integrated circuit) may be used as appropriate to realize each functional block.

[0057] The information acquisition unit 26 is capable of acquiring various pieces of information from, for example, each part of the HMD 2 and outputting the information to each functional block of the controller 21. For example, information acquisition unit 26 can acquire image information captured by inward-facing camera 9 and outward-facing camera 10, detection results of each sensor of sensor unit 19 (sensor information), and the like. The information acquisition unit can also acquire various pieces of information received from other devices via the communication unit 16. For example, it is possible to receive various information related to the AR application that the user 3 is experiencing from a predetermined server on the network, etc. For example, reservation information for a location to experience the AR world, the number of participating users experiencing the same AR world, billing information for experiencing the AR world, etc. may be acquired. It may also be possible to acquire images from cameras installed on walls, poles, etc. in the real world where the AR world is constructed, and sensor information such as distance measurement sensors.

[0058] The surrounding environment determination unit 27 generates surrounding information relating to the surroundings of the user 3 (HMD 2). For example, the surrounding information determines the surrounding environment of user 3 based on image information captured by inward-facing camera 9 and outward-facing camera 10, detection results (sensor information) of each sensor of sensor unit 19, and the like. Images from cameras installed on walls or poles in the real world where the AR world is constructed, sensor information from distance sensors, etc. may also be used, making it possible to generate surrounding information with high accuracy. For example, the following information can be given as peripheral information: Presence or absence of an object: The range to be determined may be specified, such as the presence or absence of a nearby object. Object types: people, animals, vehicles, trains, roads, trees, buildings, stationary objects, moving objects, etc. Object state: object orientation, posture, stationary state, in motion, etc. Object movement: walking (including direction and speed), running (including direction and speed), jumping (including height), talking, etc. Shape of the object (including size, etc.) Expected object movement Distance to object Any other information relating to the surroundings of the user 3 may be generated.

[0059] The method for generating the peripheral information is not limited, and any technique may be used. For example, any image recognition technique may be used, such as matching processing using a model image of the object, edge detection, projective transformation, etc. Furthermore, skeletal estimation (bone estimation) or the like may also be used. In addition, an externally constructed library with existing functions such as image processing and machine learning may be used. To generate the peripheral information, any machine learning algorithm using, for example, a DNN (Deep Neural Network) may be used. For example, by using AI (artificial intelligence) that performs deep learning, it is possible to improve the accuracy of generating the peripheral information. For example, by performing semantic segmentation on image information, it is possible to determine the type of object for each pixel in the image. It should be noted that the application of machine learning algorithms may be performed on any process within the present disclosure.

[0060] Furthermore, in this embodiment, the surrounding environment determination unit 27 defines a coordinate system for the space within the effective field of view on which the virtual object 4 can be superimposed. For example, coordinate values ​​(e.g., XYZ coordinate values) defined by an absolute coordinate system (world coordinate system) may be used. Alternatively, coordinate values ​​(e.g., XYZ coordinate values ​​or UVD coordinate values) defined by a relative coordinate system with a predetermined point as the reference (origin) may be used. When a relative coordinate system is used, the reference origin may be set arbitrarily. For example, a relative coordinate system may be defined that has the world coordinates of a predetermined real object within the field of view as its origin. The surrounding environment determination unit 27 generates surrounding information such as the position and orientation of an object by appropriately using the defined coordinate values.

[0061] The shooting range of outward-facing camera 10 and the measurement range of distance sensor 24 may be set to face not only the space within the effective field of view but also the left, right, or rearward. In this case, a coordinate system may be defined for the space outside the effective field of view, and peripheral information may be generated. For example, the range in which coordinates are defined and the range in which peripheral information is generated may be set appropriately to one or more directions of front, back, left, right, or multiple directions, or the entire 360-degree perimeter.

[0062] The surrounding environment determination unit 27 may estimate the self-position of the user 3 (HMD 2). For example, the self-position includes the position and orientation of the HMD 2. For example, by estimating the self-position, it is possible to calculate position information of the HMD 2 within the map data and orientation information such as the direction in which the HMD 2 is facing. The self-position of HMD 2 is calculated based on, for example, the detection result from sensor unit 19 and the images captured by inward-facing camera 9 and outward-facing camera 10. For example, the calculation unit 21 calculates position coordinates in a three-dimensional coordinate system (XYZ coordinate system) defined by the surrounding environment determination unit 27. In addition, the calculation unit 21 calculates the pitch angle, roll angle, and yaw angle of a predetermined reference axis extending in front of the user 3 (HMD 2) when the X axis is the pitch axis, the Y axis is the roll axis, and the Z axis is the yaw axis. Of course, the specific format of the position information and posture information of the user 3 (HMD 2) is not limited. The algorithm for estimating the self-position of the HMD 2 is not limited, and any algorithm such as SLAM (Simultaneous Localization and Mapping) may be used. Alternatively, any machine learning algorithm may be used. Based on the self-position, three-dimensional coordinates of the surroundings may be defined. Furthermore, the self-position estimation unit may be configured as a functional block different from the surrounding environment determination unit 27.

[0063] The avoidance area information generating unit 28 generates the avoidance area information. In this embodiment, the avoidance area information is generated based on the surrounding information.

[0064] The AR playback control unit 29 controls the playback of the virtual content for the user 3 . For example, how the virtual object 4 moves and how the virtual sound is heard are determined according to the worldview of the AR world. Then, the virtual object 4 is displayed on the transparent display 8 so that the determined content is realized. Also, the virtual sound is output from the speaker 14. Regarding the display of the virtual object 4, for example, the display position of the virtual object 4 is calculated based on the three-dimensional coordinates defined by the surrounding environment determination unit 27. The calculated display position (three-dimensional coordinates) is converted into two-dimensional coordinates (display coordinates on the transmissive display 8) by projective transformation or the like. The virtual object 4 is displayed at the converted display coordinates. This realizes an AR space in which the virtual object 4 exists at a desired position in the real space. Regarding the output of virtual sound, the position where the virtual sound is generated (the position of the virtual sound source) is calculated based on, for example, three-dimensional coordinates defined by the surrounding environment determination unit 27. By controlling the speaker 14 and adjusting the localization of the sound, an AR space is realized in which the virtual sound is heard from a desired position (desired direction) in the real space. There are no particular limitations on the algorithm for playing back the virtual content, and any control may be performed.

[0065] The transmission control unit 30 can transmit various information to other devices via the communication unit 16. For example, the transmission control unit 30 can transmit to other devices information such as information acquired by the information acquisition unit 26, surrounding information generated by the surrounding environment determination unit 27, avoidance area information generated by the avoidance area information generation unit 28, the display position of the virtual object 4 calculated by the AR playback control unit 29, and information stored in the storage unit 20.

[0066] In the present embodiment, the avoidance area information generation unit 28 corresponds to an embodiment of a generation unit according to the present technology. In the present embodiment, the avoidance area information generation unit 28 generates and acquires the avoidance area information. Therefore, the avoidance area information generation unit 28 also functions as an embodiment of an acquisition unit according to the present technology. The AR playback control unit 29 corresponds to an embodiment of the playback control unit according to the present technology.

[0067] [Virtual content playback behavior] FIG. 5 is a flowchart showing an example of the operation of playing back virtual content using the HMD 2. 6 to 14 are schematic diagrams for explaining the steps shown in FIG. In this embodiment, the AR space is realized by executing the process shown in Fig. 5 at a predetermined frame rate, but the present invention is not limited to this.

[0068] The surrounding environment determining unit 27 generates surrounding information (step 201). Moreover, the surrounding environment determining unit 27 determines whether or not there is an object nearby (step 202). For example, it is determined whether an object exists within a range up to a predetermined distance from the user 3 (HMD 2) within the detection range of the outward-facing camera 10 and the distance measurement sensor 24. For example, it is determined whether an object exists within a range of 10 m from the user 3 (HMD 2). Of course, the range is not limited to 10 m, and any range may be set.

[0069] As shown in FIG. 6, if there is no nearby object around the user 3 viewing the virtual object 4 using the HMD 2 (No in step 202), the AR playback control unit 29 executes playback control of the normal virtual content (step 203). The normal playback control of virtual content is not based on the avoidance area information, and for example, the display of the virtual object 4 and the output of virtual sound are executed in accordance with a normal AR application. User 3 can fully enjoy the worldview of the AR world.

[0070] 7 to 9, it is assumed that another person 32 is present nearby around the user 3. In this case, it is determined that there is a nearby object (Yes in step 202), and the avoidance area information generating unit 28 generates avoidance area information (step 204). In the examples shown in FIGS. 7 to 9, avoidance area information is generated in which the avoidance area 33 is an area of ​​a predetermined range based on the position (center position, etc.) of the other person 32. If a person or other object is present, the area around the person or other object is set as an avoidance area 33 since there is a high possibility that the person or other object will move.

[0071] 7, a circular area with a radius of a predetermined length and centered on the other person 32 is set as the avoidance area 33. Of course, the shape of the avoidance area 33 is not limited, and it may be an ellipse or a polygon. Furthermore, the size of the avoidance area 33 is not limited, and may be set to any arbitrary value, such as a radius of 3 m.

[0072] 8, the direction of another person 32 is used as a parameter to set the avoidance area 33. Since moving objects including people often move in the direction they are facing, the avoidance area 33 is set based on the direction. For example, an avoidance area 33 having a predetermined shape such as a triangle, a square, a circle, an ellipse, or a polygon is set in front of the other person 32 along the direction in which the other person 32 is facing. 8, when an isosceles triangular avoidance area 33 is used, the vertex of the isosceles triangle is aligned with the position of the other person 32. Then, the perpendicular line extending from the vertex angle to the base of the isosceles triangle is aligned with the direction in which the other person 32 is facing. Alternatively, any setting method that reflects the orientation of the other person 32 may be adopted.

[0073] 9, the movement of the other person 32 is used as a parameter. Specifically, the movement of the other person 32 is predicted, and a motion vector representing the predicted movement direction (orientation) and movement speed of the other person 32 is calculated. An avoidance area 33 is set based on the motion vector. For example, an avoidance area 33 having a predetermined shape such as a triangle, a rectangle, a circle, an ellipse, or a polygon is set in front of the other person 32 along the direction of the motion vector. The size of the avoidance area 33 may be adjusted as appropriate depending on the size of the motion vector. 9, when an isosceles triangular avoidance area 33 is used, the vertex of the isosceles triangle is aligned with the position of another person 32. Then, the perpendicular line extending from the vertex to the base of the isosceles triangle is aligned with the direction of the motion vector. Alternatively, the isosceles triangular avoidance area 33 may be set so that the magnitude of the perpendicular line matches the magnitude of the motion vector. In this way, avoidance area information may be generated based on the movement of the other person 32. By performing movement prediction, it becomes possible to set the avoidance area 33 with high accuracy.

[0074] The avoidance area 33 is set based on three-dimensional coordinates defined by the surrounding environment determination unit 27, for example. A three-dimensional area may be set as the avoidance area 33. For example, the avoidance area 33 may be set to have a conical shape, a square pyramid shape, or the like, extending forward of the other person 32, with the position of the other person 32 as its apex. Typically, the avoidance area 33 is not displayed so as to be visible to the user 3, but is instead stored as data. This makes it possible to prevent the worldview of the AR world from being disrupted. On the other hand, when applying this technology, it is not excluded that the avoidance area 33 may be displayed to the user 3.

[0075] Returning to FIG. 5, the AR playback control unit 29 controls the playback of the virtual content based on the avoidance area information (step 205). 10 and 11, playback of virtual content is executed to prevent the virtual object 4 from entering the avoidance area 33. That is, playback control is executed to prevent the virtual object 4 from entering the avoidance area 33. The virtual content is typically played back while maintaining the worldview of the AR world.

[0076] As shown in FIG. 10, if the virtual object 4 exists within the avoidance area 33, that is, if the position of the virtual object 4 is included within the avoidance area 33, the position of the virtual object 4 is moved outside the avoidance area 33. For example, the virtual object 4 is moved in a natural manner so as not to disrupt the worldview of the AR world. For example, a monster character is displayed walking, running, jumping, or the like outside the avoidance area 33. For example, when an arrow indicating a running route is displayed as the virtual object 4, the display of the arrow is changed so that the route avoids the avoidance area 33. Any other AR representation may be employed.

[0077] As shown in FIG. 11, virtual content may be played back to guide the line of sight of the user 3 so that the virtual object 4 for the user 3 included in the avoidance area 33 is outside the effective field of view of the user 3. 11, a virtual object 4 of another monster character is displayed on the opposite side of the virtual object 4 included in the avoidance area 33. This makes it possible to make the user 3 turn around and move the virtual object 4 that overlaps with the avoidance area 33 out of the effective field of view. For example, the virtual object 4 of another character may be displayed at any position that does not fall within the effective field of view (angle of view) of the original virtual object 4. For example, in the case of AR glasses with a horizontal angle of view of 90 degrees, if the original virtual object 4 is located at the center of the angle of view, another virtual object 4 is displayed between 45 degrees and 315 degrees, with the forward direction being 0 degrees. In this way, by directing the interest of the user 3 in another direction, it is possible to prevent the virtual object 4 from entering the avoidance area 33. In this case, it can be said that viewing in a state where the virtual object 4 has entered the avoidance area 33 is avoided. Any other playback control may be executed as the playback of the virtual content to guide the line of sight of the user 3. For example, a virtual sound like a monster calling out to the user 3 from the opposite side may be output. Any other AR representation may be employed.

[0078] The playback control based on the avoidance area information may end, for example, when the virtual object 4 is avoided from entering the avoidance area 33. Thereafter, normal playback control may be resumed. Alternatively, it may be determined whether the virtual object 4 would have been prevented from entering the avoidance area 33 if the original playback control had been executed. If the determination result is positive and it is determined that there is no problem in returning to normal playback control, the playback control may be switched to normal playback control. Furthermore, as playback control based on avoidance area information, the content of playback control may be set to differ depending on the environment in which the user 3 is located, such as morning / night, city / countryside, etc.

[0079] 12 to 14, playback of virtual content may be executed to prevent the user 3 from entering the avoidance area 33. In other words, playback control may be executed to prevent the user 3 from entering the avoidance area 33. The virtual content is typically played back while maintaining the worldview of the AR world.

[0080] 12, a virtual display is executed in which virtual objects 4 made up of multiple monsters dance around the periphery 34 of the avoidance area 33, blocking the path of the user 3. This makes it possible for the user 3 to avoid entering the avoidance area 33. 12 , various virtual displays may be executed, such as a virtual object 4 such as a character moving between the avoidance area 33 and the user 3 or talking to the user 3. Such virtual displays make it possible to prevent the user 3 from moving to the other side of the virtual object 4 such as a character.

[0081] In the example shown in FIG. 13, virtual content for stopping the movement of the user 3 is being played back. Specifically, a virtual display is executed in which the monster virtual object 4 dances at the feet of the user 3, so close that the user 3 cannot move. This makes it possible for the user 3 to avoid entering the avoidance area 33. Any other playback control may be executed that can stop the movement of the user 3 while maintaining the worldview of the AR world. For example, a virtual sound such as "Close your eyes right there and chant a spell" may be output in the voice of a monster. Or, a virtual sound such as "Stop right there and catch your breath" may be output in the voice of a running trainer.

[0082] In the example shown in FIG. 14, a virtual pitfall object 4 is displayed on the periphery 34 of the avoidance area 33. Depending on the content of the AR world, pitfalls may deviate from the worldview. On the other hand, pitfalls are widely known as objects that should not be entered, and are effective objects for stopping the progress of the user 3. Therefore, by displaying a virtual pitfall object 4 between the avoidance area 33 and the user 3, it is possible to avoid entering the avoidance area 33. In this way, the user 3 may be prevented from entering the avoidance area 33 by using a virtual object that deviates from the worldview of the AR world. For example, first, playback control is executed while maintaining the worldview of the AR world, as shown in Fig. 12 and Fig. 13, so as to prevent the user 3 from approaching the avoidance area 33. After that, if the user 3 inevitably gets too close to the avoidance area 33, a virtual pitfall object 4 is displayed, as shown in Fig. 14. Such a processing flow can also be adopted. Regarding the display of the virtual object 4, a method of subtracting the real world, such as a pitfall, may be adopted, or a method of adding to the real world, such as displaying a no entry sign, may be adopted.

[0083] In order to maintain the worldview of the AR world, it is effective to execute normal playback of virtual content as much as possible. Alternatively, it is desirable to minimize changes in playback control based on avoidance area information relative to normal playback control. From this perspective, the specific content of the playback control based on the avoidance area information may be determined as appropriate. For example, it is possible to set only the playback of virtual content for preventing the virtual object 4 from entering the avoidance area 33 as shown in Fig. 10 and Fig. 11, and not the playback of virtual content for preventing the user 3 from entering the avoidance area 33 as shown in Fig. 12 to Fig. 14. For example, depending on the content of the AR application, it may be possible that other persons 32, etc. may be able to adequately avoid the overlap between user 3 and avoidance area 33 by simply avoiding the overlap between virtual object 4 and avoidance area 33. The specific content of the playback control based on the avoidance area information may be determined appropriately based on the content of the AR application.

[0084] As described above, in the HMD 2 according to this embodiment, the playback of virtual content for user 3 is controlled based on avoidance area information regarding the avoidance area 33 in which at least one of the entry of user 3, who is the target of playing back virtual content, or the entry of a virtual object for user 3 displayed to user 3, is avoided. This makes it possible to avoid situations where interference with surrounding people or other people disrupts the worldview of the AR space you are enjoying. Furthermore, this avoidance can be achieved without disrupting the worldview of the AR space. As a result, it is possible to achieve a high-quality AR experience.

[0085] <Second embodiment> An AR provision system according to a second embodiment of the present technology will be described. In the following description, the description of the same configurations and operations as those of the AR provision system 1 and the HMD (information processing device) 2 described in the above embodiment will be omitted or simplified.

[0086] [AR provision system] FIG. 15 is a schematic diagram illustrating an overview of the AR provision system according to the second embodiment. The AR provision system 1 according to this embodiment is effective in a situation where multiple users 3 are wearing the HMD 2 and enjoying different AR worlds. That is, the AR provision system 1 functions with multiple HMDs 2. The following description will be given taking two HMDs 2a and 2b as an example, as shown in FIG.

[0087] 15, an AR space (World A) is constructed for a user 3a wearing an HMD 2a, based on the real world (World R). For example, an AR world in which cute monsters play an active role as virtual objects 4a is constructed in a city in the real world. As the virtual sound, any sound may be output, such as the cry of a character such as a monster, a siren sound, or the sound effect of a door closing. For user 3b wearing HMD 2b, an AR space (World B) is constructed based on the real world (World R). For example, an AR world is constructed in the middle of a city in the real world, where dancers sing and dance as virtual objects 4b. As the virtual sound, any sound may be output, such as the singing voices of dancers, the sound of stepping on the ground, or music playing in the background.

[0088] User 3a cannot see virtual object 4 (dancer) in AR space (World B), nor can he hear virtual sounds generated in AR space (World B). Similarly, user 3b cannot see virtual object 4 (monster) in AR space (World A), nor can he hear virtual sounds generated in AR space (World A). Of course, not only the monsters and dancers, but also the various virtual objects 4 that make up each AR space are displayed only to the user 3 experiencing that AR space.

[0089] The HMDs 2a and 2b are connected to each other so that they can communicate with each other. There are no limitations on the communication format for connecting the multiple HMDs 2 so that they can communicate with each other, and any communication technology may be used. For example, wireless network communication such as WiFi, short-range wireless communication such as Bluetooth (registered trademark), etc. may be used. The HMDs 2a and 2b may be able to communicate with each other via a server or the like on a network.

[0090] For example, the technology described in the first embodiment may be applied to each of the HMDs 2a and 2b. That is, when the users 3a and 3b are close to each other, the HMDs 2a and 2b may each control the playback of virtual content based on the avoidance area information. Even in this case, a high-quality AR experience can be achieved.

[0091] On the other hand, in this embodiment, various information such as avoidance area information and priority is exchanged (transmitted / received) between the HMD 2a and the HMD 2b. Specifically, each of the HMD 2a and the HMD 2b generates avoidance area information for the other user 3 (for the other HMD 2). That is, it is possible for each of the HMDs 2a and 2b to generate and transmit to the other user avoidance area information that is used when the other user's HMD 2 executes playback control based on the avoidance area information. This makes it possible to appropriately generate and transmit to the other user effective avoidance area information that does not disrupt the worldview of one's own AR world. In this embodiment, the HMD 2a and the HMD 2b each calculate a priority for playing back virtual content based on the avoidance area information, and the generated priority is transmitted to the other party. The two priorities calculated by the HMD 2a and the HMD 2b are compared, and the HMD 2 with the higher priority executes normal playback control without executing playback control based on the avoidance area information, thereby realizing a normal AR world. In the HMD 2 with the lower priority, playback control of the virtual content is executed based on the avoidance area information received from the other party. As described in the first embodiment, it is possible to play back the virtual content without disrupting the worldview of the AR world. As a result, it becomes possible for both users 3a and 3b to avoid situations in which their view of the AR world is disrupted due to interference with other users 3, thereby enabling a high-quality AR experience.

[0092] In this embodiment, each of the plurality of HMDs 2 can function as one embodiment of an information processing device according to the present technology. Also, each of the plurality of HMDs 2 can function as another information processing device according to the present technology. In the following description, the HMD 2a shown in FIG. 15 will be described as one embodiment of an information processing device according to the present technology, and the HMD 2b will be described as one embodiment of another information processing device according to the present technology. Therefore, the user 3a wearing the HMD 2a is the target person for whom the virtual content is to be played back. The virtual content (virtual object 4a and virtual sound) played back for the user 3a by the HMD 2a is the virtual content (virtual object 4a and virtual sound) for the target person. The HMD 2b is an embodiment of another information processing device. The user 3b wearing the HMD 2b becomes another object. Furthermore, the user 3b wearing the HMD 2b becomes another target person who is the target of playback of virtual content by another information processing device. The virtual content (virtual object 4b and virtual sound) reproduced for the user 3b by the HMD 2b becomes virtual content (virtual object 4a and virtual sound) for another object (for another target person). Of course, the technology described below can be applied as is even when the HMD 2b is an embodiment of an information processing device according to the present technology and the HMD 2a is an embodiment of another information processing device according to the present technology.

[0093] FIG. 16 is a block diagram showing an example of the functional configuration of the HMDs 2a and 2b. In this embodiment, the controller 21 further controls the priority calculation unit 36 ​​and the priority comparison unit 37. The priority calculation unit 36 ​​calculates the priority regarding the playback of the virtual content based on the avoidance area information. The priority comparison unit 37 compares the priority generated by itself with the priority regarding the other HMD 2 received from the partner HMD 2.

[0094] In this embodiment, the HMD 3 on the other side generates avoidance area information to be used by itself. Then, the generated avoidance area information is acquired by the information acquisition unit 26 via the communication unit 16. Therefore, in this embodiment, the information acquisition unit 26 is one embodiment of the acquisition unit related to the present technology. The avoidance area information generating unit 28 functions as an embodiment of a generating unit according to the present technology. The AR playback control unit 29 corresponds to an embodiment of the playback control unit according to the present technology. The priority calculation unit 36 ​​corresponds to an embodiment of the priority calculation unit according to the present technology. The transmission control unit 30 corresponds to an embodiment of the transmission control unit according to the present technology.

[0095] [Virtual content playback behavior] 17 is a flowchart showing an example of the operation of playing back virtual content using the HMDs 2a and 2b. Here, the explanation will be centered on the HMD 2a. 18 to 28 are schematic diagrams for explaining the steps shown in FIG. In this embodiment, the AR space is realized by executing the process shown in Fig. 4 at a predetermined frame rate. Of course, this is not limited to being executed for each frame.

[0096] The surrounding environment determination unit 27 monitors whether or not there is another user 3 nearby (step 301). For example, processing similar to steps 201 and 202 shown in FIG. 5 may be executed, and if there is a nearby object (Yes in step 202), it may be determined whether the object is another user 3 or not. The other user 3 corresponds to a user 3 wearing an HMD 2 (another information processing device) that can function as one element of the AR provision system 1 according to the present technology. For example, a signal is sent to inquire whether it is capable of functioning as one element of the AR provision system 1 according to the present technology (for example, whether it has a functional block such as that shown in FIG. 16), and if a response signal in the affirmative is received, it is determined that there is another user 3 nearby. Alternatively, any other method may be employed that can confirm whether or not a nearby object is another user 3.

[0097] For example, if step 202 shown in Fig. 5 is Yes, the process proceeds to step 301 shown in Fig. 17. If step 301 is No, the process proceeds to step 204 in Fig. 5, and if step 301 is Yes, the process proceeds to step 302 shown in Fig. 17. Such a processing flow is also possible. This makes it possible to execute processing that integrates the processing flow shown in Fig. 5 and the processing flow shown in Fig. 17.

[0098] Alternatively, a device search signal, an inquiry signal, or the like may be periodically transmitted over broadband by the HMD 2. Then, when a response signal is received from another HMD 2, it may be determined that another user 3 is nearby. Furthermore, another HMD 2 may periodically output a beacon signal or the like including information indicating that the HMD 2 itself can function as one element of the AR provision system 1 according to the present technology. When the beacon signal is received, it may be determined that another user 3 is nearby. Alternatively, any other method or algorithm may be employed to perform the determination in step 301 . For example, in the processing flow shown in Fig. 5, a step of determining whether or not the nearby object is the user 3 may be added, and if the nearby object is another user 3, the processing flow may return to step 201. Such a processing flow and the processing flow shown in Fig. 17 may be executed in parallel. This makes it possible to execute processing that integrates the processing flow shown in Fig. 5 and the processing flow shown in Fig. 17.

[0099] As shown in FIG. 18, the range of the effective field of view (angle of view) may be used as a parameter that serves as a criterion for the determination in step 301. It may be determined that a nearby user 3 exists when users 3a and 3b are within each other's field of view 38 (38a and 38b), or when at least one user 3 is within the other user's field of view 38. 18, virtual objects 4a and 4b often exist within the visual fields 38a and 38b, and there is a high possibility that the user's virtual object 4 will interfere with the other user's 3. Therefore, by using the visual fields as a parameter that serves as the determination criterion in step 301, it becomes possible to adequately deal with such interference. For example, by exchanging position information, orientation information, and effective visual field information between the HMDs 2a and 2b, it is possible to perform judgment using the effective visual field. It is also possible to determine whether the other person is within the effective visual field using sensor information from the outward-facing camera 10 and the distance sensor 24.

[0100] Furthermore, the distance to user 3b may be used as a parameter serving as a criterion for determination in step 301. For example, if the distance between oneself (user 3a) and the other party (user 3b) is short, there is a high possibility that interference will occur. For example, if the distance between oneself (user 3a) and the other party (user 3b) is 20 m or less, it may be determined that there is a nearby user 3. For example, it is possible to detect the distance to the other user 3 based on the sensor information of the distance measurement sensor 24. Alternatively, by transmitting position information acquired by the GPS 23 or position information calculated by self-position estimation, it is possible to perform a determination based on the distance between the users 3. Of course, the specific distance may be set arbitrarily. Depending on the content of the AR application, the distance is not limited to the distance at which the users 3 can see each other, and a distance of several kilometers may be adopted as the determination criterion.

[0101] Furthermore, as a parameter serving as a criterion for the determination in step 301, the distance between the virtual object 4a for the user (user 3a) and the other user (user 3b) or the distance between the virtual objects 4a and 4b may be used. If these distances are short, there is a high possibility of interference occurring. For example, if the distance between the virtual object 4a for the user (user 3a) and the other user (user 3b) is within 10 meters, it is determined that a nearby user 3 exists. Alternatively, if the distance between the virtual objects 4a and 4b is within 10 meters, it is determined that a nearby user 3 exists. Of course, the specific distance may be set arbitrarily. Such a determination can be made based on, for example, the position information of each user 3 and the display position information of each virtual object 4.

[0102] If there is another nearby user 3b (Yes in step 301), the avoidance area information generating unit 28 generates avoidance area information for the other user 3b (step 302). For example, it is possible to generate avoidance area information for another user 3b based on the position of the user (user 3a). For example, the avoidance area 33 shown in Figs. 7 to 9 is set by replacing the other person 32 with the user (user 3a). That is, the avoidance area for the other user 3b is set based on the direction and movement (vector) of the user (user 3a) based on the position of the user (user 3a). Avoidance area information including this avoidance area is generated as the avoidance area information for the other user 3b.

[0103] As shown in FIGS. 19 to 21, avoidance area information for another user 3b may be generated based on the position of a virtual object 4a for the user (user 3a). In FIGS. 19 to 21, an area of ​​a predetermined range based on the position (display position, etc.) of the virtual object 4a for the user (user 3a) is set as an avoidance area 40 for another user 3b. If the other person (user 3b) passes around the virtual object 4a for the user (user 3a) or if the other person's (user 3b's) virtual object 4b comes near the virtual object 4a for the user (user 3a), the other person (user 3b) will also approach that area. Therefore, the area around the virtual object 4a for the user (user 3a) is set as an avoidance area 40.

[0104] 19, a circular area with a predetermined radius and centered on the virtual object 4a for the user (user 3a) is set as the avoidance area 40. Of course, the shape of the avoidance area 40 is not limited, and may be an ellipse or a polygon. Furthermore, the size of the avoidance area 40 is not limited, and may be set to any arbitrary value, such as a radius of 3 m.

[0105] 20, the orientation of the virtual object 4a for the user 3a (user 3a) is used as a parameter to set the avoidance area 40. Since the virtual object 4a often moves in the direction that it is facing, the avoidance area 40 is set based on the orientation. For example, an avoidance area 40 having a predetermined shape such as a triangle, a rectangle, a circle, an ellipse, a polygon, or the like is set in front of the virtual object 4a for the user (user 3a) along the direction in which the virtual object 4a for the user (user 3a) is facing. 20, when an isosceles triangular avoidance area 40 is used, the vertex of the isosceles triangle is aligned with the position of the virtual object 4a for the user (user 3a). Then, the perpendicular line extending from the vertex to the base of the isosceles triangle is aligned with the orientation of the virtual object 4a for the user (user 3a). Alternatively, any setting method that reflects the orientation of the virtual object 4a for the user 3a may be adopted.

[0106] 21, the movement of the virtual object 4a for the user 3a is used as a parameter. Specifically, a motion vector representing the moving direction (orientation) and moving speed of the virtual object 4a for the user 3a is calculated. An avoidance area 40 is set based on the motion vector. It is possible to obtain the movement of the virtual object 4a for the user 3a from the content of the AR application. For example, if the movement of the virtual object 4a is determined by a program, it is possible to easily generate the avoidance area 40 based on the movement of the virtual object 4a. For example, an avoidance area 40 having a predetermined shape such as a triangle, a rectangle, a circle, an ellipse, a polygon, or the like is set in front of the virtual object 4a for the user (user 3a) along the direction of the motion vector. The size of the avoidance area 40 may be adjusted as appropriate depending on the size of the motion vector. 21, when an isosceles triangular avoidance area 40 is used, the vertex of the isosceles triangle is aligned with the position of the virtual object 4a for the user (user 3a). Then, a perpendicular line extending from the vertex to the base of the isosceles triangle is aligned with the direction of the motion vector. Alternatively, the avoidance area 40 made of an isosceles triangle may be set so that the magnitude of the perpendicular line matches the magnitude of the motion vector. In this way, the avoidance area information may be generated based on the movement of the virtual object 4a for the user 3a. By performing the movement prediction, it becomes possible to set the avoidance area 40 with high accuracy.

[0107] The avoidance area 40 is set based on three-dimensional coordinates defined by the surrounding environment determination unit 27, for example. A three-dimensional area may be set as the avoidance area 40. For example, the avoidance area 40 may be set to have a conical shape, a square pyramid shape, or the like, extending forward from the virtual object 4a for the user 3a (the user 3a), with the position of the virtual object 4a for the user 3a as its apex.

[0108] In the first embodiment, it is assumed that when the nearby object is another user 3, it is possible to acquire information such as the position of the virtual object 4 for the other user 3. In this case, in step 204 shown in FIG. 5 , avoidance area information may be generated based on the position and movement of the virtual object 4 for the other user 3. For example, avoidance area information including an avoidance area 33 set in the same manner as the avoidance area 40 illustrated in FIGS. 19 to 21 may be generated.

[0109] As shown in FIG. 22, it is also possible to set an avoidance area 40 based on the direction (path 41) of movement of a virtual object 4a for the user (user 3a). For example, if the path 41 of the virtual object 4a is determined by a program, it is possible to easily generate the avoidance area 40 based on the path. For example, the width of the avoidance area 40 is set arbitrarily based on the width of the virtual object 4a, etc. The avoidance area 40 is set along the path 41 with the set width. For example, if the width w of the virtual object 4a is 10 cm, the width of the avoidance area 40 is 3w=30 cm. Of course, the width is not limited to this setting.

[0110] As shown in FIG. 23, it is also possible to set an area between the virtual object 4a for the user (user 3a) and the user (user a) as an avoidance area 40. Since the user 3 is likely to move in the direction of the virtual object 4 displayed for the user 3, the area between them is set as an avoidance area 40. For example, a thick line of a predetermined width (e.g., 60 cm) connects the virtual object 4a for the user (user 3a) and the user (user a), and the area of ​​the line is set as the avoidance area 40. Of course, the setting is not limited to this, and the avoidance area 40 may be set to have a predetermined shape such as a circle, an ellipse, or a polygon.

[0111] As shown in FIG. 24, it is also possible to set an avoidance area 40 based on the direction (route 42) of movement of the user 3a. In the AR world, a path 42 along which the user 3a will move is predicted, and the path is set as an avoidance area 40. In the example shown in FIG. 24, it is assumed that the content of the AR application is, for example, listening to a virtual object 4a-1 and conveying the information to a virtual object 4a-2. The HMD 2a predicts a path 42 in which the user approaches the virtual object 4a-1 and then approaches the virtual object 4a-2 along the road. Then, an area having a predetermined width and along the path 42 is set as the avoidance area 40. The avoidance area 40 is not limited to a route 42 to a very close location as shown in FIG. 24, and depending on the content of the AR application, it is possible that the avoidance area 40 is set along a route 42 spanning several kilometers. In the first embodiment, avoidance area information may be generated that includes the avoidance area 33 that is set in the same manner as the avoidance area 40 illustrated in FIGS. 22 to 24 and the like.

[0112] Returning to FIG. 17, the priority calculation unit 36 ​​calculates the priority (step 303). The priority is information for determining whether to play back the virtual content based on the avoidance area information received from the other party, or to play back the normal virtual content. This process is performed every frame, or once when another user 3b is discovered.

[0113] Examples of priority calculation are described below, and each calculation example may be combined as appropriate. Priority is given to applications / people who have reserved a place. For example, information regarding priority is managed within the AR provision system 1, and priority is determined according to the reservation status of the place. For example, if a famous scramble intersection is rented out to one AR application between 6:00 p.m. and 7:00 p.m., a predetermined priority (e.g., determined by a score from 0 to 100) is set for user 3 participating in that AR application. For example, in a reserved location, a high priority is set for the user of the reserved AR application. As a result, the AR world related to the reserved AR application is realized preferentially in that location while maintaining the worldview. In other words, playback of normal virtual content is executed preferentially. Of course, a high priority may be set according to the reserved time slot.

[0114] Priority is given to applications / people with a large number of users in multiplayer. In other words, the more users who participate in the same AR world, the higher the priority is set. For example, if the number of users 3 participating in the AR space (World A) shown in Fig. 15 is 10, the priority for each user 3 is set to 10. If the number of users 3 participating in the AR space (World B) is 5, the priority for each user 3 is set to 5. In this way, the number of participants can be used directly as the priority. This allows AR worlds with a large number of participants to be prioritized while maintaining their worldview.

[0115] Priority may be set based on billing information for experiencing the AR world. For example, priority is given to users 3 who have paid or who have paid a large amount. For example, priority can be set according to the amount of billing. This allows the AR world enjoyed by paying users 3 or users 3 who pay a large amount to be realized with priority while maintaining the worldview.

[0116] The priority may be set based on the moving speed of the user 3 or the virtual object 4. For example, if a stationary user 3 can change his / her movement more flexibly than a walking user 3, the priority is set to the speed of user 3 or virtual object 4. This allows the AR world of user 3 or virtual object 4, which is moving at a higher speed, to be realized with priority while maintaining the worldview. Conversely, a higher priority may be set for a stationary user 3 or virtual object 4. For example, a setting such as priority=1 / speed of user 3 or virtual object 4 is also possible.

[0117] Priority is given to user 3 who is speaking. Since speaking tends to cause discomfort compared to avoidance using only visuals, the priority is set higher for user 3 who is speaking. For example, the priority is set to the duration of the speech. This allows the AR world of the speaking user 3 to be realized with priority so as to maintain the worldview.

[0118] The priority may be set based on the degree of attention to the virtual object 4. In the AR world, the priority is set depending on whether the user 3 is paying attention to the virtual object 4. If the user 3 is not paying attention to the virtual object 4, changing the display method of the virtual object 4 will not cause discomfort. For example, it is possible to set the priority as 1 / distance between the gaze point and the virtual object 4. The gaze point of user 3 can be detected from images of the left and right eyes of user 3 captured by inward-facing camera 9, for example.

[0119] The transmission control unit 30 transmits the avoidance area information for the other user 3b and the priority of the user 3a (user 3a) to the HMD 2b (step 304). Also, from the HMD 2b, the avoidance area information for another user (that is, the user 3a) as seen from the HMD 2b and the priority of the user 3b are received (also step 304). The avoidance area information and priority received from the HMD 2b are acquired by the information acquisition unit 26 and output to each block in the controller 21.

[0120] The priority comparison unit 37 compares the priorities and determines whether the priority of the other user 3b is higher (step 305). If the priority of the other user 3b is lower, that is, if the priority of the user 3a is higher (No in step 305), the AR playback control unit 29 executes normal playback control of the virtual content (step 306). That is, the playback control of the virtual content based on the avoidance area information received from the HMD 2b is not executed. If the priority for the other user 3b is higher, i.e., if the priority for the user 3a is lower (Yes in step 305), the AR playback control unit 29 executes playback control of the virtual content based on the avoidance area information received from the HMD 2b (step 307).

[0121] For example, as described with reference to Figures 10 and 11, playback of virtual content for preventing the virtual object 4a for user 3a from entering the avoidance area 33 is executed. That is, playback control is executed to prevent the virtual object 4a from entering the avoidance area 33. Alternatively, the virtual content may be played back to prevent the user 3a from entering the avoidance area, as described with reference to Figures 12 to 14. That is, playback control may be executed to prevent the user 3a from entering the avoidance area. Various other playback controls may be executed based on the avoidance area to avoid situations in which the worldview of the AR space is disrupted. Furthermore, for example, when multiple users 3 participate in an AR application with a low priority, different playback controls may be executed for each user 3. Alternatively, a common playback control may be executed for each user 3.

[0122] 25 to 28 are schematic diagrams for explaining other examples of playback control of virtual content. There may be cases where it is difficult to prevent the user (user 3a) or the virtual object 4a for the user (user 3a) from entering the avoidance area 40 included in the avoidance area information received from the HMD 2b. For example, as shown in Figures 25 to 27, there may be cases where the avoidance area 40 is very large, or where the environment in which the user 3a is located makes avoidance difficult (such as a road that is too narrow), or there may be cases where data on the avoidance area 40 is missing from the received avoidance area information.

[0123] In such a case, for example, in an emergency situation, a general-purpose virtual object 4 that stops the movement of the user 3a may be displayed. The generic virtual object 4 is a virtual display of a generic image or object that may deviate from the worldview of the AR world, but can make the user 3 aware that it has stopped moving. For example, as shown in Fig. 25, a text image (character string) of "STOP!!" is displayed as a virtual object 4a. A virtual sound of "STOP!!" may also be output. When instructions are directly presented as text, the user 3a often follows them.

[0124] In the example shown in FIG. 26, an image of a fence is displayed as the virtual object 4a. In the example shown in FIG. 27, an image of a triangular cone is displayed as the virtual object 4a. Any other object that can make the player realize that it is not possible to proceed further in the real world, such as a tape that says "Keep Out," may be displayed.

[0125] By using a general-purpose virtual object 4 such as those illustrated in FIGS. 25 to 27, it is possible to stop the user 3a reliably. 25 to 27, the general-purpose virtual object 4 is displayed at a position very close to the user 3a. In this way, the display position of the general-purpose virtual object 4 may be set according to the position of the user 3a. On the other hand, a general-purpose virtual object 4 may be displayed fixedly at a predetermined position in the real world or at a predetermined position on the transmissive display 8. Also, an alert display or alert sound may be output.

[0126] For monsters and other characters, expressions may be used that show them flying off into the distance, gradually disappearing, or otherwise releasing them from the state of entering the avoidance area 40.

[0127] As shown in FIG. 28, a transformation (substitution) of the virtual object 4 may be performed. As shown in FIG. 28A, it is assumed that another user 3b is standing still and having a conversation with a virtual object 4b (dancer) for the other user 3b. For the user 3a with a low priority, the area around the virtual object 4b becomes an avoidance area 40, and the user 3a and the virtual object 4a for the user 3a cannot approach. On the other hand, user 3a cannot see the virtual object 4b for other user 3b, and does not understand why user 3b is talking to himself or herself.Furthermore, user 3a cannot grasp the avoidance area 40 around the virtual object 4b for other user 3b. In such a situation, as shown in Fig. 28B, instead of the virtual object 4b (dancer) for the other user 3b, a virtual object 4a (monster) for the user 3a is displayed at the same position. Then, an expression is adopted that makes it appear as if the user 3a is having a conversation with the user 3b. For example, the monster's voice may be output as virtual sound. By displaying the virtual object 4a that matches the user's own AR world view instead, the user 3a can understand why the user 3b is speaking. It also makes it possible for the user 3a to avoid approaching the virtual object 4a with which the user 3b is talking. Note that a representation that allows the user 3a to recognize that the virtual object 4 is displayed in place of the virtual object 4b for the other user 3b may be presented.

[0128] <Other embodiments> The present technology is not limited to the above-described embodiments, and various other embodiments can be realized.

[0129] 29 to 34 are schematic diagrams for explaining the setting of the avoidance request level. An avoidance request level may be set for the avoidance area 33 and the avoidance area 40 set for the other user 3b. The avoidance requirement level is information indicating the degree to which avoidance is required, and can also be called the degree of avoidance. The higher the avoidance requirement level, the more avoidance is required, and the lower the avoidance requirement level, the less avoidance is required. The avoidance requirement level is set, for example, for each frame, and is set for the avoidance areas 33 and 40 in accordance with the generation of the avoidance area information.

[0130] As shown in FIG. 35, for example, the controller 21 configures an avoidance request degree setting unit 50 for setting the avoidance request degree. For example, in the first embodiment, a step for setting the avoidance requirement level for the avoidance area 33 is provided between step 204 and step 205 shown in FIG. In the second embodiment, for example, a step of setting an avoidance request degree for the avoidance area 40 is provided between step 302 and step 303. The avoidance request degree generated in this step is transmitted to the HMD 2b by the transmission control unit 30. Therefore, the avoidance request degree set in this step becomes the avoidance request degree for the other user 3b.

[0131] In the examples shown in Figs. 29 and 30, the closer a person is to the other person 32 in the avoidance area 33, the higher the avoidance requirement degree is set. The further away a person is from the other person 32, the lower the avoidance requirement degree is set. For example, the avoidance requirement degree is set to the maximum of 1.0 at the position of the other person 32. The further away a person is from the other person 32, the smaller the avoidance requirement degree becomes in a linear function fashion, and when a predetermined distance away, the avoidance requirement degree becomes the minimum of 0. For example, an avoidance requirement level of 0 may be set at a position on the periphery 34 of the avoidance area 33 that is farthest from the other person 32 .

[0132] In the examples shown in FIGS. 31 to 33, the closer a user is to the virtual object 4a relative to the avoidance region 40 for the other user 3b, the higher the avoidance requirement level is set. The further away the user is from the virtual object 4a, the lower the avoidance requirement level is set. For example, the avoidance requirement level is set to the maximum of 1.0 at the position of the virtual object 4a. The further away the user is from the virtual object 4a, the smaller the avoidance requirement level becomes in a linear function fashion, and when the user is a predetermined distance away, the avoidance requirement level becomes the minimum of 0. For example, an avoidance requirement level of 0 may be set at a position on the periphery 34 of the avoidance area 40 that is farthest from the virtual object 4a. In the example shown in FIG. 33, the further away from the virtual object 4a along the path 41, the lower the avoidance requirement level.

[0133] The setting of the avoidance requirement level may be changed as appropriate depending on the type of virtual object 4a. For example, the highest avoidance requirement level may be set at the position of the virtual object 4a as appropriate depending on the type of virtual object 4a. For example, for a virtual object 4a of an important character such as the main character, the maximum avoidance requirement level of 1.0 is set at the position of the virtual object 4a, and the avoidance requirement level decreases with increasing distance from the virtual object 4a. On the other hand, for a virtual object 4a of a character that is not very important, such as a supporting character, a relatively low avoidance requirement of 0.5 is set for the position of the virtual object 4a, and the avoidance requirement decreases from 0.5 as the distance from the virtual object 4a increases. Such a setting makes it possible to preferentially avoid interference with the virtual object 4a of an important character. In the cases shown in FIGS. 29 and 30, the setting of the avoidance requirement level may be changed as appropriate depending on the type of other object.

[0134] The AR playback control unit 29 controls the playback of the virtual content based on the avoidance request level set for the avoidance areas 33 and 40. For example, playback control is executed to more effectively prevent the user 3 or the virtual object 4a from entering a position for which a high avoidance requirement level is set. 34, when moving the virtual object 4 from the avoidance area 33, the virtual object 4 is moved in a direction that decreases the avoidance requirement level. For example, if the virtual object 4 is at a position where the avoidance requirement level is 0.8, the virtual object 4 is moved from there in a direction that decreases the avoidance requirement level to 0. Movement in a direction that increases the avoidance requirement level is not executed. This allows for more natural AR expression and maintains the worldview of the AR world.

[0135] Based on the type of the virtual object 4, the virtual object 4 may be classified into a virtual object 4 whose entry into the avoidance area 33 (40) should be avoided and a virtual object 4 whose entry into the avoidance area 33 (40) does not need to be avoided. For example, a virtual object 4 such as a character is classified as a virtual object 4 that should avoid entering the avoidance area 33 (40). On the other hand, virtual objects 4 that form the background of walls, buildings, etc., and that do not disrupt the worldview of the AR world even if entry into the avoidance area 33 (40) is avoided, are classified as virtual objects 4 that do not need to be avoided from entering the avoidance area 33 (40). For example, the various playback controls described above are executed for a virtual object 4 that is classified as a virtual object 4 that should avoid entering the avoidance area 33 (40).

[0136] In the above, an example was given in which AR glasses are used as the HMD 2, but the present technology is not limited to this and can also be applied when VR (Virtual Reality) glasses or the like are used as a video see-through HMD. Furthermore, the present technology is not limited to devices such as HMDs worn on the user's head, and can also be applied to AR expression using, for example, smartphones, tablet terminals, and PCs (Personal Computers).

[0137] The technology described in the first embodiment and the technology described in the second embodiment may be executed in an integrated manner. Alternatively, a configuration in which only the technology described in the first embodiment or only the technology described in the second embodiment can be executed may be adopted.

[0138] In the above, an example has been given in which the HMD 2 functions as an embodiment of the information processing device according to the present technology. However, this is not limiting, and an embodiment of the information processing device according to the present technology may be realized by any computer such as a PC connected to the HMD 2 via a network or the like, and the information processing method according to the present technology may be executed. Furthermore, an embodiment of the information processing device according to the present technology may be realized and the information processing method according to the present technology may be executed by the HMD 2 and a computer on the network working together. For example, some or all of the functional blocks realized by the controller 21 shown in FIG. 5 or FIG. Furthermore, the "information processing device" and the "other information processing device" may be realized by a common computer.

[0139] FIG. 36 is a block diagram showing an example of the hardware configuration of a computer 60 to which the present technology can be applied. The computer 60 includes a CPU 61, a ROM (Read Only Memory) 62, a RAM 63, an input / output interface 65, and a bus 64 that interconnects these components. The input / output interface 65 is connected to a display unit 66, an input unit 67, a storage unit 68, a communication unit 69, a drive unit 70, and the like. The display unit 66 is a display device using, for example, a liquid crystal display, an electroluminescent display, etc. The input unit 67 is, for example, a keyboard, a pointing device, a touch panel, or other operating device. When the input unit 67 includes a touch panel, the touch panel can be integrated with the display unit 66. The storage unit 68 is a non-volatile storage device such as a HDD, flash memory, or other solid-state memory. The drive unit 70 is a device capable of driving a removable storage medium 71 such as an optical storage medium or magnetic recording tape. The communication unit 69 is a modem, router, or other communication device that can be connected to a LAN, WAN, or the like and that communicates with other devices. The communication unit 69 may communicate either wired or wirelessly. The communication unit 69 is often used separately from the computer 60. Information processing by the computer 60 having the above-described hardware configuration is realized by cooperation between software stored in the storage unit 68, the ROM 62, etc. and the hardware resources of the computer 60. Specifically, the information processing method according to the present technology is realized by loading a program constituting the software stored in the ROM 62, etc., into the RAM 63 and executing the program. The program is installed in the computer 60 via, for example, a recording medium 61. Alternatively, the program may be installed in the computer 60 via a global network or the like. Any other computer-readable non-transitory storage medium may be used.

[0140] An information processing device according to the present technology may be constructed by a plurality of computers connected to each other so as to be able to communicate via a network or the like working together to execute an information processing method and program according to the present technology. That is, the information processing method and program according to the present technology can be executed not only in a computer system configured by a single computer, but also in a computer system in which multiple computers operate in conjunction with each other. In this disclosure, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems. The information processing method and program execution according to the present technology by a computer system include both cases where, for example, acquisition of avoidance area information, generation of avoidance area information, calculation of priority, setting of avoidance request level, playback control of virtual content, etc. are executed by a single computer, and cases where each process is executed by a different computer. Furthermore, execution of each process by a specific computer includes having another computer execute part or all of the process and obtaining the results. In other words, the information processing method and program according to the present technology can also be applied to a cloud computing configuration in which a single function is shared and processed jointly by multiple devices via a network.

[0141] The configurations and processing flows of the AR provision system, HMD, virtual object, etc. described with reference to the drawings are merely one embodiment and can be modified as desired without departing from the spirit of the present technology. In other words, any other configurations, algorithms, etc. for implementing the present technology may be adopted.

[0142] In this disclosure, to facilitate understanding of the explanation, words such as "approximately," "almost," and "roughly" are used as appropriate. However, there is no clear difference between using and not using words such as "approximately," "almost," and "roughly." That is, in the present disclosure, concepts that define shape, size, positional relationship, state, etc., such as "center," "central," "uniform," "equal," "same," "orthogonal," "parallel," "symmetrical," "extended," "axial direction," "cylindrical," "cylindrical," "ring-shaped," and "annular," are concepts that include "substantially center," "substantially central," "substantially uniform," "substantially equal," "substantially the same," "substantially orthogonal," "substantially parallel," "substantially symmetrical," "substantially extended," "substantially axial direction," "substantially cylindrical," "substantially cylindrical," "substantially ring-shaped," "substantially annular," and the like. For example, this also includes states that fall within a specified range (for example, a range of ±10%) based on criteria such as "perfectly centered," "perfectly central," "perfectly uniform," "perfectly equal," "perfectly the same," "perfectly perpendicular," "perfectly parallel," "perfectly symmetrical," "perfectly extended," "perfectly axial," "perfectly cylindrical," "perfectly cylindrical," "perfectly ring-shaped," and "perfectly annular." Therefore, even if the words "roughly," "almost," "approximately," etc. are not added, it may include concepts that can be expressed by adding "roughly," "almost," "approximately," etc. Conversely, a state expressed by adding "roughly," "almost," "approximately," etc. does not necessarily exclude a complete state.

[0143] In this disclosure, expressions using "more than," such as "greater than A" and "smaller than A," are expressions that comprehensively include both concepts that include equivalent to A and concepts that do not include equivalent to A. For example, "greater than A" is not limited to cases that do not include equivalent to A, but also includes "A or greater." Furthermore, "smaller than A" is not limited to "less than A," but also includes "A or less." When implementing the present technology, specific settings and the like may be appropriately adopted from the concepts included in "greater than A" and "smaller than A" so as to achieve the effects described above.

[0144] It is also possible to combine at least two of the features of the present technology described above. That is, the various features described in each embodiment may be arbitrarily combined without distinction between the embodiments. Furthermore, the various effects described above are merely examples and are not limiting, and other effects may also be achieved.

[0145] The present technology can also be configured as follows. (1) an acquisition unit that acquires avoidance area information regarding an avoidance area in which at least one of an entry of the target person or an entry of a virtual object for the target person that is displayed for the target person is to be avoided, the avoidance area information being generated based on at least one of a position of another object different from the target person for whom virtual content is to be played or a position of a virtual object for the other object that is displayed for the other object; a playback control unit that controls playback of virtual content for the target person based on the acquired avoidance area information; An information processing device comprising: (2) The information processing device according to (1), The playback control unit controls at least one of displaying a virtual object for the subject and outputting a virtual sound to the subject. Information processing device. (3) The information processing device according to (1) or (2), The playback control unit executes at least one of playback of the virtual content for preventing the target person from entering the avoidance area and playback of the virtual content for preventing a virtual object for the target person from entering the avoidance area. Information processing device. (4) An information processing device according to any one of (1) to (3), The playback control unit moves a position of a virtual object for the target person to outside the avoidance area. Information processing device. (5) An information processing device according to any one of (1) to (4), The playback control unit plays the virtual content for guiding the line of sight of the subject so that a virtual object for the subject included in the avoidance area is out of the effective field of view of the subject. Information processing device. (6) The information processing device according to any one of (1) to (5), further comprising: A generating unit generates the avoidance area information based on surrounding information about the surroundings of the target person. Information processing device. (7) The information processing device according to (6), The generation unit generates the avoidance area information based on at least one of a movement of the other object and a movement of a virtual object for the other object. Information processing device. (8) The information processing device according to (6) or (7), The generation unit generates the avoidance area information, which defines an area of ​​a predetermined range based on the position of the other object as the avoidance area. Information processing device. (9) An information processing device according to any one of (1) to (8), The other object is another target person who is a target of playback of virtual content by another information processing device. Information processing device. (10) The information processing device according to (9), The generation unit generates avoidance area information for the other target person based on at least one of a position of the target person and a position of a virtual object for the target person. Information processing device. (11) The information processing device according to (9) or (10), further comprising: a transmission control unit that transmits avoidance area information for the other target person to the other information processing device; Information processing device. (12) The information processing device according to any one of (9) to (11), further comprising: a priority calculation unit that calculates a priority regarding the reproduction of the virtual content based on the avoidance area information; Information processing device. (13) The information processing device according to (12), The transmission control unit transmits the calculated priority to the other information processing device. Information processing device. (14) The information processing device according to (13), The playback control unit compares the priority calculated by the priority calculation unit with the priority for the other target person transmitted from the other information processing device, and if the priority for the other target person is higher, controls playback of the virtual content based on the avoidance area information transmitted from the other information processing device. Information processing device. (15) The information processing device according to (13) or (14), The playback control unit compares the priority calculated by the priority calculation unit with the priority for the other target person transmitted from the other information processing device, and if the priority for the other target person is lower, does not execute control of playback of the virtual content based on the avoidance area information transmitted from the other information processing device. Information processing device. (16) The information processing device according to any one of (1) to (15), further comprising: an avoidance request level setting unit that sets an avoidance request level indicating a level to which avoidance is required for the avoidance area included in the avoidance area information; The playback control unit controls playback of the virtual content based on the avoidance request degree. Information processing device. (17) The information processing device according to any one of (10) to (15), the avoidance request degree setting unit sets the avoidance request degree for an avoidance area included in the avoidance area information for the other target person, The transmission control unit transmits the set avoidance request degree to the other information processing device. Information processing device. (18) acquires avoidance area information regarding an avoidance area in which at least one of an entry of the target person or an entry of a virtual object for the target person that is displayed for the target person is to be avoided, the avoidance area information being generated based on at least one of a position of another object different from the target person for whom virtual content is to be played back or a position of a virtual object for the other object that is displayed for the other object; Based on the obtained avoidance area information, playback of virtual content for the target person is controlled. An information processing method implemented by a computer system. (19) acquiring avoidance area information about an avoidance area in which at least one of an entry of the target person or an entry of a virtual object for the target person displayed for the target person is to be avoided, the avoidance area information being generated based on at least one of a position of another object different from the target person for whom virtual content is to be played or a position of a virtual object for the other object displayed for the other object; controlling playback of virtual content for the target person based on the obtained avoidance area information; A program that causes a computer system to execute the following. [Explanation of symbols]

[0146] 1. AR provision system 2, 2a, 2b...HMD 3, 3a, 3b...User 4, 4a, 4b...Virtual Objects 21...Controller 26…Information acquisition department 27...Surrounding Environment Assessment Section 28…Avoidance area information generation unit 29...AR playback control unit 30...Transmission control unit 32...Other people 33…Avoidance area 36…Priority calculation unit 37…Priority comparison section 38, 38a, 38b...effective field of view 40...Avoidance area for other users 60...Computer

Claims

1. an acquisition unit that acquires avoidance area information regarding an avoidance area in which at least one of an entry of the target person or an entry of a virtual object for the target person that is displayed for the target person is to be avoided, the avoidance area information being generated based on at least one of a position of another object different from the target person for whom virtual content is to be played or a position of a virtual object for the other object that is displayed for the other object; a playback control unit that controls playback of virtual content for the target person based on the acquired avoidance area information; a priority calculation unit that calculates a priority regarding the reproduction of the virtual content based on the avoidance area information; Equipped with The other object is another target person who is a target of playback of virtual content by another information processing device. Information processing device.

2. 2. The information processing device according to claim 1, The playback control unit controls at least one of displaying a virtual object for the subject and outputting a virtual sound to the subject. Information processing device.

3. 2. The information processing device according to claim 1, The playback control unit executes at least one of playback of the virtual content for preventing the target person from entering the avoidance area and playback of the virtual content for preventing a virtual object for the target person from entering the avoidance area. Information processing device.

4. 2. The information processing device according to claim 1, The playback control unit moves a position of a virtual object for the target person to outside the avoidance area. Information processing device.

5. 2. The information processing device according to claim 1, The playback control unit plays the virtual content for guiding the line of sight of the subject so that a virtual object for the subject included in the avoidance area is out of the effective field of view of the subject. Information processing device.

6. The information processing device according to claim 1, further comprising: A generating unit generates the avoidance area information based on surrounding information about the surroundings of the target person. Information processing device.

7. 7. The information processing device according to claim 6, The generation unit generates the avoidance area information based on at least one of a movement of the other object and a movement of a virtual object for the other object. Information processing device.

8. 7. The information processing device according to claim 6, The generation unit generates the avoidance area information, which defines an area of ​​a predetermined range based on the position of the other object as the avoidance area. Information processing device.

9. 7. The information processing device according to claim 6, The generation unit generates avoidance area information for the other target person based on at least one of a position of the target person and a position of a virtual object for the target person. Information processing device.

10. The information processing device according to claim 1, further comprising: a transmission control unit that transmits avoidance area information for the other target person to the other information processing device; Information processing device.

11. The information processing device according to claim 10, The transmission control unit transmits the calculated priority to the other information processing device. Information processing device.

12. The information processing device according to claim 11, The playback control unit compares the priority calculated by the priority calculation unit with the priority for the other target person transmitted from the other information processing device, and if the priority for the other target person is higher, controls playback of the virtual content based on the avoidance area information transmitted from the other information processing device. Information processing device.

13. The information processing device according to claim 11, The playback control unit compares the priority calculated by the priority calculation unit with the priority for the other target person transmitted from the other information processing device, and if the priority for the other target person is lower, does not execute control of playback of the virtual content based on the avoidance area information transmitted from the other information processing device. Information processing device.

14. The information processing device according to claim 1, further comprising: an avoidance request level setting unit that sets an avoidance request level indicating a level to which avoidance is required for the avoidance area included in the avoidance area information; The playback control unit controls playback of the virtual content based on the avoidance request degree. Information processing device.

15. The information processing device according to claim 14, further comprising: a transmission control unit that transmits avoidance area information for the other target person to the other information processing device; the avoidance request degree setting unit sets the avoidance request degree for an avoidance area included in the avoidance area information for the other target person, The transmission control unit transmits the avoidance request degree set for the avoidance area included in the avoidance area information for the other target person to the other information processing device. Information processing device.

16. an acquisition unit that acquires avoidance area information regarding an avoidance area in which at least one of an entry of the target person or an entry of a virtual object for the target person that is displayed for the target person is to be avoided, the avoidance area information being generated based on at least one of a position of another object different from the target person for whom virtual content is to be played or a position of a virtual object for the other object that is displayed for the other object; a playback control unit that controls playback of virtual content for the target person based on the acquired avoidance area information; an avoidance request level setting unit that sets an avoidance request level indicating the degree to which avoidance is required for the avoidance area included in the avoidance area information; Equipped with The playback control unit controls playback of the virtual content based on the avoidance request degree. Information processing device.

17. An information processing method executed by a computer system, comprising: acquires avoidance area information regarding an avoidance area in which at least one of an entry of the target person or an entry of a virtual object for the target person that is displayed for the target person is to be avoided, the avoidance area information being generated based on at least one of a position of another object different from the target person for whom virtual content is to be played back or a position of a virtual object for the other object that is displayed for the other object; controlling playback of virtual content for the target person based on the acquired avoidance area information; Calculating a priority for playing the virtual content based on the avoidance area information This includes: The other object is another target person who is a target of playback of virtual content by another information processing device. Information processing methods.

18. An information processing method executed by a computer system, comprising: acquires avoidance area information regarding an avoidance area in which at least one of an entry of the target person or an entry of a virtual object for the target person that is displayed for the target person is to be avoided, the avoidance area information being generated based on at least one of a position of another object different from the target person for whom virtual content is to be played back or a position of a virtual object for the other object that is displayed for the other object; controlling playback of virtual content for the target person based on the acquired avoidance area information; An avoidance requirement level indicating the degree to which avoidance is required is set for the avoidance area included in the avoidance area information. This includes: The process of controlling the reproduction of the virtual content controls the reproduction of the virtual content based on the avoidance request degree. Information processing methods.

19. A program for causing a computer system to execute an information processing method, comprising: The information processing method includes: acquires avoidance area information regarding an avoidance area in which at least one of an entry of the target person or an entry of a virtual object for the target person that is displayed for the target person is to be avoided, the avoidance area information being generated based on at least one of a position of another object different from the target person for whom virtual content is to be played back or a position of a virtual object for the other object that is displayed for the other object; controlling playback of virtual content for the target person based on the acquired avoidance area information; Calculating a priority for playing the virtual content based on the avoidance area information This includes: The other object is another target person who is a target of playback of virtual content by another information processing device. program.

Images