Information processing device, information processing method, and program

Abstract

Provided is an information processing device (500) which comprises a control unit (504) that controls a display of a virtual object on each display device according to an expression method of an image, the expression method being allocated to display the images on a plurality of the respective display devices that display the images pertaining to an identical virtual object, and that dynamically changes each parameter pertaining to displaying the virtual object.

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] In recent years, technology called Augmented Reality (AR), which presents virtual objects to a user by superimposing them on a real space as additional information to the real world, and Mixed Reality (MR), which reflects information from the real space in a virtual space, have been attracting attention. In light of this background, various user interfaces assuming the use of AR and MR technologies have been studied. For example, Patent Document 1 below discloses a technology for displaying the display content of a display unit of a mobile terminal held by a user as a virtual object on a virtual space displayed on a head mounted display (HMD) worn by the user. According to the above technology, a user can use a mobile terminal as a controller by touching the mobile terminal while viewing a virtual object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-036974 A Summary of the Invention [Problem to be solved by the invention]

[0004] With further use of AR and MR technologies, it is expected that a user will use multiple display devices simultaneously, as in Patent Document 1. However, in the conventional technology, there has been no sufficient consideration given to improving the user experience and operability when using multiple display devices that simultaneously display the same virtual object.

[0005] Therefore, the present disclosure proposes an information processing device, an information processing method, and a program that can further improve the user experience and operability when using multiple display devices that simultaneously display the same virtual object. [Means for solving the problem]

[0006] According to the present disclosure, there is provided an information processing device including a control unit that dynamically changes each parameter related to the display of a virtual object, the control unit being assigned to each of a plurality of display devices for displaying the image, and that controls the display of the virtual object on each of the display devices in accordance with a method of representing the image.

[0007] Furthermore, according to the present disclosure, there is provided an information processing method, in which an information processing device dynamically changes each parameter related to the display of the virtual object, the parameter being assigned to each of a plurality of display devices that display an image related to the same virtual object, for displaying the image, and controlling the display of the virtual object on the respective display devices in accordance with a representation method of the image.

[0008] Furthermore, according to the present disclosure, there is provided a program that causes a computer to function as a control unit that dynamically changes each parameter related to the display of a virtual object, the control unit controlling the display of the virtual object on each of a plurality of display devices that display an image related to the same virtual object in accordance with a representation method of the image, the control unit being assigned to each of the display devices for displaying the image. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram for explaining an overview of the present disclosure. [Diagram 2] 1 is a block diagram showing an example of a configuration of an information processing system 10 according to a first embodiment of the present disclosure. [Diagram 3] 11 is a flowchart illustrating an example of an information processing method according to the embodiment. [Figure 4] FIG. 11 is an explanatory diagram (part 1) for explaining an example of a display according to the embodiment. [Diagram 5] FIG. 11 is an explanatory diagram (part 2) for explaining an example of a display according to the embodiment. [Figure 6] FIG. 11 is an explanatory diagram (part 3) for explaining an example of a display according to the embodiment. [Figure 7] 11 is an explanatory diagram for explaining an example of display control according to the embodiment. FIG. [Figure 8] FIG. 11 is an explanatory diagram for explaining an overview of a second embodiment of the present disclosure. [Figure 9] 11 is a flowchart illustrating an example of an information processing method according to the embodiment. [Figure 10] 11 is an explanatory diagram for explaining an example of display control according to the embodiment. FIG. [Figure 11] FIG. 11 is an explanatory diagram for explaining an example of a display according to the embodiment. [Figure 12] FIG. 11 is an explanatory diagram for explaining an overview of a third embodiment of the present disclosure. [Figure 13] 11 is a flowchart (part 1) illustrating an example of an information processing method according to the embodiment. [Figure 14] 13 is a flowchart (part 2) illustrating an example of the information processing method according to the embodiment. [Figure 15] 11 is an explanatory diagram illustrating an example of a method for identifying a selected device according to the embodiment. FIG. [Figure 16] FIG. 13 is an explanatory diagram for explaining an overview of a modified example of the third embodiment of the present disclosure. [Figure 17] FIG. 2 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of a control unit 500. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functional configurations are denoted by the same reference numerals, and redundant description will be omitted.

[0011] In addition, in this specification and drawings, multiple components having substantially the same or similar functional configurations may be distinguished by adding different alphabets after the same reference numeral. However, when there is no particular need to distinguish each of multiple components having substantially the same or similar functional configurations, only the same reference numerals are used.

[0012] In the following description, a virtual object means a virtual object that can be perceived by a user as if it were a real object existing in real space. Specifically, a virtual object can be, for example, an animation of a game character or item that is displayed or projected, or an icon, text (button, etc.) as a user interface.

[0013] In the following description, AR display refers to displaying the virtual object so as to be superimposed on the real space visually recognized by the user so as to extend the real world. The virtual object presented to the user as additional information on the real world by such AR display is also called an annotation. In the following description, non-AR display refers to a display other than a display in which additional information is superimposed on the real space to extend the real world, and in this embodiment, for example, includes displaying a virtual object on a virtual space or simply displaying only a virtual object.

[0014] The explanation will be given in the following order. 1. Overview 1.1 Background 1.2 Overview of the embodiments of the present disclosure 2. First embodiment 2.1 General configuration of information processing system 10 2.2 Detailed configuration of the control unit 500 2.3 Information Processing Method 3. Second embodiment 3.1 Detailed configuration of the control unit 500 3.2 Information Processing Method 4. Third embodiment 4.1 Detailed configuration of the control unit 500 4.2 Information Processing Method 4.3 Variations 5. Summary 6. Hardware Configuration 7. Supplementary Information

[0015] <<1. Overview>> <1.1 Background> First, the background of the present disclosure will be described with reference to Fig. 1. Fig. 1 is an explanatory diagram for explaining an outline of the present disclosure. In the present disclosure, as shown in Fig. 1, an information processing system 10 that can be used in a situation in which a user 900 uses two devices to view a virtual object 600 and control the virtual object 600 will be considered.

[0016] In detail, one of the two devices is an AR device (first display device) 100 that can superimpose a virtual object 600 onto real space so that the user 900 can perceive it as a real object (actual object) existing in real space, such as an HMD (Head Mounted Display) shown in FIG. 1.

[0017] That is, the AR device 100 can be said to be a display device that uses the above-mentioned AR display as a method of expressing an image. Also, one of the above two devices is a non-AR device (second display device) 200 that does not display an object so as to be perceived by the user 900 as a real object (real object) existing in real space, such as a smartphone shown in Fig. 1, but can display a virtual object 600. That is, the non-AR device 200 can be said to be a display device that uses the above-mentioned non-AR display as a method of expressing an image.

[0018] In the present disclosure, a situation is assumed in which the user 900 can view the same virtual object 600 using the AR device 100 and the non-AR device 200, and can perform operations on the virtual object 600. More specifically, in the present disclosure, a situation is assumed in which, for example, the user 900 uses the AR device 100 to interact with a character that is the virtual object 600 that is perceived as existing in the same space as the user 900, and uses the non-AR device 200 to check the overall image and profile information of the character, a video from the character's viewpoint, a map, and the like.

[0019] The inventors have considered that, in the situation envisioned in the present disclosure, since the user 900 will have a different perception of the virtual object 600 displayed on two devices using different expression methods, it would be preferable to display the virtual object 600 so that it has a form that corresponds to the expression method assigned to each device.

[0020] In detail, the inventors of the present invention have chosen to control the display of the virtual object 600 to change depending on the distance between the user 900 and the virtual position of the virtual object 600 in the real space and the position of the viewpoint of the user 900 so that the user 900 can perceive the virtual object 600 as if it were a real object existing in the real space in the AR device 100. On the other hand, the inventors of the present invention have considered that the display of the virtual object 600 does not need to change depending on the distance or the position of the viewpoint in the non-AR device 200 because it is not required to be able to perceive the virtual object 600 as if it were a real object existing in the real space. In other words, the inventors of the present invention have chosen to independently control the display of the virtual object 600 in the non-AR device 200 without depending on the distance or the position of the viewpoint.

[0021] That is, the inventors considered that in the above situation, in order to enable the display of a natural virtual object 600 and to further improve the user experience and operability, it is preferable that the display of the virtual object 600 on two devices using different expression methods has a different form, a different change, or a different reaction to an operation from the user 900. Based on this consideration, the inventors have come to create the embodiment of the present disclosure.

[0022] <1.2 Overview of the embodiment of the present disclosure> In an embodiment of the present disclosure created by the present inventors, in a situation where the same virtual object 600 is displayed by a plurality of display devices including the above-mentioned AR device 100 and non-AR device 200, different controls are performed on the display of the virtual object 600 according to the representation method assigned to each display device. In detail, in the embodiment of the present disclosure, the display of the virtual object 600 in the AR device 100 is controlled using parameters that change dynamically according to the distance between the user 900 and the virtual position of the virtual object 600 and the position of the viewpoint of the user 900 in the real space. On the other hand, in the embodiment of the present disclosure, the display of the virtual object 600 in the non-AR device 200 is controlled using predefined parameters that do not change dynamically according to the distance or the position.

[0023] In this manner, in the embodiment of the present disclosure, the display of the virtual object 600 in the AR device 100 and the non-AR device 200 using different expression methods that are perceived by the user 900 will have different forms, different changes, or different reactions to operations from the user 900. Therefore, in the embodiment of the present disclosure, it is possible to display a more natural virtual object 600, and it is possible to further improve the user experience and operability. Hereinafter, the details of each embodiment of the present disclosure will be described in order.

[0024] <<2. First embodiment>> <2.1 Schematic configuration of information processing system 10> First, a schematic configuration of an information processing system 10 according to a first embodiment of the present disclosure will be described with reference to Fig. 2. Fig. 1 is a block diagram showing an example of a configuration of the information processing system 10 according to the present embodiment. As shown in Fig. 1, the information processing system 10 according to the present embodiment may include, for example, an AR device (first display device) 100, a non-AR device (second display device) 200, a depth measurement unit (real space information acquisition device) 300, a gaze sensor unit (gaze detection device) 400, and a control unit (information processing device) 500.

[0025] In this embodiment, the AR device 100 may be a device integrated with one, two, or all of the depth measurement unit 300, the gaze sensor unit 400, and the control unit 500, that is, each may not be realized by a single device. Also, the number of the AR device 100, the non-AR device 200, the depth measurement unit 300, and the gaze sensor unit 400 included in the information processing system 10 is not limited to the number shown in FIG. 2, and may be greater.

[0026] In addition, the AR device 100, the non-AR device 200, the depth measurement unit 300, the gaze sensor unit 400, and the control unit 500 can communicate with each other via various wired or wireless communication networks. The type of the communication network is not particularly limited. As a specific example, the network may be configured with a mobile communication technology (including GSM (registered trademark), UMTS, LTE, LTE-Advanced, 5G or subsequent technologies), a wireless LAN (Local Area Network), a dedicated line, or the like. In addition, the network may include multiple networks, and may be configured with a part being wireless and the rest being wired. Below, an overview of each device included in the information processing system 10 according to this embodiment will be described.

[0027] (AR device 100) The AR device 100 is a display device that AR displays a scene of a real space in which a virtual object 600 is virtually arranged, as viewed from a first viewpoint defined as a viewpoint of a user 900 in the real space. In detail, the AR device 100 can change and display the form of the virtual object 600 according to the distance between the user 900 and the virtual position of the virtual object 600 in the real space and the position of the viewpoint of the user 900. Specifically, the AR device 100 can be an HMD, a HUD (Head-Up Display) that is provided in front of the user 900 and displays an image of the virtual object 600 superimposed on the real space, or a projector that can project and display an image of the virtual object 600 in the real space. That is, the AR device 100 is a display device that displays the virtual object 600 superimposed on an optical image of a real object located in the real space, as if the virtual object 600 exists at a virtually set position in the real space.

[0028] As shown in FIG. 2, the AR device 100 also includes a display unit 102 that displays a virtual object 600, and a control unit 104 that controls the display unit 102 according to control parameters from a control unit 500, which will be described later.

[0029] Hereinafter, a configuration example of the display unit 102 will be described in a case where the AR device 100 according to the present embodiment is, for example, an HMD that is used by being worn on at least a part of the head of the user 900. In this case, examples of the display unit 102 to which AR display can be applied include a see-through type, a video see-through type, and a retinal projection type.

[0030] The see-through display unit 102 holds a virtual image optical system consisting of a transparent light guide unit or the like in front of the user 900 using, for example, a half mirror or a transparent light guide plate, and displays an image inside the virtual image optical system. Therefore, the user 900 wearing the HMD having the see-through display unit 102 can view the scenery of the external real space even while viewing the image displayed inside the virtual image optical system. With this configuration, the see-through display unit 102 can superimpose an image of the virtual object 600 on an optical image of a real object located in the real space based on, for example, AR display.

[0031] When the video see-through display unit 102 is worn on the head or face of the user 900, it is worn so as to cover the eyes of the user 900 and is held in front of the eyes of the user 900. In addition, the HMD having the video see-through display unit 102 has an outward camera (not shown) for capturing images of the surrounding scenery, and displays an image of the scenery in front of the user 900 captured by the outward camera on the display unit 102. With this configuration, the user 900 wearing the HMD having the video see-through display unit 102 can check the external scenery (real space) from the image displayed on the display, although it is difficult for the user 900 to directly view the external scenery. In addition, at this time, the HMD can superimpose an image of the virtual object 600 on the image of the external scenery based on, for example, AR display.

[0032] The retinal projection display unit 102 has a projection unit (not shown) held in front of the user 900's eyes, and the projection unit projects an image toward the user 900's eyes so that the image is superimposed on the external scenery. More specifically, in an HMD having a retinal projection display unit 102, an image is directly projected from the projection unit onto the retina of the user 900's eyes, and the image is formed on the retina. With this configuration, even in the case of a user 900 who is myopic or hyperopic, it becomes possible to view a clearer image. In addition, the user 900 who wears the HMD having the retinal projection display unit 102 can view the external scenery (real space) even while viewing the image projected from the projection unit. With this configuration, the HMD having the retinal projection display unit 102 can superimpose an image of a virtual object 600 on an optical image of a real object located in a real space based on, for example, AR display.

[0033] Furthermore, in this embodiment, the AR device 100 may be a smartphone, tablet, or the like that is held by the user 900 and that can display the virtual object 600 by superimposing it on an image of the real space seen from the position of a camera (not shown) mounted on the smartphone. In such a case, the above-mentioned first viewpoint is not limited to being the viewpoint of the user 900 in the real space, but is the position of the camera of the smartphone held by the user 900.

[0034] The control unit 104 of the AR device 100 controls the overall operation of the display unit 102 according to parameters and the like from a control unit 500 described later. The control unit 104 can be realized by electronic circuits of a microprocessor such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The control unit 104 may also include a ROM (Read Only Memory) for storing programs to be used, calculation parameters, and the like, and a RAM (Random Access Memory) for temporarily storing parameters and the like that change appropriately. For example, the control unit 104 controls the display of the virtual object 600 on the display unit 102 to be dynamically changed according to the distance between the user 900 and the virtual position of the virtual object 600 in the real space according to the parameters from the control unit 500.

[0035] The AR device 100 may also have a communication unit (not shown) that is a communication interface that can connect to an external device via wireless communication, etc. The communication unit is realized by a communication device such as a communication antenna, a transmission / reception circuit, or a port.

[0036] Furthermore, in this embodiment, the AR device 100 may be provided with a button (not shown) or a switch (not shown) (an example of an operation input unit) for the user 900 to perform an input operation. In addition to the above-mentioned operation on the button or the like, various input methods such as voice input, gesture input with the hand or head, and gaze input can be selected as the input operation of the user 900 to the AR device 100. Note that the input operation by these various input methods can be acquired by various sensors (sound sensor (not shown), camera (not shown), motion sensor (not shown)) provided in the AR device 100. In addition, the AR device 100 may be provided with a speaker (not shown) that outputs sound to the user 900.

[0037] Furthermore, in this embodiment, the AR device 100 may be provided with a depth measurement unit 300, an eye gaze sensor unit 400, and a control unit 500, which will be described later.

[0038] Additionally, in this embodiment, the AR device 100 may be provided with a positioning sensor (not shown). The positioning sensor is a sensor that detects the position of the user 900 wearing the AR device 100, and specifically, may be a GNSS (Global Navigation Satellite System) receiver or the like. In this case, the positioning sensor can generate sensing data indicating the latitude and longitude of the current location of the user 900 based on a signal from a GNSS satellite. Also, in this embodiment, for example, since it is possible to detect the relative positional relationship of the user 900 from information such as RFID (Radio Frequency Identification), a Wi-Fi access point, and a wireless base station, it is also possible to use a communication device for such communication as the positioning sensor. Furthermore, in this embodiment, the position and attitude of the user 900 wearing the AR device 100 may be detected by processing (accumulation calculation, etc.) sensing data of an acceleration sensor, a gyro sensor, a geomagnetic sensor, and the like included in the above-mentioned motion sensor (not shown).

[0039] (Non-AR device 200) The non-AR device 200 is a display device capable of non-AR display of an image of a virtual object 600 toward the user 900. In detail, the non-AR device 200 can display the virtual object 600 viewed from a second viewpoint at a position different from a first viewpoint defined as the viewpoint of the user 900 in the real space. In this embodiment, the second viewpoint may be a position virtually set in the real space, may be a position a predetermined distance away from the position of the virtual object 600 or the user 900 in the real space, or may be a position set on the virtual object 600. The non-AR device 200 may be, for example, a smartphone or tablet PC (Personal Computer) carried by the user 900, or a smart watch worn on the wrist of the user 900. Furthermore, as shown in FIG. 2, the non-AR device 200 has a display unit 202 that displays the virtual object 600, and a control unit 204 that controls the display unit 202 according to control parameters and the like from a control unit 500 described later.

[0040] The display unit 202 is provided on the surface of the non-AR device 200, and is controlled by the control unit 204 to display the virtual object 600 in a non-AR manner to the user 900. The display unit 202 can be realized by a display device such as a liquid crystal display (LCD) device or an organic light emitting diode (OLED) device.

[0041] The control unit 204 controls the overall operation of the display unit 202 according to control parameters and the like from the control unit 500 described below. The control unit 204 is realized by electronic circuits of a microprocessor such as a CPU or GPU. The control unit 204 may also include a ROM for storing programs to be used, calculation parameters, and the like, and a RAM for temporarily storing parameters that change as appropriate.

[0042] Furthermore, the non-AR device 200 may have a communication unit (not shown) that is a communication interface that can connect to an external device via wireless communication, etc. The communication unit is realized by a communication device such as a communication antenna, a transmission / reception circuit, or a port.

[0043] Furthermore, in this embodiment, the non-AR device 200 may be provided with an input unit (not shown) for the user 900 to perform an input operation. The input unit is configured with an input device such as a touch panel or a button. In this embodiment, the non-AR device 200 can function as a controller that can change the operation, position, etc. of the virtual object 600. In addition, the non-AR device 200 may be provided with a speaker (not shown) that outputs sound toward the user 900, a camera (not shown) that can capture an image of a real object in the real space or the user 900, and the like.

[0044] Furthermore, in this embodiment, the non-AR device 200 may be provided with a depth measurement unit 300, a gaze sensor unit 400, and a control unit 500, as described below. In addition, in this embodiment, the non-AR device 200 may be provided with a positioning sensor (not shown). Furthermore, in this embodiment, the non-AR device 200 may be provided with a motion sensor (not shown) including an acceleration sensor, a gyro sensor, a geomagnetic sensor, etc.

[0045] (Depth Measurement Unit 300) The depth measurement unit 300 can acquire three-dimensional information of the real space around the user 900. In detail, as shown in FIG. 2, the depth measurement unit 300 has a depth sensor unit 302 that can acquire three-dimensional information and a storage unit 304 that stores the acquired three-dimensional information. For example, the depth sensor unit 302 may be an imaging device such as a TOF (Time Of Flight) sensor (distance measuring device) that acquires depth information of the real space around the user 900, and a stereo camera or a structured light sensor. In this embodiment, the three-dimensional information of the real space around the user 900 acquired by the depth sensor unit 302 is not only used as environmental information around the user 900, but can also be used to obtain position information including distance information and positional relationship information between the virtual object 600 and the user 900 in the real space.

[0046] In detail, the TOF sensor irradiates irradiation light such as infrared light into the real space around the user 900, and detects the reflected light reflected by the surface of a real object (such as a wall) in the real space. The TOF sensor can obtain the distance (depth information) from the TOF sensor to the real object by calculating the phase difference between the irradiation light and the reflected light, and can therefore obtain a distance image including the distance information (depth information) to the real object as three-dimensional shape data of the real space. Note that the method of obtaining distance information by the phase difference as described above is called an indirect TOF method. In addition, in this embodiment, it is also possible to use a direct TOF method that can obtain the distance (depth information) from the TOF sensor to the real object by detecting the round-trip time of light from the time when the irradiation light is emitted until the irradiation light is reflected by the real object and received as reflected light.

[0047] Here, the distance image is, for example, information generated by linking distance information (depth information) acquired for each pixel of the TOF sensor to position information of the corresponding pixel. Also, the three-dimensional information here is three-dimensional coordinate information in real space (more specifically, a collection of multiple pieces of three-dimensional coordinate information) generated by converting the position information of the pixel in the distance image into coordinates in real space based on the position of the TOF sensor in real space, and linking the corresponding distance information to the coordinates obtained by conversion. In this embodiment, by using such a distance image and three-dimensional information, the position and shape of an obstacle (wall, etc.) in real space can be grasped.

[0048] Furthermore, in this embodiment, when a TOF sensor is provided in the AR device 100, the position and posture of the user 900 in the real space may be detected by comparing the three-dimensional information obtained by the TOF sensor with a three-dimensional information model (position and shape of a wall, etc.) of the same real space (such as a room) previously acquired. Also, in this embodiment, when a TOF sensor is installed in the real space (such as a room), the position and posture of the user 900 in the real space may be detected by extracting a human shape from the three-dimensional information obtained by the TOF sensor. In this embodiment, the position information of the user 900 thus detected can be used to obtain position information including distance information and positional relationship information between the virtual object 600 and the user 900 in the real space. Furthermore, in this embodiment, a virtual landscape of the real space (an illustration imitating the real space) based on the above three-dimensional information may be generated and displayed on the above-mentioned non-AR device 200, etc.

[0049] The Structured Light sensor can obtain a distance image including the distance (depth information) from the Structured Light sensor to a real object based on the deformation of the predetermined pattern obtained from the imaging result by irradiating the real space around the user 900 with light such as infrared light and capturing the image. Furthermore, the stereo camera can simultaneously capture the real space around the user 900 from two different directions with two cameras and obtain the distance (depth information) from the stereo camera to a real object by utilizing the parallax of these cameras.

[0050] Furthermore, the storage unit 304 can store a program for the depth sensor unit 302 to execute sensing, and three-dimensional information obtained by sensing. The storage unit 304 is realized by, for example, a magnetic recording medium such as a hard disk (HD) or a non-volatile memory such as a flash memory.

[0051] The depth measurement unit 300 may also have a communication unit (not shown) that is a communication interface that can connect to an external device via wireless communication, etc. The communication unit is realized by a communication device such as a communication antenna, a transmission / reception circuit, or a port.

[0052] Furthermore, in this embodiment, as described above, the depth measurement unit 300 may be provided in the above-mentioned AR device 100 or the non-AR device 200. Alternatively, in this embodiment, the depth measurement unit 300 may be installed in a real space (e.g., indoors, etc.) around the user 900, and in this case, position information of the depth measurement unit 300 in the real space is assumed to be known.

[0053] (Gaze sensor unit 400) The gaze sensor unit 400 can capture an image of the eyeball of the user 900 and detect the gaze of the user 900. The gaze sensor unit 400 will be mainly used in the embodiment described later. The gaze sensor unit 400 can be configured as an inward-facing camera (not shown) in the HMD, which is the AR device 100. Then, the gaze direction of the user 900 is detected by analyzing the captured image of the eye of the user 900 acquired by the inward-facing camera. In this embodiment, the gaze detection algorithm is not particularly limited, but gaze detection can be realized based on, for example, the positional relationship between the inner corner of the eye and the iris, or the positional relationship between the corneal reflex (Purkinje image, etc.) and the pupil. In this embodiment, the gaze sensor unit 400 is not limited to the inward-facing camera described above, and may be a camera that can capture the eyeball of the user 900, or an electrooculography sensor that measures the electrooculography by attaching electrodes around the eyes of the user 900. Furthermore, in this embodiment, the gaze direction of the user 900 may be recognized by using a model obtained by machine learning. Details of the recognition of the gaze direction will be described in the embodiment described later.

[0054] The gaze sensor unit 400 may also have a communication unit (not shown) that is a communication interface that can connect to an external device via wireless communication, etc. The communication unit is realized by a communication device such as a communication antenna, a transmission / reception circuit, or a port.

[0055] Furthermore, in this embodiment, as described above, the gaze sensor unit 400 may be provided in the above-mentioned AR device 100 or the non-AR device 200. Alternatively, in this embodiment, the gaze sensor unit 400 may be installed in a real space (for example, indoors, etc.) around the user 900, and in this case, position information of the gaze sensor unit 400 in the real space is assumed to be known.

[0056] (Control unit 500) The control unit 500 is a device for controlling the display in the above-mentioned AR device 100 and the non-AR device 200. In detail, in this embodiment, the AR display of the virtual object 600 by the AR device 100 is controlled by the control unit 500 using parameters that change dynamically according to the distance between the user 900 and the virtual position of the virtual object 600 in the real space and the position of the viewpoint of the user 900. Furthermore, in this embodiment, the display of the virtual object 600 by the non-AR device 200 is also controlled by the control unit 500 using predefined parameters. Moreover, the control unit 500 can be mainly configured with a CPU, a RAM, a ROM, and the like. Furthermore, the control unit 500 may have a communication unit (not shown) that is a communication interface that can be connected to an external device by wireless communication or the like. The communication unit is realized by a communication device such as a communication antenna, a transmission / reception circuit, or a port.

[0057] In addition, in this embodiment, as described above, the control unit 500 may be provided in the above-mentioned AR device 100 or the non-AR device 200 (provided as an integrated unit), and in this way, delays during display control can be suppressed. Alternatively, in this embodiment, the control unit 500 may be provided as a device separate from the AR device 100 or the non-AR device 200 (for example, it may be a server or the like present on a network). The detailed configuration of the control unit 500 will be described later.

[0058] <2.2 Detailed configuration of the control unit 500> Next, a detailed configuration of the control unit 500 according to this embodiment will be described with reference to Fig. 2. As described above, the control unit 500 can control the display of the virtual object 600 displayed on the AR device 100 and the non-AR device 20. In detail, as shown in Fig. 2, the control unit 500 mainly includes a three-dimensional information acquisition unit (position information acquisition unit) 502, an object control unit (control unit) 504, an AR device rendering unit 506, a non-AR device rendering unit 508, a detection unit (selection result acquisition unit) 510, and a gaze evaluation unit 520. Details of each functional unit of the control unit 500 will be described below in order.

[0059] (3D information acquisition unit 502) The three-dimensional information acquisition unit 502 acquires three-dimensional information of the real space around the user 900 from the depth measurement unit 300 described above, and outputs it to the object control unit 504 described later. The three-dimensional information acquisition unit 502 may extract information such as the position, posture, and shape of a real object in the real space from the three-dimensional information, and output it to the object control unit 504. The three-dimensional information acquisition unit 502 may also refer to position information in the real space virtually assigned for displaying the virtual object 600, generate position information including distance information and positional relationship information between the virtual object 600 and the user 900 in the real space based on the three-dimensional information, and output it to the object control unit 504. Furthermore, the three-dimensional information acquisition unit 502 may acquire position information of the user 900 in the real space not only from the depth measurement unit 300 but also from the positioning sensor (not shown) described above.

[0060] (Object control unit 504) The object control unit 504 controls the display of the virtual object 600 in the AR device 100 and the non-AR device 200 according to the representation method assigned to each of the AR device 100 and the non-AR device 200 for displaying the virtual object 600. In detail, the object control unit 504 dynamically changes each parameter related to the display of the virtual object 600 (for example, the display change amount of the virtual object 600 in the moving image display, the display change amount changed by the input operation of the user 900, etc.) according to the representation method assigned to each of the AR device 100 and the non-AR device 200 for displaying the virtual object 600. Then, the object control unit 504 outputs the parameters changed in this way to the AR device rendering unit 506 and the non-AR device rendering unit 508 described later. The output parameters are used to control the display of the virtual object 600 in the AR device 100 and the non-AR device 200.

[0061] More specifically, the object control unit 504 dynamically changes parameters related to the display of the virtual object 600 on the AR device 100 in accordance with position information including the distance between the virtual object 600 and the user 900 in real space, based on the above-mentioned three-dimensional information acquired from the depth measurement unit 300, for example.

[0062] More specifically, the size of the virtual object 600 displayed by the AR device 100 becomes smaller as the distance between the virtual object 600 and the user 900 increases so that the user 900 perceives the virtual object 600 as a real object existing in real space. Therefore, the visibility of the virtual object 600 for the user 900 decreases, and for example, when the virtual object 600 is a game character, it becomes difficult to see the minute movements of the character. Therefore, in this embodiment, the object control unit 504 changes the parameters so that the display change amount (degree of quantization of the movement (jump, etc.) of the virtual object 600) in the moving image display of the virtual object 600 displayed by the AR device 100 increases as the distance increases. In addition, the object control unit 504 changes the parameters so that the trajectory of the moving image display of the virtual object 600 displayed by the AR device 100 becomes smoother as the distance increases. In this way, in this embodiment, even if the size of the virtual object 600 displayed on the AR device 100 is reduced so that the user 900 perceives it as a real object existing in real space, it is possible to suppress a decrease in visibility of the movement of the virtual object 600.

[0063] In addition, in this embodiment, although the user 900 is less likely to perceive the virtual object 600 as a real object existing in real space, the object control unit 504 may change the parameters so that the display area of ​​the virtual object 600 displayed on the AR device 100 increases as the distance between the virtual object 600 and the user 900 increases. Furthermore, in this embodiment, the object control unit 504 may change the parameters so that the virtual object 600 can more easily get closer to or move away from other virtual objects displayed on the AR device 100, or can more easily perform an action such as an attack.

[0064] Meanwhile, in this embodiment, the object control unit 504 uses predefined parameters (for example, fixed values) as parameters related to the display of the virtual object 600 on the non-AR device 200. Note that in this embodiment, the predefined parameters may be used for displaying the virtual object 600 on the non-AR device 200 after being processed according to a predetermined rule.

[0065] (AR device rendering unit 506) The AR device rendering unit 506 uses the parameters output from the object control unit 504 described above to perform rendering processing of an image to be displayed on the AR device 100, and outputs the rendered image data to the AR device 100.

[0066] (Non-AR device rendering unit 508) The non-AR device rendering unit 508 uses parameters output from the object control unit 504 described above to perform rendering processing of an image to be displayed on the non-AR device 200, and outputs the rendered image data to the non-AR device 200.

[0067] (Detection unit 510) As shown in FIG. 2, the detection unit 510 mainly includes a gaze detection unit 512 and a gaze analysis unit 514. The gaze detection unit 512 detects the gaze of the user 900 and acquires the gaze direction of the user 900, and the gaze analysis unit 514 specifies a device that the user 900 would have selected as a controller (input device) based on the gaze direction of the user 900. The specified result (selection result) is then subjected to evaluation processing by a gaze evaluation unit 520 described later, and is then output to the object control unit 504, and is used when changing parameters related to the display of the virtual object 600. Details of the processing by the detection unit 510 will be described later in the third embodiment of the present disclosure.

[0068] (Gaze evaluation unit 520) The gaze evaluation unit 520 can evaluate the result of the identification by calculating the probability that the user 900 selects each device as a controller using a model obtained by machine learning for the device that the user 900 would have selected as a controller, which is identified by the above-mentioned detection unit 510. In this embodiment, the gaze evaluation unit 520 calculates the probability that the user 900 selects each device as a controller, and based on this, the device selected by the user 900 as a controller is finally identified. Therefore, even if the destination of the user 900's gaze is not always fixed, the device selected as a controller can be accurately identified based on the direction of the user 900's gaze. Details of the processing by the gaze evaluation unit 520 will be described later in the third embodiment of the present disclosure.

[0069] <2.3 Information processing method> Next, an information processing method according to the first embodiment of the present disclosure will be described with reference to Fig. 3 to Fig. 7. Fig. 3 is a flowchart for explaining an example of the information processing method according to the present embodiment, Fig. 4 to Fig. 6 are explanatory diagrams for explaining an example of a display according to the present embodiment, and Fig. 7 is an explanatory diagram for explaining an example of a display control according to the present embodiment.

[0070] In detail, the information processing method according to this embodiment may include steps S101 to S105 as shown in Fig. 3. Each of these steps according to this embodiment will be described in detail below.

[0071] First, the control unit 500 determines whether the display device to be controlled includes the AR device 100 that performs AR display (step S101). If the AR device 100 is included (step S101: Yes), the control unit 500 proceeds to processing of step S102, and if the AR device 100 is not included (step S101: No), the control unit 500 proceeds to processing of step S105.

[0072] Next, the control unit 500 acquires position information including information on the position and posture of the user 900 in the real space (step S102). Furthermore, the control unit 500 calculates the distance between the virtual object 600 and the user 900 in the real space based on the acquired position information.

[0073] Then, the control unit 500 controls the display of the virtual object 600 displayed on the AR device 100 according to the distance calculated in the above-mentioned step S102 (distance-dependent control) (step S103). In detail, the control unit 500 dynamically changes parameters related to the display of the virtual object 600 on the AR device 100 according to the distance and the positional relationship between the virtual object 600 and the user 900 in the real space.

[0074] More specifically, as shown in FIG. 4, the display unit 102 of the AR device 100 displays the virtual object 600 by superimposing it on an image of real space (for example, an image of a real object 800) seen from a viewpoint (first viewpoint) 700 of a user 900 wearing the AR device 100. At this time, the control unit 500 dynamically changes the above parameters so that the virtual object 600 is displayed in a form seen from the viewpoint (first viewpoint) 700 of the user 900 so that the user 900 can perceive the virtual object 600 as a real object existing in real space. Furthermore, the control unit 500 dynamically changes the above parameters so that the virtual object 600 is displayed so as to have a size according to the distance calculated in the above-mentioned step S102. Then, the control unit 500 uses the parameters obtained in this way to perform a rendering process of an image to be displayed on the AR device 100, and outputs the rendered image data to the AR device 100, thereby enabling distance-dependent control of the AR display of the virtual object 600 on the AR device 100. In this embodiment, when the virtual object 600 moves or the user 900 moves or changes his / her posture, distance-dependent control is performed on the displayed virtual object 600 accordingly. In this way, the virtual object 600 displayed in AR can be perceived by the user 900 as if it were a real object existing in real space.

[0075] Next, the control unit 500 determines whether the display device to be controlled includes a non-AR device 200 that performs non-AR display (step S104). If the non-AR device 200 is included (step S104: Yes), the control unit 500 proceeds to the process of step S105, and if the non-AR device 200 is not included (step S104: No), the control unit 500 ends the process.

[0076] Then, the control unit 500 controls the display of the virtual object 600 displayed on the non-AR device 200 according to the parameters defined (set) in advance (step S105). Then, the control unit 500 ends the processing of the information processing method.

[0077] More specifically, as shown in Fig. 4, the display unit 202 of the non-AR device 200 displays the virtual object 600 (more specifically, an image of the back of the virtual object 600) as viewed from a viewpoint (second viewpoint) 702 virtually fixed in real space. At this time, the control unit 500 selects parameters defined (set) in advance and changes the selected parameters depending on the situation. Furthermore, the control unit 500 uses the parameters to perform rendering processing of an image to be displayed on the non-AR device 200, and outputs the rendered image data to the non-AR device 200, thereby controlling the non-AR display of the virtual object 600 on the non-AR device 200.

[0078] In addition, in this embodiment, as shown in FIG. 5, when the viewpoint 702 is located on the opposite side of the user 900 from the virtual position of the virtual object 600 in real space, the display unit 202 of the non-AR device 200 may display the virtual object 600 in a form different from that shown in FIG. 4 (more specifically, the front of the virtual object 600).

[0079] 6, in this embodiment, when the viewpoint 702 is virtually placed on the virtual object 600, the display unit 202 of the non-AR device 200 may display an avatar 650 that evokes an image of the user 900 as seen from the viewpoint 702. In this case, when the virtual object 600 moves or the user 900 moves or changes his / her posture, the form of the displayed avatar 650 may be changed accordingly.

[0080] 3 may be executed repeatedly, triggered by a change in the virtual position of the virtual object 600 in the real space or a change in the position or posture of the user 900. In this way, the virtual object 600 displayed in AR by the AR device 100 can be perceived by the user 900 as if it were a real object existing in the real space.

[0081] In this embodiment, as described above, parameters related to the display of the virtual object 600 in the AR device 100 are dynamically changed (distance-dependent control) according to the distance between the virtual object 600 and the user 900 in real space. Therefore, a specific example of control of the virtual object 600 AR-displayed by the AR device 100 in this embodiment will be described with reference to FIG.

[0082] More specifically, as shown in FIG. 7, the greater the distance between the virtual object 600 and the user 900, the smaller the size of the virtual object 600 displayed by the AR device 100 becomes so that the user 900 perceives the virtual object 600 as a real object existing in real space. Therefore, the visibility of the virtual object 600 for the user 900 decreases, and for example, when the virtual object 600 is a game character, it becomes difficult to see the minute movements of the character. Therefore, in this embodiment, the control unit 500 changes the parameters so that the greater the distance, the greater the display change amount (jump amount, movement amount, direction quantization amount) in the moving image display of the virtual object 600 displayed by the AR device 100. Also, the control unit 500 changes the parameters so that the greater the distance, the greater the smoothing of the trajectory in the moving image display of the virtual object 600 displayed by the AR device 100. In this way, in this embodiment, even if the size of the virtual object 600 displayed on the AR device 100 is reduced so that the user 900 perceives it as a real object existing in real space, it is possible to suppress a decrease in visibility of the movement of the virtual object 600.

[0083] Further, in this embodiment, as shown in FIG. 7, the control unit 500 may change parameters such that the virtual object 600 moves closer to or farther from another virtual object 602 displayed on the AR device 100, for example, triggered by an operation from the user 900, according to the distance between the virtual object 600 and the user 900 in the real space. Also, in this embodiment, the control unit 500 may change parameters such that the virtual object 600 is more likely to perform an action such as an attack on another virtual object 602, for example, triggered by an operation from the user 900, according to the distance. In this way, in this embodiment, even if the size of the virtual object 600 displayed on the AR device 100 is reduced so that the user 900 perceives it as a real object existing in the real space, a decrease in the operability of the virtual object 600 can be suppressed.

[0084] In addition, in this embodiment, although the user 900 is less likely to perceive the virtual object 600 as a real object existing in real space, the control unit 500 may change parameters so that the display area of ​​the virtual object 600 displayed on the AR device 100 increases as the distance between the virtual object 600 and the user 900 increases.

[0085] As described above, according to this embodiment, in the AR device 100 and the non-AR device, which have different ways of user perception, the display of the virtual object 600 has different forms, changes differently, or reacts differently to operations from the user 900, thereby further improving the user experience and operability.

[0086] <<3. Second embodiment>> First, a situation assumed in the second embodiment of the present disclosure will be described with reference to FIG. 8. FIG. 8 is an explanatory diagram for explaining an outline of the present embodiment. For example, when a user 900 is playing a game using the information processing system 10 according to the present embodiment, as shown in FIG. 8, there may be a shield 802 such as a wall between the user 900 and the virtual object 600 in the real space, blocking the view of the user 900. In such a situation, the user 900 cannot view the virtual object 600 using the display unit 102 of the AR device 100 due to the shield 802, and therefore it becomes difficult for the user 900 to operate the virtual object 600.

[0087] Therefore, in this embodiment, the display of the virtual object 600 is dynamically changed depending on whether or not the display of the entire or part of the virtual object 600 in the AR device 100 is obstructed by the obstruction 802 (occurrence of occlusion). In detail, for example, when the virtual object 600 cannot be viewed by using the display unit 102 of the AR device 100 due to the presence of the obstruction 802, the display position of the virtual object 600 is changed to a position where the viewing is not obstructed by the obstruction 802. By doing so, in this embodiment, even if the obstruction 802 that obstructs the view of the user 900 exists between the user 900 and the virtual object 600 in the real space, the user 900 can easily view the virtual object 600 using the display unit 102 of the AR device 100. As a result, according to this embodiment, it becomes easy for the user 900 to operate the virtual object 600.

[0088] In this embodiment, the display of the virtual object 600 on the AR device 100 may be dynamically changed not only when the virtual object 600 cannot be seen due to the presence of the obstruction 802, but also when the depth measurement unit 300 cannot acquire depth information of the surroundings of the virtual object 600 in the real space (for example, when a transparent real object or a black real object exists in the real space, or when noise or the like occurs in the depth sensor unit 302). Alternatively, in this embodiment, the AR device 100 may superimpose (AR display) another virtual object 610 (see FIG. 11) in the real space in an area where depth information cannot be acquired. Details of this embodiment will be described below.

[0089] <3.1 Detailed configuration of the control unit 500> The configuration example of the information processing system 10 and the control unit 500 according to this embodiment is similar to that of the first embodiment described above, so the description will be omitted here. However, in this embodiment, the object control unit 504 of the control unit 500 also has the following functions.

[0090] In detail, in this embodiment, when a shielding object (shielding object) 802, which is a real object, is present between the virtual object 600 and the user 900 in the real space based on the above three-dimensional information, the object control unit 504 sets the region where the shielding object 802 exists as an occlusion region. Furthermore, the object control unit 504 changes parameters to change the display position or display form of the virtual object 600 in the AR device 100, or the amount of movement in the moving image display of the virtual object 600, so as to reduce the region where the virtual object 600 and the occlusion region overlap.

[0091] In addition, in this embodiment, when an area in which three-dimensional information cannot be acquired is detected (for example, when a transparent real object or a black real object exists in the real space, or when noise or the like occurs in the depth sensor unit 302), the object control unit 504 sets the area as an undefined area. Furthermore, the object control unit 504 changes parameters to change the display position or display form of the virtual object 600 in the AR device 100, or the amount of movement in the moving image display of the virtual object 600, so as to reduce the area in which the virtual object 600 and the undefined area overlap. Furthermore, in this embodiment, the object control unit 504 may generate parameters for displaying another virtual object (another virtual object) 610 (see FIG. 11) in the undefined area.

[0092] <3.2 Information processing method> Next, an information processing method according to a second embodiment of the present disclosure will be described with reference to Fig. 9 to Fig. 11. Fig. 9 is a flowchart for explaining an example of the information processing method according to the present embodiment, Fig. 10 is an explanatory diagram for explaining an example of display control according to the present embodiment, and Fig. 11 is an explanatory diagram for explaining an example of display according to the present embodiment.

[0093] In detail, as shown in Fig. 9, the information processing method according to this embodiment can include steps S201 to S209. Each of these steps according to this embodiment will be described in detail below. In the following description, only the points different from the first embodiment described above will be described, and the points in common with the first embodiment will not be described.

[0094] Steps S201 and S202 are similar to steps S101 and S102 in the first embodiment shown in FIG. 3, and therefore the description thereof will be omitted here.

[0095] First, the control unit 500 determines whether or not three-dimensional information on the periphery of the setting position of the virtual object 600 in the real space has been acquired (step S203). If the control unit 500 has acquired three-dimensional information on the periphery of the virtual object 600 in the real space (step S203: Yes), the control unit 500 proceeds to processing of step S204, and if the control unit 500 has not acquired three-dimensional information on the periphery of the virtual object 600 in the real space (step S203: No), the control unit 500 proceeds to processing of step S205.

[0096] Step S204 is similar to step S103 in the first embodiment shown in FIG. 3, and therefore a description thereof will be omitted here.

[0097] Next, the control unit 500 determines whether or not three-dimensional information about the periphery of the virtual object 600 cannot be acquired due to the obstruction 802 (step S205). That is, if three-dimensional information (position, orientation, shape) about the obstruction 802 can be acquired but three-dimensional information about the periphery of the setting position of the virtual object 600 in real space cannot be acquired (step S205: Yes), the process proceeds to step S206, and if three-dimensional information about the periphery of the virtual object 600 cannot be acquired due to, for example, noise of the depth sensor unit 302 rather than the presence of the obstruction 802 (step S205: No), the process proceeds to step S207.

[0098] Next, the control unit 500 sets the region where the obstruction 802 exists as an occlusion region. Then, the control unit 500 changes the display position or display form of the virtual object 600 in the AR device 100, or the amount of movement in the moving image display of the virtual object 600, so as to reduce the region where the virtual object 600 and the occlusion region overlap (distance-dependent control of the occlusion region) (step S206).

[0099] More specifically, in this embodiment, as shown in Fig. 10, when the virtual object 600 is in a position where the whole or part of the virtual object 600 is hidden by an obstruction 802, the amount of movement in the parallel direction is increased (the movement speed is increased or warped) so that the virtual object 600 can be seen, or a situation where the virtual object 600 can be seen is soon to come. In a similar case, in this embodiment, the virtual object 600 may be controlled to jump high so that the virtual object 600 can be seen, as shown in Fig. 10. Furthermore, in this embodiment, the direction in which the virtual object 600 can move may be limited (for example, movement in the depth direction in Fig. 10 is limited) so that the virtual object 600 can be seen.

[0100] Next, the control unit 500 sets an area around the virtual object 600 where three-dimensional information could not be acquired due to noise or the like as an undefined area. Then, similar to step S206 described above, the control unit 500 changes the display position or display form of the virtual object 600 in the AR device 100, or the amount of movement in the moving image display of the virtual object 600, so as to reduce the area where the virtual object 600 and the undefined area overlap (distance-dependent control of the undefined area) (step S207).

[0101] More specifically, in step S207, similar to step S206 described above, when the entire or part of the virtual object 600 is in a position where it is hidden in the indefinite region, the amount of movement in the parallel direction is increased (the movement speed is increased or the object is warped) so that the virtual object 600 can be seen, or a situation where the virtual object 600 can be seen is soon to come. In a similar case, in step S207, similar to step S206 described above, the virtual object 600 may be controlled to jump high so that the virtual object 600 can be seen. Furthermore, in this embodiment, the direction in which the virtual object 600 can move may be restricted so that the virtual object 600 can be seen.

[0102] Furthermore, in step S207, as shown in FIG. 11, the AR device 100 may display another virtual object (another virtual object) 610 so as to correspond to the undefined region.

[0103] Steps S208 and S209 are similar to steps S104 and S105 in the first embodiment shown in FIG. 3, and therefore the description thereof will be omitted here.

[0104] 9 may be executed repeatedly, triggered by a change in the virtual position of the virtual object 600 in the real space or a change in the position or posture of the user 900. In this way, the virtual object 600 displayed in AR by the AR device 100 can be perceived by the user 900 as if it were a real object existing in the real space.

[0105] As described above, according to the present embodiment, even if an obstruction 802 that blocks the view of the user 900 is present between the user 900 and the virtual object 600 in real space, the user 900 can easily view the virtual object 600 using the display unit 102 of the AR device 100. As a result, according to the present embodiment, the user 900 can easily operate the virtual object 600.

[0106] <<4. Third embodiment>> First, a situation assumed in the third embodiment of the present disclosure will be described with reference to FIG. 12. FIG. 13 is an explanatory diagram for explaining an outline of the present embodiment. For example, when a user 900 plays a game using the information processing system 10 according to the present embodiment, the user 900 visually recognizes the same virtual object 600 using both the AR device 100 and the non-AR device 200, and is able to operate the virtual object, as shown in FIG. 12. That is, operations on the virtual object 600 using the AR device 100 and the non-AR device 200 are not exclusive.

[0107] In such a situation, it is required to control the display of the virtual object 600 according to the device selected by the user 900 as a controller (operation device) from the AR device 100 and the non-AR device 200. That is, in such a situation, even if the operation on the virtual object 600 is the same for the user 900, it is required to further improve the user experience and operability by changing the form (e.g., the amount of change, etc.) of the virtual object 600 displayed in each case according to the device selected as the controller.

[0108] Therefore, in this embodiment, the device selected by the user 900 as a controller is identified based on the line of sight of the user 900, and the display of the virtual object 600 is dynamically changed based on the identification result. In this embodiment, for example, when the user 900 selects the AR device 100, the display of the virtual object 600 is controlled in a distance-dependent manner as described above, and when the user 900 selects the non-AR device 200, the display of the virtual object 600 is controlled with predefined parameters. According to this embodiment, by performing such control, even if the operation on the virtual object 600 is the same for the user 900, the form of the displayed virtual object 600 changes depending on the device selected as a controller, and therefore the user experience and operability can be further improved.

[0109] Furthermore, in this embodiment, the device selected by the user 900 as a controller is identified based on the direction of the user's line of sight. However, in the above-mentioned situation, since the user 900 can use both the AR device 100 and the non-AR device 200, it is assumed that the direction of the user's line of sight is not fixed but constantly moving. Therefore, when the direction of the line of sight is not always fixed, it is difficult to identify the device based on the direction of the user's line of sight, and further, it is difficult to identify the device with high accuracy. In addition, if the device selected based on the direction of the user's line of sight is simply identified and the display of the virtual object 600 is dynamically changed based on the identification result, the movement of the virtual object 600 becomes discontinuous every time the identified device changes, and it is considered that operability is deteriorated.

[0110] Therefore, in this embodiment, the probability that the user 900 will select each device as a controller is calculated, and the device selected by the user 900 as a controller is identified based on the probability, and the display of the virtual object 600 is dynamically changed based on the identification result. According to this embodiment, by doing so, even if the direction of the user 900's gaze is not always fixed, the device selected as the controller can be accurately identified based on the direction of the user 900's gaze. Furthermore, according to this embodiment, by doing so, it is possible to prevent the movement of the virtual object 600 from becoming discontinuous, and to avoid deterioration of operability.

[0111] <4.1 Detailed configuration of the control unit 500> The configuration examples of the information processing system 10 and the control unit 500 according to this embodiment are similar to those of the first embodiment, and therefore will not be described here. However, in this embodiment, the control unit 500 also has the following functions.

[0112] In detail, in this embodiment, the object control unit 504 can dynamically change parameters related to the display of the virtual object 600 depending on the device selected by the user 900 as a controller, for example, so that the amount of display change caused by the input operation of the user 900 changes.

[0113] <4.2 Information processing method> Next, an information processing method according to a third embodiment of the present disclosure will be described with reference to Fig. 13 to Fig. 15. Fig. 13 and Fig. 14 are flowcharts for explaining an example of the information processing method according to this embodiment, and in detail, Fig. 14 is a sub-flowchart of step S301 shown in Fig. 13. Also, Fig. 15 is an explanatory diagram for explaining an example of a method for identifying a selected device according to this embodiment.

[0114] In detail, as shown in Fig. 13, the information processing method according to this embodiment can include steps S301 to S305. Each of these steps according to this embodiment will be described in detail below. Note that in the following description, only the points different from the first embodiment described above will be described, and the description of points in common with the first embodiment will be omitted.

[0115] First, the control unit 500 identifies the device selected by the user 900 as a controller based on the line of sight of the user 900 (step S301). Note that detailed processing of step S301 will be described later with reference to FIG.

[0116] Next, the control unit 500 determines whether the device identified in the above-mentioned step S301 is the AR device 100 (step S302). If the identified device is the AR device 100 (step S302: Yes), the process proceeds to step S303, and if the identified device is the non-AR device 200 (step S302: No), the process proceeds to step S305.

[0117] Steps S303 to S305 are similar to steps S102, S103, and S105 in the first embodiment shown in FIG. 3, and therefore a description thereof will be omitted here.

[0118] 13 may be repeatedly executed, triggered by a change in the virtual position of the virtual object 600 in the real space or a change in the position or posture of the user 900. In this manner, the virtual object 600 AR-displayed by the AR device 100 can be perceived by the user 900 as a real object existing in the real space. Furthermore, in this embodiment, the information processing method shown in FIG. 13 may be repeatedly executed, triggered by a change in the device selected as a controller by the user 900 based on the line of sight of the user 900.

[0119] Next, detailed processing of step S301 in Fig. 13 will be described with reference to Fig. 14. In detail, as shown in Fig. 14, step S301 according to this embodiment can include sub-steps from step S401 to step S404. Details of each of these steps according to this embodiment will be described below.

[0120] First, the control unit 500 identifies the gaze direction of the user 900 based on sensing data from the gaze sensor unit 400 that detects the movement of the user's 900's eyeball (step S401). In detail, the control unit 500 can identify the gaze direction of the user 900 based on the positional relationship between the inner corner of the eye and the iris, for example, using an image of the user's eyeball obtained by the gaze sensor unit 400. Note that in this embodiment, the gaze direction of the user 900 identified within a certain time period may have multiple results because the user's eyeball is constantly moving. In addition, in step S401, the gaze direction of the user 900 may be identified using a model obtained by machine learning.

[0121] Next, the control unit 500 identifies the virtual object 600 that the user 900 focuses on based on the gaze direction identified in step S401 described above (step S402). For example, as shown in FIG. 15, it is possible to identify whether the virtual object 600 that the user 900 focuses on is the virtual object 600a displayed on the AR device 100 shown in the upper part of FIG. 15 or the virtual object 600b displayed on the non-AR device 200 shown in the lower part of FIG. 15, based on the gaze direction angles a and b relative to the horizontal line extending from the eye 950 of the user 900. Note that in this embodiment, when results of a plurality of gaze directions are obtained, the virtual object 600 that the user 900 focuses on corresponding to each gaze direction is identified. Furthermore, in step S402, the virtual object 600 that the user 900 focuses on may be identified using a model obtained by machine learning.

[0122] Next, the control unit 500 evaluates the identified results by calculating the probability that the user 900 is paying attention to the virtual objects 600 identified in the above-mentioned step S402, and by calculating the probability that the user 900 will select the device displaying each virtual object 600 as a controller (step S403).

[0123] In detail, for example, a moving virtual object 600 has a high probability of attracting the user 900's attention, and for example, a vividly colored virtual object 600 has a high probability of attracting the user 900's attention. Furthermore, for example, a virtual object 600 displayed with a voice output (effect) that sounds like a voice is speaking also has a high probability of attracting the user 900's attention. In addition, if the virtual object 600 is a game character, the probability of attracting the user 900's attention varies depending on the profile (role (protagonist, companion, enemy, etc.)) assigned to the character. Therefore, the probability of each virtual object 600 attracting the user 900's attention is calculated based on information (motion, size, shape, color, profile) and the like related to such identified virtual object 600. In addition, at this time, the control unit 500 may calculate the above probability by using a model obtained by machine learning, and may also calculate the above probability by using the movement of the user 900 detected by a motion sensor (not shown) provided in the AR device 100, or the position or posture of the non-AR device 200 detected by a motion sensor (not shown) provided in the non-AR device 200. Furthermore, when the user 900 is playing a game using the information processing system 10 according to this embodiment, the control unit 500 may calculate the above probability by using the situation in the game. In this embodiment, the calculated probability may be used when changing a parameter related to the display of the virtual object 600.

[0124] Then, the control unit 500 identifies the selected device based on the calculated probability (step S404). In this embodiment, for example, if the calculated probability is equal to or greater than a predetermined value, the device that displays the virtual object 600 corresponding to the probability is identified as the selected device selected by the user 900 as the controller. Also, in this embodiment, for example, the device that displays the virtual object 600 corresponding to the highest probability is identified as the selected device. Also, in this embodiment, the selected device may be identified by performing statistical processing such as extrapolation using the calculated probability. According to this embodiment, by doing so, even if the direction of the user 900's gaze is not always fixed, the device selected as the controller can be accurately identified based on the direction of the user 900's gaze.

[0125] As described above, in this embodiment, the device selected as a controller by the user 900 is identified based on the line of sight of the user 900, and the display of the virtual object 600 can be dynamically changed based on the identification result. According to this embodiment, by performing control in this manner, even if the operation on the virtual object 600 is the same for the user 900, the form of the displayed virtual object 600 changes depending on the device selected as a controller, thereby making it possible to further improve the user experience and operability.

[0126] Furthermore, in this embodiment, the probability that the user 900 selects each device as a controller (more specifically, the probability that the user 900 focuses on the virtual object 600) is calculated, and based on the calculated probability, the device selected by the user 900 as a controller is identified, and the display of the virtual object 600 is dynamically changed based on the identification result. According to this embodiment, by doing so, even if the direction of the user 900's gaze is not always fixed, the device selected as the controller can be accurately identified based on the direction of the user 900's gaze. Furthermore, according to this embodiment, by doing so, it is possible to prevent the movement of the virtual object 600 from becoming discontinuous, and to avoid deterioration of operability.

[0127] Furthermore, in this embodiment, in order to prevent the movement of the virtual object 600 from becoming discontinuous due to frequent changes in the parameters (control parameters) related to the display of the virtual object 600 caused by the movement of the line of sight of the user 900, the parameters related to the display of the virtual object 600 may be adjusted (interpolated) using the probability of selecting each device, rather than directly selecting the parameters related to the display of the virtual object 600 using the probability of selecting each device. For example, the probability of a device being selected as a controller based on the direction of the line of sight of the user 900 is assumed to be 0.3 for device a and 0.7 for device b. The control parameter when device a is selected as a controller is assumed to be Ca, and the control parameter when device b is selected is assumed to be Cb. In such a case, instead of setting the final control parameter C to Cb based on device b, which has a high probability of being selected as a controller, the final control parameter C may be obtained by interpolating, for example, C=0.3×Ca+0.7×Cb using the probability of selecting each device. In this way, it is possible to prevent the movement of the virtual object 600 from becoming discontinuous.

[0128] In this embodiment, in order to prevent the movement of the virtual object 600 from becoming discontinuous due to frequent changes in the parameters related to the display of the virtual object 600 caused by the movement of the gaze of the user 900, the frequency and amount of change of the parameters may be limited by the user 900 by setting them in advance. In addition, in this embodiment, for example, the parameters related to the display of the virtual object 600 may be restricted not to change while the operation of the user 900 is being performed continuously. In addition, in this embodiment, the parameters related to the display of the virtual object 600 may be changed using the detection that the user 900 has been gazing at a specific virtual object 600 for a predetermined time or more as a trigger. Furthermore, in this embodiment, the parameters related to the display of the virtual object 600 may be changed using the detection that not only the selection device is specified by the direction of the gaze of the user 900 but also the user 900 has performed a predetermined operation as a trigger.

[0129] Furthermore, in this embodiment, in order to allow the user 900 to recognize which device is identified as the controller, for example, when the AR device 100 is identified as the controller, the non-AR device 200 may not display an image from a viewpoint 702 provided on the virtual object 600. Similarly, for example, when the non-AR device 200 is identified as the controller, the AR device 100 may be caused to display the same image as the image displayed by the non-AR device 200.

[0130] <4.3 Modification> Furthermore, in this embodiment, the selection device selected by the user 900 as a controller may be identified not only by detecting the direction of the line of sight of the user 900 but also by detecting a gesture of the user 900. Hereinafter, such a modified example of this embodiment will be described with reference to Fig. 16. Fig. 16 is an explanatory diagram for explaining an overview of a modified example of the third embodiment of the present disclosure.

[0131] In detail, in this modified example, when the control unit 500 detects a predetermined gesture such as that shown in FIG. 16 from an image captured by an imaging device (gesture detection device) (not shown) that captures the movement of the hand 920 of the user 900, the control unit 500 identifies the selected device selected by the user 900 as a controller based on the detected gesture.

[0132] In addition, in this modification, if the AR device 100 is an HMD, a motion sensor (not shown) provided in the HMD may detect head movement of the user 900 wearing the HMD, and the selected device selected by the user 900 as a controller may be identified based on the detected head movement. Furthermore, in this modification, if a sound sensor (not shown) is provided in the AR device 100, the non-AR device 200, or the like, the selected device selected by the user 900 as a controller may be identified based on the voice of the user 900 or a predetermined phrase extracted from the voice.

[0133] <<5. Summary>> As described above, in each embodiment of the present disclosure, when multiple display devices that simultaneously display the same virtual object 600 are used, the display of the virtual object 600 takes on different forms, changes differently, or responds differently to operations from the user 900 in the AR device 100 and non-AR device, which have different ways of being perceived by users, thereby further improving the user experience and operability.

[0134] As described above, in each embodiment of the present disclosure, the virtual object 600 is not limited to being a game character, item, or the like, but may be, for example, an icon, text (button, etc.), a three-dimensional image, or the like as a user interface for other purposes (business tools), and is not particularly limited.

[0135] <<6. Hardware Configuration>> An information processing device such as the control unit 500 according to each of the above-described embodiments is realized by a computer 1000 having a configuration as shown in FIG. 17, for example. The control unit 500 according to the embodiment of the present disclosure will be described below as an example. FIG. 17 is a hardware configuration diagram showing an example of the computer 1000 that realizes the functions of the control unit 500. The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. Each part of the computer 1000 is connected by a bus 1050.

[0136] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processes corresponding to the various programs.

[0137] The ROM 1300 stores boot programs such as a basic input output system (BIOS) executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000.

[0138] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records an information processing program according to the present disclosure, which is an example of program data 1450.

[0139] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0140] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from the input / output device 1650, such as a keyboard, a mouse, or a microphone, via the input / output interface 1600. The CPU 1100 also transmits data to an output device, such as a display, a speaker, or a printer, via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading a program or the like recorded on a predetermined recording medium. The medium may be, for example, an optical recording medium, such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disk), a magneto-optical recording medium, such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0141] For example, when the computer 1000 functions as the control unit 500 according to an embodiment of the present disclosure, the CPU 1100 of the computer 1000 executes a program stored in the RAM 1200 to realize functions of the control unit 200, etc. Also, the information processing program according to the present disclosure is stored in the HDD 1400. The CPU 1100 reads and executes the program data 1450 from the HDD 1400, but as another example, the CPU 1100 may obtain these programs from other devices via the external network 1550.

[0142] Furthermore, the information processing device according to the present embodiment may be applied to a system consisting of a plurality of devices, which is premised on connection to a network (or communication between devices), such as cloud computing, etc. In other words, the information processing device according to the present embodiment described above can also be realized as an information processing system according to the present embodiment, for example, by a plurality of devices.

[0143] The above shows an example of the hardware configuration of the control unit 500. Each of the above components may be configured using general-purpose materials, or may be configured using hardware specialized for the function of each component. Such a configuration may be appropriately changed depending on the technical level at the time of implementation.

[0144] <<7. Supplementary Information>> The above-described embodiment of the present disclosure may include, for example, an information processing method executed by the information processing device or information processing system described above, a program for causing the information processing device to function, and a non-transitory tangible medium on which the program is recorded. The program may also be distributed via a communication line (including wireless communication) such as the Internet.

[0145] In addition, each step in the information processing method of the embodiment of the present disclosure described above does not necessarily have to be processed in the order described. For example, each step may be processed in a different order as appropriate. Also, each step may be partially processed in parallel or individually instead of being processed in chronological order. Furthermore, each step may not necessarily be processed in the manner described, and may be processed in another manner by another functional unit, for example.

[0146] Although the preferred embodiment of the present disclosure has been described in detail above with reference to the attached drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person having ordinary knowledge in the technical field of the present disclosure can conceive of various modified or amended examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally belong to the technical scope of the present disclosure.

[0147] In addition, the effects described in this specification are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that are apparent to a person skilled in the art from the description of this specification, in addition to or in place of the above effects.

[0148] The present technology can also be configured as follows. (1) a control unit that dynamically changes each parameter related to the display of the virtual object, the control unit being assigned to each of a plurality of display devices that display images related to the same virtual object for the display of the images, and that controls the display of the virtual object on each of the display devices in accordance with a representation method of the image; Information processing device. (2) The plurality of display devices include a first display device that is controlled to display a scene of the real space in which the virtual object is virtually arranged, the scene being viewed from a first viewpoint defined as a user's viewpoint in the real space; and a second display device controlled to display an image of the virtual object; Including, The information processing device according to (1) above. (3) the control unit dynamically changes the parameters for controlling the first display device in response to three-dimensional information of the real space around the user from a real space information acquisition device. The information processing device according to (2) above. (4) The real space information acquisition device is an imaging device that captures the real space around the user, or a distance measuring device that acquires depth information of the real space around the user. The information processing device according to (3) above. (5) The control unit is When a region in which an occluding object located between the virtual object and the user in the real space exists or a region in which the three-dimensional information cannot be acquired is detected based on the three-dimensional information, Setting the region as an occlusion region; changing a display position or a display form of the virtual object, or a movement amount of the virtual object in a moving image display, on the first display device so as to reduce an overlapping area between the virtual object and the occlusion area; The information processing device according to (3) or (4) above. (6) The control unit controls the first display device to display another virtual object in an undefined area where the three-dimensional information cannot be acquired. The information processing device according to (5) above. (7) a position information acquisition unit that acquires position information including distance information and positional relationship information between the virtual object and the user in the real space, The control unit dynamically changes the parameters for controlling the first display device in accordance with the position information. The information processing device according to any one of (2) to (6) above. (8) The information processing device according to (7) above, wherein the control unit controls the first display device to display a larger display area of ​​the virtual object as the distance between the virtual object and the user increases. (9) The information processing device described in (7) above, wherein the control unit controls the first display device to display a larger amount of display change in the video display of the virtual object as the distance between the virtual object and the user increases. (10) The information processing device according to (7) above, wherein the control unit controls the first display device to smooth a trajectory of the virtual object in a moving image display as the distance between the virtual object and the user becomes longer. (11) The control unit dynamically changes a display change amount of the virtual object to be displayed on the first display device, the display change amount being changed by an input operation of the user, in accordance with the position information. The information processing device according to (7) above. (12) The control unit is controlling the second display device to display an image of the virtual object viewed from a second viewpoint in the real space, the second viewpoint being different from the first viewpoint; The information processing device according to any one of (2) to (11) above. (13) The information processing device according to (12) above, wherein the second viewpoint is virtually placed on the virtual object. (14) the control unit changes a display change amount in a moving image display of the virtual object to be displayed on each of the first and second display devices, according to a representation method of the image assigned to each of the first and second display devices for displaying the image. The information processing device according to (2) above. (15) a selection result acquisition unit that acquires a selection result as to whether the user has selected one of the first display device and the second display device as an input device, The control unit dynamically changes a display change amount of the virtual object that changes in response to an input operation by the user in accordance with the selection result. The information processing device according to (2) above. (16) The information processing device according to (15) above, wherein the selection result acquisition unit acquires the selection result based on a detection result of the user's gaze from a gaze detection device. (17) The information processing device according to (15) above, wherein the selection result acquisition unit acquires the selection result based on a detection result of the user's gesture from a gesture detection device. (18) The first display device includes: an image of the virtual object is displayed superimposed on the image of the real space; projecting and displaying an image of the virtual object in the real space; Or, projecting and displaying an image of the virtual object on the user's retina; The information processing device according to any one of (2) to (17) above. (19) an information processing device controls display of the virtual object on each of a plurality of display devices, the display devices being assigned for displaying the image, according to a representation method of the image, and dynamically changes each parameter related to the display of the virtual object; 4. An information processing method comprising: (20) Computer, a control unit that dynamically changes each parameter related to the display of the virtual object, the control unit being assigned to each of a plurality of display devices that display an image related to the same virtual object, and that controls the display of the virtual object on each of the display devices in accordance with a representation method of the image; program. [Explanation of symbols]

[0149] 10. Information Processing Systems 100 AR Devices 102, 202 Display section 104, 204 Control unit 200 Non-AR Devices 300 Depth Measuring Unit 302 Depth sensor part 304 Storage section 400 Eye gaze sensor unit 500 Control Unit 502 3D information acquisition unit 504 Object Control Unit 506 AR Device Rendering Department 508 Non-AR Device Rendering Section 510 Detection unit 512 Line of Sight Detection Unit 514 Gaze Analysis Department 520 Gaze Evaluation Department 600, 600a, 600b, 602, 610 Virtual objects 650 Avatar 700, 702 Viewpoint 800 Real Objects 802 Shield 900 users 920 Hands 950 eyes a, b angle

Claims

1. a control unit that dynamically changes each parameter related to the display of the virtual object, the control unit being assigned to each of a plurality of display devices that display images related to the same virtual object for the purpose of displaying the images, and that controls the display of the virtual object on each of the display devices in accordance with a representation method of the image; a position information acquisition unit that acquires position information including distance information and positional relationship information between the virtual object and a user in a real space; Equipped with The plurality of display devices include a first display device that is controlled to display a scene of the real space in which the virtual object is virtually arranged, the scene being viewed from a first viewpoint defined as a viewpoint of the user in the real space; and a second display device controlled to display an image of the virtual object; Including, the control unit dynamically changes the parameter for controlling the first display device such that a display area of ​​the virtual object displayed on the first display device increases as a distance between the virtual object and the user increases. Information processing device.

2. the control unit dynamically changes the parameters for controlling the first display device in accordance with three-dimensional information of the real space around the user from a real space information acquisition device. The information processing device according to claim 1 .

3. The real space information acquisition device is an imaging device that captures the real space around the user, or a distance measuring device that acquires depth information of the real space around the user. The information processing device according to claim 2 .

4. The control unit is When a region in which an occluding object located between the virtual object and the user in the real space exists or a region in which the three-dimensional information cannot be acquired is detected based on the three-dimensional information, Setting the region as an occlusion region; changing a display position or a display form of the virtual object, or a movement amount of the virtual object in a moving image display, on the first display device so as to reduce an overlapping area between the virtual object and the occlusion area; 4. The information processing device according to claim 2 or 3.

5. the control unit controls the first display device to display another virtual object in an indefinite area where the three-dimensional information cannot be acquired. The information processing device according to claim 4.

6. A control unit that dynamically changes each parameter related to the display of the virtual object, the control unit being assigned to each of a plurality of display devices for displaying an image related to the same virtual object, and that controls the display of the virtual object on each of the display devices in accordance with a representation method of the image; a position information acquisition unit that acquires position information including distance information and positional relationship information between the virtual object and a user in a real space; Equipped with The plurality of display devices include a first display device that is controlled to display a scene of the real space in which the virtual object is virtually arranged, the scene being viewed from a first viewpoint defined as a viewpoint of the user in the real space; and a second display device controlled to display an image of the virtual object; Including, the control unit dynamically changes the parameter for controlling the first display device such that an amount of change in a moving image of the virtual object displayed on the first display device increases as a distance between the virtual object and the user increases. Information processing device.

7. A control unit that dynamically changes each parameter related to the display of the virtual object, the control unit being assigned to each of a plurality of display devices for displaying an image related to the same virtual object, and that controls the display of the virtual object on each of the display devices in accordance with a representation method of the image; a position information acquisition unit that acquires position information including distance information and positional relationship information between the virtual object and a user in a real space; Equipped with The plurality of display devices include a first display device that is controlled to display a scene of the real space in which the virtual object is virtually arranged, the scene being viewed from a first viewpoint defined as a viewpoint of the user in the real space; and a second display device controlled to display an image of the virtual object; Including, the control unit dynamically changes the parameter for controlling the first display device so as to smooth a trajectory of the virtual object in a moving image displayed on the first display device as a distance between the virtual object and the user increases. Information processing device.

8. The control unit is controlling the second display device to display an image of the virtual object viewed from a second viewpoint in the real space, the second viewpoint being different from the first viewpoint; The information processing device according to any one of claims 1 to 7.

9. The information processing device according to claim 8 , wherein the second viewpoint is virtually positioned on the virtual object.

10. The first display device includes: an image of the virtual object is displayed superimposed on the image of the real space; projecting and displaying an image of the virtual object in the real space; Or, projecting and displaying an image of the virtual object on the user's retina; The information processing device according to any one of claims 1 to 9.

11. an information processing device controls display of the virtual object on each of a plurality of display devices, the display devices being assigned for displaying the image, according to a representation method of the image, and dynamically changes each parameter related to the display of the virtual object; An information processing method comprising: The plurality of display devices include a first display device that is controlled to display a scene of the real space in which the virtual object is virtually arranged, the scene being viewed from a first viewpoint defined as a user's viewpoint in the real space; and a second display device controlled to display an image of the virtual object; Including, The information processing device, acquiring position information including distance information and positional relationship information between the virtual object and the user in the real space; dynamically changing the parameter for controlling the first display device such that a display area of ​​the virtual object displayed on the first display device increases as a distance between the virtual object and the user increases; Further including: Information processing methods.

12. Computer, a control unit that dynamically changes each parameter related to the display of the virtual object, the control unit being assigned to each of a plurality of display devices that display images related to the same virtual object for the purpose of displaying the images, and that controls the display of the virtual object on each of the display devices in accordance with a representation method of the image; a position information acquisition unit that acquires position information including distance information and positional relationship information between the virtual object and a user in a real space; To function as A program, The plurality of display devices include a first display device that is controlled to display a scene of the real space in which the virtual object is virtually arranged, the scene being viewed from a first viewpoint defined as a viewpoint of the user in the real space; and a second display device controlled to display an image of the virtual object; Including, the control unit dynamically changes the parameter for controlling the first display device such that a display area of ​​the virtual object displayed on the first display device increases as a distance between the virtual object and the user increases. program.

Images