Information processing apparatus, information processing method, and program

Abstract

The present technology relates to an information processing apparatus, an information processing method, and a program capable of appropriately displaying an object in a virtual space. The present invention includes a setting unit configured to set an initial position at which an object is initially displayed on a display unit, the initial position being set based on presentation position information regarding a presentation position of the object in a virtual three-dimensional space in which a presentation is superimposed on a real world, size information regarding a size of the object, and a field-of-view region information regarding a field-of-view region of the display unit in which the object is displayed. The setting unit further sets the initial position in a depth direction with respect to a position indicating the presentation position information in a coordinate system using the real world as an index. The present technology can be applied to, for example, an information processing apparatus that provides augmented reality (AR).

Description

Technical Field

[0001] This technology relates to information processing apparatus, information processing method, and program, and to, for example, an information processing apparatus, information processing method, and program for performing a display preferred by the user when displaying a virtual object. Background Technology

[0002] A technology known as Augmented Reality (AR) involves overlaying additional information onto the real world and presenting it to the user. The information presented to the user in AR is also called annotation. Annotations are visualized using various forms of virtual objects, such as text, icons, and animations.

[0003] Patent document 1 proposes to appropriately control the display of virtual objects so as not to confuse users due to interference from the display of virtual objects.

[0004] Citation List

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2012-221250 Summary of the Invention

[0007] The problem to be solved by the present invention

[0008] When displaying an object, if the positional relationship between the user and the object is inappropriate, the object will not be displayed properly. For example, when an object is displayed in front of the user's eyes (close to the user), only a portion of the object can be viewed, which is not an appropriate display and may cause discomfort or unease to the user.

[0009] This technology was developed in view of such circumstances, and its purpose is to properly display objects, etc.

[0010] Solution to the problem

[0011] According to one aspect of the present technology, an information processing apparatus includes a setting unit that sets an initial position of an object initially displayed on a display unit based on presentation position information relating to the presentation position of an object superimposed and displayed in the real world to be presented in a virtual three-dimensional space, size information relating to the size of the object, and field of view information relating to the field of view of the display unit displaying the object. The setting unit sets the initial position in a real-world coordinate system to a side deeper in the depth direction than the position indicated by the presentation position information.

[0012] According to one aspect of the present technology, the information processing method includes: using an information processing device, setting an initial position for an object to be initially displayed on a display unit based on presentation position information related to the presentation position of an object superimposed and displayed in the real world to be presented in a virtual three-dimensional space, size information related to the size of the object, and field of view information related to the field of view of the display unit displaying the object; and setting the initial position in a coordinate system based on the real world to a side deeper in the depth direction than the position indicated by the presentation position information.

[0013] According to one aspect of the present technology, a program for a computer to perform a process comprising the following steps: setting an initial position for an object to be initially displayed on a display unit based on presentation position information relating to the presentation position of an object superimposed and displayed in the real world to be presented in a virtual three-dimensional space, size information relating to the size of the object, and field of view information relating to the field of view of the display unit displaying the object; and setting the initial position in a coordinate system based on the real world to a side deeper in the depth direction than the position indicated by the presentation position information.

[0014] In one aspect of the present technology, an information processing apparatus, information processing method, and program set the initial position of the object displayed on the display unit based on presentation position information relating to the presentation position of the object superimposed and displayed in the real world to be presented in a virtual three-dimensional space, size information relating to the size of the object, and field-of-view information relating to the field-of-view area of ​​the display unit displaying the object. In a coordinate system based on the real world, the initial position is set on a side deeper in the depth direction than the position indicated by the presentation position information.

[0015] Note that the information processing device can be a standalone device or an internal block included in a device.

[0016] In addition, the program can be provided by transmission via a transmission medium or by recording on a recording medium. Attached Figure Description

[0017] Figure 1 This is a diagram illustrating the configuration of an implementation of an information processing system according to the present disclosure.

[0018] Figure 2 This is a diagram showing an example of a display device.

[0019] Figure 3 This is a diagram illustrating the appearance configuration of an AR-HMD that applies the technology according to this disclosure.

[0020] Figure 4 This is a block diagram illustrating a configuration example of an AR-HMD as an information processing device.

[0021] Figure 5 is a block diagram showing an example of the functional configuration of an AR-HMD.

[0022] Figure 6 is a diagram for explaining the virtual screen size.

[0023] Figure 7 is a diagram showing an example of the user's action.

[0024] Figure 8 is a diagram showing an example of the screen to be displayed.

[0025] Figure 9 is a diagram showing an example of the screen to be displayed.

[0026] Figure 10 is a diagram showing an example of the screen to be displayed.

[0027] Figure 11 is a diagram for explaining the display settings.

[0028] Figure 12 is a diagram showing an example of the screen to be displayed.

[0029] Figure 13 is a diagram for explaining the display settings.

[0030] Figure 14 is a diagram for explaining the display settings.

[0031] Figure 15 is a diagram for explaining the display settings.

[0032] Figure 16 is a flowchart for explaining the operation of the information processing device.

[0033] Figure 17 is a diagram for explaining the recording medium. Detailed implementation

[0034] The following is a description of the mode for executing the present technology (this mode will be referred to as the implementation mode hereinafter).

[0035] <Display device for AR content>

[0036] Figure 1 is a diagram showing an example of the configuration of an information processing system according to an embodiment of the present technology.

[0037] Figure 1 The information processing system in

[0038] is configured by connecting the AR-HMD 1 and the information processing device 2 via a network 3 such as a local area network (LAN) or the Internet. Figure 1As shown, the AR-HMD 1 is a glasses-type wearable terminal including a transmissive display unit. The AR-HMD 1 displays a video image including various objects such as a person on the display unit under the control of the information processing device 2 executed via the network 3. The user sees the object superimposed on the scenery in front of the user.

[0039] The method of projecting the video image including the object may be a virtual image projection method or a retinal projection method that directly forms an image on the retina of the user's eye.

[0040] The information processing device 2 reproduces the AR content and sends the video data obtained by the reproduction to the AR-HMD 1 to display the video image of the AR content on the AR-HMD 1. The information processing device 2 includes, for example, a personal computer (PC).

[0041] Instead of the AR-HMD 1, the AR-HMD 1A as the video transmissive HMD shown in A of [[ID=१०]] Figure 2 or a mobile terminal such as the smart phone 1B shown in B of Figure 2 can be used as a display device for the AR content.

[0042] When the AR-HMD 1A is used as a display device, the video image of the AR content reproduced by the information processing device 2 is displayed as superimposed on the image of the scenery in front of the AR-HMD 1A captured by the imaging device provided in the AR-HMD 1A. A display is provided in front of the eyes of the user wearing the AR-HMD 1A, and the display shows the AR content to be superimposed on the image captured by the imaging device.

[0043] In addition, when using the smart phone 1B, the video image of the AR content reproduced by the information processing device 2 is displayed as superimposed on the image of the scenery in front of the smart phone 1B captured by the imaging device provided on the back surface of the smart phone 1B. A display for displaying various images is provided on the front surface of the smart phone 1B.

[0044] A projector that projects a video image onto the surface of an object existing in an actual scene can be used as a display device for the AR content. Various devices such as a tablet terminal and a television receiver can be used as a display device for the AR content.

[0045] The display device and the information processing device 2 can be connected by wire instead of being wirelessly connected via the network 3.

[0046] <Appearance configuration of the AR-HMD>

[0047] In the following description, as Figure 3 shown, an example of a wearable terminal having a glasses-type shape as an information processing device to which this technology is applied will be described.

[0048] Figure 3 The AR-HMD 1 shown has an overall eyeglass shape and includes a display unit 11 and a camera device 12.

[0049] The display unit 11 corresponds to the lens portion of the glasses and is configured as a transmissive display, for example. Therefore, the display unit 11 transparently overlays and displays annotations (virtual objects) on a real-world image (real object) that is directly visually recognized by the user.

[0050] The camera device 12 is positioned at the end of the display unit 11 corresponding to the left eye of the user wearing the AR-HMD 1, and captures images of the real space including the user's field of view. The camera device 12 includes solid-state imaging elements, such as a charge-coupled device (CCD) image sensor, a complementary metal-oxide-semiconductor (CMOS) image sensor, etc. Note that multiple sensors can be used for each type of sensor. That is, the camera device 12 can be configured as a stereoscopic camera device.

[0051] The image acquired by the camera device 12 can be displayed on the display unit 11, and annotations can be superimposed and displayed on the image.

[0052] In addition, although not shown, various sensors, buttons, speakers, etc. are housed or mounted in a housing corresponding to the eyeglass frame in AR-HMD 1.

[0053] Note that the shape of AR-HMD 1 is not limited to... Figure 3 The shape shown can be varied, such as a hat shape, a band that is fixed around the user's head, or a helmet shape that covers the user's entire head. That is, the technology according to this disclosure can be applied to all HMDs.

[0054] <Configuration example of AR-HMD as an information processing device>

[0055] Figure 4 This is a block diagram showing an example configuration of the AR-HMD 1 as an information processing device.

[0056] Figure 4 The AR-HMD 1 includes a central processing unit (CPU) 31, a memory 32, a sensor unit 33, an input unit 34, an output unit 35, and a communication unit 36. These units are interconnected via a bus 37.

[0057] The CPU 31 performs processing based on the programs, data, etc. stored in the memory 32 to implement various functions of the AR-HMD 1.

[0058] The memory 32 includes a storage medium such as a semiconductor memory or a hard disk, and stores programs and data to be processed by the CPU 31.

[0059] The sensor unit 33 includes various sensors, such as Figure 3 the imaging device 12, a microphone, a gyro sensor, and an acceleration sensor in

[0060] The input unit 34 includes buttons, keys, a touch panel, etc. The output unit 35 includes Figure 3 the display unit 11, a speaker, etc. in

[0061] <Functional configuration example of an AR-HMD>

[0062] Figure 5 is a block diagram showing a functional configuration example of an AR-HMD 1 to which the technology according to the present disclosure is applied.

[0063] Figure 3 The AR-HMD 1 in

[0064] includes a control unit 51, a sensor unit 52, a display unit 53, a speaker 54, a communication unit 55, an operation input unit 56, and a storage unit 57. Figure 4 [[ID=]26] The control unit 51 corresponds to

[0065] the CPU 31 in Figure 3 and executes processing for implementing various functions of the AR-HMD 1.

[0066] Specifically, the sensor unit 52 includes Figure 3 an outward imaging device 52a corresponding to the imaging device 12 in [[ID=]38]

[0067] an inward imaging device 52b that captures an image of the user wearing the AR-HMD 1, and a microphone 52c that collects sounds around the AR-HMD 1. Specifically, the inward imaging device 52b can be used to detect the user's line of sight.

[0068] In addition, sensor unit 52 includes a positioning unit 52g for locating a position via a satellite positioning system such as the Global Positioning System (GPS), and a biosensor 52h for acquiring biological information (heart rate, body temperature, brain waves, etc.) of the user wearing AR-HMD 1.

[0069] The various sensor information acquired by these sensors is used for processing performed by the control unit 51.

[0070] Display unit 53 corresponds to Figure 3 The display unit 11 in the middle displays annotations or images acquired by the external camera device 52a according to the control unit 51.

[0071] The speaker 54 serves as the sound source for the sound to be output to the user, and outputs sound according to the control unit 51.

[0072] Communication unit 55 corresponds to Figure 4 The communication unit 36 ​​in the middle, and performs other functions with other devices (e.g., information processing device 2). Figure 1 Various communications.

[0073] Operation input unit 56 corresponds to Figure 4 The input unit 34 in the AR-HMD 1 receives user input for operation.

[0074] The control unit 51 controls the output associated with the position of the real or virtual object based on the positional relationship between the display area of ​​the display unit 53 of the AR-HMD 1 and the real or virtual object arranged in three-dimensional space, as well as user action information representing user actions (hereinafter referred to as action information). The three-dimensional space here can be real space or virtual space.

[0075] Specifically, the control unit 51 executes a predetermined program to realize the sensor information acquisition unit 71, the head position and orientation detection unit 72, the object size setting unit 73, the layout parameter calculation unit 74, the layout determination unit 75, and the output control unit 76.

[0076] The sensor information acquisition unit 71 acquires sensor information from the sensor unit 52 and, based on the sensor information, acquires user motion information representing the actions of the user wearing the AR-HMD 1. The user motion information includes dynamic information related to the actions of the user's entire body or each part thereof, the movement of the user's gaze (change in gaze position), changes in the distance between the user and objects, etc. Furthermore, based on the sensor information acquired from the sensor unit 52, the sensor information acquisition unit 71 acquires user position and posture information (hereinafter also simply referred to as position and posture information) representing the position and posture of the user wearing the AR-HMD 1. The user position and posture information includes static information related to the user's posture and position, the distance between the user and objects, etc.

[0077] The head position and orientation detection unit 72 detects the position of the user's head and the orientation of the user's head (the direction the user is looking) from the information acquired by the sensor information acquisition unit 71.

[0078] The object size setting unit 73 sets the size, display position, etc. of the object to be displayed. As will be described later, when there is an object to be displayed, the size and display position of the object are set considering its positional relationship with the user, so as to display the object to the user without causing discomfort or discomfort.

[0079] The layout parameter calculation unit 74 calculates parameters representing the user's actions, position, state, etc., based on the sensor information acquired by the sensor information acquisition unit 71, specifically, the user action information, user position and posture information acquired from the sensor information, and based on the size and display position of the object set by the object size setting unit 73.

[0080] The layout determination unit 75 determines the output format of the output related to the object to be displayed in the display area of ​​the display unit 53 based on the parameters calculated by the layout parameter calculation unit 74.

[0081] The output control unit 76 uses the output format determined by the layout determination unit 75 to control the output related to objects in the display area of ​​the display unit 53.

[0082] Storage unit 57 stores various data required by control unit 51 to perform control. A portion of storage unit 57 and control unit 51 may be included in information processing device 2. Figure 1 )middle.

[0083] <Situations where objects cause insufficient perspective>

[0084] The AR-HMD 1 using this technology sets the size and display position of the objects to be displayed based on the situation. Here, the objects are, for example, text, icons, or animated content.

[0085] Reference Figure 6 Describes the relationship between the size of the object and the virtual image. In AR-HDM1, the maximum size of the object that can be displayed depends on the wearer of AR-HDM1 ( Figure 6 User 101 and object (in the middle) Figure 6 The size of the displayable objects varies depending on the distance between objects 121 in the virtual screen.

[0086] The virtual screen is a screen that is virtually displayed on the display unit 53. The size of the virtual screen can be calculated based on the following expression (1).

[0087] [Expression 1]

[0088]

[0089] In expression (1), d represents the distance between user 101 and the virtual screen. FOV(h) represents the horizontal or vertical angle of the FOV. FOV stands for Field of View and indicates the field of view. Typically, the FOV of AR-HDM1 has a horizontal angle of 40 degrees and a vertical angle of 20 degrees. In the following description, we will continue to describe the cases where these values ​​are set to the values ​​of FOV by way of example.

[0090] When using expression (1) to calculate the maximum content size (virtual screen size) that can be displayed at a position of 100cm (1m), the maximum content size is 73cm wide and 35cm high. Furthermore, when calculating the maximum content size (virtual screen size) that can be displayed at a position of 300cm (3m), the maximum content size is 160cm wide and 105cm high. Moreover, when calculating the maximum content size (virtual screen size) that can be displayed at a position of 500cm (5m), the maximum content size is 267cm wide and 176cm high.

[0091] Reference Figure 6 The virtual image displayed at a position of 100cm (1m) is defined as virtual image 122, and the virtual image displayed at a position of 500cm (5m) is defined as virtual image 123. Figure 6 In the diagram, object 121 is represented by a circle.

[0092] When object 121 is displayed on virtual screen 122 located 1 meter away, insufficient field of view occurs. Figure 6 In the shown state, only the central portion of object 121 is displayed. Note that in the following description, insufficient viewing angle means that the object is displayed in a state where it does not fall within the display unit 53. Furthermore, insufficient viewing angle means that the object is displayed in a state where it does not fall within the user's field of view.

[0093] When object 121 is displayed on a virtual screen 123 located at 5m, there is insufficient viewing angle in the vertical direction (the object does not fall within the field of view), but the object falls within the viewing angle (field of view) in the horizontal direction. Therefore, in Figure 6 In the state shown, the vertical direction of object 121 is partially invisible, but the entire horizontal direction is visible.

[0094] For example, in Figure 6 In the case of the displayed state on the virtual screen 122, the user may not understand what is being displayed and may be in an unwanted state such as panic or discomfort. Furthermore, there is a possibility that the user may not understand the situation, not know what to do, and may stop moving.

[0095] Existence Figure 6 Even in the case of the display state shown on the virtual screen 123, there are areas with insufficient viewing angle, but the entire object is visible in the horizontal direction (left and right direction in the figure), and therefore, the user can identify the size of object 121. Therefore, for example, the user can determine that if the user moves backward, the entire object 121 can be viewed, and can perform an action based on this determination.

[0096] Therefore, it is preferable that the following does not occur. Figure 6 The image shown depicts the display state of virtual image 122, where the viewing angle is insufficient at a distance d = 1m. Regarding this, please refer to... Figure 7 Add a description. Note that unless otherwise specified in the description below, coordinates, positions, etc., are coordinates, positions, etc., in the world coordinate system.

[0097] Note that the position of the displayed object can be defined based on its position in the real world or based on its position in a virtual 3D space. Here, we will continue the description by illustrating the case where the position is defined in a world coordinate system set up in a virtual 3D space.

[0098] For example, such as Figure 7 As shown in Figure A, object A is set to be displayed at position P1 defined in the world coordinate system. When user 151 is at position P2 and looking at object A, object A is displayed without causing insufficient viewpoint. For example, as... Figure 8 As shown in Figure A, object A (an object with a ghost-like shape) is displayed on display unit 53 without causing insufficient viewing angle.

[0099] like Figure 7As shown in Figure B, user 151 is located at position P3 near position P1 of displayed object A, and is facing direction D2. Direction D2 is the X-axis direction in the world coordinate system, and is a direction different from the Z-axis direction of displayed object A. Therefore, when user 151 is facing direction D2, even if object A is displayed at position P1, user 151 is not in a state where he / she can see object A.

[0100] When user 151 changes the orientation from facing direction D2 to facing direction D1, object A is displayed in front of the user's eyes. For example, as... Figure 8 As shown in Figure B, object A appears large on display unit 53, resulting in insufficient viewing angle. As described above, for example, if a portion of object A suddenly appears on the screen when user 151 looks back, the user may not know what happened and may panic or stop moving.

[0101] Object A can be, for example: Figure 9 The immovable object shown. For example, a sign is described as an example of an immovable object. Figure 9 A is when object A and user 151 are in such a state Figure 7 The screen is displayed on display unit 53 when the appropriate positional relationship of A is shown, and the entire signboard as the object is displayed.

[0102] Figure 9 B is when object A and user 151 are in such a state Figure 7 When the positional relationship of B is inappropriate, the image displayed on the display unit 53 is shown as part of a signboard representing the object. Because only a portion of the signboard is displayed, it is difficult for the user to identify what is shown.

[0103] Object A could be an object that communicates information to the user via text. For example, ... Figure 10 As shown in A, the message "Target Store" can be displayed as object A on display unit 53. Figure 10 A is when object A and user 151 are in such a state Figure 7 The screen is displayed on the display unit 53 when the appropriate position of A is displayed, and the entire message that the notification was executed is displayed as the object, so that the user can read the message.

[0104] Figure 10 B is when object A and user 151 are in such a state Figure 7 When the position of B is inappropriate, the screen displayed on display unit 53 is displayed as part of a notification (message) for the object. Since only a part of the message is displayed, the user cannot read the displayed message.

[0105] As mentioned above, when the positional relationship between an object and the user is inappropriate, there is a possibility of inconvenience such as the user not being able to recognize the object. Furthermore, depending on the user's actions, the object may be displayed suddenly or in a state where the user cannot recognize it. Therefore, as described below, the display of an object is controlled by the object's attributes.

[0106] Setting the display position and size of objects>

[0107] <First Display Settings>

[0108] The object A being described is, for example, Figure 8 The example shown is of a movable object, specifically object A. Object A is movable and will not cause discomfort to the user even when displaying actions. For example... Figure 11 The display shown is shown below.

[0109] Reference Figure 11 User 151 is located at position P3 at time t1. Furthermore, position P3 is near position P1 of displayed object A. In this state, as referenced... Figure 7 In the situation described in B, when object A is displayed at position P1, the screen presented to the user is as follows: Figure 8 As shown in B, the image has an insufficient field of view.

[0110] Although object A is present at time t1 and is intended to be shown to user 151, if displaying object A at position P1, where it is set to be displayed, would result in display inconvenience such as insufficient viewing angle, object A will be displayed at position P4 at time t2. Position P4 is a position farther from user 151 than position P1. Object A is displayed at such a position P4, which is far from user 151.

[0111] For example, such as Figure 8 As shown in Figure A, position P4 is the position where object A is displayed without causing insufficient viewing angle. In other words, this position is where object A is displayed as if it falls within the user's field of view. The object is temporarily moved backward to position P4, where the entire object A is displayed. At time t3 thereafter, object A is displayed at position P1. An animation is performed showing object A gradually approaching the user from position P4 to position P1.

[0112] Using such a display, for example, Figure 12 The screen shown is presented to user 151. At time t2, since object A is displayed at a distant location, the entire object A is displayed. The display of object A gradually increases from time t2 to time t3.

[0113] Object A at time t2', next to time t2, appears larger than object A at time t2. Furthermore, object A at time t2”, next to time t2', appears larger than object A at time t2'. Then, object A at time t3, next to time t2”, appears larger than object A at time t2”.

[0114] When viewed from user 151, the display shows object A gradually approaching the user.

[0115] The position P1 where the object is set to be displayed is appropriately described as the predetermined presentation position. Furthermore, the position where the object is first displayed is appropriately described as the initial position, as this will prevent insufficient viewing angle.

[0116] Positions such as the predetermined presentation position or initial position are the positions where virtual objects are displayed in three-dimensional space, and represent the positions of the user wearing the AR-HMD 1 when viewing the display unit 53. Figure 5 When the screen is displayed, the virtual object is perceived to exist in a three-dimensional position in the superimposed and displayed real space.

[0117] This technology is applied to Figures 1 to 3 In the case of AR-HMD 1 shown, AR-HMD 1 presents a screen displaying real or virtual objects arranged in three-dimensional space to the user. This position is the location of the real or virtual object on the screen presented to the user in three-dimensional space as perceived (visually recognized).

[0118] When an object has the property of being movable, AR-HMD 1 sets an initial position, temporarily displays the object at the initial position, and then displays an animation until the preset scheduled presentation position.

[0119] The initial position can be set, for example, by the following expression (2).

[0120] [Expression 2]

[0121]

[0122] Expression (2) is used to calculate the initial position when the user's position is (0, 0, 0). In expression (2), (x, y, z) represents the coordinates of the predetermined rendering position, and (X, Y, Z) represents the initial position. In expression (2), d is the depth information (Z coordinate), which is obtained through expression (3).

[0123] [Expression 3]

[0124]

[0125] Expression (3) is a modification of expression (1). That is, d is the distance between the user and the virtual screen. The initial position is set by expressions (2) and (3). Furthermore, in the following description, the initial position will continue to be calculated based on expressions (2) and (3).

[0126] Note that this description is based on the following assumption: Object A is displayed at position P4 (initial position), but it is sufficient that there is no viewfinder in at least one direction (horizontal or vertical) of object A. In other words, the initial position is set at a position where there is no viewfinder in at least one direction (horizontal or vertical) of object A.

[0127] In other words, adjustments are made by adjusting the size of the display unit 53 in the horizontal direction and the size of the object A in the horizontal direction, so that the object A falls within the field of view; or adjustments are made by adjusting the size of the display unit 53 in the vertical direction and the size of the object A in the vertical direction, so that the object A falls within the field of view.

[0128] The initial position of object A is set to fall within the field of view, but its depth may be limited. For example, if a movable range for object A is set, the initial position can be set to fall within that range.

[0129] Furthermore, the description will continue assuming a change in the coordinates of the predetermined presentation position P4 in the Z-axis direction (depth direction); however, for example, in order to perform an animation in which position P4 gradually approaches from the upward direction (or downward direction), the coordinates of the predetermined presentation position in the Z-axis direction and the Y-axis direction can be changed.

[0130] Note that the field of view is set to have a field of view angle that is less than or equal to the human's field of view angle (e.g., 180 degrees), and the field of view information used when performing the above processing is the information of the set field of view angle.

[0131] <Second Display Settings>

[0132] Reference Figure 13 Describe other display methods when object A is a movable object. Figure 13 The state shown at time t11 is related to Figure 11 The state at time t1 is similar to the state shown in the image, and when object A is displayed at position P1, the image presented to the user 151 is one with insufficient viewing angle. Therefore, in such a case, as referenced... Figure 11 The position P4, where there is no insufficient view, is set as the initial position, and object A is displayed.

[0133] but, Figure 13The state shown at time t11 is a state where an obstacle, such as a wall, exists at position P5, between positions P1 and P4. This obstacle can be a real-world obstacle or an obstacle set up in a virtual 3D space.

[0134] As mentioned above, if there is an obstacle such as a wall between the desired initial position and the user's current position P3, object A cannot be displayed with position P4 as the initial position.

[0135] In this case, the initial position is set to the location that is as far away from the user as possible (151). Figure 13 As shown at time t11, the position P5 of the wall is set as the initial position. Figure 13 As shown at time t12, position P6 is set as the initial position. Specifically, position P6 is closer to position P1 than position P5. Furthermore, position P6 is located in front of position P5 (on the user 151 side) in the depth direction, and is a predetermined distance away from position P5.

[0136] As described above, in the case of an area where objects cannot be displayed due to obstacles in the depth direction, the distance (depth) of the obstacles is detected, and the initial position is set in front of the depth.

[0137] At time t12, object A is displayed at position P6 (initial position). At time t13, object A is displayed at position P1 (predetermined presentation position). From position P6 to position P1, an animation is performed in which object A gradually approaches user 151.

[0138] If an object has the property of being movable, an initial position is set, the object is temporarily displayed at the initial position, and then animated until a predetermined display position is reached. Furthermore, if there is an obstacle, such as a wall, between the location where the initial position is desired and the predetermined display position, the position of the obstacle can be set as the initial position.

[0139] exist Figure 13 At time t12, object A is displayed at position P6, which is set as the initial position. However, position P6 is a position where the viewpoint may be insufficient. Therefore, in cases where the viewpoint may be insufficient even at the set initial position, object A can be temporarily displayed smaller, such as... Figure 14 The time interval t12'→t13 is shown in the figure. "Displayed smaller" means that object A is displayed smaller than the size indicated by the size information.

[0140] At time t12', object A is displayed at position P6 at a size that does not result in underexposure. In other words, object A appears small at position P6. Thereafter, until time t13, object A is displayed with an animation in which object A gradually enlarges and gets closer to the user.

[0141] This display method can also be applied to cases where the object is non-movable. It will describe the object as... Figure 9 The case shown is that of an immovable object. If object A is an immovable object, and if the operation is performed with a reference... Figure 11 and Figure 12 If the description of a movable object is displayed in the same way, then the display of a non-movable object will be executed, which will cause discomfort to the user. Therefore, in the case of an object that is non-movable, as shown in the reference... Figure 14 The object can be displayed at its original size (the size indicated by the size information) after being temporarily displayed smaller.

[0142] However, in the case of immovable objects, instead of animing the object to move like movable objects, the predetermined presentation position is set to the initial position, and the object is temporarily displayed smaller at that initial position. (Refer to...) Figure 15 Describe this display.

[0143] Figure 15 The state shown at time t21 is related to Figure 11 The state at time t1 is similar to the state shown in the image, and when object A is displayed at position P1, the screen presented to user 151 is an image with insufficient viewpoint. When the properties of an immovable object are assigned to object A, the predetermined presentation position is set to the initial position.

[0144] At time t22, object A is displayed smaller at position P1. At time t23, object A is displayed at position P1 at its original display size.

[0145] Between time t22 and time t23, an animation can be executed to gradually increase the size of object A. Alternatively, if object A is a non-movable object, the display can be switched from a small to a large size without executing the animation.

[0146] In the alternative setting, the object can be displayed brightly when the display is small. Alternatively, instead of small, the object can be displayed brightly and then darkened.

[0147] One objective is to make the user aware that the immovable object is located at position P1. When the user recognizes the presence of the immovable object, if the user intends to see the entire immovable object, the user will perform an action such as backing up. To enable the user to take such an autonomous action, the object is temporarily displayed smaller or brighter.

[0148] Note that if object A is a non-movable object, it's conceivable that displaying it as described above might cause more discomfort for the user. Even when the viewing angle is insufficient, it's possible to configure the display so that object A is shown at a predetermined position with the set size, without changing its position or size, even when object A is a non-movable object.

[0149] Furthermore, in the case of non-movable objects, sub-properties can be provided, which can be used to perform the display with a changed display as described above and to perform the display without a changed display as described above, and the display can be controlled through the properties.

[0150] <Third Display Settings>

[0151] Some immovable objects are configured to begin acting, such as speaking, when the user sees the immovable object (when it enters the user's field of view). In the case of such objects, for example, if object A speaks while in a state of limited visibility, such as... Figure 9 As shown in B, users are likely to be confused if they don't know where the voice is coming from.

[0152] Therefore, as mentioned above, object A can be temporarily displayed in a small size so that the user can recognize the presence of object A, and then sound can be output.

[0153] Alternatively, when an object is immovable and an action is performed while the object is within the user's field of view, a setting can be made so that no action is taken if the object is not visible from a certain angle. The condition for the action to occur can be set to be met when conditions 1 and 2 below are satisfied.

[0154] Condition 1

[0155] The object is within the user's field of view (the entire object is displayed on display unit 53).

[0156] Condition 2

[0157] The distance between the object and the user is within a certain range (less than or equal to the threshold).

[0158] Condition 1 is the condition for starting an action when the object is visible to the user and when there is no insufficient view. As mentioned above, this is a condition used to prevent the action from starting when there is insufficient view. Condition 1 can be determined, for example, by the following deterministic expression (4).

[0159] [Expression 4]

[0160]

[0161] [Expression 5]

[0162] pt=M World ×M Projection ×M Objpos (5)

[0163] In expression (4), pt.w, pt.x, and pt.y represent the values ​​representing the width, the x-axis direction, and the y-axis direction, respectively, calculated by expression (5). Furthermore, expression (4) uses the values ​​obtained by dividing the length in the x-axis direction and the length in the y-axis direction by the width w to obtain a normalized calculation expression.

[0164] Expression (4) is used to determine whether the length of the hypotenuse of the right triangle with the length of the object in the x-axis direction and the length in the y-axis direction as the two sides falls within 1. In other words, it is used to determine whether the diagonal of the quadrilateral surrounding the object falls within 1. When this determining expression is satisfied, the object is determined to be within the field of view.

[0165] In expression (5), M world M is a matrix in world coordinates. projection It is a projection-related matrix, and M objpos It is a matrix associated with the object and an expression used to project the object's coordinates onto the view frustum.

[0166] Condition 1 can be determined by whether the above expression is satisfied, or by other methods.

[0167] Using only condition 1, it's possible to determine that all objects located far from the user satisfy condition 1. Therefore, condition 2 determines whether the user exists near an object. In other words, condition 2 determines whether the user exists within the range where an action is set to be performed. Using condition 2, it's possible to prevent objects located far from the user from triggering actions.

[0168] Condition 2 can be determined, for example, by the following deterministic expression (6).

[0169] [Expression 6]

[0170] d=|M Objpos -M Camera |<Const.···(6)

[0171] In expression (6), M objpos Indicates the position of the object, and M camera The position of the camera device is indicated, which is the position of the user. Expression (6) is an expression that indicates the difference between the positions of the object and the user is the distance d and is used to determine whether the distance d is less than the preset threshold Const.

[0172] When it is determined according to condition 2 that the user has entered the area set for the object to activate the action, and when it is determined according to condition 1 that the object is displayed without causing insufficient viewing angle within the field of view, the action associated with the object is activated.

[0173] <Fourth Display Settings>

[0174] Reference Figure 15 The described display method can also be applied to references Figure 10 The object A being described is a notification such as text. In the case of text, as referenced... Figure 10 As stated in B, even if part of the message is displayed, it is difficult to convey the message to the user, and as referenced... Figure 10 As mentioned in A, it is conceivable that a message can be conveyed by making the entire message visible.

[0175] Therefore, as Figure 10 As shown in B, when only a portion of the message is displayed, the message (object A) is temporarily displayed smaller to allow the user to recognize its existence. At this point, whether the user can actually read the smaller version of the message is not the issue; it is sufficient for the user to recognize its presence.

[0176] Figure 15 The state shown at time t21 is related to Figure 11 The state at time t1 is similar to the state shown in the image, and when object A is displayed at position P1, the screen presented to user 151 is an image with insufficient viewing angle. When the attribute of text (message) is assigned to object A, the predetermined presentation position is set to the initial position. That is, in this case, the position P1, which is set as the predetermined presentation position, is set to the initial position.

[0177] At time t22, object A is displayed smaller at position P1. At time t23, object A is displayed at position P1 at its original display size.

[0178] Between time t22 and time t23, an animation can be executed to gradually increase the size of object A (message). Alternatively, the display can be switched from a small to a large size without executing the animation.

[0179] By causing the user to recognize that the message is located at position P1, if the user wants to view the entire message according to the user's intention, the user performs an action such as backing up. For example, at time t23, even in the state where a part of the message is displayed as shown in B of Figure 10 if the user recognizes that there is a message, the user can back up to form a state where the entire message is displayed as shown in A of Figure 10 and the user can read the message.

[0180] As described above, the display method is set by the attributes of the object. The operation of the AR-HMD 1 will be described, where the display method is set as described above.

[0181] <Operation of AR-HMD 1>

[0182] The operation of the AR-HMD 1 that performs such display will be described with reference to the flowchart of Figure 16

[0183] In step S11, environmental information is acquired. The environmental information is the environmental information around the user and is information about the real scene that the user views via the AR-HMD 1. For example, information such as surrounding depth information, object recognition, the shape of the recognized object, and the presence or absence of moving objects is acquired as environmental information.

[0184] In step S12, the position information of the user is acquired. Here, the body information of the user is acquired, for example, the position of the head and the line-of-sight direction. The processes in steps S11 and S12 are executed by acquiring and processing the sensor information from the sensor unit 52 by the sensor information acquisition unit 71.

[0185] In step S13, it is determined whether there is an object to be displayed. It is determined whether there is a predetermined object to be displayed within the user's field of view. In the case where it is determined in step S13 that there is no object to be displayed, the process returns to step S11, and the subsequent processes are repeated.

[0186] On the other hand, in the case where it is determined in step S13 that there is an object to be displayed, the process proceeds to step S14. In step S14, object size information is acquired. As the object size information, the presentation position information (information of the predetermined presentation position) of the object to be displayed and the information about the object size (size information) are acquired. As the size information, 3D model data can be used.

[0187] In step S15, the object size information obtained in step S14 is used to determine whether there is insufficient viewing angle (whether the object falls within the field of view). If it is determined in step S15 that there is no insufficient viewing angle, the process proceeds to step S20. In step S20, output processing is performed, in which case the object is displayed.

[0188] On the other hand, if insufficient viewing angle is determined in step S15, the process proceeds to step S16. In step S16, the attributes of the object to be displayed are referenced. The attributes are information used to determine which of the first to fourth display settings to use when insufficient viewing angle occurs.

[0189] As described above, attribute information can be information representing characteristics of the object to be displayed, such as a movable object, a non-movable object, or text. Furthermore, attribute information can be information that directly represents one of the first to fourth display settings, for example, information that is "1" when displaying the first display setting and "2" when displaying the second display setting.

[0190] Furthermore, as in the first and second display settings, the display used to set the initial position can be set to a position change attribute, and as in the fourth display setting, the display can be set to a size change attribute for changing the size displayed at a predetermined presentation position, and such information can be used as attribute information. In the following description, the use of position change attributes and size change attributes as attribute information will be further described by way of example.

[0191] Such attribute information can be obtained together with the object size information as accompanying object size information.

[0192] If, in step S16, it is determined that the attribute of the object to be displayed is a position change attribute, the process proceeds to step S17. In step S17, it is determined whether there is a display restriction in the depth direction. The depth direction is the Z-axis direction in the world coordinate system, and it is determined whether there is a display restriction in the Z-axis direction.

[0193] For example, as referenced Figure 13 In step S17, if there is an obstacle such as a wall along the depth direction and the object cannot be displayed at a position in front of such an obstacle, it is determined that there is a display restriction in the depth direction.

[0194] If it is determined in step S17 that there is no display limitation in the depth direction, the process proceeds to step S18. In step S18, an initial position is set to offset the position of the object in the Z direction. When the first display setting is executed, and as referenced... Figure 11In the case of the execution display, the processing of step S18 is performed.

[0195] Step S18 is to set a reference. Figure 11 The description refers to the processing of the initial position, referred to as position P4. Upon setting the initial position, the process proceeds to step S19. In step S19, a method for displaying the object at the initial position is generated and an animation is performed until a predetermined presentation position (in...) is reached. Figure 11 The display data (referred to as position P1, etc.) is generated, and in step S20, the display is performed based on the generated display data.

[0196] On the other hand, if it is determined in step S17 that there is a display limitation in the depth direction, the process proceeds to step S21. In step S21, it is determined whether the object has any action. If it is determined in step S21 that the object does not have any action, the process proceeds to step S22.

[0197] When the second display settings are executed, the processing in step S22 is performed. (This has already been referred to...) Figure 13 , Figure 14 and Figure 15 The second display setting is described. (See reference...) Figure 13 and Figure 14 The initial position can be set as the position in the depth direction in front of the obstacle (the side facing the user), preferably a position in the depth direction separated from the obstacle by a predetermined distance in front of the obstacle. The display can be performed by animation from the initial position to the predetermined presentation position (referred to as the 2-1 display setting). Alternatively, as shown in the reference... Figure 15 The predetermined display position can be set as the initial position, and the object can be displayed small at the initial position and then large (Display Settings 2-2).

[0198] In the case of objects with display limitations in the depth direction that do not cause action, the property information can be provided as whether the display is performed according to the display setting in 2-1 or the display setting in 2-2. Furthermore, in the case of movable objects, the display can be performed according to the display setting in 2-1, and in the case of immovable objects, the display can be performed according to the display setting in 2-2.

[0199] Furthermore, when there are display limitations in the depth direction, the predetermined rendering position can be set to the initial position. In such a configuration, the object is displayed at the predetermined rendering position.

[0200] In step S22, the initial position is set at the position in the depth direction in front of the obstacle or at a predetermined presentation position. In step S23, display data is generated for displaying the object at the initial position and performing animation display until the predetermined presentation position, and in step S20, display is performed based on the generated display data.

[0201] Note that in step S23, the animation display can be set to be executed based on the object. In step S23, the setting to not execute the animation display can be made, and data can be generated so that the object is displayed at the initial position and then switched to the display at the predetermined presentation position.

[0202] On the other hand, if it is determined in step S21 that the object has an action, the process proceeds to step S24. When the third display setting is executed, the process of step S24 is executed.

[0203] The third display setting is to activate the display of associated actions (e.g., vocalizations) when the user is near an object and the entire object (without any underexposure in at least one of the horizontal or vertical directions) is displayed within the user's field of view.

[0204] Therefore, since the initial position is the predetermined presentation position, the setting of the initial position is omitted (processing to set the predetermined presentation position as the initial position can be performed). In step S24, a trigger is set. This trigger is a trigger that is preset as a condition for activating an action, and as described above, in this case, it is the condition that the object is displayed within the user's field of view. In other words, when conditions 1 and 2 above are met, a trigger for activating an action is set.

[0205] Note that condition 2 is satisfied when it is determined in step S13 that an object to be displayed exists. That is, when the user enters the set area, it is determined in step S13 that an object to be displayed exists. Therefore, it is sufficient for the trigger set in step S24 to be a trigger related to condition 1.

[0206] Note that, as mentioned above, the third display setting can be combined with the second display setting. That is, it is also possible to set a standby state after an object is temporarily displayed smaller or its display position is changed, until a trigger is generated.

[0207] Furthermore, here, as an example, the case where the attribute information of the object associated with the action is a position change attribute is described; however, the attribute information can also be a size change attribute, and in the case of a size change attribute, it can be interchanged. Figure 16 The process shown is used to handle the task.

[0208] In step S25, a standby state is set until a trigger is generated. When a trigger is generated, the process proceeds to step S20, and the action associated with the object is activated. That is, multiple conditions for activating the action are preset, and the action is activated when at least some conditions are determined, in which case the condition of displaying the object in the field of view of the user who is set to trigger is met.

[0209] On the other hand, if it is determined in step S16 that the object to be displayed does not have a position change attribute, the process proceeds to step S26. In step S26, it is determined whether the attribute is a size change attribute. If it is determined in step S26 that the attribute is a size change attribute, the process proceeds to step S27.

[0210] When the fourth display setting is executed, and when as referenced Figure 15 During the execution of the display, the processing of step S27 is performed. That is, an object such as text is temporarily displayed at a predetermined display position, with a size smaller than the size to be displayed at the predetermined display position, and is then displayed as if enlarged to the predetermined display size.

[0211] In step S27, the initial output size is changed. In step S28, the object is displayed at the initial output size, display data is generated for performing animation display until a predetermined rendering size is reached, and in step S20, display based on the generated display data is performed.

[0212] On the other hand, if it is determined in step S26 that the attribute is not a size change attribute, the process proceeds to step S20. In this case, there is a possibility that the object may be displayed in a state of insufficient viewing angle, but for the object, settings are made so that even in such a state, the initial position change or size change display is not performed. Therefore, in step S20, the object is displayed at a predetermined presentation position and a predetermined presentation size.

[0213] As mentioned above, in situations where there is a possibility of insufficient viewing angle, the display can be controlled by changing the initial position or size. Performing such a display before insufficient viewing angle occurs allows the user to temporarily confirm the entire object, and prevents the user from becoming aware of the insufficient viewing angle even if it subsequently occurs.

[0214] When insufficient field of view occurs and the user becomes aware of it, there is a possibility that the immersion in the AR experience may be reduced; however, according to this technology, it is possible to prevent the immersion from being reduced and to make the user enjoy the AR experience more.

[0215] Note that in the above implementation, more detailed display settings can be performed by setting attribute information. Designers can set attribute information and display settings.

[0216] In the above embodiments, if the process involves setting an initial position, performing a display at the initial position, then performing a display at a predetermined presentation position, and then performing a display at the initial position again, the initial position can be set again and similar displays can be repeated. Alternatively, in the second and subsequent instances, the display can be performed at the predetermined presentation position without setting an initial position.

[0217] Note that if the display position is changed, for example, in the first display setting described above, there is a possibility that the association may be meaningless if the object is associated with a location or object in the real world. Therefore, an attribute that prevents the display position from being changed (adjusted) can be assigned to objects associated with locations or objects in the real world, thus preventing changes in the display position.

[0218] Furthermore, in the case of such objects, it is permissible to change the display size within the scope where the association is not broken, and such attributes can be provided.

[0219] <Recording Medium>

[0220] The aforementioned series of processing steps can be performed by hardware or by software. When the processing steps are performed by software, the program constituting the software is installed in the computer. Here, "computer" includes computers integrated with dedicated hardware, as well as computers capable of performing various functions by installing various programs, such as general-purpose personal computers.

[0221] Figure 17 This is a block diagram illustrating an example hardware configuration of a computer that executes the aforementioned series of processing steps via a program. In the computer, a central processing unit (CPU) 1001, a read-only memory (ROM) 1002, and a random access memory (RAM) 1003 are connected to each other via a bus 1004. Furthermore, an input / output interface 1005 is connected to the bus 1004. The input / output interface 1005 is connected to an input unit 1006, an output unit 1007, a storage unit 1008, a communication unit 1009, and a driver 1010.

[0222] Input unit 1006 includes a keyboard, mouse, microphone, etc. Output unit 1007 includes a display, speakers, etc. Storage unit 1008 includes a hard disk, non-volatile memory, etc. Communication unit 1009 includes a network interface, etc. Driver 1010 drives removable recording media 1011 such as disk, optical disk, magneto-optical disk, or semiconductor memory.

[0223] In a computer configured as described above, for example, CPU 1001 loads a program stored in storage unit 1008 into RAM 1003 via input / output interface 1005 and bus 1004 to execute the series of processing steps described above.

[0224] For example, the program executed by the computer (CPU 1001) can be provided by recording it in a removable recording medium 1011, such as a packaging medium. Alternatively, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0225] In a computer, a program can be installed into the storage unit 1008 via the input / output interface 1005 by installing the removable recording medium 1011 into the drive 1010. Alternatively, the program can be received by the communication unit 1009 via a wired or wireless transmission medium and installed into the storage unit 1008. Furthermore, the program can be pre-installed into the ROM 1002 and the storage unit 1008.

[0226] Note that a program executed by a computer can be a program that is processed sequentially in the order described herein, and can be a program that is processed in parallel or at necessary time intervals, such as when an execution call is made.

[0227] Furthermore, in this specification, "system" refers to an entire device comprising multiple devices.

[0228] Note that the beneficial effects described in the specification are merely examples, and the beneficial effects of this technology are not limited to those described in the specification, and may include other effects.

[0229] Note that the implementation of this technology is not limited to the above-described implementation, and various modifications are possible without departing from the scope of this technology.

[0230] Note that this technology can also be configured as follows. (1)

[0232] The setting unit sets the initial position of the object displayed on the display unit based on presentation position information related to the presentation position of the object superimposed and displayed in the real world to be presented in the virtual three-dimensional space, size information related to the size of the object, and field of view information related to the field of view of the display unit displaying the object.

[0233] The setting unit sets the initial position in the coordinate system based on the real world to a position deeper in the depth direction than the position represented by the presented position information. (2)

[0235] According to the information processing apparatus described in (1), wherein,

[0236] When the object is displayed at the presentation position with the size represented by the size information, the setting unit sets the initial position when it is determined that at least a portion of the object does not fall within the field of view. (3)

[0238] According to the information processing apparatus described in (1) or (2), wherein,

[0239] The setting unit sets the initial position to the position where the object falls within the field of view. (4)

[0241] The information processing apparatus according to any one of (1) to (3), wherein,

[0242] The setting unit sets the initial position to a position where at least one of the horizontal or vertical directions of the object falls within the field of view. (5)

[0244] The information processing apparatus according to any one of (1) to (4), wherein,

[0245] The information processing device displays an animation of moving the object from the initial position to the presentation position. (6)

[0247] The information processing apparatus according to any one of (1) to (5), wherein,

[0248] If an obstacle exists in the depth direction, the setting unit sets the initial position based on the position of the obstacle. (7)

[0250] According to the information processing apparatus described in (6), wherein,

[0251] If the obstacle exists in the depth direction, the setting unit sets the position in the depth direction before the position of the obstacle as the initial position. (8)

[0253] According to the information processing apparatus described in (6) or (7), wherein,

[0254] If the obstacle is present in the depth direction, the setting unit sets the presentation position to the initial position. (9)

[0256] The information processing apparatus according to any one of (6) to (8), wherein,

[0257] If the obstacle is present in the depth direction, the setting unit displays the object at the set initial position with a size smaller than the size indicated by the size information. (10)

[0259] The information processing apparatus according to any one of (1) to (9), wherein,

[0260] The setting unit sets the presentation position to the initial position and sets the object to have the following size: the object falls within the field of view area at this size. (11)

[0262] The information processing apparatus according to any one of (1) to (10), wherein,

[0263] When a predetermined action is associated with the object, and the setting unit determines that at least a portion of the conditions for activating the predetermined action are met when it is determined that the object falls within the field of view. (12)

[0265] The information processing apparatus according to any one of (1) to (11), wherein,

[0266] The information processing device sets the initial position based on the attributes of the object. (13)

[0268] According to the information processing apparatus described in (12), wherein,

[0269] When the attribute of the object is that of a movable object, the setting unit sets the initial position to the position where the object falls within the field of view. (14)

[0271] According to the information processing apparatus described in (12), wherein,

[0272] If the attribute of the object is an attribute of a non-movable object, the setting unit sets the presentation position to the initial position. (15)

[0274] According to the information processing apparatus described in (12), wherein,

[0275] When the object's attribute is a size change attribute, the setting unit changes the size of the displayed object. (16)

[0277] According to the information processing apparatus described in (14), wherein,

[0278] The object is text. (17)

[0280] An information processing method, comprising:

[0281] Perform the following operations using an information processing device:

[0282] Based on presentation position information related to the presentation position of the object superimposed and displayed in the real world to be presented in the virtual 3D space, size information related to the size of the object, and field-of-view information related to the field-of-view area of ​​the display unit displaying the object, the initial position of the object initially displayed on the display unit is set; and

[0283] In the coordinate system based on the real world, the initial position is set on a side deeper in the depth direction than the position represented by the presented position information. (18)

[0285] A program for causing a computer to perform a process including the following steps:

[0286] Based on presentation position information related to the presentation position of the object superimposed and displayed in the real world to be presented in the virtual 3D space, size information related to the size of the object, and field-of-view information related to the field-of-view area of ​​the display unit displaying the object, the initial position of the object initially displayed on the display unit is set; and

[0287] In the coordinate system based on the real world, the initial position is set on a side deeper in the depth direction than the position represented by the presented position information.

[0288] List of reference numerals

[0289] 1 AR-HMD

[0290] 2. Information processing device

[0291] 3. Network

[0292] 11 Display Units

[0293] 12 camera devices

[0294] 31 CPU

[0295] 32 Memory

[0296] 33 Sensor Units

[0297] 34 Input Units

[0298] 35 Output Units

[0299] 36 Communication Units

[0300] 37 bus

[0301] 51 Control Unit

[0302] 52 sensor units

[0303] 53 display units

[0304] 54 speakers

[0305] 55 Communication Units

[0306] 56 Operation Input Unit

[0307] 57 storage units

[0308] 71 Sensor Information Acquisition Unit

[0309] 72 Head position and orientation detection units

[0310] 73 Object Size Setting Unit

[0311] 74 Layout Parameter Calculation Unit

[0312] 75 Layout Determination Unit

[0313] 76 Output Control Unit

[0314] 101 users

[0315] 121 Object

[0316] 122, 123 (virtual screen)

Claims

1. An information processing apparatus, comprising: The setting unit sets the initial position of the object displayed on the display unit based on presentation position information related to the presentation position of the object superimposed and displayed in the real world to be presented in the virtual three-dimensional space, size information related to the size of the object, and field of view information related to the field of view of the display unit displaying the object. The setting unit, in the coordinate system based on the real world, sets the initial position to a side deeper in the depth direction than the position represented by the presented position information, and The setting unit is implemented via at least one processor.

2. The information processing apparatus according to claim 1, wherein, When the object is displayed at the presentation position with the size represented by the size information, the setting unit sets the initial position when it is determined that at least a portion of the object does not fall within the field of view.

3. The information processing apparatus according to claim 1, wherein, The setting unit sets the initial position to the position where the object falls within the field of view.

4. The information processing apparatus according to claim 1, wherein, The setting unit sets the initial position to a position where at least one of the horizontal or vertical directions of the object falls within the field of view.

5. The information processing apparatus according to claim 1, wherein, The information processing device displays an animation of moving the object from the initial position to the presentation position.

6. The information processing apparatus according to claim 1, wherein, If an obstacle exists in the depth direction, the setting unit sets the initial position based on the position of the obstacle.

7. The information processing apparatus according to claim 6, wherein, If the obstacle exists in the depth direction, the setting unit sets the position in the depth direction before the position of the obstacle as the initial position.

8. The information processing apparatus according to claim 6, wherein, If the obstacle is present in the depth direction, the setting unit sets the presentation position to the initial position.

9. The information processing apparatus according to claim 6, wherein, If the obstacle is present in the depth direction, the setting unit displays the object at the set initial position with a size smaller than the size indicated by the size information.

10. The information processing apparatus according to claim 1, wherein, The setting unit sets the presentation position to the initial position and sets the object to have the following size: the object falls within the field of view area at this size.

11. The information processing apparatus according to claim 1, wherein, When a predetermined action is associated with the object, and the setting unit determines that at least a portion of the conditions for activating the predetermined action are met when it is determined that the object falls within the field of view.

12. The information processing apparatus according to claim 1, wherein, The information processing device sets the initial position based on the attributes of the object.

13. The information processing apparatus according to claim 12, wherein, When the attribute of the object is that of a movable object, the setting unit sets the initial position to the position where the object falls within the field of view.

14. The information processing apparatus according to claim 12, wherein, If the attribute of the object is an attribute of a non-movable object, the setting unit sets the presentation position to the initial position.

15. The information processing apparatus according to claim 12, wherein, When the object's attribute is a size change attribute, the setting unit changes the size of the displayed object.

16. The information processing apparatus according to claim 15, wherein, The object is text.

17. An information processing method, comprising: Perform the following operations using an information processing device: The initial position of the object is set on the display unit based on presentation position information related to the presentation position of the object superimposed and displayed in the real world in the virtual three-dimensional space, size information related to the size of the object, and field of view information related to the field of view of the display unit displaying the object. as well as In the coordinate system based on the real world, the initial position is set on a side deeper in the depth direction than the position represented by the presented position information.

18. A computer program product comprising a computer program / instructions, wherein the computer program / instructions, when executed by a processor, implement an information processing method, the information processing method comprising: The initial position of the object is set on the display unit based on presentation position information related to the presentation position of the object superimposed and displayed in the real world in the virtual three-dimensional space, size information related to the size of the object, and field of view information related to the field of view of the display unit displaying the object. as well as In the coordinate system based on the real world, the initial position is set on a side deeper in the depth direction than the position represented by the presented position information.

19. A computer-readable storage medium having a computer-executable program stored thereon, which, when executed by a processor, causes the processor to perform the information processing method according to claim 17.

Images