Information processing device, method for controlling the same, and program

Abstract

To provide a technique capable of easily performing suitable display (display capable of suitably performing stereoscopic vision) even when other object existing behind an object is visualized.SOLUTION: An information processing device includes determination means for determining a third viewpoint and generation means. When a second object is present in front of a first object as viewed from the third viewpoint, the generation means draws an image of the first object in front of the second object to generate an image as seen from a first viewpoint corresponding to a left eye and an image as seen from a second viewpoint corresponding to a right eye.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, a control method for an information processing device, and a program, and more particularly to virtual reality, augmented reality, and mixed reality. [Background technology]

[0002] Virtual reality (VR), mixed reality (MR), and augmented reality (AR) are all referred to as XR (Cross Reality). A known way to experience XR is to use a head-mounted display (HMD).

[0003] In an HMD, stereoscopic vision is achieved by presenting (displaying) two images with parallax to each of the left and right eyes. To achieve stereoscopic vision, the user must achieve "fusion," which is the recognition and integration of an object in one image and an object in the other image as the same object, and the two images must be consistent. If the two images are not consistent, fusion will not occur, and double images or binocular rivalry will occur, causing discomfort.

[0004] In XR, it is sometimes necessary to visualize another object (including another object inside an object) that exists behind an object. For example, it is sometimes necessary to display an indicator showing the position of a diseased organ in a patient's body. It is also sometimes necessary to display the internal structure on the surface of a mechanical structure. If object B behind an object A is displayed while object A in the foreground is visible, the geometrical front-to-back relationship and the front-to-back relationship of how objects A and B appear contradict each other (are not consistent), and therefore fusion is not performed properly and stereoscopic vision is not performed properly. For example, the user feels a sense of discomfort that is a mixture of the sense that object B exists behind object A (sense according to the geometrical front-to-back relationship) and the sense that object B exists in front of object A (sense according to the front-to-back relationship of how objects appear).

[0005] Patent Document 1 discloses a technique for detecting an inconsistency in the positional relationship between a real object and a virtual object. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2018-106262 A Summary of the Invention [Problem to be solved by the invention]

[0007] In the case of VR, when visualizing another object behind an object, if the object in front is displayed semi-transparently, the user can have a comfortable stereoscopic view without feeling uncomfortable. In the case of AR or MR, the object in front may be a real object. In that case, in order to display the real object semi-transparently, it is necessary to estimate the image behind the real object using a technology called dimming reality, and it is not easy to perform a suitable display (a display that allows suitable stereoscopic vision). Another object that exists inside the object can also be visualized by image processing such as drilling a hole in the surface of the object. However, when comparing the object in front with the object inside (behind), image processing that significantly changes the appearance of the object is not desirable.

[0008] The present invention provides a technique that can easily perform a suitable display (a display that can perform a suitable stereoscopic view) even when another object that exists behind an object is visualized. The purpose of this document is to: [Means for solving the problem]

[0009] A first aspect of the present invention is an information processing device characterized by having a determination means for determining a third viewpoint, and a generation means for, when a second object is present in front of a first object when viewed from the third viewpoint, drawing an image of the first object in front of the second object, thereby generating an image as it appears from the first viewpoint corresponding to the left eye and an image as it appears from the second viewpoint corresponding to the right eye.

[0010] A second aspect of the present invention is a control method for an information processing device, characterized by comprising the steps of: determining a third viewpoint; and, when a second object is present in front of a first object as viewed from the third viewpoint, drawing an image of the first object in front of the second object to generate an image as it appears from the first viewpoint corresponding to the left eye and an image as it appears from the second viewpoint corresponding to the right eye.

[0011] A third aspect of the present invention is a program for causing a computer to function as each of the means of the information processing device described above. Effect of the Invention

[0012] According to the present invention, even when an object existing behind an object is visualized, a suitable display (a display that allows suitable stereoscopic viewing) can be easily achieved. [Brief description of the drawings]

[0013] [Figure 1] FIG. 2 is a block diagram showing an example of a system configuration. [Diagram 2] FIG. 2 is a block diagram showing an example of a hardware configuration of an information processing device. [Diagram 3] 11 is a flowchart showing an example of the operation of the system. [Figure 4] FIG. 2 is a schematic diagram showing an example of the positional relationship between each viewpoint and each object. [Diagram 5] FIG. 4 is a schematic diagram for explaining an example of the process of step S3030 in FIG. [Figure 6]FIG. 4 is a schematic diagram for explaining an example of the process of step S3060 in FIG. [Figure 7] FIG. 13 is a block diagram showing a modified example of the system configuration. [Figure 8] 10 is a flowchart showing a modified example of the operation of the system. [Figure 9] 11A to 11C are schematic diagrams showing modified examples of the positional relationship between each viewpoint and each object. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an embodiment of the present invention will be described. In this embodiment, an example in which the present invention is applied to a video see-through type MR that displays a composite image in which a CG object (virtual object) is superimposed on an image of a real space captured by an imaging device such as a digital camera will be described, but the present invention is not limited to this. For example, the present invention may be applied to an optical see-through type AR that superimposes (displays) a virtual object on a real space that is not an image. The present invention may be applied to a VR that displays a composite image in which a virtual object is superimposed on an image of a virtual space.

[0015] Fig. 1 is a block diagram showing an example of the configuration of a system according to this embodiment. The system in Fig. 1 is configured with an information processing device 101 and a head mounted display (HMD) 102. The information processing device 101 may be a stationary computer such as a personal computer or a workstation, or may be a mobile terminal such as a smartphone or a wearable device. The information processing device 101 may be a part of the HMD 102.

[0016] The HMD 102 will now be described. The stereo imaging unit 103 captures two images with parallax as images of the real space. The stereo display unit 104 is a unit that displays the images of the real space in the information processing device 101 (synthesizing unit). The two composite images generated by the HMD 102 and 113) are presented (displayed) to the two left and right eyes of the user wearing the HMD 102 on the head. The method of presenting (displaying) the images is not particularly limited. For example, the images may be displayed on a display panel such as a liquid crystal panel or an organic EL panel provided in a position facing the user's eyes, or the images may be projected directly onto the user's retina using a laser.

[0017] The information processing device 101 will be described. FIG. 2 is a block diagram showing an example of a hardware configuration of the information processing device 101. The CPU 201 is a control unit that controls the entire information processing device 101. Each functional unit of the information processing device 101 shown in FIG. 1 is realized by, for example, the CPU 201 (or a GPU not shown). The ROM 202 is a non-volatile memory that non-temporarily stores various data (information) such as image data, audio data, other data, and various programs for the CPU 201 to operate. The RAM 203 is a volatile memory that temporarily stores various data. The CPU 201 executes each function of the information processing device 101 using the RAM 203 as a work memory according to, for example, a program stored in the ROM 202. The input I / F 204 is an interface for inputting various data (information) such as image data, audio data, operation signals, and other data to the information processing device 101. The output I / F 205 is an interface for outputting various data (information) such as image data, audio data, control signals, and other data from the information processing device 101. These pieces of hardware are connected to a bus 206 and can transmit and receive data via the bus 206. The information processing device 101 may be divided into multiple devices. At least one of the multiple functions of the information processing device 101 may be executed on the cloud or in the HMD 102.

[0018] The first / second viewpoint estimation unit 105 in FIG. 1 estimates a first viewpoint (position and orientation of the first viewpoint) corresponding to the left eye of the user and a second viewpoint (position and orientation of the second viewpoint) corresponding to the right eye of the user. The estimation method is not particularly limited, and for example, the first viewpoint and the second viewpoint may be estimated by estimating the position and orientation of the HMD 102. An inside-out technique such as Simultaneous Localization and Mapping (SLAM) may be used. In this case, for example, the stereo imaging unit 103 or another camera provided in the HMD 102 is used. An outside-in technique may be used. In this case, for example, motion capture is performed using a camera outside the HMD 102.

[0019] The third viewpoint determination unit 106 determines the third viewpoint (position and orientation of the third viewpoint). In this embodiment, the third viewpoint is used for rendering the first object selected by the first object selection unit 108 among the objects (back objects) hidden by the objects (second objects) in front. In rendering the first object, the first object is drawn in a view from the third viewpoint. The third viewpoint determination unit 106 determines the third viewpoint based on at least one of the first viewpoint and the second viewpoint (the first viewpoint, the second viewpoint, or both of them) estimated by the first / second viewpoint estimation unit 105, for example. As a result, the third viewpoint close to both the first viewpoint and the second viewpoint is determined, so that the first object can be drawn in a view close to both the view from the first viewpoint and the view from the second viewpoint.

[0020] The CG model data storage unit 107 stores various data such as shape data, material data, and scene graph data of the CG (object) to be presented to the user.

[0021] The first object selection unit 108 selects a first object from one or more objects. The selection method is not particularly limited. For example, the first object selection unit 108 may select an object designated by a user as the first object. The user may specify (select) an object using a tree UI that represents the structure of a scene graph. The first object selection unit 108 may automatically select an object that satisfies a predetermined condition as the first object. The predetermined condition may be a condition related to the attribute of the object that is set in advance, or a condition related to the tag information of the object that is set in advance.

[0022] The second object generating unit 109 generates a second object (shape information of the second object). In the case of AR or MR, the second object may be a real object. In that case, the second object generating unit 109 generates a virtual object that reproduces the real object by 3D reconstruction as the second object. The method of 3D reconstruction is not particularly limited. For example, the second object may be generated using two images (stereo images) obtained by the stereo imaging unit 103, or the second object may be generated using a depth sensor. The second object may be generated from a plurality of images corresponding to a plurality of viewpoints using Structure from Motion. In the case of VR, the second object may be generated from a CG model.

[0023] The first object rendering unit 110 renders an image of the first object on a rendering target as seen from a third viewpoint.

[0024] A third object generating unit 111 generates a third object in front of the second object when viewed from a third viewpoint.

[0025] The CG drawing unit 112 pastes the image of the first object drawn by the first object drawing unit 110 onto the surface of the third object. This pasting may be interpreted as drawing the first object on the surface of the third object. Then, the CG drawing unit 112 generates a CG as seen from the first viewpoint and a CG as seen from the second viewpoint as CG (image) including the third object on which the image of the first object is drawn.

[0026] The synthesis unit 113 synthesizes the CG generated by the CG rendering unit 112 with the real image (image of real space) captured by the stereo imaging unit 103. The synthesis unit 113 synthesizes the CG corresponding to the first viewpoint with the real image corresponding to the first viewpoint to generate a synthetic image corresponding to the first viewpoint. The synthesis unit 113 also synthesizes the CG corresponding to the second viewpoint with the real image corresponding to the second viewpoint to generate a synthetic image corresponding to the second viewpoint. The synthetic image corresponding to the first viewpoint is presented to the left eye of the user, and the synthetic image corresponding to the second viewpoint is presented to the right eye of the user. In the case of VR, the synthesis unit 113 is omitted, and the CG generated by the CG rendering unit 112 (synthetic image in which a virtual object is superimposed on an image of a virtual space) is presented to the user.

[0027] Fig. 3 is a flowchart showing an example of the operation of the system according to this embodiment. Fig. 4 is a schematic plan view showing an example of the positional relationship between each viewpoint and each object.

[0028] In step S3010, the third viewpoint determination unit 106 determines the third viewpoint 404 based on at least one of the first viewpoint 402 and the second viewpoint 403. For example, the third viewpoint determination unit 106 determines the midpoint between the first viewpoint 402 and the second viewpoint 403 (the position of the midpoint between the first viewpoint 402 and the second viewpoint 403 and the average (intermediate) posture) as the third viewpoint 404 (the position and posture of the third viewpoint 404). The third viewpoint determination unit 106 may determine the first viewpoint 402 or the second viewpoint 403 (one of the first viewpoint 402 and the second viewpoint 403) as the third viewpoint 404. In that case, the third viewpoint determination unit 106 determines the third viewpoint 404 based on at least one of the first viewpoint 402 and the second viewpoint 403. The information on the first efficacy may be acquired, and one of the first and second viewpoints 402 and 403, which corresponds to the efficacy, may be determined as the third viewpoint 404. The information on the efficacy may be registered in the information processing device 101 by the user 401, for example. The user 401 may register the information on the efficacy voluntarily, or may register the information in response to a request from the information processing device 101. Note that the method of determining the third viewpoint 404 is not limited to the above method. For example, the third viewpoint determination unit 106 may determine the third viewpoint 404 by adding a predetermined offset to the first viewpoint 402 or the second viewpoint 403.

[0029] In step S3020, the first object drawing unit 110 draws an image of the first object 406 on the rendering target as viewed from the third viewpoint 404. The projection matrix for drawing the CG as viewed from the first viewpoint 402 is the first projection matrix, and the projection matrix for drawing the CG as viewed from the second viewpoint 403 is the second projection matrix. And the projection matrix for drawing the CG as viewed from the third viewpoint 404 is the third projection matrix. For example, the first projection matrix is ​​determined based on the field of view corresponding to the first viewpoint 402. The second projection matrix is ​​determined based on the field of view corresponding to the second viewpoint 403. The third projection matrix is ​​determined based on the field of view corresponding to the third viewpoint 404. The first object drawing unit 110 uses the third projection matrix to draw the image of the first object 406 on the rendering target as if the second object 405 does not exist.

[0030] It is assumed that the third viewpoint 404 is the midpoint between the first viewpoint 402 and the second viewpoint 403. It is assumed that the rendering frustum for drawing the CG as seen from the first viewpoint 402 and the rendering frustum for drawing the CG as seen from the second viewpoint 403 are symmetrical. In this case, since the first projection matrix and the second projection matrix are usually the same, a projection matrix identical to the first projection matrix and the second projection matrix is ​​used as the third projection matrix. When each of the above two frustums is asymmetrical, for example, an average (median) projection matrix of the first projection matrix and the second projection matrix is ​​used as the third projection matrix. When one of the first viewpoint 402 and the second viewpoint 403 is selected as the third viewpoint 404, for example, a projection matrix corresponding to the selected viewpoint is used as the third projection matrix.

[0031] In step S3030, the first object drawing unit 110 detects an overlapping area in which an object (second object 405) exists in front of the first object 406 as viewed from the third viewpoint 404. This process is performed, for example, based on the first object 406 drawn in step S3020 and the second object 405 generated by the second object generating unit 109. FIG. 5 is a schematic diagram for explaining a processing example of step S3030. The CG scene 501 corresponds to the view from the third viewpoint 404 (user 401), and the first object 406 and the second object 405 are arranged in the CG scene 501. The first object drawing unit 110 generates a first depth image 502 of only the second object 405, excluding the first object 406 from the CG scene 501. It is assumed that the pixel value of the depth image increases as the distance from the viewpoint increases. Similarly, the first object drawing unit 110 generates a second depth image 503 of only the first object 406, excluding the second object 405 from the CG scene 501. The first object drawing unit 110 subtracts the first depth image 502 (each pixel value) from the second depth image 503 (each pixel value) to generate a difference image 504. Then, the first object drawing unit 110 detects a positive region 505 in which the pixel value of the difference image 504 (the value obtained by subtracting the pixel value of the first depth image 502 from the pixel value of the second depth image 503) is greater than 0, as an overlapping region.

[0032] In step S3040, the first object drawing unit 110 determines an α value (blending ratio) for each position of the image of the first object 406 drawn on the rendering target based on the overlapping region 505 detected in step S3030. The α value is a blending ratio when drawing (pasting) the image of the first object 406 on the surface of the third object 407, and the larger the α value is, the lower the transparency of the image of the first object 406 is drawn on the surface of the third object 407. For example, when the α value is 0, the transparency is 100%, and the image of the first object 406 is not drawn on the surface of the third object 407. When the α value is 0.5, the transparency is 50%, and a semi-transparent image of the first object 406 is drawn on the surface of the third object 407. When the α value is 1, the transparency is 0%, and an opaque image of the first object 406 is drawn on the surface of the third object 407. For example, first object drawing unit 110 determines the α value of overlapping area 505 to be 1, and determines the α value of areas other than overlapping area 505 to be 0. This makes it possible to draw an image of only the portion of first object 406 that corresponds to overlapping area 505 on the surface of third object 407.

[0033] In step S3050, the third object generating unit 111 generates (defines) a third object 407 in front of the second object 405 as viewed from the third viewpoint 404. For example, the transparent third object 407 is generated based on the second object 405 (geometry and transformation; position, orientation, and shape) generated by the second object generating unit 109. Note that an image of the first object 406 may be drawn on the surface of the second object 405 without generating the third object 407. Consider a case where the surface of the second object 405 has a complex shape (e.g., unevenness). In that case, if the image of the first object 406 is drawn on the surface of the second object 405, the appearance of the image of the first object 406 may be significantly different between the first viewpoint 402 or the second viewpoint 403 and the third viewpoint 404. Therefore, it is preferable to generate a third object 407 having a surface that is a simplified version of the surface of the second object 405 (for example, a surface without any bumps or recesses).

[0034] In step S3060, the CG drawing unit 112 calculates the coordinates of each vertex of the third object 407 generated by the third object generation unit 111. In this embodiment, an image of the first object 406 is drawn on the surface of the third object 407 so as to project the first object 406 onto the surface of the third object 407. For this purpose, the CG drawing unit 112 calculates two-dimensional coordinates of each vertex of the third object 407 corresponding to the appearance from the third viewpoint 404 using a third projection matrix. FIG. 6 is a schematic diagram for explaining a processing example of step S3060. Coordinates 602 and 603 of each vertex 601 of the third object 407 are calculated. The coordinate 602 is a vertical coordinate, and the coordinate 603 is a horizontal coordinate. The coordinates 602 and 603 are normalized by the size of the field of view.

[0035] In step S3070, CG rendering unit 112 draws an image of first object 406 on the surface of third object 407 so as to project first object 406 onto the surface of third object 407 based on the coordinates calculated in step S3060. Then, CG rendering unit 112 generates a CG of how it is seen from first viewpoint 402 and a CG of how it is seen from second viewpoint 403 as CG (image) including third object 407 on which the image of first object 406 is drawn.

[0036] Thereafter, the synthesis unit 113 synthesizes the CG corresponding to the first viewpoint 402 with the real image corresponding to the first viewpoint 402 to generate a synthetic image corresponding to the first viewpoint 402. Also, the synthesis unit 113 synthesizes the CG corresponding to the second viewpoint 403 with the real image corresponding to the second viewpoint 403 to generate a synthetic image corresponding to the first viewpoint 402. A composite image corresponding to the second viewpoint 403 is generated by synthesizing CG corresponding to the first viewpoint 402. The composite image corresponding to the first viewpoint 402 is presented to the left eye of the user 401, and the composite image corresponding to the second viewpoint 403 is presented to the right eye of the user 401.

[0037] As described above, according to this embodiment, a third viewpoint is determined, and when a second object exists in front of a first object as viewed from the third viewpoint, an image of the first object is drawn in front of the second object. Then, an image as seen from the first viewpoint corresponding to the left eye and an image as seen from the second viewpoint corresponding to the right eye are generated. By doing so, even when another object existing behind the object is visualized, a suitable display (a display that allows suitable stereoscopic viewing) can be easily performed.

[0038] For example, when visualizing an object that exists behind another object, the inconsistency in the anteroposterior relationship of the objects can be suppressed. This allows for favorable image fusion and favorable stereoscopic vision. In addition, since complex technology such as disappearance reality is not required, favorable display can be easily achieved.

[0039] The above-described embodiment (including the modified examples) is merely an example, and the present invention also includes configurations obtained by appropriately modifying or changing the configurations of the above-described embodiment within the scope of the gist of the present invention. The present invention also includes configurations obtained by appropriately combining the configurations of the above-described embodiment.

[0040] (Modification) The following describes modified examples of the above-described embodiment. Note that, in the following, the same points as the above-described embodiment (for example, the same configuration and processing as the above-described embodiment) will be omitted, and the differences from the above-described embodiment will be described.

[0041] The longer the distance (depth) from the third viewpoint, the smaller the difference between the appearance from the first or second viewpoint and the appearance from the third viewpoint. Therefore, in the above-mentioned embodiment, an example of generating a third object based on the position, orientation, and shape of the second object has been described. However, when the second object is a real object, a three-dimensional reconstruction of the second object is required, which increases the processing load. In addition, when the above-mentioned difference in appearance is not a big problem, or when it is assumed that the second object does not exist within a range of a predetermined distance from the third viewpoint, the third object may be generated (defined) without generating the second object. In this modified example, an example of generating a third object without generating a second object will be described.

[0042] 7 is a block diagram showing an example of the configuration of a system according to this modification. The information processing device 101 does not include the second object generating unit 109. A third object generating unit 701 generates a third object in front of a second object (real object) as viewed from a third viewpoint.

[0043] Fig. 8 is a flowchart showing an example of the operation of the system according to this modified example. Fig. 9 is a schematic plan view showing an example of the positional relationship between each viewpoint and each object.

[0044] In step S8010, similar to step S3010 in FIG. 3, the third viewpoint determination unit 106 determines the third viewpoint 404. In step S8020, similar to step S3020, the first object drawing unit 110 draws an image of the first object 406 on the rendering target. In step S8030, similar to step S3040, the first object drawing unit 110 determines an α value for each position of the image of the first object 406 drawn on the rendering target. For example, the first object drawing unit 110 determines an α value for each position of the image of the first object 406 drawn on the rendering target. The alpha value of the entire area of ​​the first object 406 is determined to be 1, and the alpha value of the area outside the first object 406 is determined to be 0.

[0045] In step S8040, the third object generation unit 701 generates a third object 901 based on the third viewpoint 404. For example, a plane perpendicular to the line of sight according to the attitude of the third viewpoint 404 at a position a predetermined distance away from the third viewpoint 404 is determined as the third object 901. Note that the method of generating the third object 901 is not limited to the above method, and for example, the third object 901 may be generated so that the relative position and attitude of the third object 901 with respect to the third viewpoint 404 are predetermined position and attitude.

[0046] In step S8050, similar to step S3060, CG rendering unit 112 calculates the coordinates of each vertex of third object 901 generated by third object generation unit 701. In step S8060, similar to step S3070, CG rendering unit 112 draws an image of first object 406 on the surface of third object 901. Then, CG rendering unit 112 generates a CG of the appearance from first viewpoint 402 and a CG of the appearance from second viewpoint 403.

[0047] Thereafter, similarly to the above-described embodiment, the synthesis unit 113 generates a synthetic image corresponding to the first viewpoint 402 and a synthetic image corresponding to the second viewpoint 403. The synthetic image corresponding to the first viewpoint 402 is presented to the left eye of the user 401, and the synthetic image corresponding to the second viewpoint 403 is presented to the right eye of the user 401.

[0048] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0049] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) A determination means for determining the third viewpoint; a generating means for generating an image as seen from the first viewpoint corresponding to the left eye and an image as seen from the second viewpoint corresponding to the right eye by drawing an image of the first object in front of the second object when the second object is present in front of the first object as seen from the third viewpoint; 13. An information processing device comprising: (Configuration 2) The generating means draws an image of the first object on a surface of the second object. 2. The information processing device according to configuration 1. (Configuration 3) further comprising an object generating means for generating a third object in front of the second object, The generating means draws an image of the first object on a surface of the third object. 2. The information processing device according to configuration 1. (Configuration 4) The object generating means generates the third object based on the position, orientation, and shape of the second object. 4. The information processing device according to configuration 3. (Configuration 5) The object generating means generates the third object based on the third viewpoint. 4. The information processing device according to configuration 3. (Configuration 6) The determining means determines the third viewpoint based on at least one of the first viewpoint and the second viewpoint. 6. The information processing device according to any one of configurations 1 to 5. (Configuration 7) The determining means determines a midpoint between the first viewpoint and the second viewpoint as the third viewpoint. 7. The information processing device according to any one of configurations 1 to 6. (Configuration 8) The determining means determines the first viewpoint or the second viewpoint as the third viewpoint. 7. The information processing device according to any one of configurations 1 to 6. (Configuration 9) The second object is a real object. 9. The information processing device according to any one of configurations 1 to 8. (Configuration 10) The second object is a virtual object that is a reproduction of a real object by 3D reconstruction. 10. The information processing device according to any one of configurations 1 to 9. (Configuration 11) a selection means for selecting an object designated by a user from one or more objects as the first object; Further having 11. The information processing device according to any one of configurations 1 to 10. (Configuration 12) a selection means for automatically selecting, as the first object, an object that satisfies a predetermined condition from one or more objects; Further having 11. The information processing device according to any one of configurations 1 to 10. (Configuration 13) The image of the first object is an image obtained by projecting the first object in front of the second object. 13. The information processing device according to any one of configurations 1 to 12. (method) determining a third perspective; a step of drawing an image of the first object in front of the second object when viewed from the third viewpoint, thereby generating an image as seen from the first viewpoint corresponding to the left eye and an image as seen from the second viewpoint corresponding to the right eye; 13. A method for controlling an information processing apparatus comprising the steps of: (program) A program for causing a computer to function as each of the means of the information processing device according to any one of configurations 1 to 13. [Explanation of symbols]

[0050] 101: Information processing device 106: Third viewpoint determination unit 112: CG drawing unit

Claims

1. A determination means for determining a third viewpoint; a generating means for generating an image as seen from the first viewpoint corresponding to the left eye and an image as seen from the second viewpoint corresponding to the right eye by drawing an image of the first object in front of the second object when the second object is present in front of the first object as seen from the third viewpoint; 13. An information processing device comprising:

2. The generating means draws an image of the first object on a surface of the second object.

2. The information processing apparatus according to claim 1,

3. further comprising an object generating means for generating a third object in front of the second object; The generating means draws an image of the first object on a surface of the third object.

2. The information processing apparatus according to claim 1,

4. The object generating means generates the third object based on the position, orientation, and shape of the second object.

4. The information processing apparatus according to claim 3.

5. The object generating means generates the third object based on the third viewpoint.

4. The information processing apparatus according to claim 3.

6. The determining means determines the third viewpoint based on at least one of the first viewpoint and the second viewpoint.

2. The information processing apparatus according to claim 1,

7. The determining means determines a midpoint between the first viewpoint and the second viewpoint as the third viewpoint.

2. The information processing apparatus according to claim 1,

8. The determining means determines the first viewpoint or the second viewpoint as the third viewpoint.

2. The information processing apparatus according to claim 1,

9. The second object is a real object.

2. The information processing apparatus according to claim 1,

10. The second object is a virtual object that is a reproduction of a real object by 3D reconstruction.

2. The information processing apparatus according to claim 1,

11. a selection means for selecting an object designated by a user from one or more objects as the first object; Further having 2. The information processing apparatus according to claim 1,

12. A selection means for automatically selecting an object that satisfies a predetermined condition as the first object from one or more objects. Further having 2. The information processing apparatus according to claim 1,

13. The image of the first object is an image obtained by projecting the first object in front of the second object.

2. The information processing apparatus according to claim 1,

14. determining a third perspective; a step of drawing an image of the first object in front of the second object when viewed from the third viewpoint, thereby generating an image as seen from the first viewpoint corresponding to the left eye and an image as seen from the second viewpoint corresponding to the right eye; 13. A method for controlling an information processing apparatus comprising the steps of:

15. A program for causing a computer to function as each of the means of the information processing device according to any one of claims 1 to 13.

Images