Imaging apparatus and control method for the same

Abstract

To obtain a suitable (natural) sense of solidity for an object (subject) in a virtual space.SOLUTION: An imaging apparatus is for capturing stereo images of a subject to be placed in a virtual space using a first optical system and a second optical system in a real space. The imaging apparatus has first acquisition means for acquiring first information on a convergence angle of a person with respect to the subject in the virtual space and control means for controlling a distance between the first optical system and the second optical system on the basis of the first information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an imaging device and a control method for an imaging device. [Background technology]

[0002] A technology has been proposed that captures two images (right and left images) from two positions (viewpoints) and allows the two images to be viewed with the left and right eyes (the right image is viewed with the right eye, and the left image is viewed with the left eye). When a person views two images from different viewpoints with the left and right eyes, the brain creates a sense of three-dimensionality. Therefore, stereoscopic vision is achieved by viewing two images from different viewpoints with the left and right eyes. The greater the convergence angle, which is the angle between the straight line from one viewpoint to the subject and the straight line from the other viewpoint to the subject, the greater the sense of three-dimensionality. Hereinafter, video (images) that are viewed stereoscopically will be referred to as stereoscopic video (stereoscopic images).

[0003] Technologies have been proposed that allow people in different locations to communicate in real time using video and audio, such as online conferences and online medical examinations, and technologies have also been proposed that allow communication using 3D video.Technologies have also been proposed that provide users with the sensation that an object (such as a person) is actually present by synthesizing (displaying) a 3D image of the object in a virtual space such as the Metaverse, which is constructed using CG images or real-life images.

[0004] Patent Document 1 discloses an imaging device capable of changing the distance between two lenses for obtaining stereo images (right and left images). Changing the distance between the two lenses changes the convergence angle with respect to a subject, and changes the three-dimensional effect of the subject when viewing a stereo image. Patent Document 2 discloses a technology for changing the image quality (roughness) of a person depending on the distance from the user to the person in a virtual space. Patent Document 3 discloses a technology for artificially adjusting the three-dimensional effect by adjusting the angle (orientation) of the left and right display surfaces depending on the distance from the user to an object in a virtual space. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2020-154008 A [Patent Document 2] Japanese Patent Application Publication No. 10-40423 [Patent Document 3] Japanese Patent Application Publication No. 10-48569 Summary of the Invention [Problem to be solved by the invention]

[0006] However, conventional technologies may not provide a suitable (natural) three-dimensional effect for an object (subject) in a virtual space. For example, even if the technology disclosed in Patent Document 1 is used to capture an image at a distance (the distance between two lenses) that is designed to provide a specific three-dimensional effect, the convergence angle of the user with respect to the object changes depending on the distance from the user to the object in the virtual space, so that a suitable three-dimensional effect may not be obtained. The technology disclosed in Patent Document 2 does not change the three-dimensional effect. And, even if the technology disclosed in Patent Document 3 is used, the three-dimensional effect is merely adjusted in a pseudo manner depending on the angle (direction) of the display surface, and the displayed image does not change, so that it is insufficient to provide a suitable three-dimensional effect.

[0007] An object of the present invention is to provide a suitable (natural) three-dimensional effect for an object (subject) in a virtual space. [Means for solving the problem]

[0008] A first aspect of the present invention is an imaging device that uses a first optical system and a second optical system in real space to capture stereo images of a subject placed in a virtual space, characterized in that the imaging device has a first acquisition means that acquires first information regarding the convergence angle of a person relative to the subject in the virtual space, and a control means that controls the distance between the first optical system and the second optical system based on the first information.

[0009] A second aspect of the present invention is an imaging device that uses a first optical system and a second optical system in real space to capture stereo images of a subject placed in a virtual space, the imaging device having a control means for controlling the distance between the first optical system and the second optical system, and is characterized in that the distance between the first optical system and the second optical system is different when the distance from the subject to a person in the virtual space is a first distance and when the distance is a second distance different from the first distance.

[0010] A third aspect of the present invention is an imaging device that uses a first optical system and a second optical system in real space to capture stereo images of a subject placed in a virtual space, the imaging device having a control means for controlling the distance between the first optical system and the second optical system, and is characterized in that the distance between the first optical system and the second optical system is different when a person viewing the subject in the virtual space has a first distance between their eyes and when a person has a second distance between their eyes that is different from the first distance.

[0011] A fourth aspect of the present invention is a control method for an imaging device that captures stereo images of a subject placed in a virtual space using a first optical system and a second optical system in real space, the control method comprising the steps of acquiring information regarding a convergence angle of a person relative to the subject in the virtual space, and controlling the distance between the first optical system and the second optical system based on the information.

[0012] A fifth aspect of the present invention is a program for causing a computer to function as each of the means of the imaging device described above.A sixth aspect of the present invention is a computer-readable storage medium storing a program for causing a computer to function as each of the means of the imaging device described above. Effect of the Invention

[0013] According to the present invention, a suitable (natural) three-dimensional effect can be obtained for an object (subject) in a virtual space. [Brief description of the drawings]

[0014] [Figure 1] 1 is a configuration diagram of a system according to a first embodiment. [Diagram 2] 4 is a flowchart showing the operation of the system according to the first embodiment. [Diagram 3] FIG. 4 is an explanatory diagram of a method for calculating a convergence angle according to the first embodiment. [Figure 4] FIG. 4 is an explanatory diagram of a method for calculating an imaging base line length according to the first embodiment. [Diagram 5] FIG. 11 is a configuration diagram of a system according to a second embodiment. [Figure 6] 10 is a flowchart showing the operation of a system according to a second embodiment. [Figure 7] FIG. 11 is an explanatory diagram of a method for calculating a convergence angle according to the second embodiment. [Figure 8] 13 is an explanatory diagram of a method for calculating an imaging base line length according to the second embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] (Embodiment 1) Hereinafter, a first embodiment of the present invention will be described. FIG. 1 is a configuration diagram of a system according to the first embodiment. In FIG. 1, a stereoscopic imaging device 101 is configured to capture images of left and right lenses (optical systems) in real space. ) to capture a stereo image (three-dimensional image) of a subject. The information processing device 102 generates and manages a virtual space, and generates a display image for the display device 103 (an image in which the subject of the three-dimensional imaging device 101 is arranged in the virtual space) by synthesizing the three-dimensional image captured by the three-dimensional imaging device 101 with an image in the virtual space. The display device 103 displays the display image generated by the information processing device 102.

[0016] In the first embodiment, for example, a user A (person) is imaged by the stereoscopic imaging device 101, and a display image A' in which the user A is arranged in a virtual space is generated by the information processing device 102. Then, a user B (another person) uses a display device 103 to view the display image A' (user A in the virtual space). Also, a stereoscopic imaging device similar to the stereoscopic imaging device 101 images a user B, and a display image B' in which the user B is arranged in the virtual space is generated by the information processing device 102 (or an information processing device similar to the information processing device 102). Then, the user B views the display image B' (user B in the virtual space) using a display device similar to the display device 103. In this way, the users A and B can communicate with each other in the virtual space.

[0017] The stereoscopic imaging device 101 is, for example, a twin-lens digital camera. The information processing device 102 is, for example, a server connected to the stereoscopic imaging device 101 and the display device 103 via the Internet, or a personal computer connected to the display device 103. The display device 103 is, for example, a display device that can be worn on the head (head-mounted display). Note that the stereoscopic imaging device 101, the information processing device 102, and the display device 103 are not limited to these.

[0018] The stereoscopic imaging device 101 will be described. The optical systems 104a and 104b are a pair of left and right optical systems. Each of the optical systems 104a and 104b includes a lens and an aperture, and is configured to enable focus adjustment and exposure adjustment. The imaging elements 105a and 105b are a pair of left and right imaging elements. Each of the imaging elements 105a and 105b is, for example, a CMOS sensor, and captures an image by converting an optical image into an electrical signal (analog image signal). Each of the imaging elements 105a and 105b includes an A / D conversion circuit, converts the obtained analog image signal into digital image data, and outputs the digital image data. Note that an example in which two images are captured by the two imaging elements 105a and 105b will be described, but two images may be captured by one imaging element.

[0019] The stereoscopic image generating unit 106 uses two images (digital image data) output from the two imaging elements 105a and 105b, respectively, to generate a stereoscopic image (stereo image) in which the two images are arranged as a pair of left and right images. The communication unit 107 communicates with the information processing device 102. For example, the communication unit 107 transmits a stereoscopic image to the information processing device 102 and receives convergence angle information from the information processing device 102. The convergence angle information is information related to a convergence angle, for example, information indicating the convergence angle.

[0020] The distance measuring unit 108 acquires subject distance information by measuring the distance from the stereoscopic imaging device 101 to the subject (user A) in real space. The subject distance information is information on the distance from the stereoscopic imaging device 101 to the user A, for example, information indicating the distance. The imaging base line length determination unit 109 determines the imaging base line length (the distance between the optical system 104a and the optical system 104b) based on the convergence angle information acquired by the communication unit 107 and the subject distance information acquired by the distance measuring unit 108. Note that, when the distance from the stereoscopic imaging device 101 to the user A is determined in advance, the imaging base line length may be determined based only on the convergence angle information. The imaging base line length control unit 110 controls (changes) the imaging base line length of the stereoscopic imaging device 101 to the imaging base line length determined by the imaging base line length determination unit 109 by driving the optical systems 104a and 104b.

[0021] The information processing device 102 will be described. The communication unit 111 communicates with the stereoscopic imaging device 101 (communication unit 107). For example, the communication unit 111 transmits convergence angle information to the stereoscopic imaging device 101 and receives a stereoscopic image from the stereoscopic imaging device 101. The virtual space generation unit 112 generates and manages data of the virtual space. The virtual space generation unit 112 also manages the distance between virtual objects in the virtual space and information on users (users A and B) using the virtual space. The display image generation unit 113 synthesizes the stereoscopic image acquired by the communication unit 111 with the virtual space generated by the virtual space generation unit 112, and generates a display image representing the field of view of user B (virtual object) in the virtual space. The communication unit 114 communicates with the display device 103. For example, the communication unit 114 transmits a display image to the display device 103 and receives user baseline length information of user B from the display device 103. The user baseline length information of user B is information relating to the user baseline length of user B (the distance between the eyes of user B), and is, for example, information indicating the user baseline length.

[0022] The imaging / display control unit 115 controls the stereoscopic imaging device 101 and the display device 103. For example, the imaging / display control unit 115 acquires user distance information related to the distance from user A (virtual object) to user B (virtual object) in the virtual space from the virtual space generation unit 112, and acquires user baseline length information of user B from the communication unit 114. The user distance information indicates, for example, the distance from user A to user B in the virtual space. The convergence angle calculation unit 116 calculates the convergence angle of user B with respect to user A in the virtual space based on the user distance information and user baseline length information acquired by the imaging / display control unit 115. Convergence angle information indicating this convergence angle is transmitted to the stereoscopic imaging device 101. Note that, since the difference in the stereoscopic effect due to the difference in the user baseline length is relatively small, a fixed value determined in advance may be used as the user baseline length, and the convergence angle may be calculated based only on the user distance information.

[0023] The information processing device 102 performs the same process as above for the stereoscopic imaging device of user B and the display device of user A. As a result, convergence angle information indicating the convergence angle of user A with respect to user B in the virtual space is transmitted to the stereoscopic imaging device of user B, and a display image representing the field of view of user A in the virtual space is transmitted to the display device of user A.

[0024] The display device 103 will be described. The communication unit 117 communicates with the information processing device 102 (communication unit 114). For example, the communication unit 117 transmits user baseline length information of the user B to the information processing device 102 and receives a display image from the information processing device 102. The display unit 118 displays the display image received by the communication unit 117. The user baseline length detection unit 119 detects the user baseline length of the user B of the display device 103.

[0025] Note that a part of the processing of the information processing device 102 may be performed by the stereoscopic imaging device 101, and a part of the processing of the information processing device 102 may be performed by the display device 103. For example, user distance information (and user baseline length information) may be transmitted to the stereoscopic imaging device 101, and the stereoscopic imaging device 101 may calculate the convergence angle of user B with respect to user A in the virtual space. The system may not include the information processing device 102, and the processing of the information processing device 102 may be performed by the stereoscopic imaging device 101 and the display device 103.

[0026] The operation of the system according to the first embodiment will be described with reference to Figs. 2 to 4. Fig. 2 is a flowchart showing the operation of the system according to the first embodiment. The operation of Fig. 2 is realized, for example, by each device executing a program stored in advance in each device. Fig. 3 is an explanatory diagram of a method for calculating the convergence angle of user B with respect to user A in a virtual space, and is a diagram showing the positional relationship between user A and user B in the virtual space. Fig. 4 is an explanatory diagram of a method for calculating an imaging base line length, and is a diagram showing the positional relationship between stereoscopic imaging device 101A and user A in real space. The operation when a display image is displayed on display device 103 of user B will be described below. The operation of displaying a display image on the display device of user A is similar to the operation of displaying a display image on the display device 103 of user B, and therefore a description thereof will be omitted.

[0027] In S201, the imaging / display control unit 115 acquires user distance information indicating the distance from user A to user B in the virtual space (distance LAB in FIG. 3) from the virtual space generation unit 112. In addition, the imaging / display control unit 115 acquires user baseline length information indicating the user baseline length of user B detected by the user baseline length detection unit 119 (user baseline length WB in FIG. 3) via the communication units 114 and 117.

[0028] In S202, the convergence angle calculation unit 116 calculates the convergence angle θAB of user B with respect to user A in the virtual space from the distance LAB and the user baseline length WB acquired in S201, using the following Equation 1. θAB=2×arctan(WB / (2×LAB)) (Formula 1)

[0029] In S203, the imaging base line length determination unit 109 determines the imaging base line length based on the convergence angle information acquired by the communication unit 107 and the subject distance information acquired by the distance measurement unit 108. For example, the imaging base line length is determined so that the convergence angle of the stereoscopic imaging device 101 with respect to the user A in the real space approximately matches the convergence angle θAB of the user B with respect to the user A in the virtual space. The convergence angle information indicates the convergence angle θAB calculated in S202, and the subject distance information indicates the distance M (the distance from the stereoscopic imaging device 101 to the user A) in FIG. 4. The imaging base line length determination unit 109 calculates the imaging base line length CAB from the convergence angle θAB and the distance M using the following Equation 2. CAB=2×M×tan(θAB / 2) (Formula 2)

[0030] In S204, the imaging base line length control unit 110 drives the optical systems 104a and 104b to control (change) the imaging base line length of the stereoscopic imaging device 101 to the imaging base line length CAB calculated in S203. In this way, a stereoscopic image that can provide the stereoscopic effect when user B visually recognizes user A at the distance LAB is captured.

[0031] In S205, the image pickup elements 105a and 105b capture images of the user A, the stereoscopic image generator 106 generates a stereoscopic image, and the communication unit 107 transmits the stereoscopic image to the information processing device 102.

[0032] In S206, the display image generation unit 113 synthesizes the stereoscopic image acquired by the communication unit 111 (the stereoscopic image acquired in S205) with the virtual space generated by the virtual space generation unit 112, and generates a display image representing the field of view of the user B in the virtual space. The generated display image is transmitted to the display unit 118 via the communication units 114 and 117.

[0033] In S207, the display unit 118 displays the display image generated in S206.

[0034] In S208, the imaging / display control unit 115 determines whether or not a termination condition is satisfied. The termination condition may be a condition that termination is instructed by the user B, or a condition that the display device 103 is disconnected from the information processing device 102. S201 to S208 are repeated until the termination condition is satisfied, and when the termination condition is satisfied, the operation in FIG. 2 ends.

[0035] By repeating S201 to S208, the process from user A to user B in the virtual space is When the distance LAB to the user B changes, the imaging baseline length of the stereoscopic imaging device 101 is controlled (changed) in accordance with the change. This allows a stereoscopic image that can provide a suitable (natural) stereoscopic effect to be always captured. Since the user baseline length WB does not change, it is necessary to obtain it only the first time and it is not necessary to obtain it repeatedly. The user baseline length WB may be registered in advance in the virtual space generation unit 112 as information about user B. The previously registered user baseline length WB may then be used.

[0036] 2, even if the person as user B is constant, the imaging base line length of stereoscopic imaging device 101 will be different when the distance LAB from user A to user B in the virtual space is a first distance and when it is a second distance (≠ first distance). Also, even if the distance LAB is constant, the imaging base line length of stereoscopic imaging device 101 will be different when user B is a person whose eyes are spaced a first distance (user base line length) and when it is a person whose eyes are spaced a second distance (≠ first distance).

[0037] As described above, according to the first embodiment, the stereoscopic imaging device 101 acquires information on the convergence angle of a person with respect to a subject in a virtual space, and controls the distance between the optical systems 104a and 104b based on the information. This makes it possible to obtain a suitable (natural) stereoscopic effect for an object (subject) in the virtual space.

[0038] (Embodiment 2) Hereinafter, a second embodiment of the present invention will be described. Note that in the following, descriptions of the same points as in the first embodiment (for example, the same configurations and processes as in the first embodiment) will be omitted as appropriate.

[0039] In the first embodiment, an example in which two users communicate in a virtual space is described. In the second embodiment, an example in which more than two users communicate in a virtual space is described. The number of users is not particularly limited, but in the second embodiment, it is assumed that three users A to C communicate. When a plurality of users view the same virtual object, the convergence angle of the users with respect to the virtual object may differ among the plurality of users. For example, in the virtual space, the convergence angle of user B with respect to user A may differ from the convergence angle of user C with respect to user A. Therefore, even if the stereoscopic image of user A is a stereoscopic image that allows user B to obtain a suitable (natural) stereoscopic effect, it is not necessarily a stereoscopic image that allows user C to obtain a suitable (natural) stereoscopic effect. In the second embodiment, it is made possible for both user B and user C to obtain a suitable (natural) stereoscopic effect with respect to the stereoscopic image of user A.

[0040] FIG. 5 is a configuration diagram of a system according to the second embodiment. The system according to the second embodiment includes stereoscopic imaging devices 101A to 101C, an information processing device 102, and display devices 103A to 103C. The stereoscopic imaging device 101A is a stereoscopic imaging device for user A, the stereoscopic imaging device 101B is a stereoscopic imaging device for user B, and the stereoscopic imaging device 101C is a stereoscopic imaging device for user C. The display device 103A is a display device for user A, the display device 103B is a display device for user B, and the display device 103C is a display device for user C. Each of the stereoscopic imaging devices 101A to 101C has a configuration similar to that of the stereoscopic imaging device 101 of the first embodiment (FIG. 1). The information processing device 102 has a configuration similar to that of the information processing device 102 of the first embodiment (FIG. 1). Each of the display devices 103A to 103C has a configuration similar to that of the display device 103 of the first embodiment (FIG. 1).

[0041] The operation of the system according to the second embodiment will be described with reference to Figs. 6, 7(A) to 7(C), 8(A) and 8(B). Fig. 6 is a flowchart showing the operation of the system according to the second embodiment. The operation of Fig. 6 is realized, for example, by each device executing a program pre-stored in each device. Figs. 7(A) to 7(C) are explanatory diagrams of a method for calculating the convergence angle of users B and C with respect to user A in a virtual space, and ... 8A and 8B are explanatory diagrams of a method for calculating the imaging base line length, and are diagrams showing the positional relationship between the stereoscopic imaging device 101A and the user A in the real space. The following describes the operation when the stereoscopic image of the user A is displayed on the display devices 103B and 103C of the users B and C. The operation when the stereoscopic image of the user B is displayed on the display devices 103A and 103C of the users A and C is the same as the operation when the stereoscopic image of the user A is displayed on the display devices 103B and 103C of the users B and C. The operation when the stereoscopic image of the user C is displayed on the display devices 103A and 103B of the users A and B is also the same as the operation when the stereoscopic image of the user A is displayed on the display devices 103B and 103C of the users B and C.

[0042] In S601, the imaging / display control unit 115 acquires user distance information indicating the distance from user A to user B in the virtual space (distance LAB in FIG. 7(B)) from the virtual space generation unit 112. Similarly, the imaging / display control unit 115 acquires user distance information indicating the distance from user A to user C in the virtual space (distance LAC in FIG. 7(C)) from the virtual space generation unit 112. In addition, the imaging / display control unit 115 acquires user baseline length information indicating the user baseline length of user B (user baseline length WB in FIG. 7(B)) detected by the user baseline length detection unit 119 of the display device 103B via the communication unit 114 and the communication unit 117 of the display device 103B. Similarly, the imaging / display control unit 115 acquires user baseline length information indicating the user baseline length of user C (user baseline length WC in Figure 7(C)) detected by the user baseline length detection unit 119 of the display device 103C via the communication unit 114 and the communication unit 117 of the display device 103C.

[0043] In S602, the imaging / display control unit 115 selects a user (a display device that displays a stereoscopic image of user A) who will view the stereoscopic image of user A. The imaging / display control unit 115 selects user B or user C. Details will be described later, but in the operation of FIG. 6, S601 to S609 are repeated. It is assumed that user B and user C are alternately selected by repeating S602.

[0044] Note that S601 may be omitted, and after S602, user distance information and user baseline length corresponding to the user selected in S602 may be acquired. For example, when user B is selected, user distance information indicating the distance LAB and user baseline length information indicating the user baseline length WB may be acquired. Then, when user C is selected, user distance information indicating the distance LAB and user baseline length information indicating the user baseline length WB may be acquired.

[0045] S603 to S609 are the same as S201 to S208 in the first embodiment (FIG. 2).

[0046] In S603, the convergence angle calculation unit 116 calculates the convergence angle. When user B is selected in S602, the convergence angle θAB of user B with respect to user A in the virtual space is calculated from the distance LAB and user baseline length WB of user B using the following formula 1-1. When user C is selected in S602, the convergence angle θAC of user C with respect to user A in the virtual space is calculated from the distance LAC and user baseline length WC of user C using the following formula 1-2. θAB=2×arctan(WB / (2×LAB)) (Formula 1-1) θAC=2×arctan(WC / (2×LAC)) (Formula 1-2)

[0047] 7B and 7C, the distance LAB and the user baseline length WB of user B are different from the distance LAC and the user baseline length WC of user C. Therefore, the convergence angle θAB of user B is also different from the convergence angle θAC of user C.

[0048] In S604, the imaging base line length determination unit 109 of the stereoscopic imaging device 101A determines the imaging base line length. When user B is selected in S602, the imaging base line length CAB is calculated from the convergence angle θAB and the distance M using the following formula 2-1. When user C is selected in S602, the imaging base line length CAC is calculated from the convergence angle θAC and the distance M using the following formula 2-2. CAB=2×M×tan(θAB / 2) (Formula 2-1) CAC=2×M×tan(θAC / 2) (Formula 2-2)

[0049] In S605, the imaging base line length control unit 110 of the stereoscopic imaging device 101A controls the imaging base line length of the stereoscopic imaging device 101A by driving the optical systems 104a and 104b. When user B is selected in S602, the imaging base line length of the stereoscopic imaging device 101A is controlled (changed) to the imaging base line length CAB. This allows a stereoscopic image to be captured that provides the stereoscopic effect that occurs when user B views user A at distance LAB. When user C is selected in S602, the imaging base line length of the stereoscopic imaging device 101A is controlled (changed) to the imaging base line length CAC. This allows a stereoscopic image to be captured that provides the stereoscopic effect that occurs when user C views user A at distance LAC.

[0050] In S606, the stereoscopic imaging device 101A (the imaging elements 105a and 105b, the stereoscopic image generating unit 106, and the communication unit 107) captures an image of the user A, generates a stereoscopic image, and transmits the stereoscopic image to the information processing device 102.

[0051] In S607, the display image generation unit 113 synthesizes the stereoscopic image acquired by the communication unit 111 (the stereoscopic image acquired in S606) with the virtual space generated by the virtual space generation unit 112 to generate a display image. If the user B is selected in S602, a display image representing the field of view of the user B in the virtual space is generated, and the display image is transmitted to the display device 103B via the communication unit 114. If the user C is selected in S602, a display image representing the field of view of the user C in the virtual space is generated, and the display image is transmitted to the display device 103C via the communication unit 114.

[0052] In S608, the display image generated in S607 is displayed. If user B is selected in S602, the display unit 118 of the display device 103B displays the display image, and if user C is selected in S602, the display unit 118 of the display device 103C displays the display image.

[0053] In S609, the imaging / display control unit 115 determines whether or not a termination condition is satisfied. The termination condition may be a condition that an end instruction is issued by user B or user C, or a condition that the display device 103B or the display device 103C is disconnected from the information processing device 102. S601 to S609 are repeated until the termination condition is satisfied, and when the termination condition is satisfied, the operation in FIG. 6 ends.

[0054] According to the operation of FIG. 6, the imaging base line length of the stereoscopic imaging device 101A is alternately switched between the imaging base line length corresponding to the convergence angle of the user B and the imaging base line length corresponding to the convergence angle of the user C. Then, a stereoscopic image obtained with the imaging base line length corresponding to the convergence angle of the user B is displayed on the display device 103B, and a stereoscopic image obtained with the imaging base line length corresponding to the convergence angle of the user C is displayed on the display device 103B. This allows both the user B and the user C to obtain a suitable (natural) stereoscopic effect for the stereoscopic image of the user A. The timing of switching the imaging base line length of the stereoscopic imaging device 101A is not particularly limited, but for example, the imaging base line length of the stereoscopic imaging device 101A is switched at a predetermined time interval.

[0055] The imaging base line length of the stereoscopic imaging device 101A may be controlled based on information about the convergence angle of a user who is looking at user A in the virtual space, not based on information about the convergence angle of a user who is looking at user A in the virtual space. For example, the imaging base line length of the stereoscopic imaging device 101A may not be controlled to the imaging base line length CAB when user B is not looking at user A, and may be controlled to the imaging base line length CAB only when user B is looking at user A. Similarly, the imaging base line length of the stereoscopic imaging device 101A may not be controlled to the imaging base line length CAC when user C is not looking at user A, and may be controlled to the imaging base line length CAC only when user C is looking at user A.

[0056] The method of determining whether or not a user is looking at another user in the virtual space is not particularly limited. For example, the virtual space generation unit 112 may manage the position and orientation of a virtual object. In that case, position and orientation information regarding the position and orientation of the user (virtual object) in the virtual space for each of users A to C may be acquired from the virtual space generation unit 112. The position and orientation information is, for example, information indicating the position and orientation of the user (virtual object) in the virtual space. From the position and orientation information of users A and B, it can be determined whether or not user B is looking at user A, and from the position and orientation information of users A and C, it can be determined whether or not user C is looking at user A.

[0057] Although the example in which the imaging base line length of the stereoscopic imaging device 101A is switched among a plurality of imaging base line lengths has been described, the imaging base line length of the stereoscopic imaging device 101A may be controlled to an average imaging base line length of the plurality of imaging base line lengths. In this way, both the user B and the user C can obtain a generally favorable (natural) stereoscopic effect from the stereoscopic image of the user A.

[0058] As described above, according to the second embodiment, the stereoscopic imaging device acquires a plurality of pieces of information corresponding to a plurality of persons as information on the convergence angle, and controls the distance between the optical system 104a and the optical system 104b based on the plurality of pieces of information. This allows each person to obtain a suitable (natural) three-dimensional effect of the object (subject) in the virtual space.

[0059] The above-described embodiment (including the modified examples) is merely an example, and the present invention also includes configurations obtained by appropriately modifying or changing the configurations of the above-described embodiment within the scope of the gist of the present invention. The present invention also includes configurations obtained by appropriately combining the configurations of the above-described embodiment.

[0060] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0061] The disclosure of the present embodiment includes the following configuration, method, program, and medium. (Configuration 1) An imaging device that captures stereo images of a subject disposed in a virtual space using a first optical system and a second optical system in a real space, a first acquisition means for acquiring first information regarding a convergence angle of a person with respect to the subject in the virtual space; a control means for controlling a distance between the first optical system and the second optical system based on the first information; An imaging device comprising: (Configuration 2) and a second acquisition means for acquiring second information relating to a distance from the imaging device to the subject. In addition, The control means controls the distance between the first optical system and the second optical system based on the first information and the second information. 2. The imaging device according to configuration 1, (Configuration 3) the first information indicates a distance from the subject to the person in the virtual space; The control means calculating a convergence angle of the person with respect to the subject in the virtual space based on the first information; The distance between the first optical system and the second optical system is controlled based on the calculated convergence angle. 3. The imaging device according to configuration 1 or 2. (Configuration 4) The control means controls a distance between the first optical system and the second optical system so that a convergence angle of the imaging device with respect to the subject in the real space approximately coincides with a convergence angle of the person with respect to the subject in the virtual space. 4. The imaging device according to any one of configurations 1 to 3. (Configuration 5) the first acquisition means acquires a plurality of pieces of first information corresponding to a plurality of persons, The control means controls the distance between the first optical system and the second optical system based on the plurality of first information. 5. The imaging device according to any one of configurations 1 to 4. (Configuration 6) The control means switches the distance between the first optical system and the second optical system among a plurality of distances based on the plurality of first information, respectively. 6. The imaging device according to configuration 5, (Configuration 7) The control means switches the distance between the first optical system and the second optical system at predetermined time intervals. 7. The imaging device according to configuration 6, (Configuration 8) The control means controls a distance between the first optical system and the second optical system based on the first information when the person is looking at the subject in the virtual space. 8. The imaging device according to any one of configurations 1 to 7, (Configuration 9) the first acquisition means further acquires third information relating to a position and an orientation of the person in the virtual space; The control means determines whether the person is looking at the subject in the virtual space based on the third information. 9. The imaging device according to configuration 8, (Configuration 10) the first information indicates a distance from the subject to the person in the virtual space; the first acquisition means further acquires fourth information indicating a distance between the person's eyes; The control means Calculating a convergence angle of the person with respect to the subject in the virtual space based on the first information and the fourth information; The distance between the first optical system and the second optical system is controlled based on the calculated convergence angle. 10. The imaging device according to any one of configurations 1 to 9. (Configuration 11) An imaging device that captures stereo images of a subject disposed in a virtual space using a first optical system and a second optical system in a real space, a control unit for controlling a distance between the first optical system and the second optical system, The distance between the first optical system and the second optical system is different between a case where the distance from the subject to a person in the virtual space is a first distance and a case where the distance is a second distance different from the first distance. 1. An imaging device comprising: (Configuration 12) An imaging device that captures stereo images of a subject disposed in a virtual space using a first optical system and a second optical system in a real space, a control unit for controlling a distance between the first optical system and the second optical system, The distance between the first optical system and the second optical system is different between a person whose eyes are spaced a first distance from each other and a person whose eyes are spaced a second distance from each other, the distance being different from the first distance. 1. An imaging device comprising: (method) A method for controlling an imaging device that captures stereo images of a subject placed in a virtual space using a first optical system and a second optical system in a real space, comprising the steps of: acquiring information regarding a convergence angle of a person with respect to the subject in the virtual space; controlling a distance between the first optical system and the second optical system based on the information; A control method comprising the steps of: (program) 13. A program for causing a computer to function as each of the means of the imaging device according to any one of configurations 1 to 12. (medium) 13. A computer-readable storage medium storing a program for causing a computer to function as each of the means of the imaging device according to any one of configurations 1 to 12. [Explanation of symbols]

[0062] 101, 101A to 101C: stereoscopic imaging device 104a, 104b: Optical system 107: Communication unit 110: Imaging base line length control unit

Claims

1. An imaging device that captures stereo images of a subject disposed in a virtual space using a first optical system and a second optical system in a real space, a first acquisition means for acquiring first information regarding a convergence angle of a person with respect to the subject in the virtual space; a control means for controlling a distance between the first optical system and the second optical system based on the first information; An imaging device comprising:

2. The imaging device further includes a second acquisition unit configured to acquire second information related to a distance from the imaging device to the subject, The control means controls the distance between the first optical system and the second optical system based on the first information and the second information.

2. The imaging device according to claim 1 .

3. the first information indicates a distance from the subject to the person in the virtual space; The control means Calculating a convergence angle of the person with respect to the subject in the virtual space based on the first information; The distance between the first optical system and the second optical system is controlled based on the calculated convergence angle.

2. The imaging device according to claim 1 .

4. The control means controls a distance between the first optical system and the second optical system so that a convergence angle of the imaging device with respect to the subject in the real space approximately coincides with a convergence angle of the person with respect to the subject in the virtual space.

2. The imaging device according to claim 1 .

5. the first acquisition means acquires a plurality of pieces of first information corresponding to a plurality of persons, The control means controls the distance between the first optical system and the second optical system based on the plurality of first information.

2. The imaging device according to claim 1 .

6. The control means switches the distance between the first optical system and the second optical system among a plurality of distances based on the plurality of first information, respectively.

6. The imaging device according to claim 5.

7. The control means switches the distance between the first optical system and the second optical system at predetermined time intervals.

7. The imaging device according to claim 6.

8. The control means controls a distance between the first optical system and the second optical system based on the first information when the person is looking at the subject in the virtual space.

2. The imaging device according to claim 1 .

9. the first acquisition means further acquires third information relating to a position and an orientation of the person in the virtual space; The control means determines whether the person is looking at the subject in the virtual space based on the third information.

9. The imaging device according to claim 8.

10. the first information indicates a distance from the subject to the person in the virtual space; the first acquisition means further acquires fourth information indicating a distance between the eyes of the person; The control means Calculating a convergence angle of the person with respect to the subject in the virtual space based on the first information and the fourth information; The distance between the first optical system and the second optical system is controlled based on the calculated convergence angle.

2. The imaging device according to claim 1 .

11. An imaging device that captures stereo images of a subject disposed in a virtual space using a first optical system and a second optical system in a real space, a control unit for controlling a distance between the first optical system and the second optical system, The distance between the first optical system and the second optical system is different between a case where the distance from the subject to a person in the virtual space is a first distance and a case where the distance is a second distance different from the first distance.

1. An imaging device comprising:

12. An imaging device that captures stereo images of a subject disposed in a virtual space using a first optical system and a second optical system in a real space, a control unit for controlling a distance between the first optical system and the second optical system, The distance between the first optical system and the second optical system is different between a person whose eyes are spaced a first distance from each other and a person whose eyes are spaced a second distance from each other, the distance being different from the first distance.

1. An imaging device comprising:

13. A method for controlling an imaging device that captures stereo images of a subject placed in a virtual space using a first optical system and a second optical system in a real space, the method comprising the steps of: acquiring information regarding a convergence angle of a person with respect to the subject in the virtual space; controlling a distance between the first optical system and the second optical system based on the information; A control method comprising the steps of:

14. A program for causing a computer to function as each of the means of the imaging device according to any one of claims 1 to 12.

15. A computer-readable storage medium storing a program for causing a computer to function as each of the means of the imaging device according to any one of claims 1 to 12.

Images