Information processing device, its control method, and program

Abstract

To reduce processing when a user is not wearing an image display device.SOLUTION: An information processing device that transmits an image to an image display device worn by a recipient includes transforming means for transforming the image such that distortion is suppressed, acquiring means for acquiring information indicating the usage state of the image display device, and transmitting means for transmitting an image before being converted by converting means or after being converted by the converting means according to the information indicating the usage state acquired by the acquiring means.SELECTED DRAWING: Figure 6

Description

Technical Field

[0004] , , , , , ,

[0002] , , , , , , , , , ,

[0005] ,

[0003] , ,

[0001] The present invention relates to an information processing apparatus, a control method thereof 、 program and video call systems and a program.

Background Art

[0002] <00,00011>There is known a technique of capturing two images with a parallax using two cameras and displaying the captured two images in a stereoscopic viewable manner. Patent Document 1 discloses a camera that can capture two images with a parallax at once by mounting a lens unit having two optical systems.

[0003] <, If the two optical systems are arranged to image the same direction, an image (a hemispherical image) that images a range of 180 degrees or a stereoscopic viewable image can be generated from the two images with a parallax acquired respectively. If the two optical systems are arranged to image in opposite directions, an image (an omnidirectional image) of a range of 360 degrees can be generated from the two images acquired respectively.

[0004] Such a camera having two lens optical systems can also be used in a system for performing a video call via a communication network such as the Internet. The stereoscopic viewable image captured by the camera is converted so that the user can appropriately view it, and is transmitted from the transmitting side to a communication terminal at a remote receiving side. On the receiving side, by displaying the converted image on a display device such as a head-mounted display (HMD), the receiver can communicate as if being in the same space as the sender.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] In such systems, prolonged viewing can lead to physical discomfort such as nausea and dizziness, a condition known as VR sickness. Regularly removing the HMD (Head-Mounted Display) is effective in mitigating VR sickness. Furthermore, users may remove the HMD for reasons other than VR sickness, such as taking notes or viewing external documents. Continuously converting images uniformly even when the user is not wearing the HMD places a heavy processing load on the system.

[0007] This invention has been made in view of the above-mentioned problems, and aims to reduce the processing required when the user is not wearing an image display device. [Means for solving the problem]

[0008] The present invention In video calls, An information processing device that transmits an image to an image display device worn by a recipient, Based on the image of the aforementioned recipient, the recipient Image display device Wearing it on your head A means for obtaining information that indicates, If the acquisition means obtains information indicating that the recipient is wearing the image display device on their head, the first image captured via the first optical system and the second image captured via the second optical system are transmitted. If the acquisition means does not obtain information indicating that the recipient is wearing the image display device on their head, one of the first image and the second image is transmitted. It is characterized by having a transmitting means for transmitting. [Effects of the Invention]

[0009] According to the present invention, processing can be reduced when the user is not wearing an image display device. [Brief explanation of the drawing]

[0010] [Figure 1] This is a diagram showing the system configuration of the first embodiment. [Figure 2] This is an external view of the camera. [Figure 3] This is a block diagram showing the camera configuration. [Figure 4] This is a diagram showing the configuration of the lens unit. [Figure 5] This is a block diagram showing the configuration of an information processing device. [Figure 6]This is a flowchart showing the process of the first embodiment. [Figure 7] This is a diagram showing an example of an image. [Figure 8] This figure shows the system configuration of the second embodiment. [Figure 9] This is a flowchart showing the process of the second embodiment. [Modes for carrying out the invention]

[0011] Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. <First Embodiment> Figure 1 is a schematic diagram showing an example of the overall configuration of the system according to the first embodiment. The system according to this embodiment is configured to enable video calls between multiple viewing environments communicating via a communication network. The transmitting environment 151 is an environment where a sender 155 captures stereoscopic images. The transmitting environment 151 consists of a digital camera (camera) 100 and a personal computer (PC) 500a. On the other hand, the receiving environment 152 is an environment where a receiver 156 views the captured images. The receiving environment 152 consists of a head-mounted display (HMD) 154 and a PC 500b. The transmitting environment 151 and the receiving environment 152 can communicate with each other via a communication network 153 such as the Internet.

[0012] The camera 100 has a lens unit 300 attached (connected) thereto. By attaching the lens unit 300, the camera 100 can capture two images (still images or moving images) having a predetermined parallax at a time. Details of the lens unit 300 will be described later. The PC 500a is an information processing device that processes the images captured by the camera 100. The PC 500b is an information processing device that processes the images received from the transmission-side environment 151. The PC 500a and the PC 500b are communicably connected via a communication network 153 and can transmit and receive data to and from each other. Here, the PC 500a and the PC 500b are each configured integrally with a display device, but may be connected to a separate display device. The camera 100 and the PC 500a and the HMD 154 and the PC 500b are communicably connected to each other by wireless or wired means. When the PC 500a and the PC 500b are not distinguished, they are referred to as the PC 500.

[0013] Figs. 2(a) and (b) are external views showing an example of the appearance of the camera 100. Fig. 2(a) is a perspective view of the camera 100 seen from the front side, and Fig. 2(b) is a perspective view of the camera 100 seen from the back side. On the upper surface of the camera 100, there are a shutter button 101, a power switch 102, a mode switch 103, a main electronic dial 104, a sub electronic dial 105, a movie button 106, and an external finder display section 107. The shutter button 101 is an operation section for giving an instruction for preparation for shooting or an instruction for shooting. The power switch 1 is an operation section for switching on and off the power of the camera 100. The mode switch 103 is an operation section for switching various modes. The main electronic dial 104 is a rotary operation section for changing set values such as the shutter speed and aperture. The sub electronic dial 105 is a rotary operation section for moving a selection frame (cursor) or advancing an image. The movie button 106 is an operation section for giving an instruction to start or stop movie shooting (recording). The external finder display section 107 displays various set values such as the shutter speed and aperture.

[0014] Camera 100 has a display unit 108, a touch panel 109, directional keys 110, a SET button 111, an AE lock button 112, a zoom button 113, a playback button 114, a menu button 115, an eyepiece 116, an eyepiece detection unit 118, and a touch bar 119 on its back. The display unit 108 displays images and various information. The touch panel 109 is an operation unit that detects touch operations on the display surface (touch operation surface) of the display unit 108. The directional keys 110 is an operation unit consisting of keys that can be pressed up, down, left, and right (4-way keys). Processing can be performed according to the position pressed on the directional keys 110. The SET button 111 is an operation unit that is mainly pressed when confirming a selection item. The AE lock button 112 is an operation unit that is pressed when fixing the exposure state in shooting standby mode. The zoom button 113 is an operation unit that switches the zoom mode on and off in the live view display (LV display) of the shooting mode. When the magnification mode is on, the live view image (LV image) can be enlarged or reduced by operating the main electronic dial 104. The magnification button 113 is used in playback mode to enlarge the playback image or increase the magnification ratio. The playback button 114 is an operation unit for switching between shooting mode and playback mode. By pressing the playback button 114 in shooting mode, the camera switches to playback mode, and the latest image recorded on the recording medium 227 (described later) can be displayed on the display unit 108.

[0015] The menu button 115 is an operation button that is pressed to display a menu screen on the display unit 108 that allows for various settings. The user can intuitively make various settings using the menu screen displayed on the display unit 108 and the directional keys 110 and SET button 111. The eyepiece section 116 is the part that the user looks through when they place their eye on the eyepiece viewfinder (peep-in type viewfinder) 117. The user can view the image displayed on the EVF 217 (Electronic View Finder), which will be described later, inside the camera 100, through the eyepiece section 116. The eyepiece detection section 118 is a sensor that detects whether or not the user is looking through the eyepiece section 116 (eyepiece viewfinder 117).

[0016] The touch bar 119 is a linear touch operation unit (line touch sensor) capable of receiving touch operations. The touch bar 119 is arranged at a position where it can be touched (touched) with the right thumb in a state where the grip portion 120 is held with the right hand (a state where it is held with the little finger, ring finger, and middle finger of the right hand) so that the shutter button 101 can be pressed with the index finger of the right hand. That is, the touch bar 119 is operable in a state (shooting posture) where the user looks into the eyepiece 116 through the viewfinder 117 and can always press the shutter button 101. The touch bar 119 can receive a tap operation on the touch bar 119 (an operation of touching and releasing without moving the touch position within a predetermined period), a slide operation to the left and right (an operation of moving the touch position while keeping the touch after touching), and the like. The touch bar 119 is an operation unit different from the touch panel 109 and does not have a display function. The touch bar 119 functions as, for example, a multi-function bar (M-Fn bar) to which various functions can be assigned.

[0017] The camera 100 also includes a grip section 120, a thumb rest section 121, a terminal cover 122, a lid 123, a communication terminal 124, etc. The grip section 120 is a holding section shaped to be easily gripped by the user with their right hand when holding the camera 100. With the camera 100 held by gripping the grip section 120 with the little finger, ring finger, and middle finger of the right hand, the shutter button 101 and the main electronic dial 104 are positioned to be operated by the index finger of the right hand. Similarly, in the same position, the sub electronic dial 105 and the touch bar 119 are positioned to be operated by the thumb of the right hand. The thumb rest section 121 (thumb standby position) is a grip section located on the back of the camera 100, in a place where it is easy to rest the thumb of the right hand holding the grip section 120 when no other controls are being operated. The thumb rest section 121 is made of a rubber material or the like to enhance the holding force (grip feel). The terminal cover 122 protects connectors such as connection cables that connect the camera 100 to external equipment (external devices). The cover 123 protects the recording medium 227 and the slot for storing the recording medium 227, which will be described later, by closing the slot. The communication terminal 124 is a terminal for communicating with the camera 100 and a detachable lens unit (such as the lens unit 200 or lens unit 300, which will be described later).

[0018] Figure 3 is a block diagram showing an example of the configuration of camera 100. Components identical to those in Figure 2 are denoted by the same reference numerals, and their descriptions are omitted where appropriate. In Figure 3, a lens unit 200 is attached to camera 100. First, let's describe the lens unit 200. The lens unit 200 is a type of interchangeable lens that can be attached to and removed from the camera 100. The lens unit 200 is a single-lens reflex lens and is an example of a normal lens. The lens unit 200 includes an aperture 201, a lens 202, an aperture drive circuit 203, an AF (autofocus) drive circuit 204, a lens system control circuit 205, a communication terminal 206, etc.

[0019] The aperture 201 is configured to have an adjustable aperture diameter. The lens 202 is composed of multiple lenses. The aperture drive circuit 203 adjusts the amount of light by controlling the aperture diameter of the aperture 201. The AF drive circuit 204 drives the lens 202 to focus. The lens system control circuit 205 controls the aperture drive circuit 203, the AF drive circuit 204, etc., based on instructions from the system control unit 50, which will be described later. The lens system control circuit 205 controls the aperture 201 via the aperture drive circuit 203 and focuses by changing the position of the lens 202 via the AF drive circuit 204. The lens system control circuit 205 can communicate with the camera 100. Specifically, communication takes place via the communication terminal 206 of the lens unit 200 and the communication terminal 124 of the camera 100. The communication terminal 206 is a terminal for the lens unit 200 to communicate with the camera 100.

[0020] Next, the camera 100 will be described. The camera 100 includes a shutter 210, an imaging unit 211, an A / D converter 212, a memory control unit 213, an image processing unit 214, a memory 215, a D / A converter 216, an EVF 217, a display unit 108, and a system control unit 50. The shutter 210 is a focal-plane shutter that can freely control the exposure time of the imaging unit 211 based on instructions from the system control unit 50. The imaging unit 211 is an image sensor composed of a CCD or CMOS element, etc., which converts an optical image into an electrical signal. The imaging unit 211 may have an image plane phase difference sensor that outputs defocus amount information to the system control unit 50. The A / D converter 212 converts the analog signal output from the imaging unit 211 into a digital signal. The image processing unit 214 performs predetermined processing (pixel interpolation, resizing such as reduction, color conversion, etc.) on the data from the A / D converter 212 or the data from the memory control unit 213. The image processing unit 214 also performs predetermined calculation processing using the captured image data, and the system control unit 50 performs exposure control and distance measurement control based on the obtained calculation results. Through this processing, TTL (through-the-lens) AF processing, AE (automatic exposure) processing, EF (flash pre-flash) processing, etc. are performed. Furthermore, the image processing unit 214 performs predetermined calculations using the captured image data, and the system control unit 50 performs TTL-type AWB (auto white balance) processing based on the obtained calculation results.

[0021] Image data from the A / D converter 212 is written to the memory 215 via the image processing unit 214 and the memory control unit 213. Alternatively, image data from the A / D converter 212 is written to the memory 215 via the memory control unit 213 without going through the image processing unit 214. The memory 215 stores image data obtained by the imaging unit 211 and converted into digital data by the A / D converter 212, as well as image data for display on the display unit 108 and EVF 217. The memory 215 has sufficient storage capacity to store a predetermined number of still images, a predetermined amount of video footage, and audio. The memory 215 also serves as a memory for image display (video memory).

[0022] The D / A converter 216 converts the display image data stored in the memory 215 into an analog signal and supplies it to the display unit 108 and EVF 217. Therefore, the display image data written to the memory 215 is displayed on the display unit 108 and EVF 217 via the D / A converter 216. The display unit 108 and EVF 217 perform display according to the analog signal from the D / A converter 216. The display unit 108 and EVF 217 are, for example, displays such as LCDs and OLEDs. The digital signal, which has been A / D converted by the A / D converter 212 and stored in the memory 215, is converted into an analog signal by the D / A converter 216 and sequentially transferred to the display unit 108 and EVF 217 for display, thereby performing live view display.

[0023] The system control unit 50 is a control unit consisting of at least one processor and / or at least one circuit. That is, the system control unit 50 may be a processor, a circuit, or a combination of a processor and a circuit. The system control unit 50 controls the entire camera 100. The system control unit 50 executes the program recorded in the non-volatile memory 219. The system control unit 50 also performs display control by controlling the memory 215, D / A converter 216, display unit 108, EVF 217, etc.

[0024] The camera 100 also includes a system memory 218, a non-volatile memory 219, a system timer 220, a communication unit 221, a posture detection unit 222, and an eyepiece detection unit 118. System memory 218 is, for example, RAM. System memory 218 stores constants, variables, and programs read from non-volatile memory 219 for the operation of the system control unit 50. Non-volatile memory 219 is electrically erasable and recordable memory. For example, EEPROM is used for non-volatile memory 219. Non-volatile memory 219 stores constants, programs, and other information for the operation of the system control unit 50. The system timer 220 is a timing unit that measures time used for various controls and the time of the built-in clock.

[0025] The communication unit 221 transmits and receives video and audio signals to and from external devices connected wirelessly or via wired cables. The communication unit 221 can also connect to wireless LAN (Local Area Network) and the internet. Furthermore, the communication unit 221 can communicate with external devices using Bluetooth® and Bluetooth Low Energy. The communication unit 221 can transmit images (including live images) captured by the imaging unit 211 and images recorded on the recording medium 227. In addition, the communication unit 221 can receive images and other various information from external devices.

[0026] The attitude detection unit 222 detects the attitude of the camera 100 relative to the direction of gravity. Based on the attitude detected by the attitude detection unit 222, it is possible to determine whether the image captured by the imaging unit 211 was taken with the camera 100 held horizontally or vertically. The system control unit 50 can add orientation information corresponding to the attitude detected by the attitude detection unit 222 to the image file of the image captured by the imaging unit 211, or rotate the image according to the detected attitude. For example, an acceleration sensor or a gyroscope sensor can be used for the attitude detection unit 222. It is also possible to use the attitude detection unit 222 to detect the movement of the camera 100 (pan, tilt, lift, whether it is stationary or not, etc.).

[0027] The eyepiece detection unit 118 can detect the approach of any object to the eyepiece unit 116 (eyepiece finder 117). For example, an infrared proximity sensor can be used for the eyepiece detection unit 118. When an object approaches, infrared light emitted from the light emitter of the eyepiece detection unit 118 is reflected by the object and received by the light receiver of the infrared proximity sensor. The distance from the eyepiece unit 116 to the object can be determined by the amount of infrared light received. In this way, the eyepiece detection unit 118 performs eyepiece detection to detect the proximity distance of an object to the eyepiece unit 116. The eyepiece detection unit 118 is an eyepiece detection sensor that detects the approach (eye-catching) and departure (eye-moving) of an eye (object) to the eyepiece unit 116. When an object is detected approaching the eyepiece unit 116 within a predetermined distance from a non-eyepiece state (non-approach state), it is detected that the eye has been caught in the eye. On the other hand, if the object that was detected as approaching moves beyond a predetermined distance from the eyepiece state (close-up state), it is detected that the eye has been removed. The threshold for detecting eyepiece contact and the threshold for detecting eye removal may be different, for example, by providing hysteresis. After eyepiece contact is detected, the eyepiece state is maintained until eye removal is detected. After eye removal is detected, the non-eyepiece state is maintained until eyepiece contact is detected again. The system control unit 50 switches the display (display state) / non-display (non-display state) of the display unit 108 and EVF 217 according to the state detected by the eyepiece detection unit 118. Specifically, at least in the shooting standby state and when the display destination switching setting is set to automatic switching, when the eyepiece is not in use, the display destination is set to the display unit 108 and the display is turned on, and the EVF 217 is hidden. When eyepiece contact is made, the display destination is set to the EVF 217 and the display is turned on, and the display unit 108 is hidden. Furthermore, the eyepiece detection unit 118 is not limited to an infrared proximity sensor; any sensor capable of detecting a state that can be considered as an eyepiece may be used, and other sensors may also be used.

[0028] The camera 100 also includes an external viewfinder display unit 107, an external viewfinder display drive circuit 223, a power control unit 224, a power supply unit 225, a recording medium interface 226, an operation unit 228, and the like. The external viewfinder display unit 107 is driven by the external viewfinder display drive circuit 223 and displays various settings of the camera 100, such as shutter speed and aperture. The power control unit 224 consists of a battery detection circuit, a DC-DC converter, a switch circuit for switching which blocks are energized, and detects whether a battery is installed, the type of battery, and the remaining battery level. The power control unit 224 also controls the DC-DC converter based on the detection results and instructions from the system control unit 50, supplying the necessary voltage to each part, including the recording medium 227, for the required period. The power supply unit 225 includes primary batteries such as alkaline batteries and lithium batteries, secondary batteries such as NiCd batteries, NiMH batteries and Li batteries, and an AC adapter. The recording medium I / F 226 is an interface with the recording medium 227, such as a memory card or hard disk. The recording medium 227 is a memory card or the like for recording captured images and consists of semiconductor memory or a magnetic disk. The recording medium 227 may be detachable from the camera 100 or may be built into the camera 100.

[0029] The operation unit 228 is an input unit that receives user input (user operation) and is used to input various instructions to the system control unit 50. The operation unit 228 includes the shutter button 101, power switch 102, mode switch 103, touch panel 109, and other operation units 229. Other operation units 229 include the main electronic dial 104, sub electronic dial 105, video button 106, directional keys 110, SET button 111, AE lock button 112, zoom button 113, playback button 114, menu button 115, touch bar 119, and the like.

[0030] The shutter button 101 has a first shutter switch 230 and a second shutter switch 231. The first shutter switch 230 turns on during the operation of the shutter button 101, so-called half-press (shooting preparation instruction), and outputs a first shutter switch signal SW1. The system control unit 50 starts shooting preparation processing such as AF processing, AE processing, AWB processing, and EF processing in response to the first shutter switch signal SW1. The second shutter switch 231 turns on when the operation of the shutter button 101 is completed, so-called full-press (shooting instruction), and outputs a second shutter switch signal SW2. The system control unit 50 starts a series of shooting processes from reading the signal from the imaging unit 211 to generating an image file containing the captured image and writing it to the recording medium 227 in response to the second shutter switch signal SW2.

[0031] The mode switch 103 switches the operating mode of the system control unit 50 to one of the following: still image shooting mode, video shooting mode, playback mode, etc. Modes included in the still image shooting mode include auto shooting mode, auto scene detection mode, manual mode, aperture priority mode (Av mode), shutter speed priority mode (Tv mode), and program AE mode (P mode). There are also various scene modes and custom modes that provide shooting settings for different shooting scenes. The user can directly switch to any of the above shooting modes using the mode switch 103. Alternatively, the user can switch to the shooting mode list screen using the mode switch 103, and then selectively switch to any of the displayed modes using the operation unit 228. Similarly, the video shooting mode may also include multiple modes.

[0032] The touch panel 109 is a touch sensor that detects various touch operations on the display surface of the display unit 108 (the operating surface of the touch panel 109). The touch panel 109 and the display unit 108 can be configured as an integrated unit. For example, the touch panel 109 is mounted on the upper layer of the display surface of the display unit 108 so that its light transmittance does not interfere with the display of the display unit 108. By associating the input coordinates on the touch panel 109 with the display coordinates on the display surface of the display unit 108, a GUI (Graphical User Interface) can be configured that makes it appear as if the user can directly operate the screen displayed on the display unit 108. The touch panel 109 can use any of the following methods: resistive, capacitive, surface acoustic wave, infrared, electromagnetic induction, image recognition, or optical sensor. Depending on the method, a touch may be detected when there is contact with the touch panel 109, or when a finger or pen approaches the touch panel 109; either method is acceptable.

[0033] The system control unit 50 can detect the following operations or states on the touch panel 109. - A finger or pen that was not previously touching the touch panel 109 now touches the touch panel 109, i.e., the start of a touch (hereinafter referred to as Touch-Down). • The state in which the touch panel 109 is being touched with a finger or pen (hereinafter referred to as Touch-On). • The touch panel 109 is being moved while a finger or pen is touching it (hereinafter referred to as Touch-Move). - The finger or pen that was touching the touch panel 109 is lifted (released), meaning the touch action ends (hereinafter referred to as "Touch-Up"). • The state in which nothing is being touched on the touch panel 109 (hereinafter referred to as Touch-Off).

[0034] When a touchdown is detected, a touch-on is also detected simultaneously. After a touchdown, touch-ons are usually detected continuously unless a touch-up is detected. Touch-ons are also detected simultaneously if a touch-move is detected. Even if a touch-on is detected, a touch-move will not be detected if the touch position has not moved. After all fingers or pens that were touching have been detected as having touched up, a touch-off occurs.

[0035] These operations and states, as well as the position coordinates of the finger or pen touching the touch panel 109, are notified to the system control unit 50 via the internal bus. Based on the notified information, the system control unit 50 determines what kind of operation (touch operation) was performed on the touch panel 109. For touch moves, the direction of movement of the finger or pen moving on the touch panel 109 can also be determined for each vertical and horizontal component on the touch panel 109 based on the change in position coordinates. If a touch move of a predetermined distance or more is detected, it is determined that a slide operation was performed. An operation in which a finger is touched on the touch panel 109 and then quickly moved a certain distance and released is called a flick. In other words, a flick is an operation in which the finger is quickly traced across the touch panel 109 as if flicking it. If a touch move of a predetermined distance or more at a predetermined speed or faster is detected, and a touch-up is then detected, it is determined that a flick was performed (it can be determined that a flick followed a slide operation). Furthermore, touching multiple points (for example, two points) simultaneously (multitouch) to bring them closer together is called pinch-in, and touching them further apart is called pinch-out. Pinch-out and pinch-in are collectively referred to as pinch operations (or simply pinch).

[0036] Figure 4 is a schematic diagram showing an example of the configuration of the lens unit 300. Figure 4 shows the lens unit 300 attached to the camera 100. Note that, among the components of the camera 100 shown in Figure 4, those identical to those described in Figure 3 are denoted by the same reference numerals as in Figure 3, and their descriptions are omitted as appropriate. The lens unit 300 is a type of interchangeable lens that can be attached to and removed from the camera 100. The lens unit 300 is a twin-lens system capable of capturing parallax right and left images. In this embodiment, the lens unit 300 has two optical systems, and each of the two optical systems can capture a wide field of view of approximately 180 degrees. Specifically, each of the two optical systems of the lens unit 300 can capture a subject with a field of view (angle of view) of 180 degrees in the left-right direction (horizontal angle, azimuth angle, yaw angle) and 180 degrees in the up-down direction (vertical angle, elevation / depression angle, pitch angle). In other words, each of the two optical systems can capture an area of ​​the front hemisphere.

[0037] The lens unit 300 includes a right-eye optical system 301R having multiple lenses and a reflective mirror, a left-eye optical system 301L having multiple lenses and a reflective mirror, and a lens system control circuit 303. The right-eye optical system 301R is an example of a first optical system, and the left-eye optical system 301L is an example of a second optical system. The right-eye optical system 301R has a lens 302R positioned on the subject side, and the left-eye optical system 301L has a lens 302L positioned on the subject side. Lenses 302R and 302L face the same direction, and their optical axes are approximately parallel.

[0038] The lens unit 300 is a binocular lens (VR180 lens) for obtaining VR180 images, which are one of the VR (Virtual Reality) image formats that enable binocular stereoscopic viewing. In this embodiment, the lens unit 300 has a fisheye lens capable of capturing a range of approximately 180 degrees in each of the right eye optical system 301R and the left eye optical system 301L. The range that can be captured by the lenses of each of the right eye optical system 301R and the left eye optical system 301L may be narrower than 180 degrees, around 160 degrees. The lens unit 300 can image the right image (first image) formed via the right eye optical system 301R and the left image (second image) formed via the left eye optical system 301L onto one or two image sensors of the camera to which the lens unit 300 is attached.

[0039] The lens unit 300 is attached to the camera 100 via the lens mount portion 304 and the camera mount portion 305 of the camera 100. In this way, the system control unit 50 of the camera 100 and the lens system control circuit 303 of the lens unit 300 are electrically connected via the communication terminal 124 of the camera 100 and the communication terminal 306 of the lens unit 300.

[0040] In this embodiment, the right image formed via the right-eye optical system 301R and the left image formed via the left-eye optical system 301L are simultaneously (as a set) imaged on the imaging unit 211 of the camera 100. That is, two optical images formed by the right-eye optical system 301R and the left-eye optical system 301L are formed on a single image sensor. The imaging unit 211 converts the imaged subject (optical signal) into an analog electrical signal. By using the lens unit 300 in this way, two images with parallax can be simultaneously (as a set) acquired from two locations (optical systems) - the right-eye optical system 301R and the left-eye optical system 301L. By separating the acquired images into an image for the left eye and an image for the right eye and displaying them in VR, the user can view a three-dimensional VR image with a range of approximately 180 degrees. In other words, the user can view a VR180 image in 3D.

[0041] Here, a VR image is an image that can be displayed in VR, as described later. VR images include omnidirectional images (spherical images) captured by an omnidirectional camera (spherical camera), and panoramic images with a wider field of view (effective field of view) than the display area that can be displayed on the display unit at once. Furthermore, VR images are not limited to still images, but also include videos and live images (images acquired from the camera in near real-time). A VR image has a field of view (effective field of view) of up to 360 degrees horizontally and vertically. In addition, VR images also include images with a wider field of view than that that can be captured by a normal camera, or images with a wider field of view than the display area that can be displayed on the display unit at once, even if the field of view is less than 360 degrees horizontally or vertically. The image captured by the camera 100 using the lens unit 300 described above is a type of VR image.

[0042] The HMD154 is equipped with a display device that shows images transmitted by the PC500b. The HMD154 is an example of an image display device worn on the head of the receiver 156. The HMD154 is connected to the PC500b by wireless or wired cable. Images are displayed on the HMD154 based on control from the PC500b. Here, the HMD154 can display VR images in VR by setting the display mode to "VR View". By displaying VR images with a 360-degree field of view in VR, the user can change the orientation of the display device left or right (horizontal rotation direction) to view a seamless, all-around image in the left and right directions.

[0043] VR display (VR view) is a display method (display mode) that allows the display range to be changed, displaying images within a field of view that corresponds to the orientation of the display device. One type of VR display is "single-eye VR display (single-eye VR view)," which displays a single image by performing a transformation (distortion correction) that maps the VR image to a virtual sphere. Another type of VR display is "two-eye VR display (two-eye VR view)," which displays a VR image for the left eye and a VR image for the right eye side by side by performing a transformation that maps each to a virtual sphere. By performing "two-eye VR display" using a VR image for the left eye and a VR image for the right eye that have parallax with each other, it is possible to view these VR images in 3D. In any type of VR display, for example, when a user wears an HMD, the image displayed will correspond to the field of view that corresponds to the orientation of the user's face. For example, suppose that in a VR display, at a certain point in time, an image is displayed with a field of view centered at 0 degrees horizontally (a specific direction, e.g., north) and 90 degrees vertically (90 degrees from the zenith, i.e., horizontal). If the orientation of the display device is reversed from this state (for example, changing the display surface from facing south to facing north), the display range of the same VR image will change to an image with a field of view centered on 180 degrees horizontally (opposite direction, e.g., south) and 90 degrees vertically. In other words, if the user, while wearing the HMD, turns their face from north to south (i.e., turns their back), the image displayed on the HMD will also change from a northern image to a southern image. Note that the VR image captured using the lens unit 300 of this embodiment is an image capturing a range of approximately 180 degrees in front (180° image), and there is no image of a range of approximately 180 degrees behind. If such an image is displayed in VR and the orientation of the display device is changed to the side where there is no image, a blank area will be displayed.

[0044] By displaying VR images in VR in this way, users can visually experience the sensation (immersion) of being inside the VR image (in VR space). Note that the method of displaying VR images is not limited to changing the orientation of the display device. For example, the display range may be moved (scrolled) in response to user operations via the touch panel or directional buttons on the camera 100 or PC 500a or 500b. Furthermore, in VR display mode (display mode "VR View"), in addition to changing the display range due to changes in orientation, the display range may also be changed in response to touch movements on the touch panel, drag operations with a mouse, or pressing of directional buttons on the camera 100 or PC 500a or 500b. Note that smartphones and tablet devices mounted on VR goggles (head-mounted adapters) are a type of HMD (Head-Mounted Display).

[0045] Figure 5 is a block diagram showing an example configuration of the PC500 (500a, 500b). The control unit 501 is, for example, a Central Processing Unit (CPU) and controls the entire PC500. The Read Only Memory (ROM) 502 stores programs and parameters non-temporarily. The Random Access Memory (RAM) 503 temporarily stores programs and data supplied from external devices. The recording medium 504 is a hard disk or flash memory fixed to the PC500, or an optical disc, magnetic card, optical card, IC card, memory card, etc. that can be attached to and removed from the PC500. The operation unit 505 receives user operations on the PC500. The operation unit 505 may be buttons or a touch panel provided on the PC500 used by the user when operating the PC500, or it may be a keyboard or mouse that can be attached to and removed from the PC500. The display unit 506 displays data held by the PC500 or images transmitted from external devices based on control by the control unit 501. Unlike the HMD 154, the display unit 506 is a display device that can be viewed by the user without being worn. Furthermore, the display unit 506 is a two-dimensional display device that displays images in two dimensions. Note that the display unit 506 may be part of the PC 500, or it may be a separate display device from the PC 500. The communication unit 507 communicates with other external devices such as other PCs, the camera 100, or the HMD 154. For example, from the perspective of PC 500a, PC 500b is an external device, and from the perspective of PC 500b, PC 500a is an external device. The system bus 508 connects the components of the PC 500 in a way that enables communication between them.

[0046] Here, we will explain the characteristics of the images captured with the lens unit 300 (two-lens system). In the case of the lens unit 200 (a normal single-lens system), an image that is inverted vertically and horizontally (a 180-degree rotated image) relative to the actual view is formed on the imaging unit 211. Therefore, the entire formed image is rotated 180 degrees to acquire (image) an image that matches the actual view. On the other hand, in the case of the lens unit 300 (two-lens system), the right image and the left image are each rotated 180 degrees relative to the actual view and formed on the imaging unit 211. The arrangement of the right and left images is not particularly limited, but in this embodiment, it is assumed that the right image is formed on the right side and the left image is formed on the left side on the imaging unit 211. Then, as with the lens unit 200 (a normal single-lens system), if the entire formed image (the image including the right and left images) is rotated 180 degrees, the right and left images can be adjusted to match the actual view, but the positions of the right and left images are swapped. In other words, the left-right positional relationship is reversed, resulting in an image where the right image is positioned on the left and the left image is positioned on the right. In this embodiment, the positions of the right and left images can be displayed in a way that corresponds to the two optical systems (right eye optical system 301R and left eye optical system 301L).

[0047] Next, the processing performed by PC500a in the system configured as described above will be explained using the flowchart in Figure 6. The flowchart in Figure 6 is realized when the control unit 501 of PC500a loads the program stored in ROM 502 into RAM 503 and executes it. The flowchart in Figure 6 is started when the camera 100, which is connected to PC500a and PC500b for communication and equipped with a lens unit 300, captures an image, and is executed periodically.

[0048] In S601, the control unit 501 of PC500a acquires the image captured by camera 100. Specifically, the image captured by camera 100 is transferred to PC500a via communication unit 507, and the control unit 501 of PC500a acquires the image by reading the transferred image. The read image is image 701, which includes two circular fisheye images 702a and 702b obtained with a binocular lens, as shown in Figure 7(a). Circular fisheye image 702a is an image captured via the right eye optical system 301R, and circular fisheye image 702b is an image captured via the left eye optical system 301L. Circular fisheye images 702a and 702b are distorted more as they approach the outer edge of the circle.

[0049] In S602, the control unit 501 of PC500a obtains usage information of the HMD154 in the receiving environment 152 from PC500b via the communication unit 507. The usage information of HMD154 refers to information indicating the usage status of HMD154. PC500b has obtained usage information from HMD154, and the control unit 501 of PC500a continuously obtains the usage information of HMD154 via PC500b. Note that the processing order of S601 and S602 may be reversed, and S601 and S602 may be processed in parallel.

[0050] Here, there is a variety of information available regarding the use of the HMD154. As usage information for the HMD154, information indicating whether the HMD154's power is on or off (power on / off signal) can be used. For example, the control unit 501 of PC500a can acquire the usage information for the HMD154 by receiving the power on / off signal from the HMD154 via PC500b. Furthermore, as usage information for the HMD154, information indicating whether or not the recipient, as a user, is wearing the HMD154 (signal data from the accelerometer) can be used. By equipping the HMD154 with an accelerometer, the control unit 501 of the PC500a can receive the signal data from the accelerometer via the PC500b to obtain usage information for the HMD154.

[0051] In S603, the control unit 501 of PC500a determines, based on the acquired usage information of HMD154, whether the HMD154 in the receiving environment 152 is in use, that is, whether the HMD154 is currently in use. If it is in use, the process proceeds to S604; otherwise, it proceeds to S605. For example, if the usage information for the HMD154 is an on / off signal for the HMD154's power supply, the control unit 501 determines that the HMD154 is in use if the power supply is on, and determines that the HMD154 is not in use if the power supply is off. Furthermore, for example, if the usage information of the HMD154 is a signal from an accelerometer installed on the HMD154, the control unit 501 determines whether or not it is in use based on the signal from the accelerometer. That is, when the receiver 156 is wearing the HMD154 on their head, the signal from the accelerometer fluctuates in accordance with the movement of the head. Therefore, as long as the control unit 501 detects the next change within a certain time after detecting a change in the accelerometer signal, it determines that the HMD154 is in use. On the other hand, if the control unit 501 does not detect a change in the accelerometer signal within a certain time, it determines that the HMD154 is not in use.

[0052] In this way, the control unit 501 determines the current usage status of the HMD154 based on the power on / off signal of the HMD154 acquired in S602 and the signal from the acceleration sensor installed on the HMD154. However, the method for determining the usage status of the HMD154 is not limited to the method described above. Furthermore, the control unit 501 of PC500a in the transmitting environment 151 may determine the usage status of HMD154, or the control unit 501 of PC500b in the receiving environment 152 may determine the usage status of HMD154. In this case, the control unit 501 of PC500a can receive the determination result from PC500b and determine the usage status of HMD154 based on the determination result. When the control unit 501 of PC500a receives the determination result from PC500b in this way, the information of the determination result corresponds to the usage information of HMD154.

[0053] In step S604, the control unit 501 performs an image transformation that suppresses image distortion, which is necessary to enable VR display of the image on the HMD 154. The process proceeds to S604 when the HMD 154 is in use, i.e., when the receiver 156 is wearing the HMD 154, so here the necessary or appropriate processing is performed to enable VR display of the image on the HMD 154. Specifically, the control unit 501 applies an equirectangular transformation (first transformation) to each of the two circular fisheye images acquired in S601 to enable "two-eye VR display". Here, Figure 7(b) shows the images after applying the equirectangular transformation to each of the two circular fisheye images 702a and 702b included in image 701 shown in Figure 7(a). The transformed image is image 703, which includes images 704a and 704b as shown in Figure 7(b). Focusing on images 704a and 704b, the distortion of the outer edge is suppressed without the outer edge of the circle being stretched.

[0054] In S605, the control unit 501 does not perform the equirectangular transformation described above, but instead extracts a portion of the two circular fisheye images. The process proceeds to S605 because the HMD 154 is not in use, i.e., the receiver 156 is not wearing the HMD 154; therefore, the equirectangular transformation required to display the image in VR on the HMD 154 is not performed here. Specifically, the control unit 501 extracts only the left circular fisheye region of the two circular fisheye images acquired in S601. By not performing the equirectangular transformation in this way, the processing required to transform the image can be reduced. Here, Figure 7(c) shows the image obtained by extracting the left circular fisheye image 702b from the two circular fisheye images 702a and 702b included in image 701 shown in Figure 7(a). The transformed image is image 705, as shown in Figure 7(c). Note that while this explanation describes how to crop the left-hand circular fisheye region, you only need to crop either the left or right side; you could also crop the right-hand circular fisheye region. Furthermore, the image extracted from one side of the circular fisheye region may be subjected to necessary or appropriate transformations to enable display on a two-dimensional display device. Specifically, the control unit 501 may perform a perspective projection transformation (second transformation) on the image extracted from one side of the circular fisheye region. By performing a perspective projection transformation, distortion at the outer edge of the transformed image is suppressed. The image transformed by perspective projection is image 706 as shown in Figure 7(d), and it looks as if only the central part of the equirectangular transformation has been extracted. Compared to the circular fisheye image and the equirectangular transformation image, it has less distortion and looks natural to the user in the case of two-dimensional display.

[0055] In S606, the control unit 501 transmits the processed image to the receiving environment 152. If the HMD 154 is in use, the image after equirectangular transformation is transmitted in S604. On the other hand, if the HMD 154 is not in use, the image before equirectangular transformation is transmitted, and in S605, only the left circular fisheye region is cropped out of the image.

[0056] In the receiving environment 152, the control unit 501 of PC500b displays the received images on the HMD154 and the display unit 506, respectively. Specifically, the control unit 501 of PC500b displays the received images on the display unit 506 and also transfers them to the HMD154 so that they can be displayed on the HMD154. Here, when the HMD154 is in use, that is, when the HMD154 is worn, the image after equirectangular conversion is displayed on both the HMD154 and the display unit 506. Therefore, the receiver 156 wearing the HMD154 can view the image corresponding to "two-eye VR display" via the HMD154. Although the control unit 501 of the PC500b also displays the image after equirectangular conversion on the display unit 506, the receiver 156 is viewing the image on the HMD154. On the other hand, when the HMD154 is not in use, that is, when the HMD154 is not attached, the image before equirectangular transformation is displayed on the HMD154 and the display unit 506. Therefore, if the receiver 156 is not wearing the HMD154, they can view the image by looking at the display unit 506. Thus, even if the receiver 156 is not wearing the HMD154, they can communicate with the sender 155 in the transmitting environment 151 based on the image displayed on the display unit 506. Furthermore, if the image displayed on the display unit 506 is an image that has been perspective-projected onto only one of the circumferential fisheye regions, the receiver 156 can view an image with suppressed distortion at the outer edge.

[0057] Therefore, according to the process described above, when the receiver 156 transitions from wearing the HMD 154 to not wearing it, the image displayed on the HMD 154 and the display unit 506 switches from the image after equirectangular transformation to the image before equirectangular transformation. Specifically, it switches from image 703 shown in Figure 7(b) to image 705 shown in Figure 7(c). On the other hand, when the receiver 156 transitions from not wearing the HMD 154 to wearing it, the image displayed on the HMD 154 and the display unit 506 switches from the image before equirectangular transformation to the image after equirectangular transformation. Specifically, it switches from image 705 shown in Figure 7(c) to image 703 shown in Figure 7(b).

[0058] However, the control unit 501 of the PC500b may not display the image after equirectangular conversion on the display unit 506 when the receiver 156 is wearing the HMD 154, but may display it only on the HMD 156. Also, the control unit 501 of the PC500b may not display the image before equirectangular conversion on the HMD 156 when the receiver 156 is not wearing the HMD 154, but may display it only on the display unit 506.

[0059] As described above, according to this embodiment, since the image before or after conversion is transmitted according to information indicating the usage status of the HMD154, the processing of image conversion can be reduced. Specifically, when the HMD154 is not in use, only a portion is cropped without equirectangular conversion, thus reducing the processing of equirectangular conversion. Furthermore, even when the HMD154 is not in use and only a portion of the image is cropped without equirectangular conversion and then subjected to perspective projection conversion, the processing can be reduced because only a portion of the image is converted. In addition, since only the necessary images can be transmitted, the transmission of unnecessary images can be reduced, reducing processing and suppressing the amount of data transmitted.

[0060] <Second Embodiment> In this embodiment, we will describe the case in which an image of the receiver is used as the usage information for the HMD154. Note that reference numerals identical to those in the first embodiment are assumed to perform the same processing as in the first embodiment, and their explanations will be omitted as appropriate. Figure 8 is a schematic diagram showing an example of the overall configuration of the system according to the second embodiment. In this embodiment, the receiving environment 152 includes a camera 800 that images the receiver 156 wearing the HMD 154. The image captured by the camera 800 is input to the PC 500b and transmitted to the PC 500a of the transmitting environment 151. In the transmitting environment 151, the image captured by the camera 800 is displayed in real time on the display unit 506 provided on the PC 500a. In this embodiment, the image captured by the camera 800 is used as usage information for the HMD 154.

[0061] Next, the processing performed by PC500a in the system configured as described above will be explained using the flowchart in Figure 9. Note that in the flowchart in Figure 9, processes similar to those in the flowchart in Figure 6 will be given the same step numbers and their explanations will be omitted as appropriate. In this embodiment, S901 and S902 are added between S601 and S603.

[0062] In S901, the control unit 501 of PC500a acquires the image from the camera 800 that is imaging the receiver 156 by receiving it via the communication unit 507. Here, the image being captured by the camera 800 is taken into PC500b, and the image taken into PC500b is transmitted to PC500a, so the control unit 501 of PC500a acquires the image from the camera 800. The control unit 501 of PC500a also displays the received image on the display unit 506 provided on PC500a.

[0063] In S902, the control unit 501 of the PC500a performs image recognition processing on the image captured by the camera 800. Note that S901 and S902 may be processed in parallel with S601.

[0064] In S603, the control unit 501 of PC500a uses the image captured by camera 800 as usage information for HMD154 to determine whether or not the HMD154 in the receiving environment 152 is in use. Specifically, the control unit 501 determines whether or not the HMD154 is in use based on the results of image recognition processing of the image captured by camera 800.

[0065] For example, the control unit 501 performs image recognition to determine whether or not the HMD154 is present. If the control unit 501 can recognize the presence of the HMD154, it determines that the HMD154 is in use. If the control unit 501 cannot recognize the presence of the HMD154, it determines that the HMD154 is not in use. Furthermore, for example, the control unit 501 performs image recognition of a human eye, and if it cannot detect the eye, it determines that the HMD 154 is in use; if it can detect the eye, it determines that the HMD 154 is not in use.

[0066] If HMD154 is in use, proceed to S604; otherwise, proceed to S605. The subsequent processing from S604 to S606 is the same as in the first embodiment and will not be explained.

[0067] As described above, according to this embodiment, the current usage status of the HMD 154 is determined based on the image from the camera 800 that is capturing images of the receiver 156. Therefore, the usage status of the HMD 154 can be determined with high accuracy. The camera 800 may be provided together with the PC 500b, or it may be a separate unit from the PC 500b.

[0068] <Other Embodiments> The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or recording medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0069] Although the present invention has been described above with reference to various embodiments, the present invention is not limited to these embodiments, and modifications can be made within the scope of the present invention. In this embodiment, the case of equirectangular transformation was described as a transformation that suppresses image distortion, but the invention is not limited to this case, and other transformations may also be performed. [Explanation of symbols]

[0070] 100: Camera (imaging device) 154: HMD (head-mounted display device) 156: Receiver 500a: PC (information processing device) 501: Control unit 507: Communication unit

Claims

1. An information processing device that transmits an image to an image display device worn by a recipient during a video call, An acquisition means for acquiring information indicating that the recipient is wearing an image display device on their head, based on an image of the recipient, If the acquisition means obtains information indicating that the recipient is wearing the image display device on their head, the transmission means transmits a first image captured via a first optical system and a second image captured via a second optical system; if the acquisition means does not obtain information indicating that the recipient is wearing the image display device on their head, the transmission means transmits one of the first image and the second image. An information processing device characterized by having the following features.

2. Having a conversion means for converting the image in such a way that distortion is suppressed, The aforementioned transmission means is The information processing apparatus according to claim 1, characterized in that if information indicating that the recipient is wearing the image display device on their head has not been acquired by the acquisition means, the image before it is converted by the conversion means is transmitted.

3. Having a conversion means for converting the image in such a way that distortion is suppressed, The aforementioned transmission means is The information processing apparatus according to claim 1, characterized in that, if information indicating that the recipient is wearing the image display device on their head has not been acquired by the acquisition means, the apparatus transmits the image before it is converted by the conversion means to be displayed on a display device that can be viewed by the recipient without them wearing the device.

4. Having a conversion means for converting the image in such a way that distortion is suppressed, The aforementioned transmission means is The information processing apparatus according to claim 1, characterized in that if information indicating that the recipient is wearing the image display device on their head has not been acquired by the acquisition means, a portion of the image before it is converted by the conversion means is transmitted.

5. Having a conversion means for converting the image in such a way that distortion is suppressed, The aforementioned transmission means is The information processing apparatus according to claim 1, wherein if the acquisition means has not acquired information indicating that the recipient is wearing the image display device on their head, the apparatus transmits one of the first image and the second image, which is the image before it is converted by the conversion means.

6. Having a conversion means for converting the image in such a way that distortion is suppressed, The aforementioned transmission means is The information processing apparatus according to claim 1, characterized in that, when information indicating that the recipient is wearing the image display device on their head is acquired by the acquisition means, the image after conversion by the conversion means is transmitted.

7. The aforementioned image is an image captured through a fisheye lens, The information processing apparatus according to any one of claims 2 to 6, characterized in that the conversion means performs equirectangular transformation on the captured image.

8. If the aforementioned conversion means is designated as the first conversion means, The system has a second conversion means that converts the image in a manner different from the first conversion means, The aforementioned transmission means is The information processing apparatus according to any one of claims 2 to 7, characterized in that, if information indicating that the recipient is wearing the image display device on their head has not been acquired by the acquisition means, an image that has been converted by the second conversion means but not converted by the first conversion means is transmitted.

9. The information processing apparatus according to claim 8, characterized in that the second conversion means performs perspective projection conversion on the image.

10. The information processing apparatus according to any one of claims 1 to 9, characterized in that the information indicating that the recipient is wearing the image display device on their head is information indicating whether the power of the image display device is on or off.

11. The information processing apparatus according to any one of claims 1 to 9, characterized in that the information indicating that the recipient is wearing the image display device on their head is information indicating the state of wearing the image display device.

12. The information processing apparatus according to any one of claims 1 to 9, characterized in that the information indicating that the recipient is wearing the image display device on their head is an image of the recipient.

13. The aforementioned transmission means is The information processing apparatus according to any one of claims 2 to 9, characterized in that the image converted by the conversion means is transmitted to the image display device via an external device that controls the image display device.

14. The information processing apparatus according to any one of claims 1 to 13, characterized in that the image display device is a display device that is attached to the user's head.

15. A method for controlling an information processing device that transmits an image to an image display device worn by a recipient during a video call, An acquisition step of acquiring information indicating that the recipient is wearing an image display device on their head, based on an image of the recipient, If information indicating that the recipient is wearing the image display device on their head is obtained by the acquisition step, the transmission step transmits a first image captured via the first optical system and a second image captured via the second optical system; if information indicating that the recipient is wearing the image display device on their head is not obtained by the acquisition step, the transmission step transmits one of the first image and the second image. A control method characterized by having the following features.

16. A program for causing a computer to function as one of the means of an information processing device described in any one of claims 1 to 14.

17. A video call system comprising: an information processing device for transmitting an image captured by a sender to a display control device that displays the image on an image display device worn on the head of a receiver or a display device that is visible to the receiver without being worn; the display control device; the image display device; and the display device, The aforementioned information processing device is A first acquisition means for acquiring information from the display control device indicating that the recipient is wearing the image display device on their head, The system includes a first transmission means that, if first information indicating that the recipient is wearing the image display device on their head is acquired by the first acquisition means, transmits a first image captured via the first optical system and a second image captured via the second optical system to the display control device; and if the first information is not acquired by the first acquisition means, transmits the first image to the display control device and does not transmit the second image to the display control device. The aforementioned display control device is A second acquisition means for acquiring the first information based on an image of the receiver, When the first information is acquired by the second acquisition means, the second transmission means transmits the first information to the information processing device, The system includes a display control means that controls the display unit to display the image transmitted by the first transmission means, The display control means is When the first transmission means transmits the first image and the second image, the first image and the second image are controlled to be displayed on the image display device. If the first image is transmitted by the first transmission means and the second image is not transmitted, the first image is controlled to be displayed on the display device. When the first image and the second image are displayed on the image display device, the first image and the second image are not displayed on the display device. When the first image is displayed on the display device, the first image is not displayed on the image display device. A video call system characterized by the following features.

Images