Information processing apparatus, video conference system, and control method

Abstract

To communicate information to a particular person while a video conference is in progress in a manner of not allowing other persons to notice the communication.SOLUTION: An information processing apparatus includes: a plurality of display pixels arranged in a matrix; and a plurality of liquid crystal lenses arranged to overlap the plurality of display pixels, and controls display of a display capable of switching between a planar view display and a stereoscopic view display by adjusting a direction of light emitted from the plurality of display pixels by controlling an orientation state of a liquid crystal of each liquid crystal lens. The information processing apparatus includes: an information acquisition unit configured to acquire information to be stereoscopically displayed; and a display control unit configured to: control the display on the planar view display; and, when the information acquisition unit acquires the information to be stereoscopically displayed, stereoscopically display the acquired information in a part of a screen region of the display so as to be stereoscopically visible with respect to a designated position.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a video conference system, and a control method. [Background technology]

[0002] Video conferencing, such as web conferencing, has become widely used in the market because it allows conferences to be held online in real time via a network without the need to travel far. For example, a video conference is held between multiple participants (attendees) in a conference room where a video conferencing device running a video conferencing application is installed, and participants in other locations connected via a network, and video and audio data of each participant are transmitted and received via the video conferencing device (for example, see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-61594 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in a video conference, there are cases where you want to convey information only to a specific person among multiple participants while the conference is in progress. Currently, the only person who can convey information to a specific person without the other people knowing is the person sitting next to you, and it has been difficult to convey information to a specific person without the other people knowing while the video conference is in progress, including people participating via video.

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and one of its objects is to provide an information processing device, a video conferencing system, and a control method that can convey information to a specific person while a video conference is in progress without other people knowing. [Means for solving the problem]

[0006] The present invention has been made to solve the above-mentioned problems, and an information processing device according to a first aspect of the present invention is an information processing device that controls the display of a display that has a plurality of display pixels arranged in a matrix and a plurality of liquid crystal lenses arranged on top of a plurality of the display pixels, and that can switch between planar display and stereoscopic display by adjusting the direction of light emitted from the plurality of display pixels by controlling the orientation state of the liquid crystal of the liquid crystal lenses, and that has an information acquisition unit that acquires information to be displayed in stereoscopic view, and a display control unit that controls the display to the planar display, and when the information acquisition unit acquires information to be displayed in stereoscopic view, displays the acquired information in a part of the screen area of ​​the display so that it can be viewed stereoscopically at a specified position.

[0007] The information processing device further includes a position information acquisition unit that acquires detection information of the eye position of a person present on the side facing the display, detected from an image captured of the side facing the display, wherein the information acquisition unit acquires the information to be stereoscopically displayed in association with identification information of the person to whom the information is to be conveyed, and when the display control unit causes the information acquired by the information acquisition unit to be stereoscopically displayed so as to be stereoscopically visible at the specified position, the display control unit may cause the information to be stereoscopically displayed so as to be stereoscopically visible at the eye position of the person to whom the information is to be conveyed, based on the identification information of the person acquired by the information acquisition unit and detection information of the eye position of the person acquired by the position information acquisition unit.

[0008] In the information processing device, the display control unit may determine a position of the partial screen area of ​​the screen area of ​​the display that is controlled to be displayed stereoscopically, according to the specified position.

[0009] In the information processing device, the display control unit may determine the position of the portion of the screen area of ​​the display that is controlled to be displayed stereoscopically, depending on the position within the screen area of ​​the content displayed on the display.

[0010] In the above-mentioned information processing device, when the information acquisition unit acquires multiple pieces of information that the information acquisition unit wants to convey to each of multiple people, the display control unit may display the multiple pieces of information acquired by the information acquisition unit in a stereoscopic manner in each of the multiple partial screen areas so that the pieces of information can be viewed stereoscopically at specified positions.

[0011] In the information processing device, the display control unit may control the partial screen area to the stereoscopic display and the other screen area to the planar display by making the orientation state of the liquid crystal lens different between the partial screen area and the other screen area.

[0012] In the information processing device, the display control unit may alternately switch the display of the screen area of ​​the display between the planar display and the stereoscopic display.

[0013] In the information processing device, the display control unit may control the display on the display in an interlaced format, and alternately switch between the monoscopic display and the stereoscopic display between odd-numbered fields and even-numbered fields in one frame.

[0014] In the above information processing device, when performing the stereoscopic display, the display control unit may divide the information acquired by the information acquisition unit into information to be displayed as an image for the right eye and information to be displayed as an image for the left eye and display the information.

[0015] The information processing device further includes a transmission unit that transmits an image captured by an imaging device that captures an image including a person present on the side facing the display to another information processing device that is connected to the device for communication, and the information acquisition unit may acquire, in the other information processing device that receives the image transmitted from the transmission unit, identification information of a person selected from the people captured in the image and information to be displayed in stereoscopic vision that is to be conveyed to the person.

[0016] According to a second aspect of the present invention, there is provided a video conferencing system including a first information processing device that controls display of a first display, the first display including a plurality of display pixels arranged in a matrix and a plurality of liquid crystal lenses arranged overlying a plurality of the display pixels, and that is capable of switching between planar display and stereoscopic display by adjusting a direction of light emitted from the plurality of display pixels by controlling an orientation state of liquid crystal in the liquid crystal lenses, and a second information processing device that is communicatively connected to the first information processing device and is capable of conducting online video conferencing, the first information processing device including: a position information acquisition unit that acquires detection information of eye positions of a person present on the side facing the first display, detected from a captured image of the side facing the first display; a first transmission unit that transmits the captured image, which is captured by an imaging device that captures an image including the person present on the side facing the first display, to the second information processing device; an information acquisition unit that acquires information for the stereoscopic display from the second information processing device; and a second information processing device that controls the first display to the planar display, and when the information acquisition unit acquires the information for the stereoscopic display, transmits the acquired information to the second information processing device. and a first display control unit that causes the stereoscopic display to be stereoscopically visible at a specified position in a portion of the screen area of ​​the first display, and the second information processing device comprises: a second display control unit that causes the captured image transmitted from the first transmission unit to be displayed on the second display; a reception unit that receives a selection of a person to whom information is to be communicated from among the people captured in the captured image displayed on the second display and an input of the information to be communicated to the person; and a second transmission unit that associates and transmits the person's identification information received by the reception unit with the information to be communicated to the person, and the information acquisition unit of the first information processing device acquires the person's identification information and the information to be communicated to the person transmitted from the second transmission unit of the second information processing device as the information to be stereoscopically displayed and the identification information of the person to whom the information is to be communicated, and the first display control unit of the first information processing device, when causing the information acquired by the information acquisition unit to be stereoscopically displayed so as to be stereoscopically visible at a specified position, causes the stereoscopic display to be stereoscopically visible at the eye position of the person to whom the information is to be communicated, based on detection information of the person's eye position acquired by the position information acquisition unit.

[0017] Furthermore, according to a third aspect of the present invention, there is provided an information processing device that controls display of a display that is capable of switching between planar display and stereoscopic display by adjusting the direction of light emitted from the display pixels by controlling the orientation state of the liquid crystal of the liquid crystal lenses, the information processing device comprising: a control unit that controls the display to the planar display; and a control unit that controls the display to the stereoscopic display, and when the control unit has acquired the information to display the stereoscopic display, displays the acquired information in a part of the screen area of ​​the display so as to be stereoscopically visible from a specified position.

[0018] Furthermore, according to a fourth aspect of the present invention, there is provided a video conference system including a first information processing device that controls display of a first display, the first display including a plurality of display pixels arranged in a matrix and a plurality of liquid crystal lenses arranged so as to overlap a plurality of the display pixels, and that is capable of switching between planar display and stereoscopic display by adjusting a direction of light emitted from the plurality of display pixels by controlling an orientation state of liquid crystal of the liquid crystal lenses, and a second information processing device that is communicatively connected to the first information processing device and is capable of conducting online video conferences, the video conference system including a first information processing device and a second information processing device, the first information processing device and a second information processing device, the video conference system including a ... a receiving step of receiving a selection of a person to whom information is to be communicated from among the people captured in the captured image displayed on the first information processing device, and an input of the information to be communicated to the person; a second transmitting step of transmitting the person's identification information received in the receiving step and the information to be communicated to the person in association with each other; an acquiring step of a control unit in the first information processing device acquiring, from the second information processing device, the person's identification information and the information to be communicated to the person transmitted in the second transmitting step as information to be stereoscopically displayed; and a first display control step of controlling the first display to the planar display, and, when the information to be stereoscopically displayed is acquired in the acquiring step, stereoscopically displaying the acquired information in a part of a screen region of the first display so as to be stereoscopically visible from a designated position, wherein, in the first display control step, the control unit in the first information processing device, when stereoscopically displaying the information acquired in the acquiring step so as to be stereoscopically visible from the designated position, performs the following steps:The stereoscopic display is performed so that the information can be viewed stereoscopically at the eye position of the person to whom the information is to be conveyed. [Effects of the Invention]

[0019] According to the above aspect of the present invention, it is possible to convey information to a specific person while a video conference is in progress without the other people knowing about it. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a schematic configuration diagram showing an example of a video conference system according to a first embodiment. [Figure 2] Schematic diagram showing an example of the configuration of a display dedicated to 3D display. [Figure 3] FIG. 10 is a schematic diagram showing an example of 3D display when the position where stereoscopic viewing is possible is shifted to the right. [Figure 4] FIG. 10 is a schematic diagram showing an example of 3D display when the position where stereoscopic viewing is possible is shifted to the left. [Figure 5] 5A and 5B are schematic diagrams showing an example of a method for shifting an image to be displayed on a display pixel. [Figure 6] FIG. 2 is a schematic diagram showing an example of the configuration of a display during 3D display according to the first embodiment. [Figure 7] FIG. 2 is a schematic diagram showing an example of the configuration of a display during 2D display according to the first embodiment. [Figure 8] 3A and 3B are diagrams showing a display example in which only a part of the screen area of ​​the display according to the first embodiment is controlled to be displayed in 3D. [Figure 9] FIG. 2 is a diagram showing a first example of a message display using 3D display according to the first embodiment. [Figure 10] FIG. 10 is a view showing a second example of a message display using 3D display according to the first embodiment. [Figure 11] 3A and 3B are diagrams showing examples of a message to be displayed in 3D according to the first embodiment, divided into a message for the right eye and a message for the left eye. [Figure 12] FIG. 1 is a block diagram showing an example of the configuration of an information processing apparatus according to a first embodiment. [Figure 13]FIG. 3 is a diagram showing an example of a message receiving method according to the first embodiment. [Figure 14] 10 is a flowchart showing an example of a message transmission side process in a 3D message notification process according to the first embodiment. [Figure 15] 10 is a flowchart showing an example of a message display process in the 3D message notification process according to the first embodiment. [Figure 16] FIG. 10 is a diagram showing an example of a message display using 3D display according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0021] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. First Embodiment First, an overview of the video conference system according to this embodiment will be described. [Video conferencing system configuration] Fig. 1 is a schematic diagram showing an example of a video conference system according to this embodiment. The illustrated video conference system SYS is a system for connecting multiple locations via a communication network NW such as the Internet to hold online video conferences in real time. Fig. 1 shows an example in which two locations, location A and location B, are connected and a video conference is held by bidirectionally communicating images and audio.

[0022] Each location has the basic configuration of the video conferencing system SYS, which includes an information processing device 10, a display 20 (display device), a 2D camera 30 and a 3D camera 40 that capture images of the side facing the display 20, and an eye camera 50 for detecting the eye positions of people participating in the video conference.

[0023] The information processing device 10 is, for example, a notebook (clamshell) PC (Personal Computer) or a desktop PC. The information processing device 10 realizes the video conferencing function by executing a program that performs processing for a video conferencing system. Hereinafter, the program (application program) that performs the video conferencing processing is referred to as a "video conferencing application." Note that the information processing device 10 is not limited to a general-purpose PC, and may also be a dedicated information processing device with video conferencing functions.

[0024] The display 20 is a relatively large display device on which participants in a video conference at each location can display images of participants in the video conference at other locations. For example, the display 20 may include a liquid crystal display (LCD) or an organic electroluminescence (EL) display. Since the display 20 is a relatively large display device, it is not limited to being placed on a desk, but may also be installed on a wall or the like. The display 20 is a display device that can switch between 2D display (planar display) and 3D display (stereoscopic display). The configuration for switching between 2D display and 3D display will be described later.

[0025] The 2D camera 30 and the 3D camera 40 are imaging devices for capturing images of the side facing the display 20, and are provided to capture images of people participating in the video conference. That is, the 2D camera 30 and the 3D camera 40 capture images including people present on the side facing the display 20. The 2D camera 30 is an imaging device for capturing 2D images, and is installed, for example, on the top of the display 20. On the other hand, the 3D camera 40 is a set of two imaging devices, one for capturing an image for the right eye and one for capturing an image for the left eye, and is installed, for example, on the top of the display 20 to capture 3D images.

[0026] The eye camera 50 is an imaging device for detecting the eye positions of people present on the side facing the display 20 (i.e., people participating in the video conference). For example, two eye cameras 50 are provided, one for each of the two 3D cameras 40. The eye cameras 50 detect the facial area where the face is captured and the eye positions from the captured image. For example, the eye camera 50 detects the facial area and eye positions from the shapes and positional relationships of the eyes, nose, mouth, and other components of the face from the captured image. For example, the eye camera 50 outputs position coordinates indicating the eye positions for each detected facial area (i.e., person).

[0027] Here, the position coordinates of the eyes detected by the eye camera 50 are correlated with the position coordinates on the image captured by the 2D camera 30 or the position coordinates on the image captured by the 3D camera 40. The position coordinates of the eyes detected by the eye camera 50 may also include information on the distance from the eye camera 50 (distance from the display 20). For example, the position coordinates of the eyes detected by the eye camera 50 are output as three-dimensional position coordinates relative to the screen of the display 20.

[0028] 1, when distinguishing the devices installed at site A from the devices installed at site B, they are referred to as information processing device 10A, display 20A, 2D camera 30A, 3D camera 40A, and eye camera 50A. Similarly, when distinguishing the devices installed at site B from the devices installed at site A, they are referred to as information processing device 10B, display 20B, 2D camera 30B, 3D camera 40B, and eye camera 50B.

[0029] The information processing device 10A at site A and the information processing device 10B at site B execute a video conference application to perform video conference processing by bidirectionally communicating images and audio via the communication network NW. Here, the people participating in the video conference at site A are person A1, person A2, person A3, and person A4, and the people participating in the video conference at site B are person B1, person B2, person B3, and person B4.

[0030] In processing a video conference, the information processing device 10A transmits 2D or 3D images captured using the 2D camera 30A or 3D camera 40A to the information processing device 10B via the communication network NW. The information processing device 10B displays the 2D or 3D images received from the information processing device 10A in 2D or 3D on the display 20B. The display 20B displays an image including person A1, person A2, person A3, and person A4 participating in the video conference at location A in 2D or 3D. For example, when displaying 3D images on the display 20B, the information processing device 10B can also display the images in 3D so that the images can be viewed stereoscopically from the eye positions detected by the eye camera 50B.

[0031] In processing a video conference, the information processing device 10B transmits 2D or 3D images captured using the 2D camera 30B or 3D camera 40B to the information processing device 10A via the communication network NW. The information processing device 10A displays the 2D or 3D images received from the information processing device 10B in 2D or 3D on the display 20A. The display 20A displays an image including person B1, person B2, person B3, and person B4 participating in the video conference at location B in 2D or 3D. For example, when displaying 3D images on the display 20A, the information processing device 10A can also display the images in 3D so that the images can be viewed stereoscopically from the eye positions detected by the eye camera 50A.

[0032] [Display Configuration] Next, the configuration of a display 20 that can switch between 2D display and 3D display will be described. First, for reference, the configuration and principles of 3D display will be described using a conventional display dedicated to 3D display as an example, with reference to Figures 2 to 5. Figure 2 is a schematic diagram showing an example of the configuration of a display dedicated to 3D display, showing a portion of the cross section. A plurality of lenticular lenses are arranged over a plurality of display pixels arranged in a matrix. Through the lenticular lenses, only the image for the left eye is seen by the left eye, and only the image for the right eye is seen by the right eye, resulting in a 3D display that allows stereoscopic viewing.

[0033] In 3D displays using lenticular lenses, stereoscopic vision is only possible when the angle of the light passing through the lenticular lens matches the position of the eyes, so the positions from which stereoscopic vision is possible are limited. For example, in the schematic diagram shown in Figure 2, stereoscopic vision is possible when the eyes are positioned approximately in the center of the display screen. If the eyes are positioned away from the center, the angle of the light passing through the lenticular lens and the position of the eyes will no longer match, and not only will stereoscopic vision be impossible, but the viewer will also be unable to recognize what is being displayed. The direction from which stereoscopic vision is possible can be changed by shifting (moving) the image displayed on the display pixels relative to the lenticular lens.

[0034] Fig. 3 is a schematic diagram showing an example of 3D display when the position at which stereoscopic viewing is possible is shifted to the right. As shown in Fig. 3, when the image displayed on the display pixels is shifted to the left relative to the lenticular lens, the position at which stereoscopic viewing is possible shifts to the right. Fig. 4 is a schematic diagram showing an example of 3D display when the position at which stereoscopic viewing is possible is shifted to the left. As shown in Fig. 4, when the image displayed on the display pixels is shifted to the right relative to the lenticular lens, the position at which stereoscopic viewing is possible shifts to the left.

[0035] Next, a method for shifting an image displayed on display pixels relative to a lenticular lens will be described. FIG. 5 is a schematic diagram showing an example of a method for shifting an image displayed on display pixels. (A) of FIG. 5 shows a part of a cross section of a display, similar to the example shown in FIG. 3. (B) of FIG. 5 shows the relationship between the arrangement of display pixels and lenticular lenses when viewed from the display surface. A plurality of lenticular lenses are arranged diagonally relative to the arrangement of a plurality of display pixels arranged in a matrix. Therefore, for example, by shifting the image displayed on display pixels in the vertical direction, the image can be shifted left and right. In other words, by shifting the image displayed on display pixels in the vertical direction, the position where stereoscopic viewing is possible can be shifted left and right.

[0036] Next, a method for switching between 2D display and 3D display on the display 20 in this embodiment will be described with reference to Figs. 6 and 7. Fig. 6 is a schematic diagram showing an example of the configuration of a display during 3D display according to this embodiment. Fig. 6 shows a portion of a cross section of the display 20 during 3D display. The display 20 includes a plurality of display pixels arranged in a matrix and a plurality of liquid crystal lenses arranged to overlap the plurality of display pixels. The liquid crystal lenses include liquid crystals and optical lenses. During 3D display, a right-eye image and a left-eye image captured by a 3D camera 40 are displayed on the plurality of display pixels.

[0037] The display 20 can switch between 2D and 3D display by adjusting the direction of light emitted from multiple display pixels by controlling the orientation of the liquid crystals in the liquid crystal lens. When an AC voltage is applied to the liquid crystals in the liquid crystal lens, the liquid crystals are aligned by electric field lines, and light is bent along the aligned liquid crystals, creating the same effect as a lenticular lens. As a result, only the left-eye image is seen by the left eye, and only the right-eye image is seen by the right eye, resulting in a stereoscopic 3D display. Furthermore, as described with reference to FIG. 5, by vertically shifting the image displayed on the display pixels, the position at which stereoscopic viewing is possible can also be shifted horizontally in the 3D display of the display 20 according to this embodiment.

[0038] Fig. 7 is a schematic diagram showing an example of the configuration of a display during 2D display according to this embodiment. Fig. 7 shows a portion of the cross section of the display 20 similar to Fig. 6, but differs in that during 2D display, a 2D image captured by the 2D camera 30 is displayed on multiple display pixels and no AC voltage is applied to the liquid crystal. When no AC voltage is applied to the liquid crystal, the liquid crystal lens aligns the liquid crystal in a fixed horizontal direction relative to the screen of the display 20, so that light transmitted perpendicular to the screen of the display 20 is not bent and is transmitted as is. This results in 2D display.

[0039] Furthermore, the information processing device 10 according to this embodiment is capable of performing both 2D display and 3D display. For example, the information processing device 10 can control only a portion of the screen area of ​​the display 20 to display 3D, and control the remaining screen areas to display 2D. For example, the electrodes of the multiple liquid crystal lenses arranged on the display 20 are segmented so that an AC voltage can be applied only to the liquid crystal lenses in a portion of the screen area. The unit of the segments can be arbitrarily determined as a specification of the display 20. The information processing device 10 controls only a portion of the screen area to display 3D by applying an AC voltage only to the liquid crystal lenses in a portion of the screen area and displaying 3D images (images for the right eye and images for the left eye) on the display pixels. Furthermore, the information processing device 10 controls the screen areas other than the one controlled for 3D display to display 2D by not applying an AC voltage to the liquid crystal lenses and displaying 2D images on the display pixels.

[0040] FIG. 8 is a diagram showing a display example in which only a portion of the screen area of ​​the display according to this embodiment is controlled to 3D display. Image area G1 shown in the figure indicates the area of ​​the entire image displayed on the desktop screen of a PC. Within image area G1, image area G2 is an area for 2D image content, and image area G3 is an area for 3D image content. When displaying this image on display 20, information processing device 10 controls only the screen area of ​​display 20 in which the 3D image (the image in image area G3) is displayed to 3D display, and controls the other screen areas to 2D display. This allows information processing device 10 to display only the 3D image in a stereoscopically viewable manner, and to display the other screen areas in 2D display without degrading the resolution.

[0041] Furthermore, the information processing device 10 can convey information to only specific people by using control to display only a part of this screen area in 3D, and displaying the information in 3D so that it can be viewed stereoscopically at a specified position, even if there are multiple people facing the display 20. For example, the information processing device 10 displays the information in 3D so that it can be viewed stereoscopically at the eye position of the person to whom the information is to be conveyed, based on the eye position of the person detected by the eye camera 50 (see FIG. 1).

[0042] Information that is to be conveyed only to specific people is visual information such as text, pictures such as stamps or illustrations, or photographs, and is a so-called message from a sender (the sender) to another receiver (the recipient). In the following, we will explain a display example, configuration, and processing for conveying (sending) a message from site A to a specific person at site B, or conveying (sending) a message from site B to a specific person at site A, using control to display only a portion of the screen area of ​​display 20 in 3D in the video conference system SYS shown in Figure 1.

[0043] With reference to FIG. 9, a display example using 3D display to convey a message only to a specific person will be described. FIG. 9 is a diagram showing a first example of a message display using 3D display according to this embodiment. Of the image area G1 shown in the figure, image area G3 contains a message to be conveyed to a specific person α (Person-α) as 3D image content. Image area G2 is an area for 2D image content. When displaying this image on display 20, information processing device 10 detects the position of the eyes of specific person α (Person-α) and controls only the screen area of ​​display 20 where the 3D image (image in image area G3) is displayed to be displayed in 3D so that it can be viewed stereoscopically from the position of the eyes of specific person α (Person-α). Furthermore, information processing device 10 controls the other screen areas to be displayed in 2D.

[0044] This allows the information processing device 10 to display on the display 20 a message that can only be seen by a specific person α (Person-α). That is, the information processing device 10 can convey a message to only the specific person by utilizing the 3D display of the display 20 so that the message cannot be seen by other people. Meanwhile, the screen area other than the 3D display area of ​​the display 20 is a 2D display. Therefore, the 2D image displayed in the 2D display screen area can be seen not only by the specific person α (Person-α) but also by other persons β (for example, Person-β).

[0045] 9 is an example in which a message that can only be seen by one person is displayed, but the information processing device 10 may also display a message that can only be seen by each of a plurality of people. A display example in which a message that can only be seen by each of a plurality of people is displayed using 3D display will be described with reference to FIG. 10. FIG. 10 is a diagram showing a second example of a message display using 3D display according to this embodiment.

[0046] Of the image area G1 shown in the figure, image area G3a and image area G3b contain messages to be conveyed to specific people as 3D image content. The message contained in image area G3a is a message to be conveyed to person α (Person-α), and the message contained in image area G3b is a message to be conveyed to person β (Person-β). Image area G2 is an area for 2D image content.

[0047] When displaying this image on the display 20, the information processing device 10 detects the eye position of person α (Person-α) and controls the screen region of the display 20 where the 3D image of image region G3a is displayed to be displayed in 3D, allowing stereoscopic viewing from the eye position of person α (Person-α). The information processing device 10 also detects the eye position of person β (Person-β) and controls the screen region of the display 20 where the 3D image of image region G3b is displayed to be displayed in 3D, allowing stereoscopic viewing from the eye position of person β (Person-β). The information processing device 10 also controls the other screen regions to be displayed in 2D.

[0048] This allows the information processing device 10 to display a message that can only be seen by each of the multiple people on the display 20. That is, the information processing device 10 can convey a message to each of the multiple people in a way that the other people cannot see, by utilizing the 3D display of the display 20. On the other hand, the screen area other than the 3D display area on the display 20 is displayed in 2D.

[0049] Furthermore, when controlling a portion of the screen area of ​​the display 20 to be displayed in 3D, the information processing device 10 may determine the position of the portion of the screen area controlled to be displayed in 3D according to the position of a person to whom a message is to be conveyed. For example, as shown in Fig. 10, the information processing device 10 may display a message for person α (Person-α) who is on the right side of the screen area of ​​the display 20 on the right side. On the other hand, the information processing device 10 may display a message for person β (Person-β) who is on the left side of the screen area of ​​the display 20 on the left side. If the position of the screen area to be displayed in 3D is far from the position of the person's eyes, it may be difficult to ensure visibility through 3D display, so it is preferable to position the screen area to be displayed in 3D close to the position of the person's eyes.

[0050] Furthermore, when a message is displayed in 3D so that it can be viewed stereoscopically only by a specific person, the message may be displayed divided into information to be displayed as an image for the right eye and information to be displayed as an image for the left eye.

[0051] Fig. 11 is a diagram showing a display example in which a message to be displayed in 3D according to this embodiment is divided into those for the right eye and those for the left eye. In the example shown in Fig. 11, the message to be displayed in 3D, "Eye tracking 3D is key technology for immersive experience," is alternately extracted and divided into those for the right eye and those for the left eye, character by character. Specifically, the message to be displayed as an image for the right eye is "ytakn 3 I kyehooyoimr I exe I ne," and the message to be displayed as an image for the left eye is "E erc I g D setcnlgfrmesveprec."

[0052] By dividing the message into right and left eye segments in this way, only those in a position where the 3D displayed message can be viewed stereoscopically can see the right eye image with their right eye and the left eye image with their left eye, allowing them to view the message displayed in 3D. For those in other positions, the characters are completely invisible, or they cannot be viewed in the correct order. While an example has been described in which the message is divided into segments for the right and left eyes alternately, the number of characters to be divided is not limited to one character. For example, every two characters may be alternately extracted for the right and left eyes.

[0053] [Configuration of information processing device] Next, with reference to Fig. 12, a detailed description will be given of the configuration of an information processing device 10 used in the video conference system SYS shown in Fig. 1. Fig. 12 is a block diagram showing an example of the configuration of an information processing device according to this embodiment. The information processing device 10 is connected to a display 20, a 2D camera 30, a 3D camera 40, and an eye camera 50, and is communicatively connected to other information processing devices 10, thereby enabling online video conferencing in real time (see Fig. 1). For example, the information processing device 10 includes a communication unit 11, a storage unit 12, an input unit 13, an output unit 14, a 2D / 3D control unit 15, and a control unit 16. These units are communicatively connected via a bus or the like.

[0054] The communication unit 11 is configured to include, for example, a plurality of Ethernet (registered trademark) ports, and communication devices that perform wireless communication such as Bluetooth (registered trademark) and Wi-Fi (registered trademark). For example, the communication unit 11 has a function as a transmitter that transmits image data and various information to other information processing devices 10 via a communication network NW (see FIG. 1), and a function as a receiver that receives image data and various information from other information processing devices 10.

[0055] The storage unit 12 is configured to include one or more of an SSD (Solid State Drive), an HDD (Hard Disk Drive), a ROM (Residual Only Memory), a Flash-ROM, etc. For example, the storage unit 12 stores system programs and setting data that control the system of the information processing device 10, application programs that run on the system, various data used by the applications, etc.

[0056] Input devices such as a keyboard, a mouse, a touchpad, and a microphone for inputting various instructions by voice are connected to the input unit 13. These input devices may be built into the information processing device 10 or may be external devices separate from the information processing device 10.

[0057] The output unit 14 is an output device such as a speaker, or may be an earphone jack to which an external output device such as an earphone is connected.

[0058] The 2D / 3D control unit 15 controls the display on the display 20 to be a 2D display or a 3D display in accordance with control from the control unit 16. For example, the 2D / 3D control unit 15 controls the orientation state of the liquid crystal by controlling whether or not to apply an AC voltage to the liquid crystal of the liquid crystal lens, and controls the display of an image on each display pixel, thereby controlling the display to be a 2D display (see FIG. 7) or a 3D display (see FIG. 6).

[0059] The control unit 16 is configured to include processors such as a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), a chipset, and memory, and realizes various functions by executing programs (system programs, various programs such as applications that run on the system) stored in the memory unit 12, etc.

[0060] For example, the control unit 16 includes a video conference processing unit 161 as a functional configuration realized by executing a video conference application. The video conference processing unit 161 executes processing for a video conference in which images and audio are communicated bidirectionally with another information processing device 10 via the communication network NW. For example, the video conference processing unit 161 transmits a captured image (a captured image including a person present on the side facing the display 20) captured by the 2D camera 30 or the 3D camera 40 to another information processing device 10 connected to the communication network NW via the communication unit 11. The video conference processing unit 161 also receives a captured image transmitted from another information processing device 10 connected to the communication network NW via the communication unit 11, and displays the received captured image on the display 20.

[0061] 9 to 11, the control unit 16 executes a process for transmitting a message to only specific people using the 3D display. This process for transmitting a message to only specific people using the 3D display is hereinafter referred to as a "3D message notification process." A program for performing this 3D message notification process may be incorporated into a video conferencing app, for example, or may be provided as an application program separate from the video conferencing app and performed in conjunction with the video conferencing process.

[0062] The 3D message notification process includes a sending-side process for sending a message and a display-side process for receiving the message and displaying it in 3D. The control unit 16 has a functional configuration for performing the sending-side process in the 3D message notification process, which includes a person information acquisition unit 162, a message acceptance unit 163, and a message sending unit 164. The control unit 16 also has a functional configuration for performing the display-side process in the 3D message notification process, which includes a message acquisition unit 165, a position information acquisition unit 166, and a display control unit 167.

[0063] The person information acquisition unit 162 acquires person information of participants in a video conference from another information processing device 10 (an information processing device 10 that performs display-side processing) via the communication network NW and the communication unit 11. The person information of participants in a video conference is information in which eye position information (for example, position coordinates) of a person (participant) detected by another information processing device 10 (an information processing device 10 that performs display-side processing) is associated with a person ID.

[0064] The message receiving unit 163 receives a selection of a person to whom a message is to be conveyed from among the people (video conference participants) captured in the captured image of the video conference displayed on the display 20, and an input of a message to be conveyed to that person. For example, the person to whom a message is to be conveyed can be selected by performing an operation on the screen area of ​​the display 20 (such as a click operation or a touch operation using a mouse). The message receiving unit 163 receives, as the person to whom a message is to be conveyed, the person whose eye position is closest to the operated position on the screen area from among the people captured in the captured image displayed on the display 20. Note that an upper limit may be set for the distance from the operated position to the eye position, and an operation in which the eye position is not within the upper limit may be treated as invalid.

[0065] Furthermore, the message receiving unit 163 receives input of a message by displaying a text box in which text can be input on the display 20. Note that the message receiving unit 163 may also receive input of a message using stamps by displaying a list of stamps on the display 20.

[0066] Fig. 13 is a diagram showing an example of a method for receiving a message according to this embodiment. As shown in (A) of Fig. 13, when the user selects a person to whom he or she wishes to convey a message by clicking on the screen area of ​​the display 20 with a mouse or the like, a text box TB is displayed as shown in (B) of Fig. 13. When a message is entered into the text box TB, the message receiving unit 163 accepts the message as the message to be conveyed to the person whose eyes are closest to the position where the click operation was performed.

[0067] Returning to FIG. 12, the message sending unit 164 sends message information that associates the person ID of the person accepted by the message accepting unit 163 with the message to another information processing device 10 (an information processing device 10 that performs display side processing) via the communication unit 11 and the communication network NW.

[0068] The message acquisition unit 165 (an example of an information acquisition unit) acquires a message to be displayed in 3D. For example, the message acquisition unit 165 acquires message information to be displayed in 3D, which is transmitted from the information processing device 10 that performs transmission-side processing in the 3D message notification processing, via the communication network NW and the communication unit 11.

[0069] For example, the message acquisition unit 165 acquires a message to be displayed in 3D in association with the person ID of the person to whom the message is to be conveyed. That is, the message acquisition unit 165 acquires the person ID of a person selected from the people (participants in the video conference) captured in the captured image in another information processing device 10 (message sender) that received the captured image transmitted from the video conference processing unit 161, and the message to be conveyed to that person.

[0070] The position information acquisition unit 166 acquires, from the eye camera 50, detection information (position information) of the eye positions of people present on the side facing the display 20. Furthermore, the position information acquisition unit 166 associates the eye position information (e.g., position coordinates) of the people acquired from the eye camera 50 with a person ID, and transmits this as person information of the participants of the video conference to the information processing device 10 that performs the transmitting-side process in the 3D message notification process. Here, the person ID is identification information that, when the eyes of multiple people are detected by the eye camera 50, makes it possible to identify the position information of the detected multiple eyes. For example, the person ID may be a number assigned to each of the position information of the multiple eyes detected by the eye camera 50.

[0071] The display control unit 167 controls the display on the display 20. When a video conference process is being executed by the video conference application, the display control unit 167 causes the display 20 to display captured images transmitted from other information processing devices 10 and received via the communication unit 11.

[0072] Furthermore, when controlling the display of the display 20, the display control unit 167 controls the display to either 2D display or 3D display via the 2D / 3D control unit 15. For example, the display control unit 167 controls the display of the display 20 to either 2D display or 3D display.

[0073] For example, the display control unit 167 can control the display 20 to a 2D display and also control a part of the screen area of ​​the display 20 to a 3D display. For example, the display control unit 167 controls a part of the screen area of ​​the display 20 to a 3D display and controls the other screen areas to a 2D display. As an example, the display control unit 167 displays a message included in the message information acquired by the message acquisition unit 165 in a part of the screen area of ​​the display 20 in 3D so as to be stereoscopically visible at a specified position (see FIG. 9). Then, the display control unit 167 controls the screen areas other than the screen area in which the message is displayed in 3D to a 2D display.

[0074] Here, when displaying a message in 3D so that it can be viewed stereoscopically at a specified position, the display control unit 167 displays the message in 3D so that it can be viewed stereoscopically at the eye position of a person identified by the person ID (i.e., the person to whom the message is to be conveyed) based on the person ID included in the message information received by the message acquisition unit 165 (see FIG. 9). The eye position at this time is based on the latest eye position information acquired by the position information acquisition unit 166.

[0075] Furthermore, when the message acquisition unit 165 acquires multiple messages to be conveyed to multiple people, the display control unit 167 displays each of the multiple messages acquired by the message acquisition unit 165 in a 3D manner so that they can be viewed stereoscopically at the designated positions in each of multiple partial screen areas (see FIG. 10).

[0076] Furthermore, the display control unit 167 may determine the position of a portion of the screen area of ​​the display 20 that is controlled to be displayed in 3D, according to a specified position. For example, the information processing device 10 may determine the position of the portion of the screen area that is controlled to be displayed in 3D, according to the position of a person to whom a message is to be conveyed, by displaying a message for a person on the right side of the screen area of ​​the display 20 on the right side, and a message for a person on the left side of the screen area of ​​the display 20 on the left side (see FIG. 10 ).

[0077] Note that the display control unit 167 may determine the position of the partial screen area of ​​the screen area of ​​the display 20 that is controlled to be displayed in 3D, depending on the position within the screen area of ​​the content displayed on the display 20. For example, the display control unit 167 may determine the position of the partial screen area that is controlled to be displayed in 3D to be an area within the screen area of ​​the display 20 where no application window is displayed (an area such as the background).

[0078] In addition, when displaying a message in 3D, the display control unit 167 may divide the message acquired by the message acquisition unit 165 into a message (text) to be displayed as an image for the right eye and a message (text) to be displayed as an image for the left eye (see Figure 11).

[0079] [Operation by 3D message notification processing] Next, the operation of the 3D message notification process in the video conference system SYS will be described. Here, the case where a message is sent from site A to a specific person at site B in the video conference system SYS shown in FIG. 1 will be described as an example.

[0080] First, the operation of the 3D message notification process (message sending side process) executed by the information processing device 10A at the site A will be described with reference to Fig. 14. Fig. 14 is a flowchart showing an example of the message sending side process in the 3D message notification process according to the present embodiment.

[0081] (Step S101) When the information processing device 10A launches the video conference application, it starts processing the video conference using the video conference application. The information processing device 10A transmits, for example, a captured image captured by the 2D camera 30A (a captured image including persons A1 to A4 present on the side facing the display 20A) to the information processing device 10B via the communication network NW. The information processing device 10A also receives, via the communication network NW, a captured image transmitted from the information processing device 10B, and displays the received captured image on the display 20A. The captured image transmitted from the information processing device 10B is, for example, a captured image captured by the 2D camera 30B (a captured image including persons B1 to B4 present on the side facing the display 20B). Then, the process proceeds to step S103.

[0082] (Step S103) The information processing device 10A acquires person information via the communication network NW. The person information is information in which eye position information (e.g., position coordinates) of a person participating in the video conference at site B is associated with a person ID, and is information that the information processing device 10B detects and acquires using the eye camera 50B. Then, the process proceeds to step S105.

[0083] (Step S105) Information processing device 10A accepts a selection of a person to whom a message is to be sent from among the people (participants at site B) captured in the captured image of the video conference displayed on display 20A. For example, a person participating at site A selects a person to whom a message is to be sent from among people B1 to B4 participating in the video conference at site B by operating the screen area of ​​display 20A (e.g., clicking with a mouse) (see FIG. 13A). Information processing device 10A accepts, among people B1 to B4 displayed on display 20A, the person whose eye position is closest to the position operated on the screen area as the person to whom a message is to be sent. Then, the process proceeds to step S107.

[0084] (Step S107) The information processing device 10A displays on the display 20A a text box TB (see (B) of FIG. 13) into which text can be input, as a means for inputting a message to the person selected in step S105. The information processing device 10A accepts the input of a message in response to an input operation by the user into the text box TB. Note that when the information processing device 10A accepts the input of a message into the text box TB, it ends the display of the text box TB. Then, the process proceeds to step S109.

[0085] (Step S109) The information processing device 10A transmits message information, in which the person ID of the person accepted in step S105 is associated with the message accepted in step S107, to the information processing device 10B via the communication network NW. Then, the process proceeds to step S111.

[0086] (Step S111) The information processing device 10A determines whether the video conference application has been terminated. If the information processing device 10A determines that the video conference application has not been terminated (NO), the information processing device 10A continues the video conference processing and returns to step S103. On the other hand, if the information processing device 10A determines that the video conference application has been terminated (YES), the information processing device 10A terminates the 3D message notification processing.

[0087] Next, the operation of 3D message notification processing (message display processing) executed by the information processing device 10B at site B will be described with reference to Fig. 15. Fig. 15 is a flowchart showing an example of message display processing in the 3D message notification processing according to the present embodiment.

[0088] (Step S201) When the information processing device 10B launches the video conference application, it starts processing the video conference using the video conference application. The information processing device 10B transmits, for example, a captured image captured by the 2D camera 30B (a captured image including persons B1 to B4 present on the side facing the display 20B) to the information processing device 10A via the communication network NW. The information processing device 10B also receives, via the communication network NW, a captured image transmitted from the information processing device 10A, and displays the received captured image on the display 20B. The captured image transmitted from the information processing device 10A is, for example, a captured image captured by the 2D camera 30A (a captured image including persons A1 to A4 present on the side facing the display 20A). Then, the process proceeds to step S203.

[0089] (Step S203) The information processing device 10B acquires detection information (position information) of the eye position of a person present on the side facing the display 20B from the eye camera 50B, and then proceeds to the processing of step S205.

[0090] (Step S205) The information processing device 10B associates the position information (e.g., position coordinates) of the person's eyes acquired from the eye camera 50 with the person ID, and transmits the information processing device 10A via the communication network NW as person information of the person participating in the video conference at site B. Then, the process proceeds to step S207.

[0091] (Step S207) Information processing device 10B determines whether message information has been acquired from information processing device 10A. Message information refers to the message information transmitted from information processing device 10A in step S109 of FIG. 14. If information processing device 10B determines that message information has not been acquired from information processing device 10A (NO), it proceeds to processing of step S211. On the other hand, if information processing device 10B determines that message information has been acquired from information processing device 10A (YES), it proceeds to processing of step S209.

[0092] (Step S209) Information processing device 10B displays the message included in the message information acquired in step S207 in 3D so that it can be viewed stereoscopically at a specified position in a partial screen area of ​​display 20B. Specifically, information processing device 10B displays the message included in the message information acquired in step S207 in 3D so that it can be viewed stereoscopically at the eye position of a person identified by the person ID included in the message information (i.e., the person to whom the message is to be conveyed) (see FIG. 9). As a result, the message transmitted from information processing device 10A by selecting a person is displayed on display 20B so as to be visible only to the selected person.

[0093] In addition, when the information processing device 10B acquires a plurality of messages to be conveyed to a plurality of persons, the information processing device 10B may display each of the plurality of messages in a 3D manner so that the messages can be viewed stereoscopically at the eye positions of the persons to whom the messages are to be conveyed (see FIG. 10).

[0094] In addition, the information processing device 10B may control the position of a portion of the screen area to be displayed in 3D depending on the position of the person to whom the message is to be conveyed, such that a message for a person on the right side of the display 20 is displayed on the right side of the screen area of ​​the display 20, and a message for a person on the left side of the display 20 is displayed on the left side of the screen area of ​​the display 20 (see Figure 10).

[0095] Furthermore, when displaying a message in 3D, the information processing device 10B may divide the message into a message (text) to be displayed as an image for the right eye and a message (text) to be displayed as an image for the left eye and display them (see FIG. 11).

[0096] (Step S211) The information processing device 10B determines whether the video conference application has been terminated. If the information processing device 10B determines that the video conference application has not been terminated (NO), the information processing device 10B continues the video conference processing and returns to step S203. On the other hand, if the information processing device 10B determines that the video conference application has been terminated (YES), the information processing device 10B terminates the 3D message notification processing.

[0097] [Summary of the embodiment] As described above, the information processing device 10 according to this embodiment controls the display of the display 20. The display 20 includes a plurality of display pixels arranged in a matrix and a plurality of liquid crystal lenses arranged on top of the plurality of display pixels, and is capable of switching between 2D display (planar display) and 3D display (stereoscopic display) by adjusting the direction of light emitted from the plurality of display pixels by controlling the orientation state of the liquid crystal in the liquid crystal lenses. The information processing device 10 controls the display 20 to a 2D display, and when it acquires a message (an example of information) to be displayed in 3D, it displays the acquired message in a portion of the screen area of ​​the display 20 in a 3D manner so that it can be viewed stereoscopically from a specified position.

[0098] As a result, the information processing device 10 displays a message in 3D in a portion of the screen area of ​​the display 20 so that it can be viewed stereoscopically at a specified position, so that even when a video conference is in progress, the message (information) can be conveyed only to specific people without being seen by others.

[0099] For example, the information processing device 10 acquires eye position information (eye position detection information) of a person present on the side facing the display 20, detected from a captured image of the side facing the display 20. The information processing device 10 also acquires a message to be displayed in 3D and a person ID (an example of person identification information) of the person to whom the message is to be conveyed, in association with each other. Then, when the information processing device 10 displays the acquired message in 3D so as to be stereoscopically visible at a specified position, the information processing device 10 displays the message in 3D so as to be stereoscopically visible at the eye position of the person to whom the message is to be conveyed, based on the acquired person ID and the person's eye position information.

[0100] As a result, the information processing device 10 displays a 2D image in a portion of the screen area of ​​the display 20 so that it can be viewed stereoscopically at the eye position of the person to whom the message is to be conveyed, so that even when a video conference is in progress, the message (information) can be conveyed only to a specific person without being seen by other people.

[0101] Furthermore, the information processing device 10 may determine the position of a portion of the screen area of ​​the display 20 that is controlled to be displayed in 3D, according to a specified position. For example, as shown in Fig. 10, the information processing device 10 may display a message for person α (Person-α) who is on the right side of the screen area of ​​the display 20, on the right side of the screen area of ​​the display 20. On the other hand, the information processing device 10 may display a message for person β (Person-β) who is on the left side of the screen area of ​​the display 20, on the left side of the screen area of ​​the display 20.

[0102] This allows the information processing device 10 to ensure visibility through 3D display, making the message easier to see.

[0103] Note that the information processing device 10 may determine the position of a part of the screen area of ​​the display 20 that is controlled to be displayed in 3D, depending on the position of the content displayed on the display 20 within the screen area.

[0104] As a result, when the information processing device 10 displays a message in 3D on the display 20, it is possible to reduce the impact on the visibility of other displayed content.

[0105] When the information processing device 10 acquires a plurality of messages to be conveyed to a plurality of people, the information processing device 10 may display each of the acquired messages in 3D in a manner that allows it to be viewed stereoscopically at a specified position (the eye position of the person to whom the message is to be conveyed) in each of a plurality of partial screen areas of the display 20.

[0106] This allows the information processing device 10 to convey a message (information) to each of a plurality of people in a manner that is not apparent to the other people, even while a video conference is in progress.

[0107] Furthermore, the information processing device 10 controls the partial screen area of ​​the display 20 to display in 3D and the other screen area to display in 2D by making the orientation state of the liquid crystal lenses different between the partial screen area and the other screen area.

[0108] As a result, the information processing device 10 allows a specific person to see the content displayed in 2D and the message (information) displayed in 3D, while other people can only see the content displayed in 2D, so that even while a video conference is in progress, the message (information) can be conveyed only to the specific person without being seen by other people.

[0109] Furthermore, when the information processing device 10 displays the acquired message in 3D on the display 20, the information processing device 10 may divide the acquired message into a message (text) to be displayed as an image for the right eye and a message (text) to be displayed as an image for the left eye and display the message.

[0110] As a result, when the information processing device 10 displays a message in 3D on the display 20, only a person in a position where stereoscopic vision is possible can correctly read the message.

[0111] The information processing device 10 also includes a communication unit 11 (an example of a transmission unit) that transmits a captured image captured by a 2D camera 30 or a 3D camera 40 (an example of an imaging device) that captures an image including a person present on the side facing the display 20 to another information processing device 10 that is communicatively connected. Then, the information processing device 10 acquires, in the other information processing device 10 that has received the captured image transmitted from the communication unit 11, the person ID of a person selected from the people captured in the captured image and a message to be displayed in 3D that is to be conveyed to that person.

[0112] As a result, the information processing device 10 can convey a message received from another information processing device 10 by displaying the message in 3D so that it is visible only to the person to whom the message is to be conveyed (the person selected by the other information processing device 10).

[0113] In addition, the control method in the information processing device 10 according to this embodiment includes a step in which the control unit 16 acquires a message (an example of information) to be displayed in 3D, and a step in which the control unit 16 controls the display 20 to display in 2D, and when a message (an example of information) to be displayed in 3D is acquired, displays the acquired message in 3D in a part of the screen area of ​​the display 20 so that it can be viewed stereoscopically at a specified position.

[0114] As a result, the control method in the information processing device 10 displays a message in 3D in a portion of the screen area of ​​the display 20 so that it can be viewed stereoscopically at a specified position, so that even while a video conference is in progress, a message (information) can be conveyed to a specific person without being seen by other people.

[0115] The videoconferencing system SYS according to this embodiment includes an information processing device 10B (an example of a first information processing device) that controls the display of a display 20B (an example of a first display) that can switch between 2D and 3D display, and an information processing device 10A (an example of a second information processing device) that is communicatively connected to the information processing device 10B and can conduct online videoconferencing. The information processing device 10B acquires eye position information (eye position detection information) of a person present on the side facing the display 20B, detected from a captured image of the side facing the display 20B. The information processing device 10B also transmits to the information processing device 10A captured images that include the person present on the side facing the display 20B, captured by a 2D camera 30 or a 3D camera 40 (an example of an imaging device). The information processing device 10B controls the display 20B to 2D display, and when the information processing device 10B acquires a message (an example of information) to be displayed in 3D from the information processing device 10A, the information processing device 10B displays the acquired message in 3D in a portion of the screen area of ​​the display 20B so that it can be viewed stereoscopically from a specified position. The information processing device 10A displays the captured image transmitted from the information processing device 10B on the display 20A (an example of a second display). The information processing device 10A also accepts a selection of a person to whom a message is to be conveyed from among the people captured in the captured image displayed on the display 20A and an input of the message to be conveyed to the person, and transmits the message to be conveyed to the person in association with the person ID (an example of person identification information) of the received person. The information processing device 10B acquires the person ID and the message to be conveyed to the person transmitted from the information processing device 10A as a message to be displayed in 3D and the person ID of the person to whom the message is to be conveyed. When the information processing device 10B displays the acquired message in 3D so as to be stereoscopically visible at a specified position, the information processing device 10B displays the message in 3D so as to be stereoscopically visible at the eye position of the person to whom the message is to be conveyed, based on the acquired information on the position of the person's eyes.

[0116] As a result, when information processing device 10A and information processing device 10B are processing a video conference online, a person participating on information processing device 10A can select any person from among the people participating on information processing device 10B via video and send a message to the selected person, thereby allowing the video conference system SYS to convey the message only to the selected person. Thus, even while a video conference is in progress, the video conference system SYS can convey a message (information) only to a specific person without other people knowing.

[0117] Furthermore, the control method in the video conference system SYS according to this embodiment includes a position information acquisition step in which the control unit 16 in the information processing device 10B acquires eye position information (detected information of eye position) of a person present on the side facing the display 20B, detected from a captured image of the side facing the display 20B; a first transmission step in which the control unit 16 in the information processing device 10B transmits to the information processing device 10A a captured image captured by the 2D camera 30 or the 3D camera 40 (an example of an imaging device) that captures a captured image including the person present on the side facing the display 20B; a second display control step in which the control unit 16 in the information processing device 10A displays the captured image transmitted in the first transmission step on the display 20A (an example of a second display); The information processing device 10B includes a receiving step of receiving a selection of a person to whom information is to be conveyed from among the information processing devices 10A and an input of a message to be conveyed to the person (an example of information), a second transmission step of associating the person ID (an example of person identification information) of the person received in the receiving step with the message to be conveyed to the person and transmitting them, an acquisition step of a control unit 16 in the information processing device 10B acquiring from the information processing device 10A the person ID and the message to be conveyed to the person transmitted in the second transmission step as a message to be displayed in 3D, and a first display control step of controlling the display 20B to a 2D display and, if the message to be displayed in 3D is acquired in the acquisition step, displaying the acquired message in 3D so as to be stereoscopically visible at a specified position in a part of the screen area of ​​the display 20B. Then, when the control unit 16 in the information processing device 10B displays the message acquired in the acquisition step in 3D so as to be stereoscopically visible at a specified position in the first display control step, the control unit 16 displays the message acquired in the acquisition step in 3D so as to be stereoscopically visible at the position of the eyes of the person to whom the information is to be conveyed, based on detection information of the position of the person's eyes acquired in the position information acquisition step.

[0118] As a result, the control method in the video conference system SYS allows, when information processing device 10A and information processing device 10B are processing a video conference online, a person participating on information processing device 10A to select any person from among the people participating on information processing device 10B via video and send a message to the selected person, thereby allowing the message to be conveyed only to the selected person. Thus, the control method in the video conference system SYS allows a message (information) to be conveyed only to a specific person without being known to other people, even while the video conference is in progress.

[0119] <Second embodiment> Next, a second embodiment will be described. In the first embodiment, when a message is displayed in 3D by the 3D message notification process, part of the screen area of ​​the display 20 is controlled to be displayed in 3D and the other screen area is controlled to be displayed in 2D, but the 2D display and the 3D display may be alternately switched. This embodiment is similar to the first embodiment in that both the 2D display and the 3D display are performed, but instead of separating the screen area for 2D display and the screen area for 3D display, the 2D display and the 3D display are alternately switched.

[0120] Fig. 16 is a diagram showing an example of a message display using 3D display according to this embodiment. The information processing device 10 controls the display 20 in an interlaced manner, and alternates between 2D display and 3D display for odd fields and even fields in one frame. In the example shown in Fig. 16, the information processing device 10 controls the odd fields to 2D display and the even fields to 3D display. The information processing device 10 displays a video conference image when controlling the display 20 to 2D display, and displays a message when controlling the display to 3D display.

[0121] Here, the information processing device 10 displays a message to be displayed when controlled to 3D display in a part of the screen area of ​​the display 20, and displays it in 3D so that it can be seen stereoscopically at the eye position of the person to whom the message is to be conveyed. This is the same as in the first embodiment. For example, the information processing device 10 displays a message to be conveyed to person α (Person-α) in 3D display so that it can be seen stereoscopically at the eye position of person α (Person-α). Furthermore, the information processing device 10 displays a message to be conveyed to person β (Person-β) in 3D display so that it can be seen stereoscopically at the eye position of person β (Person-β).

[0122] This allows the information processing device 10 to display a message that you want to convey to person α (Person-α) so that it is visible only to person α (Person-α), and to display a message that you want to convey to person β (Person-β) so that it is visible only to person β (Person-β).

[0123] In addition, the information processing device 10 may position the part of the screen area controlled to 3D display in an area (such as the background) excluding the area where the video conferencing app window is displayed (the area where an image is displayed) within the screen area of ​​the display 20 when controlled to 2D display.

[0124] Furthermore, in this embodiment, the information processing device 10 sets the display frame rate of the display 20 to twice that of the first embodiment (for example, 120 Hz) and alternates between 2D display and 3D display using odd and even fields, thereby preventing a decrease in resolution and frame rate during 2D display. Note that the screen brightness during 2D display is half that of the first embodiment, but because display screen brightness has been increasing, the image is still visible even at half the brightness.

[0125] As described above, the information processing device 10 according to this embodiment alternately switches the display of the screen area of ​​the display 20 between 2D display and 3D display.

[0126] As a result, the information processing device 10 alternates between a 2D display that is visible to all people facing the display 20 and a 3D display that is visible only to people in positions where stereoscopic viewing is possible, so that a specific person can see the content displayed in 2D and the message (information) displayed in 3D, while other people can see only the content displayed in 2D. Therefore, by displaying a message in 3D so that it can be viewed stereoscopically at the position of a specific person, the information processing device 10 can convey a message (information) only to a specific person so that other people cannot see it, even while a video conference is in progress.

[0127] For example, the information processing device 10 controls the display on the display 20 in an interlaced manner, and alternates between 2D display and 3D display between odd and even fields in one frame.

[0128] As a result, the information processing device 10 alternates between a 2D display that is visible to all people facing the display 20 and a 3D display that is visible only to people in positions where stereoscopic viewing is possible, using odd and even fields, so that a specific person can see the content displayed in 2D and the message (information) displayed in 3D, while other people can see only the content displayed in 2D. Therefore, by displaying a message in 3D so that it can be viewed stereoscopically at the position of a specific person, the information processing device 10 can convey a message (information) only to a specific person so that other people cannot see it, even while a video conference is in progress.

[0129] Although an example in which odd fields are controlled to 2D display and even fields are controlled to 3D display has been described above, it is also possible to configure odd fields to be controlled to 3D display and even fields to be controlled to 2D display.Furthermore, instead of a configuration in which odd fields and even fields alternate between 2D display and 3D display, it is also possible to configure a configuration in which 2D display and 3D display are alternately switched for each frame.

[0130] Although the embodiments of the present invention have been described above in detail with reference to the drawings, the specific configurations are not limited to the above-described embodiments, and the present invention also includes designs that do not deviate from the gist of the present invention. For example, the configurations described in the above-described embodiments can be combined in any manner.

[0131] In the above embodiment, when there are multiple people participating in the video conference on the message sending side, the information processing device 10 may display the text box TB displayed on the display 20 when the message is being input in 3D so that it is visible (stereoscopic) only to the person inputting the message. This makes it possible to make the message invisible to people other than the person inputting the message among the people participating in the video conference on the message sending side. Note that a separate information processing device 10, such as a laptop PC, may be provided for each participant.

[0132] In the above embodiment, a video conference is described as connecting two locations, location A and location B, as shown in Fig. 1, but the video conference is not limited to two locations and may involve three or more locations. Furthermore, the number of people participating in a video conference at one location is not limited to four, but may be one, two, three, or five or more.

[0133] In the above embodiment, an example has been described in which a process in cooperation with a video conference app accepts a selection of a person to whom a message is to be sent from among people (video conference participants) captured in a captured image of the video conference. However, this is not limiting. For example, regardless of the process by the video conference app, people facing the display 20 may be detected, and icons generated for each detected person may be displayed on the display 20 as options for people to whom a message is to be sent. In this way, it is also possible to perform 3D message notification processing without cooperation with a video conference app. In this case, the 3D message notification processing can be applied to any video conference app that does not support 3D message notification processing.

[0134] The 2D camera 30, the 3D camera 40, and the eye camera 50 may be built into the display 20 or the information processing device 10. The 2D camera 30, the 3D camera 40, and the eye camera 50 do not have to be installed on the display 20, and may be installed on a wall or ceiling in the space where the video conference is held, as long as they can capture images of the people participating in the video conference (those facing the display 20). When the eye camera 50 detects the position of the person's eyes, the position of the eyes detected by the eye camera 50 is corrected to the position of the person's eyes relative to the display 20 based on the positional relationship between the display 20 and the eye camera 50.

[0135] In addition, in the above embodiment, an example of a configuration in which the eye camera 50 is used to detect the position of a person's eyes is described, but the configuration in which the eye position of a person is detected using an image captured by a 2D camera 30 or a 3D camera 40 instead of the eye camera 50 may also be used.

[0136] Furthermore, in the above embodiment, when transmitting a message from the information processing device 10A at site A to site B, the information processing device 10B at site B acquires the eye position information of the people participating in the video conference at site B. However, the information processing device 10A at site A may detect the eye positions of the people from a captured image of the video conference transmitted from the information processing device 10B at site B. For example, by transmitting message information in which position information indicating the eye position of a person to whom a message is to be conveyed, among the eye positions of people detected by the information processing device 10A, is associated with a message to the information processing device 10B, the message may be displayed in 3D so as to be stereoscopically visible at the closest eye position among the eye positions of people detected on the information processing device 10B side, based on the message information received by the information processing device 10B.

[0137] 12, the 2D / 3D control unit 15 and the control unit 16 are shown as separate components, but the 2D / 3D control unit 15 may be included in the control unit 16.

[0138] The information processing device 10 described above includes an internal computer system. A program for implementing the functions of each component of the information processing device 10 may be recorded on a computer-readable recording medium, and the program may be loaded into a computer system and executed to perform processing in each component of the information processing device 10. Here, "loading a program recorded on a recording medium into a computer system and executing it" includes installing the program into a computer system. The term "computer system" here includes hardware such as an OS and peripheral devices. The term "computer system" may also include multiple computers connected via a network, including the Internet, a WAN, a LAN, a dedicated line, or other communication lines. The term "computer-readable recording medium" refers to portable media such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built into a computer system. The recording medium storing the program may also be a non-transitory recording medium such as a CD-ROM.

[0139] The recording medium also includes internal or external recording media accessible from a distribution server for distributing the program. The program may be divided into multiple parts, downloaded at different times, and then combined by each component of the information processing device 10, or each divided program may be distributed by a different distribution server. Furthermore, the term "computer-readable recording medium" also includes a medium that stores a program for a certain period of time, such as volatile memory (RAM) within a computer system that serves as a server or client when a program is transmitted over a network. The program may also be a medium that realizes part of the above-described functions. Furthermore, the program may be a so-called differential file (differential program) that can realize the above-described functions in combination with a program already stored in the computer system.

[0140] Furthermore, some or all of the functions of the information processing device 10 in the above-described embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each function may be individually implemented as a processor, or some or all of the functions may be integrated into a processor. The integrated circuit method is not limited to LSI, and may be implemented using a dedicated circuit or a general-purpose processor. Furthermore, if an integrated circuit technology that can replace LSI emerges due to advances in semiconductor technology, an integrated circuit based on that technology may be used. [Explanation of symbols]

[0141] 10 Information processing device, 11 Communication unit, 12 Memory unit, 13 Input unit, 14 Output unit, 15 2D / 3D control unit, 16 Control unit, 20 Display, 30 2D camera, 40 3D camera, 50 Eye camera, 161 Video conference processing unit, 162 Person information acquisition unit, 163 Message reception unit, 164 Message transmission unit, 165 Message acquisition unit, 166 Location information acquisition unit, 167 Display control unit, SYS Video conference system

Claims

1. An information processing device that controls display of a display that is capable of switching between planar display and stereoscopic display, the display comprising a plurality of display pixels arranged in a matrix and a plurality of liquid crystal lenses arranged overlying a plurality of the display pixels, and that adjusts the direction of light emitted from the plurality of display pixels by controlling the orientation state of liquid crystal of the liquid crystal lenses, an information acquisition unit that acquires information to be displayed stereoscopically; a display control unit that controls the display to the planar display, and when the information acquisition unit acquires information for the stereoscopic display, causes the acquired information to be stereoscopically displayed in a part of a screen area of ​​the display so as to be stereoscopically visible from a designated position; An information processing device comprising:

2. a position information acquisition unit that acquires detection information of eye positions of a person present on the side facing the display, detected from a captured image of the side facing the display; Equipped with The information acquisition unit acquiring the information to be stereoscopically displayed in association with identification information of a person to whom the information is to be conveyed; The display control unit When the information acquired by the information acquisition unit is stereoscopically displayed so as to be stereoscopically visible at the specified position, the information is stereoscopically displayed so as to be stereoscopically visible at the eye position of a person to whom the information is to be conveyed, based on identification information of the person acquired by the information acquisition unit and detection information of the eye position of the person acquired by the position information acquisition unit. The information processing device according to claim 1 .

3. The display control unit determining a position of the partial screen area of ​​the display that is to be controlled to the stereoscopic display in accordance with the specified position; The information processing device according to claim 1 .

4. The display control unit determining a position of the partial screen area of ​​the display that is to be controlled to the stereoscopic display in accordance with a position within the screen area of ​​the content displayed on the display; The information processing device according to claim 1 .

5. The display control unit When the information acquisition unit acquires a plurality of pieces of information to be conveyed to each of a plurality of people, stereoscopically displaying each of the plurality of pieces of information acquired by the information acquisition unit in each of the plurality of partial screen regions so that the pieces of information can be stereoscopically viewed at a designated position; The information processing device according to claim 1 .

6. The display control unit By making the orientation state of the liquid crystal lens different between the partial screen region and the other screen region, the partial screen region is controlled to display the stereoscopic image and the other screen region is controlled to display the monoscopic image. The information processing device according to claim 1 .

7. The display control unit The display of the screen area of ​​the display is alternately switched between the monoscopic display and the stereoscopic display. The information processing device according to claim 1 .

8. The display control unit controlling the display on the display in an interlaced manner, and alternately switching between the monoscopic display and the stereoscopic display in odd and even fields within one frame; The information processing device according to claim 7 .

9. The display control unit When performing the stereoscopic display, the information acquired by the information acquisition unit is divided into information to be displayed as an image for the right eye and information to be displayed as an image for the left eye and then displayed. The information processing device according to claim 1 .

10. a transmitting unit that transmits an image captured by an imaging device that captures an image including a person present on the side facing the display to another information processing device that is communicatively connected; Equipped with The information acquisition unit and acquiring, in the other information processing device that has received the captured image transmitted from the transmission unit, identification information of a person selected from the people captured in the captured image and information to be stereoscopically displayed that is to be conveyed to the person. The information processing device according to claim 2 .

11. A video conferencing system comprising: a first information processing device that controls display of a first display, the first display having a plurality of display pixels arranged in a matrix and a plurality of liquid crystal lenses arranged to overlap the plurality of display pixels, and that is capable of switching between planar display and stereoscopic display by adjusting a direction of light emitted from the plurality of display pixels by controlling an orientation state of liquid crystal in the liquid crystal lenses; and a second information processing device that is communicatively connected to the first information processing device and is capable of conducting online video conferencing, The first information processing device a position information acquisition unit that acquires detection information of eye positions of a person present on the side facing the first display, detected from a captured image of the side facing the first display; a first transmission unit that transmits to the second information processing device an image captured by an imaging device that captures an image including a person present on a side facing the first display; an information acquisition unit that acquires the information to be stereoscopically displayed from the second information processing device; a first display control unit that controls the first display to the planar display, and when the information acquisition unit acquires information for the stereoscopic display, causes the acquired information to be stereoscopically displayed in a part of a screen area of ​​the first display so as to be stereoscopically visible from a designated position; Equipped with The second information processing device a second display control unit that causes a second display to display the captured image transmitted from the first transmission unit; a reception unit that receives a selection of a person to whom information is to be communicated from among the people captured in the captured image displayed on the second display and an input of information to be communicated to the person; a second transmission unit that transmits the person's identification information received by the reception unit in association with information to be conveyed to the person; Equipped with The information acquisition unit of the first information processing device acquiring the person's identification information and the information to be conveyed to the person, which are transmitted from the second transmission unit of the second information processing device, as the information to be stereoscopically displayed and the identification information of the person to whom the information is to be conveyed; The first display control unit of the first information processing device, when the information acquired by the information acquisition unit is stereoscopically displayed so as to be stereoscopically visible at a designated position, the information is stereoscopically displayed so as to be stereoscopically visible at an eye position of a person to whom the information is to be conveyed, based on detection information of an eye position of the person acquired by the position information acquisition unit; Video conferencing system.

12. A control method for an information processing device that controls display of a display that is switchable between planar display and stereoscopic display by controlling an orientation state of liquid crystal in the liquid crystal lenses, the display comprising a plurality of display pixels arranged in a matrix and a plurality of liquid crystal lenses arranged overlying a plurality of the display pixels, the display being capable of switching between planar display and stereoscopic display, the method comprising: The control unit acquiring information to be displayed stereoscopically; controlling the display to the planar display and, when information for the stereoscopic display is acquired, stereoscopically displaying the acquired information in a part of a screen area of ​​the display so as to be stereoscopically visible from a designated position; A control method comprising:

13. A control method for a video conference system including a first information processing device that controls display of a first display, the first display having a plurality of display pixels arranged in a matrix and a plurality of liquid crystal lenses arranged overlying a plurality of the display pixels, and that is capable of switching between planar display and stereoscopic display by adjusting a direction of light emitted from the plurality of display pixels by controlling an orientation state of liquid crystal in the liquid crystal lenses, and a second information processing device that is communicatively connected to the first information processing device and is capable of conducting online video conferences, the method comprising: a control unit in the first information processing device, a position information acquisition step of acquiring detection information of eye positions of a person present on the side facing the first display, detected from a captured image of the side facing the first display; a first transmission step of transmitting, to the second information processing device, a captured image captured by an imaging device that captures an image including a person present on a side facing the first display; a control unit in the second information processing device, a second display control step of displaying the captured image transmitted in the first transmission step on a second display; a receiving step of selecting a person to whom information is to be communicated from among the people captured in the captured image displayed on the second display and receiving input of information to be communicated to the person; a second transmission step of associating the person's identification information received in the receiving step with information to be communicated to the person and transmitting the information; a control unit in the first information processing device, an acquisition step of acquiring, from the second information processing device, the person's identification information transmitted in the second transmission step and information to be conveyed to the person as information to be stereoscopically displayed; a first display control step of controlling the first display to the planar display and, when information for the stereoscopic display is acquired in the acquisition step, stereoscopically displaying the acquired information in a part of a screen area of ​​the first display so as to be stereoscopically visible from a designated position; Including, a control unit in the first information processing device, In the first display control step, when the information acquired in the acquisition step is stereoscopically displayed so as to be stereoscopically visible at a designated position, the information is stereoscopically displayed so as to be stereoscopically visible at an eye position of a person to whom the information is to be conveyed, based on detection information of the eye position of the person acquired in the position information acquisition step. Control method.

Images