Remote communication system, remote communication method, relay server, and program

Abstract

To provide a remote communication system for enabling a local user to capture an area to which a remote user pays attention.SOLUTION: The remote communication system is provided with an image capturing unit for capturing images, a light emitting unit having a light emitting area with a width corresponding to a viewing angle of the image capturing unit, an area extracting unit for extracting some areas from the images captured by the image capturing unit, a display unit for displaying images of some areas extracted by the area extracting unit, an attention level determining unit for determining the attention level of a user viewing the image displayed by the display unit, and a light emitting control unit for emitting light in a light emitting area corresponding to a portion of the area extracted by the area extracting unit with a light emitting method corresponding to the attention level determined by the attention level determining unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a remote communication system, a remote communication method, a relay server, and a program. [Background technology]

[0002] Currently, remote communication by a network conference system that displays a conversation among many people and a face image of a speaker is common. Patent Document 1 discloses a system including a 360-degree camera that captures an image for a remote user, and a lighting device configured to generate visual output for one or more local users that represents the viewing direction or number of the remote users based on mapping of the region of interest of the remote viewer, the lighting device being associated with the 360-degree camera. Non-Patent Document 1 also discloses a system that evaluates a concentration state from a pulse wave and a body movement. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-102845 A [Non-patent literature]

[0004] [Non-Patent Document 1] Subaru Uchida et al., "Verification of Accuracy of Estimating Concentration State Using EEG and Body Movement Information," Multimedia, Distributed, Cooperative and Mobile (DICOMO2017) Symposium, pp.876-879, June 2017 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the system of Patent Document 1 mentioned above, although the local user can know in which direction the remote user is outputting the image from the 360-degree camera, there is a problem in that the local user may not be able to know whether the remote user is paying attention to that area, or is just outputting that area by chance without any particular awareness.

[0006] The present invention has been made in consideration of the above circumstances, and provides a remote communication system, a remote communication method, a relay server, and a program that enable a local user to grasp an area that a remote user is paying attention to. [Means for solving the problem]

[0007] The present invention has been made to solve the above-mentioned problems, and one aspect of the present invention is a remote communication system comprising an imaging unit that captures an image, a light-emitting unit having a light-emitting area with a width corresponding to the angle of view of the imaging unit, an area extraction unit that extracts a partial area from the image captured by the imaging unit, a display unit that displays an image of the partial area extracted by the area extraction unit, an attention determination unit that determines the attention level of a user viewing the image displayed by the display unit, and an illumination control unit that causes a portion of the light-emitting area corresponding to the partial area extracted by the area extraction unit to emit light in a lighting manner corresponding to the attention level determined by the attention determination unit.

[0008] Another aspect of the present invention is the above-mentioned remote communication system, in which the light emission method according to the level of attention is a light emission method in which at least one of the brightness, color, blinking period, and blinking duty ratio differs depending on the level of attention.

[0009] Another aspect of the present invention is the above-mentioned remote communication system, wherein the attention determination unit determines the attention level based on any one of the size of the portion of the area extracted by the area extraction unit, the user's degree of attention to the display unit, or the user's vital signs.

[0010] Another aspect of the present invention is the above-mentioned remote communication system, wherein the area extraction section changes the extracted area in accordance with the movement of an object designated by the user.

[0011] Another aspect of the present invention is the above-mentioned remote communication system, wherein the light emission control unit determines the light emission method based on the fact that the area extraction unit is changing the extracted area in accordance with the movement of an object specified by the user.

[0012] Another aspect of the present invention is a remote communication method having a first step of capturing an image, a second step of extracting a partial area from the image captured in the first step, a third step of displaying an image of the partial area extracted in the second step, a fourth step of determining the level of attention of a user viewing the image displayed in the third step, and a fifth step of illuminating a portion of an illumination area having a width corresponding to the angle of view of the image captured in the first step, the portion corresponding to the partial area extracted in the second step, in an illumination manner corresponding to the level of attention determined in the fourth step.

[0013] Another aspect of the present invention is a relay server in a remote communication system comprising an imaging unit that captures an image, a light-emitting unit having a light-emitting area with a width corresponding to the angle of view of the imaging unit, an area extraction unit that extracts a partial area from the image captured by the imaging unit, a display unit that displays an image of the partial area extracted by the area extraction unit, and an attention determination unit that determines the attention level of a user viewing the image displayed by the display unit, and the relay server further comprises an illumination control unit that causes a portion of the light-emitting area corresponding to the partial area extracted by the area extraction unit to be illuminated in an illumination manner corresponding to the attention level determined by the attention determination unit.

[0014] Another aspect of the present invention is a program for causing a computer in a remote communication system comprising an imaging unit for capturing an image, a light-emitting unit having a light-emitting area with a width corresponding to the angle of view of the imaging unit, an area extraction unit for extracting a partial area from the image captured by the imaging unit, a display unit for displaying an image of the partial area extracted by the area extraction unit, and an attention determination unit for determining the attention level of a user viewing the image displayed by the display unit, to function as an illumination control unit that causes a portion of the light-emitting area corresponding to the partial area extracted by the area extraction unit to emit light in a lighting manner corresponding to the attention level determined by the attention determination unit. Effect of the Invention

[0015] According to the present invention, the remote communication system, the remote communication method, the relay server, and the program enable a local user to grasp an area that a remote user is paying attention to. [Brief description of the drawings]

[0016] [Figure 1] 1 is a schematic block diagram showing a device configuration of a remote communication system 1 according to an embodiment of the present invention. [Diagram 2] 2 is a schematic block diagram showing the functional configuration of a relay server 30, a remote terminal 50, and a local terminal 60 in the embodiment. [Diagram 3] 10 is a flowchart illustrating an example of the operation of a light emission control unit 34 in the embodiment. [Figure 4] 10 is a flowchart illustrating an example of the operation of a region extraction unit 53 in the embodiment. [Diagram 5] 2A to 2C are schematic diagrams showing an example of light emission from a ring light 20 in the embodiment. [Figure 6] 11 is a diagram showing an example of a display image of a remote terminal 50 in the embodiment. FIG. [Figure 7] FIG. 13 is a schematic diagram illustrating a modified example of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing the device configuration of a remote communication system 1 according to an embodiment of the present invention. The remote communication system 1 is a system for communication between a local user in a commercial facility such as a store, an appreciation facility such as a museum, or a work facility such as a factory (hereinafter referred to as a local facility) and a remote user who is remote from these local facilities. Note that remote users may also be able to communicate with each other. Also, although commercial facilities, appreciation facilities, and work facilities have been given as examples of local facilities, local facilities may be other than these.

[0018] The remote communication system 1 includes an omnidirectional camera 10, a ring light 20, a relay server 30, a control microcomputer 40, one or more remote terminals 50, and a local terminal 60. The omnidirectional camera 10, the relay server 30, the one or more remote terminals 50, and the local terminal 60 are communicatively connected to each other via a communication network such as an IP (Internet Protocol) network. The relay server 30 and the control microcomputer 40 may be communicatively connected to each other via a communication network, a USB (Universal Serial Bus), Bluetooth (registered trademark), or the like. The omnidirectional camera 10 (imaging unit) is a camera capable of simultaneously capturing images in all directions, and is installed in a local facility or can be moved within the local facility. For example, if the local facility is a commercial facility, the omnidirectional camera 10 may be installed in a shopping cart. The ring light 20 (light emitting unit) is a light in which LEDs (Light Electric Diodes, light emitting areas) are arranged on a circumference, and is installed so that the relative position with respect to the omnidirectional camera 10 does not change.

[0019] The relay server 30 transmits images captured by the omnidirectional camera 10 to the remote terminals 50. The relay server 30 also determines how to light up each LED of the ring light 20 based on information acquired from each remote terminal 50, and instructs the control microcomputer 40. The relay server 30 also relays video chats between one or more remote terminals 50 and a local terminal 60. The relay server 30 may be realized by one or more computers reading and executing a program.

[0020] The control microcomputer 40 controls the emission of each LED of the ring light 20 in accordance with instructions from the relay server 30. The control microcomputer 40 may be realized by one or more computers reading and executing a program.

[0021] The remote terminal 50 is a terminal used by a remote user, and extracts and displays an area that the remote user pays attention to from an image received from the relay server 30. The remote terminal 50 also transmits information about the area and information indicating the remote user's attention level to the relay server 30. The remote terminal 50 also performs video chats with the local terminal 60 and other remote terminals 50. The remote terminal 50 may be realized by one or more computers reading and executing a program. The program may also be transmitted to the remote terminal 50 by the relay server 30.

[0022] The local terminal 60 is a terminal used by a local user, and performs video chat with one or more remote terminals 50. The local terminal 60 may be realized by one or more computers reading and executing a program. The program may be transmitted to the local terminal 60 by the relay server 30.

[0023] The above-mentioned computer may be any type of computer, such as a smartphone, a tablet, a notebook personal computer, a desktop personal computer, or a rack-mounted server.

[0024] 2 is a schematic block diagram showing the functional configuration of the relay server 30, the remote terminal 50, and the local terminal 60 in this embodiment. Although only one remote terminal 50 is shown in FIG. 2, there may be multiple remote terminals. In this case, the video chat control unit 31, the image transmission unit 33, and the light emission control unit 34 described later communicate with the multiple remote terminals 50.

[0025] The relay server 30 includes a video chat control unit 31, an image acquisition unit 32, an image transmission unit 33, and a light emission control unit 34. The video chat control unit 31 transmits and receives images and audio of a video chat between one or more remote terminals 50 and a local terminal 60. The video chat control unit 31 also transmits information (display area information described later) received from each remote terminal 50 to the other remote terminals 50 and the local terminal 60. This enables each remote terminal 50 and the local terminal 60 to display an area in the panoramic image displayed on the other remote terminal 50. The image acquisition unit 32 acquires an image (panoramic image) captured by the panoramic camera 10 from the panoramic camera 10. The image transmission unit 33 transmits the image acquired by the image acquisition unit 32 to each remote terminal 50 and the local terminal 60.

[0026] The light emission control unit 34 determines the light emission method of the ring light 20 based on information received from each remote terminal 50 (display area information and attention level information, which will be described later), and instructs the control microcomputer 40 on the result of the determination. For example, the light emission control unit 34 determines to emit light from the LEDs of the ring light 20 in a portion corresponding to the area extracted from the panoramic image and displayed on the remote terminal 50. At this time, the light emission method determined by the light emission control unit 34 is a light emission method according to the attention level determined by the remote terminal 50. The attention level is a value indicating the degree of attention paid by the user using the remote terminal 50 to the image extracted from the panoramic image and displayed by the remote terminal 50. The light emission method according to the attention level is a light emission method in which at least any of the brightness, color, blinking period, and blinking duty ratio differs depending on the attention level. For example, the brightness may be increased, the color saturation may be increased, the blinking period may be shortened, or the blinking duty ratio may be increased as the attention level increases. The color to be emitted may be a color corresponding to the remote terminal 50 or the user using the remote terminal 50. Furthermore, when the LED of the ring light 20 is made to blink, the brightness of the LED may be a square wave, a sine wave, or another waveform such as a sawtooth wave.

[0027] The local terminal 60 includes a video chat processing unit 61, a UI (User Interface) unit 62, and an image receiving unit 63. The video chat processing unit 61 transmits and receives images and audio of a video chat to and from the video chat control unit 31. The video chat processing unit 61 outputs the received images and audio from the UI unit 62. The images and audio are images and audio of a remote user captured and collected by the remote terminal 50. The video chat processing unit 61 also receives display area information of the remote terminal 50 from the video chat control unit 31, and outputs from the UI unit 62 an area indicated by the display area information among the full sky image received by the image receiving unit 63. The received display area information may be display area information of the remote terminal 50 of a partner user (remote user) with whom the local user is video chatting.

[0028] Furthermore, the video chat processing unit 61 acquires images and audio of the local user from the UI unit 62 and transmits them to the video chat control unit 31. The UI unit 62 includes a display (such as a liquid crystal display or an organic EL display) for displaying images, a camera for capturing images, a speaker for outputting audio, and a microphone for collecting audio. The UI unit 62 may further include input / output devices such as a touch panel, a keyboard, a mouse, a vibration element, and an LED lamp. The image receiving unit 63 receives the full sky image transmitted by the image transmitting unit 33.

[0029] The remote terminal 50 includes a video chat processing unit 51, an image receiving unit 52, an area extracting unit 53, a UI unit 54, a vital sensor unit 55, an attention level determining unit 56, and an information transmitting unit 57. The video chat processing unit 51 transmits and receives images and audio of a video chat to and from the video chat control unit 31. The video chat processing unit 51 outputs the received image and audio signals from the UI unit 54. The images and audio are images and audio of a local user or a remote user captured and collected by the local terminal 60 or another remote terminal 50. The video chat processing unit 51 also receives display area information of another remote terminal 50 from the video chat control unit 31, and outputs from the UI unit 52 an area indicated by the display area information among the full sky image received by the image receiving unit 52. The received display area information may be display area information of the remote terminal 50 of a partner user (remote user) with whom the remote user of the remote terminal 50 is video chatting. In addition, the video chat processing unit 51 acquires the image and voice of the remote user from the UI unit 54 and transmits them to the video chat control unit 31 .

[0030] The image receiving unit 52 receives the panoramic image transmitted by the image transmitting unit 33. The area extracting unit 53 extracts a partial area from the panoramic image received by the image receiving unit 52, and displays the image of the area on the UI unit 54. This area may be an area designated by a remote user using the UI unit 54, or may be an area changed in accordance with the movement of an object that at least partially overlaps with the area designated by the remote user.

[0031] The UI unit 54 (display unit) includes a display (such as a liquid crystal display or an organic EL display) for displaying an image including an image of the region extracted by the region extraction unit 53, a camera for capturing an image, a speaker for outputting sound, and a microphone for collecting sound. The UI unit 54 may further include input / output devices such as a touch panel, a keyboard, a mouse, a vibration element, and an LED lamp.

[0032] The vital sensor unit 55 is a sensor that detects at least one of the brain waves, pulse waves, and body movements of the remote user. The attention level determination unit 56 determines the attention level of the remote user viewing the image displayed by the UI unit 54. The attention level determination unit 56 may determine the attention level based on any one of the size of the area extracted by the area extraction unit 53, the remote user's attention level to the display of the UI unit 54, and the remote user's vital signs, or a combination of some or all of these.

[0033] For example, the attention level determination unit 56 may determine that the smaller the size of the area extracted by the area extraction unit 53 (the size in the panoramic image) is, the higher the attention level is. Also, the attention level determination unit 56 may determine that the higher the degree of attention of the remote user to the display of the UI unit 54 is. Note that the attention level may be determined to be high when the direction of the face of the remote user in the image captured by the camera of the UI unit 54 is facing the display of the UI unit 54. The vital signs of the remote user are the detection results by the vital sensor unit 55, and the attention level determination unit 56 may determine the concentration state by, for example, a method disclosed in Non-Patent Document 1, and determine that the more concentrated the user is, the higher the attention level is.

[0034] The information transmitting unit 57 transmits attention level information indicating the attention level determined by the attention level determining unit 56 and display area information regarding the area extracted by the area extracting unit 53 to the light emission control unit 34. The display area information may be information indicating coordinates and size of the area extracted from the panoramic image, or information indicating the direction (horizontal coordinates) and width (horizontal width) of the area extracted from the panoramic image. The information transmitting unit 57 may transmit information indicating whether the area extracting unit 53 is changing the area in accordance with the movement of an object to the light emission control unit 34, and the light emission control unit 34 may change the light emission method based on the information.

[0035] FIG. 3 is a flowchart for explaining an example of the operation of the light emission control unit 34 in this embodiment. First, the light emission control unit 34 repeats the process from step Sa2 to step Sa5 for each remote terminal 50 (steps Sa1 and Sa6). In step Sa2, the light emission control unit 34 acquires display area information and attention level information from the remote terminal 50 of the repetition. Next, in step Sa3, the light emission control unit 34 determines which LED of the ring light 20 to emit light on the basis of the display area information acquired in step Sa2. Next, in step Sa4, the light emission control unit 34 determines the color to emit light on the basis of the remote terminal 50 of the repetition or its remote user. Next, in step Sa5, the light emission control unit 34 determines the light emission method of the LED based on at least the attention level information. Note that when determining the light emission method, either the width of the area indicated by the display area information or information indicating whether the area is changed in accordance with the movement of an object, or both, may be further used.

[0036] After repeating the processes from step Sa2 to step Sa5 for all remote terminals 50, the light emission control unit 34 determines whether there is any overlap in the LEDs to be illuminated determined in step Sa3 (step Sa7). If it is determined that there is any overlap (step Sa7-Yes), the light emission control unit 34 changes the light emission method and color of the overlapping LEDs (step Sa8). The changed light emission method and color may be a light emission method and color indicating an overlap, a light emission method for a case with a higher attention level, or a color obtained by combining the colors of the original overlapping LEDs.

[0037] Following step Sa8, and if it is determined in step S7 that there is no overlap (step Sa7-Yes), the light emission control unit 34 instructs the control microcontroller 40 to emit light from the LED of the ring light 20 in the light emission method and color determined up to that point (step Sa9).

[0038] FIG. 4 is a flowchart for explaining an example of the operation of the region extraction unit 53 in this embodiment. First, the region extraction unit 53 acquires the panoramic image received by the image receiving unit 52 (step Sb1). Next, the region extraction unit 53 acquires a designation of a region to be extracted, which is designated by a remote user (step Sb2). The designation is performed using the UI unit 54. For example, the UI unit 54 may display the panoramic image, and a region to be extracted from the displayed panoramic image may be designated using a pointing device such as a touch panel or a mouse provided in the UI unit 54. Alternatively, the UI unit 54 may display a part or the whole of the panoramic image, and a region to be extracted may be designated by swiping, pinching, or the like using a touch panel provided in the UI unit 54.

[0039] Next, the area extraction unit 53 judges whether or not the object, which is at least partially overlapping with the area to be extracted and designated in step Sb2, is moving (step Sb3). This judgment is made by comparing the previous image with the current image. For example, the area extraction unit 53 may use an object recognition technique to detect an area that has changed between the previous image and the current image, and judge whether or not the object is moving. The judgment in step Sb3 may be made only when it is determined that the omnidirectional camera 10 is stationary. If no judgment is made, it may be assumed that the object, which is at least partially overlapping with the area to be extracted, is not moving. The judgment whether or not the omnidirectional camera 10 is stationary may be made based on whether or not there is a change in the low-resolution omnidirectional image, or based on the position detected by a sensor provided in the omnidirectional camera 10.

[0040] If it is determined in step Sb3 that the object is moving (step Sb3-Yes), the area extraction unit 53 changes the area to be extracted in accordance with the movement of the object (step Sb4). After step Sb4, or if it is determined in step Sb3 that the object is not moving (step Sb3-No), the area extraction unit 53 extracts an image of the area to be extracted from the panoramic image (step Sb5). Next, the area extraction unit 53 outputs the display area information to the information transmission unit 57 and the attention determination unit 56 (step Sb6).

[0041] Next, the area extraction unit 53 determines whether or not the area to be extracted has been re-designated by the remote user (step Sb7). The method of re-designation is the same as the method of designation in step Sb2. If it is determined that there has been a re-designation (step Sb7-Yes), the area extraction unit 53 changes the area to be extracted in accordance with the re-designation (step Sb8). After step Sb8, or if it is determined in step Sb7 that there has been a re-designation (step Sb7-No), the area extraction unit 53 acquires a panoramic image newly received by the image receiving unit 52 (step Sb9) and returns to step Sb3.

[0042] FIG. 5 is a schematic diagram showing an example of the illumination of the ring light 20 in this embodiment. Each of the multiple rectangles in FIG. 5 is an LED of the ring light 20 arranged on a circumference. The example in FIG. 5 shows an example of the illumination of the ring light 20 when four remote users are using the remote terminal 50. The areas indicated by the arrows A, B, C, and D correspond to the areas extracted and displayed in the remote terminal 50 used by the first, second, third, and fourth remote users, respectively. The LED in the area indicated by the arrow A is illuminated in a color corresponding to the first remote user, and is illuminated in a lighting method according to the attention level of the first remote user. The LED in the area indicated by the arrow B is illuminated in a color corresponding to the second remote user, and is illuminated in a lighting method according to the attention level of the second remote user.

[0043] The area indicated by arrow A is smaller than the area indicated by arrow B, so the first remote user has a higher level of attention than the second remote user. Thus, the LEDs in the area indicated by arrow A are emitting light in a manner corresponding to a higher level of attention than the LEDs in the area indicated by arrow B.

[0044] The area indicated by arrow C and the area indicated by arrow D overlap. For this reason, the LEDs in the overlapping area emit light in a different manner or color than the LEDs in the non-overlapping parts of the areas indicated by arrows C and D. The LEDs in the overlapping area may emit light with a higher brightness than the other LEDs, or may emit light in a color that is a combination of the color of the LEDs in the non-overlapping parts of the area indicated by arrow C and the color of the LEDs in the non-overlapping parts of the area indicated by arrow D.

[0045] Fig. 6 is a diagram showing an example of a display image of the remote terminal 50 in this embodiment. Display image G1 in Fig. 6 is an example of a display image on the display of the UI unit 54 of the remote terminal 50. Display image G1 includes a full sky image G11, an extracted image G12, an image G13 of a local user, an image G14 of a first remote user (user X), and an image G15 of a second remote user (user Y).

[0046] The omnidirectional image G11 is an omnidirectional image captured by the omnidirectional camera 10. The extracted image G12 is an image of a region extracted from the omnidirectional image by the region extraction unit 53. A rectangle R11 in the omnidirectional image G11 indicates the position of the image G12 in the omnidirectional image G11. The mark R12 is a mark designated by any remote user as a location of interest. The color of the mark R12 may be a color corresponding to the designated remote user. The button B11 is a button for switching the display of the extracted image G12 and the image G13 of the local user. The image G13 of the local user is an image of a video chat, and is an image captured by the camera of the UI unit 62 of the local terminal 60.

[0047] The image G14 of the first remote user and the image G15 of the second remote user are images of a video chat, and are respectively images taken by the camera of the UI unit 54 of the remote terminal 50 of the first remote user and the camera of the UI unit 54 of the remote terminal 50 of the second remote user. The color of the frame of the image G14 of the first remote user and the title part displaying the user name are the color corresponding to the first remote user, and are the same as the color that lights up the LED of the ring light 20 corresponding to the first remote user. Similarly, the color of the frame of the image G15 of the second remote user and the title part displaying the user name are the color corresponding to the second remote user and are the same as the color that lights up the LED of the ring light 20 corresponding to the second remote user.

[0048] FIG. 7 is a schematic diagram for explaining a modified example of this embodiment. In the above-mentioned embodiment, an all-sky camera 10 having a 360-degree angle of view is used as the imaging unit, and a ring light 20 is used as the light-emitting unit. However, the angle of view of the imaging unit does not have to be 360 ​​degrees, and the light-emitting unit only needs to have a light-emitting area with a width according to the angle of view of the imaging unit. In this modified example, the angle of view Ra of the camera 10a, which is the imaging unit, is smaller than 360 degrees. Also, the line LED 20a, which is the light-emitting unit, has multiple LEDs arranged in a straight line.

[0049] The present invention may be embodied as follows. (1) One embodiment is a remote communication system comprising an imaging unit that captures an image, a light-emitting unit having a light-emitting area with a width corresponding to an angle of view of the imaging unit, an area extraction unit that extracts a portion of the image captured by the imaging unit, a display unit that displays an image of the portion of the area extracted by the area extraction unit, an attention determination unit that determines the attention level of a user viewing the image displayed by the display unit, and a light-emitting control unit that causes a portion of the light-emitting area corresponding to the portion of the area extracted by the area extraction unit to be illuminated in a light-emitting manner corresponding to the attention level determined by the attention determination unit.

[0050] This allows the local user to know the area that the remote user is paying attention to.

[0051] (2) Another embodiment is a remote communication system as described in (1), in which the light emission method according to the level of attention is a light emission method in which at least one of the brightness, color, blinking period, and blinking duty ratio differs depending on the level of attention.

[0052] This allows the local user to know the area that the remote user is paying attention to based on the lighting method.

[0053] (3) Another embodiment is a remote communication system described in (1) or (2), in which the attention determination unit determines the attention level based on any one of the size of the portion of the area extracted by the area extraction unit, the user's degree of attention to the display unit, or the user's vital signs.

[0054] This allows the remote communication system to determine whether the remote user is paying attention.

[0055] (4) Another embodiment is a remote communication system described in any one of (1) to (3), wherein the area extraction unit changes the extracted area in accordance with the movement of an object specified by the user.

[0056] This allows the remote communication system to continue displaying the object of interest, and further allows the local user to become aware of a remote user attempting to speak to the local user if the object of interest that is being displayed continuously is a local user.

[0057] (5) Another embodiment is the remote communication system described in (4), wherein the light emission control unit determines the light emission method based on the fact that the area extraction unit is changing the extracted area in accordance with the movement of an object specified by the user.

[0058] This allows the local user to know whether or not the light-emitting portion is being changed due to the movement of the object by the remote user's operation when the light-emitting portion is being changed due to the movement of the object.

[0059] (6) Yet another embodiment is a remote communication method comprising a first step of capturing an image, a second step of extracting a partial area from the image captured in the first step, a third step of displaying an image of the partial area extracted in the second step, a fourth step of determining a level of attention of a user viewing the image displayed in the third step, and a fifth step of illuminating a portion of an illumination area having a width corresponding to the angle of view of the image captured in the first step, the portion corresponding to the partial area extracted in the second step, in an illumination manner corresponding to the level of attention determined in the fourth step.

[0060] (7) Yet another embodiment is a relay server in a remote communication system including an imaging unit that captures an image, a light-emitting unit having a light-emitting area with a width corresponding to the angle of view of the imaging unit, an area extraction unit that extracts a portion of the image captured by the imaging unit, a display unit that displays an image of the portion of the area extracted by the area extraction unit, and an attention determination unit that determines the attention level of a user viewing the image displayed by the display unit, and the relay server includes an illumination control unit that causes a portion of the light-emitting area corresponding to the portion of the area extracted by the area extraction unit to be illuminated in an illumination manner corresponding to the attention level determined by the attention determination unit.

[0061] (8) Yet another embodiment is a program for causing a computer in a remote communication system including an imaging unit that captures an image, a light-emitting unit having a light-emitting area with a width corresponding to the angle of view of the imaging unit, an area extraction unit that extracts a partial area from the image captured by the imaging unit, a display unit that displays an image of the partial area extracted by the area extraction unit, and an attention determination unit that determines the attention level of a user viewing the image displayed by the display unit, to function as an illumination control unit that causes a portion of the light-emitting area corresponding to the partial area extracted by the area extraction unit to emit light in a lighting manner corresponding to the attention level determined by the attention determination unit.

[0062] 1 may be recorded on a computer-readable recording medium, and the programs recorded on the recording medium may be read into a computer system and executed to realize the relay server 30, the control microcomputer 40, the remote terminal 50, and the local terminal 60. Note that the term "computer system" here includes hardware such as an OS and peripheral devices.

[0063] In addition, if a WWW system is used, the "computer system" also includes the homepage providing environment (or display environment). In addition, "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. Furthermore, "computer-readable recording medium" also includes those that dynamically hold a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, and those that hold a program for a certain period of time, such as volatile memory inside a computer system that serves as a server or client in such cases. Furthermore, the above program may be one that realizes part of the above-mentioned functions, or may be one that can realize the above-mentioned functions in combination with a program already recorded in the computer system. [Explanation of symbols]

[0064] 1 Remote communication system 10 Panoramic Camera 10a Camera 20 Ring Light 20a Line LED 30 Relay Server 31 Video chat control section 32 Image acquisition section 33 Image transmission unit 34 Light emission control unit 40 Control microcomputer 50 Remote Terminal 51 Video chat processing unit 52 Image receiving unit 53 Region extraction part 54 UI section 55 Vital Sensor Unit 56 Attention Degree Judgment Unit 57 Information Transmission Unit 60 Local Terminal 61 Video chat processing unit 62 UI section

Claims

1. An imaging unit that captures an image; a light emitting unit having a light emitting region having a width corresponding to an angle of view of the imaging unit; an area extraction unit that extracts a partial area from the image captured by the imaging unit; a display unit that displays an image of the partial area extracted by the area extraction unit; an attention level determination unit that determines an attention level of a user viewing an image displayed by the display unit; a light emission control unit that causes a portion of the light emission area corresponding to the part of the area extracted by the area extraction unit to emit light in a light emission method according to the attention level determined by the attention level determination unit; A remote communication system comprising:

2. The remote communication system according to claim 1 , wherein the light emission method according to the attention level is a light emission method in which at least one of brightness, color, blinking cycle, and blinking duty ratio is different depending on the attention level.

3. The remote communication system of claim 1 or claim 2, wherein the attention level determination unit determines the attention level based on any one of the size of the portion of the area extracted by the area extraction unit, the user's degree of attention to the display unit, and the user's vital signs.

4. The remote communication system according to claim 1 , wherein the area extraction unit changes the area to be extracted in accordance with a movement of the object designated by the user.

5. The remote communication system according to claim 4 , wherein the light emission control unit determines the light emission method based on the fact that the area extraction unit changes the extracted area in accordance with a movement of the object designated by the user.

6. A first step of capturing an image; a second step of extracting a partial area from the image captured in the first step; a third step of displaying an image of the partial area extracted in the second step; a fourth step of determining a degree of attention of a user viewing the image displayed in the third step; a fifth step of lighting a portion of the light emitting area having a width corresponding to an angle of view of the image captured in the first step, the portion corresponding to the partial area extracted in the second step, in a lighting method corresponding to the attention level determined in the fourth step; A remote communication method comprising:

7. An imaging unit that captures an image; a light emitting unit having a light emitting region having a width corresponding to an angle of view of the imaging unit; an area extraction unit that extracts a partial area from the image captured by the imaging unit; a display unit that displays an image of the partial area extracted by the area extraction unit; an attention level determination unit that determines an attention level of a user viewing an image displayed by the display unit; A relay server in a remote communication system comprising: a light emission control unit that causes a portion of the light emission area corresponding to the part of the area extracted by the area extraction unit to emit light in a light emission method according to the attention level determined by the attention level determination unit; A relay server comprising:

8. An imaging unit that captures an image; a light emitting unit having a light emitting region having a width corresponding to an angle of view of the imaging unit; an area extraction unit that extracts a partial area from the image captured by the imaging unit; a display unit that displays an image of the partial area extracted by the area extraction unit; an attention level determination unit that determines an attention level of a user viewing an image displayed by the display unit; A computer in a remote communication system comprising: a light emission control unit that causes a portion of the light emission area corresponding to the part of the area extracted by the area extraction unit to emit light in a light emission method according to the attention level determined by the attention level determination unit; A program to function as a

Images