Information presentation method, information presentation system, and program

Abstract

An information presentation method according to an embodiment of the present invention includes: displaying a voice waveform of a voice acquired by a remote operation device on a display screen (S6); bringing a robot to a first distance from a person being addressed in a state where the voice waveform is displayed on the display screen (S7); displaying a real video image of an operator of the remote operation device on the display screen (S8); moving the robot to a position, at which an angle relative to the person being addressed is a first angle, in a state where the real video image is displayed on the display screen (S9); displaying an avatar of the operator on the display screen (S10); and bringing the robot from the position to a second distance from the person being addressed in a state where the avatar is displayed on the display screen, the second distance being shorter than the first distance (S11).

Description

Information presentation method, information presentation system, and program

[0001] One aspect of the present invention relates to an information presentation method, an information presentation system, and a program for remote communication.

[0002] In recent years, various human activities that were previously conducted face-to-face or in close proximity are rapidly becoming remote. For example, hybrid work has become established as a working system in offices. Hybrid work is a work style that combines traditional office work in the office with remote work from home or in a shared office. Remote communication is also becoming more common in commercial stores and facilities.

[0003] Telepresence robots are attracting attention as a remote communication tool. Telepresence robots are robots that combine video conferencing functions with remote control technology. Some telepresence robots have mobile capabilities. By using a physical device such as a telepresence robot, a caller who is face-to-face with the robot can more easily sense the presence of a caller operating the telepresence robot in a remote location.

[0004] For example, Non-Patent Document 1 shows an example of using a telepresence robot in an office space, and Non-Patent Document 2 shows an example of using a telepresence robot in a commercial space (store).

[0005] Maral Muratbekova-Touron, Emmanuelle Leon, "Is there anybody out there? Using a telepresence robot to engage in face time at the office," Information Technology & People, 2023, Vol. 36, No. 1, pp. 48-65; Kazuyoshi Wada, "Robotics in Retail Spaces," Journal of the Japan Society of Mechanical Engineers, May 2021, Vol. 124, No. 1230, pp. 34-35; Hiroyuki Hito, Daisuke Taniyama, and Hitomi Inagaki, "Consumer Acceptance and Challenges Regarding the Introduction of Robots and AI Technology: Implications from a Survey in Japan, the United States, and Germany," Intellectual Asset Creation, May 2016, pp. 108-125

[0006] In remote communication, when conversing with someone you've never met before via a telepresence robot, there is a stronger psychological resistance than when conversing with someone you know well. For example, compared to an office space where people know each other, in a commercial space where staff and visitors are meeting for the first time, conversations via a telepresence robot are more likely to cause psychological resistance and communication is less likely to be accepted. For example, Non-Patent Document 3 presents the results of a survey on acceptance of robots.

[0007] The present invention has been made in view of these points, and aims to reduce psychological resistance to conversation with a robot in an information presentation method for remote communication using a robot.

[0008] One aspect of the information presentation method is an information presentation method using an information presentation system that includes a robot that includes a moving mechanism, a display screen, and a video call function, and a remote control device for the robot, and includes the steps of: displaying an audio waveform of audio acquired by the remote control device on the display screen; bringing the robot closer to a person being spoken to within a first distance while the audio waveform is displayed on the display screen; displaying a live-action video of the operator of the remote control device on the display screen; moving the robot to a position where its angle with respect to the person being spoken to is a first angle while the live-action video is displayed on the display screen; displaying an avatar of the operator on the display screen; and bringing the robot closer to a person being spoken to within a second distance from the position where the avatar is displayed on the display screen, the second distance being shorter than the first distance.

[0009] According to an embodiment, in remote communication using a telepresence robot having a screen display function and a movement function, the information presentation method can control the display screen and the approach method to a person when the telepresence robot approaches a person to attempt a conversation, thereby improving the acceptance of the telepresence robot by the person being spoken to by the telepresence robot.

[0010] FIG. 1 is a block diagram showing an example of the overall configuration of an information presentation system according to an embodiment. FIG. 2 is a block diagram showing an example of the circuit configuration of a remote control device included in the information presentation system according to an embodiment. FIG. 3 is a block diagram showing an example of the circuit configuration of a telepresence robot included in the information presentation system according to an embodiment. FIG. 4 is a block diagram showing an example of the functional configuration of the information presentation system according to an embodiment. FIG. 5 is a diagram showing a specific example of a screen display on a display screen of a remote control device included in the information presentation system according to an embodiment. FIG. 6 is a diagram showing a specific example of a screen display on a display screen of a telepresence robot included in the information presentation system according to an embodiment. FIG. 7 is a diagram showing a specific example of a direction in which a telepresence robot included in the information presentation system according to an embodiment approaches a local speaker. FIG. 8 is a diagram showing a specific example of a moving direction of a telepresence robot included in the information presentation system according to an embodiment and a screen display on a display screen when the telepresence robot approaches a local speaker. FIG. 9 is a flowchart showing the overall flow of an information presentation method in an information presentation system according to an embodiment.

[0011] Hereinafter, embodiments will be described with reference to the drawings. In the following description, components having the same functions and configurations will be given the same reference numerals.

[0012] 1. Overall Configuration of Information Presentation System First, an example of the overall configuration of an information presentation system 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the overall configuration of the information presentation system 1.

[0013] 1 , for example, an information presentation system 1 includes a remote control device 10 and a telepresence robot 20 (hereinafter simply referred to as a "robot") that can communicate with the remote control device 10. In the following description, the location where the remote control device 10 is operated will be referred to as a "remote location." Furthermore, the location where the telepresence robot 20 is operating will be referred to as an "on-site location."

[0014] The remote control device 10 is, for example, a personal computer. The remote control device 10 may be a tablet terminal such as a smartphone, or may be another computer device. The remote control device 10 has a function for remotely controlling the telepresence robot 20. The remote control device 10 also has a function for performing a video call (video chat) with the telepresence robot 20. For example, a video of the local area is displayed on the display screen 1000 of the remote control device 10. A person operating the remote control device 10 (hereinafter referred to as "operator HM1") may operate the telepresence robot 20 via the remote control device 10. For example, the operator HM1 may operate the telepresence robot 20 to search for a local person to talk to or to move the telepresence robot 20 toward the person to talk to. Alternatively, the operator HM1 may use the video call function between the remote control device 10 and the telepresence robot 20 to talk to a person to talk to who is facing the telepresence robot 20 (hereinafter referred to as "local speaker HM2").

[0015] The telepresence robot 20 includes a moving mechanism 21, a support 22, and a video call unit 23. Note that the telepresence robot 20 may be a mobile humanoid robot.

[0016] The movement mechanism 21 moves the telepresence robot 20 based on operation by the remote control device 10. The movement mechanism 21 includes a base, wheels, a wheel drive (rotation) motor, etc. The wheel drive motor (not shown) is installed inside the base. In the example of FIG. 1 , two wheels are provided on the sides (left and right) of the cylindrical base. Separate wheel drive motors are attached to each of the two wheels. The two wheels are controlled independently of each other. By controlling the rotation of the left and right wheels, the telepresence robot 20 can run and turn. Note that the configuration and arrangement of the base and wheels are not limited to this. For example, the base may be a rectangular parallelepiped. Furthermore, three or more wheels may be provided.

[0017] The support 22 is provided on the moving mechanism 21. A video call unit 23 is provided on the support 22. The support 22 may be provided with an extension mechanism, and may also be provided with a rotation mechanism for rotating the video call unit 23 (display screen 2000). For example, the telepresence robot 20 can adjust the length of the support 22 to match the height of the local speaker HM2. Furthermore, the telepresence robot 20 can change the orientation of the video call unit 23 (display screen 2000) depending on the traveling direction of the moving mechanism 21 and the positional relationship with the local speaker HM2.

[0018] The video call unit 23 is used for video calls. The video call unit 23 includes a display screen 2000. The display screen 2000 displays an image of a remote location (an image of the operator HM1).

[0019] 1.1 Circuit Configuration of Remote Control Device Next, an example of the circuit configuration of the remote control device 10 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the circuit configuration of the remote control device 10.

[0020] 2, the remote control device 10 includes a control circuit 11, a memory 12, an input unit 13, a presentation unit 14, a camera 15, a microphone 16, a speaker 17, and a communication unit 18. These are connected to each other so as to be able to transmit and receive information to and from each other.

[0021] The control circuit 11 is, for example, a CPU (Central Processing Unit). The control circuit 11 controls the entire remote control device 10. The control circuit 11 executes a program stored in a program storage unit 121 of the memory 12.

[0022] The memory 12 includes a program storage unit 121 and a data storage unit 122. The memory 12 may include various storage devices as external storage, such as a magnetic disk storage device (HDD: Hard Disk Drive) or a solid state drive (SSD: Solid State Drive).

[0023] The program storage unit 121 may be configured as a non-transitory storage medium, for example, a non-volatile memory such as a read-only memory (ROM). The program storage unit 121 stores programs used to execute the information presentation system 1. For example, the programs include an operation program for the telepresence robot 20 and a video call application for use with the telepresence robot 20.

[0024] The data storage unit 122 may be configured as a storage medium, for example, a volatile memory such as a RAM (Random Access Memory). The data storage unit 122 stores data necessary for executing the information presentation system 1.

[0025] The input unit 13 includes input devices such as a keyboard, a mouse, a touch panel, or button switches. The operator HM1 can operate the telepresence robot 20 using the input devices.

[0026] The presentation unit 14 includes a display screen 1000 (e.g., an LCD (Liquid Crystal Display) or an EL (Electroluminescence) display) as a display device. For example, the presentation unit 14 displays an image captured on-site (hereinafter referred to as "on-site image") on the display screen 1000 based on the control of the control circuit 11.

[0027] The camera 15 is used to capture images of a remote location, i.e., to capture images of the operator HM1, based on the control of the control circuit 11. Hereinafter, images captured by the camera 15 at a remote location will be referred to as "remote location images."

[0028] The microphone 16 is used, under the control of the control circuit 11, to acquire voice from a remote location (hereinafter referred to as "remote voice"), that is, the voice of the operator HM1.

[0029] The speaker 17 is used to play back audio captured locally (hereinafter referred to as "local audio") under the control of the control circuit 11.

[0030] The communication unit 18 is used for wireless communication with the telepresence robot 20. For example, the communication unit 18 transmits remote video and remote audio (video and audio of the operator HM1) and operation signals of the telepresence robot 20 to the telepresence robot 20. The communication unit 18 also receives local video and audio, as well as information acquired by the telepresence robot 20, from the telepresence robot 20.

[0031] The microphone 16 and the speaker 17 may be built into the remote control device 10 or may be earphones or headphones with a microphone connected to the remote control device 10 .

[0032] 1.2 Circuit Configuration of Telepresence Robot 20 Next, the circuit configuration of the telepresence robot 20 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the circuit configuration of the telepresence robot 20.

[0033] 3 , the telepresence robot 20 includes a control circuit 30, a memory 31, an input unit 32, a presentation unit 33, a camera 34, a microphone 35, a speaker 36, a communication unit 37, a sensor 38, and a drive control unit 39. These are connected to each other so that they can transmit and receive information. For example, the presentation unit 33, the camera 34, the microphone 35, and the speaker 36 are provided in the video call unit 23. The drive control unit 39 is provided in the movement mechanism 21. The control circuit 30, the memory 31, the communication unit 37, and the sensor 38 may be provided in the movement mechanism 21 or in the video call unit 23.

[0034] The control circuit 30 is, for example, a CPU. The control circuit 30 controls the entire telepresence robot 20. More specifically, the control circuit 30 controls the movement of the telepresence robot 20. The control circuit 30 also controls video calls with the operator HM1. The control circuit 30 executes a program stored in a program storage unit 41 of the memory 31.

[0035] The memory 31 includes a program storage unit 41 and a data storage unit 42. The memory 31 may include various storage devices such as a magnetic disk storage device (HDD) or a solid state drive (SSD) as external storage.

[0036] The program storage unit 41 may be configured as a non-transitory storage medium, for example, a non-volatile memory such as a read-only memory (ROM). The program storage unit 41 stores the execution program of the telepresence robot 20. For example, the program includes a movement control program for the telepresence robot 20, a video processing application, and a video call application for communication with the remote control device 10.

[0037] The data storage unit 42 may be configured as a storage medium, for example, a volatile memory such as a RAM (Random Access Memory). The data storage unit 42 stores data necessary for the operation of the telepresence robot 20.

[0038] The input unit 32 includes input devices such as a touch panel or button switches. For example, the local speaker HM2 may use these input devices to send and receive information to and from the operator HM1. These input devices may also be used during maintenance of the telepresence robot 20.

[0039] The presentation unit 33 includes a display screen 2000 (for example, an LCD or EL display) as a display device. For example, the presentation unit 33 presents a remote video (video of the operator HM1) based on the control of the control circuit 30. The video displayed on the presentation unit 33 is switched based on the positional relationship between the telepresence robot 20 and the local speaker HM2.

[0040] The camera 34 is used to capture local video including the local speaker HM2 under the control of the control circuit 30.

[0041] The microphone 35 is used to acquire local voices including the voice of the local speaker HM2 under the control of the control circuit 30.

[0042] The speaker 36 is used to reproduce the remote voice (voice of the operator HM1) based on the control of the control circuit 30.

[0043] The communication unit 37 is used for wireless communication with the remote control device 10. For example, the communication unit 37 transmits local video and audio, as well as information acquired by the telepresence robot 20, to the remote control device 10. The communication unit 37 also receives remote video and audio, as well as operation signals for the telepresence robot 20, from the remote control device 10.

[0044] The telepresence robot 20 includes multiple sensors 38. For example, the telepresence robot 20 includes a range sensor (laser sensor) or a depth sensor (infrared sensor) to measure the distance to a person or an obstacle. The telepresence robot 20 also includes a gyro sensor or an acceleration sensor to measure the inclination of the support pole 22, the position and movement speed of the telepresence robot 20, etc. Information from these sensors 38 is transmitted to the control circuit 30. For example, the control circuit 30 controls the movement mechanism 21 (drive control unit 39) based on the information from the sensors 38.

[0045] The drive control unit 39 controls the driving of the wheel drive motor of the movement mechanism 21 based on the control of the control circuit 30 .

[0046] 1.3 Functional Configuration of Information Presentation System Next, the functional configuration of the information presentation system 1 will be described with reference to Fig. 4. Fig. 4 is a block diagram showing an example of the functional configuration of the information presentation system 1.

[0047] First, the functional configuration of the remote control device 10 will be described.

[0048] 4, the remote control device 10 includes, as information presentation processing functions, an operation acquisition unit 111, a video acquisition unit 112, and a voice acquisition unit 113. The operation acquisition unit 111, the video acquisition unit 112, and the voice acquisition unit 113 are realized by causing the control circuit 11 to execute a program stored in the program storage unit 121. Note that part or all of the operation acquisition unit 111, the video acquisition unit 112, and the voice acquisition unit 113 may be realized using hardware such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit).

[0049] The operation acquisition unit 111 acquires operation instructions for operating the telepresence robot 20 from the operator HM1. More specifically, the operation acquisition unit 111 acquires local information from the local information presentation unit 304 of the telepresence robot 20. The local information includes local audio, local video, and information about people detected from the local video. The local information may also include position information of the telepresence robot 20 and distance information between the telepresence robot 20 and the detected people. The operation acquisition unit 111 displays the local information on the display screen 1000. The operator HM1 refers to the local information displayed on the display screen 1000 and instructs the telepresence robot 20 to operate. For example, if only one person is detected from the local video, the operator HM1 determines whether to select that person as a person to be spoken to, i.e., as a local speaker HM2. Furthermore, if multiple people are detected from the local video, the operator HM1 can select a local speaker HM2 from among them. The operation acquisition unit 111 generates an operation signal instructing the operator HM1 to approach the local speaker HM2 (person to be spoken to) selected by the operator HM1, and transmits the operation signal to the local information presentation unit 304.

[0050] The video acquisition unit 112 acquires a remote location video of, for example, the operator HM1 from the camera 15 of the remote operation device 10. The video acquisition unit 112 transmits the acquired remote location video to the remote location video acquisition unit 307 of the telepresence robot 20.

[0051] The voice acquisition unit 113 acquires the remote voice including the conversation of the operator HM1 from the microphone 16 of the remote operation device 10. The voice acquisition unit 113 transmits the acquired remote voice to the remote voice acquisition unit 308 of the telepresence robot 20.

[0052] Next, a description will be given of the functional configuration of the telepresence robot 20. The telepresence robot 20 includes, as information presentation processing functions, a local audio acquisition unit 301, a local video acquisition unit 302, a person information acquisition unit 303, a local information presentation unit 304, a movement control unit 305, a movement processing unit 306, a remote location video acquisition unit 307, a remote location audio acquisition unit 308, a remote location audio output unit 309, a display screen synthesis unit 310, and a screen display unit 311. The local audio acquisition unit 301, the local video acquisition unit 302, the person information acquisition unit 303, the local information presentation unit 304, the movement control unit 305, the movement processing unit 306, the remote location video acquisition unit 307, the remote location audio acquisition unit 308, the remote location audio output unit 309, the display screen synthesis unit 310, and the screen display unit 311 are realized by causing the control circuit 30 to execute a program stored in the program storage unit 41. In addition, some or all of the local audio acquisition unit 301, local video acquisition unit 302, person information acquisition unit 303, local information presentation unit 304, movement control unit 305, movement processing unit 306, remote video acquisition unit 307, remote audio acquisition unit 308, remote audio output unit 309, display screen synthesis unit 310, and screen display unit 311 may be realized using hardware such as an LSI or an ASIC.

[0053] The local voice acquisition unit 301 acquires local voice from the microphone 35. The local voice acquisition unit 301 transmits the acquired local voice to the local information presentation unit 304.

[0054] The local image acquisition unit 302 acquires local images from the camera 34. The local image acquisition unit 302 transmits the local images to the person information acquisition unit 303.

[0055] The person information acquisition unit 303 analyzes the local video to acquire person information. The person information includes information about a person area and the facial orientation of the person. A person area is an area in the local video where a person is determined to be captured. The person information acquisition unit 303 can detect multiple people from the local video. For example, YOLO (You Only Look Once) or the like may be used as an algorithm for person (object) detection. The person information acquisition unit 303 can also detect the facial orientation of a person in a person area by facial orientation detection. The person information acquisition unit 303 can also detect the direction of movement of the person. The person information acquisition unit 303 can detect the direction of the person's body (front direction) by detecting the facial orientation or the direction of movement. This person information is used by the operator HM1 to grasp the local situation and to process the movement of the telepresence robot 20. The person information acquisition unit 303 transmits the local video and person information to the local information presentation unit 304 and the movement processing unit 306.

[0056] The local information presentation unit 304 generates local information including local audio, local video, and person information. The local information presentation unit 304 presents (transmits) the local information to the operation acquisition unit 111 of the remote control device 10. The local information presentation unit 304 also transmits the local information and the operation signal received from the operation acquisition unit 111 to the movement control unit 305.

[0057] The movement control unit 305 controls the movement of the telepresence robot 20. More specifically, the movement control unit 305 recognizes the local speaker HM2 (the person to be spoken to) based on the local information and operation signals received from the local information presentation unit 304. The movement control unit 305 acquires, for example, from the sensor 38, the results of measuring the distance between the telepresence robot 20 and the local speaker HM2 (hereinafter referred to as the "face-to-face distance"). The movement control unit 305 transmits this information about the local speaker HM2 to the movement processing unit 306. The movement control unit 305 controls the movement of the telepresence robot 20 based on the movement instruction signal received from the movement processing unit 306. More specifically, the movement control unit 305 transmits a drive signal for the wheel drive motor of the movement mechanism 21 to the drive control unit 39.

[0058] When the telepresence robot 20 moves toward the local speaker HM2, the movement processing unit 306 selects a method of approaching the local speaker HM2 and the content of the screen display on the display screen 2000 based on the positional relationship between the telepresence robot 20 and the local speaker HM2. More specifically, the movement processing unit 306 receives information about the local speaker HM2 from the movement control unit 305. The movement processing unit 306 also receives local video and person information. The movement processing unit 306 calculates the positional relationship between the telepresence robot 20 and the local speaker HM2 from this information. The positional relationship information includes the face-to-face distance between the telepresence robot 20 and the local speaker HM2 and the angle at which the telepresence robot 20 is facing from the front of the local speaker HM2. The movement processing unit 306 determines a method of movement of the telepresence robot 20 and the content of the screen display to be presented on the display screen 2000 based on the positional relationship information. Details of the method of approaching the local speaker HM2 and the content of the screen display will be described later. Based on the approach method, the movement processing unit 306 determines the movement direction of the telepresence robot 20. The movement processing unit 306 transmits a movement instruction signal to the movement control unit 305 instructing the movement direction of the telepresence robot 20. The movement processing unit 306 also transmits a display instruction signal to the display screen synthesis unit 310 instructing the content of the screen display.

[0059] The remote location image acquisition unit 307 acquires a remote location image, i.e., an image of the operator HM1, from the image acquisition unit 112 of the remote operation device 10. The remote location image acquisition unit 307 transmits the remote location image to the display screen synthesis unit 310.

[0060] The remote voice acquisition unit 308 acquires the remote voice, i.e., the voice of the operator HM1, from the voice acquisition unit 113 of the remote operation device 10. The remote voice acquisition unit 308 transmits the remote voice to the remote voice output unit 309 and the display screen synthesis unit 310.

[0061] The remote location audio output unit 309 outputs the remote location audio to the speaker 36 of the telepresence robot 20 .

[0062] The display screen synthesis unit 310 receives remote video, i.e., an image of the operator HM1, from the remote video acquisition unit 307. The display screen synthesis unit 310 receives remote audio, i.e., the voice of the operator HM1, from the remote audio acquisition unit 308. The display screen synthesis unit 310 also receives a display instruction signal from the movement processing unit 306. The display screen synthesis unit 310 reads out screen display data 421 from the data storage unit 42 based on the display instruction signal. The display screen synthesis unit 310 synthesizes an image to be displayed on the display screen 2000 based on the video and audio of the operator HM1, the display instruction signal, and the screen display data 421.

[0063] The screen display unit 311 displays the video acquired from the display screen synthesis unit 310 as a remote location video on the display screen 2000 of the telepresence robot 20 .

[0064] 2. Specific Examples of Screen Displays in the Remote Control Device Next, a specific example of a screen display in the remote control device 10 will be described with reference to Fig. 5. Fig. 5 is a diagram showing a specific example of a screen display on the display screen 1000 of the remote control device 10.

[0065] As shown in FIG. 5, for example, a local video display window 1100, a remote video display window 1200, and a controller display window 1300 are displayed on a display screen 1000 of the presentation unit 14.

[0066] The local image display window 1100 displays local image and person information. In the example of FIG. 5, an image of a shopping mall is shown as the local image. In this image, the person area is indicated by frame FR. In the example of FIG. 5, three frames FRa, FRb, and FRc are shown. Note that one frame FR may contain multiple people. For example, frame FRa contains a woman. Frame FRb contains three parents and their children. Frame FRc contains a male-female duo. In each frame FR, the direction of the detected person's face (movement direction) is indicated by an arrow. In the example of FIG. 5, frame FRb, which is indicated by a double frame, is shown in a selected state.

[0067] The remote location video display window 1200 displays the video of the operator HM1.

[0068] Four arrow keys indicating up, down, left and right are displayed on the controller display window 1300. The controller display window 1300 is used to select the frame FR (local speaker HM2).

[0069] There are, for example, two methods for selecting a frame FR. One is to select a frame FR by operating (clicking) an arrow key in the controller display window 1300. The other is to select a frame FR by clicking within the target frame FR. For example, the remote control device 10 transmits an operation signal to the telepresence robot 20 instructing it to approach the selected local speaker HM2 (the person being spoken to).

[0070] 3. Specific Examples of Screen Displays in Telepresence Robot Next, a specific example of screen displays in the telepresence robot 20 will be described with reference to Fig. 6. Fig. 6 is a diagram showing a specific example of screen displays on the display screen 2000 of the telepresence robot 20.

[0071] As shown in FIG. 6 , for example, the display screen 2000 of the presentation unit 33 displays one of a live-action video of the operator HM1's face (remote video), a video of the operator HM1's avatar, and a video of the operator HM1's voice waveform (audio waveform of the remote voice). For example, the avatar may display facial elements such as the eyes, nose, and mouth. Note that the voice waveform does not need to be displayed as a circular waveform as shown in FIG. 6 as long as it presents the status of the operator HM1's speech. The display screen synthesis unit 310 synthesizes one of the videos based on a display instruction signal received from the movement processing unit 306. For example, the display screen synthesis unit 310 synthesizes an avatar based on the live-action video of the operator HM1 and the screen display data 421. The display screen synthesis unit 310 then changes the facial expression of the avatar based on the voice of the operator HM1.

[0072] The anthropomorphism of the images decreases in the order of live-action images, avatars, and audio waveforms. The information presentation system 1 can control the acceptance of the local speaker HM2 by displaying images with adjusted anthropomorphism on the display screen 2000 based on the positional relationship between the telepresence robot 20 and the local speaker HM2.

[0073] 4. Specific Examples of Directions in Which the Telepresence Robot Approaches the Local Speaker Next, a specific example of the direction in which the telepresence robot 20 approaches the local speaker HM2 will be described with reference to Fig. 7. Fig. 7 is a diagram showing a specific example of the direction in which the telepresence robot 20 approaches the local speaker HM2.

[0074] As shown in Figure 7, the telepresence robot 20 can control the direction of approach when approaching the local speaker HM2, i.e., the approach angle. For example, when the telepresence robot 20 approaches the local speaker from the front, the approach angle θ is set to 0°. When the telepresence robot 20 approaches the local speaker from diagonally to the right, the approach angle θ is set to 45°. When the telepresence robot 20 approaches the local speaker from the right side, the approach angle θ is set to 90°. The information presentation system 1 can control the sense of acceptance of the local speaker HM2 by adjusting the approach angle θ based on the positional relationship between the telepresence robot 20 and the local speaker HM2.

[0075] 5. Method of Approaching a Local Speaker Next, an example of a method of the telepresence robot 20 approaching the local speaker HM2 will be described with reference to Fig. 8. Fig. 8 is a diagram showing a specific example of the movement direction of the telepresence robot 20 and the screen display on the display screen 2000 when the telepresence robot 20 approaches the local speaker HM2.

[0076] As shown in Figure 8, when the telepresence robot 20 approaches the local speaker HM2, the movement direction of the telepresence robot 20 and the content displayed on the display screen 2000 are divided into, for example, three stages depending on the face-to-face distance D between the telepresence robot 20 and the local speaker HM2.

[0077] First, in the first stage, the face-to-face distance D between the telepresence robot 20 and the local speaker HM2 is set to a predetermined first distance d ｔｈ＿ａｐｐA longer (farther) stage than (D>d ｔｈ＿ａｐｐ ) The telepresence robot 20 approaches the local speaker HM2 from a relatively long distance. In the first stage, the approach angle θ is not limited. For example, if the telepresence robot 20 approaches the local speaker HM2 from directly in front of him (approach angle θ = 0°), the local speaker HM2 is likely to notice the presence of the telepresence robot 20. In the first stage, a voice waveform with a low anthropomorphic quality is displayed on the display screen 2000 so that the local speaker HM2 is not made aware that he is being spoken to. Displaying a voice waveform increases the local speaker HM2's acceptance of the voice waveform compared to when live-action video or an avatar is displayed.

[0078] In the second stage, the telepresence robot 20 moves at a first distance d ｔｈ＿ａｐｐ The stage where the speaker has approached the local speaker HM2 (D = d ｔｈ＿ａｐｐ ) In the second stage, the telepresence robot 20 adjusts the approach angle θ. For example, the approach angle θ is preferably 30°≦θ≦60° (right diagonal) or −30°≦θ≦−60° (left diagonal). For example, by positioning the telepresence robot 20 diagonally to the right or left relative to the local speaker HM2, the psychological resistance of the local speaker HM2 to the telepresence robot 20 is reduced. At this time, the telepresence robot 20 adjusts the approach angle θ when the face-to-face distance D is equal to or less than the first distance d ｔｈ＿ａｐｐ For example, the telepresence robot 20 moves within a radius d ｔｈ＿ａｐｐ In the second stage, an actual video image of the operator HM1 is displayed on the display screen 2000 so as not to induce any psychological resistance from the local speaker HM2.

[0079] In the third stage, the telepresence robot 20 moves to the position (first distance d ｔｈ＿ａｐｐ and the approach angle θ) to the second distance d ｔｈ＿ｆｉｎ When approaching to (d ｔｈ＿ａｐｐ >D≧d ｔｈ＿ｆｉｎ ) The second distance d ｔｈ＿ｆｉｎ is the first distance d ｔｈ＿ａｐｐ The telepresence robot 20 is positioned at a second distance d ｔｈ＿ｆｉｎWhen it approaches the target position, it stops moving. Because the telepresence robot 20 approaches from an oblique position, the local speaker HM2 feels less psychological resistance. In the third stage, an avatar with a medium level of anthropomorphism is displayed on the display screen 2000 in order to reduce the local speaker HM2's psychological resistance to the telepresence robot 20. The telepresence robot 20 speaks to the local speaker HM2 with the avatar displayed on the display screen 2000. When an avatar is displayed, the local speaker HM2 is more receptive to it than when live video or audio waveforms are displayed.

[0080] The moving speed of the telepresence robot 20 may be different in each stage. For example, the moving speed in the third stage may be slower than the first and second stages.

[0081] 6. Overall Flow of Information Presentation Method Next, an example of the flow of the information presentation method will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the overall flow of the information presentation method.

[0082] 9, first, the telepresence robot 20 acquires the local conditions (S1). More specifically, the local audio acquisition unit 301 acquires local audio. The local video acquisition unit 302 acquires local video.

[0083] The person information acquisition unit 303 acquires person information (information such as person area and face direction) from the on-site video (S2).

[0084] The local information presentation unit 304 presents the local information (local audio, local video, and person information) to the operation acquisition unit 111 of the remote control device 10 (S3).

[0085] The local information presentation unit 304 acquires an operation signal from the remote operation device 10 (S4). More specifically, local information is displayed on the display screen 1000 of the remote operation device 10. The operator HM1 refers to the display screen 1000 and selects a local speaker HM2. The operation acquisition unit 111 transmits an operation signal for the telepresence robot 20 to the local information presentation unit 304 based on the selection result of the operator HM1.

[0086] The telepresence robot 20 starts moving the telepresence robot 20 and displaying the screen 2000 based on the operation signal (S5). More specifically, the movement processing unit 306 checks the positional relationship between the telepresence robot 20 and the local speaker HM2. If the face-to-face distance D is equal to or smaller than the first distance d ｔｈ＿ａｐｐ If it is longer, the movement processing unit 306 selects the first stage of the approach method. The movement processing unit 306 transmits a movement instruction signal to the movement control unit 305 and transmits a display instruction signal to the display screen synthesis unit 310.

[0087] The display screen synthesis unit 310 synthesizes the audio waveform when it receives a display instruction signal for the first stage screen display from the movement processing unit 306. The screen display unit 311 displays the audio waveform on the display screen 2000 (S6).

[0088] When the movement control unit 305 receives the first-stage movement instruction signal from the movement processing unit 306, the movement control unit 305 moves the telepresence robot 20 to the first distance d ｔｈ＿ａｐｐ That is, in the first stage, the telepresence robot 20 moves to the first distance d while displaying the audio waveform on the display screen 2000 (S7). ｔｈ＿ａｐｐ Approach the local speaker HM2 until

[0089] When the first stage of the approach method is completed, the movement processing unit 306 moves to the second stage of the approach method. When the display screen composition unit 310 receives a display instruction signal for the second stage screen display from the movement processing unit 306, it selects the live-action video of the operator HM1. The screen display unit 311 displays the live-action video of the operator HM1 on the display screen 2000 (S8).

[0090] When the movement control unit 305 receives the second-stage movement instruction signal from the movement processing unit 306, it moves the telepresence robot 20 to a position where the approach angle θ is 30°≦θ≦60° (right diagonal) or −30°≦θ≦−60° (left diagonal) (S9). That is, in the second stage, the telepresence robot 20 moves to the preset approach angle θ while displaying the live-action video of the operator HM1 on the display screen 2000.

[0091] When the second stage of the approach method is completed, the movement processing unit 306 moves to the third stage of the approach method. When the display screen synthesis unit 310 receives a display instruction signal for the third stage screen display from the movement processing unit 306, it synthesizes an avatar of the operator HM1 based on the live video and audio of the operator HM1. The screen display unit 311 displays the avatar of the operator HM1 on the display screen 2000 (S10).

[0092] When the movement control unit 305 receives the third-stage movement instruction signal from the movement processing unit 306, the movement control unit 305 moves the telepresence robot 20 to a second distance d from the position to which the telepresence robot 20 was moved in the second stage. ｔｈ＿ｆｉｎ That is, in the third stage, the telepresence robot 20 moves the avatar to the second distance d while displaying the avatar on the display screen 2000 (S11). ｔｈ＿ｆｉｎ Approach the local speaker HM2 until

[0093] The operator HM1 speaks to the local speaker HM2 via the telepresence robot 20 (S12). Note that the operator HM1 may start speaking to the local speaker HM2 from the first stage or the second stage.

[0094] 7. Effects of this Embodiment With the configuration according to this embodiment, the information presentation system 1 can switch the moving direction of the telepresence robot 20 and the contents displayed on the display screen 2000 according to the distance between the telepresence robot 20 and the local speaker HM2 when the telepresence robot 20 approaches the local speaker HM2. For example, in the first stage, the telepresence robot 20 displays a voice waveform on the display screen 2000 while moving at a first distance d ｔｈ＿ａｐｐ In the second stage, the telepresence robot 20 can move to a position where 30°≦θ≦60° (right diagonal) or −30°≦θ≦−60° (left diagonal) while displaying a live-action video of the operator HM1 on the display screen 2000. In the third stage, the telepresence robot 20 can move to a position where 30°≦θ≦60° (right diagonal) or −30°≦θ≦−60° (left diagonal) while displaying an avatar on the display screen 2000. ｔｈ＿ｆｉｎ The telepresence robot 20 can approach the local speaker HM2 by the second distance d ｔｈ＿ｆｉｎFurthermore, the local speaker HM2 can be spoken to at a position where 0°≦θ≦60° (diagonal right angle) or -30°≦θ≦-60° (diagonal left angle). Depending on the positional relationship between the telepresence robot 20 and the local speaker HM2, a movement direction and screen display content that poses relatively little psychological resistance to the local speaker HM2 can be selected. This improves the local speaker HM2's acceptance of the telepresence robot 20 when spoken to by the telepresence robot 20.

[0095] 8. Others Various modifications can be applied to the above-described embodiment.

[0096] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in combination as appropriate as possible, and in such cases, the combined effects can be obtained. Furthermore, the above-described embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining the disclosed multiple constituent elements.

[0097] 1...Information presentation system 10...Remote control device 11...Control circuit 12...Memory 13...Input unit 14...Presentation unit 15...Camera 16...Microphone 17...Speaker 18...Communication unit 20...Telepresence robot 21...Movement mechanism 22...Support 23...Video call unit 30...Control circuit 31...Memory 32...Input unit 33...Presentation unit 34...Camera 35...Microphone 36...Speaker 37...Communication unit 38...Sensor 39...Drive control unit 41...Program memory unit 42...Data memory unit 111...Operation acquisition unit 112...Video acquisition unit 113...Audio acquisition unit 121...Program memory unit 122...Data memory unit 301...Local audio acquisition unit 302...Local video acquisition unit 303...Person information acquisition unit 304...Local information presentation unit 305...Movement control unit 306...Movement processing unit 307...Remote location video acquisition unit 308...Remote location audio acquisition unit 309...Remote location audio output unit 310...Display screen synthesis unit 311...Screen display unit 421...Screen display data 1000...Display screen 1100...Local location video display window 1200...Remote location video display window 1300...Controller display window 2000...Display screen HM1...Operator HM2...Local speaker

Claims

1. A method of presenting information using an information presentation system including a robot having a moving mechanism, a display screen, and a video calling function, and a remote control device for the robot, the method comprising: displaying an audio waveform of audio acquired by the remote control device on the display screen; moving the robot closer to a person to be spoken to to a first distance while the audio waveform is displayed on the display screen; displaying a live-action video of the operator of the remote control device on the display screen; moving the robot to a position where its angle with respect to the person to be spoken to is a first angle while the live-action video is displayed on the display screen; displaying an avatar of the operator on the display screen; and moving the robot closer to the person to be spoken to to a second distance shorter than the first distance while the avatar is displayed on the display screen.

2. The information presentation method according to claim 1, wherein the first angle is greater than or equal to 30° and less than or equal to 60°, or greater than or equal to -30° and less than or equal to -60°.

3. The information presentation method according to claim 1, further comprising: capturing on-site video of the robot in operation; acquiring personal information from the on-site video; and presenting the on-site video and the personal information to the remote control device.

4. The information presentation method according to claim 3, wherein acquiring the person information includes: detecting a person area from the on-site image; and detecting a facial direction of the person within the person area.

5. The information presentation method described in claim 1, wherein displaying the avatar includes: receiving video and audio of the operator from the remote control device; and synthesizing the avatar based on the video and audio of the operator.

6. The information presentation method according to claim 1, further comprising the step of: said robot receiving, from said remote control device, an operation signal based on the result of said operator selecting said target person.

7. An information presentation system comprising: a robot including a moving mechanism and a display screen and having a video call function; and a remote control device for the robot, wherein, when the distance between the robot and a person to be spoken to is longer than a first distance, the robot approaches the person to be spoken to with an audio waveform of audio acquired by the remote control device displayed on the display screen; when the distance between the robot and the person to be spoken to is the first distance, the robot moves to a position where its angle with respect to the person to be spoken to is a first angle with a live-action image of the operator of the remote control device displayed on the display screen; and, after moving to the position, the robot approaches the person to a second distance shorter than the first distance from the position with an avatar of the operator displayed on the display screen.

8. A program for causing a computer provided in an information presentation system to execute the processing of the information presentation method according to any one of claims 1 to 6.

Images