Information processor, display control method and display control program in information processor

Abstract

To provide an information processor communicable with a plurality of terminal devices, which achieves suitable and smooth reception for a user using each of the terminal devices.SOLUTION: An on-line reception system (10) enables an operator using an operator terminal (20) as an information processor to remotely respond to a user using each of M (M; natural number of 2 or greater) user terminals (30) as M terminal devices. An operation screen 600 is displayed on the operation terminal. The operation screen 600 includes an enlarged room image 602 corresponding to a user terminal as an opposite party of the reception, and M reduced room images 606 corresponding to N (N; natural number of less than M) user terminals. The M reduced room images 606 are displayed in a vertical row, and the arrangement accords with importance of the user terminal corresponding to each of the M reduced room images 606.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a display control method in the information processing device, and a display control program, and more particularly to an information processing device capable of communicating with a plurality of terminal devices, a display control method in the information processing device, and a display control program. [Background technology]

[0002] An example of this type of technology is disclosed in Patent Document 1. The technology disclosed in Patent Document 1 is applied to an ordering system for a restaurant or other establishment. According to the technology disclosed in Patent Document 1, multiple operator terminals serving as information processing devices, multiple tabletop terminals, and a control center device responsible for managing each of these operator terminals and tabletop terminals are connected via a network. Each operator terminal is used by an operator who is a store staff member, specifically an operator fluent in a language other than the language used in the store, and is installed, for example, in an operator room or the operator's home. Each tabletop terminal is installed at a table in the store and used by a customer seated at that table. Although Patent Document 1 does not specify the installation location of the control center device, it is presumed that the control center device is installed in an appropriate location such as a management center. When a tabletop terminal requests assistance in a language other than the language used in the store, the control center device transmits a response request to an operator terminal of an operator who can respond in that language. If the operator terminal responds that the operator terminal can respond, the control center device initiates a link between the operator terminal and the tabletop terminal.

[0003] The operator terminal is, for example, a computer (strictly speaking, a personal computer), and when linking with one of the table terminals is initiated, an operation screen is displayed on the display unit. On this operation screen, an image representing the linked table terminal, particularly an image captured by a camera equipped in the table terminal, is enlarged, and to the side of the enlarged image, an image representing the table terminal waiting for an operator to respond is displayed in a reduced size. Patent Document 1 (e.g., FIG. 19) shows an example in which three reduced images (i.e., images representing table terminals in a standby state) are arranged vertically to the right of the enlarged image (i.e., the image representing the linked table terminal). On this operation screen, the operator can attend to customers while viewing the enlarged image, and can also recognize the three standby table terminals from the three reduced images. Therefore, after finishing serving the current customer, the operator selects one of the three standby table terminals and begins serving the customer at that table terminal. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-151848 Summary of the Invention [Problem to be solved by the invention]

[0005] Thus, with the technology disclosed in Patent Document 1, the operator can recognize three tabletop terminals in standby mode from the three screens displayed in reduced size on the operation screen, but cannot determine which of the three tabletop terminals is the most appropriate to select. For example, there is a view that, of three tabletop terminals in standby mode, the one with the longest waiting time, which is the elapsed time from the time the terminal entered standby mode, should be selected with the highest priority, or in other words, has the highest importance (priority). However, with the technology disclosed in Patent Document 1, the operator cannot make such a judgment from the three screens displayed in reduced size on the operation screen. This makes it impossible to provide accurate and smooth service to users (customers) of the tabletop terminals.

[0006] Therefore, the present invention aims to provide a new technology in an information processing device capable of communicating with multiple terminal devices, a display control method in an information processing device, and a display control program, which can provide accurate and smooth responses to users of each terminal device. [Means for solving the problem]

[0007] To achieve this object, the present invention includes a first invention relating to an information processing device, a second invention relating to a display control method in an information processing device, and a third invention relating to a display control program in an information processing device.

[0008] A first aspect of the present invention relates to an information processing device capable of communicating with M (M is a natural number equal to or greater than 2) terminal devices, and includes a display means, an operation accepting means, and a display control means. The display means is a type of information output means and outputs various information in a visual form, for example, a screen. The operation accepting means can accept operations by an operator of the information processing device. The display control means causes the display means to display various screens. In particular, when the operation accepting means accepts an operation instructing the selection of any one of the M terminal devices as a communication partner, the display control means causes the display means to display an operation screen. This operation screen includes a communication partner image corresponding to the communication partner and N first standby terminal images corresponding to N (N is a natural number less than M) of the M terminal devices. The N first standby terminal images are displayed in a manner according to the importance of each of the N terminal devices corresponding to the N first standby terminal images.

[0009] It is desirable that the N terminal devices are selected from the M terminal devices in descending order of importance.

[0010] Furthermore, it is appropriate that the N terminal devices do not include the communication partner.

[0011] Furthermore, it is desirable that the value of N, that is, the number of first standby terminal images included on the operation screen, can be set arbitrarily.

[0012] In addition, each terminal device may be provided with a camera that can capture an image of the user using the terminal device. In this case, it is desirable that the communication partner image includes an image captured by the camera provided in the communication partner.

[0013] Additionally, it is preferable that each of the N first standby terminal images also includes an image captured by a camera provided in the corresponding terminal device.

[0014] It is also desirable that the communication partner image be displayed larger than each of the N first standby terminal images, and that each of the N first standby terminal images be displayed in a more conspicuous manner the higher the importance of the terminal device corresponding to it.

[0015] For example, the N first standby terminal images may be displayed in a vertical row to the side of the communication partner image. In this case, the N first standby terminal images may be displayed higher in the order of importance of the terminal device corresponding to the first standby terminal image.

[0016] The importance level referred to here may be determined based on some or all of the waiting time, which is the elapsed time from the point at which the user of each terminal device wishes to become a communication partner, the behavior of the user, and the inherent importance level, which is an inherent importance level preset for each terminal device.

[0017] Furthermore, the display control means may cause the display means to display a standby screen when the aforementioned communication partner is in an unselected state. This standby screen includes P second standby terminal images corresponding to P (P; a natural number greater than N and equal to or less than M) of the M terminal devices. The P second standby terminal images are displayed in a manner according to the importance of each of the P terminal devices corresponding to the P second standby terminal images.

[0018] A display control method for an information processing device according to a second aspect of the present invention is based on the premise that the information processing device is capable of communicating with M (M is a natural number equal to or greater than 2) terminal devices and includes a display means and an operation receiving means. Here, the display means is a type of information output means and outputs various information in a visual form, for example, a screen. The operation receiving means can receive operations from an operator of the information processing device. Furthermore, the second aspect of the present invention includes a display control step. In this display control step, various screens are displayed on the display means. In particular, in the display control step, when the operation receiving means receives an operation instructing the selection of any one of the M terminal devices as a communication partner, the display means is caused to display an operation screen. This operation screen includes a communication partner image corresponding to the communication partner and N first standby terminal images corresponding to N (N is a natural number less than M) of the M terminal devices. The N first standby terminal images are displayed in a manner according to the importance of each of the N terminal devices corresponding to the N first standby terminal images.

[0019] A display control program for an information processing device according to a third aspect of the present invention is based on the premise that the information processing device is capable of communicating with M (M is a natural number equal to or greater than 2) terminal devices and includes a display means and an operation accepting means. Here, the display means is a type of information output means and outputs various information in a visual form, i.e., a screen. The operation accepting means can accept operations by an operator of the information processing device. The third aspect of the present invention then causes a computer of the information processing device to execute a display control procedure. In this display control procedure, various screens are displayed on the display means. In particular, in the display control procedure, when the operation accepting means accepts an operation instructing the selection of any one of the M terminal devices as a communication partner, the display means is caused to display an operation screen. The operation screen includes a communication partner image corresponding to the communication partner and N first standby terminal images corresponding to N (N is a natural number less than M) of the M terminal devices. The N first standby terminal images are displayed in a manner according to the importance of each of the N terminal devices corresponding to the N first standby terminal images. [Effects of the Invention]

[0020] According to the present invention, it is possible to provide appropriate and smooth responses to users of the respective terminal devices. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 1 is a diagram showing the overall configuration of an online customer service system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing the electrical configuration of an operator terminal in one embodiment of the present invention. [Figure 3] FIG. 3 is a diagram showing a state in which a user terminal is installed in one embodiment of the present invention. [Figure 4] FIG. 4 is a block diagram showing the electrical configuration of a user terminal in one embodiment of the present invention. [Figure 5] FIG. 5 is a diagram showing an example of a standby screen in an embodiment of the present invention. [Figure 6] FIG. 6 is a diagram showing an example of an operation screen in an embodiment of the present invention. [Figure 7] FIG. 7 is a diagram showing an example of an operation screen in a virtual configuration as a comparison target for an embodiment of the present invention. [Figure 8] FIG. 8 is a diagram conceptually showing the configuration of a room management table in one embodiment of the present invention. [Figure 9] FIG. 9 is a diagram conceptually showing the configuration of a room status management table in one embodiment of the present invention. [Figure 10] FIG. 10 is a memory map conceptually showing the internal configuration of the RAM of the main storage unit of the operator terminal 20 in one embodiment of the present invention. [Figure 11] FIG. 11 is a flow diagram showing the flow of a room state management task in one embodiment of the present invention. [Figure 12] FIG. 12 is a flowchart showing the flow of a part of the standby screen display task in one embodiment of the present invention. [Figure 13]FIG. 13 is a flow diagram showing the flow of the remaining part of the standby screen display task in one embodiment of the present invention. [Figure 14] FIG. 14 is a flowchart showing the flow of a part of the operation screen display task in one embodiment of the present invention. [Figure 15] FIG. 15 is a flow diagram showing the flow of the remaining part of the operation screen display task in one embodiment of the present invention. [Figure 16] FIG. 16 is a diagram showing another example of an operation screen according to an embodiment of the present invention. [Figure 17] FIG. 17 is a diagram showing yet another example of an operation screen according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0022] An embodiment of the present invention will be described by taking an online customer service system 10 shown in FIG. 1 as an example.

[0023] The online customer service system 10 according to this embodiment is applied to, for example, a hotel and includes a plurality of operator terminals 20, 20, ... and a plurality of user terminals 30, 30, .... Each operator terminal 20 is an example of an information processing device according to the present invention, and is used by an operator who is a hotel staff member, and may be installed, for example, in the hotel office, a contact center, or even in the operator's home. On the other hand, each user terminal 30 is an example of a terminal device according to the present invention, and is used by an unspecified number of users, including hotel guests, and is installed in an appropriate location in the hotel, such as the front counter or entrance.

[0024] Each operator terminal 20 and each user terminal 30 is connected to a network 40, and two-way communication can be performed via the network 40 using the P2P (Peer to Peer) method. The network 40 is, for example, a LAN within a hotel, but it may also be a WAN that includes the LAN, or it may include the Internet. To achieve two-way communication using the P2P method, WebRTC (Web Real-Time Communication) is used, and for this purpose, appropriate elements such as a signaling server, STUN server, and TURN server (not shown) are provided. However, since these are well known, detailed explanations of each will be omitted.

[0025] According to this online customer service system 10, operators using the respective operator terminals 20 can remotely respond to (serve) users using the respective user terminals 30. In this case, a user desiring remote service performs a call operation to call a hotel staff member using any of the user terminals 30. The fact that this call operation has been performed is notified to each operator terminal 20. Upon receiving the call notification indicating that this call operation has been performed, one of the hotel staff operators performs a response start operation using his or her own operator terminal 20. This establishes a state for two-way communication using the P2P method between the operator terminal 20 that accepted the response start operation and the user terminal 30 that accepted the call operation, and enables the two-way communication. This enables remote service, particularly real-time service.

[0026] Each operator terminal 20 is, for example, a personal computer, and as shown in Fig. 2, has a control unit 202, an input / output interface (I / F) 204, an input device 206, a display device 208, a microphone 210, a speaker 212, a camera 214, an auxiliary storage unit 216, a communication unit 218, etc. Note that Fig. 2 is a block diagram showing the electrical configuration of each operator terminal 20, but elements not directly related to the present invention are omitted from Fig. 2.

[0027] The control unit 202 is a control means responsible for overall control of the operator terminal 20. The control unit 202 includes a computer, such as a CPU 202a, as a control execution means. The control unit 202 also includes a main memory unit 202b, which is directly accessible by the CPU 202a. The main memory unit 202b includes, for example, a ROM and a RAM. The ROM stores firmware, including BIOS. The RAM provides a working area and a buffer area for the CPU 202a when it executes processes based on various programs included in the firmware and various software. The control unit 202 may be a SoC (System on a Chip). The control unit 202 also serves as an example of a display control means according to the present invention. The control unit 202 as the display control means displays various screens, such as a standby screen 500 and an operation screen 600 (described later), on the display device 208. A GPU (Graphics Processing Unit) may be provided to reduce the processing load on the control unit 202 as the display control means, particularly the CPU 202a.

[0028] The input / output interface 204 is an interface means, including a chipset, that acts as a bridge between the control unit 202, particularly the CPU 202a, and each element such as the input device 206. For this reason, the input / output interface 204 is connected to the control unit 202, as well as each element such as the input device 206, more specifically, the input device 206, the display device 208, the microphone 210, the speaker 212, the camera 214, the auxiliary storage unit 216, the communication unit 218, etc.

[0029] The input device 206 is an example of an operation receiving unit according to the present invention, and receives operations from an operator. The input device 206 includes, for example, a keyboard and a mouse.

[0030] The display device 208 is an example of a display means according to the present invention, and as described above, various screens such as the standby screen 500 and the operation screen 600 are displayed on the display device 208. The display device 208 is, for example, a flat panel display such as a liquid crystal display or an organic EL display, but is not limited to this.

[0031] The microphone 210 is an operator voice detection means for detecting the voice uttered by the operator using the operator terminal 20, that is, the operator voice. Naturally, the microphone 210 is provided so as to be able to detect the operator voice. Note that a headset integrated with a speaker 212, which will be described next, may be used as the microphone 210.

[0032] The speaker 212 is a user voice output means, so to speak, for outputting the voice of the user receiving the remote response described above, or more precisely, for outputting voice corresponding to the user voice described below. Naturally, therefore, the speaker 212 is provided so that the operator can hear the output sound from the speaker 212. Note that a headset integrated with the microphone 210 described above may be used as the speaker 212.

[0033] Camera 214 is, so to speak, an operator imaging means for imaging an operator using operator terminal 20, more specifically for obtaining a moving image of the operator. Naturally, camera 214 is therefore provided so as to be able to image the operator.

[0034] The auxiliary storage unit 216 is an auxiliary storage means and includes, for example, a hard disk drive. Various software such as an operating system and various application software is stored in this auxiliary storage unit 216. Additionally, the auxiliary storage unit 216 stores various data such as the operator's voice detected by the microphone 210 and the operator image captured by the camera 214. Note that the auxiliary storage unit 216 may include a rewritable nonvolatile memory such as a flash memory instead of or in addition to the hard disk drive.

[0035] The communication unit 218 is a communication connection means that is responsible for connection with the above-mentioned network 40. This connection with the network 40 may be wired or wireless.

[0036] Such an operator terminal 20 may be a desktop personal computer or a notebook personal computer. Alternatively, a portable computer such as a tablet or a smartphone may be adopted as the operator terminal 20. In this case, the touch panel of the portable computer serves as both the input device 206 and the display device 208.

[0037] Meanwhile, each user terminal 30, for example, a user terminal 30 installed at the front counter, is a so-called self-check-in / check-out machine that allows the hotel guest to check in / check out themselves, and as shown in Fig. 3, has a device main body 302. The device main body 302 has an operation unit 302a that accepts operations by the user, and this operation unit 302a is provided with a touch panel 302b. A procedure operation screen (not shown) required for performing the check-in / check-out procedure is displayed on the touch panel 302b. The user can perform the check-in / check-out procedure on this procedure operation screen.

[0038] When carrying out this check-in / check-out procedure, the user can receive remote support or other assistance as necessary. To this end, the user terminal 30 has a display device 304. This display device 304 is, for example, a flat panel display having a relatively large display screen, more specifically, a display screen larger than the touch panel 302b of the device main body 302, and is provided near the device main body 302. An avatar 304a, which is a representative character of the operator who will be providing remote support, is displayed on the display device 304. The avatar 304a moves in accordance with the operator's actions, or more precisely, moves in accordance with the operator image described above.

[0039] Additionally, the user terminal 30 has a speaker 306. This speaker 306 is an operator voice output means for outputting the voice of the operator, or more precisely, for outputting a voice corresponding to the above-mentioned operator voice. Naturally, therefore, the speaker 306 is provided so that the user can hear the output sound from the speaker 306. Note that the speaker 306 is provided, for example, on the top of the display device 304, but is not limited to this and may be built into the display device 304, for example. The output sound from the speaker 306 may also be a voice processed by a voice changer.

[0040] Furthermore, the user terminal 30 has a microphone 308 as a user voice detection means for detecting the voice uttered by the user, that is, the user voice. Naturally, the microphone 308 is provided so as to be able to detect the user voice, and is provided, for example, on the upper part of the display device 304. In addition, the user terminal 30 has a user shooting camera 310 as a user shooting means for shooting the user, more specifically, for obtaining a moving image of the user. Naturally, the user shooting camera 310 is provided so as to be able to shoot the user, and is provided, for example, on the upper part of the display device 304. Note that in the configuration shown in FIG. 3, the microphone 308 is built into the user shooting camera 310, but this is not a limitation, and the microphone 308 may be provided separately (independently) from the user shooting camera 310, for example.

[0041] In addition, user terminal 30 has operation unit photographing camera 312 as operation unit photographing means for photographing operation unit 302a of device main body 302, more specifically for obtaining a moving image of operation unit 302a. Naturally, operation unit photographing camera 312 is therefore provided so as to be able to photograph operation unit 302a of device main body 302, and is fixed to an appropriate location such as the ceiling via appropriate fixing member 312a, for example.

[0042] FIG. 4 is a block diagram showing the electrical configuration of the user terminal 30 shown in FIG. 3, i.e., the user terminal 30 is a self-check-in / check-out machine. As shown in FIG. 4, the user terminal 30 has a control unit 320, which is connected to an input / output interface (I / F) 322. The input / output interface 322 is connected to an operation unit 302a, a display device 304, a speaker 306, a microphone 308, a user image capturing camera 310, and an operation unit image capturing camera 312 of the device main body 302. The input / output interface 322 is also connected to an auxiliary memory unit 324, a communication unit 326, and the like. The control unit 320, the input / output interface 322, the auxiliary memory unit 324, and the communication unit 326 are provided within the device main body 302. Elements not directly related to the present invention are omitted from FIG. 4.

[0043] The control unit 320 is a control means (control means of the user terminal 30) that controls the entire user terminal 30. Therefore, the control unit 320 includes a computer, such as a CPU 320a, as a control execution means. The control unit 320 also includes a main memory unit 320b as a main memory unit (main memory unit of the user terminal 30) that can be directly accessed by the CPU 320a. The main memory unit 320b includes, for example, a ROM and a RAM. The ROM stores firmware, including embedded software. The RAM provides a working area and a buffer area when the CPU 320a executes processing based on a control program included in the firmware. The control unit 320 may be an SoC. The control unit 320 also functions as a display control means (display control means of the user terminal 30) that displays the aforementioned avatar 304a and the like on the display device 304. A GPU may be provided to reduce the processing load on the control unit 320, particularly the CPU 320a, as a display control means.

[0044] Input / output interface 322 is an interface means (interface means of user terminal 30) that acts as a bridge between control unit 320, particularly CPU 320a, and each element such as operation unit 302a, and includes, for example, a chipset. Therefore, as described above, input / output interface 322 is connected to control unit 320, as well as operation unit 302a, display device 304, speaker 306, microphone 308, user image capturing camera 310, operation unit image capturing camera 312, auxiliary storage unit 324, communication unit 326, etc.

[0045] The auxiliary storage unit 324 is an auxiliary storage means (auxiliary storage means of the user terminal 30) and has a rewritable nonvolatile memory such as a flash memory. This auxiliary storage unit 324 stores, for example, update programs for the embedded software described above. In addition, the auxiliary storage unit 324 stores various data such as the user's voice detected by the microphone 308, user images captured by the user capturing camera 310, and operation unit images captured by the operation unit capturing camera 312. Note that the auxiliary storage unit 324 may have a hard disk drive instead of or in addition to the rewritable nonvolatile memory.

[0046] The communication unit 326 is a communication connection means (communication connection means of the user terminal 30) that is responsible for connection with the above-mentioned network 40. This connection with the network 40 may be wired or wireless.

[0047] The device main body 302 including the operation unit 302a, the display device 304, the speaker 306, the microphone 308, the camera for capturing images of the user 310, and the camera for capturing images of the operation unit 312 are as described above. The display device 304 may be a touch panel, and in this case, the display device 304 may also serve as part of the operation unit 302a.

[0048] As described above, the user terminal 30 described with reference to FIGS. 3 and 4 is a self-check-in / check-out machine installed at the front counter. However, user terminals 30 installed at locations other than the front counter have a different configuration from the self-check-in / check-out machine. While detailed explanations, including illustrations, are omitted, user terminals 30 installed at locations other than the front counter include, for example, a personal computer, and in particular, a display device 304 with a relatively large display screen. This display device 304 is a touch panel and also serves as the operation unit 302a. Additionally, a speaker 306, a microphone 308, and a camera 310 for photographing users are provided, but a camera 312 for photographing the operation unit is not provided. Note that a portable computer such as a tablet or smartphone may also be used as the user terminal 30 installed at a location other than the front counter. In this case, the touch panel of the portable computer serves as both the operation unit 302a and the display device 304, and in addition, another display device with a relatively large display screen may be provided.

[0049] As described above, according to the online customer service system 10 of this embodiment, an operator using each operator terminal 20 can remotely respond to a user using each user terminal 30. To achieve this, application software called operator remote response software and a web browser are installed on each operator terminal 20. The user terminal 30 installed at the front counter is also installed with the above-mentioned embedded software, including the embedded remote response software and browser. Furthermore, the user terminal 30 installed at a location other than the front counter is also installed with application software called user remote response software and a web browser.

[0050] Then, the operator remote response software is started on each operator terminal 20. At the same time, the user remote response software is started on the user terminal 30 installed in a location other than the front counter. For the user terminal 30 installed at the front counter, when the power is turned on, the embedded software including the embedded remote response software is started.

[0051] Then, when the above-mentioned calling operation is performed on any of the user terminals 30, the fact that this calling operation has been performed is notified to each operator terminal 20. When a response start operation is performed by any of the operator terminals 20 in response to the calling notification indicating that this calling operation has been performed, a state for two-way communication using the P2P method is established between the operator terminal 20 on which the response start operation was performed and the user terminal 30 on which the calling operation was performed, and this two-way communication becomes possible. Then, remote response is initiated. This remote response is performed via a web browser.

[0052] In this remote response, a user image (strictly speaking, the data) of the user terminal 30 is transmitted to the operator terminal 20 and received by the operator terminal 20. The user image received by the operator terminal 20 is then displayed on the display device 208 of the operator terminal 20; more precisely, an operation screen 600 including the user image is displayed on the display device 208, as will be described later. At the same time, the user voice (strictly speaking, the data) of the user terminal 30 is also transmitted to the operator terminal 20 and received by the operator terminal 20. The user voice received by the operator terminal 20 is then output from the speaker 212 of the operator terminal 20; more precisely, a voice corresponding to the user voice is output from the speaker 212. Note that if the user terminal 30 is installed at a front counter, an operation unit image of the user terminal 30 is transmitted to the operator terminal 20 together with the user image and received by the operator terminal 20. Then, the user image and operation unit image received by the operator terminal 20 are displayed on the display device 208 of the operator terminal 20, and strictly speaking, an operation screen 600 including the user image and operation unit image is displayed on the display device 208.

[0053] Furthermore, in remote response, the aforementioned avatar 304a is displayed on the display device 304 of the user terminal 30. This avatar 304a moves in accordance with the operator image on the operator terminal 20 side. To this end, avatar data for displaying the avatar 304a moving in accordance with the operator image is generated in the operator terminal 20. In this process, the operator terminal 20 performs a well-known motion detection process on the operator image, thereby detecting the operator's movements, and the detection results are reflected in the avatar data. This avatar data is transmitted to the user terminal 30 and received by the user terminal 30. In the user terminal 30 that has received this avatar data, display screen data for displaying the avatar 304a is generated based on the avatar data, and the avatar 304a is displayed on the display device 304 based on the display screen data.

[0054] At the same time, the operator voice (strictly speaking, the data) on the operator terminal 20 side is also transmitted to the user terminal 30 and received by the user terminal 30. The operator voice received by the user terminal 30 is then output from the speaker 306 of the user terminal 30; strictly speaking, a voice corresponding to the operator voice is output from the speaker 306. As described above, the output sound from the speaker 306 may be a voice processed by a voice changer. This voice changer is provided in the operator terminal 20, for example, and specifically, is configured by the control unit 202 of the operator terminal 20. However, it may also be provided in the user terminal 30, and specifically, may be configured by the control unit 320 of the user terminal 30.

[0055] Although not shown, each user terminal 30 has a call button for accepting the above-mentioned call operation. This call button is enabled only when the user terminal 30 is in a standby state, i.e., not being used for remote call answering. That is, when the call button of a user terminal 30 in a standby state is operated, a call notification indicating that the call operation has been performed is sent to each operator terminal 20. Furthermore, if the display device 304 of a user terminal 30 in a standby state is a touch panel, an operator (button image) functioning as a call button may be displayed on the display device 304. Furthermore, even if a user speaks into the microphone 308 of a user terminal 30 in a standby state, or more precisely, even if the microphone 308 detects a voice level above a predetermined threshold level, the call operation is treated as having been performed. Therefore, in each user terminal 30, the microphone 308 remains enabled even when the user terminal 30 is in a standby state.

[0056] In addition, although not shown in the figure, a message screen containing information informing the user that remote response is possible and that if remote response is desired, the user should operate the call button (or an operator that functions as a call button) or speak into the microphone 308 is displayed on the display device 304 of the user terminal 30 in standby mode.

[0057] Here, the space in which each user terminal 30 is installed is referred to as a "room." It is assumed that there are M rooms, that is, M user terminals 30, 30, ... are installed, and the value of M is, for example, 10 or more.

[0058] Then, when attention is paid to each operator terminal 20, particularly to an operator terminal 20 in a standby state that is not being used for remote response, a standby screen 500 such as that shown in FIG. 5 is displayed on the display device 208. This standby screen 500 includes P (P is a natural number equal to or less than M), for example, 10, room images 502, 502, .... Each room image 502 represents one room, and more specifically, represents a room in a standby state (where a user terminal 30 is installed). Although detailed illustration is omitted, each room image 502 includes an image of a user in the room corresponding to the room image 502. In addition, near, for example below, each room image 502, a character string 504 indicating the location of the room corresponding to the room image 502 is displayed. Therefore, from each room image 502 and the character string 504, the operator can intuitively recognize (understand) the status of the room, including whether a user is present in the room corresponding to the room image 502, and can, in other words, monitor the status of the room. To achieve this monitoring, in each room (user terminal 30), the user image capturing camera 310 is enabled even when the room is in standby mode. Each room image 502 is an example of a second standby terminal image according to the present invention.

[0059] The room images 502, 502, ... are displayed in a so-called 2-row, 5-column array, for example, two vertically and five horizontally. Furthermore, if the total number of rooms is greater than 10, that is, if the value of M described above is greater than the value of P, although not shown, the standby screen 500 can be scrolled, for example, horizontally, thereby allowing all the room images 502, 502, ... corresponding to all the rooms to be displayed on the standby screen 500. Note that the scrolling direction of the standby screen 500 may be vertical instead of horizontal.

[0060] The arrangement (order) of each room image 502, 502, ... displayed on the standby screen 500 is determined according to the importance (priority) of the room corresponding to each room image 502, 502, .... The importance here refers to the degree to which remote assistance is required; for example, the higher the importance, the higher the degree to which remote assistance is required, and the lower the importance, the lower the degree to which remote assistance is required. This importance is determined based on multiple indicators, including the user's waiting time, the user's behavior (attitude), and a specific importance previously set for each room (user terminal 30).

[0061] The waiting time of a user is the time elapsed from the time the call operation was made. The longer the waiting time of a user, the more urgent the need for a response, and therefore the higher the importance.

[0062] A user's behavior refers to the user's speech and actions, such as the volume of the user's voice and panicked movements. For example, as described above, when a user speaks into the microphone 308 of a user terminal 30 in standby mode, this is treated as a call operation. However, the louder the detection level of the voice by the microphone 308, i.e., the call audio level, the more urgent the user is considered to be, and therefore the higher the importance. Furthermore, when a user is in an urgent or panicked state, the user's movements (fluctuations in their movements) during the call operation tend to be larger, so the louder the call action, which is the movement during the call operation, the higher the importance. The amplitude of this call action is detected by performing a well-known motion detection process on the user's image. Furthermore, when a user performs a call operation frequently, it is also considered as an urgent situation, so the more the number of calls, which is the number of times the call operation is performed, the higher the importance. Thus, a user's behavior includes multiple elements, such as the call audio level, the call action, and the number of calls.

[0063] The inherent importance is a unique importance that is set in advance for each room. For example, each room is assigned a room number, which is an individual identification number starting from "1." The smaller the room number, the higher the inherent importance, and therefore the higher the importance. In particular, rooms that are expected to have many users, such as the front counter, are assigned a small room number, i.e., a high inherent importance is set. The inherent importance may also be set individually for different time periods. As an example, for the front counter, a high inherent importance is set for the morning and evening time periods when many check-in / check-out procedures are expected. As another example, for a restaurant, a high inherent importance is set for mealtimes. As yet another example, for a banquet hall, a high inherent importance is set for the time period when banquets are scheduled. In other words, the inherent importance is set appropriately according to the characteristics (attributes) of the room.

[0064] The importance is determined based on multiple indicators, such as the user's waiting time, the user's behavior, and the specific importance previously set for each room, and details of this will be described later. Note that if two or more rooms have the same importance, or more precisely, if the overall evaluation points described later are the same, the room with the smaller room number will be treated as having a higher importance.

[0065] 5, when the standby screen 500 is initially displayed (before being scrolled), the room image 502 corresponding to the room with the highest importance is displayed in the upper left, that is, at the left end of the top row. The room image 502 corresponding to the room with the second highest importance is displayed second from the left in the top row, and the room image 502 corresponding to the room with the third highest importance is displayed third from the left in the top row. Thereafter, in accordance with a Z-shaped pattern of visual guidance, the room images 502, 502, ... are displayed in an arrangement according to the importance of the corresponding room. In short, the room images 502 displayed on the initial standby screen 500 are filtered and the arrangement of the room images 502, 502, ... is determined based on the importance of each room; in other words, the room images 502, 502, ... are rearranged.

[0066] Furthermore, a room image 502 corresponding to a room in which a call operation has been performed (strictly speaking, the room is in a state where, after the call operation has been performed, it is waiting for an operator to perform a response start operation in response to the call operation) displays, for example, an appropriate call mark 506 near the upper left corner, indicating that the call operation has been performed. Additionally, an appropriate decoration indicating that the call operation has been performed is applied to the periphery of the room image 502 corresponding to the room in which the call operation has been performed, for example, a red frame is added. Note that in Figure 5, the addition of a thick solid frame to the periphery of the room image 502 corresponding to the room in which the call operation has been performed indicates that a red frame has been added to the periphery of the room image 502.

[0067] Therefore, the operator can intuitively recognize the importance of each room corresponding to each room image 502, 502, ... from the arrangement of each room image 502, 502, ..., and in particular, can intuitively recognize that the room corresponding to room image 502 displayed in the upper left has the highest importance. Note that Fig. 5 shows an example of the standby screen 500 when "Front Desk (Front Counter) A" as "Room 1" with room number "1" has the highest importance, and a call operation is being performed at "Front Desk A," and further, call operations are also being performed at "Entrance" as "Room 11," "Restaurant" as "Room 18," and "Lobby Lounge" as "Room 15." In addition, in the example shown in Figure 5, six room images 502, 502, ... corresponding to six rooms, namely, "Front B" as "Room 2," "Front C" as "Room 3," "Front D" as "Room 4," "Front E" as "Room 5," "Front F" as "Room 6," and "Front G" as "Room 7," among the rooms for which no call operation has been performed, are displayed in order according to the importance of the corresponding rooms.

[0068] The operator can then intuitively recognize from the call mark 506 that a call operation has been performed in the room corresponding to the room image 502 on which the call mark 506 is displayed. In addition, the operator can also intuitively recognize from the fact that a red frame is added to the periphery of the room image 502 on which the call mark 506 is displayed that a call operation has been performed in the room corresponding to the room image 502. The room in which the call operation has been performed is determined based on the above-mentioned call notification.

[0069] Furthermore, each of the room images 502, 502, ... on the standby screen 500 functions as an operator that accepts an operation to select a party to be served remotely, in other words, a communication partner for the two-way communication described above. That is, when any of the room images 502 is operated (pressed), the room (user terminal 30) corresponding to the operated room image 502 is selected as the party to be served remotely. Then, instead of the standby screen 500, an operation screen 600 as shown in FIG. 6 is displayed on the display device 208.

[0070] When selecting a remote party to be served, it is desirable to select the room with the highest importance as the party to be served; that is, it is desirable to operate the room image 502 displayed in the upper left corner of the room images 502, 502, ... on the standby screen 500. Therefore, for example, on the standby screen 500 shown in FIG. 5, it is desirable to operate the room image 502 corresponding to "Front A" as "Room 1." FIG. 6 shows an example of an operation screen 600 that is displayed when the room image 502 corresponding to "Front A" as "Room 1" is operated on the standby screen 500 shown in FIG. 5. The standby screen 500 also includes an end button (button image) (not shown), and operating this end button terminates the display of the standby screen 500 and terminates the remote reception software for the operator described above.

[0071] In the operation screen 600 shown in FIG. 6 , a room image 602 corresponding to the room selected as the remote contact, i.e., the room image 602 corresponding to "Front Desk A" as "Room 1," is displayed in an enlarged state. This so-called enlarged room image 602 is an example of a communication partner image according to the present invention and is displayed, for example, in a position toward the upper right of the operation screen 600. Additionally, near, for example below, the enlarged room image 602, a character string 604 indicating the location of the room corresponding to the enlarged room image 602 is displayed. Although not shown, the enlarged room image 602 includes a user image of the room selected as the remote contact. Furthermore, if the room selected as the remote contact is a front desk counter, i.e., if the user terminal 30 installed in the room selected as the remote contact is a self-service check-in / check-out machine, the enlarged room image 602 includes a user image and an operation unit image. In this case, the user image and the operation unit image are displayed side by side, for example, but are not limited to this.

[0072] Furthermore, at an appropriate position on the operation screen 600, for example, to the left of the enlarged room image 602, N (N is a natural number less than P) room images 606, 606, ... are displayed in a vertical row in a reduced state. These four, so-called reduced room images 606, 606, ... correspond to four rooms of high importance among those other than the room selected for remote interaction. Therefore, on the operation screen 600 shown in FIG. 6, four reduced room images 606, 606, ... corresponding to four rooms, namely, "Entrance" as "Room 11," "Restaurant" as "Room 18," "Lobby Lounge" as "Room 15," and "Front Desk B" as "Room 2," are displayed in a vertical row. In addition, near, for example, below, each reduced room image 606, a character string 608 indicating the location of the room corresponding to the reduced room image 606 is displayed. Although not shown, each reduced room image 606 includes an image of the user of the room corresponding to the reduced room image 606. Each reduced room image 606 is an example of a first standby terminal image according to the present invention.

[0073] The arrangement (order) of each reduced room image 606, 606, ... depends on the importance of the room corresponding to each reduced room image 606, 606, .... Specifically, the higher the importance of the room corresponding to each reduced room image 606, the higher the reduced room image 606 is displayed. Therefore, on the operation screen 600 shown in FIG. 6, the reduced room images 606, 606, ... are displayed from top to bottom in the following order: "Entrance" as "Room 11," "Restaurant" as "Room 18," "Lobby Lounge" as "Room 15," and "Front B" as "Room 2." In short, based on the importance of each room, the reduced room images 606 displayed on the operation screen 600 are filtered and the arrangement of each reduced room image 606, 606, ... is determined; in other words, the reduced room images 606, 606, ... are rearranged.

[0074] In addition, an appropriate call mark 610 indicating that the call operation has been performed is displayed, for example, near the upper left corner, on each of the reduced room images 606, 606, ... that corresponds to a room for which a call operation has been performed. At the same time, an appropriate decoration indicating that the call operation has been performed is applied to the periphery of the reduced room image 606 corresponding to the room for which a call operation has been performed, for example, a red frame is added. Note that in Figure 6, a thick solid frame is added to the periphery of the reduced room image 606 corresponding to the room for which a call operation has been performed, thereby expressing that a red frame has been added to the periphery of the reduced room image 606.

[0075] Furthermore, three operators (button images) 612, 614, and 616 are displayed side by side at an appropriate position on the operation screen 600, for example, near the bottom right of the operation screen 600. Of these three operators 612, 614, and 616, operator 612 on the left end is a [Back] button that instructs the display device 208 to end the display of the operation screen 600 and display the standby screen 500 again, in other words, to return to that state. Operator 614 in the center is a [Start Response] button that instructs the start of a remote response with the room selected as the remote party. Furthermore, operator 616 on the right end is an [End Response] button that instructs the end of the remote response.

[0076] That is, when the [Back] button 612 on the operation screen 600 is operated, the standby screen 500 is displayed again on the display device 208 in place of the operation screen 600, that is, the display device 208 returns to that state. Then, when the [Start Response] button 614 is operated, a remote response is started with the room selected as the remote party, and strictly speaking, the remote response becomes possible. The operation of this [Start Response] button 614 corresponds to the response start operation. Furthermore, when a remote response is being performed, if the [End Response] button 616 is operated, the remote response ends.

[0077] When the operation screen 600 is first displayed, the "End Response" button 616 is in a state where it cannot accept any operation, and is, for example, grayed out. When a remote response is being performed, the "Back" button 612 and the "Start Response" button 614 are in a state where it cannot accept any operation, and are, for example, grayed out.

[0078] According to this operation screen 600, particularly when a call is being received remotely, the operator can converse with the user of the other party's room while checking the status of the other party's room through the enlarged room image 602. At this time, the above-mentioned avatar 304a is displayed in the other party's room (display device 304). Note that an image similar to the avatar 304a may also be displayed on the operation screen 600; for example, an image similar to the avatar 304a may be displayed at an appropriate position on the enlarged room image 602, but smaller than the enlarged room image 602.

[0079] In addition, the operator can intuitively recognize, or monitor, the status of a room, including whether a user is present in the room corresponding to each reduced room image 606, from each reduced room image 606 and the character string 608 on the operation screen 600. Furthermore, the operator can intuitively recognize the importance of each room corresponding to each reduced room image 606, 606, ..., from the arrangement of each reduced room image 606, 606, ..., and in particular, can intuitively recognize that the room corresponding to the reduced room image 606 displayed at the top has the highest importance. In addition, the operator can intuitively recognize from the call mark 610 that a call operation has been performed in the room corresponding to the reduced room image 606 in which the call mark 610 is displayed. Furthermore, the operator can intuitively recognize from the red frame around the reduced room image 606 in which the call mark 610 is displayed that a call operation has been performed in the room corresponding to the reduced room image 606. Therefore, after finishing the service with the user currently being served, the operator can intuitively recognize the next user (room) to be served.

[0080] Furthermore, when a remote call is not being received, each of the reduced room images 606, 606, ... on the operation screen 600 functions as an operator that accepts a caller change operation to change the caller being called remotely. That is, when a call is not being received remotely and one of the reduced room images 606 is operated, the room corresponding to the operated reduced room image 606 is selected as the new caller, i.e., the caller is changed. Then, in accordance with the room selected as the new caller, the enlarged room image 602 is changed, and the character string 604 below the enlarged room image 602 is also changed. Furthermore, the position of each of the reduced room images 606, 606, ... is changed, and the character string 608 below each reduced room image 606 is also changed. When a call is being received remotely, each of the reduced room images 606, 606, ... is unable to accept operations.

[0081] As described above, the four (N) reduced room images 606, 606, ... on the operation screen 600 correspond to the four most important rooms among those other than the room selected as the remote contact, that is, arranged in order of importance. Therefore, the operator can intuitively recognize the importance of each room corresponding to each reduced room image 606, 606, ... from the arrangement of the reduced room images 606, 606, ..., and can intuitively recognize that the room corresponding to the reduced room image 606 displayed at the top has the highest importance. Additionally, the operator can intuitively recognize from the call mark 610 that a call operation has been performed in the room corresponding to the reduced room image 606 in which the call mark 610 is displayed. Furthermore, the operator can intuitively recognize from the red frame around the reduced room image 606 in which the call mark 610 is displayed that a call operation has been performed in the room corresponding to the reduced room image 606 in which the call mark 610 is displayed that the operator also intuitively recognizes that a call operation has been performed in the room corresponding to the reduced room image 606. This allows the operator to intuitively recognize the next user (room) to be served, greatly contributing to the realization of accurate and smooth remote service.

[0082] In contrast to this, for example, a configuration is assumed in which the four (N) reduced room images 606, 606, ... on the operation screen 600 correspond to the four rooms with the smallest room numbers among the rooms other than the room selected as the remote contact, that is, in order of room number. According to this virtual configuration, when the room image 502 corresponding to "Front A" as "Room 1" is operated on the standby screen 500 in the state shown in Fig. 5, for example, an operation screen 600 as shown in Fig. 7 is displayed on the display device 208 in place of the standby screen 500. On the operation screen 600 shown in Fig. 7, the reduced room images 606, 606, ... are displayed in the following order from top to bottom: "Front B" as "Room 2," "Front C" as "Room 3," "Front D" as "Room 4," and "Front E" as "Room 5." That is, even though the room in which the call operation was performed exists among rooms other than the room selected as the remote party, the reduced room image 606 corresponding to the room in which the call operation was performed is not displayed on the operation screen 600. From such an operation screen 600, the operator cannot recognize the room in which the call operation was performed, and cannot even recognize that the room in which the call operation was performed existed before that. This makes it impossible to perform accurate and smooth remote responses.

[0083] Thus, according to this embodiment, and particularly according to the operation screen 600, the operator can intuitively recognize the importance of each room corresponding to each reduced room image 606, 606, ... from the arrangement of the reduced room images 606, 606, ..., and in particular, can intuitively recognize that the room corresponding to the reduced room image 606 displayed at the top has the highest importance. Therefore, the operator can intuitively recognize the room to attend to next.

[0084] Also, on the standby screen 500, the operator can intuitively recognize the importance of each room corresponding to each room image 502, 502, ... from the arrangement of the room images 502, 502, ..., and in particular, can intuitively recognize that the room corresponding to the room image 502 displayed in the upper left has the highest importance. Therefore, the operator can intuitively recognize which room should be attended to first from among the rooms corresponding to each room image 502, 502, ....

[0085] In this embodiment, a room management table 700 is provided as shown in Fig. 8. This room management table 700 is a table for managing each room (user terminal 30), and is provided in each operator terminal 20, and more specifically, is stored in the auxiliary storage unit 216 of the operator terminal 20 in accordance with the above-mentioned remote response software for the operator.

[0086] This room management table 700 records data related to the IP address, location, unique importance setting conditions, etc. for each room, using the room number as a key. In particular, the unique importance setting conditions are conditions under which a high unique importance is set. FIG. 8 shows an example in which, for each front desk, including "Front Desk A" as "Room 1," the unique importance is set high for the time period "17:00-20:00," when many check-in procedures are expected, and the time period "08:00-10:00," when many check-out procedures are expected. As mentioned above, for example, a high unique importance is set for a restaurant during meal times, and a high unique importance is set for a banquet hall during times when banquets are planned. These settings are also recorded as data related to the unique importance setting conditions, etc. The unique importance setting conditions can be set arbitrarily, and can be set arbitrarily each time, including when a banquet is scheduled. Furthermore, the unique importance also depends on the room number. Specifically, as mentioned above, the smaller the room number, the higher the unique importance is set.

[0087] Additionally, in this embodiment, a room status management table 800 as shown in Fig. 9 is provided. This room status management table 800 is a table for managing the status of each room (user terminal 30). Like the room management table 700, this room status management table 800 is also provided in each operator terminal 20, and more specifically, is stored in the auxiliary storage unit 216 in accordance with the above-mentioned remote operator response software.

[0088] This room status management table 800 records data regarding the response status, importance, overall evaluation score, waiting time, call voice level, call operation, number of calls, inherent importance, etc. for each room (user terminal 30) using the room number as a key.

[0089] The service status indicates which of the four states each room is in: "waiting," "calling," "serving," and "stopped." Of these, "waiting" indicates that the room is in the standby state described above. "Calling" indicates that the room has performed the call operation described above (more precisely, after the call operation has been performed, the room is waiting for an operator to perform a response start operation in response to the call operation). "Serving" indicates that the room is currently receiving a remote call. "Stopped" indicates that the room is not currently receiving a remote call; that is, the user terminal 30 is turned off, or, for user terminals 30 installed in locations other than the front counter, the user remote response software described above is not running. Note that Figure 9 shows an example in which "Room 1" (Front A) is in the "calling" state, "Room 2" (Front B) is in the "waiting" state, and "Room 3" (Front C) is also in the "waiting" state.

[0090] The importance is expressed by natural numbers starting from "1". The smaller the value representing the importance, the higher the importance. In other words, a room with an importance value of "1" is the most important room among all the rooms. In the example shown in FIG. 9, "Room 1" has the highest importance, "Room 2" has the fifth highest importance, and "Room 3" has the sixth highest importance.

[0091] The overall rating score is the sum of the ratings based on multiple indices, including the waiting time, which will be described below. The overall rating score determines the importance of a room; specifically, the higher the overall rating score, the higher the importance. In the example shown in FIG. 9, the overall rating score for "Room 1" is 37 points, and the importance based on this overall rating score of 37 points is the highest among the rooms, and therefore "Room 1" has the highest importance. Also, in the example shown in FIG. 9, the overall rating score for "Room 2" is 19 points, and based on this overall rating score of 19 points, "Room 2" has the fifth highest importance. The overall rating score for "Room 3" is 18 points, and based on this rating score of 18 points, "Room 3" has the sixth highest importance. Note that if two or more rooms have the same overall rating score, the room with the smaller room number is considered to be more important.

[0092] As mentioned above, the overall evaluation score is the sum of evaluation scores based on multiple indicators, including the user's waiting time, the user's behavior, and the specific importance of each room. The user's behavior also includes multiple factors, such as the call voice level, the call operation, and the number of calls. Data related to these factors is recorded in the room status management table 800.

[0093] Focusing on the waiting time, in the example shown in FIG. 9, the waiting time for "Room 1" is 5.70 seconds. The longer this waiting time, the higher the evaluation point will be. Note that in the example shown in FIG. 9, "Room 2" is in the "waiting" state as described above, so the waiting time is not recorded, and therefore no evaluation point based on that waiting time is assigned (set to 0 points). Similarly, "Room 3" is in the "waiting" state, so the waiting time is not recorded, and therefore no evaluation point based on that waiting time is assigned.

[0094] The call audio level is evaluated, for example, in three stages according to its volume (height), and more specifically, is represented by one of three values: "1," "2," and "3." The larger the value representing the call audio level, the higher the call audio level and the higher the evaluation point assigned. In the example shown in FIG. 9, the value of the call audio level for "Room 1" is "2," and an evaluation point corresponding to this call audio level value of "2" is assigned to "Room 1." As for "Room 2" and "Room 3," since they are in the "standby" state as described above, the call audio level is not recorded, and therefore no evaluation point based on the call audio level is assigned. The call audio level is not limited to being evaluated in three stages, and may be evaluated in any number of stages.

[0095] The calling operation is also evaluated, for example, on a three-point scale according to its magnitude, and more specifically, is represented by one of three values: "1," "2," and "3." The larger the value representing this calling operation, the greater the calling operation and the higher the evaluation point assigned. In the example shown in FIG. 9, the value of the calling operation for "Room 1" is "1," and an evaluation point corresponding to this value of "1" is assigned to "Room 1." As for "Room 2" and "Room 3," since they are in the "standby" state as described above, no calling operation is recorded, and therefore no evaluation point is assigned based on this calling operation. This calling operation is also not limited to being evaluated on a three-point scale, and may be evaluated on a scale other than the three-point scale.

[0096] The larger the number of calls, the larger the evaluation score will be. In the example shown in Figure 9, the number of calls for "Room 1" is "1," and an evaluation score corresponding to this number of calls, "1," will be assigned to "Room 1." As for "Room 2" and "Room 3," since they are in the "waiting" state as described above, the number of calls is not recorded, and therefore evaluation scores based on the number of calls will not be assigned.

[0097] Furthermore, as described above, the inherent importance is set for each room according to the room number of the room. In addition, the inherent importance may be set according to the inherent importance setting conditions in the room management table 700. For example, for each front desk, including "Front Desk A" as "Room 1," a high inherent importance may be set during a time period according to the inherent importance setting conditions. In the example shown in FIG. 9 , the inherent importance value of "Room 1" is "20," and an evaluation point corresponding to this inherent importance value of "20" is assigned to "Room 1." Furthermore, the inherent importance value of "Room 2" is "19," and an evaluation point corresponding to this inherent importance value of "19" is assigned to "Room 2." Furthermore, the inherent importance value of "Room 3" is "18," and an evaluation point corresponding to this inherent importance value of "18" is assigned to "Room 3."

[0098] In this way, evaluation points are assigned to each room based on the waiting time, call voice level, call operation, number of calls, and inherent importance, and the importance is determined by the total evaluation point, which is the sum of these evaluation points. The layout of each room image 502, 502, ... on the standby screen 500 is determined according to the importance, and the layout of each reduced room image 606, 606, ... on the operation screen 600 is also determined according to the importance. Note that when assigning evaluation points based on each of the waiting time, call voice level, call operation, number of calls, and inherent importance, appropriate weighting may be applied to each of the waiting time, call voice level, call operation, number of calls, and inherent importance. Furthermore, the overall evaluation point may be calculated from evaluation points based on only a portion of the waiting time, call voice level, call operation, number of calls, and inherent importance, and the importance may then be determined. In other words, the importance may be determined based on only a portion of the waiting time, call voice level, call operation, number of calls, and inherent importance.

[0099] FIG. 10 is a memory map 900 conceptually showing the internal configuration of the RAM of the main storage unit 202b of the operator terminal 20. As shown in FIG.

[0100] As shown in this memory map 900, the RAM has a program storage area 910 and a data storage area 950. Various programs are stored in the program storage area 910. The various programs include an operation detection program 912, a display control program 914, a voice detection program 916, a voice output program 918, a photography program 920, a communication program 922, and a remote response program 924.

[0101] The operation detection program 912 is a program for detecting the operation state of the input device 206. The display control program 914 is a program for generating display screen data necessary for displaying various screens, such as the standby screen 500 and the operation screen 600, on the display device 208. The audio detection program 916 is a program for detecting audio using the microphone 210. The audio output program 918 is a program for outputting audio from the speaker 212. The imaging program 920 is a program for capturing moving images using the camera 214. The communication program 922 is a program for communicating via the communication unit 218. The remote response program 924 is a program based on the above-mentioned remote response software for the operator. This remote response program 924 includes various programs for realizing remote response, including the above-mentioned avatar data generation program for generating avatar data.

[0102] Meanwhile, various types of data are stored in the data storage area 950. The various types of data include operation data 952, display image generation data 954, operator voice data 956, operator image data 958, room status data 960, person being served data 962, response transmission data 964, response reception data 966, etc.

[0103] The operation data 952 is data representing the operation status of the input device 206 detected by the operation detection program 912. The display image generation data 954 is data such as polygon data and texture data used to generate display screen data based on the display control program 914. The operator voice data 956 is data of the operator voice detected by the microphone 210. The operator image data 958 is data of the operator image captured by the camera 214. The room status data 960 is data representing the status of each room. This room status data 960 includes data representing the user voice, user image, and call operation status of each room. The call recipient data 962 is data representing the remote call recipient, such as data representing the room number of the call recipient. The call transmission data 964 is data to be transmitted to the remote call recipient. This call transmission data 964 includes the operator voice, avatar data, and the like. The call reception data 966 is data received from the remote call recipient. This response reception data 966 includes the user's voice, user's image, and the like.

[0104] According to this embodiment, remote response is performed in the manner described above, and to achieve this, the CPU 202a of each operator terminal 20 executes a room status management task in accordance with a room status management program included in the remote response program 924. The flow of this room status management task is shown in Fig. 11. This room status management task is executed repeatedly in the background, for example, at a relatively short interval of less than one second.

[0105] According to this room status management task, the CPU 202a first performs a room status confirmation process in step S1 to confirm the status of each room. In this room status confirmation process in step S1, it is confirmed whether a call operation has been performed in each room, and if such a call operation has been performed, the aforementioned waiting time, call audio level, call operation, number of calls, etc. are confirmed. Then, in the following step S3, the CPU 202a updates the room status management table 800 based on the confirmation result in the room status confirmation process in step S1. With this, the CPU 202a ends the room status management task.

[0106] Additionally, the CPU 202a executes a standby screen display task in accordance with a standby screen display program included in the remote response program 924. The flow of this standby screen display task is shown in Figs. 12 and 13. This standby screen display task is executed immediately after the above-mentioned remote response software for the operator is started. In addition, the standby screen display task is executed in response to operation of the [Back] button 612 on the operation screen 600.

[0107] According to this standby screen display task, the CPU 202a first refers to the room status management table 800 in step S101. Then, in the following step S103, the CPU 202a displays the standby screen 500 on the display device 208 based on the result of referring to the room status management table 800 in step S101. As described above, on the standby screen 500, ten (P) room images 502, 502, ... are displayed in an arrangement according to the importance of the rooms corresponding to each of these room images 502, 502, .... Furthermore, a call mark 506 is displayed on the room image 502 corresponding to the room for which the call operation has been performed. Additionally, a red frame is added to the periphery of the room image 502 corresponding to the room for which the call operation has been performed.

[0108] After executing step S103, the CPU 202a proceeds to step S105, where it resets and starts a counter for counting the update cycle Ta of the display of the standby screen 500. The update cycle Ta of the display of the standby screen 500 is equal to or longer than the execution cycle of the room state management task, for example, one second. The counter for counting the update cycle Ta of the display of the standby screen 500 is, for example, a software counter configured by the CPU 202a, but may also be a hardware counter.

[0109] After executing step S105, the CPU 202a proceeds to step S107, where it determines whether or not the standby screen 500 has received any operation. If the standby screen 500 has received any operation (S107: YES), the CPU 202a proceeds to step S117, which will be described later. On the other hand, if the standby screen 500 has not received any operation (S107: NO), the CPU 202a proceeds to step S109.

[0110] In step S109, the CPU 202a determines whether or not the update cycle Ta of the display of the standby screen 500 has elapsed. If the update cycle Ta of the display of the standby screen 500 has not elapsed (S109: NO), the CPU 202a returns the process to step S107. On the other hand, if the update cycle Ta of the display of the standby screen 500 has elapsed (S109: YES), the CPU 202a advances the process to step S111.

[0111] In step S111, the CPU 202a refers to the room status management table 800. Then, in the following step S113, the CPU 202a determines whether there has been a change in the content of the room status management table 800. If there has been no change in the content of the room status management table 800 (S113: NO), the CPU 202a returns the process to step S105. On the other hand, if there has been a change in the content of the room status management table 800 (S113: YES), the CPU 202a proceeds to step S115. Then, in step S115, the CPU 202a updates the display of the standby screen 500 based on the result of referring to the room status management table 800 in step S111, and then returns the process to step S105.

[0112] Furthermore, when the CPU 202a advances the process from step S107 to step S117, it determines in step S117 whether the operation accepted in step S107 is a person-to-be-served selection operation, that is, whether the operation is an operation on any of the room images 502, 502, ... If it is determined that a person-to-be-served selection operation has been accepted, that is, if it is determined that any of the room images 502 on the standby screen 500 has been operated (S117: YES), the CPU 202a advances the process to step S119. On the other hand, if it is determined that a person-to-be-served selection operation has not been accepted, that is, if it is determined that none of the room images 502 has been operated (S117: NO), the CPU 202a advances the process to step S125, which will be described later.

[0113] In step S119, the CPU 202a sets a remote contact person, and more specifically, stores the room number of the contact person as the above-mentioned contact person data 962. Then, in the following step S121, the CPU 202a ends the display of the standby screen 500. Furthermore, in the following step S123, the CPU 202a starts execution of an operation screen display task, which will be described later. With this, the CPU 202a ends the standby screen display task.

[0114] On the other hand, when the CPU 202a advances the process from step S117 to step S125, it determines in step S125 whether the operation accepted in the above-mentioned step S107 is an end operation, that is, whether an operation is performed on an end button (not shown) on the standby screen. If it is determined that an end operation has not been accepted, that is, if it is determined that the end button on the standby screen 500 has not been operated (S125: NO), the CPU 202a advances the process to step S129, which will be described later. On the other hand, if it is determined that an end operation has been accepted, that is, if it is determined that the end button on the standby screen 500 has been operated (S125: YES), the CPU 202a advances the process to step S127. Then, in step S127, the CPU 202a ends the display of the standby screen 500, thereby ending the standby screen display task.

[0115] Furthermore, when the CPU 202a advances the process from step S125 to step S129, it performs, in step S129, a process according to the operation accepted in the above-mentioned step S107. The process in step S107 includes a process of scrolling the standby screen 500 left and right. Then, the CPU 202a returns the process to step S107.

[0116] Furthermore, the CPU 202a executes an operation screen display task in accordance with an operation screen display program included in the remote response program 924. The flow of this operation screen display task is shown in Figures 14 and 15. Execution of this operation screen display task starts in step S123 of the standby screen display task.

[0117] According to this operation screen display task, the CPU 202a first confirms the remote party in step S201. This confirmation of the remote party is performed by referencing the party data 962 described above. Then, in the following step S203, the CPU 202a refers to the room status management table 800. Furthermore, in the following step S205, the CPU 202a displays the operation screen 600 on the display device 208 based on the confirmation result of the party in step S201 and the reference result of the room status management table 800 in step S203. As described above, in the operation screen 6500, an enlarged room image 602 of the party is displayed, and four (N) reduced room images 606, 606, ... are displayed vertically to the left of the enlarged room image 602. The arrangement of each reduced room image 606, 606, ... depends on the importance of the room corresponding to each reduced room image 606, 606, .... Furthermore, a call mark 610 is displayed on the reduced room image 606 corresponding to the room where the call operation was performed. In addition, a red frame is added to the periphery of the room image 606 corresponding to the room where the call operation was performed.

[0118] After executing step S205, the CPU 202a advances the process to step S207, where it resets and starts a counter for counting the update period Tb of the display of the operation screen 600. Note that the update period Tb of the display of the operation screen 600 is equal to or longer than the execution period of the room status management task, similar to the update period Ta of the display of the standby screen 500, and is, for example, one second. The counter for counting the update period Tb of the display of the operation screen 600 is, for example, a software counter configured by the CPU 202a, but may also be a hardware counter.

[0119] After executing step S207, the CPU 202a advances the process to step S209, where it determines whether or not the operation screen 600 has received any operation. If the operation screen 600 has received any operation (S209: YES), the CPU 202a advances the process to step S219, which will be described later. On the other hand, if the operation screen 600 has not received any operation (S209: NO), the CPU 202a advances the process to step S211.

[0120] In step S211, the CPU 202a determines whether or not the update cycle Tb of the display of the operation screen 600 has elapsed. If the update cycle Tb of the display of the operation screen 600 has not elapsed (S211: NO), the CPU 202a returns the process to step S209. On the other hand, if the update cycle Tb of the display of the operation screen 600 has elapsed (S211: YES), the CPU 202a advances the process to step S213.

[0121] In step S213, the CPU 202a refers to the room status management table 800. Then, in the following step S215, the CPU 202a determines whether there has been a change in the content of the room status management table 800. If there has been no change in the content of the room status management table 800 (S215: NO), the CPU 202a returns the process to step S207. On the other hand, if there has been a change in the content of the room status management table 800 (S215: YES), the CPU 202a proceeds to step S217. Then, in step S217, the CPU 202a updates the display of the operation screen 600 based on the result of referring to the room status management table 800 in step S213, and then returns the process to step S207.

[0122] Furthermore, when the CPU 202a advances the process from step S209 to step S219, it determines in step S219 whether the operation accepted in step S209 is the operation of the [Start Handling] button 614. If it is determined that the [Start Handling] button 614 has been operated (S219: YES), the CPU 202a advances the process to step S221. On the other hand, if it is determined that the [Start Handling] button 614 has not been operated (S219: NO), the CPU 202a advances the process to step S223, which will be described later.

[0123] In step S221, the CPU 202a starts remote response. Although detailed explanation including illustrations will be omitted, the remote response is performed by a separate task called a response task. Then, the CPU 202a returns the process to step S209.

[0124] In contrast, when the CPU 202a advances the process from step S219 to step S223, it determines in step S223 whether the operation accepted in the above-mentioned step S209 is the operation of the [End Call] button 616. If it is determined that the [End Call] button 616 has been operated (S223: YES), the CPU 202a advances the process to step S225. On the other hand, if it is determined that the [End Call] button 616 has not been operated (S223: NO), the CPU 202a advances the process to step S227, which will be described later.

[0125] In step S225, the CPU 202a ends the remote response, that is, ends the response task, and returns the process to step S209.

[0126] Furthermore, when the CPU 202a advances the process from step S223 to step S227, it determines in step S227 whether the operation accepted in the above-mentioned step S209 is the operation of the [Back] button 612. If it is determined that the [Back] button 612 has been operated (S227: YES), the CPU 202a advances the process to step S229. On the other hand, if it is determined that the [Back] button 612 has not been operated (S223: NO), it determines that an operation to change the person being served has been accepted, that is, it determines that one of the reduced-room images 606, 606, ... has been operated, and the CPU 202a advances the process to step S235, which will be described later.

[0127] In step S229, the CPU 202a cancels the remote contact setting, and more specifically, clears the contact data 962 described above. Then, in the following step S231, the CPU 202a ends the display of the operation screen 600. Furthermore, in the following step S233, the CPU 202a starts execution of a standby screen display task. With this, the CPU 202a ends the operation screen display task.

[0128] On the other hand, when the CPU 202a advances the process from step S227 to step S235, the CPU 202a changes the remote contact person in step S235, and more specifically, stores the room number of the room selected as the new contact person as the contact person data 962. Then, the CPU 202a returns the process to step S201.

[0129] Thus, according to this embodiment, and particularly according to the operation screen 600, the operator can intuitively recognize the importance of each room corresponding to each reduced room image 606, 606, ... from the arrangement of the reduced room images 606, 606, ..., and in particular, can intuitively recognize that the room corresponding to the reduced room image 606 displayed at the top has the highest importance. Therefore, the operator can intuitively recognize the room to attend to next.

[0130] Also, on the standby screen 500, the operator can intuitively recognize the importance of each room corresponding to each room image 502, 502, ... from the arrangement of the room images 502, 502, ..., and in particular, can intuitively recognize that the room corresponding to the room image 502 displayed in the upper left has the highest importance. Therefore, the operator can intuitively recognize which room should be attended to first from among the rooms corresponding to each room image 502, 502, ....

[0131] It should be noted that this embodiment is merely a specific example of the present invention and does not limit the technical scope of the present invention. The present invention can also be applied to aspects other than this embodiment.

[0132] For example, in this embodiment, the value of N representing the number of reduced room images 606 on the operation screen 600 is set to "4," but this is not limited to this. As an example, the value of N may be set to "3" as shown in FIG. 16, or in an extreme case, the value of N may be set to "1" as shown in FIG. 17. The value of N may be set (changed) arbitrarily, and may be set appropriately depending on the ability (response ability) of the operator in particular. However, it is essential that the value of N is smaller than the value of P representing the number of room images 502 on the standby screen 500.

[0133] Furthermore, the area of ​​the operation screen 600 in which each reduced room image 606, 606, ... is displayed may be configured to be scrollable, for example, to be scrollable in the vertical direction. With this configuration, all reduced room images 606, 606, ... corresponding to all rooms can be displayed on the operation screen 600. Even with this configuration, it is essential that the arrangement of each reduced room image 606, 606, ... follows the importance of the room corresponding to each reduced room image 606, 606, ....

[0134] Additionally, although each of the reduced room images 606, 606, ... is displayed to the left of the enlarged room image 602, this is not limiting, and they may be displayed, for example, to the right of the enlarged room image 602. Also, each of the reduced room images 606, 606, ... may be displayed above or below the enlarged room image 602, and in this case, it is desirable that the reduced room images 606 be displayed further to the left the higher the importance of the room they correspond to.

[0135] Furthermore, each miniature room image 606 may be displayed in a more prominent manner the more important the room it corresponds to, for example, it may be displayed in a larger size, have a darker border around it, or flash more frequently.

[0136] The value of P, which represents the number of room images 502 on the standby screen 500, is set to "10," but is not limited to this. However, it is essential that the value of P be equal to or less than the value of M, which represents the total number of rooms.

[0137] Furthermore, the room image 502 on the standby screen 500 may be displayed in a more conspicuous manner the higher the importance of the room corresponding to it, for example, by being displayed in a larger size, by having a dark border around its periphery, or by flashing frequently.

[0138] Furthermore, in this embodiment, an avatar 304a is displayed in the room (user terminal 30) where remote consultations are received, but instead of the avatar 304a, an operator image of the operator in charge of consultations may be displayed.

[0139] In addition, although the user's waiting time, user behavior, and the specific importance of each room have been listed as multiple indices for determining importance, other indices may be added in place of or in addition to at least some of these.Furthermore, while the user's behavior has been described as including multiple elements such as the call voice level, the call behavior, and the number of calls, other elements may be added in place of or in addition to at least some of these.

[0140] Furthermore, in this embodiment, two-way communication is performed between each operator terminal 20 and each user terminal 30 using the P2P method, but this is not limiting and two-way communication may be performed using, for example, a client-server method. Incidentally, the P2P method in this embodiment is a pure P2P method, but this is not limiting and a hybrid P2P method or a supernode type P2P method may also be adopted.

[0141] In the extreme, only one operator terminal 20 may be provided. That is, a single operator terminal 20 may be configured to correspond to all rooms.

[0142] Furthermore, although the present embodiment has been described with reference to the online customer service system 10 for a hotel as an example, the present invention can also be applied to other uses. In particular, the user terminal 30 may be a physically mobile terminal such as a robot avatar. In this case, the inherent importance of the user terminal 30 may change depending on the location and time of day of the user terminal 30, and the importance of the user terminal 30 may also change.

[0143] In addition, the present invention is not limited to the form of an apparatus, namely, an information processing apparatus, but can also be provided in the form of a method, namely, a display control method in an information processing apparatus, and in the form of a program, namely, a display control program in an information processing apparatus.

[0144] In addition, the present invention can also be provided in the form of a computer-readable, non-transitory recording medium on which a display control program for an information processing device is recorded. In this case, the display control program recorded on the recording medium is read by the computer of the information processing device and executed by the computer, thereby realizing an information processing device similar to the present invention. The recording medium referred to here includes the same semiconductor media and disk media as described above. Furthermore, instead of portable media, built-in (internal) media such as ROMs and hard disk drives that are incorporated into the information processing device (computer system) can also be used as the recording medium referred to here. [Explanation of symbols]

[0145] 10... Online customer service system 20... Operator terminal 30...User terminal 202 ... Control section 202a...CPU 202b... Main memory section 206 ... Input device 208 … Display device 310 ... Camera for taking photos of users 500 … Standby screen 502 … Room image 600 … Operation screen 602 … Enlarged room image 606 … Miniature room image

Claims

1. An information processing device comprising a display means and an operation receiving means capable of receiving an operation by an operator, and capable of communicating with M (M being a natural number equal to or greater than 2) terminal devices, further comprising a display control means for causing the display means to display various screens, When the operation receiving means receives an instruction to select any one of the M terminal devices as a communication partner, the display control means causes the display means to display an operation screen including a communication partner image corresponding to the communication partner and N first standby terminal images corresponding to N (N; a natural number less than M) of the M terminal devices, and displays the N first standby terminal images in a manner according to the importance of each of the N terminal devices.

2. The information processing apparatus according to claim 1 , wherein the N terminal devices are selected from the M terminal devices in descending order of importance.

3. The information processing device according to claim 1 , wherein the N terminal devices do not include the communication partner.

4. The information processing device according to claim 1 , wherein the value of N can be set arbitrarily.

5. Each of the terminal devices includes a camera that is capable of photographing a user who uses the terminal device, The information processing device according to claim 1 , wherein the communication partner image includes an image captured by the camera provided in the communication partner.

6. The information processing apparatus according to claim 5 , wherein each of the N first standby terminal images includes an image captured by the camera provided in the terminal device corresponding to the first standby terminal image.

7. the communication partner image is larger than each of the N first standby terminal images; The information processing device according to claim 1 , wherein each of the N first standby terminal images is displayed in a more conspicuous manner as the importance of the corresponding terminal device increases.

8. The information processing device according to claim 7, wherein the N first standby terminal images are displayed in a vertical row horizontally to the communication partner image, and each of the N first standby terminal images is displayed higher the higher the importance of the terminal device corresponding to it.

9. The information processing device according to claim 1, wherein the importance is determined based on part or all of a waiting time, which is the elapsed time from the point at which a user of each of the terminal devices wishes to become the communication partner, the behavior of the user, and an inherent importance, which is an inherent importance that is set in advance in the terminal device.

10. The information processing device described in claim 1, wherein the display control means, when the communication partner is in an unselected state, causes the display means to display a standby screen including P second standby terminal images corresponding to P (P; a natural number greater than N and less than M) of the M terminal devices, and displays the P second standby terminal images in a manner according to the importance of each of the P terminal devices.

11. A display control method for an information processing device that includes a display unit and an operation receiving unit that can receive an operation from an operator, and that can communicate with M (M is a natural number equal to or greater than 2) terminal devices, comprising: a display control step of displaying various screens on the display means, In the display control step, when the operation receiving means receives an instruction to select any one of the M terminal devices as a communication partner, the display means displays an operation screen including a communication partner image corresponding to the communication partner and N first standby terminal images corresponding to N (N; a natural number less than M) of the M terminal devices, and displays the N first standby terminal images in a manner according to the importance of each of the N terminal devices.

12. A display control program for an information processing device that includes a display unit and an operation receiving unit that can receive an operation from an operator, and that can communicate with M (M is a natural number equal to or greater than 2) terminal devices, causing a computer of the information processing device to execute a display control procedure for displaying various screens on the display means; In the display control procedure, when the operation instructing to select any one of the M terminal devices as a communication partner is accepted by the operation accepting means, the display means displays an operation screen including a communication partner image corresponding to the communication partner and N first standby terminal images corresponding to N (N; a natural number less than M) of the M terminal devices, and displays the N first standby terminal images in a manner according to the importance of each of the N terminal devices.

Images