System, method and control device

Abstract

To provide a communication system that can improve the business efficiency.SOLUTION: A communication system 100 in which a first terminal device 10 operated by an operator and a second terminal device 70 operated by a guest communicate with each other via a network includes a proposal unit 98 that determines a proposal to the operator on the basis of voice data of the operator transmitted from the first terminal device or voice data of the guest transmitted from the second terminal device, and a first communication unit 91 that transmits the proposal to the first terminal device, and the first terminal device includes a second communication unit 11 that receives the proposal, and a display control unit 17 that displays the proposal received by the second communication unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a communication system, a terminal device, a communication method, and a program. [Background technology]

[0002] There is known a communication system in which terminal devices at remote locations transmit and receive images and audio to each other, display the images on a display, and output the audio from a speaker, thereby enabling remote communication such as a conference for users at remote locations. Each terminal device has a camera, a microphone, a display, and a speaker. An image captured by the camera is displayed on the display of the terminal device of the communication partner, and audio captured by the microphone is output from the speaker of the terminal device of the communication partner. By using such a communication system, for example, an operator at a call center can talk to (for example, serve) a guest who visits a store while looking at his or her face, and the guest can also look at the operator's face and receive an explanation from him or her.

[0003] Also, a technology for assisting conversation is known (see, for example, Patent Document 1). Patent Document 1 discloses a conversation analysis device that analyzes conversation in real time, analyzes the receptiveness of the user's conversation partner and the user's conversational skill, and extracts users (influencers) who have a large influence on others on a certain topic. Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional method of displaying the results of conversation analysis had the problem that it was difficult to improve the efficiency of various tasks using communication systems. For example, if the results of conversation analysis were simply displayed as numbers, it would take time for an operator to understand how to apply them to customer service.

[0005] In view of the above problems, an object of the present invention is to provide a communication system capable of improving business efficiency. [Means for solving the problem]

[0006] In view of the above problems, the present invention provides a communication system in which a first terminal device operated by an operator and a second terminal device operated by a guest communicate with each other via a network, the system including: a proposal unit that determines a proposal to the operator based on voice data of the operator transmitted from the first terminal device or voice data of the guest transmitted from the second terminal device; a first communication unit that transmits the proposal to the first terminal device; The first terminal device is characterized by having a second communication unit that receives the proposal, and a display control unit that displays the proposal received by the second communication unit. Effect of the Invention

[0007] It is possible to provide a communication system that can improve business efficiency. [Brief description of the drawings]

[0008] [Figure 1] FIG. 13 is a diagram illustrating how a communication system provides a suggestion for improving customer service efficiency. [Diagram 2] FIG. 1 is a schematic configuration diagram of an example of a communication system. [Diagram 3] FIG. 2 is a diagram illustrating a hardware configuration of an example of a terminal device. [Figure 4] FIG. 2 is a diagram illustrating a hardware configuration of an example of a management system. [Diagram 5] FIG. 2 is a diagram illustrating a software configuration of an example of a terminal device. [Figure 6] 2 is a functional block diagram of an example of a terminal device and a management system included in the communication system. [Figure 7] 2A to 2C are diagrams illustrating an example of a destination management DB, a presence management DB, and a destination management DB. [Figure 8] FIG. 2 is a functional block diagram of an example that explains the functions of the proposed control device by dividing them into blocks. [Figure 9] FIG. 4 is a diagram showing information held in a storage unit. [Figure 10] FIG. 11 is a sequence diagram illustrating an example of a process in which a terminal device logs in to a management system. [Figure 11] FIG. 13 is a diagram showing an example of a reception screen. [Figure 12] FIG. 11 is a sequence diagram showing an example of a process for transmitting presence information. [Figure 13] FIG. 11 is a flow diagram showing an example of a process for managing the presence of each feature. [Figure 14] FIG. 11 is a sequence diagram illustrating an example of a process for starting communication. [Figure 15] FIG. 13 is a diagram showing an example of a feature list screen displayed when the presence management table is in the state of Table 2. [Figure 16] FIG. 11 is a sequence diagram illustrating an example of a process for establishing communication between terminal devices. [Figure 17] FIG. 13 is a diagram illustrating an example of a message display. [Figure 18] 11 is an example of a sequence diagram relating to a process in which a proposal control device proposes a method for improving speaking style. [Figure 19] 11 is an example of a sequence diagram illustrating a process in which a proposal control device makes a proposal in response to a request from an operator. [Figure 20] FIG. 13 is a diagram showing an example of a call in progress screen displayed on an operator's terminal device; [Figure 21] FIG. 13 is a diagram showing an example of suggestions using icons. [Figure 22] FIG. 13 is a diagram showing an example of suggested conversation topics displayed on a screen during a call. [Figure 23] FIG. 13 is an example of a flowchart illustrating a process performed by the proposed control device to prompt the operator to speak louder. [Figure 24] FIG. 13 is an example of a flowchart illustrating a process performed by the proposed control device to prompt an operator to slow down the speaking speed. [Diagram 25] 11 is an example of a flowchart illustrating a process performed when a proposal control device proposes a conversation topic to an operator. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A communication system and a communication method performed by the communication system will be described below as an example of an embodiment of the present invention with reference to the drawings.

[0010] <Overall operation overview> FIG. 1 is a diagram for explaining how a communication system 100 of this embodiment provides a proposal for improving customer service efficiency. In FIG. 1, remote communication is taking place between two terminal devices 10 and 70. The remote communication is a meeting held at a remote base. A meeting is when people get together for consultation, discussion, etc. There are various types of meetings, such as customer service, conferences, gatherings, meetings, study sessions, classes, seminars, and presentations. It does not necessarily have to be two-way communication.

[0011] The terminal device 10 is assumed to be placed in a call center, and the terminal device 70 is assumed to be placed in a store. The terminal device 10 is operated by an operator a, and the terminal device 70 is operated by a guest g who visits the store. A Web camera 101 and a document camera 102 are connected to the terminal devices 10 and 70. In addition, a touch panel 190 is connected to the terminal device 70, so that the guest g can handwrite information.

[0012] (1) As shown in FIG. 1(a), during a call (while serving a customer), the terminal device 10 (an example of a first terminal device) displays an image (which may be a moving image or a still image) captured by the terminal device 70 (an example of a second terminal device) with the Web camera 101 or the document camera 102 on the display 720. Similarly, the terminal device 70 displays an image (which may be a moving image or a still image) captured by the terminal device 10 with the Web camera 101 or the document camera 102 on the display 720. The operator a and the guest g can talk to each other while looking at each other's faces.

[0013] (2) During a call, the management system 50 detects the number of times the guest asks the operator to repeat what he or she says, or at least the speaking speed of the operator. The reason why the guest asks the operator to repeat what he or she says is because it is difficult for the guest to hear what the operator says, and if the operator's voice is too quiet, the guest may have difficulty hearing what the operator says. Also, if the operator's speaking speed is too fast, the guest may have difficulty hearing what the operator says. Therefore, when at least one of the number of times the guest asks to repeat what he or she says or the speaking speed exceeds a threshold, the proposed control device 3 suggests to the operator how to improve his or her speaking style.

[0014] (3) Figure 1(b) is an example of a suggestion for improvement displayed on the operator's terminal device 10. In Figure 1(b), "Speech speed: You may want to slow down your speech speed a little." Therefore, the operator can continue serving the customer by slowing down the speech speed.

[0015] In this way, the communication system of the present embodiment proposes the results of analysis of voice data as specific improvement methods in real time, thereby making it possible to improve the efficiency of operations such as customer service.

[0016] <Terminology> A proposal refers to the submission of a proposal or a proposal that has been submitted. In this embodiment, any information that leads to improved business efficiency may be used. For example, the average time required for customer service may be shortened. The proposal may be displayed, output as a voice to a headset, or be suggested by an alarm sound.

[0017] <System configuration example> Fig. 2 is a schematic configuration diagram of an example of a communication system 100 of this embodiment. As shown in Fig. 2, the communication system 100 of this embodiment includes a plurality of terminal devices 10, 70 installed at each base, one or more relay devices 30, one or more management systems 50, and a proposed control device 3. Here, a base is a place that serves as a base for activities. The installation location of the terminal devices 10, 70 or the location of the user is a base. Three bases (a call center 8, a store 7X, and a store 7Y) are shown in Fig. 2.

[0018] Terminal devices 10a to 10d are arranged in the call center 8, terminal device 70x is arranged in the store 7X, and terminal device 70y is arranged in the store 7Y. The numbers of these are not limited to those shown in the figure. In addition, any one of the multiple terminal devices 70x, 70y will be referred to as a "terminal device 70", and any one of the multiple terminal devices 10a to 10d will be referred to as a "terminal device 10". Any one of the multiple stores 7X, 7Y will be referred to as a "store 7".

[0019] The call center 8 is a department that responds to inquiries from guests. In this embodiment, the person who responds to inquiries at the call center 8 is called an operator. They may also be called concierges, explainers, instructors, or advisors. The store 7 is a facility for conducting business. Actual products or service brochures are available, and guests can visit them. For example, a travel agency, a bank, a hospital, or any other facility that users visit may be considered. A guest who visits the store 7 operates a terminal device 70 that has been placed in advance to start a video conference with a terminal device 10 of the call center 8. This makes it possible to minimize the number of staff members stationed at the store 7, and to enable highly specialized staff members to hold video conferences with guests from various locations.

[0020] 2, the terminal device 10 for the operator is located in the call center 8, but the terminal device 10 for the operator may also be located in the store 7. For example, this is to allow the staff of the store 7 to act as an operator when the store 7 is quiet. As described below, the terminal devices 10 and 70 may be general information processing devices, and can be switched between operator and guest use by application software (application software for video conferencing).

[0021] The terminal devices 10 and 70 may be general information processing devices such as PCs (Personal Computers). They may also be dedicated terminals. Application software for operators is installed in the terminal device 10, and application software for guests is installed in the terminal device 70. Therefore, attribute information of each of the terminal devices 10 and 70 is determined by the application software. In this way, the attribute information may be determined by the application software, or the attribute information may be determined by a communication ID, which will be described later. In this case, each of the terminal devices 10 and 70 can use a table in which attribute information is associated with a communication ID.

[0022] The terminal devices 10 and 70 are each a general information processing device, and therefore can execute software such as presentation software and browser software in addition to application software for video conferencing. The terminal devices 10 and 70 can be PCs, tablet terminals, smartphones, PDAs (Personal Digital Assistants), etc. However, any device can be used as long as it is capable of executing application software. For example, the terminal devices 10 and 70 can be car navigation devices, television receivers, digital signage terminals, electronic whiteboards, projectors, etc. Also, the terminal devices 10 and 70 do not have to be the same type of information processing device.

[0023] It should be noted that dedicated terminal devices for video conferencing may be used as the terminal devices 10 and 70. As one form, there is a dedicated terminal device that logs in to the management system 50 and transmits and receives video and audio simply by turning on the power.

[0024] The communication system 100 of this embodiment is described as being applied to a video conference. However, the communication system 100 is also applied to a communication system or a data communication system. The communication system 100 also includes a data providing system that transmits image data or audio data unidirectionally from one terminal device 10 to multiple terminal devices 10 via a server. A video conference may also be called a video conference. In addition to a video conference, it may also be called a remote conference, remote consultation, or remote consulting. Three or more terminal devices 70 can communicate simultaneously.

[0025] The terminal device 10, the management system 50, and the relay device 30 are connected to a communication network 1 or 2. The communication network 1 may be a general network such as a LAN (Local Area Network), WiFi (Wireless Fidelity), or the Internet. For example, the communication network 1 may include a dedicated line such as a wide area Ethernet (registered trademark) or a VPN (Virtual Private Network). In the case of wireless communication, the terminal device connects to the communication network 2 (mobile communication network) using a base station 2a as an access point. The communication network 2 includes locations where wireless communication such as 3G, 4G, LTE (Long Term Evolution), 5G, and Bluetooth (registered trademark) is performed. The communication networks 1 and 2 are connected by a gateway.

[0026] In the communication system 100 of this embodiment, a plurality of terminal devices 10, 70 transmit and receive image data (including still images and moving images, which may also be referred to as video data), audio data, and the like, via a relay device 30. At this time, a management information session for transmitting and receiving various types of management information is established between the plurality of terminal devices 10, 70 via a management system 50. Also, a data session for transmitting and receiving image data and audio data is established between the plurality of terminal devices 10, 70 via the relay device 30. Here, in particular, the image data transmitted and received in the data session is scalably encoded data, and for example, encoded data of high quality video, encoded data of medium quality video, and encoded data of low quality video are transmitted and received on separate channels.

[0027] As a function of the video conference, the terminal device 10, 70 has a function to transmit image data captured by a camera (imaging device), audio data collected by a microphone, and material data used for explanations, etc., to the terminal device 10, 70 at another base participating in the video conference via the relay device 30. It can also transmit handwritten information to the terminal device 10, 70 at another base. It also has a function to display image data, material data, and handwritten information received from the terminal device 10, 70 at another base on a display, and output received audio data from a speaker. Image data, material data, handwritten information, and audio data may be collectively or one or more of them may be referred to as content data.

[0028] When a video conference is started, the terminal devices 10 and 70 transmit image data, audio data, material data, and handwritten information to the relay device 30, and the relay device 30 transmits image data, audio data, material data, and handwritten information to the other terminal devices participating in the video conference. In addition, the terminal devices 10 and 70 receive image data, audio data, material data, and handwritten information from the other terminal device 10 via the relay device 30.

[0029] The management system 50 manages the communication IDs of the terminal devices 10 and 70, and functions as a call control server that performs call control such as starting a session between two or more terminal devices when an incoming call is received. As the management system 50 is a server, it has the functions of an information processing device. Call control refers to a series of processes for making and disconnecting calls, such as making a call to start communication, receiving the call, responding to the call, and disconnecting the communication. In addition, the management system 50 often performs authentication, searches, and alive monitoring of the terminal devices 10 and 70. Furthermore, the management system 50 also manages a database necessary for controlling the communication system 100, such as authentication information for the terminal devices 10 and 70.

[0030] The relay device 30 is a server (information processing device) that relays image data, audio data, document data, and handwritten data from a terminal device 10, 70 at one base to a terminal device 10, 70 at another base. The management system 50 monitors the bandwidth of the communication network 2 and the communication load of the relay device 30, and assigns an appropriate relay device 30 to the terminal devices 10, 70.

[0031] The management terminal 40 is an information processing device used by a store manager or the like. Browser software runs on the management terminal 40. The management terminal 40 may be a PC, a tablet terminal, a smartphone, a PDA, or the like, as long as the browser software runs on it.

[0032] The proposal control device 3 analyzes the voice data of the operator and each guest. The terminal devices 10 and 70 transmit the voice data to the relay device 30, which then transfers the voice data to the proposal control device 3. Alternatively, the relay device 30 may also function as the proposal control device 3, or the management system 50 may also function as the proposal control device 3. The proposal control device 3 performs voice recognition on the voice data of the operator and the guest, and converts it into text data. This makes it possible to detect whether the guest has asked for replies and the speaking speed of the operator. The proposal control device 3 can also analyze words contained in the text data to suggest conversation themes.

[0033] <Hardware configuration example> <<Terminal Device>> Fig. 3 is a hardware configuration diagram of an example of the terminal device 10, 70. As shown in Fig. 3, the terminal device 10, 70 includes a CPU 701 that controls the operation of the entire terminal device, a ROM 702 (Read Only Memory) that stores a program, a RAM 703 used as a work area for the CPU 701, a HDD 704 (Hard Disk Drive) that reads or writes data under the control of the CPU 701, a media I / F 707 that controls reading or writing (storing) data from or to a recording medium 706 such as a flash memory, and cameras 712a, 712b that capture an image of a subject and obtain image data under the control of the CPU 701.

[0034] The HDD 704 stores an operating system (OS) executed by the CPU 701, other programs, and various data. The cameras 712a and 712b are charge-coupled devices that convert light into electric charges to digitize an image of a subject. The cameras 712a and 712b may be complementary metal oxide semiconductor (CMOS) sensors or charge coupled device (CCD) sensors. There may be three or more cameras 712a and 712b. Any one of the cameras 712a and 712b is referred to as "camera 712."

[0035] Furthermore, the terminal device 10, 70 is equipped with a microphone 714 that converts sound into a sound signal, a speaker 715 that converts a sound signal into sound, an antenna 711a, a communication unit 71 that uses this antenna 711a to communicate with the nearest base station 2a via wireless communication signals, a display 720 such as a liquid crystal or organic EL display that displays an image of a subject and various icons, etc., a touch panel 721 placed on this display 720 and composed of a pressure-sensitive or electrostatic panel, and that detects a touch position on the display 720 by touch with a finger, a touch pen, etc., and a bus line 710 such as an address bus or a data bus for electrically connecting each of the above-mentioned units.

[0036] Also, in FIG. 3, there is one microphone 714 and one speaker 715, but there may be two or more microphones 714 and speakers 715.

[0037] <<Management System>> 4 is a hardware configuration diagram of an example of the management system 50. The management system 50 includes a CPU 201 for controlling the operation of the entire management system 50, a ROM 202 storing a program used to drive the CPU 201, such as an IPL (Initial Program Reader), a RAM 203 used as a work area for the CPU 201, a HD 204 for storing various data such as programs for the management system 50, a HDD 205 (Hard Disk Drive) for controlling reading or writing of various data from the HD 204 under the control of the CPU 201, a media I / F 207 for controlling reading or writing (storing) of data from a recording medium 206 such as a flash memory, a display 208 for displaying various information such as a cursor, a menu, a window, a character, or an image, a network I / F 209 for data communication using the communication network 2, a keyboard 211 having a plurality of keys for inputting characters, numerical values, various instructions, etc., a mouse 212 for selecting and executing various instructions, selecting a processing target, moving the cursor, etc., a CD-ROM 213 (Compact Disc Read Only Memory) as an example of a removable recording medium, and a memory card 214 (RAM). The computer system includes a CD-ROM drive 214 that controls the reading and writing of various data from and to the CD-ROM (CD-ROM memory), and bus lines 210 such as an address bus and a data bus for electrically connecting the above components as shown in FIG.

[0038] The relay device 30 or the management terminal 40 has the same hardware configuration as the management system 50, and therefore a description thereof will be omitted.

[0039] <Application software> Fig. 5(a) is a software configuration diagram of the terminal device 10 according to an embodiment. An operator's application software A1 is installed in the terminal device 10. As shown in Fig. 5(a), an OS 1020 and the operator's application software A1 run on a working area 1010 of a RAM 703 of the terminal device 10. Of these, the OS 1020 is basic software that provides basic functions and manages the entire terminal device 10. The operator's application software A1 is an application for communicating with other terminal devices 10, 70.

[0040] Fig. 5(b) is a software configuration diagram of a terminal device 70 according to an embodiment. A guest application software A7 is installed in the terminal device 70. As shown in Fig. 5(b), an OS 1020 and the guest application software A7 run on a working area 1010 of a RAM 703. Of these, the OS 1020 is basic software that provides basic functions and manages the entire terminal device 70. The guest application software A7 is an app for communicating with other terminal devices 10, 70.

[0041] However, the guest application software A7 has limited functionality compared to the operator application software A1. This is due to a design policy that essentially the operator performs all operations, and the guest only needs to perform minimal operations. As an example, the operations that the guest can perform are as follows: 1. Press the operator call button 2. Adjusting the volume of the Speaker 715 3. Switch between displaying and hiding your location's image Examples of communication protocols for application software include (1) SIP (Session Initiation Protocol), (2) H.323, (3) an extended protocol of SIP, (4) an instant messenger protocol, (5) a protocol using the MESSAGE method of SIP, (6) an Internet Relay Chat protocol (IRC (Internet Relay chat)), and (7) an extended protocol of the instant messenger protocol. Among these, the (4) instant messenger protocol is, for example, (4-1) XMPP (Extensible Messaging and Presence Protocol), or (4-2) a protocol used in ICQ (registered trademark), AIM (registered trademark), or Skype (registered trademark), etc. Also, the (7) extended protocol of the instant messenger protocol is, for example, Jingle.

[0042] <About the function> 6 is a functional block diagram of an example of the terminal devices 10, 70 and the management system 50 included in the communication system 100 of the present embodiment. Note that the relay device 30 only relays image data, so its function is omitted in FIG.

[0043] <<Functional configuration of terminal device>> The following describes the functions of the terminal device 10 used by the operator of the call center 8 and the terminal device 70 used by the guest visiting the store 7. The difference in these functions is the difference in application software. However, the functions may be different if the terminal device of the call center 8 and the terminal device of the store 7 run the same application software and provide operations that can only be performed by the operator after authentication.

[0044] ·Call center terminal equipment 10 The terminal device 10 of the call center 8 has a communication unit 11, an operation input receiving unit 12, a startup unit 13, a display control unit 17, an imaging unit 18, an audio output unit 19, an audio acquisition unit 20, and a storage / readout processing unit 29. Each of these units is a function or means realized by any of the components shown in FIG.

[0045] The terminal device 10 also has a storage unit 1000 constructed by the RAM 703 and HDD 704 shown in Fig. 3. The storage unit 1000 stores application software A1 for the operator downloaded from the management system 50 or the like, and also temporarily holds image data, voice data, document data, and handwritten information. Attribute information (operator) set by the application software A1 for the operator is also stored.

[0046] The storage unit 1000 also stores a communication ID (Identification) for identifying the terminal device 10, a password, and the like. ID is an abbreviation of Identification and means an identifier or identification information. An ID refers to a name, a code, a character string, a number, or a combination of one or more of these, used to uniquely distinguish a specific object from multiple objects. The same applies to other IDs.

[0047] The communication unit 11 (an example of a second communication unit) transmits and receives various data (or information) to and from the management system 50 and the relay device 30 (which communicates with other terminal devices 10 via these) via the communication network 2. The communication unit 11 starts receiving each piece of status information indicating the status of each terminal as a candidate for a communication partner from the management system 50 before starting a call with a desired destination terminal.

[0048] The operation input receiving unit 12 receives various inputs to the terminal device 10 by an operator of the call center 8. For example, when a user turns on a power switch, the operation input receiving unit 12 receives the power-on command and notifies the start-up unit 13.

[0049] The startup unit 13 starts the terminal device 10 by turning on the power. For example, it executes an IPL, reads out the operator's application software A1 from the storage unit 1000, and transfers it to the RAM 703 in Fig. 3. Furthermore, the startup unit 13 causes the CPU 701 to execute the operator's application software A1, thereby performing processing such as logging the terminal device 10 into the management system 50.

[0050] The display control unit 17 performs control for sending the image data, document data, and handwritten information transmitted from the relay device 30 to the display 720. The display control unit 17 also causes the display 720 to display various screens.

[0051] The imaging unit 18 converts the imaging data obtained by imaging a subject with the cameras 712a and 712b into predetermined image data and outputs the image data. The imaging unit 18 can output images by switching between the cameras 712a and 712b.

[0052] The audio output unit 19 reproduces and outputs the audio data transmitted from the relay device 30 from the speaker 715 .

[0053] After the operator's voice is converted by the microphone 714 into a voice signal, the voice acquisition unit 20 converts the voice signal into predetermined voice data and outputs it.

[0054] The storage / read processing unit 29 performs processing such as storing various data in the storage unit 1000 and reading out various data stored in the storage unit 1000.

[0055] ·Store terminal device 70 The terminal device 70 in the store 7 has a communication unit 71, an operation input receiving unit 72, a startup unit 73, a display control unit 77, an imaging unit 78, an audio output unit 79, an audio acquisition unit 80, a status management unit 81, a transmission control unit 82, and a storage / readout processing unit 89. In the explanation of these, differences from the terminal device 10 for a call center will be mainly explained.

[0056] The status management unit 81 is realized by an instruction from the CPU 701, and manages the presence of destination candidates (operators) based on the presence information sent from the terminal device 10.

[0057] The transmission control unit 82 is realized by an instruction from the CPU 701, and controls the transmission of a call start request to a destination candidate based on the state of the destination candidate.

[0058] In the storage unit 7000 of the terminal device 70 in the store 7, in addition to attribute information (guest) 7004, a destination management DB 7001, a presence management DB 7002, and a transmission destination management DB 7003 are constructed.

[0059] 7(a) shows an example of a destination management table stored in the destination management DB 7001. In the destination management table, the communication ID of the terminal device 10 of the operator who is a destination candidate, the presence indicating the state of the destination candidate, and characteristic information indicating the characteristics of this destination candidate are associated and managed. This characteristic information is an abbreviation of the item. The communication ID is information for identifying the communication destination in the communication system 100. The communication ID is not particularly limited, but includes, for example, identification information of the terminal device 10, 70, identification information of the user of the terminal device 10, 70, an email address, etc. The presence will be described in detail later.

[0060] 7(b) is a conceptual diagram showing a presence management table stored in the presence management DB 7002 (an example of a state management information storage unit). In the presence management table, characteristic information indicating the characteristics of the destination candidate, a presence integrating each presence of each destination candidate of the characteristic, a communication ID of a terminal device 10 having a presence of "online" that is available for a call among the destination candidates of the characteristic, and a communication ID of a terminal device 10 having a presence of "chat" that is in a call among the destination candidates of the characteristic are associated and managed. The process of integrating each presence of each destination candidate to generate a presence of the characteristic will be described later.

[0061] 7(c) is a conceptual diagram showing a destination management table stored in the destination management DB 7003. The terminal device 70 is used by a guest, and the terminal device 10 is used by an operator. The terminal device 70 can transmit call start requests to multiple terminal devices 10 at the same time. The destination management table manages the communication IDs of the terminal devices 10 to which the current call start request is transmitted by the terminal device 70. The number of communication IDs managed in the destination management table changes depending on the process of transmitting the call start request by the terminal device 70 and the process of canceling the call start request.

[0062] <Management system functional configuration> Returning to Fig. 6, the management system 50 has a communication unit 51, an authentication unit 52, a management unit 53, a session control unit 54, a screen information generation unit 57, and a storage / readout processing unit 59. These units are functions realized when any of the components shown in Fig. 4 operates in response to an instruction from the CPU 201 in accordance with a program for the management system 50 that is loaded from the HD 204 onto the RAM 203. The management system 50 also has a storage unit 5000 constructed by the HD 204. Furthermore, in the storage unit 5000, each DB is constructed by each table as shown below.

[0063] 7(d) is a conceptual diagram showing an authentication management table stored in the authentication management DB 5001. In the authentication management table, an authentication password is associated with each communication ID of all the terminal devices 10 and 70 managed by the management system 50 and managed.

[0064] 7(e) is a conceptual diagram showing a terminal management table stored in the terminal management DB 5002. In the terminal management table, the IP addresses of the terminal devices 10 and 70 are associated with and managed for each communication ID of the terminal devices 10 and 70.

[0065] <Functional configuration of the management system> Next, a detailed description will be given of each functional configuration of the management system 50. The communication unit 51 is realized by commands from the CPU 201 and the network I / F 209, and transmits and receives various data (or information) to and from each terminal device or system via the communication network 2.

[0066] The authentication unit 52 is realized by instructions from the CPU 201, and searches the authentication management table using the communication ID and password received by the communication unit 51 as search keys, and authenticates the terminal by determining whether the same communication ID and password are managed in this authentication management table.

[0067] The management unit 53 is realized by an instruction from the CPU 201, and manages the terminal devices 10 and 70 by recording information about the terminal devices 10 and 70 in the terminal management table shown in FIG.

[0068] The session control unit 54 controls a session for transmitting content data between the terminal devices 10 and 70 in response to an instruction from the CPU 201. This control includes control for establishing a session, control for making the terminal devices 10 and 70 participate in the established session, control for leaving the session, and the like.

[0069] The screen information generating unit 57 generates screen information for a screen displayed by the management terminal 40 as a Web server. The screen information is created using HTML, XML, CSS (Cascade Style Sheet), JavaScript (registered trademark), and the like. In this embodiment, screen information for displaying a call in progress screen is generated. Such a screen may be called a Web page, and the Web page may be provided by a Web application. A Web application refers to software or a mechanism thereof that operates by cooperation between a program written in a programming language (e.g., JavaScript (registered trademark)) that runs on a browser and a program on the Web server side, and that is executed on a browser. Web pages can be dynamically changed by the Web application.

[0070] The storage / readout processing unit 59 is realized by instructions from the CPU 201 and the HDD 205, or by instructions from the CPU 201, and performs processing such as storing various data in the memory unit 5000 and extracting various data stored in the memory unit 5000.

[0071] <Administrative terminal> The management terminal 40 has a communication unit 41, an operation reception unit 42, and a display control unit 43. Each of these functional units of the management terminal 40 is realized by any of the components shown in Fig. 4 operating in response to an instruction from the CPU 201 in accordance with a program for the management terminal 40 that is loaded from the HD 204 onto the RAM 203. This program is assumed to be browser software. The management terminal 40 is used by the store manager to obtain information on each operator from the proposed control device 3 and display it.

[0072] The communication unit 41 included in the management terminal 40 transmits and receives various information to and from the management system 50, etc. For example, the communication unit 41 requests a web page of a portal site by HTTP communication, and transmits various operation contents for the web page to the management system 50.

[0073] The operation reception unit 42 receives various operations for the Web page (screen) displayed by the management terminal 40. The display control unit 43 analyzes various screen information from the management system 50 and displays it on the display 208.

[0074] <Proposed control device> Fig. 8 is a functional block diagram of an example in which the functions of the proposed control device 3 are divided into blocks. The proposed control device 3 has a communication unit 91, a voice recognition unit 92, a volume recording unit 93, a tone determination unit 94, a repeat detection unit 95, a speaking speed detection unit 96, a video analysis unit 97, and a proposal unit 98. These units are functions realized by operating any of the components shown in Fig. 4 by an instruction from the CPU 201 according to a program for the proposed control device 3 that is expanded from the HD 204 onto the RAM 203. The proposed control device 3 also has a memory unit 99 constructed by the HD 204. Furthermore, in the memory unit 99, each memory unit is constructed by each table as shown below.

[0075] FIG. 9(a) is a conceptual diagram showing a utterance management table stored in the utterance storage unit 3001. In the utterance management table, text data uttered by the operator and guest is recorded for each word. However, for the operator, it is sufficient to convert the data into text data, and it is not necessary to classify the data into words. The text data is classified into parts of speech by morphological analysis. In addition, the volume of the voice, tone, facial expression, etc. when the operator or guest uttered the word are registered. The volume of the voice, tone, or facial expression do not have to be registered for each word, but may be registered for a chunk of utterance, such as a sentence, a certain period of time, or until the conversation stops. Tone is a tone of voice, and represents the pitch and brightness of the voice.

[0076] 9(a) is generated for each store, but if there are multiple guests, the proposed control device 3 may identify the multiple guests by voice recognition and generate a comment management table for each guest. The same applies to the following tables.

[0077] 9B is a conceptual diagram showing a re-listening management table stored in the re-listening count storage unit 3002. In the re-listening management table, the number of times a guest has asked a question back for each conference is registered. The number of times a question has been asked back is initialized at regular intervals.

[0078] 9(c) is a conceptual diagram showing a conversation speed management table stored in the conversation speed storage unit 3003. In the conversation speed management table, the conversation speed [number of characters / 10 sec] between the operator and guest is registered for each conference. The guest's conversation speed is not required. The unit of conversation speed [number of characters / 10 sec] is just an example, and it may be the number of words per certain time. Also, it does not have to be per 10 seconds, such as the number of characters per minute. The conversation speed is updated at regular intervals.

[0079] FIG. 9(d) is a conceptual diagram showing the proposed information management table 1 stored in the proposed information storage unit 1 (reference numeral 3004). In the proposed information management table 1, a suggestion and a display method are registered in association with the number of times the guest has asked to repeat. For example, if the number of times the guest has asked to repeat is 20, the suggestion is "Speak a little louder," and the display method of the suggestion is "red text." The suggestion is not limited to text, and may be an icon, a voice message, an alarm sound, or the like.

[0080] FIG. 9(e) is a conceptual diagram showing the proposed information management table 2 stored in the proposed information storage unit 2 (reference numeral 3005). In the proposed information management table 2, a suggestion and a display method are registered in association with the speed difference. For example, when the speed difference is "-10", the suggestion is "please speak a little more slowly" and the display method is "red text". The speed difference is the difference between the operator's speaking speed and a predetermined reference value. If the operator's speaking speed is faster than the reference value, the value is a negative value, and if the opposite is true, the value is a positive value. The speed difference may be the difference in speaking speed between the guest and the operator.

[0081] FIG. 9(f) is a conceptual diagram showing a time information management table stored in the time information storage unit 3006. In the time information management table, destinations and average customer service times are registered. In the case of a travel agency, the customer service time varies depending on the guest's destination, so the average customer service time is registered for each destination. The destination may be a more detailed area name. The average customer service time is the average of the time each guest takes to serve. The time required for customer service is the time from the start to the end of a video conference (call). Based on the average customer service time, the proposed control device 3 can propose a guideline for the customer service time.

[0082] <<Functional configuration of the proposed control device>> Next, a detailed description will be given of each functional configuration of the proposal control device 3. The communication unit 91 (an example of a first communication unit) receives audio data and video data from the relay device 30 via the communication network 2. The communication unit 91 also transmits suggestions regarding ways to improve speaking styles and conversation themes to the terminal device 10 of the operator.

[0083] The voice recognition unit 92 recognizes the voice data received by the communication unit 91 and converts it into text data. Any method of voice recognition may be used. For example, a specific method is a method of maximum likelihood estimation of statistics of a generative model represented by an HMM (hidden Markov model) or an N-gram model based on training data. Alternatively, a method of applying a long short-term memory (LSTM) to a recurrent neural network (RNN), which is a form of deep learning, is known.

[0084] Furthermore, the speech recognition unit 92 performs morphological analysis to convert the text data into words with parts of speech. This makes it possible to extract words and parts of speech. It is also possible to convert homonyms into appropriate words (for example, "umi" (ocean) can be converted into "umi" (sea) for detection). The speech recognition unit 92 registers the extracted words in the utterance storage unit 3001. It is preferable to record in the utterance storage unit 3001 for both the operator and the guest, but in this embodiment, it is sufficient to perform this for at least the guest. For the operator, conversion to text data is sufficient to detect the speaking speed.

[0085] The loudness recording unit 93 records at least the loudness (volume of voice) of the guest's speech in association with the word. The loudness of the speech may be the amplitude of the voice data (for example, the bit value at the time of PCM (pulse code modulation) conversion). Preferably, it is judged for each word, but it may be registered for a chunk of speech, such as a sentence, a certain period of time, or until the conversation is interrupted.

[0086] The tone judgment unit 94 judges the tone of the voice. The voice data is converted into a frequency-intensity relationship using a Fourier transform or the like, and a possible frequency range is determined for each guest. For example, the frequency range taken in the first few minutes may be used. Then, the range is divided into, for example, two equal regions, and it is determined which of the two regions the strongest frequency belongs to. The tone judgment unit 94 records at least the guest's tone in association with words. It is preferable to judge for each word, but it is sufficient to register for a chunk of speech, such as a sentence, a certain period of time, or until the conversation stops.

[0087] The re-listening detection unit 95 detects re-listening by comparing the text data detected by the voice recognition unit 92 with character strings stored in advance. For example, character strings such as "Once more," "Huh?", "I can't hear you," and "What did you say" are stored in advance, and when text data of a guest that matches these is detected, it detects that a re-listening has occurred. The re-listening detection unit 95 increments the number of re-listenings in the re-listening count storage unit 3002 by one each time it detects a re-listening.

[0088] The speaking speed detection unit 96 detects at least the speaking speed of the operator. The number of characters (syllables) of the text data recognized by the voice recognition unit 92 is counted at regular intervals to determine the speaking speed. The speaking speed of the guest may also be recorded. This regular interval may be the same as the proposed period, but does not have to be the same. Operator conversation speed SO = number of characters / fixed time Guest speaking speed SG=number of characters / certain time The conversation speed detection unit 96 periodically registers the conversation speed between the operator and the guest in the conversation speed storage unit 3003. The average of the past conversation speeds may also be registered.

[0089] The video analysis unit 97 analyzes the video data of the operator and the guest. For example, it is determined whether or not they are smiling. For the operator, whether or not they are smiling is, for example, periodically registered, and for the guest, facial expressions (smiling, normal) are registered in the utterance storage unit 3001. For example, deep learning may be used to detect smiling faces. The information processing device learns image data in advance, which is given as a teacher signal as to whether or not a face is smiling, using a neural network such as a CNN (Convolutional Neural Network). Through learning, the video analysis unit 97 can output whether or not the input video data is smiling.

[0090] The suggestion unit 98 transmits a suggestion according to the number of times to ask again or the speaking speed to the operator's terminal device 10 at regular time intervals or in response to a request from the operator. For example, in the case where the suggestion is based on the number of times to ask again, it determines that a suggestion should be transmitted if the number of times to ask again is greater than the number in the proposed information storage unit 1. In the case where the suggestion is based on the speaking speed, it calculates a "reference value - SO" and determines that a suggestion should be transmitted if this value is greater than the speed difference in the proposed information storage unit 2.

[0091] The suggestion unit 98 can also represent the list of words stored in the utterance storage unit 3001 as visual information such as a graph, and transmit the same to the terminal device 10 of the operator.

[0092] <Communication system processing and operation> Next, a process in which the terminal devices 10 and 70 log in to the management system 50 will be described with reference to Fig. 10. Fig. 10 is a sequence diagram showing an example of a process in which the terminal devices 10 and 70 log in to the management system 50.

[0093] When a staff member or the like of the store 7 turns on the power switch of the terminal device 70, the operation input reception unit 72 receives the power ON and starts up the terminal device 70 (step S1). When the terminal device 70 starts up, the start-up unit 73 starts up an application for guests installed in the terminal device 70 (step S2). Thereafter, processing in the terminal device 70 is executed according to commands from the application for guests.

[0094] The communication unit 71 of the terminal device 70 transmits a login request to the management system 50 via the communication network 2 (step S3). This login request includes a communication ID for identifying the terminal that is the source of the login request, and a password. In this embodiment, the communication ID of the terminal device 70 is represented by the initial letter "C". Hereinafter, the communication IDs of the terminal device 70X and the terminal device 70Y are assumed to be "C01, C02".

[0095] The communication unit 51 of the management system 50 receives the login request. When the login request is transmitted from the terminal device 70 to the management system 50, the management system 50, which is the receiving side, can obtain the IP address of the terminal device 70, which is the transmitting side.

[0096] Next, the authentication unit 52 of the management system 50 searches the authentication management table shown in FIG. 7(d) in the memory unit 5000 using the communication ID and password included in the login request as search keys, and performs authentication by determining whether the same communication ID and password are managed in this authentication management table (step S4).

[0097] If the authentication unit 52 authenticates that the login request is from a terminal device 70 with valid usage authority, the management unit 53 records the communication ID and IP address of the terminal device 70 that made the login request in the terminal management table shown in Figure 7 (e) in association with each other (step S5).

[0098] The communication unit 51 of the management system 50 transmits authentication result information indicating the authentication result by the authentication unit 52 to the terminal device 70 that has made the login request, via the communication network 2 (step S6). As a result, the communication unit 71 of the terminal device 70 receives the authentication result information. Below, a case will be described where the authentication of the terminal device 70X, 70Y that has made the login request is successful, and the terminal device 70X, 70Y has logged in to the management system 50.

[0099] On the other hand, when the operator of the terminal device 10 on the operator side turns on the power switch, the operation input reception unit 12 receives the power ON and starts up the terminal device 10 (step S11). When the terminal device 10 starts up, the start-up unit 13 starts up the application software A1 for the operator installed in the terminal device 10 (step S12). Thereafter, the processing in the terminal device 10 is executed according to the commands of the application software A1 for the operator.

[0100] The display control unit 17 of the terminal device 10 outputs to the display 720 a reception screen for receiving input of the characteristics of an operator who is a user of the terminal device 10 .

[0101] 11 is a diagram showing an example of a reception screen. In addition to fields 141, 142 for receiving input of a communication ID and a password, the reception screen 140 includes options 143 for describing countries or regions that can be handled as a travel destination as an operator's characteristics, and a selection check box 144. When the operator selects a country or region that can be handled and performs an operation of pressing an OK button 145 on the reception screen 140, the operation input reception unit 12 receives the input of the operator's characteristics (step S13-1).

[0102] The storage / read processing unit 29 stores the feature information corresponding to the selected feature in the storage unit 1000. Hereinafter, the feature information corresponding to the features "Hokkaido, Tohoku, Kanto, Chubu, Kinki, Sanin Sanyo, Shikoku, Kyushu" is respectively "hd, th, kt, cb, kk, ss, sk, ks". Therefore, in the checked state of Fig. 11, "hd, th, ks" are the feature information of this operator.

[0103] The terminal device 10 transmits a login request to the management system 50 and logs in to the management system 50 (steps S13-2, S14, S15, S16). This process is similar to the processes of steps S3, S4, S5, S6 between the terminal device 70 and the management system 50, so a description thereof will be omitted. However, the communication ID transmitted from the terminal device 10 to the management system 50 in the login request is represented by the initial letter "O" so that it can be identified as the terminal device 10 on the operator side. Hereinafter, it is assumed that the communication IDs of the terminal devices 10a, 10b, 10c are "O01, O02, O03". When the authentication of the terminal devices 10a, 10b, 10c that have made the login request is successful, the terminal devices 10a, 10b, 10c log in to the management system 50.

[0104] Next, a process of transmitting presence information indicating the status of the operator from the terminal device 10 to the terminal device 70 will be described with reference to Fig. 12. Fig. 12 is a sequence diagram showing an example of a process of transmitting presence information.

[0105] The application software A1 for the operator manages, in the storage unit 1000, events that trigger state transitions and the presence after transitions due to the occurrence of an event. The presence is used to identify whether the operator, who is the user of the terminal device 10, is in a state in which a call can be started. The presence indicating a state in which a call can be started includes "online", which indicates a state in which the terminal device 10 is logged in to the management system 50 and is not communicating with the terminal device 70 on the guest side. The presence indicating a state in which a call cannot be started includes "offline", which indicates a state in which the terminal device 10 is not logged in to the management system 50, and "chat", which indicates a state in which the terminal device 10 is logged in to the management system 50 but is already communicating with the terminal device 70 on the customer side and therefore cannot start a new call. Note that the presence is not limited to the above. For example, the presence may include a presence based on a user's input. An example of a presence indicating a state in which a call cannot be started among the presences based on a user's input is "away from desk".

[0106] An example of an event that triggers a transition to the "online" presence is the terminal device 10 receiving the authentication result information in step S16 above, or receiving an input from the user requesting to end a call. An example of an event that triggers a transition to the "offline" presence is the terminal device 10 logging out from the management system 50. An example of an event that triggers a transition to the "chat" presence is the establishment of communication between the terminal devices 70, 10 (step S105 described below). Note that the event that triggers a transition of the presence is not particularly limited as long as it is detectable, and can be set arbitrarily depending on the communication protocol and the processing contents by the application software.

[0107] When any of the above events occurs in the terminal device 10a (step S21), the storage / readout processing unit 29 reads out the characteristic information from the storage unit 1000. The characteristic information read out corresponds to the characteristic such as "Hokkaido" input by the operator, who is the user of the terminal device 10a, at the time of login on the reception screen 140 of FIG. 11. The communication unit 11 of the terminal device 10a transmits the presence information including the presence indicating the state after the transition due to the event occurrence in step S21, the characteristic information read out from the storage unit 1000, and the communication ID "O01" of the terminal device 10a that is the transmission source, to the management system 50 (step S22). For example, when the terminal device 10a receives the authentication result information in step S16, it transmits the presence information including the presence "online" to the management system 50. Also, when the terminal device 10a receives the request for logout, it transmits the presence information including the presence "offline" to the management system 50. Furthermore, in response to the establishment of communication with the terminal device 70, the terminal device 10a transmits presence information including the presence "chat" to the management system 50. If the characteristic information is not updated after the characteristic information is recorded in the storage unit 1000 in step S13-1, the terminal device 10a transmits presence information including the same characteristic information to the management system 50 every time an event occurs.

[0108] When the communication unit 51 of the management system 50 receives the presence information transmitted by the terminal device 10a, it transmits the received presence information to the terminal devices 70x, 70y of the store 7 that are logged in to the management system 50 (steps S23, S24). There are no particular limitations on the method for identifying the terminal device 70 on the store side that is logged in to the management system 50, but one example is a method of reading out a communication ID whose initial letter is "C" from the terminal management table in Fig. 7(e).

[0109] The communication unit 71 of the terminal devices 70x and 70y receives the presence information transmitted from the terminal device 10a of the call center 8 via the management system 50. The storage / read processing unit 89 of the terminal devices 70x and 70y associates the communication ID, presence, and feature information contained in the received presence information and records them in the destination management table of FIG. 7(a) (steps S25 and S26). If a record with the same communication ID as the communication ID contained in the received presence information exists in the destination management table, the storage / read processing unit 89 overwrites and updates the presence recorded in this record based on the newly received presence information. This allows the terminal devices 70x and 70y to manage the latest presence on the terminal device 10a side.

[0110] When the above event occurs in the terminal device 10b on the operator side, the terminal device 10b, the management system 50, and the terminal devices 70x and 70y execute processes similar to those in steps S21, S22, S23, S24, S25, and S26 between the terminal device 10a, the management system 50, and the terminal devices 70x and 70y (steps S31, S32, S33, S34, S35, and S36).When the above event occurs in the terminal device 10c on the operator side, the terminal device 10c, the management system 50, and the terminal devices 70x and 70y execute processes similar to those in steps S21, S22, S23, S24, S25, and S26 between the terminal device 10a, the management system 50, and the terminal devices 70x and 70y (steps S41, S42, S43, S44, S45, and S46). This allows the terminal devices 70x and 70y to manage the latest presence information on the terminal devices 10b and 10c.

[0111] Next, a process for managing the presence for each feature will be described with reference to FIG. 13. FIG. 13 is a flow diagram showing an example of a process for managing the presence for each feature. In the following, assuming that a guest side requests a call to the operator side, the operator side is represented as a destination candidate. If the destination candidate operator selects multiple features (for example, "Hokkaido, Tohoku, Kyushu") in step S13-1, the destination candidate belongs to a group of multiple features. The presence "online" of a feature indicates that, among the destination candidates of this feature, one of the destination candidates is in the presence "online" where a call can be made. The presence "chat" of a feature indicates that none of the destination candidates of this feature group is in the presence "online" state, and one or more destination candidates are in the presence "chat" where a call is being made.

[0112] The presence of a feature "offline" indicates that all of the destination candidates of this feature have the presence "offline." Below, the process of the terminal device 70x will be described, but the terminal device 70y can also execute the same process.

[0113] In steps S26, S36, and S46, each time the presence in the destination management table shown in FIG. 7(a) is updated, the status management unit 81 of the terminal device 70x acquires the communication ID, presence, and characteristic information recorded in the updated record (step S51).

[0114] The state management unit 81 of the terminal device 70x updates each record including the characteristic information acquired in step S51 in the presence management table shown in Fig. 7(b). That is, when the characteristic information "hd,th,ks" is acquired in step S51, the state management unit 81 updates each record including the characteristic information "hd,th,ks" in the presence management table.

[0115] As an example, the following describes the process when the presence information "online, (hd, th, ks), O01" is transmitted from the terminal device 10a to the terminal device 70x and this information is acquired in step S51 (online in step S52).

[0116] First, the process of updating the record in the presence management table in which the characteristic information "hd" is recorded in the loop process will be described. The state management unit 81 of the terminal device 70x refers to the field of the callable communication ID in the record in the presence management table in which the characteristic information "hd" is recorded. As a result, the state management unit 81 determines whether the communication ID "O01" acquired in step S51 is included in the field of the callable communication ID (step S53).

[0117] If it is determined that the communication ID "O01" acquired in step S51 is not included in the field of the callable communication ID in the presence management table (NO in step S53), the status management unit 81 adds the acquired communication ID "O01" to the field of the callable communication ID (step S54).

[0118] When the process of step S54 is completed or when the answer of step S53 is YES, the state management unit 81 of the terminal device 70x refers to the field of the busy communication ID in the record in which the characteristic information "hd" is recorded in the presence management table. As a result, the state management unit 81 determines whether the communication ID "O01" acquired in step S51 is included in the field of the busy communication ID (step S55).

[0119] If it is determined that the communication ID "O01" acquired in step S51 is included in the field of the busy communication ID in the presence management table (YES in step S55), the acquired communication ID is deleted from the field of the busy communication ID (step S56). By the process up to this point, the communication ID of the terminal device 10a is recorded only in the field of the callable communication ID in the record in which the characteristic information "hd" is recorded in the presence management table, regardless of whether the presence on the terminal device 10a side transitions from "chat" to "online" or from "offline" to "online".

[0120] When the process of step S56 is completed, or when the result of step S55 is NO, the state management unit 81 of the terminal device 70x refers to the presence of the characteristic in the record in which the characteristic information "hd" is recorded in the presence management table. As a result, the state management unit 81 determines whether the presence of the group is "online" (step S57).

[0121] If it is determined that the feature presence in the record in which the feature information "hd" is recorded is not "online" (NO in step S57), the state management unit 81 updates the feature presence to "online." If it is determined that the feature presence in the record in which the feature information "hd" is recorded is "online" (YES in step S57), the state management unit 81 does not update the feature presence in the record in which the feature information "hd" is recorded. With the processing up to this point, the feature presence in the record in which the feature information "hd" is recorded in the presence management table becomes "online" (step S58).

[0122] Since the characteristic information "hd,th,ks" was acquired in step S51, the state management unit 81 performs a loop process to similarly update each record in the presence management table in which the characteristic information "th,ks" is recorded. When the loop process is completed, the presence management table in Fig. 7(b) is updated as shown in Table 1. The change in presence is also reflected in the destination management table in Fig. 7(a).

[0123] [Table 1] Next, the presence information "chat, (hd, th, ks), O01" is transmitted from the terminal device 10a to the terminal device 70x, and the process when this presence information is acquired in step S51 will be described (chat in step S52).

[0124] First, the process of updating the record in the presence management table in which the characteristic information "hd" is recorded in the loop process will be described. The state management unit 81 of the terminal device 70x refers to the field of the callable communication ID in the record in the presence management table in which the characteristic information "hd" is recorded (see Table 1). As a result, the state management unit 81 determines whether the field of the callable communication ID includes the communication ID "O01" acquired in step S51 (step S63).

[0125] If it is determined that the communication ID "O01" acquired in step S51 is included in the field of the callable communication ID in the presence management table (YES in step S63), the acquired communication ID is deleted from the field of the callable communication ID (step S64).

[0126] When the process of step S64 is completed or when step S63 is NO, the state management unit 81 of the terminal device 70x refers to the field of the busy communication ID in the record in which the characteristic information "hd" is recorded in the presence management table (see Table 1). As a result, the state management unit 81 determines whether the communication ID "O01" acquired in step S51 is included in the field of the busy communication ID (step S65). If it is determined that the communication ID "O01" acquired in step S51 is not included in the field of the busy communication ID in the presence management table (NO in step S65), the acquired communication ID "O01" is added to the field of the busy communication ID (step S66). By the process up to this point, the communication ID of the terminal device 10a is recorded only in the field of the busy communication ID in the record in which the characteristic information "hd" is recorded in the presence management table.

[0127] When the process of step S66 is completed, or when the answer of step S65 is YES, the state management unit 81 of the terminal device 70x refers to the field of the callable communication ID in the record in which the characteristic information "hd" is recorded in the presence management table. As a result, the state management unit 81 determines whether the number of communication IDs recorded in the field of the callable communication ID in the record in which the characteristic information "hd" is recorded is 0 (step S67).

[0128] If the number of communication IDs recorded in the call-available communication ID field in the record in which the characteristic information "hd" is recorded is 0 (YES in step S67), the status management unit 81 updates the presence of the characteristic in the record in which the characteristic information "hd" is recorded to "chat" (step S68).

[0129] If the number of communication IDs recorded in the callable communication ID field in the record in which the characteristic information "hd" is recorded is not 0 (NO in step S67), the state management unit 81 does not update the presence of the feature in the record in which the characteristic information "hd" is recorded. In other words, if a communication ID is recorded in the callable communication ID field in the record in which the characteristic information "hd" is recorded, the corresponding presence remains "online" and is not updated.

[0130] Since the characteristic information "hd,th,ks" is acquired in step S51, the state management unit 81 similarly updates each record in which the characteristic information "th,ks" is recorded in the presence management table in the loop process. As a result, the presence management table of Table 1 becomes as shown in Table 2.

[0131] [Table 2] Next, a case will be described in which the presence information "offline, (hd, kt, cb), O03" is transmitted from the terminal device 10c to the terminal device 70x, and this presence information is acquired in step S51 (offline in step S52).

[0132] First, the process of updating the record in the presence management table in which the characteristic information "hd" is recorded in the loop process will be described. The state management unit 81 of the terminal device 70x refers to the call available field in the record in the presence management table (see Table 2) in which the characteristic information "hd" is recorded. As a result, the state management unit 81 determines whether the communication ID "O03" acquired in step S51 is included in the call available communication ID field (step S73).

[0133] If it is determined that the communication ID "O03" acquired in step S51 is included in the field of the callable communication ID in the presence management table (YES in step S63), the acquired communication ID "O03" is deleted from the field of the callable communication ID (step S74).

[0134] When the process of step S74 is completed or when the result of step S73 is NO, the state management unit 81 of the terminal device 70x refers to the field of the busy communication ID in the record in which the characteristic information "hd" is recorded in the presence management table (see Table 2). As a result, the state management unit 81 determines whether the communication ID "O03" acquired in step S51 is included in the field of the busy communication ID (step S75).

[0135] If it is determined that the communication ID "O03" acquired in step S51 is included in the field of the busy communication ID in the presence management table (YES in step S75), the acquired communication ID "O03" is deleted from the field of the busy communication ID (step S76). Through the process up to this point, the communication ID of the terminal device 10c is deleted from the record in which the characteristic information "hd" is recorded in the presence management table.

[0136] When the process of step S76 is completed or when the answer is NO in step S75, the state management unit 81 of the terminal device 70x refers to the fields of the available communication ID and the busy communication ID in the record in which the characteristic information "hd" is recorded in the presence management table (see Table 2). As a result, the state management unit 81 judges whether the conditions IF1 and IF2 are satisfied (step S77). The condition IF1 is a case in which the number of communication IDs recorded in the available communication ID field in the record in which the characteristic information "hd" is recorded is 0, and the number of communication IDs recorded in the busy communication ID field is not 0. The condition IF2 is a case in which the number of communication IDs recorded in the available communication ID field in the record in which the characteristic information "hd" is recorded is 0, and the number of communication IDs recorded in the busy communication ID field is 0.

[0137] If the number of communication IDs recorded in the field of the available communication ID in the record in which the characteristic information "hd" is recorded is 0 and the number of communication IDs recorded in the field of the in-progress communication ID is not 0 (IF1 in step S77), the state management unit 81 updates the presence of the characteristic in the record in which the characteristic information "hd" is recorded in the presence management table to "chat" (step S78).

[0138] If the number of communication IDs recorded in the field of available communication IDs in the record in which the characteristic information "hd" is recorded is 0 and the number of communication IDs recorded in the field of in-progress communication IDs is 0 (IF2 in step S77), the status management unit 81 updates the presence of the characteristic in the record in which the characteristic information "hd" is recorded in the presence management table to "offline" (step S79).

[0139] If neither IF1 nor IF2 applies in step S77 (NO in step S77), the state management unit 81 does not update the presence of the feature in the record in which the feature information "hd" is recorded, and keeps it at "online."

[0140] Since the characteristic information "hd, kt, cb" is acquired in step S51, the state management unit 81 performs a loop process to update each record in the presence management table in which the characteristic information "kt, cb" is recorded. As a result, the presence management table of Table 2 becomes as shown in Table 3.

[0141] [Table 3] Next, the process of starting communication will be described with reference to Fig. 14. Fig. 14 is a sequence diagram showing an example of the process of starting communication. As an example, the process of starting communication by the terminal device 70x will be described below, but the terminal device 70y can also start communication by a similar process. The case will be described where, at the time of starting the process of Fig. 14, the destination management table is in the state of Fig. 7(a) and the presence management table is in the state of Table 2.

[0142] The operation input receiving unit 72 of the terminal device 70x receives a display request for a list of characteristics of the destination candidates in response to an operation input by a user. In response to the display request, the display control unit 77 acquires a set of each piece of characteristic information recorded in the presence management table (see Table 2) and the presence of the group corresponding to the characteristic information. The storage unit 7000 stores in advance a display image corresponding to the characteristic information and data of the display image corresponding to the presence of the characteristic. As an example, the data of the display image corresponding to the characteristic information "hd" is text data indicating "Hokkaido". Also, as an example, the data of the display image corresponding to the presence of the characteristic "online, offline, chat" is image data of normal brightness, low brightness, and normal brightness + speech bubble icon, respectively. The display control unit 77 acquires a set of data of the display image of the corresponding text and icon from the storage unit 7000 for each acquired set of characteristic information and presence. The display control unit 77 generates image data of the feature list by arranging the acquired sets of data of the display image in association with the same row in a predetermined display area. The display control unit 77 outputs image data of the generated feature list to the display 720 (step S81).

[0143] FIG. 15 is a diagram showing an example of a feature list screen displayed when the presence management table is in the state of Table 2. In FIG.

[0144] The display control unit 77 repeats the above process every time the presence management table is updated, whereby a feature list showing the latest presence for each feature is displayed on the display 720.

[0145] In step S81, the display control unit 77 outputs an item (button) with normal brightness for the item corresponding to "online," outputs an item (button) with low brightness for the item corresponding to "offline," and outputs an item with normal brightness + a speech bubble icon (button) for the item corresponding to "chat." In Fig. 15, speech bubble icons 111 are displayed for "Tohoku" 110b and "Kyushu" 110h. Also, "Sanin-Sanyo" 110f is displayed with low brightness.

[0146] For example, when a user presses an item corresponding to "Tohoku" or "Kyushu" in the feature list shown in Fig. 15, the operation input receiving unit 72 receives the operation input and displays a message that the call is in progress. On the other hand, when a user presses an item corresponding to "Sanin-Sanyo" in the feature list shown in Fig. 15, the operation input receiving unit 72 receives the operation input and displays a message that the call is in progress.

[0147] The following describes the process when the selection of the feature "Hokkaido" is accepted. The operation input accepting unit 72 accepts the selection of the feature corresponding to "Hokkaido" (step S82).

[0148] When the selection of the characteristic "Hokkaido" is accepted, the storage / read processing unit 89 searches the destination management table of FIG. 7(a) using the characteristic information "hd" indicating the characteristic selected in step S82 as a search key, and reads out the corresponding communication ID and presence pairs "O01, chat", "O02, online", and "O03, online" (step S83).

[0149] The communication unit 71 of the terminal device 70x transmits a call start request to the management system 50 (step S84-1), which includes the communication ID "O02" whose corresponding presence is "online" among the communication IDs read out in step S83, the communication ID "C01" of the terminal device 70x itself, and the feature information "hd" indicating the feature selected in step S82.

[0150] Similarly, the communication unit 71 of the terminal device 70x transmits a call start request to the management system 50 (step S86-1), which includes the communication ID "O03" whose corresponding presence is "online" among the communication IDs read out in step S83, the communication ID "C01" of the terminal device 70x itself, and the feature information "hd" indicating the feature selected in step S82.

[0151] When the communication unit 51 of the management system 50 receives a call start request transmitted by the terminal device 70x, the communication unit 51 transmits the call start request to the terminal device 10b identified by the communication ID "O02" included in the call start request (step S84-2). Similarly, when the communication unit 51 of the management system 50 receives a call start request transmitted by the terminal device 70x, the communication unit 51 transmits the call start request to the terminal device 10c identified by the communication ID "O03" included in the call start request (step S86-2).

[0152] The transmission control unit 82 of the terminal device 70x updates the destination management table by recording the communication ID "O02" of the terminal device 10b, which is the destination of the call start request, in the destination management table (step S85). Similarly, the transmission control unit 82 of the terminal device 70x updates the destination management table by recording the communication ID "O03" of the terminal device 10c, which is the destination of the call start request, in the destination management table (step S87).

[0153] When the input of a request to end the call is accepted at the terminal device 10a on the operator side (step S21'), the communication unit 11 transmits presence information including the characteristic information "hd, th, ks" read from the memory unit 1000, the communication ID "O01" of the terminal device 10a, and the presence "online" to the management system 50 (step S22').

[0154] When the communication unit 51 of the management system 50 receives the presence information transmitted by the terminal device 10a, it transmits the received presence information to the terminal device 70x on the guest side that has logged in to the management system 50 (step S24').

[0155] The communication unit 71 of the terminal device 70x receives the presence information transmitted from the terminal device 10a on the operator side via the management system 50. The storage / read processing unit 89 of the terminal device 70x records the communication ID, presence, and feature information included in the received presence information in the destination management table (step S26'). As a result, the destination management table is updated from the state of FIG. 7(a) to the state of Table 4.

[0156] [Table 4] The transmission control unit 82 of the terminal device 70x judges whether to transmit a call start request to the terminal device 10a that is the sender of the presence information received in step S24' (step S88). In this process, the transmission control unit 82 of the terminal device 70x refers to the presence of the terminal device 10a updated in the destination management table and judges whether this presence is "online". If the presence of the terminal device 10a is not "online", the transmission control unit 82 judges not to transmit a call start request and ends the process.

[0157] If it is determined that the presence of the terminal device 10a is "online", the destination management table refers to the characteristic information "hd, th, ks" of the terminal device 10a to determine whether any of the characteristic information indicates the characteristic "Hokkaido" selected in step S82. If none of the characteristic information indicates the selected characteristic, the transmission control unit 82 determines not to transmit a start request and ends the process.

[0158] If any of the feature information indicates the selected feature, the transmission control unit 82 of the terminal device 70x determines to transmit a call start request. In response to the determination to transmit a call start request, the communication unit 71 of the terminal device 70x transmits a call start request including the communication ID "O01" of the terminal device 10a whose presence information is to be transmitted, the communication ID "C01" of the terminal device 70x itself, and the feature information "hd" indicating the feature selected in step S82 to the management system 50 (step S89-1).

[0159] When the communication unit 51 of the management system 50 receives the call start request sent by the terminal device 70x, it transmits the call start request to the terminal device 10a identified by the communication ID "O01" included in the call start request (step S89-2).

[0160] Next, a process of establishing communication between the terminal devices 10 and 70 will be described with reference to Fig. 16. Fig. 16 is a sequence diagram showing an example of a process of establishing communication between the terminal devices 10 and 70. Hereinafter, a case where the terminal device 10a responds to a call start request from the terminal device 70x will be described.

[0161] The communication unit 11 of the terminal device 10a receives the call start request transmitted from the terminal device 70x via the management system 50 (see step S89-2). When the call start request is received, the display control unit 17 of the terminal device 10a outputs to the display 720 a message including request source information corresponding to the communication ID "CO1" included in the call start request and the characteristic "Hokkaido" corresponding to the characteristic information "hd" included in the call start request.

[0162] 17 is an example of a message display. By displaying the feature 151 in the message 150, the operator, who is the user of the terminal device 70, can know in advance, before starting a call, which country or region he or she will be asked to explain.

[0163] 17 includes a response button 152 for accepting permission to start a call. When the user presses the response button 152 on the screen of the terminal device 10a, the operation input acceptance unit 12 accepts the operation input for permission to respond (step S101). When the operation input for permission to respond is accepted, the communication unit 11 of the terminal device 10a transmits a response including the communication ID "C01" of the source of the call start request and the communication ID "O01" of the terminal device 10a to the management system 50 (step S102).

[0164] The communication unit 51 of the management system 50 receives the response transmitted by the terminal device 10a. The communication unit 51 of the management system 50 transmits the received response to the terminal device 70x identified by the communication ID “C01” included in the response (step S103).

[0165] When the communication unit 71 of the terminal device 70x receives the response, it transmits a communication establishment request to the management system 50 to establish communication with the terminal device 10a that transmitted the response (step S104). The communication establishment request includes the communication ID "C01" of the terminal device itself and the communication ID "O01" of the terminal device 10a that transmitted the response.

[0166] The communication unit 71 of the management system 50 receives the communication establishment request. The session control unit 54 of the management system 50 performs control to establish a session for transmitting content data, such as image data and sound data, between the terminal devices 10a and 70x identified by the communication ID "O01, C01" included in the communication establishment request via the relay device 30 (step S105). Note that the session is not limited to the above, and may be a session for directly transmitting content data between the terminal devices 10a and 70x via the communication network 2.

[0167] When communication between the terminal devices 10a and 70x is established, the terminal devices 10a and 70x transmit image data of an image captured by the terminal device itself and sound data collected by the terminal device itself to the terminal devices 70x and 10a on the other side via the relay device 30. The display control units 17 and 77 of the terminal devices 10a and 70x output the received content data. This allows a call to be started between the users.

[0168] <Proposal process> Next, the process in which the proposal control device 3 proposes a way to improve speaking to the operator after communication is established will be described with reference to Fig. 18 and Fig. 19. Fig. 18 is an example of a sequence diagram regarding the process in which the proposal control device 3 proposes a way to improve speaking. First, Fig. 18 describes the case in which proposals are made at regular intervals.

[0169] S201 and S202: When communication is established, the communication unit 11 of the terminal device 10 and the communication unit 71 of the terminal device 70 repeatedly transmit audio data and video data to the relay device 30.

[0170] S203: The relay device 30 receives these and transmits the audio data and video data to the proposed control device 3. Note that a communication ID is transmitted to the relay device 30 together with the audio data and video data, and the communication ID can be used to determine whether the data was transmitted by the guest or the operator.

[0171] S204: The communication unit 91 of the proposal control device 3 receives the voice data and the video data. The voice recognition unit 92 performs voice recognition and further performs morphological analysis, and registers the words and parts of speech in the utterance storage unit 3001 in association with each other.

[0172] S205: The loudness recorder 93 of the proposed control device 3 detects the loudness of the voice from the amplitude of the voice data and registers it in association with the word. The tone judger 94 performs a Fourier transform on the voice data, judges the tone according to the frequency of the maximum intensity, and registers it in association with the word. The video analyzer 97 analyzes the video data and registers the facial expression, whether it is a smile or not, in association with the word.

[0173] S206: The re-asking detection unit 95 of the proposal control device 3 detects re-asking by determining whether a pre-registered character string is included in the list of words uttered by the guest. If re-asking is detected, the number of re-askings in the re-asking count storage unit 3002 is updated (increased by 1). The conversation speed detection unit 96 detects the conversation speed between the operator and the guest based on the number of characters per certain period of time, and registers it in the conversation speed storage unit 3003. It is preferable to register the latest value, but it is also possible to register the average of past conversation speeds.

[0174] S207: The suggestion unit 98 judges whether or not to suggest to the operator a way to improve his / her speaking style at regular intervals, as described below. For example, it judges whether or not the number of times the guest asks to repeat exceeds the number in the suggested information storage unit 1, and if so, judges to make a suggestion. It also judges whether or not the difference between the operator's speaking speed and a reference value exceeds the speed difference in the suggested information storage unit 2, and if so, judges to make a suggestion. The suggestion unit 98 also compares the customer service time from the start of communication with the average customer service time in the time information storage unit 3006, and if the average customer service time has passed, judges to make a suggestion.

[0175] S208: If it is determined that a proposal should be made, the proposal unit 98 of the proposal control device 3 acquires a proposal and a display method from the proposal information storage unit 1 or the proposal information storage unit 2.

[0176] S209: The suggestion unit 98 transmits the suggestion and the display method to the terminal device 10 of the operator.

[0177] S210: The communication unit 11 of the operator's terminal device 10 receives the proposal and the display method, and the display control unit 17 displays the proposal on the in-call screen. An example of the in-call screen is shown in FIG.

[0178] In this way, the operator's terminal device 10 can display ways to improve speaking style at regular intervals during a call, making it easier to improve customer service efficiency.

[0179] FIG. 19 is an example of a sequence diagram illustrating a process in which the proposal control device 3 makes a proposal in response to a request from an operator.

[0180] S301 to S306: These may be the same as steps S201 to S206.

[0181] S307: The operator wants to know how to improve his / her speaking style, and inputs a predetermined operation into the terminal device 10. For example, the operator presses a proposal request button displayed on the operator's in-call screen. The operation input receiving unit 12 receives the press, and the communication unit 11 transmits a proposal request together with the communication ID to the management system 50. Note that proposal requests include not only those related to the number of times to ask again and the conversation speed, but also those related to the conversation theme. The operator may specify the content of the proposal and request it, or may request all possible proposals. In the explanation of FIG. 19, it is assumed that all proposals are requested.

[0182] S308: The communication unit 51 of the management system 50 receives the proposal request and transfers it to the proposal control device 3.

[0183] S309: The suggestion unit 98 of the proposal control device 3 judges whether or not to make a suggestion based on the number of times the guest has asked to repeat and the speaking speed. This judgment method may be the same as that shown in FIG. 18. If no suggestion is necessary, the suggestion unit 98 transmits a notice to that effect to the terminal device 10. As for suggestions related to conversation themes, a suggestion is always made regardless of whether or not to make a suggestion. In other words, the words stored in the utterance memory unit 3001 are weighted by one or more of the volume, tone, and facial expression of the voice, and a list of words in which the guest showed interest during the conversation is provided. Details will be described later.

[0184] S310 to S312: The proposal control device 3 transmits the proposal in the same manner as in steps S208 to S210, and the display control unit 17 of the operator's terminal device 10 displays it.

[0185] The suggestion unit 98 may suggest ways to improve speaking style based not only on the number of times the customer has to ask to repeat or the speaking speed as shown in Figs. 18 and 19, but also on the volume or tone of the voice. For example, if the volume of the voice is below a threshold, a suggestion is made such as "Speak louder." If the tone is low, a suggestion is made such as "Speak brighter." If there are few smiling faces (below a threshold) in the facial expressions analyzed from the video data, a suggestion is made such as "Serve customers with a smile."

[0186] <Example of suggestions displayed on the call screen> FIG. 20 is an example of a call in progress screen 300 displayed by the operator's terminal device 10. Since this is the operator's call in progress screen 300, guest g is shown large, and an image of operator a is displayed in picture-in-pin at the bottom right. In addition, a proposal request button 301 is displayed at the top right of the call in progress screen 300. The proposal request button 301 is a button that allows the operator to request a way to improve speaking style at any time. When the proposal request button 301 is pressed, a pull-down menu of request items 302 is displayed, and the operator can select a request item from "number of times to ask again," "conversation speed," "conversation topic," or "all."

[0187] When the suggestion control device 3 transmits a suggestion regarding "number of times to ask again" and "conversation speed," the in-call screen 300 displays the suggestion. In FIG. 20, a suggestion 303 saying "Speak a little more slowly" is displayed at the bottom of the in-call screen 300. The operator can see the suggestion and improve their speaking style while serving the customer. The suggestion 303 may be displayed anywhere. It may be displayed as a pop-up.

[0188] The suggestion 303 may be displayed as an icon or image data instead of text. FIG. 21 is a diagram showing an example of a suggestion using an icon. First, FIG. 21(a) shows an icon 304 that makes a suggestion regarding the conversation speed. For example, this icon 304 always moves on the in-call screen 300 at a speed corresponding to the conversation speed (the suggestion notifies the conversation speed at regular intervals), and when the conversation speed exceeds a threshold, the suggestion unit 98 calls attention by changing the color, for example. Alternatively, the icon 304 is not usually displayed on the in-call screen 300, and when the conversation speed exceeds the threshold, the suggestion unit 98 notifies the conversation speed, and the operator's terminal device 10 may display the icon 304 that moves at a speed corresponding to the conversation speed.

[0189] 21(b) also shows an icon 305 that indicates the speaking speed. This icon 305 imitates a vehicle speedometer, and is always displayed on the in-call screen 300. The faster the speaking speed, the more the needle 305a rotates clockwise and points to the surrounding color, so the operator can judge whether the speaking speed is appropriate or not by the color indicated by the needle 305a.

[0190] 21(c) shows an icon 306 that indicates the volume of the voice according to the number of times the user has asked to listen again. This icon 306 is a bar that imitates a volume meter, and is always displayed on the in-call screen 300. The louder the voice, the longer the bar becomes, and the more attention-grabbing the color displayed on the left side is, such as red. In other words, the bar is displayed in different colors from left to right. Therefore, when the volume is below a certain level, the icon 306 is displayed in red, and the operator can determine that the voice is quiet when the bar is only displayed in red.

[0191] Fig. 22 is a diagram showing an example of suggested conversation themes displayed on during-call screen 300. In Fig. 22, a pie chart 310 is displayed as a pop-up on during-call screen 300. The display method is one example, and the pie chart may be displayed on a separate screen or as part of during-call screen 300.

[0192] A conversation theme is an example of a conversation suggestion in which the proposal control device 3 suggests appropriate words to the operator from words contained in past conversations when the guest is unsure of which plan to choose, for example. The operator can easily identify words (themes) that the guest is interested in among various conversations that the guest has had.

[0193] In FIG. 22, the score calculated by weighting the words (nouns) registered in the utterance storage unit 3001 is displayed as a percentage. The calculation method will be described. The suggestion unit 98 acquires, for example, nouns from the words stored in the utterance storage unit 3001. If a noun has a synonym, it may be considered to be the same noun. For example, diving and skin diving are considered to be the same noun. This type of processing is called data cleansing. Data cleansing refers to finding duplications, typos, and variations in spelling in data stored in a database, and deleting, correcting, normalizing, and the like to improve the quality of the data.

[0194] Next, the loudness of the voice (loud / medium / small), the high / low tone, and the smiling / normal facial expression associated with the noun are converted into coefficients α, β, and γ, respectively, and weighted. The loudness of the voice corresponds to the coefficients αn (α1-α3), the high / low tone corresponds to the coefficients βn (β1, β2), and the smiling / normal facial expression corresponds to the coefficients γn (γ1, γ2).

[0195] Therefore, the score for each noun is calculated as follows: Score = Σ(αn × βn × γn) Σ indicates addition of the same noun.

[0196] The suggestion unit 98 calculates a ratio by, for example, adding up the scores of nouns whose scores are equal to or greater than a threshold (that is, removing nouns that appear only once) and dividing the score of each noun by the total value. Proportion of each noun = score of each noun / total score of each noun Create a pie chart like the one in Figure 22 based on the ratio of nouns.

[0197] It is even better to detect and weight the relationship between adjectives, adjectival verbs, or auxiliary verbs and nouns. For example, when a guest says "beautiful sea," he or she uses "sea" in a positive sense, but when a guest says "the sea was dirty," he or she uses "sea" in a negative sense. For this reason, when a positive adjective is detected for a noun, the score is increased, and when a negative adjective is detected for a noun, the score is decreased. Adjectival verbs can be judged in the same way. For example, auxiliary verbs are "tai" in "uchi ni kitai" (want to go to the sea) and "taku" and "nai" in "uchi ni kitai" (don't want to go to the sea). The former is positive about the sea and the latter is negative, so the score can be increased or decreased in the same way.

[0198] <Detailed explanation of the processing of the proposed control device> Fig. 23 is a flow chart for explaining the process in the case where the proposed control device 3 prompts the operator to speak louder. The process in Fig. 23 starts, for example, when a call is started.

[0199] First, the suggestion unit 98 acquires the information stored in the suggestion information storage unit 1, the suggestion information storage unit 2, and the time information storage unit 3006 (S401).

[0200] The communication unit 91 of the proposal control device 3 receives the voice data and the video data (S402). The voice recognition unit 92 analyzes the voice data and the video data and registers words and the like in the utterance storage unit 3001 (S403).

[0201] The re-listening detection unit 95 judges whether or not the guest has re-listened, and if so, updates the number of re-listening times in the re-listening times storage unit 3002 (S404).

[0202] The suggestion unit 98 judges whether a certain period of time has elapsed (S405), and every time the certain period of time has elapsed, judges whether the number of times to ask again stored in the number of times to ask again storage unit 3002 has exceeded the number (threshold) in the proposal information storage unit 1 (S406). Note that the same judgment is made not only after the certain period of time, but also when requested by the operator.

[0203] If the determination in step S406 is Yes, the suggestion unit 98 transmits a suggestion to the operator's terminal device 10 to encourage the operator to speak louder (S407).

[0204] The suggestion unit 98 initializes the number of times of relistening stored in the number-of-times-of-relistening storage unit 3002 (S408). In other words, it resets it to zero.

[0205] In this manner, the suggestion unit 98 can provide the operator with a suggestion to speak louder at regular intervals or when requested by the guest.

[0206] Fig. 24 is a flow chart for explaining the process in the case where the proposed control device 3 prompts the operator to slow down the speaking speed. The process in Fig. 24 starts, for example, when a call is started.

[0207] First, the suggestion unit 98 acquires the information stored in the suggestion information storage unit 1, the suggestion information storage unit 2, and the time information storage unit 3006 (S501).

[0208] The communication unit 91 of the proposal control device 3 receives the voice data and the video data (S502). The voice recognition unit 92 analyzes the voice data and the video data and registers words and the like in the utterance storage unit 3001.

[0209] The speaking speed detection unit 96 counts the number of characters spoken by the operator (S503). This number of characters is the number of syllables before being converted into kanji.

[0210] The conversation speed detection unit 96 determines whether a certain period of time has elapsed (S504), and each time the certain period of time elapses, calculates the conversation speed by dividing the number of characters spoken by the guest by the certain period of time (S505). The conversation speed is registered in the conversation speed storage unit 3003. The conversation speed is also calculated when requested by the operator. In this case, the conversation speed detection unit 96 calculates the conversation speed not based on the certain period of time, but based on the time from when the number of characters starts to be counted until the operator makes a request.

[0211] The suggestion unit 98 judges whether the difference between the conversation speed and the reference value exceeds the speed difference (threshold value) in the suggestion information storage unit 2 (S506).

[0212] If the determination in step S506 is Yes, the suggestion unit 98 transmits a suggestion to the operator's terminal device 10 to encourage the operator to slow down the speaking speed (S507).

[0213] The suggestion unit 98 initializes the speaking speed stored in the speaking speed storage unit 3003 (S508), i.e., resets it to zero.

[0214] In this manner, the suggestion unit 98 can provide the operator with suggestions to slow down their speaking speed at regular intervals or when requested by the guest.

[0215] In addition, in either or both of Figures 23 and 24, it may be determined whether the time elapsed from the start of a call exceeds the average customer service time, and a proposal to shorten the call time may be made. The proposal may be made not based on the average customer service time itself, but in cases such as average customer service time x 0.8 < call time, average customer service time x 1.2 < call time, etc. The operator can provide service while keeping track of the call time.

[0216] Fig. 25 is a flow chart for explaining the process when the proposal control device 3 proposes a conversation theme to an operator. The process in Fig. 25 starts when a proposal request is received from a guest.

[0217] First, the communication unit 91 of the proposal control device 3 receives a proposal request from the operator's terminal device 10 (S601).

[0218] The suggestion unit 98 acquires all words, voice volume, tone, and facial expressions recorded during a call from the utterance storage unit 3001 (S602).

[0219] Next, the suggestion unit 98 performs data cleansing on the acquired data (S603). The suggestion unit 98 converts the words after data cleansing into scores. The nouns that are the conversation topic are processed into visual information such as a pie chart and transmitted to the operator's terminal device 10 (S604). This allows the operator's terminal device 10 to display a pie chart 310 or the like. In addition to a pie chart, a bar graph or a word cloud may also be used. A word cloud is a method of dividing text data into words and illustrating each extracted word with a size according to its frequency of appearance (score) in the text.

[0220] In this way, since the suggestion unit 98 provides conversation topics when requested by a guest, the operator can guide the guest to topics that interest him or her, thereby improving customer service efficiency.

[0221] <Major Effects> As described above, the communication system of this embodiment proposes the results of analyzing voice data as specific improvement methods, thereby improving the efficiency of operations such as customer service. In addition, since the conversation topic is displayed, the operator can guide the conversation to a topic that interests the guest, thereby improving the efficiency of operations.

[0222] <Other application examples> The above describes the best mode for carrying out the present invention using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0223] For example, in the present embodiment, the method for improving customer service efficiency has been described using the example of customer service at a travel agency, but the present invention can also be suitably applied to other industries, such as finance, health consultation, and medical consultation.

[0224] In addition, it can also be applied to local government offices. For example, staff can serve customers according to the type of business they are responsible for (resident registration, family register, tax payment, bulky waste, pensions, etc.).

[0225] Furthermore, the management system 50 may have the functions of the relay device 30, or the relay device 30 may have the functions of the management system 50. Furthermore, the functions of the management system 50 may be realized by a plurality of servers, or a plurality of management systems 50 may exist.

[0226] In addition, the configuration examples shown in the above embodiments in Figures 6, 8, etc. are divided according to the main functions in order to facilitate understanding of the processing of the management system 50 and the terminal devices 10, 70. However, the present invention is not limited by the manner in which each processing unit is divided or the names of each processing unit. The management system 50 and the terminal devices 10, 70 can also be divided into more processing units according to the processing content. Also, one processing unit can be divided so as to include more processes.

[0227] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and a device such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), or a conventional circuit module designed to execute each function described above.

[0228] Also, in some embodiments, the management system, relay device, or proposed control device includes multiple computing devices, such as a server cluster, configured to communicate with each other over any type of communication link, including a network, shared memory, etc., to perform the processes disclosed herein. Similarly, the terminal device 10, 70 may include multiple computing devices configured to communicate with each other.

[0229] Furthermore, the management system, relay device, or proposed control device can be configured to share the disclosed processing steps in various combinations. For example, some of the processes performed by the management system, relay device, or proposed control device can be performed by the terminal device 10, 70. Also, some of the processes performed by the terminal device 10, 70 can be performed by the management system, relay device, or proposed control device. [Explanation of symbols]

[0230] 3 Proposed control device 7 stores 8. Call Center 50 Management Systems 10,70 Terminal equipment 100 Communication Systems [Prior art documents] [Patent documents]

[0231] [Patent Document 1] JP 2016-103081 A

Claims

1. A system including a first terminal device and a control device, The control device includes: a first receiving means for receiving voice data including a user's speech transmitted from the first terminal device when a call between the first terminal device and a second terminal device is started; a first transmitting means for transmitting the received voice data to the second terminal device; a second receiving means for receiving a request transmitted from the first terminal device in response to an operation of a button included in a call in progress screen displayed on the first terminal device while the call is being made, the button receiving a request for a proposal to the user; and a second transmission means for transmitting a proposal to the first terminal device based on the voice data when the request is received, The first terminal device is a first display means for displaying the call in progress screen including the button on a display unit; a third transmission means for transmitting the request to the control device when the button is operated; a second display unit that displays a proposal to the user transmitted from the control device on the display unit; A system having

2. The system of claim 1, wherein the suggestions are suggestions regarding how to improve the user's speaking style.

3. The system described in claim 2, wherein the suggestion is to increase the volume of the user's voice as a way to improve the user's speaking style.

4. The system described in claim 2, wherein the suggestions are suggestions regarding speech speed as a way to improve the user's speaking style.

5. The system of claim 4, wherein the suggestions regarding conversation speed are either text, an icon, a voice message, or an alert sound.

6. The system of claim 1, wherein the suggestions are conversation topic suggestions.

7. The system described in claim 1, wherein the first display means displays the in-call screen on the display unit, the in-call screen including a plurality of buttons each requesting a plurality of suggestions.

8. The system of claim 7, wherein the plurality of buttons includes a button requesting suggestions regarding speaking speed.

9. The system of claim 7, wherein the plurality of buttons includes a button requesting suggested conversation topics.

10. A method performed by a system including a first terminal device and a control device, The control device includes: When a call is started between the first terminal device and a second terminal device, a process of receiving voice data including a user's speech transmitted from the first terminal device; A process of transmitting the received voice data to the second terminal device; A process of receiving a request transmitted from the first terminal device in response to an operation of a button included in a call in progress screen displayed on the first terminal device while the call is being made, the button receiving a request for a proposal to the user; When the request is received, a process of transmitting a proposal to the user based on the voice data to the first terminal device; The first terminal device is A process of displaying the call in progress screen including the button on a display unit; a process of transmitting the request to the control device when the button is operated; A process of displaying a proposal to the user transmitted from the control device on the display unit; How to do it.

11. A control device capable of communicating with a first terminal device, a first receiving means for receiving voice data including a user's speech transmitted from the first terminal device when a call between the first terminal device and a second terminal device is started; a first transmitting means for transmitting the received voice data to the second terminal device; a second receiving means for receiving a request transmitted from the first terminal device in response to an operation of a button included in a call in progress screen displayed on the first terminal device while the call is being made, the button receiving a request for a proposal to the user; and a second transmission means for transmitting a proposal to the first terminal device based on the voice data when the request is received, The first terminal device is a first display means for displaying the call in progress screen including the button on a display unit; a third transmission means for transmitting the request to the control device when the button is operated; a second display unit that displays a proposal to the user transmitted from the control device on the display unit; A control device having the above configuration.

Images