Communication system, terminal device, communication method, and program

Abstract

To provide a communication system that can improve the business efficiency.SOLUTION: A communication system 100 in which a first terminal device 10 operated by an operator and a second terminal device 70 operated by a guest communicate with each other via a network includes a proposal unit 98 that determines a proposal to the operator on the basis of voice data of the operator transmitted from the first terminal device or voice data of the guest transmitted from the second terminal device, and a first communication unit 91 that transmits the proposal to the first terminal device, and the first terminal device includes a second communication unit 11 that receives the proposal, and a display control unit 17 that displays the proposal received by the second communication unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a communication system, a terminal device, a communication method, and a program. [Background technology]

[0002] There is known a communication system in which terminal devices in remote locations transmit and receive images and audio to and from each other, displaying the images on a display and outputting the audio from a speaker, thereby enabling remote users to hold remote communications such as conferences. Each terminal device has a camera, a microphone, a display, and a speaker. Images captured by the camera are displayed on the display of the terminal device of the communication partner, and audio captured by the microphone is output from the speaker of the terminal device of the communication partner. Using such a communication system, for example, an operator at a call center can talk to (e.g., serve) a guest visiting a store while looking at their face, and the guest can also look at the operator's face and receive explanations.

[0003] Furthermore, a technology for assisting conversations is known (see, for example, Patent Document 1). Patent Document 1 discloses a conversation analysis device that analyzes conversations in real time, analyzes the receptivity of the user's conversation partner and the user's conversational skill, and extracts users (influencers) who have a large influence on others on a certain topic. Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional methods for displaying the results of conversation analysis have had the problem of making it difficult to improve the efficiency of various tasks using communication systems. For example, if the results of conversation analysis were simply displayed as numbers, it would take time for operators to understand how to apply them to customer service.

[0005] In view of the above-mentioned problems, an object of the present invention is to provide a communication system that can improve business efficiency. [Means for solving the problem]

[0006] In view of the above-mentioned problems, the present invention provides a communication system in which a first terminal device operated by an operator and a second terminal device operated by a guest communicate with each other via a network, the system including: a proposal unit that determines a proposal to the operator based on voice data of the operator transmitted from the first terminal device or voice data of the guest transmitted from the second terminal device; a first communication unit that transmits the proposal to the first terminal device; The first terminal device is characterized by having a second communication unit that receives the proposal, and a display control unit that displays the proposal received by the second communication unit. [Effects of the Invention]

[0007] It is possible to provide a communication system that can improve business efficiency. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 10 is a diagram illustrating how a communication system provides suggestions for improving customer service efficiency. [Figure 2] FIG. 1 is a schematic diagram illustrating an example of a communication system. [Figure 3] FIG. 2 is a diagram illustrating a hardware configuration of an example of a terminal device. [Figure 4] FIG. 2 is a diagram illustrating a hardware configuration of an example of a management system. [Figure 5] FIG. 2 is a diagram illustrating a software configuration of an example of a terminal device. [Figure 6] FIG. 2 is a functional block diagram of an example of a terminal device and a management system included in the communication system. [Figure 7] 3A and 3B are diagrams illustrating examples of a destination management DB, a presence management DB, and a transmission destination management DB. [Figure 8] FIG. 2 is a functional block diagram of an example illustrating the functions of the proposed control device divided into blocks. [Figure 9] FIG. 2 is a diagram illustrating information held in a storage unit. [Figure 10] FIG. 10 is a sequence diagram illustrating an example of a process in which a terminal device logs in to a management system. [Figure 11] FIG. 10 is a diagram illustrating an example of a reception screen. [Figure 12] FIG. 10 is a sequence diagram illustrating an example of a process for transmitting presence information. [Figure 13] FIG. 10 is a flowchart illustrating an example of a process for managing presence for each feature. [Figure 14] FIG. 10 is a sequence diagram illustrating an example of a process for starting communication. [Figure 15] FIG. 10 is a diagram showing an example of a feature list screen displayed when the presence management table is in the state of Table 2. [Figure 16] FIG. 10 is a sequence diagram illustrating an example of a process for establishing communication between terminal devices. [Figure 17] FIG. 10 is a diagram illustrating an example of a message display. [Figure 18] 10 is an example of a sequence diagram relating to a process in which a proposal control device proposes a method for improving speaking style. [Figure 19] 10 is an example of a sequence diagram illustrating a process in which a proposal control device makes a proposal in response to a request from an operator. [Figure 20] FIG. 10 is a diagram showing an example of a call in progress screen displayed on the operator's terminal device. [Figure 21] FIG. 10 is a diagram illustrating an example of suggestions using icons. [Figure 22] FIG. 10 is a diagram showing an example of suggested conversation topics displayed on a screen during a call. [Figure 23] FIG. 10 is an example of a flowchart illustrating a process performed by the proposed control device to prompt the operator to speak louder. [Figure 24] FIG. 10 is an example of a flowchart illustrating a process performed by the proposed control device to prompt the operator to slow down the speaking speed. [Figure 25] 10 is an example of a flowchart illustrating a process performed when a proposal control device proposes a conversation topic to an operator. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A communication system and a communication method performed by the communication system will be described below as an example of an embodiment of the present invention with reference to the drawings.

[0010] <Overall operation overview> FIG. 1 is a diagram illustrating how a communication system 100 of this embodiment provides suggestions for improving customer service efficiency. In FIG. 1, remote communication is taking place between two devices, a terminal device 10 and a terminal device 70. The remote communication is a meeting held at a remote location. A meeting is when people get together for consultation, discussion, etc. Meetings can take various forms, such as customer service, conferences, gatherings, meetings, study sessions, classes, seminars, and presentations. It does not necessarily have to be two-way communication.

[0011] It is assumed that the terminal device 10 is placed in a call center, and the terminal device 70 is placed in a store. The terminal device 10 is operated by an operator a, and the terminal device 70 is operated by a guest g who visits the store. A web camera 101 and a document camera 102 are connected to the terminal devices 10 and 70. In addition, a touch panel 190 is connected to the terminal device 70, allowing the guest g to write information by hand.

[0012] (1) As shown in FIG. 1(a), during a call (while serving a customer), the terminal device 10 (an example of a first terminal device) displays an image (which may be a moving image or a still image) captured by the web camera 101 or the document camera 102 of the terminal device 70 (an example of a second terminal device) on the display 720. Similarly, the terminal device 70 displays an image (which may be a moving image or a still image) captured by the web camera 101 or the document camera 102 of the terminal device 10 on the display 720. The operator a and the guest g can converse while looking at each other's faces.

[0013] (2) During a call, the management system 50 detects the number of times the guest asks the operator to repeat what he or she says, or at least the operator's speaking speed. Guests ask the operator to repeat what he or she says because it is difficult for them to hear what the operator says. If the operator's voice is too quiet, the guest may have difficulty hearing what the operator says. Also, if the operator's speaking speed is too fast, the guest may have difficulty hearing what the operator says. Therefore, when at least one of the number of times the guest asks to repeat what he or she says or the speaking speed exceeds a threshold, the proposal control device 3 suggests to the operator how to improve their speaking style.

[0014] (3) Figure 1(b) is an example of a suggestion for improvement displayed on the operator's terminal device 10. In Figure 1(b), it is displayed as "Speech speed: You may want to slow down your speech speed a little." Therefore, the operator can continue serving the customer by slowing down the speech speed.

[0015] In this way, the communication system of this embodiment proposes the results of analyzing voice data as specific improvement methods in real time, thereby improving the efficiency of operations such as customer service.

[0016] <Terminology> A proposal refers to the submission of a proposal or a proposal that has been submitted. In this embodiment, any information that leads to improved business efficiency may be used. For example, the average time required for customer service may be shortened. The proposal may be displayed, output as audio to a headset, or sounded as an alarm.

[0017] <System configuration example> Fig. 2 is a schematic configuration diagram of an example of a communication system 100 of this embodiment. As shown in Fig. 2, the communication system 100 of this embodiment includes a plurality of terminal devices 10, 70 installed at each base, one or more relay devices 30, one or more management systems 50, and a proposed control device 3. Here, a base is a location that serves as a base for activities. The installation location of the terminal devices 10, 70 or the location of the user is a base. Three bases (a call center 8, a store 7X, and a store 7Y) are shown in Fig. 2.

[0018] Terminal devices 10a to 10d are arranged in the call center 8, terminal device 70x is arranged in the store 7X, and terminal device 70y is arranged in the store 7Y. The numbers of these are not limited to those shown in the figure. Furthermore, any one of the multiple terminal devices 70x, 70y will be referred to as a "terminal device 70," and any one of the multiple terminal devices 10a to 10d will be referred to as a "terminal device 10." Any one of the multiple stores 7X, 7Y will be referred to as a "store 7."

[0019] The call center 8 is a department that responds to inquiries from guests. In this embodiment, the person who responds to inquiries at the call center 8 is called an operator. They may also be called concierges, explainers, instructors, or advisors. The store 7 is a facility for conducting business. Actual products or service brochures are available for guests to visit. Examples include travel agencies, banks, and hospitals, but any facility that users visit can be used. A guest who visits the store 7 operates a pre-installed terminal device 70 to start a video conference with a terminal device 10 at the call center 8. This makes it possible to minimize the number of staff members stationed at the store 7 and enables highly specialized staff members to hold video conferences with guests from various locations.

[0020] In FIG. 2, the terminal device 10 for the operator is located in the call center 8, but the terminal device 10 for the operator may also be located in the store 7. For example, this is so that when the store 7 is quiet, a staff member of the store 7 can act as an operator. As will be described below, the terminal devices 10 and 70 may be general information processing devices, and can be switched between operator and guest use by application software (application software for video conferencing).

[0021] The terminal devices 10 and 70 may be general information processing devices such as PCs (Personal Computers). They may also be dedicated terminals. Application software for operators is installed in the terminal device 10, and application software for guests is installed in the terminal device 70. Therefore, the attribute information of each of the terminal devices 10 and 70 is determined by the application software. In this way, the attribute information may be determined by the application software, or the attribute information may be determined by a communication ID, which will be described later. In this case, each of the terminal devices 10 and 70 can use a table in which attribute information is associated with a communication ID.

[0022] The terminal devices 10 and 70 are each general information processing devices, and therefore can run software such as presentation software and browser software in addition to videoconferencing application software. The terminal devices 10 and 70 may be PCs, tablet devices, smartphones, PDAs (Personal Digital Assistants), etc. However, they may be any devices capable of running application software. For example, they may be car navigation devices, television sets, digital signage terminals, electronic whiteboards, projectors, etc. Furthermore, the terminal devices 10 and 70 do not have to be the same type of information processing device.

[0023] Dedicated terminal devices dedicated to video conferencing may be used as the terminal devices 10 and 70. One form of such a terminal device is a dedicated terminal device that logs in to the management system 50 and transmits and receives video and audio simply by turning on the power.

[0024] The communication system 100 of this embodiment will be described as being applied to a video conference. However, the communication system 100 can also be applied to a communication system or a data communication system. The communication system 100 also includes a data providing system that transmits image data or audio data unidirectionally from one terminal device 10 to multiple terminal devices 10 via a server. A video conference may also be called a video conference. In addition to a video conference, it may also be called a remote conference, remote consultation, remote consulting, etc. Three or more terminal devices 70 can communicate simultaneously.

[0025] The terminal device 10, the management system 50, and the relay device 30 are connected to a communication network 1 or 2. The communication network 1 may be a general network such as a LAN (Local Area Network), WiFi (Wireless Fidelity), or the Internet. For example, the communication network 1 may include a dedicated line such as Wide Area Ethernet (registered trademark) or VPN (Virtual Private Network). In the case of wireless communication, the terminal device connects to the communication network 2 (mobile communication network) using a base station 2a as an access point. The communication network 2 includes locations where wireless communication such as 3G, 4G, LTE (Long Term Evolution), 5G, and Bluetooth (registered trademark) is performed. The communication networks 1 and 2 are connected by a gateway.

[0026] In the communication system 100 of this embodiment, a plurality of terminal devices 10, 70 transmit and receive image data (including still images and moving images, which may also be referred to as video data), audio data, and the like via a relay device 30. At this time, a management information session for transmitting and receiving various types of management information is established between the plurality of terminal devices 10, 70 via a management system 50. Furthermore, a data session for transmitting and receiving image data and audio data is established between the plurality of terminal devices 10, 70 via the relay device 30. Here, in particular, the image data transmitted and received in the data session is scalably encoded data, and for example, encoded data of high-quality video, encoded data of medium-quality video, and encoded data of low-quality video are transmitted and received on separate channels.

[0027] As a function of the video conference, the terminal devices 10, 70 have the function of transmitting image data captured by a camera (image capture device), audio data collected by a microphone, and material data used for explanations, etc., to the terminal devices 10, 70 at other locations participating in the video conference via the relay device 30. They can also transmit handwritten information to the terminal devices 10, 70 at other locations. They also have the function of displaying image data, material data, and handwritten information received from the terminal devices 10, 70 at other locations on a display, and outputting received audio data from a speaker. Image data, material data, handwritten information, and audio data may be collectively or individually referred to as content data.

[0028] When a video conference is started, the terminal devices 10 and 70 transmit image data, audio data, document data, and handwritten information to the relay device 30, and the relay device 30 transmits image data, audio data, document data, and handwritten information to the other terminal devices participating in the video conference. In addition, the terminal devices 10 and 70 receive image data, audio data, document data, and handwritten information from the other terminal devices 10 via the relay device 30.

[0029] The management system 50 manages the communication IDs of the terminal devices 10 and 70, and functions as a call control server that performs call control such as initiating a session between two or more terminal devices when an incoming call is received. As the management system 50 is a server, it also has the functions of an information processing device. Call control refers to a series of processes for making and disconnecting calls, such as making a call to start communication, receiving the call, answering the call, and disconnecting the communication. In addition, the management system 50 often performs authentication, searching, and alive monitoring of the terminal devices 10 and 70. Furthermore, the management system 50 also manages databases necessary for controlling the communication system 100, such as authentication information for the terminal devices 10 and 70.

[0030] The relay device 30 is a server (information processing device) that relays image data, audio data, document data, and handwritten data from a terminal device 10, 70 at one base to a terminal device 10, 70 at another base. The management system 50 monitors the bandwidth of the communication network 2 and the communication load of the relay device 30, and assigns an appropriate relay device 30 to the terminal device 10, 70.

[0031] The management terminal 40 is an information processing device used by a store manager or the like. Browser software runs on the management terminal 40. The management terminal 40 may be a PC, tablet terminal, smartphone, PDA, or the like, as long as it runs browser software.

[0032] The proposal control device 3 analyzes the voice data of the operator and each guest. The terminal devices 10 and 70 transmit the voice data to the relay device 30, which then forwards the voice data to the proposal control device 3. Alternatively, the relay device 30 may also function as the proposal control device 3, or the management system 50 may also function as the proposal control device 3. The proposal control device 3 recognizes the voice data of the operator and guest and converts it into text data. This makes it possible to detect whether the guest has asked for clarification and the operator's speaking speed. The proposal control device 3 can also analyze words contained in the text data to suggest conversation topics.

[0033] <Hardware configuration example> <<Terminal Device>> Fig. 3 is a hardware configuration diagram of an example of the terminal device 10, 70. As shown in Fig. 3, the terminal device 10, 70 includes a CPU 701 that controls the overall operation of the terminal device, a ROM 702 (Read Only Memory) that stores programs, a RAM 703 that is used as a work area for the CPU 701, a HDD 704 (Hard Disk Drive) that reads or writes data under the control of the CPU 701, a media I / F 707 that controls reading or writing (storing) data from or to a recording medium 706 such as a flash memory, and cameras 712a, 712b that capture an image of a subject and obtain image data under the control of the CPU 701.

[0034] The HDD 704 stores an operating system (OS) executed by the CPU 701, other programs, and various data. The cameras 712a and 712b are charge-coupled devices that convert light into electric charges to digitize images of a subject. The cameras 712a and 712b may be complementary metal oxide semiconductor (CMOS) sensors or charge coupled device (CCD) sensors. There may be three or more cameras 712a and 712b. Any of the cameras 712a and 712b will be referred to as "camera 712."

[0035] Furthermore, the terminal device 10, 70 includes a microphone 714 that converts sound into sound signals, a speaker 715 that converts sound signals into sound, an antenna 711a, a communication unit 71 that uses this antenna 711a to communicate with the nearest base station 2a via wireless communication signals, a display 720 such as a liquid crystal or organic EL display that displays an image of a subject, various icons, etc., a touch panel 721 that is placed on this display 720 and is composed of a pressure-sensitive or electrostatic panel and detects a touch position on the display 720 by touch with a finger, a touch pen, etc., and a bus line 710 such as an address bus or a data bus for electrically connecting the above-mentioned components.

[0036] Also, in FIG. 3, there is one microphone 714 and one speaker 715, but there may be two or more microphones 714 and speakers 715.

[0037] <<Management System>> 4 is a hardware configuration diagram of an example of a management system 50. The management system 50 includes a CPU 201 that controls the overall operation of the management system 50, a ROM 202 that stores programs used to drive the CPU 201, such as an IPL (Initial Program Reader), a RAM 203 that is used as a work area for the CPU 201, a HD 204 that stores various data such as programs for the management system 50, a HDD 205 (Hard Disk Drive) that controls reading or writing of various data from or to the HD 204 under the control of the CPU 201, a media I / F 207 that controls reading or writing (storing) of data from or to a recording medium 206 such as a flash memory, a display 208 that displays various information such as a cursor, menus, windows, characters, or images, a network I / F 209 for data communication using the communication network 2, a keyboard 211 with multiple keys for inputting characters, numbers, various instructions, etc., a mouse 212 that selects and executes various instructions, selects a processing target, moves the cursor, etc., and a CD-ROM 213 (Compact Disc Read Only Memory) as an example of a removable recording medium. The CD-ROM drive 214 controls the reading and writing of various data from and to the CD-ROM memory, and bus lines 210 such as an address bus and a data bus for electrically connecting the above components as shown in FIG.

[0038] The relay device 30 or the management terminal 40 has the same hardware configuration as the management system 50, and therefore a description thereof will be omitted.

[0039] <Application software> FIG. 5(a) is a software configuration diagram of a terminal device 10 according to an embodiment. An operator application software A1 is installed in the terminal device 10. As shown in FIG. 5(a), an OS 1020 and the operator application software A1 run on a working area 1010 of the RAM 703 of the terminal device 10. Of these, the OS 1020 is basic software that provides basic functions and manages the entire terminal device 10. The operator application software A1 is an application for communicating with other terminal devices 10, 70.

[0040] FIG. 5(b) is a software configuration diagram of a terminal device 70 according to one embodiment. Guest application software A7 is installed in the terminal device 70. As shown in FIG. 5(b), the OS 1020 and the guest application software A7 run on a working area 1010 of the RAM 703. Of these, the OS 1020 is basic software that provides basic functions and manages the entire terminal device 70. The guest application software A7 is an app for communicating with other terminal devices 10, 70.

[0041] However, the guest application software A7 has limited functionality compared to the operator application software A1. This is due to the design policy that, in principle, operations are performed by the operator, and guests are only required to perform minimal operations. As an example, the operations that guests can perform are as follows: 1. Press the operator call button 2. Adjusting the volume of the Speaker 715 3. Switching between displaying and hiding your own location's image Examples of communication protocols for application software include (1) SIP (Session Initiation Protocol), (2) H.323, (3) an extended SIP protocol, (4) an instant messenger protocol, (5) a protocol using the SIP MESSAGE method, (6) an Internet Relay Chat protocol (IRC (Internet Relay chat)), and (7) an extended instant messenger protocol. Of these, (4) an instant messenger protocol is, for example, (4-1) XMPP (Extensible Messaging and Presence Protocol) or (4-2) a protocol used in ICQ (registered trademark), AIM (registered trademark), or Skype (registered trademark). Furthermore, (7) an extended instant messenger protocol is, for example, Jingle.

[0042] <About the function> 6 is a functional block diagram of an example of the terminal devices 10, 70 and the management system 50 included in the communication system 100 of this embodiment. Note that the function of the relay device 30 is omitted in FIG. 6 because it only relays image data.

[0043] <<Functional configuration of terminal device>> Below, we will explain the functions of the terminal device 10 used by the operator of the call center 8 and the terminal device 70 used by the guest who visits the store 7. The difference in these functions is the difference in the application software. However, the terminal device of the call center 8 and the terminal device of the store 7 may run the same application software, and the functions may be different if operations that can only be performed by the operator are prepared after authentication.

[0044] Call center terminal equipment 10 The terminal device 10 of the call center 8 has a communication unit 11, an operation input reception unit 12, a startup unit 13, a display control unit 17, an imaging unit 18, an audio output unit 19, an audio acquisition unit 20, and a storage / readout processing unit 29. Each of these units is a function or means realized when any of the components shown in Fig. 3 operates in response to an instruction from the CPU 701 in accordance with the operator application software A1 loaded from the HDD 704 onto the RAM 703.

[0045] The terminal device 10 also has a storage unit 1000 constructed by the RAM 703 and HDD 704 shown in Fig. 3. The storage unit 1000 stores application software A1 for operators downloaded from the management system 50 or the like, and also temporarily stores image data, audio data, document data, and handwritten information. Attribute information (operator) set by the application software A1 for operators is also stored.

[0046] The storage unit 1000 also stores a communication ID (Identification) for identifying the terminal device 10, a password, and the like. ID is an abbreviation for Identification and means an identifier or identification information. An ID refers to a name, code, character string, numeric value, or a combination of one or more of these that is used to uniquely distinguish a specific object from multiple objects. The same applies to other IDs.

[0047] The communication unit 11 (an example of a second communication unit) transmits and receives various data (or information) to and from the management system 50 and the relay device 30 (which communicates with other terminal devices 10 via these) via the communication network 2. Before starting a call with a desired destination terminal, the communication unit 11 starts receiving, from the management system 50, each piece of status information indicating the status of each terminal as a candidate for a communication partner.

[0048] The operation input receiving unit 12 receives various inputs to the terminal device 10 by an operator of the call center 8. For example, when a user turns on the power switch, the operation input receiving unit 12 receives the power-on command and notifies the activation unit 13.

[0049] The startup unit 13 starts the terminal device 10 by turning on the power. For example, it executes IPL to read the operator application software A1 from the storage unit 1000 and transfers it to the RAM 703 in Fig. 3. Furthermore, the startup unit 13 causes the CPU 701 to execute the operator application software A1, thereby performing processing such as logging the terminal device 10 into the management system 50.

[0050] The display control unit 17 performs control to send the image data, document data, and handwritten information transmitted from the relay device 30 to the display 720. The display control unit 17 also causes the display 720 to display various screens.

[0051] The imaging unit 18 converts the imaging data obtained by imaging a subject with the cameras 712a and 712b into predetermined image data and outputs the image data. The imaging unit 18 can output images by switching between the cameras 712a and 712b.

[0052] The audio output unit 19 reproduces and outputs the audio data transmitted from the relay device 30 from the speaker 715 .

[0053] After the operator's voice is converted into a voice signal by the microphone 714, the voice acquisition unit 20 converts this voice signal into predetermined voice data and outputs it.

[0054] The storage / readout processing unit 29 stores various data in the storage unit 1000 and reads out various data stored in the storage unit 1000 .

[0055] Store terminal device 70 The terminal device 70 of the store 7 has a communication unit 71, an operation input reception unit 72, a startup unit 73, a display control unit 77, an imaging unit 78, an audio output unit 79, an audio acquisition unit 80, a status management unit 81, a transmission control unit 82, and a storage / readout processing unit 89. In the explanation of these components, differences from the terminal device 10 for a call center will be mainly explained.

[0056] The status management unit 81 is implemented by a command from the CPU 701, and manages the presence of destination candidates (operators) based on the presence information sent from the terminal device 10.

[0057] The transmission control unit 82 is implemented by instructions from the CPU 701, and controls the transmission of a call start request to a destination candidate based on the status of the destination candidate.

[0058] In the storage unit 7000 of the terminal device 70 of the store 7, in addition to attribute information (guest) 7004, a destination management DB 7001, a presence management DB 7002, and a transmission destination management DB 7003 are constructed.

[0059] 7(a) shows an example of a destination management table stored in the destination management DB 7001. In the destination management table, the communication ID of the terminal device 10 of the operator who is a destination candidate, presence indicating the status of the destination candidate, and characteristic information indicating the characteristics of this destination candidate are associated and managed. This characteristic information is an abbreviation of the item. The communication ID is information for identifying a communication destination in the communication system 100. The communication ID is not particularly limited, but includes, for example, identification information of the terminal device 10, 70, identification information of the user of the terminal device 10, 70, an email address, etc. Details of the presence will be described later.

[0060] 7(b) is a conceptual diagram showing a presence management table stored in the presence management DB 7002 (an example of a state management information storage unit). The presence management table associates and manages characteristic information indicating the characteristics of the destination candidate, a presence integrating each presence of each destination candidate of that characteristic, the communication ID of a terminal device 10 with the presence "online" available for a call among the destination candidates of that characteristic, and the communication ID of a terminal device 10 with the presence "chat" active among the destination candidates of that characteristic. The process of integrating each presence of each destination candidate to generate a presence of the characteristic will be described later.

[0061] 7(c) is a conceptual diagram showing a destination management table stored in the destination management DB 7003. The terminal device 70 is used by a guest, and the terminal device 10 is used by an operator. The terminal device 70 can send call start requests to multiple terminal devices 10 at the same time. The destination management table manages the communication IDs of the terminal devices 10 to which the terminal device 70 is currently sending a call start request. The number of communication IDs managed in the destination management table changes depending on the process of sending a call start request and the process of canceling a call start request by the terminal device 70.

[0062] <Management system functional configuration> Returning to Fig. 6, the management system 50 has a communication unit 51, an authentication unit 52, a management unit 53, a session control unit 54, a screen information generation unit 57, and a storage / readout processing unit 59. Each of these units has a function realized by operating any of the components shown in Fig. 4 in response to an instruction from the CPU 201 in accordance with a program for the management system 50 that is loaded from the HD 204 onto the RAM 203. The management system 50 also has a storage unit 5000 constructed by the HD 204. Furthermore, in the storage unit 5000, each DB is constructed using each table as shown below.

[0063] 7(d) is a conceptual diagram showing an authentication management table stored in the authentication management DB 5001. In the authentication management table, an authentication password is associated with each communication ID of all the terminal devices 10 and 70 managed by the management system 50 and managed.

[0064] 7(e) is a conceptual diagram showing a terminal management table stored in the terminal management DB 5002. In the terminal management table, the IP addresses of the terminal devices 10 and 70 are associated with and managed for each communication ID of the terminal device 10 and 70.

[0065] <Management system functional configuration> Next, a detailed description will be given of each functional configuration of the management system 50. The communication unit 51 is realized by commands from the CPU 201 and the network I / F 209, and transmits and receives various data (or information) to and from each terminal device or system via the communication network 2.

[0066] The authentication unit 52 is realized by an instruction from the CPU 201, and searches the authentication management table using the communication ID and password received by the communication unit 51 as search keys, and authenticates the terminal by determining whether the same communication ID and password are managed in this authentication management table.

[0067] The management unit 53 is implemented by an instruction from the CPU 201, and manages the terminal devices 10 and 70 by recording information about the terminal devices 10 and 70 in the terminal management table shown in FIG. 7(e).

[0068] The session control unit 54 controls a session for transmitting content data between the terminal devices 10 and 70 in response to instructions from the CPU 201. This control includes control for establishing a session, control for allowing the terminal devices 10 and 70 to participate in the established session, control for leaving the session, and the like.

[0069] The screen information generation unit 57 generates screen information for the screen displayed by the management terminal 40 as a web server. The screen information is created using HTML, XML, CSS (Cascade Style Sheet), JavaScript (registered trademark), etc. In this embodiment, screen information for displaying a call in progress screen is generated. Such a screen may be called a web page, and the web page may be provided by a web application. A web application is software or its mechanism that runs on a browser and operates through cooperation between a program written in a programming language (for example, JavaScript (registered trademark)) that runs on a browser and a program on the web server. Web pages can be dynamically changed by the web application.

[0070] The storage / readout processing unit 59 is realized by instructions from the CPU 201 and the HDD 205, or by instructions from the CPU 201, and performs processing such as storing various data in the memory unit 5000 and extracting various data stored in the memory unit 5000.

[0071] <Administrative terminal> The management terminal 40 has a communication unit 41, an operation reception unit 42, and a display control unit 43. Each of these functional units of the management terminal 40 is realized by one of the components shown in FIG. 4 operating in response to an instruction from the CPU 201 in accordance with a program for the management terminal 40 that is loaded from the HD 204 onto the RAM 203. This program is assumed to be browser software. The management terminal 40 is used by the store manager to obtain and display information about each operator from the proposal control device 3.

[0072] The communication unit 41 included in the management terminal 40 transmits and receives various information to and from the management system 50, etc. For example, the communication unit 41 requests a web page of a portal site via HTTP communication and transmits various operation contents for the web page to the management system 50.

[0073] The operation reception unit 42 receives various operations for the web page (screen) displayed by the management terminal 40. The display control unit 43 analyzes various screen information from the management system 50 and displays it on the display 208.

[0074] <Proposed control device> FIG. 8 is a functional block diagram of an example that explains the functions of the proposed control device 3 by dividing them into blocks. The proposed control device 3 has a communication unit 91, a voice recognition unit 92, a volume recording unit 93, a tone determination unit 94, a reflection detection unit 95, a speaking speed detection unit 96, a video analysis unit 97, and a proposal unit 98. Each of these units is realized by the operation of any of the components shown in FIG. 4 in response to an instruction from the CPU 201 in accordance with a program for the proposed control device 3 that is loaded from the HD 204 onto the RAM 203. The proposed control device 3 also has a memory unit 99 constructed by the HD 204. Furthermore, each memory unit in the memory unit 99 is constructed using the tables shown below.

[0075] FIG. 9(a) is a conceptual diagram showing a utterance management table stored in the utterance storage unit 3001. In the utterance management table, text data uttered by the operator and guest is recorded for each word. However, for the operator, it is sufficient to convert the text data into text data, and it does not need to be divided into words. The text data is divided into parts of speech using morphological analysis. In addition, the volume, tone, and facial expression of the voice when the operator or guest uttered the word are registered. The volume, tone, or facial expression of the voice do not have to be registered for each word, but may be registered for a group of utterances, such as a sentence, a certain period of time, or until a pause in the conversation occurs. Tone is a tone of voice and represents the pitch and brightness of the voice.

[0076] 9(a) is generated for each store, but if there are multiple guests, the proposal control device 3 may identify the multiple guests using voice recognition and generate a comment management table for each guest. The same applies to the following tables.

[0077] 9(b) is a conceptual diagram showing the re-listening management table stored in the re-listening count storage unit 3002. The re-listening management table registers the number of times a guest has asked for re-listening for each conference. The number of times a guest has asked for re-listening is initialized at regular intervals.

[0078] FIG. 9(c) is a conceptual diagram showing a conversation speed management table stored in the conversation speed storage unit 3003. In the conversation speed management table, the conversation speed [number of characters / 10 seconds] of the operator and guest is registered for each conference. The guest's conversation speed is not required. The unit of conversation speed [number of characters / 10 seconds] is an example, and it may be the number of words per fixed time. It may also be the number of characters per minute, or it may not be per 10 seconds. The conversation speed is updated at regular intervals.

[0079] FIG. 9(d) is a conceptual diagram showing the proposed information management table 1 stored in the proposed information storage unit 1 (reference numeral 3004). In the proposed information management table 1, suggestions and display methods are registered in association with the number of times the guest has asked to repeat themselves. For example, if the number of times the guest has asked to repeat themselves is 20, the suggestion is "Please speak a little louder," and the display method for the suggestion is "in red." The suggestion is not limited to text, and may be an icon, a voice message, an alarm sound, or the like.

[0080] FIG. 9(e) is a conceptual diagram showing the proposed information management table 2 stored in the proposed information storage unit 2 (reference numeral 3005). In the proposed information management table 2, a suggestion and a display method are registered in association with the speed difference. For example, when the speed difference is "-10", the suggestion is "Please speak a little more slowly" and the display method is "Red text". The speed difference is the difference between the operator's speaking speed and a predetermined reference value. If the operator's speaking speed is faster than the reference value, the value is a negative value, and if the opposite is true, the value is a positive value. The speed difference may also be the difference in speaking speed between the guest and the operator.

[0081] FIG. 9(f) is a conceptual diagram showing a time information management table stored in the time information storage unit 3006. In the time information management table, destinations and average customer service times are registered. In the case of a travel agency, the customer service time varies depending on the guest's destination, so the average customer service time is registered for each destination. The destination may also be a more detailed area name. The average customer service time is the average time required for each guest to be served. The time required for customer service is the time from the start to the end of a video conference (call). Based on the average customer service time, the proposal control device 3 can propose a guideline for customer service times.

[0082] <<Functional configuration of the proposed control device>> Next, a detailed description will be given of each functional configuration of the proposal control device 3. The communication unit 91 (an example of a first communication unit) receives audio data and video data from the relay device 30 via the communication network 2. The communication unit 91 also transmits suggestions regarding methods for improving speaking style and conversation topics to the terminal device 10 of the operator.

[0083] The speech recognition unit 92 recognizes the speech data received by the communication unit 91 and converts it into text data. Any method of speech recognition may be used. For example, there is a method of maximum likelihood estimation of statistics of a generative model, such as an HMM (hidden Markov model) or an N-gram model, based on training data. Alternatively, there is a known method of applying a long short-term memory (LSTM) to a recurrent neural network (RNN), which is a form of deep learning.

[0084] The speech recognition unit 92 also performs morphological analysis to convert the text data into words with parts of speech. This allows words and parts of speech to be extracted. Homonyms can also be converted into appropriate words (for example, "umi" can be converted into "umi" for detection). The speech recognition unit 92 registers the extracted words in the utterance storage unit 3001. While it is preferable to record in the utterance storage unit 3001 for both the operator and the guest, in this embodiment, it is sufficient to perform this for at least the guest. For the operator, conversion into text data is sufficient to detect the speaking speed.

[0085] The volume recording unit 93 records at least the volume (voice volume) of the guest's speech in association with each word. The volume of speech may be the amplitude of the audio data (for example, the bit value at the time of PCM (pulse code modulation) conversion). Preferably, it is determined for each word, but it may also be registered for a group of speech, such as a sentence, a certain period of time, or until the conversation stops.

[0086] The tone determination unit 94 determines the tone of the voice. It converts the audio data into a frequency-intensity relationship using a Fourier transform or similar method, and determines the range of frequencies that can be taken for each guest. For example, it can be the range of frequencies taken over the first few minutes. Then, it divides the range into, for example, two equal regions, and determines which of the two regions the frequency with the highest intensity belongs to. The tone determination unit 94 records at least the guest's tone in association with each word. Preferably, it is sufficient to make a determination for each word, but it is also sufficient to register the tone for a chunk of speech, such as a sentence, a certain period of time, or until the conversation stops.

[0087] The repetition detection unit 95 detects repetition by comparing the text data detected by the speech recognition unit 92 with character strings stored in advance. For example, character strings such as "Once again," "Huh?", "I can't hear you," and "What did you say" are stored in advance, and when text data of a guest that matches these is detected, it detects that a repetition has occurred. Every time the repetition detection unit 95 detects a repetition, it increments the number of repetitions stored in the repetition count storage unit 3002 by one.

[0088] The speaking speed detection unit 96 detects at least the speaking speed of the operator. The number of characters (syllables) in the text data recognized by the speech recognition unit 92 is counted at regular intervals to determine the speaking speed. The speaking speed of the guest may also be recorded. This regular interval may be the same as the proposed period, but does not have to be the same. Operator conversation speed SO = number of characters / fixed time Guest speaking speed SG=number of characters / certain time The conversation speed detection unit 96 periodically registers the conversation speed between the operator and the guest in the conversation speed storage unit 3003. The average of the past conversation speeds may also be registered.

[0089] The video analysis unit 97 analyzes the video data of the operator and guest. For example, it determines whether they are smiling. For the operator, whether they are smiling or not is registered, for example, periodically, and for the guest, facial expressions (smiling, neutral) are registered in the utterance storage unit 3001. Smile detection may be performed using, for example, deep learning. Image data that has been given in advance as a teacher signal indicating whether or not a smile is present is trained by the information processing device using a neural network such as a CNN (Convolutional Neural Network). Through this learning, the video analysis unit 97 can output whether or not the input video data is smiling.

[0090] The suggestion unit 98 transmits a suggestion to the operator's terminal device 10 at regular intervals or in response to a request from the operator, based on the number of times the speaker has asked to repeat the speech or the speaking speed. For example, in the case where the suggestion unit 98 determines based on the number of times the speaker has asked to repeat the speech, the suggestion unit 98 determines to send a suggestion when the number of times the speaker has asked to repeat the speech exceeds the number stored in the suggestion information storage unit 1. In the case where the suggestion unit 98 determines based on the speaking speed, the suggestion unit 98 calculates a "reference value - SO" and determines to send a suggestion when this value exceeds the speed difference stored in the suggestion information storage unit 2.

[0091] The proposing unit 98 can also represent the list of words stored in the utterance storage unit 3001 as visual information such as a graph and transmit it to the terminal device 10 of the operator.

[0092] <Communication system processing and operation> Next, a process in which the terminal devices 10 and 70 log in to the management system 50 will be described with reference to Fig. 10. Fig. 10 is a sequence diagram showing an example of a process in which the terminal devices 10 and 70 log in to the management system 50.

[0093] When a staff member or the like at the store 7 turns on the power switch of the terminal device 70, the operation input receiving unit 72 receives the power-on command and starts up the terminal device 70 (step S1). When the terminal device 70 starts up, the starting unit 73 starts up an application for guests that has been installed in the terminal device 70 (step S2). Thereafter, processing in the terminal device 70 is executed according to commands from the application for guests.

[0094] The communication unit 71 of the terminal device 70 transmits a login request to the management system 50 via the communication network 2 (step S3). This login request includes a communication ID for identifying the terminal that is the source of the login request, and a password. In this embodiment, the communication ID of the terminal device 70 is represented by the initial letter "C." Hereinafter, the communication IDs of the terminal device 70X and the terminal device 70Y are assumed to be "C01" and "C02."

[0095] The communication unit 51 of the management system 50 receives the login request. When the login request is sent from the terminal device 70 to the management system 50, the management system 50, which is the receiving side, can obtain the IP address of the terminal device 70, which is the sending side.

[0096] Next, the authentication unit 52 of the management system 50 searches the authentication management table shown in Figure 7(d) in the memory unit 5000 using the communication ID and password included in the login request as search keys, and performs authentication by determining whether the same communication ID and password are managed in this authentication management table (step S4).

[0097] If the authentication unit 52 authenticates that the login request is from a terminal device 70 with valid usage rights, the management unit 53 records the communication ID and IP address of the terminal device 70 that made the login request in association with each other in the terminal management table shown in Figure 7(e) (step S5).

[0098] The communication unit 51 of the management system 50 transmits authentication result information indicating the authentication result by the authentication unit 52 to the terminal device 70 that issued the login request via the communication network 2 (step S6). As a result, the communication unit 71 of the terminal device 70 receives the authentication result information. Below, a case will be described in which the authentication of the terminal devices 70X and 70Y that issued the login request is successful and the terminal devices 70X and 70Y log in to the management system 50.

[0099] On the other hand, when the operator of the terminal device 10 on the operator side turns on the power switch, the operation input receiving unit 12 receives the power-on command and starts up the terminal device 10 (step S11). When the terminal device 10 starts up, the starting unit 13 starts up the application software A1 for the operator that is installed in the terminal device 10 (step S12). Thereafter, processing in the terminal device 10 is executed according to commands from the application software A1 for the operator.

[0100] The display control unit 17 of the terminal device 10 outputs to the display 720 a reception screen for receiving input of characteristics of an operator who is a user of the terminal device 10.

[0101] 11 is a diagram showing an example of a reception screen. In addition to fields 141 and 142 for receiving input of a communication ID and a password, reception screen 140 includes options 143 for describing countries or regions that can be handled as travel destinations as characteristics of the operator, and a selection check box 144. When the operator selects a country or region that can be handled and presses OK button 145 on reception screen 140, operation input reception unit 12 receives input of the operator's characteristics (step S13-1).

[0102] The storage / readout processing unit 29 stores the feature information corresponding to the selected feature in the storage unit 1000. Hereinafter, the feature information corresponding to the features "Hokkaido, Tohoku, Kanto, Chubu, Kinki, Sanin-Sanyo, Shikoku, Kyushu" is assumed to be "hd, th, kt, cb, kk, ss, sk, ks," respectively. Therefore, in the checked state of FIG. 11, "hd, th, ks" are the feature information of this operator.

[0103] The terminal device 10 sends a login request to the management system 50 and logs in to the management system 50 (steps S13-2, S14, S15, and S16). This process is similar to the processes in steps S3, S4, S5, and S6 between the terminal device 70 and the management system 50, so a description thereof will be omitted. However, the communication ID sent from the terminal device 10 to the management system 50 in the login request is represented by the initial letter "O" so that it can be identified as the terminal device 10 on the operator's side. Hereinafter, it is assumed that the communication IDs of the terminal devices 10a, 10b, and 10c are "O01, O02, and O03." When the authentication of the terminal devices 10a, 10b, and 10c that have made the login request is successful, the terminal devices 10a, 10b, and 10c log in to the management system 50.

[0104] Next, a process of transmitting presence information indicating the status of the operator from the terminal device 10 to the terminal device 70 will be described with reference to Fig. 12. Fig. 12 is a sequence diagram showing an example of a process of transmitting presence information.

[0105] The operator application software A1 manages, in the storage unit 1000, events that trigger state transitions and presences after transitions due to the occurrence of events. Presences are used to determine whether the operator, who is the user of the terminal device 10, is in a state where he or she can initiate a call. Presences indicating a state where he or she can initiate a call include "online," which indicates that the terminal device 10 is logged in to the management system 50 but is not communicating with the guest terminal device 70. Presences indicating a state where he or she cannot initiate a call include "offline," which indicates that the terminal device 10 is not logged in to the management system 50, and "chat," which indicates that the terminal device 10 is logged in to the management system 50 but is already communicating with the customer terminal device 70 and therefore cannot initiate a new call. Note that presences are not limited to those described above. For example, presences based on user input may also be included. Among presences based on user input, an example of a presence indicating a state where he or she cannot initiate a call is "away from desk."

[0106] Examples of events that trigger a transition to the "online" presence include the terminal device 10 receiving the authentication result information in step S16 above, or receiving an input from the user requesting to end a call. An example of an event that triggers a transition to the "offline" presence is the terminal device 10 logging out from the management system 50. An example of an event that triggers a transition to the "chat" presence is the establishment of communication between the terminal devices 70 and 10 (step S105 described below). Note that the event that triggers a transition of the presence is not particularly limited as long as it is detectable, and can be set arbitrarily depending on the communication protocol and the processing content of the application software.

[0107] When any of the above events occurs in the terminal device 10a (step S21), the storage / readout processing unit 29 reads out characteristic information from the storage unit 1000. The read-out characteristic information corresponds to a characteristic, such as "Hokkaido," entered by the operator, who is the user of the terminal device 10a, at the time of login on the reception screen 140 of FIG. 11. The communication unit 11 of the terminal device 10a transmits to the management system 50 (step S22) presence information including a presence indicating a state after the transition due to the occurrence of the event in step S21, the characteristic information read out from the storage unit 1000, and the communication ID "O01" of the terminal device 10a that is the sender. For example, when the terminal device 10a receives authentication result information in step S16, it transmits presence information including the presence "online" to the management system 50. Furthermore, when a logout request is accepted, the terminal device 10a transmits presence information including the presence "offline" to the management system 50. Furthermore, in response to the establishment of communication with the terminal device 70, the terminal device 10a transmits presence information including the presence "chat" to the management system 50. Note that if the characteristic information is not updated after being recorded in the storage unit 1000 in step S13-1, the terminal device 10a transmits presence information including the same characteristic information to the management system 50 every time an event occurs.

[0108] When the communication unit 51 of the management system 50 receives the presence information transmitted by the terminal device 10a, it transmits the received presence information to the terminal devices 70x and 70y of the store 7 that are logged in to the management system 50 (steps S23 and S24). There are no particular limitations on the method for identifying the terminal device 70 on the store side that is logged in to the management system 50, but one example is a method of reading out a communication ID whose initial letter is "C" from the terminal management table of Fig. 7(e).

[0109] The communication unit 71 of the terminal devices 70x and 70y receives presence information transmitted from the terminal device 10a of the call center 8 via the management system 50. The storage and readout processing unit 89 of the terminal devices 70x and 70y associates the communication ID, presence, and feature information contained in the received presence information and records them in the destination management table of FIG. 7(a) (steps S25 and S26). If a record with the same communication ID as the communication ID contained in the received presence information exists in the destination management table, the storage and readout processing unit 89 overwrites and updates the presence recorded in this record based on the newly received presence information. This allows the terminal devices 70x and 70y to manage the latest presence on the terminal device 10a side.

[0110] When the above event occurs in the terminal device 10b on the operator side, the same processes as steps S21, S22, S23, S24, S25, and S26 between the terminal device 10a, the management system 50, and the terminal devices 70x and 70y are executed between the terminal device 10b, the management system 50, and the terminal devices 70x and 70y (steps S31, S32, S33, S34, S35, and S36).When the above event occurs in the terminal device 10c on the operator side, the same processes as steps S21, S22, S23, S24, S25, and S26 between the terminal device 10a, the management system 50, and the terminal devices 70x and 70y are executed between the terminal device 10c, the management system 50, and the terminal devices 70x and 70y (steps S41, S42, S43, S44, S45, and S46). This allows the terminal devices 70x and 70y to manage the latest presence information on the terminal devices 10b and 10c side.

[0111] Next, a process for managing presence for each feature will be described with reference to FIG. 13. FIG. 13 is a flow diagram showing an example of a process for managing presence for each feature. Hereinafter, assuming that a guest requests a call from the operator, the operator will be referred to as a destination candidate. If the destination candidate operator selects multiple features (e.g., "Hokkaido, Tohoku, Kyushu") in step S13-1, the destination candidate will belong to a group of multiple features. The presence "online" of a feature indicates that, among the destination candidates for this feature, one of the destination candidates has the presence "online" and is available for a call. The presence "chat" of a feature indicates that none of the destination candidates for this feature group has the presence "online," and that one or more destination candidates have the presence "chat" and are currently in a call.

[0112] The presence of a feature "offline" indicates that all of the destination candidates for this feature have the presence "offline." Below, the processing of the terminal device 70x will be described, but the terminal device 70y can also execute the same processing.

[0113] In steps S26, S36, and S46, each time the presence in the destination management table shown in FIG. 7(a) is updated, the status management unit 81 of the terminal device 70x acquires the communication ID, presence, and characteristic information recorded in the updated record (step S51).

[0114] 7(b), the state management unit 81 of the terminal device 70x updates each record including the characteristic information acquired in step S51. That is, when the characteristic information "hd,th,ks" is acquired in step S51, the state management unit 81 updates each record including the characteristic information "hd,th,ks" in the presence management table.

[0115] As an example, the following describes the processing when the presence information "online, (hd, th, ks), O01" is sent from terminal device 10a to terminal device 70x and this information is acquired in step S51 (online in step S52).

[0116] First, a process of updating a record in the presence management table in which the characteristic information "hd" is recorded in the loop process will be described. The state management unit 81 of the terminal device 70x references the callable communication ID field in the record in the presence management table in which the characteristic information "hd" is recorded. As a result, the state management unit 81 determines whether the callable communication ID field contains the communication ID "O01" acquired in step S51 (step S53).

[0117] If it is determined that the communication ID "O01" acquired in step S51 is not included in the field of the communication ID that can be called in the presence management table (NO in step S53), the status management unit 81 adds the acquired communication ID "O01" to the field of the communication ID that can be called (step S54).

[0118] When the process of step S54 is completed or when the answer to step S53 is YES, the state management unit 81 of the terminal device 70x refers to the field of the busy communication ID in the record in which the characteristic information "hd" is recorded in the presence management table. As a result, the state management unit 81 determines whether the communication ID "O01" acquired in step S51 is included in the field of the busy communication ID (step S55).

[0119] If it is determined that the communication ID "O01" acquired in step S51 is included in the field of busy communication ID in the presence management table (YES in step S55), the acquired communication ID is deleted from the field of busy communication ID (step S56). By the processing up to this point, whether the presence on the terminal device 10a side transitions from "chat" to "online" or from "offline" to "online", the communication ID of the terminal device 10a is recorded only in the field of callable communication ID in the record in which the characteristic information "hd" is recorded in the presence management table.

[0120] When the process of step S56 is completed, or if step S55 is NO, the state management unit 81 of the terminal device 70x refers to the presence of the characteristic in the record in which the characteristic information "hd" is recorded in the presence management table. As a result, the state management unit 81 determines whether the presence of the group is "online" (step S57).

[0121] If it is determined that the feature presence in the record in which the feature information "hd" is recorded is not "online" (NO in step S57), the state management unit 81 updates the feature presence to "online". If it is determined that the feature presence in the record in which the feature information "hd" is recorded is "online" (YES in step S57), the state management unit 81 does not update the feature presence in the record in which the feature information "hd" is recorded. As a result of the processing up to this point, the feature presence in the record in the presence management table in which the feature information "hd" is recorded becomes "online" (step S58).

[0122] Since the characteristic information "hd,th,ks" was acquired in step S51, the state management unit 81 performs a loop process to similarly update each record in the presence management table in which the characteristic information "th,ks" is recorded. When the loop process is completed, the presence management table in FIG. 7(b) is updated as shown in Table 1. Note that the presence change is also reflected in the destination management table in FIG. 7(a).

[0123] [Table 1] Next, the presence information "chat, (hd, th, ks), O01" is sent from the terminal device 10a to the terminal device 70x, and the processing when this presence information is acquired in step S51 will be described (chat in step S52).

[0124] First, the process of updating the record in the presence management table in which the characteristic information "hd" is recorded in the loop process will be described. The state management unit 81 of the terminal device 70x references the callable communication ID field in the record in the presence management table in which the characteristic information "hd" is recorded (see Table 1). As a result, the state management unit 81 determines whether the callable communication ID field contains the communication ID "O01" acquired in step S51 (step S63).

[0125] If it is determined that the communication ID "O01" acquired in step S51 is included in the field of the communication ID that can be called in the presence management table (YES in step S63), the acquired communication ID is deleted from the field of the communication ID that can be called (step S64).

[0126] When the process of step S64 is completed or when the result of step S63 is NO, the state management unit 81 of the terminal device 70x refers to the field of the busy communication ID in the record in which the characteristic information "hd" is recorded in the presence management table (see Table 1). As a result, the state management unit 81 determines whether the field of the busy communication ID contains the communication ID "O01" acquired in step S51 (step S65). If it is determined that the communication ID "O01" acquired in step S51 is not included in the field of the busy communication ID in the presence management table (NO in step S65), the acquired communication ID "O01" is added to the field of the busy communication ID (step S66). By the processing up to this point, the communication ID of the terminal device 10a is recorded only in the field of the busy communication ID in the record in which the characteristic information "hd" is recorded in the presence management table.

[0127] When the process of step S66 is completed, or when the answer is YES in step S65, the state management unit 81 of the terminal device 70x refers to the field of the callable communication ID in the record in which the characteristic information "hd" is recorded in the presence management table. As a result, the state management unit 81 determines whether the number of communication IDs recorded in the field of the callable communication ID in the record in which the characteristic information "hd" is recorded is 0 (step S67).

[0128] If the number of communication IDs recorded in the callable communication ID field in the record in which the characteristic information "hd" is recorded is 0 (YES in step S67), the status management unit 81 updates the presence of the characteristic in the record in which the characteristic information "hd" is recorded to "chat" (step S68).

[0129] If the number of communication IDs recorded in the callable communication ID field in the record in which the characteristic information "hd" is recorded is not 0 (NO in step S67), the state management unit 81 does not update the presence of the feature in the record in which the characteristic information "hd" is recorded. In other words, if a communication ID is recorded in the callable communication ID field in the record in which the characteristic information "hd" is recorded, the corresponding presence remains "online" and is not updated.

[0130] Since the characteristic information "hd,th,ks" was acquired in step S51, the state management unit 81 similarly updates each record in the presence management table in which the characteristic information "th,ks" is recorded in the loop processing. As a result, the presence management table of Table 1 becomes as shown in Table 2.

[0131] [Table 2] Next, a case will be described in which the presence information "offline, (hd, kt, cb), O03" is transmitted from the terminal device 10c to the terminal device 70x, and this presence information is acquired in step S51 (offline in step S52).

[0132] First, a process of updating a record in the presence management table in which the characteristic information "hd" is recorded in the loop process will be described. The state management unit 81 of the terminal device 70x references the "call available" field in the record in the presence management table (see Table 2) in which the characteristic information "hd" is recorded. As a result, the state management unit 81 determines whether the communication ID "O03" acquired in step S51 is included in the "call available communication ID" field (step S73).

[0133] If it is determined that the communication ID "O03" acquired in step S51 is included in the field of the communication ID that can be called in the presence management table (YES in step S63), the acquired communication ID "O03" is deleted from the field of the communication ID that can be called (step S74).

[0134] When the process of step S74 is completed or when the result of step S73 is NO, the state management unit 81 of the terminal device 70x refers to the field of the busy communication ID in the record in which the characteristic information "hd" is recorded in the presence management table (see Table 2). As a result, the state management unit 81 determines whether the communication ID "O03" acquired in step S51 is included in the field of the busy communication ID (step S75).

[0135] If it is determined that the communication ID "O03" acquired in step S51 is included in the field of the busy communication ID in the presence management table (YES in step S75), the acquired communication ID "O03" is deleted from the field of the busy communication ID (step S76). Through the processing up to this point, the communication ID of the terminal device 10c is deleted from the record in the presence management table in which the characteristic information "hd" is recorded.

[0136] When the processing of step S76 is completed or when the determination in step S75 is NO, the state management unit 81 of the terminal device 70x refers to the fields of the available communication ID and the busy communication ID in the record in which the characteristic information "hd" is recorded in the presence management table (see Table 2). As a result, the state management unit 81 determines whether conditions IF1 and IF2 are met (step S77). Condition IF1 is a case in which the number of communication IDs recorded in the available communication ID field in the record in which the characteristic information "hd" is recorded is 0, and the number of communication IDs recorded in the busy communication ID field is not 0. Condition IF2 is a case in which the number of communication IDs recorded in the available communication ID field in the record in which the characteristic information "hd" is recorded is 0, and the number of communication IDs recorded in the busy communication ID field is 0.

[0137] If the number of communication IDs recorded in the field of available communication IDs in the record in which the characteristic information "hd" is recorded is 0 and the number of communication IDs recorded in the field of currently used communication IDs is not 0 (IF1 in step S77), the state management unit 81 updates the presence of the characteristic in the record in which the characteristic information "hd" is recorded in the presence management table to "chat" (step S78).

[0138] If the number of communication IDs recorded in the field of available communication IDs in the record in which the characteristic information "hd" is recorded is 0 and the number of communication IDs recorded in the field of currently used communication IDs is 0 (IF2 in step S77), the state management unit 81 updates the presence of the characteristic in the record in which the characteristic information "hd" is recorded in the presence management table to "offline" (step S79).

[0139] If neither IF1 nor IF2 applies in step S77 (NO in step S77), the state management unit 81 does not update the presence of the feature in the record in which the feature information "hd" is recorded, and leaves it as "online."

[0140] Since the characteristic information "hd, kt, cb" was acquired in step S51, the state management unit 81 similarly updates each record in the presence management table in which the characteristic information "kt, cb" is recorded, in a loop process. As a result, the presence management table of Table 2 becomes as shown in Table 3.

[0141] [Table 3] Next, the process of starting communication will be described with reference to Fig. 14. Fig. 14 is a sequence diagram showing an example of the process of starting communication. As an example, the process of starting communication by terminal device 70x will be described below, but terminal device 70y can also start communication by similar processing. The case will be described where, at the time the process of Fig. 14 is started, the destination management table is in the state of Fig. 7(a) and the presence management table is in the state of Table 2.

[0142] The operation input receiving unit 72 of the terminal device 70x receives a request to display a list of characteristics of candidate destinations in response to an operation input by a user. In response to the display request, the display control unit 77 acquires a set of each piece of characteristic information recorded in the presence management table (see Table 2) and the presence of the group corresponding to the characteristic information. The storage unit 7000 pre-stores display image data corresponding to the characteristic information and the presence of the characteristic. As an example, the display image data corresponding to the characteristic information "hd" is text data indicating "Hokkaido." Also, as an example, the display image data corresponding to the characteristic presences "online, offline, chat" are image data of normal brightness, low brightness, and normal brightness + speech bubble icon, respectively. The display control unit 77 acquires a set of display image data of corresponding text and icon from the storage unit 7000 for each acquired set of characteristic information and presence. The display control unit 77 generates image data of the list of characteristics by arranging the acquired sets of display image data in association with the same row in a predetermined display area. The display control unit 77 outputs image data of the generated feature list to the display 720 (step S81).

[0143] FIG. 15 is a diagram showing an example of a feature list screen displayed when the presence management table is in the state of Table 2.

[0144] The display control unit 77 repeats the above process every time the presence management table is updated, thereby displaying on the display 720 a list of features showing the latest presence for each feature.

[0145] In step S81, the display control unit 77 outputs an item (button) corresponding to "online" at normal brightness, an item (button) corresponding to "offline" at low brightness, and an item corresponding to "chat" at normal brightness + a speech bubble icon (button). In Fig. 15, speech bubble icons 111 are displayed for "Tohoku" 110b and "Kyushu" 110h. Also, "San'in Sanyo" 110f is displayed at low brightness.

[0146] For example, when the user presses the item corresponding to "Tohoku" or "Kyushu" in the feature list shown in Fig. 15, the operation input receiving unit 72 receives the operation input and displays a message that the call is in progress. On the other hand, when the user presses the item "San'in Sanyo" in the feature list shown in Fig. 15, the operation input receiving unit 72 receives the operation input and displays a message that the call is out.

[0147] The following describes the process when the selection of the feature "Hokkaido" is accepted: The operation input accepting unit 72 accepts the selection of the feature corresponding to "Hokkaido" (step S82).

[0148] When the selection of the feature "Hokkaido" is accepted, the storage / readout processing unit 89 searches the destination management table of FIG. 7(a) using the feature information "hd" indicating the feature selected in step S82 as a search key, and reads out the corresponding communication ID and presence pairs "O01, chat", "O02, online", and "O03, online" (step S83).

[0149] The communication unit 71 of the terminal device 70x transmits a call start request to the management system 50 (step S84-1), which includes the communication ID "O02" whose corresponding presence is "online" among the communication IDs read out in step S83, the communication ID "C01" of the terminal device 70x itself, and the feature information "hd" indicating the feature selected in step S82.

[0150] Similarly, the communication unit 71 of the terminal device 70x sends a call start request to the management system 50 (step S86-1), which includes the communication ID "O03" whose corresponding presence is "online" among the communication IDs read out in step S83, the communication ID "C01" of the terminal device 70x itself, and the feature information "hd" indicating the feature selected in step S82.

[0151] When the communication unit 51 of the management system 50 receives the call start request transmitted by the terminal device 70x, it transmits this call start request to the terminal device 10b identified by the communication ID "O02" included in the call start request (step S84-2). Similarly, when the communication unit 51 of the management system 50 receives the call start request transmitted by the terminal device 70x, it transmits this call start request to the terminal device 10c identified by the communication ID "O03" included in the call start request (step S86-2).

[0152] The transmission control unit 82 of the terminal device 70x updates the destination management table by recording the communication ID "O02" of the terminal device 10b, which is the destination of the call start request, in the destination management table (step S85). Similarly, the transmission control unit 82 of the terminal device 70x updates the destination management table by recording the communication ID "O03" of the terminal device 10c, which is the destination of the call start request, in the destination management table (step S87).

[0153] When the input of a request to end the call is accepted at the terminal device 10a on the operator side (step S21'), the communication unit 11 transmits presence information including the characteristic information "hd, th, ks" read from the memory unit 1000, the communication ID "O01" of the terminal device 10a, and the presence "online" to the management system 50 (step S22').

[0154] When the communication unit 51 of the management system 50 receives the presence information transmitted by the terminal device 10a, it transmits the received presence information to the terminal device 70x on the guest side that has logged in to the management system 50 (step S24').

[0155] The communication unit 71 of the terminal device 70x receives the presence information transmitted from the terminal device 10a on the operator side via the management system 50. The storage / readout processing unit 89 of the terminal device 70x records the communication ID, presence, and feature information contained in the received presence information in the destination management table (step S26'). As a result, the destination management table is updated from the state of FIG. 7(a) to the state of Table 4.

[0156] [Table 4] The transmission control unit 82 of the terminal device 70x determines whether to transmit a call start request to the terminal device 10a that is the sender of the presence information received in step S24' (step S88). In this process, the transmission control unit 82 of the terminal device 70x references the presence of the terminal device 10a that has been updated in the destination management table and determines whether this presence is "online." If the presence of the terminal device 10a is not "online," the transmission control unit 82 determines not to transmit a call start request and ends the process.

[0157] If it is determined that the presence of the terminal device 10a is "online", the destination management table is referenced to the characteristic information "hd, th, ks" of the terminal device 10a to determine whether any of this characteristic information indicates the characteristic "Hokkaido" selected in step S82. If none of the characteristic information indicates the selected characteristic, the transmission control unit 82 determines not to send a start request and ends the process.

[0158] If any of the feature information indicates the selected feature, the transmission control unit 82 of the terminal device 70x determines to transmit a call start request. In response to the determination to transmit a call start request, the communication unit 71 of the terminal device 70x transmits a call start request to the management system 50, the call start request including the communication ID "O01" of the terminal device 10a whose presence information is to be transmitted, the communication ID "C01" of the terminal device 70x itself, and the feature information "hd" indicating the feature selected in step S82 (step S89-1).

[0159] When the communication unit 51 of the management system 50 receives the call start request sent by the terminal device 70x, it sends this call start request to the terminal device 10a identified by the communication ID "O01" included in the call start request (step S89-2).

[0160] Next, a process for establishing communication between the terminal devices 10 and 70 will be described with reference to Fig. 16. Fig. 16 is a sequence diagram showing an example of a process for establishing communication between the terminal devices 10 and 70. Below, a case where the terminal device 10a responds to a call start request from the terminal device 70x will be described.

[0161] The communication unit 11 of the terminal device 10a receives the call start request transmitted from the terminal device 70x via the management system 50 (see step S89-2). When the call start request is received, the display control unit 17 of the terminal device 10a outputs to the display 720 a message including request source information corresponding to the communication ID "CO1" included in the call start request and the characteristic "Hokkaido" corresponding to the characteristic information "hd" included in the call start request.

[0162] 17 is an example of a message display. By displaying the feature 151 in the message 150, the operator, who is the user of the terminal device 70, can know in advance, before starting the call, which country or region the explanation will be asked about.

[0163] 17 includes a response button 152 for accepting permission to start a call. When the user presses the response button 152 on the screen of the terminal device 10a, the operation input accepting unit 12 accepts the operation input for permission to respond (step S101). When the operation input for permission to respond is accepted, the communication unit 11 of the terminal device 10a transmits a response including the communication ID "C01" of the source of the call start request and the communication ID "O01" of the terminal device 10a itself to the management system 50 (step S102).

[0164] The communication unit 51 of the management system 50 receives the response transmitted by the terminal device 10a. The communication unit 51 of the management system 50 transmits the received response to the terminal device 70x identified by the communication ID “C01” included in the response (step S103).

[0165] When the communication unit 71 of the terminal device 70x receives the response, it transmits a communication establishment request to the management system 50 to establish communication with the terminal device 10a that sent the response (step S104). The communication establishment request includes the communication ID "C01" of the terminal device itself and the communication ID "O01" of the terminal device 10a that sent the response.

[0166] The communication unit 71 of the management system 50 receives the communication establishment request. The session control unit 54 of the management system 50 controls the establishment of a session for transmitting content data, such as image data and sound data, between the terminal devices 10a and 70x identified by the communication ID "O01, C01" included in the communication establishment request via the relay device 30 (step S105). Note that the session is not limited to the above, and may be a session for directly transmitting content data between the terminal devices 10a and 70x via the communication network 2.

[0167] When communication between the terminal devices 10a and 70x is established, the terminal devices 10a and 70x transmit image data of images captured by the terminals themselves and sound data collected by the terminals themselves to the terminal devices 70x and 10a on the other side via the relay device 30. The display control units 17 and 77 of the terminal devices 10a and 70x output the received content data, thereby enabling a call to be initiated between the users.

[0168] <Proposal process> Next, the process in which the proposal control device 3 proposes ways to improve speaking style to the operator after communication is established will be described using Fig. 18 and Fig. 19. Fig. 18 is an example of a sequence diagram relating to the process in which the proposal control device 3 proposes ways to improve speaking style. First, Fig. 18 describes the case in which proposals are made at regular intervals.

[0169] S201, S202: When communication is established, the communication unit 11 of the terminal device 10 and the communication unit 71 of the terminal device 70 repeatedly transmit audio data and video data to the relay device 30.

[0170] S203: The relay device 30 receives these and transmits the audio data and video data to the proposed control device 3. A communication ID is also transmitted to the relay device 30 along with the audio data and video data, and the communication ID can be used to determine whether the data was transmitted by the guest or the operator.

[0171] S204: The communication unit 91 of the proposal control device 3 receives the audio data and video data. The audio recognition unit 92 performs audio recognition and further morphological analysis, and associates words with parts of speech and registers them in the utterance storage unit 3001.

[0172] S205: The loudness recording unit 93 of the proposed control device 3 detects the loudness of the voice from the amplitude of the voice data and registers it in association with a word. The tone determination unit 94 performs a Fourier transform on the voice data, determines the tone according to the frequency of the maximum intensity, and registers it in association with a word. The video analysis unit 97 analyzes the video data and registers the facial expression, whether it is a smile or not, in association with a word.

[0173] S206: The re-asking detection unit 95 of the proposal control device 3 detects a re-asking by determining whether a pre-registered character string is included in the list of words uttered by the guest. If a re-asking is detected, the number of re-askings stored in the re-asking count storage unit 3002 is updated (increased by one). The conversation speed detection unit 96 detects the conversation speed between the operator and the guest based on the number of characters per fixed time period, and registers this in the conversation speed storage unit 3003. It is advisable to register the latest value, but it is also possible to register an average of past conversation speeds.

[0174] S207: The suggestion unit 98 determines at regular intervals whether to suggest to the operator ways to improve their speaking style, as will be described later. For example, it determines whether the number of times the guest has asked to repeat themselves exceeds the number stored in the suggestion information storage unit 1, and if so, it determines to make a suggestion. It also determines whether the difference between the operator's speaking speed and a reference value exceeds the speed difference stored in the suggestion information storage unit 2, and if so, it determines to make a suggestion. The suggestion unit 98 also compares the service time from the start of communication with the average service time stored in the time information storage unit 3006, and if the average service time has passed, it determines to make a suggestion.

[0175] S208: If it is determined that a proposal should be made, the proposal unit 98 of the proposal control device 3 acquires a proposal and a display method from the proposal information storage unit 1 or the proposal information storage unit 2.

[0176] S209: The suggestion unit 98 transmits the suggestion and the display method to the terminal device 10 of the operator.

[0177] S210: The communication unit 11 of the operator's terminal device 10 receives the proposal and the display method, and the display control unit 17 displays the proposal on the in-call screen. An example of the in-call screen is shown in FIG.

[0178] In this way, the operator's terminal device 10 can display ways to improve speaking style at regular intervals during a call, which makes it easier to improve customer service efficiency.

[0179] FIG. 19 is an example of a sequence diagram illustrating a process in which the proposal control device 3 makes a proposal in response to a request from an operator.

[0180] S301 to S306: These steps may be the same as steps S201 to S206.

[0181] S307: The operator wants to know how to improve his or her speaking style, and inputs a predetermined operation into the terminal device 10. For example, the operator presses the proposal request button displayed on the operator's call screen. The operation input receiving unit 12 receives the press, and the communication unit 11 sends the proposal request together with the communication ID to the management system 50. Note that proposal requests may relate not only to the number of times to ask again and the conversation speed, but also to the conversation topic. The operator may specify the content of the proposal and request, or may request all possible proposals. In the explanation of Figure 19, it is assumed that all proposals are requested.

[0182] S308: The communication unit 51 of the management system 50 receives the proposal request and transfers it to the proposal control device 3.

[0183] S309: The suggestion unit 98 of the suggestion control device 3 determines whether to make a suggestion based on the number of times the guest has asked to repeat the question and the speaking speed. This determination method may be the same as that shown in FIG. 18. If no suggestion is necessary, the suggestion unit 98 transmits a message to that effect to the terminal device 10. Suggestions related to conversation themes are always made regardless of whether they should be suggested or not. That is, the words stored in the utterance storage unit 3001 are weighted by one or more of the volume of voice, tone, and facial expression, and a list of words that the guest showed interest in during the conversation is provided. Details will be given later.

[0184] S310 to S312: The proposal control device 3 transmits the proposal in the same manner as in steps S208 to S210, and the display control unit 17 of the operator's terminal device 10 displays it.

[0185] The suggestion unit 98 may suggest ways to improve speaking style based not only on the number of times the customer has asked to repeat themselves or the speaking speed as shown in Figs. 18 and 19, but also on the volume or tone of the voice. For example, if the volume of the voice is below a threshold, a suggestion is made to "speak louder." If the tone is low, a suggestion is made to "speak brighter." If the facial expressions analyzed from the video data show few smiling faces (below a threshold), a suggestion is made to "serve customers with a smile."

[0186] <Example of suggestions displayed on the call screen> FIG. 20 is an example of a call in progress screen 300 displayed on the operator's terminal device 10. Since this is the operator's call in progress screen 300, guest g is shown large, and an image of operator a is displayed in picture-in-pin at the bottom right. Also, a proposal request button 301 is displayed at the top right of the call in progress screen 300. The proposal request button 301 is a button that the operator can use to request ways to improve their speaking style at any time. When the proposal request button 301 is pressed, a pull-down menu of request items 302 is displayed, and the operator can select a request item from "number of times to ask for replies," "conversation speed," "conversation topic," or "all."

[0187] When the suggestion control device 3 transmits suggestions regarding "number of times to ask again" and "conversation speed," the in-call screen 300 displays the suggestions. In FIG. 20, a suggestion 303 saying "speak a little more slowly" is displayed at the bottom of the in-call screen 300. The operator can see the suggestion and improve their speaking style while serving the customer. The suggestion 303 can be displayed anywhere. It may also be displayed as a pop-up.

[0188] The suggestions 303 may be displayed as icons or image data instead of text. FIG. 21 is a diagram showing an example of suggestions using icons. First, FIG. 21(a) shows an icon 304 that makes suggestions regarding the conversation speed. For example, this icon 304 constantly moves on the in-call screen 300 at a speed corresponding to the conversation speed (the suggestions notify the conversation speed at regular intervals), and when the conversation speed exceeds a threshold, the suggestion unit 98 alerts the user by changing the color, for example. Alternatively, the icon 304 may not normally be displayed on the in-call screen 300, and when the conversation speed exceeds the threshold, the suggestion unit 98 notifies the user of the conversation speed, and the operator's terminal device 10 may display the icon 304 that moves at a speed corresponding to the conversation speed.

[0189] 21(b) also shows an icon 305 that indicates the speaking speed. This icon 305 is modeled after a vehicle speedometer, and is always displayed on the call in progress screen 300. The faster the speaking speed, the more the needle 305a rotates clockwise and indicates the surrounding color, so the operator can determine whether the speaking speed is appropriate by the color indicated by the needle 305a.

[0190] 21(c) shows an icon 306 that indicates the volume of the voice according to the number of times the user has asked to repeat the call. This icon 306 is a bar that resembles a volume meter, and is always displayed on the call in progress screen 300. The louder the voice, the longer the bar becomes, and the more attention-grabbing the color displayed on the left side, such as red, becomes. In other words, the bar is displayed in different colors from left to right. Therefore, if the volume is below a certain level, the icon 306 is displayed in red, and the operator can determine that the voice is quiet when the bar is only displayed in red.

[0191] Fig. 22 is a diagram showing an example of suggested conversation topics displayed on during-call screen 300. In Fig. 22, a pie chart 310 is displayed as a pop-up on during-call screen 300. The display method is one example, and the pie chart may be displayed on a separate screen or as part of during-call screen 300.

[0192] A conversation theme is an example of a conversation suggestion in which the proposal control device 3 suggests appropriate words to the operator from words contained in past conversations when, for example, a guest is unsure which plan to choose. This makes it easier for the operator to identify words (themes) that the guest is interested in from various conversations the guest has had.

[0193] In FIG. 22, the score calculated by weighting the words (nouns) registered in the utterance storage unit 3001 is displayed as a percentage. The calculation method will be explained. The suggestion unit 98 acquires, for example, a noun from the words stored in the utterance storage unit 3001. If a noun has synonyms, it may be considered the same noun. For example, diving and skin diving are considered the same noun. This type of processing is called data cleansing. Data cleansing is the process of finding duplications, typographical errors, and spelling variations in data stored in a database, and deleting, correcting, normalizing, etc. to improve the quality of the data.

[0194] Next, the loudness (loud / medium / quiet), high / low tone, and smiling / normal facial expression associated with the noun are converted into coefficients α, β, and γ, respectively, and weighted. The loudness (loud / medium / quiet) corresponds to the coefficients αn (α1-α3), the high / low tone corresponds to the coefficients βn (β1, β2), and the smiling / normal facial expression corresponds to the coefficients γn (γ1, γ2).

[0195] Therefore, the score for each noun is calculated as follows: Score = Σ(αn × βn × γn) Σ indicates addition for the same noun.

[0196] The proposing unit 98 calculates the ratio by, for example, adding up the scores of nouns whose scores are equal to or greater than a threshold (that is, removing nouns that appear only once) and dividing the score of each noun by the total value. Proportion of each noun = Score of each noun / Sum of each score Create a pie chart like the one shown in Figure 22 based on the ratio of nouns.

[0197] It is even better to detect and weight the relationship between adjectives, adjectival verbs, or auxiliary verbs and nouns. For example, when a guest says "a beautiful sea," they are using "sea" in a positive sense, but when they say "the sea was dirty," they are using "sea" in a negative sense. Therefore, if a positive adjective is detected for a noun, the score is increased, and if a negative adjective is detected for a noun, the score is decreased. Adjectival verbs can be judged in a similar way. For example, auxiliary verbs are "tai" (want to go to the sea) in "I want to go to the sea" and "taku" (do not want to go to the sea) and "nai" (not want to go to the sea). The former is positive about the sea, and the latter is negative, so the score can be increased or decreased in a similar way.

[0198] <Detailed explanation of the processing of the proposed control device> 23 is a flowchart illustrating the process when the proposed control device 3 prompts the operator to speak louder. The process in FIG. 23 starts, for example, when a call is started.

[0199] First, the proposing unit 98 acquires the information stored in the proposed information storage unit 1, the proposed information storage unit 2, and the time information storage unit 3006 (S401).

[0200] The communication unit 91 of the proposal control device 3 receives the voice data and the video data (S402). The voice recognition unit 92 analyzes the voice data and the video data and registers words and the like in the utterance storage unit 3001 (S403).

[0201] The repeat-listening detection unit 95 determines whether or not the guest has repeated the question, and if so, updates the number of times the guest has repeated the question in the repeat-listening count storage unit 3002 (S404).

[0202] The suggestion unit 98 determines whether a certain time has elapsed (S405), and every time the certain time has elapsed, determines whether the number of times of asking again stored in the number of times of asking again storage unit 3002 has exceeded the number (threshold) of the suggested information storage unit 1 (S406). Note that the same determination is made not only after the certain time has elapsed, but also when requested by the operator.

[0203] If the determination in step S406 is Yes, the suggestion unit 98 transmits a suggestion to the operator's terminal device 10 to encourage the operator to speak louder (S407).

[0204] The suggestion unit 98 initializes the number of times of re-listening stored in the number-of-times-of-re-listening storage unit 3002 (S408), i.e., resets it to zero.

[0205] In this way, the suggestion unit 98 can provide the operator with a suggestion to speak louder at regular intervals or when requested by the guest.

[0206] 24 is a flowchart illustrating the process when the proposed control device 3 prompts the operator to slow down the speaking speed. The process in FIG. 24 starts, for example, when a call is started.

[0207] First, the proposing unit 98 acquires the information stored in the proposed information storage unit 1, the proposed information storage unit 2, and the time information storage unit 3006 (S501).

[0208] The communication unit 91 of the proposal control device 3 receives the voice data and the video data (S502). The voice recognition unit 92 analyzes the voice data and the video data and registers words and the like in the utterance storage unit 3001.

[0209] The speaking speed detection unit 96 counts the number of characters spoken by the operator (S503). This number of characters is the number of syllables before being converted into kanji.

[0210] The speaking speed detection unit 96 determines whether a certain time has elapsed (S504), and each time the certain time has elapsed, calculates the speaking speed by dividing the number of characters spoken by the guest by the certain time (S505). The speaking speed is registered in the speaking speed storage unit 3003. The speaking speed is also calculated when requested by the operator. In this case, the speaking speed detection unit 96 calculates the speaking speed not based on the certain time, but on the time from when it starts counting the number of characters until the operator makes the request.

[0211] The proposing unit 98 determines whether the difference between the conversation speed and the reference value exceeds the speed difference (threshold value) in the proposed information storage unit 2 (S506).

[0212] If the determination in step S506 is Yes, the suggestion unit 98 transmits a suggestion to the operator's terminal device 10 to encourage the operator to slow down the speaking speed (S507).

[0213] The proposing unit 98 initializes the speaking speed stored in the speaking speed storage unit 3003 (S508), that is, resets it to zero.

[0214] In this way, the suggestion unit 98 can provide the operator with a suggestion to slow down their speaking speed at regular intervals or when requested by the guest.

[0215] It is also possible to propose shortening the call time by determining whether the time elapsed since the start of the call has exceeded the average customer service time in either or both of Figures 23 and 24. The proposal may be made not based on the average customer service time itself, but in cases such as average customer service time x 0.8 < call time, average customer service time x 1.2 < call time, etc. The operator can provide service while keeping track of the call time.

[0216] 25 is a flowchart illustrating the process when the proposal control device 3 proposes a conversation theme to an operator. The process in FIG. 25 starts when a proposal request is received from a guest.

[0217] First, the communication unit 91 of the proposal control device 3 receives a proposal request from the operator's terminal device 10 (S601).

[0218] The suggestion unit 98 acquires all words, voice volume, tone, and facial expressions recorded during the call from the utterance storage unit 3001 (S602).

[0219] Next, the suggestion unit 98 performs data cleansing on the acquired data (S603). The suggestion unit 98 converts the words after data cleansing into scores. The suggestion unit 98 processes the nouns that are the conversation topic into visual information such as a pie chart and transmits it to the operator's terminal device 10 (S604). This allows the operator's terminal device 10 to display a pie chart 310 or the like. In addition to a pie chart, a bar graph or a word cloud may also be used. A word cloud is a method of dividing text data into words and illustrating each extracted word in a size corresponding to the frequency of appearance (score) in the text.

[0220] In this way, the suggestion unit 98 provides conversation topics when requested by the guest, so the operator can guide the guest to topics that interest them, thereby improving customer service efficiency.

[0221] <Major Effects> As described above, the communication system of this embodiment proposes specific improvement methods based on the analysis results of voice data, thereby improving operational efficiency in customer service, etc. Furthermore, by displaying the conversation topic, the operator can guide the conversation to topics that interest the guest, thereby improving operational efficiency.

[0222] <Other application examples> The best mode for carrying out the present invention has been described above using examples, but the present invention is not limited to these examples in any way, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention.

[0223] For example, in this embodiment, a method for improving customer service efficiency has been described using the example of customer service at a travel agency, but the method can also be suitably applied to other industries, such as finance, health consultation, and medical consultation.

[0224] This can also be applied to local government offices. For example, staff can serve customers by the type of business they are responsible for (resident registration, family register, tax payment, bulky waste, pensions, etc.).

[0225] Furthermore, the management system 50 may have the functions of the relay device 30, or the relay device 30 may have the functions of the management system 50. Furthermore, the functions of the management system 50 may be realized by multiple servers, or multiple management systems 50 may exist.

[0226] 6, 8, and other configuration examples shown in the above embodiments are divided according to the main functions to facilitate understanding of the processing of the management system 50 and the terminal devices 10, 70. However, the present invention is not limited by the manner in which each processing unit is divided or the names of each processing unit. The management system 50 and the terminal devices 10, 70 can also be divided into more processing units depending on the processing content. Furthermore, one processing unit can also be divided so that it includes more processes.

[0227] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or a conventional circuit module designed to perform each function described above.

[0228] Also, in some embodiments, the management system, relay device, or proposed control device includes multiple computing devices, such as a server cluster, configured to communicate with each other via any type of communication link, including a network, shared memory, etc., and to perform the processes disclosed herein. Similarly, the terminal device 10, 70 may include multiple computing devices configured to communicate with each other.

[0229] Furthermore, the management system, relay device, or proposed control device can be configured to share the disclosed processing steps in various combinations. For example, some of the processes performed by the management system, relay device, or proposed control device can be performed by the terminal device 10, 70. Also, some of the processes performed by the terminal device 10, 70 can be performed by the management system, relay device, or proposed control device. [Explanation of symbols]

[0230] 3 Proposed control device 7 stores 8. Call Center 50 Management Systems 10,70 Terminal equipment 100 Communication Systems [Prior art documents] [Patent documents]

[0231] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-103081

Claims

1. A communication system in which a first terminal device operated by an operator and a second terminal device operated by a guest communicate with each other via a network, a proposal unit that determines a proposal to the operator based on the voice data of the operator transmitted from the first terminal device or the voice data of the guest transmitted from the second terminal device; a first communication unit that transmits the proposal to the first terminal device; The first terminal device a second communication unit for receiving the proposal; a display control unit that displays the proposal received by the second communication unit; A communication system comprising:

2. the suggestion unit analyzes the voice data of the operator transmitted from the first terminal device or the voice data of the guest transmitted from the second terminal device, and determines a suggestion regarding a method for improving the speaking style of the operator; The first terminal device 2. The communication system according to claim 1, wherein suggestions on how to improve the speaking style of the operator are displayed.

3. a repeat-listening detection unit that analyzes the voice data of the guest transmitted from the second terminal device and detects the repeat-listening of the guest; 3. The communication system according to claim 2, wherein the suggestion unit determines a suggestion regarding how to improve the speaking style of the operator in accordance with the number of times the operator has asked to reflect on something detected by the reflection detection unit.

4. a speech speed detection unit that analyzes the voice data of the operator transmitted from the first terminal device and detects the speech speed of the operator; 3. The communication system according to claim 2, wherein the suggestion unit determines a suggestion regarding a method for improving the speaking style of the operator in accordance with the speaking speed detected by the speaking speed detection unit.

5. 4. The communication system according to claim 3, wherein the suggestion unit determines a suggestion to increase the volume of the operator's voice as a method for improving the operator's speaking style.

6. 5. The communication system according to claim 4, wherein the suggestion unit determines a suggestion to slow down the speaking speed as a method for improving the speaking style of the operator.

7. The first terminal device displaying a button for accepting an operation by the operator to request the proposal; When the button is pressed, the second communication unit transmits a proposal request to the communication system; 7. The communication system according to claim 1, wherein the proposal unit determines a proposal to the operator in response to the proposal request.

8. a voice recognition unit that performs voice recognition on the voice data of the guest transmitted from the second terminal device and stores words included in the voice data in a utterance storage unit; the suggestion unit processes the list of words stored in the utterance storage unit into visual information in response to the suggestion request; the first communication unit transmits the visual information to the first terminal device; 8. The communication system according to claim 7, wherein the first terminal device displays the visual information.

9. the first communication unit receives video data showing the guest from the second terminal device; a video analysis unit that analyzes the video data and stores the facial expressions of the guests in association with the words in the utterance storage unit; the suggestion unit weights the expressions to the words stored in the utterance storage unit in response to the suggestion request, and processes the list of words into visual information; 9. The communication system according to claim 8, wherein the first terminal device displays the visual information.

10. a second terminal device operated by a guest and a terminal device operated by an operator that communicates via the system, a second communication unit that receives from the system a proposal to the operator, the proposal being determined based on voice data of the operator transmitted from a terminal device or voice data of the guest transmitted from the second terminal device; a display control unit that displays the proposal received by the second communication unit; A terminal device comprising:

11. A communication method performed in a communication system in which a first terminal device operated by an operator and a second terminal device operated by a guest communicate via a network, comprising: a proposing unit determining a proposing option for the operator based on the voice data of the operator transmitted from the first terminal device or the voice data of the guest transmitted from the second terminal device; a first communication unit transmitting the proposal to the first terminal device; The first terminal device a second communication unit receiving the proposal; a display control unit displaying the proposal received by the second communication unit; A communication method comprising:

12. A second terminal device operated by a guest and a terminal device operated by an operator communicating through the system are a second communication unit that receives from the system a proposal to the operator, the proposal being determined based on voice data of the operator transmitted from a terminal device or voice data of the guest transmitted from the second terminal device; a display control unit that displays the proposal received by the second communication unit; A program to function as a

Images