Communication System

Abstract

To provide a communication log review function for supporting mutual recognition in information sharing and communication.SOLUTION: In a communication system of an embodiment, multiple users communicate with each other through a group call in which the users participate, received speech data is concurrently distributed to a plurality of other users, and while accumulating a speech recognition result of the received speech data and input transmission information in text format as a communication log, a text distribution is performed to each user. Furthermore, an information identification tag that can be assigned to each communication log is stored, and the communication log is accumulated in association with the information identification tag. Based on filtering conditions including information identification tags, the accumulated communication logs are then retrieved, and one or more retrieved communication logs are provided to the users.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a technology for supporting communication (contact, shared awareness, communication, etc.) using voice and text. [Background technology]

[0002] An example of voice communication is a transceiver. A transceiver is a radio device that has both radio wave transmission and reception functions, allowing one user to communicate with multiple users (one-way or two-way information transmission). Examples of transceiver use can be seen at construction sites, event venues, hotels, etc. Taxi radios are also an example of transceiver use. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-7005 A Summary of the Invention [Problem to be solved by the invention]

[0004] The objective of the present invention is to provide a communication system that provides a function for reviewing communication logs in a group call environment in which many users participate, and supports mutual recognition in information sharing and communication. [Means for solving the problem]

[0005] A communication system according to an embodiment is a communication system in which multiple users communicate with each other through a group call. The system includes a communication control unit that performs a first process of broadcasting speech data received from a user terminal to each of multiple other user terminals, and a second process of chronologically accumulating, as communication logs, speech recognition results obtained by speech recognition processing of the received speech data and text-format communication information input from the user terminals, and performing text delivery so that the communication logs are displayed synchronously on each of the user terminals, and a storage unit that stores information identification tags that can be assigned to each of the communication logs. The communication control unit accumulates the communication logs in association with the information identification tags, and performs text delivery of the communication logs including the information identification tags in the second process, and searches the accumulated communication logs based on filtering conditions including the information identification tags and provides one or more of the searched communication logs to the user terminals. and an agent control unit that searches for the communication log that matches a set determination condition and makes a predetermined contact with the user terminals in the communication group based on whether or not there is a communication log that matches the determination condition. . [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a network configuration diagram of a communication system according to a first embodiment. [Figure 2] 1 is a configuration block diagram of a communication management device according to a first embodiment. [Figure 3] FIG. 2 is a block diagram illustrating the configuration of a user terminal according to the first embodiment. [Figure 4] FIG. 3 is a diagram illustrating an example of various information according to the first embodiment. [Figure 5] FIG. 10 is a diagram showing an example of communication history (log) information according to the first embodiment, and is an example of a communication screen including a watch list. [Figure 6] 10A and 10B are diagrams showing an example of a communication by text input on a communication screen and an example of a filtering condition setting screen for a watch list in the first embodiment. [Figure 7]10 is an example of an information identification tag (semantic tag) addition / edit screen in the first embodiment. [Figure 8] FIG. 10 is a diagram showing an example of how a watch list is displayed on the communication screen in the first embodiment. [Figure 9] 10A and 10B are diagrams illustrating an example of tag update and an example of adding a completion tag to a communication history in the first embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of non-display control based on a completion tag in the first embodiment. [Figure 11] FIG. 2 is a diagram showing an example of a communication screen on the mobile communication terminal of the first embodiment. [Figure 12] 3A to 3C are diagrams showing an example of a watch list setting screen and an example of a watch list screen of the mobile communication terminal of the first embodiment. [Figure 13] FIG. 2 is a diagram showing a processing flow of the communication system of the first embodiment. [Figure 14] FIG. 10 is a diagram showing a processing flow relating to tagging operations, setting of filtering conditions, and display of a watch list in the first embodiment. [Figure 15] FIG. 10 is a configuration block diagram of a communication management device according to a second embodiment. [Figure 16] FIG. 10 is a diagram showing a processing flow of automatic assignment control of information identification tags according to the second embodiment. [Figure 17] FIG. 10 is a configuration block diagram of a communication management device according to a third embodiment. [Figure 18] FIG. 11 is a diagram showing a processing flow relating to an assistance function by an agent in the third embodiment. BEST MODE FOR CARRYING OUT THE INVENTION

[0007] (First embodiment) 1 to 14 are diagrams showing a first embodiment. Fig. 1 is a network configuration diagram of a communication system according to this embodiment. The communication system is centered around a communication management device (hereinafter referred to as the management device) 100, and mainly provides an information transmission support function via group calls.

[0008] Below, we will explain how the communication system is applied to hotel room management as an example, but the system is not limited to this. In other words, the communication system can be applied to various situations in which multiple users communicate within a communication group, and is provided as a communication field for both the contactor and the contactee.

[0009] In hotel guest room management, a user terminal 300 operated by a task requester is connected to the management device 100. The task requester is a person who directly contacts guests, including through conversation, or a manager who manages guest rooms other than the front desk clerk. In addition to the manager, for example, a task manager (group leader) who oversees multiple users can also operate his or her own user terminal 300 and communicate with each user through each user terminal 500. In this case, the task manager acting as the task requester can contact each worker on behalf of the other users based on a message received from the manager, or, even without receiving a message from the manager, can carry out the work at his or her own discretion based on schedule information managed by the manager and contact each worker on behalf of the other users with a task request, etc.

[0010] The work requester can input text of the communication information using manual input means such as a keyboard, mouse, screen keyboard, or touch panel keyboard (or numeric keypad). The user terminal 300 transmits the information input through the manual input means (communication information in text format) to the management device 100. Note that although text input is given as an example, voice input is also possible.

[0011] The user terminal 300 is a desktop computer or a tablet computer, and is equipped with a data communication function and a calculation function (CPU, etc.) via an IP (Internet protocol) network or a mobile communication network. The user terminal 300 also has a display device (or a touch panel display device) and the above-mentioned manual character input means.

[0012] The user terminal 500 is a mobile communication terminal carried by a worker, and is, for example, a portable mobile terminal such as a multi-function mobile phone such as a smartphone, a PDA (Personal Digital Assistant), a tablet terminal, etc. The user terminal 500 has a communication function, a calculation function, and an input function, and connects to the management device 100 by wireless communication via an IP (Internet protocol) network or a mobile communication network to perform data communication.

[0013] The management device 100 receives manually input text-formatted communication information from the user terminal 300 and transmits it to each user terminal 500 operated by a worker. Furthermore, the management device 100 generates synthetic speech based on the text data input by the work requester, in addition to the text data, and transmits the generated synthetic speech data to each user terminal 500. The worker (contact recipient) receives the contact from the work requester (contacting party) and performs an action in response to the contact (for example, the requested work). At this time, when performing the action, the worker responds with spoken voice in response to the received contact, and the worker's spoken voice data is transmitted to the management device 100. The management device 100 can record spoken text based on the spoken voice data as a communication history and broadcast the spoken voice data to the user terminals 300, 500 in the communication group.

[0014] In this way, the management device 100 of this embodiment broadcasts the spoken text obtained by speech recognition of the input text and spoken voice data to each terminal in the communication group, and also broadcasts the voice synthesis data and spoken voice data based on the text input to each terminal in the communication group, thereby supporting communication between multiple users.

[0015] 2 is a configuration block diagram of the management device 100. The management device 100 includes a control device 110, a storage device 120, and a communication device 130. The communication device 130 manages communication connections and controls data communication between the user terminal 300 and multiple user terminals 500. The communication device 130 controls broadcast distribution to simultaneously send voice data and text data of the same content to multiple user terminals 500.

[0016] The control device 110 includes a user management unit 111, a communication control unit 112, a voice recognition unit 113, a voice synthesis unit 114, and an information type tag setting unit 115. The storage device 120 includes user information 121, group information 122, communication history (communication log) information 123, a voice recognition dictionary 124, a voice synthesis dictionary 125, and information type tag master information 126.

[0017] 3 is a block diagram of each configuration of the user terminals 300 and 500. The user terminal 300 is, for example, a desktop computer device, and is configured to include a communication / talk unit 310, a communication App control unit 320, a microphone 330, a speaker 340, a display unit 350 which is a display device, a storage unit 360, and an input operation unit 370 such as a keyboard. Also, as an input means other than a keyboard, voice input is possible via the microphone 330.

[0018] The user terminal 500 includes a communication / call unit 510, a communication App control unit 520, a microphone 530, a speaker 540, a display / input unit 550 such as a touch panel, and a storage unit 560. A pin microphone can be connected to the microphone 530, and earphones or headphones can be connected to the speaker 540, allowing hands-free listening and speaking. A headset microphone with an integrated speaker can also be connected to the microphone 530 and the speaker 540. These can be realized by wired or wireless connection. A similar configuration can also be applied to the user terminal 300.

[0019] The user terminal 300 and the user terminal 500 both have the same basic functional configuration, and for example, a work requester can also use the user terminal 500. Furthermore, the user terminals 300 and 500 each have a communication app control unit 320 or 520 that controls and processes in the system, and performs screen control and input control at each terminal. Note that screen information and various information displayed on the screen (e.g., information identification tags) can be obtained in advance from the management device 100 when the communication app control units 320 or 520 are started and stored in the storage units 360 or 560. Furthermore, each screen control may be configured to communicate with the management device 100 and obtain necessary information each time.

[0020] FIG. 4 is a diagram showing an example of various types of information. User information 121 is registration information for users who use this communication system. The user management unit 111 controls the setting of user IDs, user names, attributes, and groups through a specified management screen. Group information 122 is group identification information that partitions communication groups. The sending and receiving of communication information and broadcast distribution are controlled by communication group ID, and control is performed to prevent information from being mixed between different communication groups. In the user information 121, the communication groups registered in the group information 122 can be linked to each user.

[0021] Such groupings could be used, for example, to split guest room management into multiple departments, with bellpersons (luggage carriers), concierges, and housekeeping (cleaning) each assigned to a different group, creating a more segmented communication environment for each group. Another consideration is whether communication is unnecessary based on role. For example, food servers and bellpersons (luggage carriers) can be placed in separate groups because they do not need to communicate directly. There are also cases where communication is unnecessary due to geographical location. For example, if Branch A and Branch B are geographically distant and do not need to communicate frequently, they can be placed in separate groups.

[0022] Fig. 5 is a diagram showing an example of communication history (log) information, and is an example of a communication screen including a watch list. The example in Fig. 5 is an example of a screen displayed on the user terminal 300. The watch list WR will be described later.

[0023] In Figure 5, the communication screen is composed of a first display area W1 in which the communication log is displayed chronologically, a second display area W2 including an input field in which communication information is entered, and a third display area W3 in which the watch list WR is displayed.

[0024] The first display area W1 displays the communication logs of multiple users participating in the group call in chronological order in speech bubbles. That is, the communication history information 123 includes text entered by the work requester in the second display area W2 (or spoken text based on the work requester's spoken voice), text spoken by the worker, and the like, stored in chronological order along with time information. The spoken text is the speech recognition result obtained by speech recognition processing of the spoken voice data. The communication history information 123 is generated and stored for each communication group.

[0025] Note that the speech data (excluding synthesized speech data) may be configured to be stored as an audio file in a predetermined storage area. For example, the communication control unit 112 may be configured to record the storage location of the audio file in the communication history 123 in association with the corresponding communication log. In the example of Fig. 5, the second display area W2 including an input field for inputting communication information may include a "microphone" button, and when the "microphone" button is selected, the communication app control unit 320 activates the microphone 330 and performs sound collection processing.

[0026] The second display area W2, which includes an input field for inputting communication information, can also be configured to include a function for uploading any image. For example, by selecting the "Attach Image" button, image data (still image or moving image) can be uploaded to the management device 100. For example, image data accompanying text-entered communication information can be uploaded, or image data captured for purposes such as reporting work completion or the status of the work site can be uploaded. In this case, the communication history 123 records an image upload history, which can be linked to the uploaded image data for log management.

[0027] The communication control unit 112 of the management device 100 functions as each of a first control unit and a second control unit. The first control unit controls broadcast distribution of uttered voice data received from one user terminal 500 to each of the other multiple user terminals 500. The second control unit accumulates the uttered voice recognition results obtained by processing the received uttered voice data into voice recognition data in chronological order as a communication history 123 between users, and controls text distribution so that the communication history 123 is displayed synchronously on all user terminals 500, including the user terminal 500 of the user who made the utterance.

[0028] The function of the first control unit is to broadcast speech data, providing a group call in which one user's speech is delivered to multiple other users. Group calls can be performed using either half-duplex or full-duplex communication, and the choice of communication method is up to the user. In other words, this system can provide one-to-many or many-to-many group call functionality.

[0029] The speech data is speech data uttered by the user. As described above, speech synthesis data artificially generated from text information input from the user terminal 300 is also subject to broadcast distribution by the first control unit. The communication control unit 112 may be configured to also distribute the speech data received from the user terminal 500 to the user terminal 300 and play the speech data on the user terminal 300. However, depending on how the user terminal 300 is used, that is, when the user terminal 300 is used in a location where speech playback is inappropriate, the user terminal 300 may be configured not to play the speech data received by the user terminal 300, or may be configured to exclude the speech data from the broadcast distribution of the first control unit and not to distribute the speech data itself.

[0030] The function of the second control unit is to broadcast the input text and the speech recognition results of the spoken speech data. All speech input to the user terminal 500 and speech played back on the user terminal 500 are converted into text and stored in chronological order in the communication history 123, and are controlled so that they are displayed synchronously on each user terminal 500. The speech recognition unit 113 performs speech recognition processing using the speech recognition dictionary 124, and outputs the speech recognition results in text format. Note that known techniques can be applied to the speech recognition processing.

[0031] The speech synthesis unit 114 uses the speech synthesis dictionary 125 to synthesize speech data corresponding to the characters of the input text data, thereby generating synthetic speech data. At this time, pre-registered voices may be used as the material for the speech data that constitutes the synthetic speech data. For example, sound material of the voices of a task requester, a group leader, etc. may be pre-registered and used to generate the synthetic speech data.

[0032] Next, the information identification tag of this embodiment will be described. As shown in Fig. 4, the information identification tag master information 125 is arbitrary tag information set for each communication group. The information identification tag is tag information assigned to each communication log recorded as the communication history 123, and is applied to both communication logs entered as text and communication logs based on spoken voice.

[0033] An information identification tag is tag information for grouping each communication log based on, for example, the meaning or intent of the content of text input or spoken voice. A communication log is a collection of many logs that are sequential in chronological order and accumulate over time. For this reason, it is difficult to check whether there are any missed contacts or whether there are any responses to contacts among the many communication logs. Therefore, by assigning an information identification tag corresponding to each communication group to each communication log (each text input and each utterance) within the communication group, it becomes easy to check whether there are any missed contacts or whether there are any responses to contacts.

[0034] This embodiment provides a review confirmation function using information identification tags, reduces communication errors caused by the person requesting a task failing to request a task and the person receiving the request failing to accept the task in a group call environment in which many users participate, and supports mutual recognition in information sharing and communication. Information identification tags are registered in the management device 100 through a tag registration and editing function provided by the information identification tag setting unit 115.

[0035] FIG. 6 shows an example of a communication screen by text input and an example of a filtering condition setting screen for a watch list.

[0036] As an example, when transmitting information, an information identification tag can be assigned to match the information to be input. In the example of FIG. 6, when contact is made by text input, "User 3, please clean room 301" is input into the text input field of the second display area W2 in the user terminal 300. At this time, by performing a predetermined operation that is set automatically or arbitrarily, an information identification tag list R is displayed, and one (or more) information identification tags are selected from the multiple information identification tags in the list R. This allows the information to be assigned an information identification tag to be transmitted and distributed to each user terminal 500. The communication log accumulated in the communication history 123 is linked to the selected information identification tag. In the example of FIG. 6, the communication log "User 3, please clean room 301" is linked to and displayed with the information identification tag "Request."

[0037] 7, information identification tags (semantic tags) can be registered through a predetermined addition / edit screen Ra. The information identification tag addition / edit screen Ra can be displayed in conjunction with the communication screen by selecting "Add / Edit Tag" in the information identification tag list R when inputting information to be transmitted. Furthermore, the information identification tag addition / edit screen Ra can be displayed on the user terminal 300 as a master registration screen (not shown) independent of the communication screen, and the addition / edit content of the information identification tag input through this screen can be registered in the management device 100. These processes are performed by the information identification tag setting unit 115, but the screen control may be configured to be performed by the communication control unit of the user terminal 300.

[0038] Returning to Figure 6, the watchlist WR is provided in the left area of ​​the communication screen. The watchlist WR is controlled so that it can switch between the watchlist setting screen WR1 shown in Figure 6, where filtering conditions are set, and the watchlist screen WR2 shown in Figure 5. For example, by clicking "∨" or "∧" at the top of the watchlist WR, you can switch between the two screens.

[0039] The watch list setting screen WR1 allows you to select the speech information (speech, user name, tag) to be filtered. If you select "tag" as the filtering target, you can select each information identification tag linked to the communication group to which you belong. In the example of Figure 6, the filtering target "tag" is selected, and four information identification tags, "request," "question," "approval," and "rejection," are selected.

[0040] On the other hand, for the filtering target "utterance", any keyword is set, and the corresponding communication log is extracted using the set keyword as a filtering condition (search condition). Also, for the filtering target "user", a user name in a communication group is set as a keyword. As with the filtering target "utterance", the user name can be accepted as an arbitrary keyword input, or a user list based on the user information 121 can be displayed for selection.

[0041] After the filtering conditions have been set, switching to the watch list screen WR2 will display a list of communication logs that match the filtering conditions. In the example of Figure 5, communication logs that include any of the filtering target "tags" - "request," "question," "approval," and "rejection" - are picked up. In other words, the picked information from communication logs filtered by information identification tags is displayed on the watch list screen WR2.

[0042] In this embodiment, two or more information identification tags can be selected (assigned) to one group utterance (one communication log). For example, a message such as "User 2, can you clean the front desk for me?" includes a question to a worker, so two information identification tags, "request" and "question," can be selected.

[0043] Fig. 8 is a diagram showing an example of the display of a watch list WR on a communication screen. The example in Fig. 8 is a display example of a watch list that includes the "Request" tag as a filtering condition and the keyword "User 3" in the communication log. In this way, you can arbitrarily combine filtering targets to look back and check the desired communication log.

[0044] Fig. 9 is a diagram showing an example of adding a completion tag to a communication history and an example of updating the tag, Fig. 10 is a diagram showing an example of non-display control based on the completion tag.

[0045] First, an information identification tag can be newly assigned to a communication log at any time after the information is sent, or an information identification tag can be added / edited to a communication log to which an information identification tag has already been assigned.

[0046] 9, when each communication log displayed on the watch list screen WR2 is selected, the communication App control unit 320 displays a selection pop-up of "Complete" and "Update Tag". When "Complete" is selected, the management device 100 transmits the assignment information of the "Complete" tag to the corresponding communication log to the management device 100, and the management device 100 (communication control unit 112) updates the communication history 123. Based on the update process of the communication history 123, the management device 100 notifies each user terminal 300, 500 that the "Complete" tag has been assigned to the corresponding communication log, or the updated communication log with the "Complete" tag assigned is shared with all users.

[0047] When the user terminals 300, 500 receive from the management device 100 that a "Completed" tag has been added to the corresponding communication log, they perform control to display the "Completed" tag on the corresponding communication log displayed in the first display area W1 and the watch list screen WR2. Furthermore, when an updated communication log with a "Completed" tag added is received, they perform control to update and display the corresponding communication log itself displayed in the first display area W1 and the watch list screen WR2 as is.

[0048] Similarly, in the example of Fig. 9, when "Update Tag" is selected in the selection pop-up, the communication app control unit 320 causes the information identification tag list R shown in Fig. 6 to be displayed on the user terminal 300. At this time, the information identification tag list R can be configured to include information identification tags that have already been assigned.

[0049] 9, the user selects a new information identification tag or cancels the assignment of an already assigned information identification tag in the information identification tag list R. The communication App control unit 320 transmits information about adding / editing the information identification tag for the corresponding communication log to the management device 100, and the management device 100 (communication control unit 112) updates the communication history 123. Based on the update process of the communication history 123, the management device 100 broadcasts, to each user terminal 300, 500, tag update information indicating that the information identification tag for the corresponding communication log has been updated, or a communication log including the updated information identification tag.

[0050] When the user terminals 300, 500 receive tag update information for the corresponding communication log from the management device 100, they update the tag display of the corresponding communication displayed in the first display area W1 and the watch list screen WR2 based on the assigned tag information included in the tag update information. Furthermore, when they receive a communication log including an updated information identification tag, they control the corresponding communication log itself displayed in the first display area W1 and the watch list screen WR2 to be updated and displayed as is.

[0051] Next, the "Completed" tag is used for hide control on the watch list screen WR2. As shown in Fig. 10, the communication App control unit 320 can perform hide control so that communication logs tagged with "Completed" among the communication logs extracted based on the filtering conditions are not displayed on the watch list screen WR2. By configuring in this way, it is possible to organize information within a communication group by dividing it into completed and incomplete.

[0052] Note that the watch list setting screen WR1 can also be controlled so that only communication logs tagged with "Complete" are displayed on the watch list screen WR2. Also, the watch list screen WR2 can be provided with a display / hide button for the "Complete" tag, and the display / hide switching operation can be performed using the display / hide button, so that the logs can be displayed without setting filtering conditions. On the other hand, in the first display area W1, application of control to hide communication logs tagged with "Complete" is optional, but since the first display area W1 is meant to be a display area for looking back and viewing past communication logs, it is not necessary to apply hide control based on the "Complete" tag in the first display area W1.

[0053] Fig. 11 is a diagram showing an example of a communication screen on a mobile communication terminal, a user terminal 500. Fig. 12 is a diagram showing a watchlist setting screen WR11 and a watchlist screen WR22 of the watchlist on the user terminal 500.

[0054] The user terminal 500 is a mobile terminal designed for portability and therefore has a small screen. Therefore, a communication screen with a divided watch list can be applied, as shown in Figures 11 and 12. The example in Figure 11 shows a communication screen made up of a first display area W1 in which a communication log is displayed and a second display area W2 including a text input field. The operation of assigning information identification tags in text input is basically the same as the example in Figure 6, and an information identification tag list R is displayed and the user is prompted to select one.

[0055] When a predetermined screen switching operation is performed on the communication screen of Fig. 11, a watch list setting screen WR11 and a watch list screen WR22 are displayed on the user terminal 500. These display controls are performed by the communication App control unit 520. When the work requester uses the user terminal 500, the various functions shown in Figs. 5 to 10 can be used through the screens exemplified in Figs. 11 and 12.

[0056] FIG. 13 is a diagram showing a processing flow of the communication system.

[0057] Each user, who is a worker, starts the communication app control unit 520 on the user terminal 500, and the communication app control unit 520 performs connection processing with the management device 100. Then, the user logs in to the management device 100 by entering their own user ID and password on a predetermined login screen. The login authentication processing is performed by the user management unit 111. After the first login, the user can omit the operation of entering the user ID and password, and the login processing can be automatically performed using the user ID and password entered at the time of the first login when the communication app control unit 520 is started.

[0058] After logging in, the management device 100 automatically establishes a communication channel in group call mode for each of the multiple user terminals 500, opening a group call channel centered around the management device 100. After logging in, each user terminal 500 performs information acquisition processing with the management device 100 at any timing or at predetermined time intervals.

[0059] Meanwhile, the work requester also starts the communication App control unit 320, which then performs connection processing with the management device 100, a similar login authentication process is performed, and a communication screen is displayed on the user terminal 300 based on the authentication result.

[0060] The work requester inputs the text of the request (transmission information) on the communication screen. At this time, the work requester selects any information identification tag (one or more) from the information identification tag list R according to the content of the text input information (S301, S302). The communication App control unit 320 uploads the transmission information including the information identification tag selection information to the management device 100 (S303).

[0061] When the management device 100 receives communication information including the selection information of the information identification tag, it records this in the communication history 123 and accumulates it together with time information (S101). Then, it generates synthetic voice data of the received communication information (S102) and broadcasts the generated synthetic voice data and text data of the communication information to each user terminal 500. At this time, the management device 100 also transmits the text data to the work requester as a display synchronization of the log accumulated in the communication history 123 (S103).

[0062] The communication App control unit 520 of each user terminal 500 performs automatic playback processing of the received speech voice data and outputs the speech voice (S501a, S501b, S501c), and all user terminals 500 display the speech content in text format corresponding to the output voice voice (S502a, S502b, S502c).

[0063] Next, when user A speaks, the communication App control unit 520 collects the spoken voice and transmits the spoken voice data to the management device 100 (S503a). The voice recognition unit 113 of the management device 100 performs voice recognition processing on the received spoken voice data and outputs the voice recognition result of the spoken content. The communication control unit 112 stores the voice recognition result in the communication history 123 and stores the spoken voice data in the storage device 120 (S104).

[0064] The communication control unit 112 broadcasts the speech data of user A to each of the other user terminals 500 other than the user A who spoke. In addition, the communication control unit 112 transmits the speech content (text) of user A stored in the communication history 123 to each user terminal 500 in the communication group including user A himself / herself for display synchronization (S105).

[0065] The communication App control unit 520 of each user terminal 500 other than user A performs automatic playback processing of the received speech voice data and outputs the speech voice (S503b, S503c), and all user terminals 500 including user A display the speech content in text format corresponding to the speech voice output (S505a, S504b, S504c).

[0066] The user terminal 300 also receives the text-formatted speech content corresponding to the speech of user A and displays it on the communication screen. At this time, if the user terminal 300 is set to output the speech data as audio, the communication app control unit 320 may be configured to play the audio, just like the user terminal 500.

[0067] Although the embodiment has been described with reference to an example in which the speaker's own voice data is not transmitted to the user terminal 500, it is also possible to configure the system so that the voice data is uniformly distributed to all users, and the user terminals 300, 500 control whether or not to play the voice data. In other words, the management device 100 uniformly distributes voice data to all users in the communication group, and the communication App control units 320, 520 can control the system so that if the voice data is the speaker's own, the voice data is not played back, based on information identifying the speaker contained in the voice data. This configuration allows the management device 100 to reduce its processing load.

[0068] FIG. 14 is a diagram showing a processing flow relating to tagging operations, setting of filtering conditions, and display of a watch list in this embodiment.

[0069] 14, on the user terminal 300, the work requester can perform a tagging operation for each utterance (communication log) as shown in Fig. 9 (S310). The user terminal 300 transmits addition / edit information of the information identification tag for the corresponding communication log to the management device 100.

[0070] The management device 100 updates the communication history 123 (S110). Based on the update process of the communication history 123, the management device 100 broadcasts to each of the user terminals 300, 500 tag update information indicating that the information identification tag of the corresponding communication log has been updated, or a communication log including the updated information identification tag.

[0071] When the user terminal 300, 500 receives tag update information for the relevant communication log from the management device 100, it updates the tag display of the relevant communication displayed in the first display area W1 and the watch list screen WR2 based on the assigned tag information contained in the tag update information (S510a, S510b, S510c).

[0072] In the user terminal 300, the work requester sets or inputs filtering conditions on the watch list setting screen WR1. The user terminal 300 transmits the set filtering conditions to the management device 300, and the communication control unit 112 extracts communication logs that match the filtering conditions (S112). The communication control unit 112 transmits the extracted communication logs and displays them on the watch list screen WR2 (S113). The user terminal 300 displays the received communication logs on the watch list screen WR2 (S312).

[0073] As described above, the display control of the watch list screen WR2 is performed by the communication control unit 112 on the management device 100 side providing a search function to provide and display communication logs that meet the filtering conditions, but other methods may also be used. For example, the user terminal 300 (500) is provided with a synchronous communication history 123, and the communication logs can be stored as text data in the storage unit 360 (560). Therefore, the communication App control unit 320 (520) can provide a search function similar to that of the communication control unit 112, extract communication logs that meet the filtering conditions from the storage unit 360 (560), and control the display of the communication logs on the watch list screen WR2. Of course, for communication logs including information identification tags, the communication logs accumulated and managed by the management device 100 are regarded as the original, and the display control on the watch list screen WR2 can be performed for the latest communication logs received from the management device 100.

[0074] Similarly, in the user terminal 500, the operator sets or inputs filtering conditions on the watch list setting screen WR11 (S511a). The user terminal 500 transmits the set filtering conditions to the management device 300, and the communication control unit 112 extracts communication logs that match the filtering conditions (SS114). The communication control unit 112 transmits the extracted communication logs and displays them on the watch list screen WR22 (S115). The user terminal 500 displays the received communication logs on the watch list screen WR22 (S512a).

[0075] In this embodiment, communication logs are organized using information identification tags, and necessary communication logs can be grouped and displayed as a list. Therefore, in a group call in which multiple users participate, it is possible to look back on past communications using information identification tags.

[0076] That is, in cases where a requester issues instructions to multiple people in communication such as work instructions, if the requester wants to check whether the instructions were properly conveyed to the requested person, they can look back and check the large number of conversation threads (communication logs). Also, since there are cases where the requested person does not notice that their request is buried in the large number of conversation threads, if they temporarily leave the group call for a break or something, they can look back and check the large number of conversation threads.

[0077] This makes it possible to check for miscommunication and dialogue in the transmission of information, reducing omissions and oversights, and supporting mutual understanding in information sharing and communication.

[0078] The timing of the above-mentioned review can vary, and it can be used at various times, such as reviewing the day after completing the day's work, or as a real-time review such as confirming completion of work on a request during work.

[0079] As described above, if an information identification tag is set for a name, the user (worker) can select "Requests" and "User name (your name)" as the items to be displayed in the watch list on the user terminal 500 used by the user, and thus only requests made to the user can be displayed. This can be used as a real-time review to see if there are any requests made to the user that have not yet been addressed.

[0080] Furthermore, if the communication log showing a request does not include the name of the requestee (the person in charge of the work), it will be unclear who the request is directed to, but if the person in charge of receiving the request includes their own name in their speech to make it clear that they are in charge, the work for the request will not be left unattended within the group. In this way, using information identification tags makes it possible to proceed with work smoothly.

[0081] In this embodiment, all filtering conditions that can be set on the tag watch list setting screen WR1 can also be considered as information identification tags. In other words, in addition to assigning pre-registered information identification tags to speech logs, the user name and any keywords contained in the speech content themselves can also serve as information identification tags. Therefore, the user name and any keywords are tag information that can be used to distinguish and group speeches directed to oneself and to whom from multiple chronologically consecutive communication logs. By combining them with information identification tags, retrospective confirmation can be realized, making it possible to more smoothly share information and mutual recognition in communication.

[0082] (Second embodiment) 15 and 16 are diagrams showing a second embodiment. The second embodiment has a function for automating the process of assigning information identification tags. Note that the same components as those in the first embodiment are given the same reference numerals and their description will be omitted.

[0083] FIG. 15 is a functional block diagram of a communication management device 100 according to this embodiment, which includes a tagging control unit 116 and tagging control information 127 in addition to the first embodiment.

[0084] The tagging control unit 116 performs processing for assigning information identification tags based on tagging control information to each communication log such as text-entered transmission information and uttered voice.

[0085] Fig. 16 is a diagram showing a processing flow of automatic information identification tag assignment control. As shown in Fig. 16, if the received transmission information is voice data (YES in S1001), the management device 100 performs voice recognition processing and outputs the voice recognition result (S1002). If the received transmission information is text information, the management device 100 performs analysis processing as is (S1003). Details of the analysis processing will be described later.

[0086] The management device 100 selects an information identification tag based on the analysis processing result and automatically assigns the information identification tag to the received communication information (S1004). Then, the management device 100 stores a communication log including the information identification tag in the communication history 123 (S1005).

[0087] For display synchronization, the received transmission information (including the automatically assigned information identification tag) is subjected to text broadcasting processing, and synthetic voice data or spoken voice data based on the transmission information is subjected to broadcasting processing (S1006, S1007).

[0088] The analysis process will now be described. There are various types of analysis processes, and known techniques can be used. Representative examples of analysis processes are shown below.

[0089] The first analysis process analyzes the content of the text-format communication information (including the speech recognition results obtained by speech recognition processing of the speech data) and assigns an information identification tag according to the content of the speech. For example, as shown in FIG. 16, the utterance "User 3, please clean room 301" includes the sentence "please." The tagging control unit 116 extracts "please" from the content of the utterance as a sentence expressing the linguistic intention. The tagging control information 127 pre-stores exemplary sentences expressing the linguistic intention, along with tag definition information defining the intention of the sentence. For example, for the intended word "please," tags such as request, favor, offer, and request can be defined. Furthermore, this tag definition information can include additional information such as the frequency of use of each tag. For example, a "request" tag can be assigned to "please" and a "offer" tag can be assigned to "please."

[0090] For example, if a request is not addressed for a while and the requester issues another request, a tag with a higher priority may be assigned. For example, this can be achieved by defining a tag that includes a priority element such as "urgent."

[0091] The second analysis process analyzes the emotion (tone of voice) of the utterance and assigns an information identification tag based on the emotion. For example, when someone says "Please confirm," the listener's impression will change depending on whether the person says it in a polite and warm manner or in a blunt and cold manner. Therefore, the emotions of the work requester, such as "polite" or "blunt," can be registered in the tagging control information 127, and tag information can be defined for each emotion.

[0092] The third analysis process analyzes the speaker's speech tendencies in advance, estimates the tendency of the speech content, and extracts and assigns tag information that matches that tendency. For example, as an example of trend analysis, if a worker receives a request from a work requester and usually responds within 5 seconds, based on this speech tendency, if a request is made and the worker does not respond for 10 seconds or more, an "incomplete" tag is assigned, thereby alerting the worker. In this case, the response time, which is the speaker's tendency, is registered in the tagging control information 127, and tag information can be defined for that tendency to assign tags.

[0093] In the fourth analysis process, when an image is attached to an utterance, the image is analyzed and an information identification tag is assigned to the utterance. For example, if a job requester requests cleaning of a room and the worker sends back a photo of the room after cleaning, the quality of the cleaning is analyzed from the photo and the quality is assigned as an information identification tag. In this case, the work quality can be registered in the tagging control information 127 and tag information can be defined for that quality, allowing tags to be assigned.

[0094] Note that the tags assigned based on the results of each of the above-described analysis processes can also be configured to work in conjunction with the information identification tag master information 126 set through the information identification tag setting unit 115. That is, in this embodiment, even in an aspect that does not have the manual tag assignment processing function provided by the information identification tag setting unit 115 and the information identification tag master information 126 of the first embodiment, information identification tags can be automatically assigned, but the name of the information identification tag to be assigned can be set by the user, and analysis of the utterance content and tag assignment based on the analysis results can be automated. For example, in the first analysis process, for the intended language "please," tags such as request, wish, proposal, and favor are defined in the tagging control information 127, but since the "request" tag is registered in the information identification tag master information 126, control can be exercised to assign the common "request" tag instead of the "request" tag.

[0095] Furthermore, in terms of linking with the information identification tag master information 126, tagging control information 127 may be generated by associating each information identification tag registered in the information identification tag master information 126 with an intended language such as "please." In other words, the tag information registered in the information identification tag master information 126 and the tagging control information 127 can be treated as registered information identification tags that can be added to a communication log.

[0096] According to this embodiment, an information identification tag can be automatically assigned to a communication log uttered by a worker using the user terminal 500, eliminating the need for the worker to assign an information identification tag or for the work requester to assign an information identification tag. In other words, as described above, there are cases where the worker listens and speaks hands-free while performing work, and is unable to perform the operation to assign an information identification tag. However, by automating the tag assignment process, it is possible to quickly assign an information identification tag to a communication log without interfering with the work.

[0097] Furthermore, through the analysis process, information identification tags are automatically generated and assigned according to the linguistic intent, emotion, speech tendency, etc., so that information identification tags suitable for the content of the speech can be easily assigned.

[0098] (Third embodiment) 17 and 18 are diagrams showing a third embodiment. The third embodiment is a system configuration equipped with an assist function by an agent using an information identification tag. Note that the same components as those in the first and second embodiments are given the same reference numerals and their description will be omitted. Also, the example in FIG. 17 is a configuration in which the assist function of this embodiment is applied to the system configuration of FIG. 15 shown in the second embodiment, but the assist function of this embodiment may also be applied to the system configuration of FIG. 2 shown in the first embodiment.

[0099] FIG. 17 is a functional block diagram of a communication management device 100 according to this embodiment, which differs from the first and second embodiments in that it includes an agent control unit 117 and agent control information 128.

[0100] 18 is a diagram showing a processing flow relating to the assist function by an agent. The agent of this embodiment searches for a communication log that matches the judgment conditions set in the agent control information 128, and provides an assist function that contacts users in a communication group based on the presence or absence of a communication log that matches the judgment conditions (judgment result).

[0101] As shown in FIG. 18, for example, the agent control information includes an accommodation schedule and a judgment condition. An example of the accommodation schedule is information such as "Room 301, Check-in at 17:00." The accommodation schedule can be received from a separate accommodation schedule management system or uploaded to this system by an administrator. An example of the judgment condition is "whether or not there is a cleaning request and user action in the communication log for the accommodation schedule."

[0102] Therefore, the determination process in this case is to check whether "the cleaning request and user action for Room 301 at Check-In at 17:00 are in the communication log" (S2001).

[0103] For example, in step S2001, the agent control unit 117 searches for communication logs with the keyword "Room 301" at 16:00, and if it is determined that there are no communication logs with the "request" tag and the "approval" tag, it outputs a judgment result 1 of "contact required." Also, if there are communication logs with the keyword "Room 301" at 16:00 and there are communication logs with the "request" tag but there are no communication logs with the "approval" tag, it outputs a judgment result 2 of "contact required." Furthermore, if there are communication logs with the keyword "Room 301" at 16:00 and there are communication logs with both the "request" tag and the "approval" tag, it outputs a judgment result 3 of "no contact required."

[0104] If it is determined that "contact is required" based on the determination result of step S2001 (YES in S2002), the agent control unit 117 generates a contact text (S2003). The contact text is a preset template, and the contact text is generated by extracting specific keywords such as "Room 301" and "cleaning" from the accommodation schedule and determination conditions and incorporating them into the template.

[0105] Furthermore, contact information can be set in advance in the agent control information 128. For example, in the user information 121 of Fig. 4, the group leader and the work requester (caller) can be registered as contact information. At this time, the voice notification can also be set as "necessary" or "unnecessary."

[0106] The agent control unit 117 determines whether or not a voice notification is necessary, and if it is determined that a voice notification is necessary (YES in S2004), it generates voice synthesis data based on the generated contact text through the voice synthesis unit 114 (S2005), and delivers the synthesized voice data to the corresponding contact (S2006).

[0107] Processing based on the above judgment conditions can be handled by using speech rules that clearly include information about the corresponding location, such as "Clean room 301 accepted" or "Cleaning room 301 completed." If speech rules are not set up, and the corresponding location is not clearly stated because the speech is not made according to the rules, similar assistance can be provided by processing such as the following:

[0108] The agent control unit 117 checks the response log within a predetermined time from the time of the request utterance, and if there is an utterance log of "approved" or "cleaning completed", it determines that "contact is required" and generates a contact text to confirm whether the corresponding utterance log is a response to the cleaning request for "Room 301". It also generates synthesized voice data based on the generated contact text through the voice synthesis unit 114. The generated contact text data and synthesized voice data are delivered to the contact (in this case, the worker).

[0109] Furthermore, the agent control unit 117 distributes the message text as agent utterance text regardless of whether synthetic voice data is distributed. In this case, the distribution destination of the agent utterance text may be the entire communication group, or only the distribution destinations of the synthetic voice data described above. When distributed to the entire communication group, the agent utterance text is stored in the communication history 123, and distribution control is performed to synchronize the display for all users belonging to the group. On the other hand, when sending agent utterance text only to the distribution destinations of the synthetic voice data, the text is also stored in the communication history 123, but distribution control can be performed to synchronize the display for only users who are the distribution destinations of the synthetic voice data.

[0110] According to this embodiment, using information identification tags as a starting point, it is possible to easily select information that requires assistance, and to provide an agent-assisted function with a simple configuration. In other words, extracting information that meets the judgment criteria from a large number of chronologically consecutive communication logs and checking the content places a heavy processing load on the system and reduces processing speed. However, by using information identification tags, content checking can be performed quickly and easily. This makes it possible to realize a smooth agent-assisted function.

[0111] The present embodiment has been described above, but control of the communication screen displayed on the user terminal 300, 500 can be realized in two ways. For example, the user terminal 300, 500 can be equipped with a screen control application such as a browser, in which case the management device 100 (communication control unit 112) can provide each screen information to the user terminal 300, 500 in response to user operations and control the screen. Alternatively, the screen information can be downloaded to the user terminal 300, 500 in advance, and the management device 100 (communication control unit 112) can provide various information displayed on each screen each time in response to user operations.

[0112] Another example of application beyond the hotel room management example mentioned above is building management in the security industry. In this case, the information source is a central center that monitors the status of each building, the control center staff in each building are the work requesters, and the workers are security personnel. Another example is berth management in the logistics industry. In this case, the information source is the warehouse manager (who monitors the availability of the entire warehouse), the berth manager in charge is the requester, and the driver is the worker. Applying this system can reduce baggage waiting time and improve the efficiency of loading and unloading operations.

[0113] Each function constituting the above-mentioned communication management device 100 and user terminals 300, 500 can be realized by a program, and a computer program prepared in advance to realize each function is stored in an auxiliary memory device, and a control unit such as a CPU reads the program stored in the auxiliary memory device into a main memory device, and the control unit executes the program read into the main memory device, thereby operating the functions of each unit.

[0114] Furthermore, the functions of the speech recognition unit 113 and speech synthesis unit 114 may be implemented using an external speech recognition server or speech synthesis server, and the communication management device 100 may not be provided with a direct processing function. That is, the communication management device 100 may be configured to send speech data to an external speech recognition server and receive speech recognition results (text), or to send text to an external speech synthesis server and receive speech synthesis data. The speech recognition and speech synthesis functions may also be provided in the user terminal 300 or the terminal 500. In this case, the worker's speech may be automatically converted into text data and the speech data and the automatically converted text data may be sent to the communication management device 100, or speech may be synthesized based on the received text information to output a command voice.

[0115] The program can also be provided to a computer in a state in which it is recorded on a computer-readable recording medium. Examples of computer-readable recording media include optical discs such as CD-ROMs, phase-change optical discs such as DVD-ROMs, magneto-optical discs such as MOs (Magnet Optical) and MDs (Mini Disks), magnetic discs such as floppy disks and removable hard disks, and memory cards such as CompactFlash (registered trademark), SmartMedia, SD memory cards, and memory sticks. Also included as recording media are hardware devices such as integrated circuits (e.g., IC chips) specially designed and configured for the purposes of the present invention.

[0116] Although the embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. This novel embodiment can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, and are also included in the inventions and their equivalents as set forth in the claims. [Explanation of symbols]

[0117] 100 Communication Management Device 110 Control device 111 User Management Department 112 Communication Control Unit 113 Voice Recognition Unit 114 Speech synthesis unit 115 Information identification tag setting unit 120 Storage device 121 User Information 122 Group Information 123 Communication history information 124 Voice Recognition Dictionary 125 Text-to-Speech Dictionary 126 Information Identification Tag Master Information 130 Communication equipment 300,500 user terminals 310,510 Communications and Telephone Department 320,520 Communication App Control Unit 330,530 Microphone (sound collection part) 340,540 Speaker (audio output section) 350 Display section 360,560 storage section 370 Input operation section 550 Display and input section R Information Identification Tag List Ra Add / Edit screen W1 1st display area W2 2nd display area W3 3rd display area WR1,WR11 watch list setting screen WR2,WR22 Watchlist screen

Claims

1. A communication system in which multiple users communicate with each other through a group call, a communication control unit that performs a first process of broadcasting speech data received from a user terminal to each of a plurality of other user terminals, and a second process of accumulating, in chronological order, a speech recognition result obtained by speech recognition processing of the received speech data and communication information in text format input from the user terminal as a communication log, and performing text distribution so that the communication log is displayed synchronously on each of the user terminals; a storage unit that stores an information identification tag that can be assigned to each of the communication logs, the communication control unit accumulates the communication log in association with the information identification tag, and in the second processing, performs text distribution of the communication log including the information identification tag, searches the accumulated communication log based on filtering conditions including the information identification tag, and provides the searched one or more communication logs to the user terminal; A communication system characterized by further comprising an agent control unit that searches for communication logs that match set judgment conditions and makes predetermined contact with the user terminals within the communication group based on the presence or absence of communication logs that match the judgment conditions.

2. further comprising an information identification tag setting unit that registers the information identification tag; the communication control unit transmits the registered information identification tag to the user terminal; The communication control unit The communication system described in claim 1, characterized in that the text-format communication information is stored as the communication log by being linked to one or more information identification tags selected in conjunction with the input of the text-format communication information at the user terminal.

3. a communication screen is displayed on the user terminal, the communication screen including a first display area in which the communication logs are displayed continuously in chronological order, a second display area including an input field for the text-formatted transmission information, and a third display area in which a watch list setting screen in which the filtering conditions are input and a watch list screen in which one or more communication logs searched based on the filtering conditions are displayed alternately; the watch list setting screen accepts selection of the information identification tag and input of any keyword included in the communication log; 3. The communication system according to claim 1, wherein the filtering conditions include selected information of the information identification tag and the arbitrary keyword.

4. further comprising an information identification tag setting unit that registers the information identification tag; a tag addition / edit screen for the information identification tag is displayed on the user terminal; A communication system as described in any one of claims 1 to 3, characterized in that the information identification tag setting unit accepts the information identification tag added or edited through the tag addition / editing screen from the user terminal and performs registration processing.

5. The communication control unit receives update information of the information identification tag for the accumulated communication log from the user terminal, and performs the second processing on the communication log after the information identification tag has been updated in conjunction with the update processing of the information identification tag linked to the corresponding communication log.A communication system as described in any one of claims 1 to 4, characterized in that

6. the information identification tag includes non-display setting information, A communication system described in any one of claims 1 to 5, characterized in that the hidden setting information is used to control the display of one or more communication logs searched based on the filtering conditions on the user terminal.

7. The communication system described in any one of claims 1 to 6, further comprising a tagging control unit that analyzes the content of the speech recognition result and the text-format communication information input from the user terminal, and automatically assigns the information identification tag based on the analysis result.

8. A program executed by a computer that controls communication among multiple users through a group call, a first function that performs a first process of broadcasting speech data received from a user terminal to each of a plurality of other user terminals, and a second process of accumulating, in chronological order, a speech recognition result obtained by speech recognition processing of the received speech data and communication information in text format input from the user terminal as a communication log, and performing text distribution so that the communication log is displayed synchronously on each of the user terminals; a second function of storing an information identification tag that can be assigned to each of the communication logs; the first function stores the communication log in association with the information identification tag, and in the second processing, performs text distribution of the communication log including the information identification tag, searches the stored communication log based on filtering conditions including the information identification tag, and provides the searched one or more communication logs to the user terminal; A program characterized by searching for communication logs that match set judgment conditions, and further realizing a third function of making specified contact with the user terminals within the communication group based on whether or not there are communication logs that match the judgment conditions.

9. A communication system including a communication management device that controls communication in which multiple users communicate with each other through a group call in which the users participate, and a user terminal operated by each of the users, the communication management device, a communication control unit that performs a first process of broadcasting speech data received from the user terminal to each of a plurality of other user terminals, and a second process of accumulating, in chronological order, a speech recognition result obtained by speech recognition processing of the received speech data and communication information in text format input from the user terminal as a communication log, and performing text distribution so that the communication log is displayed synchronously on each of the user terminals; a storage unit that stores an information identification tag that can be assigned to each of the communication logs; an agent control unit that searches for the communication log that matches a set judgment condition, and makes a predetermined contact with the user terminal within the communication group based on the presence or absence of the communication log that matches the judgment condition, The user terminal a communication App control unit that controls input of the text-format communication information and selection of the information identification tag through a predetermined communication screen; the communication control unit stores the communication log in association with the information identification tag, and performs text distribution of the communication log including the information identification tag in the second processing; the communication App control unit transmits filtering conditions including the information identification tag input from the communication screen to the communication management device; the communication control unit searches the accumulated communication logs based on the filtering conditions, and provides the searched one or more communication logs to the user terminal; A communication system characterized in that the communication app control unit displays the received searched one or more communication logs on the communication screen as picked-up information filtered by the information identification tag for multiple communication logs that are consecutive in chronological order and for which display synchronization is performed.

Images