Conversation control system, conversation control method, and computer program

Abstract

To provide a conversation control system, a conversation control method, and a computer program that enable presentation of a reaction image with a higher degree of freedom.SOLUTION: A conversation control system includes a control unit that displays an image representing a character string input by a user in a virtual conference room for conversations between the user and another user via a network on a user terminal of the user and on a user terminal of another user participating in the same conference room as the user for a predetermined period of time and then erases the display.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to the technology of communication provided online. [Background technology]

[0002] Conventionally, there are conversations that are conducted by outputting information aloud and conversations that are conducted by displaying information. A conversation control system has been proposed for realizing these conversations using a terminal device. In conversations that are conducted by displaying information, character strings or images that indicate the content of utterances made by participants in the conversation are displayed. For example, the technology disclosed in Patent Document 1 makes it possible to use images called stamps, and by displaying stamps instead of character strings, speakers can make more diverse utterances.

[0003] Thus, until now, services that allow conversations by displaying information have been limited to displaying text information indicating the content of a comment or images (stamps) selected in response to the content of a comment. In recent years, services that display images that show reactions to the content of a comment (hereinafter referred to as "reaction images") have also been proposed. Such reaction images are selected by the user from a plurality of reaction images (icons) prepared in advance in response to a specific comment (which may include stamps), and are displayed in association with the specific comment. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-068063 Summary of the Invention [Problem to be solved by the invention]

[0005] Previously, reaction images available were limited to those pre-prepared by chat services. As a result, the ways in which reactions could be expressed using reaction images were limited, and some users were unable to express the reactions they truly wanted to express.

[0006] The present invention has been made in view of the above circumstances, and provides a technique that enables the presentation of reaction images with a higher degree of freedom. [Means for solving the problem]

[0007] One aspect of the present invention is a conversation control system that includes a control unit that displays an image representing a string of characters entered by a user in a virtual conference room for conversations between a user and other users via a network on the user terminal of the user and on the user terminals of other users participating in the same conference room as the user for a predetermined period of time, and then erases the display.

[0008] One aspect of the present invention is the above-mentioned conversation control system, wherein the control unit displays a first operation object on the user's user terminal to display an image representing the string of characters most recently input by the user.

[0009] One aspect of the present invention is the above-mentioned conversation control system, wherein when the first operation object is operated, the control unit operates as if an instruction had been given to display an image representing the character string most recently input by the user.

[0010] One aspect of the present invention is the above-mentioned conversation control system, wherein when a predetermined operation is performed while the first operation object is displayed, the control unit operates as if an instruction has been given to display an image representing the character string most recently entered by the user.

[0011] One aspect of the present invention is the above-mentioned conversation control system, wherein the control unit displays on the user terminals of multiple users participating in the conference room the number of users who have performed an operation to display an image showing the character string corresponding to the second operation target in proximity to the second operation target, or the number of times the operation was performed.

[0012] One aspect of the present invention is the above-mentioned conversation control system, wherein the control unit displays a second operation object on the user terminals of multiple users participating in the conference room, for displaying an image representing a string of characters previously entered by a user participating in the conference room.

[0013] One aspect of the present invention is a conversation control method in which a computer displays an image representing a string of characters entered by a user in a virtual conference room where the user and other users can converse over a network on the user terminal of the user and on the user terminals of other users participating in the same conference room as the user for a predetermined period of time, and then erases the display.

[0014] One aspect of the present invention is a computer program for causing a computer to function as a conversation control system that includes a control unit that displays an image representing a string of characters entered by a user in a virtual conference room where the user and other users can converse over a network on the user terminal of the user and on the user terminals of other users participating in the same conference room as the user for a predetermined period of time, and then erases the display. [Effects of the Invention]

[0015] The present invention makes it possible to present reaction images with greater freedom. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a schematic block diagram showing the system configuration of a conversation control system 100 of the present invention. [Figure 2]2 is a schematic block diagram showing a specific example of the functional configuration of a user terminal 10. FIG. [Figure 3] 2 is a schematic block diagram showing a specific example of the functional configuration of the conversation control device 20. FIG. [Figure 4] FIG. 10 is a diagram showing a specific example of a reaction history information table. [Figure 5] FIG. 10 is a diagram showing a specific example of an individual reaction history information table. [Figure 6] 10A and 10B are diagrams showing specific examples of images displayed on the display unit 13 of the user terminal 10. FIG. [Figure 7] 10A and 10B are diagrams showing specific examples of images displayed on the display unit 13 of the user terminal 10. FIG. [Figure 8] 10A and 10B are diagrams showing specific examples of images displayed on the display unit 13 of the user terminal 10. FIG. [Figure 9] 10A and 10B are diagrams showing specific examples of images displayed on the display unit 13 of the user terminal 10. FIG. [Figure 10] 10A and 10B are diagrams showing specific examples of images displayed on the display unit 13 of the user terminal 10. FIG. [Figure 11] 10A and 10B are diagrams showing specific examples of images displayed on the display unit 13 of the user terminal 10 when a reaction action is performed. [Figure 12] 10A and 10B are diagrams showing specific examples of images displayed on the display unit 13 of the user terminal 10 when a reaction action is performed. [Figure 13] FIG. 10 is a diagram showing a specific example of a history display area 56. [Figure 14] FIG. 9 is a diagram illustrating an example of the hardware configuration of an information processing device 900 applied to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0017] Specific configuration examples of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic block diagram showing the system configuration of a conversation control system 100 of the present invention. The conversation control system 100 is a system in which users operating user terminals 10 converse with each other via a network 90. The conversation control system 100 may be configured as a system in which users operating terminal devices 10 converse with each other via a network 70 using displays. In a display-based conversation, multiple users can converse by displaying text and images. The conversation control system 100 includes multiple user terminals 10 and a conversation control device 20. The terminal device 10 may acquire speech uttered by the user of each terminal device 10 and output the speech as audio on the terminal devices 10 of other users participating in the conversation. The terminal device 10 may display text and images indicating statements made in a text-based conversation. The multiple user terminals 10 and the conversation control device 20 are communicatively connected via a network 90. The network 90 may be a network using wireless communication or a network using wired communication. The network 90 may be configured by combining multiple networks.

[0018] 2 is a schematic block diagram showing a specific example of the functional configuration of user terminal 10. User terminal 10 is configured using information devices such as a smartphone, tablet, personal computer, portable game console, stationary game console, dedicated device, etc. User terminal 10 includes a communication unit 11, an operation unit 12, a display unit 13, an audio input unit 14, an audio output unit 15, an imaging unit 16, a memory unit 17, and a control unit 18.

[0019] The communication unit 11 is a communication device. The communication unit 11 may be configured as, for example, a network interface. The communication unit 11 communicates data with other devices via a network 90 in accordance with the control of the control unit 18. The communication unit 11 may be a device that performs wireless communication or a device that performs wired communication.

[0020] The operation unit 12 is configured using existing input devices such as a keyboard, a pointing device (such as a mouse or tablet), buttons, and a touch panel. The operation unit 12 is operated by a user when inputting user instructions to the user terminal 10. The operation unit 12 may be an interface for connecting an input device to the user terminal 10. In this case, the operation unit 12 inputs an input signal generated by the input device in response to a user input to the user terminal 10. The operation unit 12 may be configured using a microphone and a voice recognition device. In this case, the operation unit 12 performs voice recognition on words spoken by the user and inputs character string information of the recognition result to the user terminal 10. In this case, the voice recognition process may be performed by the control unit 18. In this case, the microphone constituting the operation unit 12 may be configured integrally with the voice input unit 14 described below. The operation unit 12 may be configured in any way as long as it is capable of inputting user instructions to the user terminal 10.

[0021] The display unit 13 is an image display device such as a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 13 displays image data used in the conversation control system 100. The displayed image data includes character strings, images, etc. The display unit 13 may be an interface for connecting an image display device to the user terminal 10. In this case, the display unit 13 generates a video signal for displaying the image data and outputs the video signal to the image display device connected to the display unit 13.

[0022] The voice input unit 14 is configured using a microphone. The voice input unit 14 may be configured as a microphone itself, or may be configured as an interface for connecting a microphone as an external device to the user terminal 10. The microphone acquires the speech of the user who is having a conversation. The voice input unit 14 outputs data of the voice acquired by the microphone to the control unit 18.

[0023] The audio output unit 15 is configured using an audio output device such as a speaker, headphones, or earphones. The audio output unit 15 may be configured as the audio output device itself, or may be configured as an interface for connecting the audio output device to the user terminal 10 as an external device. It is desirable that the audio output device outputs audio so that the user having a conversation can hear the audio. The audio output unit 15 outputs audio corresponding to the audio signal output by the control unit 18.

[0024] The imaging unit 16 is configured using an imaging device such as a camera. The imaging unit 16 may also be an interface for connecting an imaging device such as a camera to the user terminal 10. The imaging unit 16 is preferably installed in a position where it can capture an image of the user's face. In this case, the imaging unit 16 captures an image of the user's face and generates still image or video data (hereinafter referred to as "captured image data"). The imaging unit 16 outputs the generated captured image data to the control unit 18. The captured image data is one form of provided video. The provided video is a video provided from a user terminal 10 or an information device to another user terminal 10. Specific examples of provided video include a video (captured image data) captured by the imaging unit 16 of the user terminal 10 and a video showing a screen shared on the user terminal 10 (hereinafter referred to as a "shared screen"). The shared screen is, for example, a window selected by the user, a window displaying a selected file, a selected desktop, or the like. The video showing the shared screen may reflect operations performed by the user on the screen at any time. For example, when a user operates the position of a cursor, the display position of the cursor changes on the shared screen in response to the operation.

[0025] The storage unit 17 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 17 stores data used by the control unit 18. The storage unit 17 may function as, for example, a user information storage unit 171. The user information storage unit 171 stores information about the user operating the user terminal 10 (hereinafter referred to as "user information"). The user information may include, for example, user identification information, the user's handle name, user icon data, and user attribute information. The user identification information is identification information for uniquely identifying a user in the conversation service provided by the conversation control system 100. The handle name is the name (nickname or real name) used by the user when having a conversation in the conversation control system 100. The icon data is image data used by the user when having a conversation in the conversation control system 100. The attribute information may include, for example, information about the user's age, gender, and affiliation.

[0026] The control unit 18 is configured using a processor such as a CPU (Central Processing Unit) and a memory. The control unit 18 functions as a display control unit 181, a conference control unit 182, and a conversation control unit 183 by the processor executing a program. All or part of the functions of the control unit 18 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The above program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, and a semiconductor storage device (e.g., a solid-state drive (SSD)), as well as storage devices such as a hard disk or semiconductor storage device built into a computer system. The above program may be transmitted via a telecommunications line.

[0027] The display control unit 181 receives display information from the conversation control device 20 via the communication unit 11. The display control unit 181 generates an image signal based on the acquired display information and displays it on the display unit 13. At this time, the display control unit 181 controls the display within the display area of the application in which the application of the conversation control system 100 is running. For example, if the application of the conversation control system 100 is a web browser, display control is performed within the web browser window. If the web browser is displayed full screen, display control is performed on the entire display screen of the display unit 13.

[0028] The display information may be, for example, image data that indicates the image itself to be displayed. In this case, the entity that generates the image data (image data generation unit) is the display information generation unit 233 of the conversation control device 20. The display information may be, for example, data that indicates information required to generate the image to be displayed (for example, information about the conference room (participants, number of participants, etc.), the position, size, and display mode of the conference room, and information indicating the position and icon image of each user). The display information may be, for example, information that indicates a character string or image that indicates the content of a conversation, or information that indicates an icon image of each user. In this case, the display control unit 181 generates image data to be displayed on the display unit 13 based on the display information. In this case, the entity that generates the image data (image data generation unit) is the display control unit 181.

[0029] The conference control unit 182 controls a conference virtually established in the conversation control device 20. For example, when a user operates the operation unit 12 to instruct the creation of a new conference room, the conference control unit 182 performs processing for creating the new conference room. When the user of the user's own device (user terminal 10) has set up the setting to transmit captured image data to other user terminals 10, the conference control unit 182 transmits captured image data captured by the imaging unit 16 to the other device. The other device serving as the destination may be, for example, the conversation control device 20. In this case, the captured image data is transmitted via the conversation control device 20 to other user terminals 10 connected to the same conference room as the user's own device (user terminal 10). The other device serving as the destination may be other user terminals 10 connected to the same conference room as the user's own device (user terminal 10). When the user of the user's own device (user terminal 10) has set up the setting to transmit a shared screen to other user terminals 10, the conference control unit 182 transmits video data showing the shared screen of the user's own device to the other device. The other device serving as the destination may be, for example, the conversation control device 20. In this case, the video data of the shared screen is transmitted to the other user terminals 10 connected to the same conference room as the user terminal 10 via the conversation control device 20. The other device to which the data is transmitted may be another user terminal 10 connected to the same conference room as the user terminal 10.

[0030] The conversation control unit 183 controls conversations between multiple users participating in the conversation control system 100. A conversation between multiple users may be, for example, a conversation between multiple users participating in a conference room (hereinafter referred to as a "conference room conversation"). When the user of the user's own device (user terminal 10) is participating in a conversation, the conversation control unit 183 controls the conversation between multiple users including the user of the user's own device. In this case, the conversation control unit 183 transmits, for example, audio data input from the audio input unit 14 to the conversation control device 20 via the communication unit 11. When the conversation control unit 183 receives audio data from the conversation control device 20, it outputs the received audio data from the audio output unit 15. When the user of the user's own device (user terminal 10) is not participating in the conversation, the conversation control unit 183 controls the conversation between multiple users excluding the user of the user's own device. In this case, the conversation control unit 183 receives audio data of a conversation between other users and outputs it from the audio output unit 15. The audio data output from the audio output unit 15 may be received from another user terminal 10 via the conversation control device 20, or may be received directly from another user terminal 10 connected in accordance with the control of the conversation control device 20.

[0031] 3 is a schematic block diagram showing a specific example of the functional configuration of the conversation control device 20. The conversation control device 20 is configured using an information processing device such as a personal computer or a server device. The conversation control device 20 includes a communication unit 21, a storage unit 22, and a control unit 23.

[0032] The communication unit 21 is a communication device. The communication unit 21 may be configured as, for example, a network interface. The communication unit 21 communicates data with other devices via the network 90 in accordance with the control of the control unit 23. The communication unit 21 may be a device that performs wireless communication or a device that performs wired communication.

[0033] The storage unit 22 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 22 stores data used by the control unit 23. The storage unit 22 may function as, for example, a conference room information storage unit 221, a reaction history information storage unit 222, and an individual reaction history information storage unit 223.

[0034] The conference room information storage unit 221 stores information about conference rooms (hereinafter referred to as "conference room information"). A conference room is a virtual room set up in the conversation control system 100 for users to have conversations. The conference room information may include, for example, information indicating the name set for the conference room. The conference room information may further include information indicating the users currently in the conference room.

[0035] The reaction history information storage unit 222 stores a reaction history information table. FIG. 4 is a diagram showing a specific example of the reaction history information table. The reaction history information table has a plurality of reaction history records. The reaction history record has information about reaction actions performed in the past. The reaction history record has, for example, conference room identification information, reaction identification information, reaction content, attribute information, reacter identification information, reaction date and time, consenter identification information, and the number of consenters. The conference room identification information is identification information for uniquely identifying the conference room in which the reaction of the reaction history record was performed. The reaction identification information is identification information for uniquely identifying the reaction of the reaction history record. The reaction content indicates the content of the reaction. The reaction content may, for example, indicate the character string used in the reaction action, or the image (stamp) used in the reaction action. The attribute information indicates the attribute information of the image displayed in the reaction action when a reaction action using a character string is performed. The attribute information may, for example, indicate the color of the image or the font used in the image. The attribute information may further include other information. For example, the attribute information may indicate the font size. The responder identification information is information (e.g., user identification information) for identifying the user who performed the reaction action. The reaction date and time indicates the date and time when the reaction action was performed. The consenter identification information is information (e.g., user identification information) for identifying the user (consenter) who performed an agreement action in response to the reaction action. The number of consenters indicates the number of users (consenters) who performed an agreement action in response to the reaction action.

[0036] The individual reaction history information storage unit 223 stores an individual reaction history information table. FIG. 5 is a diagram showing a specific example of an individual reaction history information table. The individual reaction history information table has multiple individual reaction records. An individual reaction history information table may be provided for each user. That is, an individual reaction history information table may be provided for each user, containing information about reaction actions performed by that user in the past. An individual reaction record contains information about reaction actions performed by that user in the past. An individual reaction record includes, for example, reaction identification information, reaction content, attribute information, and reaction date and time. The reaction identification information is identification information for uniquely identifying the reaction of that individual reaction record. The reaction content indicates the content of the reaction. The attribute information indicates the attribute information of the image shown in the reaction action when a reaction action using a character string is performed. The reaction date and time indicates the date and time when the reaction action was performed. The number of responders indicates the number of users who agreed with the reaction action.

[0037] The control unit 23 is configured using a processor such as a CPU and a memory. The control unit 23 functions as a user control unit 231, a conference room control unit 232, a display information generation unit 233, and a conversation control unit 234 by the processor executing a program. Note that all or part of the functions of the control unit 23 may be realized using hardware such as an ASIC, PLD, or FPGA. The above program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and semiconductor storage devices (e.g., SSDs), as well as storage devices such as hard disks and semiconductor storage devices built into a computer system. The above program may be transmitted via a telecommunications line.

[0038] The user control unit 231 performs control processing related to users. For example, the user control unit 231 may perform processing for logging in (for example, authentication processing) for a user who accesses the conversation control device 20.

[0039] The conference room control unit 232 performs control processing related to conference rooms. For example, when the conference room control unit 232 receives an instruction from the user terminal 10 to set up a new conference room, it may generate conference room information based on the received information and register it in the conference room information storage unit 221. When a predetermined operation to join a conference room is performed by a user, the conference room control unit 232 performs processing to allow the user to join the conference room if predetermined conditions are met. For example, the conference room control unit 232 updates the conference room information storage unit 221 to register that a new user has joined the conference room.

[0040] The display information generation unit 233 generates information (display information) required to generate an image to be displayed on the user terminal 10. The display information may include, for example, information about the currently installed conference room, information about the user of each user terminal 10 accessing the conversation control device 20, and information about a reaction action made in a conference room in which the user of the user terminal 10 is present. The display information generation unit 233 transmits the generated display information to the user terminal 10.

[0041] The conversation control unit 234 receives voice data from the user terminals 10. The conversation control unit 234 generates voice data to be output to each user terminal 10 (hereinafter referred to as "conversation voice data") and transmits the conversation voice data to each user terminal 10. The conversation control unit 234 may generate the conversation voice data based on, for example, what kind of conversation the user of each user terminal 10 is participating in. For example, the conversation control unit 234 may transmit, to each user's user terminal 10, conversation voice data including voices received from the user terminals 10 of all users participating in the conference room that the user is in.

[0042] Next, the relationship between the images displayed on the user terminal 10 and the conversational voice data output when each image is displayed will be described.

[0043] FIG. 6 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. In FIG. 6, the display unit 13 displays a conference room area image 30, a conversation area 31, a listening area 32, an icon image 41, an operation area image 501, an input area 51, and a speech button 52. The display unit 13 displays an image of a virtual space in which a conference room can be set up. The virtual space is mainly composed of a conference room area and an area outside the conference room. The conference room area is displayed as a conference room area image 30. The area outside the conference room (hereinafter referred to as the "outdoor area") is the area represented by the symbol 91, and is an area in which icon images 41 of users who are not in the conference room are displayed.

[0044] The conference room area image 30 includes a conversation area 31 and a listening area 32. A user whose icon image 41 is displayed in the conversation area 31 can converse with other users whose icon image 41 is displayed in the conversation area 31 of the same conference room. In other words, the voices of the users whose icon image 41 is displayed in the conversation area 31 are transmitted and received between them. In the following description, entering a conference room means that the icon image 41 is displayed in the conversation area 31. In other words, entering a conference room means that a user is in a state where they can converse with other users in the room.

[0045] A user whose icon image 41 is displayed in the listening area 32 (hereinafter referred to as a "listening user") can listen to the conversation of a user whose icon image 41 is displayed in the conversation area 31 of the conference room (a user who has entered the conference room). However, the listening user cannot speak in the conference room. In other words, the voices uttered by the listening user are not output to the user terminals 10 of other users who have entered the conference room or to the user terminals 10 of other listening users in the conference room. Regarding displays in response to reaction actions, which will be described later, the listening user can perform a reaction action, and displays related to reaction actions performed by other users who have entered the conference room may be displayed to the listening user. Note that the listening user may be configured not to perform a reaction action, and displays related to reaction actions performed by other users who have entered the conference room may be displayed to the listening user.

[0046] The conference room area image 30 is represented as a geometrically shaped image showing an area of a predetermined size. In the example of this embodiment, the conference room area image 30 is represented as a circular image. The name of the conference room 311 is displayed near the center of the conference room area image 30. In the example of FIG. 6, "Chat" is the name of the conference room. By displaying the name of the conference room, even a user located in the outdoor area 91 can roughly guess what kind of conversation is taking place in the conference room without entering the conference room.

[0047] An enter button 312 and a details button 313 may be displayed inside the conference room area image 30. The enter button 312 is a button for requesting entry into the conference room. In response to the entry button 312 being operated by a user located in the outdoor area 91, the conference room control unit 232 determines that the user is requesting entry into the conference room. In response to the request, the conference room control unit 232 determines whether or not to permit entry, and controls the user's entry in accordance with the determination result.

[0048] The details button 313 is a button for displaying detailed information about the conference room. In response to operating the details button 313, a details display area is displayed on the screen. The details display area displays detailed information about the conference room, as well as buttons and input boxes for performing operations on the conference room. For example, the name of the conference room, a button for changing the name of the conference room, an input box for entering the changed name of the conference room, etc. may be displayed.

[0049] The icon image 41 is an image that indicates the virtual position of the user and information about the user. The icon image 41 may be displayed according to icon data included in the user information, for example. The input area 51 and the comment button 52 will be described later.

[0050] FIG. 7 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. In the example of FIG. 7, icon images 41 (41_1 and 41_2) of two users and a conference room area image 30 of one conference room are displayed. No users have yet participated in the conference room indicated by the conference room area image 30. The two users have not participated in any conference room. Therefore, the icon images 41_1 and 41_2 are displayed in the outdoor area 91, not inside the conference room area image 30.

[0051] A user who wishes to participate in a conference room can do so (enter) the conference room by performing a predetermined operation. For example, a user can join the conference room by operating the entry button 312 in the conference room area image 30 in which the user wishes to participate. Note that the operation for entering the room does not have to be limited to the above. If entry restrictions are set for the conference room, only users who are permitted to enter according to the entry restrictions can enter the conference room.

[0052] FIG. 8 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. In the example of FIG. 8, one user is participating in a conference room with the theme of chatting. Therefore, an icon image 41 (41_2) of one user is displayed inside the conference room area image 30 showing the conference room with the theme of chatting. At this point, there are no other users participating in the same conference room as the user of icon image 41_2. Therefore, the user of icon image 41_2 cannot have a conversation with other users in the conference room with the theme of chatting.

[0053] When a user of icon image 41_1 in the outdoor area 91 performs an operation to join a conference room with a chat theme, the users participating in the conference room are notified that there is another user who wishes to join. For example, a predetermined sound (e.g., a door knock) may be output from the user terminal 10 of each user. The users to be notified may be all users participating in the conference room, or only a predetermined subset of users (e.g., the user who is the administrator of the conference room).

[0054] After the notification, if a predetermined condition is met, the user of icon image 41_1 can participate in the conference room with a chat theme. The predetermined condition may be, for example, simply that a predetermined time (e.g., 1 second, 3 seconds, 5 seconds, etc.) has passed, or that participation has been permitted by a predetermined user (e.g., the conference room administrator, a participant, etc.), or that a participation condition preset for the conference room has been met. Also, a predetermined condition may not be set. In that case, all users who wish to participate can participate in the conference room (e.g., a conference room with a chat theme).

[0055] FIG. 9 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. In the example of FIG. 9, two users are participating in a conference room with the theme of chatting. Therefore, icon images 41 (41_1 and 41_2) of the two users are displayed inside the conference room area image 30 indicating the conference room with the theme of chatting. At this point, the user of the icon image 41_1 and the user of the icon image 41_2 are participating in the same conference room. Therefore, the user of the icon image 41_1 and the user of the icon image 41_2 can have a conversation in the conference room with the theme of chatting. In other words, users participating in the same conference room can have a conversation using the user terminal 10.

[0056] More specifically, the process is as follows: The user terminal 10 of a user participating in a conference room receives and outputs voice data spoken by other users participating in the same conference room. The user terminal 10 of a user participating in a conference room also transmits the voice data spoken by that user to the conversation control device 20. This transmission causes the voice data to be output at the user terminals 10 of the other users participating in the same conference room. When a user represented by icon image 41_3 in the outdoor area 91 performs an operation to participate in a conference room with a chat theme, after certain conditions are met, the user represented by icon image 41_3 can participate in the conference room with a chat theme.

[0057] The user of the icon image 41_1 and the user of the icon image 41_2 can react in a conference room with a chat theme. That is, when one user participating in a conference room makes a reaction, a display according to the reaction is executed on the user terminals 10 of all users participating in the conference room.

[0058] FIG. 10 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10. In the example of FIG. 10, three users are participating in a conference room with the theme of chatting. Therefore, icon images 41 (41_1, 41_2, and 41_3) of the three users are displayed inside the conference room area image 30 indicating the conference room with the theme of chatting. Therefore, the user of the icon image 41_1, the user of the icon image 41_2, and the user of the icon image 41_3 can have a conversation in the conference room with the theme of chatting. The character string "Hello" is input in the input area 51. When the speech button 52 is operated, a reaction action is performed using the character string input in the input area 51 at that time.

[0059] FIG. 11 is a diagram showing a specific example of an image displayed on the display unit 13 of the user terminal 10 when a reaction action is performed. When a character string is input by the user as a reaction action, the display information generation unit 233 generates a reaction image 53 showing the input character string. The reaction action may be performed, for example, by inputting a character string in the input area 51 and operating the speech button 52. The reaction action may also be performed using an image (stamp) prepared in advance in the conversation control system 100. For example, by operating a specific button or area displayed on the screen, a list of multiple available stamps may be displayed, and a stamp (reaction image 53) may be displayed in response to an operation performed on the displayed stamp.

[0060] When a reaction action is performed, a reaction image 53 is displayed on the display unit 13 of the user terminal 10 of each user participating in the conference room where the reaction action was performed. When multiple reaction actions are performed, a reaction image 53 may be displayed for each reaction action. In other words, the reaction image 53 may be displayed the same number of times as the number of reaction actions performed. The conversation control system 100 is capable of performing reaction actions using images showing any character string input by the user, in addition to reaction actions using images (icons) prepared in advance as in the conventional case. In FIG. 11, a reaction action using the character string "Hello" is performed.

[0061] Reaction image 53 may be displayed to move. In the example of FIG. 11 , reaction image 53 is displayed to originate from boundary line 54 and move upward. For example, reaction image 53 may be displayed to originate from region 541 of boundary line 54 and move upward. In this case, reaction image 53 may be displayed to move upward while swaying left and right (moving left and right). The movement of reaction image 53 may be implemented in any manner. Reaction image 53 is displayed for a predetermined time and then disappears. The display time may be the same for each reaction image 53 or may be different. Furthermore, the distance on the screen over which reaction image 53 moves while displayed, the speed at which reaction image 53 moves, the position at which reaction image 53 appears, the direction in which reaction image 53 moves, and the like may be the same for each reaction image 53 or may be different.

[0062] When the reaction image 53 is displayed by operating the speak button 52, a first repeat button 55 (first operation target) is displayed in a part of the input area 51. The first repeat button 55 is a button for instructing a reaction action to the character string previously input using the input area 51. The first repeat button 55 may be displayed using, for example, an image showing the character string previously input using the input area 51, or a reaction image 53 displayed with the reaction action previously instructed. When the first repeat button 55 is operated, the reaction action to the character string previously input using the input area 51 is executed.

[0063] The first repeat button 55 may be configured to be displayed only on the user terminal 10 of the user who inputs a character string in the input area 51 and operates the speak button 52, but not on the user terminals 10 of other users (e.g., other users participating in the same conference room as the user who operated the speak button 52). This configuration can prevent other users from accidentally operating the first repeat button 55 unintentionally. Furthermore, when a user performs a predetermined operation (e.g., pressing the Enter key) while the first repeat button 55 is displayed, the user terminal 10 may treat this as an instruction to respond to the character string input using the input area 51 immediately before, even if no direct operation (e.g., click or tap) was performed on the first repeat button 55 itself. The predetermined operation may be treated as described above only if it is performed consecutively after, for example, inputting a character string in the input area 51 and operating the speak button 52.

[0064] FIG. 12 is a diagram showing a specific example of an image displayed on display unit 13 of user terminal 10 when a reaction action is performed. In the example of FIG. 12, repeat button 55 is operated in the state of FIG. 11. In response to the operation of repeat button 55, reaction image 53a is generated from area 541a and moves upward. Successive reaction images 53 on the time axis may be displayed so as to each be generated from a different area 541. With this configuration, multiple reaction images 53 are displayed scattered across the entire screen, improving visibility and making the entire screen appear more lively.

[0065] FIG. 13 is a diagram showing a specific example of the history display area 56. The history display area 56 displays information indicating the history of reaction actions performed in the past in conference rooms in which the user is participating (hereinafter referred to as "participating conference rooms"). The history display area 56 displays an individual history display area 57 for each reaction action performed in the past in the participating conference rooms. The individual history display area 57 displays information about each reaction action. In the example of FIG. 13, the individual history display area 57 displays a reaction image 571, the number of people who agree 572, and an icon image 573.

[0066] Reaction image 571 shows a reaction image output by the reaction action indicated by the history. Reaction image 571 may function as a second repeat button (second operation target). The second repeat button is a button for instructing a reaction action performed by a user who has entered the participating conference room. When the second repeat button is operated, the reaction action indicated by the history is executed. For example, the second repeat button may be displayed not only on the user terminal 10 of the user who performed the reaction action indicated by the history, but also on the user terminals 10 of other users (for example, other users participating in the same conference room as the user who performed the reaction action). With this configuration, when each user wants to perform the same reaction action as the reaction action performed by another user, they do not need to bother to input the same character string, and can execute the reaction action with a simple operation.

[0067] The number of agreeers 572 indicates the number of users (agreeors) who performed an agreeing action in response to the reaction action. In other words, it is shown that eight users performed an agreeing action in response to the reaction action of "like." The number of agreeers 572 may indicate the total number of users (agreeors) who performed an agreeing action. In other words, the number of agreeing actions may indicate the number of times the agreeing action was performed. In this case, for example, if the same user performs an agreeing action multiple times, the number of reaction actions is counted as the number of times. The number of agreeing actions performed by each user may be displayed in association with each user. For example, the number of times the same action was performed may be displayed below each user's icon or name. The number of agreeers 572 may be displayed in any position as long as it is associated with the reaction image 571 (second operation target). For example, it may be displayed below and adjacent to the reaction image 571 as shown in FIG. 13, or it may be displayed above or to the left or right of the reaction image 571.

[0068] A user who wishes to indicate consent to a reaction action operates the operation unit 12 of their terminal device 10 to perform a predetermined operation on the reaction image 571 of the reaction action to be consented to. For example, the operation may be a touch operation on a touch panel or a click operation with a pointing device such as a mouse. In response to this operation, the conversation control unit 234 searches the reaction history record of the reaction image 571 for which consent has been given, adds the user identification information of the user who indicated consent to the consenter identification information, and increments the number of consenters. The conversation control unit 234 updates the display of the operated reaction image 571 based on the updated reaction history record. Specifically, the conversation control unit 234 updates and displays the number of consenters for the reaction image 571.

[0069] The number of consenters may be configured to represent the number of times the second repeat button 571 has been operated, rather than the number of users who consented to the reaction image 571. In this case, in response to the consent operation, the conversation control unit 234 searches for the reaction history record of the reaction image 571 for which consent has been given. The conversation control unit 234 adds the user identification information of the user who consented to the consenter identification information if it is not registered, and does not register it if it is already registered. The conversation control unit 234 increments the number of consenters by the number of times the operation was performed, regardless of whether the user identification information is registered. The conversation control unit 234 updates the display of the operated reaction image 571 based on the updated reaction history record. Specifically, the conversation control unit 234 updates and displays the number of consenters of the reaction image 571.

[0070] The icon image 573 shows the icon image of the user who performed the reaction action. In the history display area 56, individual history display areas 57 may be displayed in order of the reaction actions.

[0071] The conversation control unit 234 may perform the following processing when generating a reaction image 53 based on an input character string. The conversation control unit 234 may select a specific color from one or more color options to represent the character string in the reaction image 53. When the conversation control unit 234 selects a color, it may select the color according to the number of characters in the input character string. In this case, a correspondence between the number of characters and the color is predetermined. When the conversation control unit 234 selects a color, it may select the color according to the number of bytes in the input character string. In this case, a correspondence between the number of bytes and the color is predetermined. When the conversation control unit 234 selects a color, it may select the color according to the character code of the input character string. In this case, a correspondence between the character code and the color is predetermined. When the conversation control unit 234 selects a color, it may select the color according to a number indicating the order of the reaction action performed in response to the utterance. In this case, a correspondence between the number indicating the order and the color is predetermined. The conversation control unit 234 may use criteria different from those described above for selecting a color. For example, the conversation control unit 234 may select a color randomly.

[0072] The conversation control unit 234 may select a specific font from one or more font options to represent the character string in the reaction image 53. When the conversation control unit 234 selects a font, it may select the font according to the number of characters in the input character string. In this case, the correspondence between the number of characters and the font is predetermined. When the conversation control unit 234 selects a font, it may select the font according to the number of bytes in the input character string. In this case, the correspondence between the number of bytes and the font is predetermined. When the conversation control unit 234 selects a font, it may select the font according to the character code of the input character string. In this case, the correspondence between the character code and the font is predetermined. When the conversation control unit 234 selects a font, it may select the font according to a number indicating the order of the reaction action performed in response to the utterance. In this case, the correspondence between the number indicating the order and the font is predetermined. The conversation control unit 234 may use criteria different from those described above for selecting a font. For example, the conversation control unit 234 may select a font randomly.

[0073] Conversation control unit 234 may insert line breaks into the input character string to generate multi-line character string reaction image 53. The position at which to insert the line break may be determined, for example, as a position of a predetermined number of bytes of the character string, or as a position of a predetermined number of characters.

[0074] The conversation control unit 234 may select from a plurality of candidates the font size of the characters in the reaction image 53. For example, the conversation control unit 234 may select the font size according to the number of lines. More specifically, if the font size used for one line is 1, a smaller font size (for example, 0.7) may be selected for two lines.

[0075] If the length of the input character string or the number of lines of the character string exceeds the maximum value, the conversation control unit 234 may instruct the user to reduce the number of characters without generating a reaction image 53. For example, the maximum value may be set to 10 characters. For example, the maximum number of lines may be set to 2 lines.

[0076] A maximum value may be defined for the number of reaction operations that can be performed on a single utterance. The reaction image 913 may be displayed over multiple lines. When the reaction image 913 is displayed over multiple lines, the vertical width of the utterance display area 91 may be enlarged. When the reaction image 913 spans multiple lines, a line break may be inserted at a portion that satisfies the line break insertion condition. As a specific example of the line break insertion condition, there is a position where the character type (Chinese characters, hiragana, katakana, alphabet, numbers, etc.) changes. For example, when a line break is made in the middle of "Thank you", a line break may be inserted between "謝" and "し", which is the boundary between the Chinese character part and the hiragana part. For example, when a line break is made in the middle of "This is the Patent Office", a line break may be inserted between "庁" and "で", which is the boundary between the Chinese character part and the hiragana part. Thus, the number of characters in the first and second lines does not have to be fixed. For example, when the maximum number of characters in the first line is 10, the 10th and 11th characters are compared to determine whether the above line break insertion condition is satisfied. Until the line break insertion condition is satisfied, each character is moved up one by one (checked like the 10th character, 9th character, 8th character...), and a line break is inserted at the portion where the line break insertion condition is satisfied. The same applies to the lines after the second line. Other conditions may be defined as the line break insertion condition. For example, a condition may be set such that a line break is inserted between words.

[0077] FIG. 14 is a diagram illustrating an example of the hardware configuration of an information processing device 900 applied to this embodiment. The information processing device 900 includes a processor 901, a main memory device 902, a communication interface 903, an auxiliary memory device 904, an input / output interface 905, and an internal bus 906. The processor 901, the main memory device 902, the communication interface 903, the auxiliary memory device 904, and the input / output interface 905 are communicably connected to each other via the internal bus 906. The information processing device 900 may be applied to, for example, a user terminal 10 and a conversation control device 20. In this case, for example, the communication units 11 and 21 may be configured using the communication interface 903. For example, the memory units 17 and 22 may be configured using the auxiliary memory device 904. Furthermore, the control units 18 and 23 may be configured using the processor 901 and the main memory device 902.

[0078] The conversation control system 100 configured in this way allows for the presentation of reaction images with a greater degree of freedom. Specifically, the conversation control system 100 generates and displays reaction images 53 using not only images (stamps) prepared in advance in the conversation control system 100, but also any character string entered by the user. This allows the user to freely enter character strings and generate and display original reaction images.

[0079] Furthermore, in the conversation control system 100, the color and font of a reaction image using a character string are selected from a plurality of options according to predetermined criteria, which makes it possible to improve the visibility of each reaction image.

[0080] (Variation) The conversation control device 20 may be implemented using multiple information processing devices. For example, the conversation control device 20 may be implemented using a device such as a cloud. For example, in the conversation control device 20, the storage unit 22 and the control unit 23 may be implemented in different information processing devices. For example, the storage unit 22 of the conversation control device 20 may be distributed and implemented in multiple information processing devices.

[0081] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0082] 100...conversation control system, 10...user terminal, 20...conversation control device, 11...communication unit, 12...operation unit, 13...display unit, 14...audio input unit, 15...audio output unit, 16...imaging unit, 17...storage unit, 171...user information storage unit, 18...control unit, 181...display control unit, 182...conference control unit, 183...conversation control unit, 21...communication unit, 22...storage unit, 221...conference room information storage unit, 222...reaction history information storage unit, 223...individual reaction history information storage unit, 23...control unit, 231...user control unit, 232...conference room control unit, 233...display information generation unit, 234...conversation control unit, 30...conference room area image, 31...conversation area, 311...conference room name, 312...enter button, 313...detail button, 32...Listening area, 41...Icon image, 51...Input area, 52...Comment button, 53...Reaction image, 54...Border, 541...Area, 55...First repeat button, 56...History display area, 57...Individual history display area, 571...Reaction image, 572...Number of agreeees, 573...Icon image, 91...Outdoor area

Claims

1. A conversation control system is provided in a virtual conference room for a user to converse with other users via a network, the conversation control system comprising a control unit that displays an image representing a character string entered by the user on the user's user terminal and on the user terminals of other users participating in the same conference room as the user for a predetermined period of time, and then erases the display.

2. The conversation control system according to claim 1 , wherein the control unit causes the user terminal of the user to display a first operation object for displaying an image representing a character string most recently input by the user.

3. 3. The conversation control system according to claim 2, wherein when the first operation object is operated, the control unit operates as if an instruction has been given to display an image representing a character string most recently input by the user.

4. 3. The conversation control system according to claim 2, wherein when a predetermined operation is performed while the first operation object is displayed, the control unit operates as if an instruction has been given to display an image representing a character string most recently input by the user.

5. The conversation control system of claim 1, wherein the control unit displays a second operation object on the user terminals of multiple users participating in the conference room to display an image representing a character string previously entered by a user participating in the conference room.

6. The conversation control system of claim 5, wherein the control unit displays, on user terminals of multiple users participating in the conference room, the number of users who have performed an operation to display an image showing the character string corresponding to the second operation target in proximity to the second operation target or the number of times the operation was performed.

7. A conversation control method in which a computer displays an image representing a character string entered by a user in a virtual conference room for conversations between the user and other users via a network on the user terminal of the user and on the user terminals of other users participating in the same conference room as the user for a predetermined period of time, and then erases the display.

8. A computer program for causing a computer to function as a conversation control system that includes a control unit that displays an image representing a string of characters entered by a user on the user's user terminal and on the user terminals of other users participating in the same conference room as the user for a predetermined period of time, and then erases the display, in a virtual conference room where a user and other users can converse over a network.

Images