Information processing apparatus, and information processing program

Abstract

To understand the status of communication performed by each user from user data posted in a chat room.SOLUTION: A chat server 6 is configured to: acquire user data posted in a chat room 10; and display, while switching multiple display forms according to a user instruction, association information generated by associating the acquired user data and at least one of replies and user reactions to the user data with a user having posted the user data and a posting time of the user data.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and an information processing program.

Background Art

[0002] Patent Document 1 discloses an information processing system that is communicably connected to a plurality of terminal devices and transmits a message received from a terminal device to another terminal device, the information processing system including a check unit that executes a spell check on the message received from the terminal device, and a transmission unit that transmits, to the other terminal device, a message including a spell in which a spelling mistake has been corrected when a spelling mistake is detected by the check unit.

[0003] Patent Document 2 discloses a program for visualizing data characteristics, the program causing a computer to execute a series of processes including a process of acquiring data labeled based on at least one variable, a process of generating a feature vector using the variable of the acquired data as a parameter, a process of mapping the generated feature vector into a multidimensional space, and a process of displaying the mapped feature vector on a screen.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] With the spread of telework, users in locations separated from each other may communicate using groupware with chat.

[0006] In a chat room where chats are conducted, conversations using user data such as characters are displayed in chronological order. Therefore, for example, in order for a user who joins the chat midway to grasp the content of the conversation being held in the chat room, it is necessary to read the conversations posted in the chat room retroactively. In addition, in a chat room, multiple topics may be conducted in parallel, and it may be difficult to understand which conversation is a statement regarding which topic and who is participating in the conversation.

[0007] An object of the present invention is to provide an information processing apparatus and an information processing program capable of grasping the communication status of each user from user data posted in a chat room.

Means for Solving the Problem

[0008] The information processing apparatus according to the first aspect includes a processor, and the processor acquires user data posted in a chat room in which a plurality of users participate, and at least one of the acquired user data and response information for the user data is associated with the user who posted the user data and the posting time of the user data. The associated information is switched and displayed in a plurality of display forms.

[0009] The information processing apparatus according to the second aspect is the information processing apparatus according to the first aspect, and the processor classifies each of the user data using at least one of the emotion information added to the user data and the response information for the user data into a plurality of categories, and displays the associated information in different display forms for each classified category.

[0010] The information processing apparatus according to the third aspect is the information processing apparatus according to the second aspect, and the processor divides a series of response information starting from common user data in the category including the user data having the response information into a group forming one communication, and changes the display form for each group according to the degree of response of the response information for the user data serving as the starting point.

[0011] The information processing apparatus according to the fourth aspect is the information processing apparatus according to the second aspect or the third aspect, wherein the processor acquires display attributes of the information device used by the user participating in the chat room, and according to the acquired display attributes of the information device for each user, changes the display form of the association information classified for each category for each information device.

[0012] The information processing apparatus according to the fifth aspect is the information processing apparatus according to any one of the first aspect to the fourth aspect, wherein the processor extracts the user data posted in the chat room according to the conditions specified by the user, and uses the extracted user data to display the association information of each user.

[0013] The information processing apparatus according to the sixth aspect is the information processing apparatus according to the fifth aspect, wherein the processor displays a list screen for displaying the user data for each user extracted in time series, and generates a report recording the activity status of the user in a specific period from the list screen for each user.

[0014] The information processing apparatus according to the seventh aspect is the information processing apparatus according to the sixth aspect, wherein the processor displays the user data on the list screen so that the communication partner based on the user data can be known.

[0015] The information processing apparatus according to the eighth aspect is the information processing apparatus according to any one of the first aspect to the seventh aspect, wherein the processor estimates the user's emotion from the voice uttered by the user, divides the voice at the turning point where the type of the estimated emotion changes, and generates the user data with the emotion information representing the estimated user emotion added to the characters representing each of the divided voices.

[0016] The information processing apparatus according to the ninth aspect is the information processing apparatus according to the eighth aspect, wherein the processor changes the emotion information added to the user data to the emotion information specified by the user.

[0017] The information processing apparatus according to the tenth aspect is the information processing apparatus according to the eighth or ninth aspect, wherein the processor re-divides the user data generated by dividing the voice at a division position designated by the user and displays it in the chat room.

[0018] The information processing apparatus according to the eleventh aspect is the information processing apparatus according to any one of the eighth to tenth aspects, wherein the processor combines the emotion of the user estimated from the voice and the emotion of the user estimated from the user image obtained by photographing the user who uttered the voice to estimate the emotion of the user.

[0019] The information processing apparatus according to the twelfth aspect is the information processing apparatus according to any one of the eighth to eleventh aspects, wherein the processor uses a balloon associated with the emotion information representing the estimated emotion of the user to display the user data in the chat room.

[0020] The information processing program according to the thirteenth aspect is a program that causes a computer to execute a process of acquiring user data posted in a chat room participated by a plurality of users, and switching and displaying the acquired user data and at least one of response information for the user data in a plurality of display forms in association with the user who posted the user data and the posting time of the user data.

Advantages of the Invention

[0021] According to the first aspect and the thirteenth aspect, there is an effect that the communication status of each user can be grasped from the user data posted in the chat room.

[0022] According to the second aspect, there is an effect that the communication status of each user can be grasped in a display form according to the classified category.

[0023] According to the third aspect, there is an effect that the communication status of each user can be grasped for each topic that is the subject of conversation.

[0024] According to the fourth aspect, there is an effect that the communication status of each user can be displayed according to the display attributes of the information device used by the user.

[0025] According to the fifth aspect, there is an effect that the communication status of each user can be displayed under the conditions specified by the user.

[0026] According to the sixth aspect, there is an effect that a daily report can be generated simply by the user chatting in the chat room.

[0027] According to the seventh aspect, there is an effect that the user can grasp the flow of communication among users in chronological order without tracing and checking the user data posted in the chat room.

[0028] According to the eighth aspect, there is an effect that the emotion with which the user spoke can be included in the user data simply by the user talking about the content to be posted in the chat room.

[0029] According to the ninth aspect, there is an effect that even when emotion information representing an emotion different from the actual emotion of the user is added to the user data, the correct emotion information can be re-added to the user data.

[0030] According to the tenth aspect, there is an effect that even when the user data generated from voice is not divided at the correct division position, the user data can be re-divided at the correct division position.

[0031] According to the eleventh aspect, there is an effect that the emotion of the user can be estimated more accurately than by estimating the emotion of the user from voice.

[0032] According to the 12th aspect, there is an effect that the user's emotion can be added to the user data without adding extra characters to the user data sent by the user to the chat room.

Brief Description of Drawings

[0033]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Mode for Carrying Out the Invention

[0034] Hereinafter, this embodiment will be described with reference to the drawings. Note that the same components and the same processing are given the same reference numerals throughout the drawings, and duplicate explanations are omitted.

[0035] FIG. 1 is a diagram showing an example of an information processing system 1 according to this embodiment. As shown in FIG. 1, the information processing system 1 includes a WEB server 4, a chat server 6, and a data server 8, and the WEB server 4, the chat server 6, and the data server 8 are connected by a LAN (Local Area Network) 5. The information processing system 1 provides a chat service to each user terminal 2 used by each user connected to the Internet 3 via the WEB server 4 connected to the Internet 3.

[0036] "Chat" means that a user performs information transmission such as real-time conversation, tweet, request, notice, memo entry, and report through characters and images. The information processing system 1 provides each user with a place where chat is performed, that is, a "chat room 10". The information posted in the chat room 10 is shared by each user who participates in the chat room 10 through the user terminal 2. Hereinafter, the information posted by the user to the chat room 10 through the user terminal 2 is referred to as "user data".

[0037] The WEB server 4 receives user data from each of the user terminals 2 and notifies the chat server 6, and transmits to the user terminal 2 an information screen for notifying the user of the status of the communication being performed in the chat room 10.

[0038] A chat server 6, which is an example of an information processing apparatus, upon receiving user data from the WEB server 4, generates association information associating at least one of the user data, response information which is user data posted in the chat room 10 as a response to the user data, with the user who posted the user data and the posting time of the user data. That is, the association information is a general term for information representing the status of the communication being performed in the chat room 10. The chat server 6 having such a function includes a chat control unit 6A and a display switching unit 6B.

[0039] The chat control unit 6A performs processing related to the management of the chat room 10 and the generation of association information. Specifically, the chat control unit 6A controls the display switching unit 6B to generate association information according to the display form instructed by the user from the user data posted in the chat room 10, and distributes to the user terminal 2 via the WEB server 4 an information screen displaying the generated association information in the instructed display form.

[0040] FIG. 2 is a diagram showing an example of the chat room 10. In the chat room 10, the user data associated with the poster of the user data is displayed in the posting order from the top. That is, the chat room 10 is an example of a display form for displaying the association information.

[0041] User data posted in the chat room is displayed surrounded by the speech bubble 12. The speech bubble 12 is associated with information representing the emotion and mental state of the user who posted the user data, that is, emotion information. When users talk to each other face to face, for example, the emotion and mental state of the speaking user can be grasped from the user's facial expression, voice volume and pitch, and speech rate. However, in a chat on the chat room 10, since the facial expression and voice of the other party cannot be known, users participating in the chat room 10 can know the emotion and mental state of the user who posted the user data, which is difficult to perceive only from the content of the user data posted in the chat room 10, through the speech bubble 12. Hereinafter, the emotion and mental state of a user that is difficult to perceive only from the content of the user data posted in the chat room 10 is referred to as "nuance".

[0042] FIG. 3 is a diagram showing an example of the speech bubble 12 used in the chat room 10. The speech bubble 12 includes, for example, a speech bubble 12A used for a tweet, a speech bubble 12B used for a memo, a speech bubble 12C used for a notice, a speech bubble 12D used for a normal post and reply, and a speech bubble 12E used for an urgent post and reply.

[0043] The speech bubble 12A used for a tweet is used for user data posted in the chat room 10 as a statement that does not particularly expect a response from other users, that is, as a monologue. Hereinafter, the user data displayed in the speech bubble 12A may be simply referred to as a "tweet".

[0044] The speech bubble 12B used for a memo is used for user data posted in the chat room 10 as a reminder to oneself. Hereinafter, the user data displayed in the speech bubble 12B may be simply referred to as a "memo".

[0045] The balloon 12C used for notifications is used for user data posted in the chat room 10 as something to be notified to the participants in the chat room 10, for example. Hereinafter, the user data displayed in the balloon 12C may be simply referred to as "notification".

[0046] The balloon 12D used for normal posts and replies is used for user data that is not, for example, a monologue, a memo to oneself, or something to be notified to other users. Hereinafter, the user data displayed in the balloon 12D may be simply referred to as "normal posts and replies".

[0047] The balloon 12E used for urgent posts and replies is used for user data that is particularly urgent to be notified to other users and user data for which some reaction is immediately desired among normal posts and replies. Hereinafter, the user data displayed in the balloon 12E may be simply referred to as "urgent posts and replies".

[0048] Note that when there is no need to distinguish and explain the balloons 12A to 12E, they are collectively referred to as "balloon 12".

[0049] As shown in FIG. 3, a predetermined icon 14 is associated with each type of balloon 12, and the chat control unit 6A controls the display switching unit 6B to display the balloon 12 with the associated icon 14 added in the chat room 10 (see FIG. 2).

[0050] Users who participate in chat room 10 can post, as a reaction to the user data, the emotions of the user pre-associated with icon 14 on chat room 10 by selecting icon 14 displayed together with the user data in chat room 10. That is, the user who posts the user data does not need to instruct chat server 6 about the icon 14 to be displayed together with their own user data. Chat control unit 6A automatically displays, together with the user data, icon 14 corresponding to the emotion information represented by speech bubble 12 according to the type of speech bubble 12 associated with the user data.

[0051] For example, for speech bubble 12A used for a tweet, icons 14 indicating ambiguous consent, empathy, and questions about the tweet are associated. Specifically, icon 14A is the icon 14 selected when there is an ambiguous question about the tweet, and icon 14B is the icon 14 selected when ambiguous consent or empathy is obtained for the tweet.

[0052] For speech bubble 12B used for a memo, icons 14 indicating a memo to oneself who posted the user data, reflection on action items, and intention to respond are associated. Specifically, icon 14C is the icon 14 selected when the content shown in the memo is an action item (also called a "ToDo item") that must be done, icon 14D is the icon 14 selected when the action item is an action item that has already started taking some action, and icon 14E is the icon 14 selected when the action item is an action item that has already been completed. For convenience of explanation, icon 14C may be referred to as "action icon 14C", icon 14D as "started icon 14D", and icon 14E as "completed icon 14E".

[0053] The speech bubble 12C used in the notification is associated with an icon 14 indicating, for example, an indication that the content of the notification has been confirmed. Specifically, the icon 14F is the icon 14 selected when expressing approval (so-called "like") for the content of the notification, and the icon 14G is the icon 14 selected when expressing a more positive approval for the content of the notification than the icon 14F.

[0054] In addition, the speech bubble 12D used for normal posts and replies is associated with an icon 14F indicating the feeling towards the post or reply, such as "like", and the speech bubble 12E used for urgent posts and replies is associated with, in addition to the icon 14F indicating that the content of the post or reply has been confirmed and approval has been expressed for the content, an icon 14C indicating reflection on action items and the like.

[0055] When there is no need to distinguish and explain the icons 14A to 14G, they are collectively referred to as "icon 14". Also, the icons 14 associated with each type of speech bubble 12 shown in FIG. 3 are examples, and the chat control unit 6A can change, delete, and add the icons 14 associated with each type of speech bubble 12 according to the user's instructions.

[0056] In this way, the reply to the user data and the user's reaction reflected in the user data by selecting the icon 14 are examples of response information to the user data.

[0057] FIG. 4 is a diagram showing an outline of a method for estimating the speech bubble 12, which explains which type of speech bubble 12 the chat control unit 6A uses to display user data in the chat room 10.

[0058] As shown in FIG. 4, when the user posts user data to the chat room 10, the user designates the nuance of the user using an input device 2A such as a keyboard or a touch panel of the user terminal 2, and transmits the nuance of the user together with the user data from the user terminal 2 to the chat server 6 via the WEB server 4. That is, when the user data is represented by characters, the chat control unit 6A specifies the speech balloon 12 of the user data displayed in the chat room 10 according to the nuance designated by the user.

[0059] Note that the user data transmitted from the user terminal 2 to the chat server 6 may be voice instead of characters. In this case, the voice uttered by the user toward the microphone 9 is changed to voice data by the user terminal 2 and transmitted to the chat server 6 via the WEB server 4.

[0060] The chat control unit 6A converts the user data represented by voice into user data represented by characters and displays it in the chat room 10, and estimates the nuance of the user from the acquired voice.

[0061] When the camera 7 is connected to the user terminal 2, the user may photograph his or her own expression while uttering voice toward the microphone 9 with the camera 7, and transmit the image captured by the camera 7 (referred to as "user image") to the chat server 6 via the WEB server 4 together with the voice. In this case, the chat control unit 6A estimates the nuance of the user from the voice and the user image.

[0062] As described above, the nuance of the user who has posted the user data can be estimated only by voice. However, hereinafter, as an example, the chat control unit 6A will estimate the nuance of the user using the voice and the user image. The method for estimating the nuance of the user using the voice and the user image will be described in detail later.

[0063] On the other hand, the display switching unit 6B in the chat server 6 in FIG. 1 generates an information screen for displaying the association information in a display form according to the instruction of the chat control unit 6A.

[0064] The data server 8 is a storage device that stores data related to the chat room 10 and the associated information.

[0065] The chat server 6 associates the user data received from the user terminal 2 with the chat room 10 and stores it in the user data DB (Database) 8A of the data server 8. Further, the chat server 6 classifies the associated information corresponding to each display form for each display form and stores it in the classification data DB8B of the data server 8. Note that the chat server 6 and the data server 8 may be integrated without separating them, and the chat server 6 may be integrated with the data server 8.

[0066] The display forms of the associated information described so far include, for example, "overhead view 16" and "history view 18".

[0067] FIG. 5 is a diagram showing an example of the overhead view 16. The overhead view 16 is an example of an information screen that classifies each of the user data posted in the chat room 10 into a plurality of categories using at least one of the types of balloons 12 representing the user's nuance and the response information for the user data, and displays the associated information in different display forms for each classified category. Specifically, the overhead view 16 uses, for example, the type of balloon 12 used for displaying user data, the posting amount of user data, the type and posting amount of response information for the user data, the posting interval and posting time of the user data and the response information, the attributes of the user who posted the user data and the response information, the content of the file posted together with the user data, and at least one of the user data that is the posting target of the response information to display the associated information in different display forms for each category classified thereby.

[0068] In the overhead view 16 shown in FIG. 5, the user data posted in the chat room 10 is classified into three categories: "dashboard", "notification", and "communication", and the communication status in the chat room 10 is displayed in different display forms for each category.

[0069] Based on the selection status of the action icon 14C, start icon 14D, and completion icon 14E in the overhead view 16, the dashboard classifies the memos posted in the chat room 10 into mere memos and memos with action items. When it is a memo with action items, the memo is further classified and displayed according to the progress status of the action items.

[0070] The announcements in the overhead view 16 collectively display only the announcements posted in the chat room 10.

[0071] The communication in the overhead view 16 divides a series of response information starting from the common user data posted in the chat room 10 into groups 17 that form one communication together with the starting user data, and changes and displays the display form for each group 17 according to the degree of response of the response information to the starting user data.

[0072] For the sake of convenience of explanation, the display range of the dashboard in the overhead view 16 is referred to as the "dashboard area", the display range of the announcements is referred to as the "announcement area", and the display range of the communication is referred to as the "communication area".

[0073] Note that the degree of response of the response information is an index representing the excitement level of the chat. For example, the higher the number of response information posts per unit time and the number of users who posted the response information, the higher the degree of response, that is, it indicates that the chat is lively. In the example of FIG. 5, the user data posted in the chat room 10 is divided into three groups 17, namely groups 17A to 17C, and is displayed in the communication area. Among them, a Hot label is displayed for group 17A where the chat is being conducted at a response degree equal to or higher than a predetermined response degree.

[0074] An icon 14H representing a file is displayed in group 17C, but the user can also attach a file to user data and post it to the chat room 10. Therefore, when displaying user data with a file attached, an icon 14H indicating that a file is attached is displayed.

[0075] On the other hand, FIG. 6 is a diagram showing an example of the history view 18. The history view 18 is an example of an information screen that extracts user data posted to the chat room 10 according to conditions specified by the user and displays association information of each user using the extracted user data.

[0076] In the example of FIG. 6, the communication statuses of Mr. A and Mr. B on the specified date specified by the user are classified for each time zone and displayed in the history view 18. The mark 13 displayed in the history view 18 is additional information that visually represents the communication status for each user and each time zone. For example, mark 13A indicates that the user is attending a meeting, mark 13B indicates that the user has posted a tweet to the chat room 10, and mark 13C indicates that the user has posted user data other than a tweet to the chat room 10 and is chatting with other users.

[0077] Whether the user is attending a meeting or not can be specified, for example, by the chat control unit 6A obtaining the schedule of each user from a schedule server (not shown). Thus, in the history view 18, in addition to the user data posted to the chat room 10, various types of information about the user, such as the schedule of the user participating in the chat room 10, the operating state of the user terminal 2, and the user's leaving work information, can be displayed. That is, in the history view 18, an action history including the communication status of the user can be grasped.

[0078] In addition, when the corresponding time period is outside the user's working hours, the time period outside working hours is displayed in a different form from the working hours. Also, when the power of the user terminal 2 is off or the user is not logged in to the user terminal 2, etc., when the user is in a state where they cannot communicate with other users in the chat room 10, "OFF" is displayed for the time period during which the user was in that state. In the example of the history view 18 shown in FIG. 6, the time periods outside the user's working hours are grayed out. Therefore, it can be seen that from 12:00 to 13:00 on December 1st, Mr. B was outside his working hours.

[0079] Note that graying out is a display form in which a specific location (in this case, within the frame of the time period) is filled with gray of a density that allows the described content to be confirmed. Naturally, a specific location may be filled with a color other than gray.

[0080] User data, the user's schedule, the operating state of the user terminal 2, and the user's leaving work information are examples of the information used for the display of the history view 18, and the information used to display such a history view 18 is referred to as "history information".

[0081] The conditions that the user can specify regarding the display of the history view 18 are not limited to the date only. For example, at least one of the user to be displayed in the history view 18, the width of the time period, the type of history information to be displayed, the background color in the display range of each time period that changes depending on the type of history information to be displayed, etc. can be specified.

[0082] The chat control unit 6A switches between the overview view 16 and the history view 18 for each user terminal 2 and displays them on the user terminal 2 according to the instructions from each user. However, according to the instructions from the user, at least two of the chat room 10, the overview view 16, and the history view 18 may be displayed on the user terminal 2 together. The overview view 16 and the history view 18 are also examples of categories that display the classified user data in different display forms.

[0083] The chat server 6 shown in FIG. 1 is configured using, for example, a computer 20. FIG. 7 is a diagram showing a configuration example of the chat server 6 using the computer 20.

[0084] The computer 20 includes a CPU (Central Processing Unit) 21 which is an example of a processor responsible for executing the functions of the chat server 6 shown in FIG. 1, a ROM (Read Only Memory) 22 that stores an information processing program for causing the computer 20 to function as the chat server 6, a RAM (Random Access Memory) 23 used as a temporary working area for the CPU 21, a non-volatile memory 24, and an input / output interface (I / O) 25. The CPU 21, ROM 22, RAM 23, non-volatile memory 24, and I / O 25 are each connected via a bus 26.

[0085] The non-volatile memory 24 is an example of a storage device that maintains the stored information even when the power supplied to the non-volatile memory 24 is cut off. For example, a semiconductor memory is used, but a hard disk may also be used. The non-volatile memory 24 stores parameters such as thresholds that the CPU 21 refers to when executing an information processing program.

[0086] Connected to the I / O 25 are, for example, a communication unit 27, an input unit 28, and a display unit 29.

[0087] The communication unit 27 is connected to the LAN 5, has a communication protocol for performing data communication with the user terminal 2 via the WEB server 4 and for performing data communication with the data server 8.

[0088] The input unit 28 is a device that receives instructions from an operator of the chat server 6 and notifies the CPU 21. For example, buttons, touch panels, keyboards, pointing devices, and mice are used.

[0089] The display unit 29 is a device that visually outputs the information processed by the CPU 21. For example, display devices such as liquid crystal displays, organic EL (Electro Luminescence) displays, and projectors that project images onto a screen are used.

[0090] Note that since the chat server 6 can also be remotely operated via the Internet 3, the input unit 28 and the display unit 29 are not necessarily essential units in the computer 20.

[0091] Next, as shown in FIG. 4, a method for estimating the nuance of a user when the user posts user data in voice to the chat room 10 will be described.

[0092] FIG. 8 is a flowchart showing an example of the flow of nuance estimation processing executed by the CPU 21 of the chat server 6 when the chat server 6 receives user data in voice from the user terminal 2 via the WEB server 4, for example.

[0093] The information processing program that defines the nuance estimation processing is stored in advance, for example, in the ROM 22 of the chat server 6. The CPU 21 of the chat server 6 reads the information processing program stored in the ROM 22 and executes the nuance estimation processing.

[0094] Note that, as already described, a user image captured by the camera 7 is added to the user data expressed in voice.

[0095] First, in step S10, the CPU 21 analyzes the voice of the user data using a known voice analysis method, converts the voice of the user data into characters, and estimates the nuance of the user. The nuance of the user estimated from the voice is referred to as "voice nuance".

[0096] For example, the CPU 21 quantifies and sets the certainty of the voice nuance obtained from the user data for each predetermined type of voice nuance.

[0097] FIG. 9 is a diagram for explaining a nuance estimation process for estimating the nuance of a user using points, and FIG. 9(A) is a diagram for explaining the quantification of voice nuances.

[0098] As shown in FIG. 9(A), when there are "muttering", "appeal", "urgency", and "business-like" as predetermined types of voice nuances, the CPU 21 sets points for each type of voice nuance. The higher the point of the voice nuance, the higher the probability that it represents the user's emotion potentially appearing in the user data expressed by voice. For convenience of explanation, the points set for the voice nuance are referred to as "voice points".

[0099] Here, as an example, the voice nuances are classified into four types: "muttering", "appeal", "urgency", and "business-like", but it goes without saying that they may be classified into other types, such as "calm" or "emotional".

[0100] In step S20, the CPU 21 analyzes the user image added to the user data using a known image analysis method and estimates the nuance of the user. The nuance of the user estimated from the user image is referred to as "facial expression nuance".

[0101] For example, the CPU 21 quantifies and sets the certainty of the facial expression nuance obtained from the user data for each predetermined type of facial expression nuance. FIG. 9(B) is a diagram for explaining the quantification of facial expression nuances.

[0102] As shown in FIG. 9(B), when there are "positive", "neutral", and "negative" as predetermined types of expression nuances, the CPU 21 sets points for each type of expression nuance. The higher the point of the expression nuance, the higher the probability that it represents the user's emotion potentially appearing in the user image. For convenience of explanation, the points set for the expression nuance are referred to as "expression points".

[0103] Here, as an example, the expression nuances are classified into three types: "positive", "neutral", and "negative". Needless to say, they may be classified into other types, such as "question" or "indifference".

[0104] In step S30, the CPU 21 calculates points for each type of speech bubble 12 using the voice points for each type of voice nuance set in step S10 and the expression points for each type of expression nuance set in step S20. For example, the CPU 21 uses a predetermined evaluation function with the voice points for each type of voice nuance and the expression points for each type of expression nuance as explanatory variables and the points for each type of speech bubble 12 (referred to as "speech bubble points") as the objective variable to calculate the speech bubble points for each type of speech bubble 12.

[0105] FIG. 9(C) is a diagram for explaining the point setting of the speech bubble 12, showing a state where speech bubble points are set for each type of speech bubble 12. In the case of FIG. 9(C), the CPU 21 selects the speech bubble 12B used for the memo with the highest speech bubble point as a candidate for the speech bubble 12 to be used for displaying the user data.

[0106] That is, the CPU 21 estimates the user's nuance by combining the user's voice nuance estimated from the voice and the user's expression nuance estimated from the user image.

[0107] In step S40, the CPU 21 generates a speech bubble selection dialog 11 that displays candidates for the speech bubble 12 used to display user data, and causes the user terminal 2 that has transmitted the user data represented by voice to display the generated speech bubble selection dialog 11.

[0108] FIG. 10 is a diagram showing an example of the speech bubble selection dialog 11. In the column of "Automatically Selected Speech Bubble" in the speech bubble selection dialog 11, the user data converted into characters is displayed by the speech bubble 12 with the highest speech bubble point selected in step S30. In the column of "View Recommended Speech Bubbles" in the speech bubble selection dialog 11, the user data converted into characters is displayed by a speech bubble 12 other than the speech bubble 12 displayed in the column of "Automatically Selected Speech Bubble", and the speech bubble point is equal to or higher than a predetermined threshold value.

[0109] The user who has posted user data by voice selects the speech bubble 12 that most correctly represents their nuance from among the candidates for the speech bubble 12 displayed in the speech bubble selection dialog 11. When the CPU 21 receives the selection result of the speech bubble from the user terminal 2 through the speech bubble selection dialog 11, it displays the user data converted into characters in the chat room 10 using the selected speech bubble 12.

[0110] Note that the CPU 21 may display an item "Select Other Speech Bubbles" in the speech bubble selection dialog 11, and when this item is selected by the user, all other types of speech bubbles 12 not displayed in the speech bubble selection dialog 11 may be displayed to allow the user to select a speech bubble 12.

[0111] Through the voice analysis in step S10 of FIG. 8, for example, it may be analyzed that the user data contains a plurality of different nuances such that the tendency is initially a tweet but switches to an appeal tendency in the middle.

[0112] In such a case, the CPU 21 may divide the user data at the turning point where the nuance changes, and select candidates for the speech bubbles 12 along with each nuance for each divided user data. That is, in the chat room 10, the user data divided by the CPU 21 is displayed with the speech bubbles 12 corresponding to the nuance of the user estimated from each divided user data.

[0113] The CPU 21 may determine the turning point of the nuance using at least one of the change in the frequency and the change in the volume included in the voice, and may determine the silent portion existing between voices, that is, the silent portion for a predetermined period or more existing between voices, as the turning point of the nuance. The silent portion refers to a portion where the volume is less than a predetermined threshold for silent determination.

[0114] Also, when connectives such as "by the way" or "come to think of it" appear in the middle of the voice, the nuance of the user may change along with the change of the topic. Therefore, the CPU 21 may identify the portion where the word corresponding to the connective is uttered from the voice by analyzing the frequency change of the voice, and may determine the portion as the turning point of the nuance. Specifically, the CPU 21 identifies the portion where the word corresponding to the connective is uttered from the voice by comparing, for example, the frequency data of the voice when the word corresponding to the connective is uttered, which is stored in advance in the non-volatile memory 14, with the user's voice.

[0115] In this case, since the user data may be divided at a location not intended by the user, the CPU 21 may accept the correct division position from the user, re-divide the user data at the division position specified by the user, and display it again in the chat room 10.

[0116] Naturally, even if the user data that the CPU 21 has not divided itself, such as user data represented by characters, the CPU 21 may receive the division position from the user, re-divide the user data at the division position specified by the user, and display it in the chat room 10 again. Further, even after the CPU 21 displays the user data in the chat room 10, the CPU 21 may receive an instruction to change the speech bubble 12 for specific user data from the user, and change the type of the speech bubble 12 of the specified user data to the type of the speech bubble 12 specified by the user.

[0117] In the above, an example in which the nuance estimation process is executed when the user data is received by voice from the user terminal 2 has been described. However, even if the user data is represented by characters, for example, the CPU 21 may perform the nuance estimation process when the nuance is not specified by the user.

[0118] FIG. 11 is a flowchart showing an example of the flow of the nuance estimation process for user data represented by characters. The differences between the flowchart shown in FIG. 11 and the flowchart of the nuance estimation process for user data represented by voice shown in FIG. 8 are that steps S10 and S20 are replaced with steps S10A and S20A, respectively, and step S15 is newly added.

[0119] In step S10A, the CPU 21 analyzes the context of the user data using a known natural language analysis method and estimates the nuance of the user. The nuance of the user estimated from the context of the user data is referred to as "character nuance".

[0120] The CPU 21 points and sets the probability of the character nuance obtained from the user data for each predetermined type of character nuance. For convenience of explanation, the points set for the character nuance are referred to as "character points".

[0121] In step S15, the CPU 21 determines whether a file is attached to the user data. If a file is attached to the user data, the process proceeds to step S20A.

[0122] In step S20A, the CPU 21 analyzes the content of the file and estimates the user's nuance. The nuance of the user estimated from the file is referred to as "file nuance".

[0123] The CPU 21 points and sets the reliability of the file nuance obtained from the file for each predetermined type of file nuance. For the sake of convenience of explanation, the points set for the file nuance are referred to as "file points".

[0124] For example, if the file is a meeting document, the CPU 21 estimates it as "business", if the file is marked with "Please contact urgently", the CPU 21 estimates it as "urgent", and if it is just a landscape image, the CPU 21 estimates it as "mutter". Also, when the file is a photo of a person, the CPU 21 may estimate the user's nuance from the expression of the person shown in the photo.

[0125] On the other hand, if it is determined in the determination process of step S15 that no file is attached to the user data, the process proceeds to step S30 without executing the process of step S20A.

[0126] Thereafter, following the processing flow described in FIG. 8, if there are character points for each type of character nuance set in step S10A and file points for each type of file nuance set in step S20A, the CPU 21 calculates blowout points for each type of speech bubble 12 using the file points, and displays a speech bubble selection dialog 11 that displays candidates for the speech bubble 12 to be used for displaying the user data on the user terminal 2.

[0127] In this way, the CPU 21 can present to the user candidates for the speech bubble 12 that follows the user's nuance based on the context of the user data represented by characters and the content of the file attached to the user data.

[0128] In the nuance estimation process shown in FIGS. 8 and 11, the CPU 21 displays candidates for the speech bubble 12 used for displaying the user data in the speech bubble selection dialog 11 and allows the user to select the speech bubble 12 to be used. However, the user data may be displayed in the chat room 10 using the speech bubble 12 with the highest speech bubble point without allowing the user to select the speech bubble 12. Whether to allow the user to select the speech bubble 12 can be changed by setting.

[0129] Next, a display example of the bird's-eye view 16 will be described.

[0130] FIG. 12 is a flowchart showing an example of the flow of the bird's-eye view display process executed by the CPU 21 of the chat server 6 when the chat server 6 receives an instruction to display the bird's-eye view 16 from the user terminal 2 via the WEB server 4.

[0131] The information processing program that defines the bird's-eye view display process is stored in advance in the ROM 22 of the chat server 6, for example. The CPU 21 of the chat server 6 reads the information processing program stored in the ROM 22 and executes the bird's-eye view display process.

[0132] In step S100, the CPU 21 executes a communication display process of displaying the situation of the communication being performed in the chat room 10 in the communication area.

[0133] In step S200, the CPU 21 executes a dashboard display process of displaying information about the memo posted in the chat room 10 in the dashboard area.

[0134] In step S300, the CPU 21 executes a notification display process of displaying information regarding the notification posted in the chat room 10 in the notification area.

[0135] FIG. 13 is a flowchart showing in detail an example of the flow of the communication display process in step S100 of FIG. 12.

[0136] First, in step S110, the CPU 21 tracks response information for user data posted in the chat room 10 according to the corresponding relationship, and generates a tree structure rooted at the user data that is the starting point of a series of response information. Each element of the generated tree structure forms an independent group 17 in the communication area.

[0137] The corresponding relationship of the response information can be tracked, for example, by referring to the destination username of the reply described in the form of "@username" in the user data if the response information is a reply, or by referring to the user data associated with the selected icon 14 if the response information is the user's reaction by selecting the icon 14.

[0138] However, among the user data, there are some that do not show the user's reaction to the user data and seem to be just simple tweets or memos at first glance, but actually are response information for other user data. If there is user data whose corresponding relationship with other user data is not explicitly shown, such user data will not be included in the tree structure generated in step S110, so the communication situation in the chat room 10 cannot be correctly reflected.

[0139] Therefore, in step S120, the CPU 21 analyzes the context of the user data using a known natural language analysis method, and performs a tree structure analysis to detect user data that, although not explicitly showing the corresponding relationship with other user data, is response information for other user data from the context and includes it in the tree structure.

[0140] In step S130, the CPU 21 performs a response analysis on the response information of each group 17, and changes the display form of each group 17 according to the degree of response of the response information represented by at least one of, for example, the number of posts of the response information and the number of users who posted the response information.

[0141] The display of the Hot label for the group 17A shown in FIG. 5 is an example of a change in the display form according to the degree of response.

[0142] In addition to this, the CPU 21 may, for example, increase the display range of the group 17 with a higher degree of response compared to other groups 17, or display the group 17 closer to the center of the communication area compared to other groups 17. Also, the CPU 21 may change the font of the characters used for the display of the group 17 according to the degree of response, or change the background color of the display range of the group 17 and the color and size of the characters for displaying the response information and the user data that is the starting point.

[0143] In step S140, the CPU 21 performs sentiment analysis using a known natural language analysis method on the response information of each group 17, and changes the display form of each group 17 depending on whether there are more opposing opinions or more approval opinions.

[0144] For example, when there are many opposing opinions against the proposal of the user data that is the starting point, the CPU 21 displays "×" for the group 17, and when there are many approval opinions, the CPU 21 displays "○" for the group 17. Also, the CPU 21 may display the ratio of approval opinions and opposing opinions numerically, or change the font of the characters used for the display of the group 17 according to the ratio of approval opinions and opposing opinions, or change the background color and the color and size of the characters of the display range of the group 17.

[0145] Furthermore, the CPU 21 may change the display position of each group 17 in the communication area according to the ratio of approval opinions to opposition opinions. For example, the CPU 21 may display the group 17 with more approval opinions than opposition opinions on the left side with the center of the communication area as the boundary, and display the group 17 with more opposition opinions than approval opinions on the right side.

[0146] In step S150, the CPU 21 performs timeline analysis on the response information of each group 17, and changes the display form of each group 17 according to the degree of response of the response information represented by at least one of the posting interval of the response information, the posting frequency per unit time, the posting time, and the elapsed time since the last response information was posted.

[0147] For example, the CPU 21 may display the response information with a newer posting time larger within the group 17, or may enlarge the display range of the group 17 with a higher posting frequency of the response information per unit time than other groups 17, or may change the background color of the display range of the group 17, the color and size of the characters for displaying the response information and the user data serving as the starting point.

[0148] In addition to this, the CPU 21 may display the time zone with the largest number of posted response information in the group 17 using at least one of characters, symbols, figures, and colors.

[0149] Note that the CPU 21 also displays the user name of the user who posted the user data and response information, the avatar or user icon representing the user (hereinafter referred to as "user information") together with the user data and response information in the group 17. User information is an example of an attribute related to the user.

[0150] With the above, the communication display process shown in FIG. 13 is terminated. By the communication display process, the communication carried out in the chat room 10 is classified and displayed for each topic in the communication area of the overhead view 16 shown in FIG. 5, and the user can grasp the excitement level of the chat for each topic.

[0151] FIG. 14 is a flowchart showing in detail an example of the flow of the dashboard display process in step S200 of FIG. 12.

[0152] In step S210, the CPU 21 acquires any one memo posted in the chat room 10.

[0153] In step S220, the CPU 21 determines whether the action icon 14C is selected for the memo (hereinafter referred to as "acquired memo") acquired in step S210. If the action icon 14C is not selected, the process proceeds to step S230.

[0154] In this case, the acquired memo is not an action item but just a memo. Therefore, in step S230, the CPU 21 associates the acquired memo with the user information of the user who posted the acquired memo and displays it in the memo column of the dashboard.

[0155] On the other hand, if it is determined in the determination process of step S220 that the action icon 14C is selected, the process proceeds to step S240.

[0156] In step S240, the CPU 21 determines whether the start icon 14D is selected for the acquired memo. If the start icon 14D is not selected, the process proceeds to step S250.

[0157] In this case, the acquired memo is a memo with action items, but the user has not taken any action. Therefore, in step S250, the CPU 21 associates the acquired memo with the user information of the user who posted the acquired memo and displays it in the unstarted column of the dashboard.

[0158] On the other hand, if it is determined in the determination process of step S240 that the start icon 14D is selected, the process proceeds to step S260.

[0159] In step S260, the CPU 21 determines whether the completion icon 14E is selected for the acquired memo. If the completion icon 14E is selected, the process proceeds to step S270.

[0160] In this case, it means that the action item represented by the acquired memo has been completed. Therefore, in step S270, the CPU 21 associates the acquired memo with the user information of the user who posted the acquired memo and displays it in the completed column of the dashboard.

[0161] On the other hand, if it is determined in the determination process of step S260 that the completion icon 14E is not selected, the process proceeds to step S280.

[0162] In this case, it means that the action item represented by the acquired memo has been started by the user but not yet completed. Therefore, in step S280, the CPU 21 associates the acquired memo with the user information of the user who posted the acquired memo and displays it in the in-progress column of the dashboard.

[0163] Since the acquisition memo and the user information of the user who posted the acquisition memo are displayed in any column of the dashboard, in step S290, the CPU 21 determines whether there is an unacquired memo in the chat room 10. If there is an unacquired memo, the process proceeds to step S210, and the CPU 21 acquires any one memo from the unacquired memos as a new acquired memo. That is, the CPU 21 repeatedly executes steps S210 to S290 until it is determined in the determination process of step S290 that there is no unacquired memo in the chat room 10, and performs a process of classifying and displaying each memo posted in the chat room 10 in any column of the dashboard.

[0164] If it is determined in the determination process of step S290 that there is no unacquired memo in the chat room 10, the dashboard display process shown in FIG. 14 is terminated.

[0165] By the dashboard display process, as in the dashboard of the bird's-eye view 16 shown in FIG. 5, the memos are classified into simple memos and memos with action items, and the memos with action items are further classified into "not started", "in progress", and "completed" according to the progress status of the action items. Note that the numbers displayed beside the "not started", "in progress", and "completed" tags on the dashboard represent the number of memos classified in each column of the action items.

[0166] FIG. 15 is a flowchart showing in detail an example of the flow of the notification display process in step S300 of FIG. 12.

[0167] In step S310, the CPU 21 acquires any one notification posted in the chat room 10.

[0168] In step S320, the CPU 21 acquires the execution start time of this process (hereinafter referred to as the "current time"), calculates the elapsed time, which is the difference between the current time and the posting time when the notification acquired in step S310 (hereinafter referred to as the "acquired notification") was posted to the chat room 10, and stores it in the RAM 23. The CPU 21 acquires the current time from, for example, a clock built into the CPU 21, but it may also acquire the current time from a time server (not shown) connected to the Internet 3.

[0169] In step S330, the CPU 21 acquires the elapsed time from the RAM 23 and determines whether the elapsed time exceeds the retention time. The retention time is the time used to determine the display form of the acquired notification based on the length of the elapsed time and is, for example, pre-stored in the non-volatile memory 24. If the elapsed time is longer than the retention time, it means that too much time has passed since the acquired notification was posted to the chat room 10, for example, the start time of the meeting notified by the acquired notification has already passed, and the value of the information notified by the acquired notification has decreased.

[0170] Therefore, if the elapsed time exceeds the retention time, the process proceeds to step S340. In step S340, the CPU 21 grays out the acquired notification so that it is less conspicuous than other notifications in the notification area and displays it in the notification area as shown in, for example, FIG. 5.

[0171] On the other hand, if it is determined in the determination process of step S330 that the elapsed time is equal to or less than the retention time, the process proceeds to step S350.

[0172] In step S350, the CPU 21 determines whether or not the elapsed time has exceeded the emphasis time. The emphasis time, like the retention time, is a time used to determine the display form of the acquisition notification based on the length of the elapsed time, and is stored in advance in the non-volatile memory 24, for example. There is a relationship of "retention time > emphasis time" between the retention time and the emphasis time, and the retention time and the emphasis time can be changed by the user. When the elapsed time is less than or equal to the emphasis time, not much time has passed since the acquisition notification was posted in the chat room 10. For example, since the meeting notified by the acquisition notification is about to start, it means that the value of the information notified by the acquisition notification is high and we want to widely inform the fact as much as possible.

[0173] Therefore, when the elapsed time is less than or equal to the emphasis time, the process proceeds to step S360. In step S360, the CPU 21 emphasizes the acquisition notification and displays it in the notification area so that the acquisition notification stands out more than other notifications in the notification area. To emphasize and display the acquisition notification means, for example, making the characters of the acquisition notification larger than those of other notifications, changing the color of the characters, changing the thickness of the characters, changing the background color, changing the font of the characters, or adding characters or images such as "important" to the acquisition notification to attract the user's attention.

[0174] On the other hand, if it is determined in the determination process of step S350 that the elapsed time has exceeded the emphasis time, the process proceeds to step S370. In this case, although it is not necessary to display the acquisition notification more prominently than other notifications, it will not be an older notification that is more grayed out.

[0175] Therefore, in step S370, the CPU 21 displays the acquisition notification in the notification area in a predetermined standard display form. The standard display form is an intermediate display form that is less prominent than the emphasized display of the acquisition notification in step S360 and more prominent than the graying out of the acquisition notification in step S340.

[0176] After the acquisition notifications are displayed in the notification area of the bird's-eye view 16 in steps S340, S360, and S370, in step S380, the CPU 21 determines whether there is an unacquired notification in the chat room 10. If there is an unacquired notification, the process proceeds to step S310, and the CPU 21 acquires any one of the unacquired notifications as a new acquisition notification. That is, the CPU 21 repeatedly executes steps S310 to S380 until it determines in the determination process of step S380 that there is no unacquired notification in the chat room 10, and performs the process of displaying each notification posted in the chat room 10 in the notification area in a display form based on the elapsed time.

[0177] If it is determined in the determination process of step S380 that there is no unacquired notification in the chat room 10, the notification display process shown in FIG. 15 is terminated.

[0178] Note that in step S340 of FIG. 15, the acquisition notification was displayed in a grayed-out state so as not to be more prominent than other notifications, but the acquisition notification may not be displayed in the notification area.

[0179] As described above, the user data posted in the chat room 10 is displayed in different display forms in the dashboard area, notification area, and communication area of the bird's-eye view 16, and the bird's-eye view display process shown in FIG. 12 is terminated.

[0180] Next, an example of the display of the history view 18 will be described.

[0181] FIG. 16 is a diagram showing an example of displaying the chat room 10 (referred to as the "chat room 10 in standard format") shown in FIG. 2 in another display form according to an instruction from the user. The chat room 10 shown in FIG. 16 is a chat room 10 in bulletin board format that displays only user data without using the speech bubble 12. In the chat room 10 in bulletin board format, since each user data is displayed in a time table format separated by a frame, the display is simpler than the chat room 10 in standard format with displays other than user data such as the speech bubble 12 and the icon 14, and the communication situation can be grasped by concentrating only on the user data. The CPU 21 switches the display between the chat room 10 in standard format and the chat room 10 in bulletin board format according to an instruction from the user. That is, the chat room 10 is also an example of a category that displays the associated information in different display forms.

[0182] The chat room 10 in bulletin board format includes a button 10A. When the user selects the button 10A on the user terminal 2, the CPU 21 displays the history view 18 shown in FIG. 17 on the user terminal 2 on which the button 10A is selected. Note that it is not always necessary to switch the display form of the chat room 10 to the bulletin board format in order to display the history view 18. For example, by also displaying the button 10A in the chat room 10 in standard format, the history view 18 can also be displayed from the chat room 10 in standard format. When the button 10A is selected, the CPU 21 may either switch the chat room 10 in standard format or bulletin board format to the history view 18 for display, or display the history view 18 together with the chat room 10 in standard format or bulletin board format.

[0183] FIG. 18 is a flowchart showing an example of the flow of the history view display process executed by the CPU 21 of the chat server 6 when, for example, the chat server 6 receives an instruction to display the history view 18 from the user terminal 2 via the WEB server 4.

[0184] The information processing program that defines the history view display process is stored in advance, for example, in the ROM 22 of the chat server 6. The CPU 21 of the chat server 6 reads the information processing program stored in the ROM 22 and executes the history view display process.

[0185] Here, as an example, a specific date is specified by the user as the extraction condition for user data, and the communication status in the chat room 10 on the specified date is classified every specified time (for example, one hour) and displayed in the history view 18. An example will be described.

[0186] In step S400, the CPU 21 divides the time of the specified date by the specified time specified by the user, that is, every hour, and sets a time zone for every hour. Specifically, for example, time zones divided every hour such as 9:00 - 10:00 and 10:00 - 11:00 are set.

[0187] In step S410, the CPU 21 selects any one time zone from the time zones set in step S400. For the sake of convenience of explanation, the time zone selected here is referred to as the "selected time zone".

[0188] In step S420, for each user participating in the chat room 10, the CPU 21 acquires history information including user data whose posting time is included in the selected time zone.

[0189] In step S430, the CPU 21 associates the history information for each user acquired in step S420 with the selected time zone and displays it on the user terminal 2.

[0190] In this case, the CPU 21 displays the mark 13 for each piece of history information according to the type of history information. Also, the CPU 21 performs graying out of the time zone and OFF display for the time zone according to the operating state of each user terminal 2 included in the history information and the leaving work information of each user.

[0191] When the history information is user data, the CPU 21 may display the icon 14 associated with the user data together with the user data. The icon 14 displayed in the history view 18 can also be selected by the user, just like the icon 14 displayed in the chat room 10. Also, when a file is attached to the user data, the CPU 21 may display an icon 14H (see FIG. 5) representing the file together with the user data.

[0192] In step S440, the CPU 21 determines whether there is an unselected time period within the time period set in step S400. If there is an unselected time period, the process proceeds to step S410, and the CPU 21 selects any one of the unselected time periods as a new time period. That is, the CPU 21 repeatedly executes steps S410 to S440 until it is determined in the determination process of step S440 that there is no unselected time period, and performs a process of associating the history information for each user with each selected time period and displaying it on the user terminal 2.

[0193] If it is determined in the determination process of step S440 that there is no unselected time period, the history view display process shown in FIG. 18 is terminated. As described above, the history view 18 as shown in FIG. 17 is displayed on the user terminal 2 that has sent the display instruction for the history view 18. The history view 18 is also an example of a list screen because it displays a list of user data for each user extracted from the chat room 10 in chronological order.

[0194] In step S430 of FIG. 18, the history view 18 in which the history information for each user is associated with the selected time zone was displayed. However, among the users displayed in the history view 18, there may be users living abroad. In such a case, due to the time difference, the time zone displayed in the history view 18 may be different from the local time of the users living abroad. Therefore, the CPU 21 may switch the time zone display of the history view 18 according to the time zone of the region where the user selected in the history view 18 lives. In addition, the CPU 21 may display, in the history view 18, the time zone displayed according to the time zone of the region where the user selected in the history view 18 lives, together with the time zone displayed according to the time zone of the region where the user viewing the history view 18 lives, so that the correspondence such as what time 9:00 for Mr. A is for Mr. B can be understood.

[0195] As described in the overhead view display process shown in FIG. 12, the CPU 21 can analyze the correspondence relationship of the response information for the user data that is the starting point in the chat room 10. Therefore, when the CPU 21 displays the history view 18 on the user terminal 2, it may explicitly display the user data in the response relationship in the history view 18.

[0196] For example, in the history view 18 shown in FIG. 17, assume that the user data "Thank you very much" of Mr. A in the time zone of 10:00 to 11:00 is a reply to the user data "FAX sent" of Mr. C in the same time zone. In this case, the CPU 21 may change the text color of "FAX sent" of Mr. C and the text color of "Thank you very much" of Mr. A to a common color different from the text colors of other user data, and display the history view 18 so that the response relationship can be understood. In addition, when the user selects the text of "FAX sent" of Mr. C, the CPU 21 may blink the text of "Thank you very much" of Mr. A, or display a pop-up dialog showing the text of "Thank you very much" of Mr. A around the text of "FAX sent" of Mr. C.

[0197] On the one hand, here, as the specification of the conditions regarding the display of the history view 18, the display process of the history view when the specified date of the history information to be displayed in the history view 18 and the length of the time period for classifying the history information are specified has been described. However, the CPU 21 may further acquire another condition regarding the display of the history view 18.

[0198] For example, when a condition is specified to display only the user data for which a specific icon 14 (for example, the icon 14F indicating approval) is selected in the history view 18, the CPU 21 extracts only the user data for which the icon 14F is selected from the chat room 10 and displays it in the history view 18 for each user and for each time period.

[0199] Also, when the user selects the icon 14H representing the file displayed in the history view 18, the CPU 21 acquires the user name of the distribution destination of the file associated with the selected icon 14H and the user data exchanged between users regarding the file from the chat room 10, and may display the user data sorted in chronological order so that the flow of communication between users can be understood, together with the user name of the user who posted the user data, in a pop-up dialog, for example.

[0200] Also, since the history information for each user is listed in chronological order in the history view 18, the CPU 21 may acquire the history information regarding the selected user from the history view 18 and create a report documenting the user's activity status during a specific period such as a daily report, a weekly report, and a monthly report. The report is created as a file, and there is no restriction on the file format of the report. For example, it may be created in a document format such as XML (Extensible Markup Language) or a CSV (Comma Separated Value) format.

[0201] Note that, as the user terminal 2, various types of information devices connectable to the Internet 3, such as a smartphone 30 and a desktop computer 32, are used. However, the display ranges of the screens for displaying the chat room 10, the bird's-eye view 16, and the history view 18 differ depending on the information device.

[0202] Therefore, the CPU 21 preferably obtains from the user terminal 2 a display attribute indicating the size of the display range of the screen in each user terminal 2, such as the screen size of the user terminal 2, and changes the display form of the associated information classified by category according to the obtained display attribute for each user terminal 2.

[0203] Specifically, when the smartphone 30 is used as the user terminal 2, the display range of the screen is narrower than that of the desktop computer 32. Therefore, when displaying the bird's-eye view 16 on the smartphone 30, the CPU 21 may display only the category specified by the user among the dashboard, notifications, and communication, instead of displaying the entire bird's-eye view 16 shown in FIG. 5.

[0204] FIG. 19 is a diagram showing a display example in which only the communication area of the bird's-eye view 16 is displayed on the smartphone 30. The smartphone 30 has a shape that is longer in the vertical direction than in the horizontal direction, and since the user scrolls the screen in the vertical direction to view the information within the screen, arranging the information in the vertical direction is easier for the user to view than arranging the information in the horizontal direction within the screen.

[0205] Therefore, the CPU 21 displays the groups 17 with a horizontal width as in the communication area of the bird's-eye view 16 shown in FIG. 5 on the desktop computer 32, and rearranges and displays them vertically as in the communication area of the bird's-eye view 16 shown in FIG. 19 on the smartphone 30. In this case, the CPU 21 may rearrange the display order of the groups 17 so that, for example, the group 17 with a higher response degree of the response information is displayed higher. Further, in the communication area of the bird's-eye view 16 shown in FIG. 19, the CPU 21 displays the group 17 with a higher response degree larger.

[0206] On the other hand, FIG. 20 is a diagram showing a display example of the bird's-eye view 16 displayed on the desktop computer 32. Even if the user terminals 2 are both the same desktop computer 32, the CPU 21 may display the dashboard of the bird's-eye view 16 shown in FIG. 5, for example, as large as the dashboard shown in FIG. 20 on the desktop computer 32 with a wider display range of the screen. In the communication area of the bird's-eye view 16 shown in FIG. 20, the CPU 21 displays the group 17 with a higher response degree larger on the screen and also displays it closer to the center of the communication area than the other groups 17.

[0207] As described above, one aspect of the chat server 6 in the information processing system 1 has been described using the embodiments. However, the disclosed form of the chat server 6 is an example, and the form of the chat server 6 is not limited to the scope described in the embodiments. Various changes or improvements can be made to the embodiments without departing from the gist of the present disclosure, and the forms with such changes or improvements are also included in the technical scope of the disclosure. For example, without departing from the gist of the present disclosure, the order of the nuance processing shown in FIGS. 8 and 11, the bird's-eye view display processing shown in FIG. 12, the communication display processing shown in FIG. 13, the dashboard display processing shown in FIG. 14, the notification display processing shown in FIG. 15, and the history view display processing shown in FIG. 18 may be changed.

[0208] In the above-described embodiment, as an example, the forms in which each of the above-described processes is realized by software have been described. However, the processes equivalent to the flowcharts of FIGS. 8, 11 to 15, and 18 may be processed by hardware. In this case, the processing speed can be increased as compared with the case where each process is realized by software.

[0209] In the above-described embodiment, the processor refers to a processor in a broad sense, and includes a general-purpose processor (for example, CPU 21) and a dedicated processor (for example, GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0210] Also, the operation of the processor in the above-described embodiment may be achieved not only by one processor but also by a plurality of physically separated processors cooperating with each other. Also, the order of each operation of the processor is not limited to the order described in the above-described embodiment, and may be changed as appropriate.

[0211] In the above-described embodiment, an example in which the information processing program is stored in the ROM 22 has been described. However, the storage destination of the information processing program is not limited to the ROM 22. The information processing program of the present disclosure can also be provided in a form recorded on a storage medium readable by the computer 20. For example, the information processing program may be provided in a form recorded on an optical disk such as a CD-ROM (Compact Disk Read Only Memory) and a DVD-ROM (Digital Versatile Disk Read Only Memory). Further, the information processing program may be provided in a form recorded on a portable semiconductor memory such as a USB (Universal Serial Bus) memory and a memory card. The ROM 22, the non-volatile memory 24, the CD-ROM, the DVD-ROM, the USB, and the memory card are examples of non-transitory storage media.

[0212] Furthermore, the chat server 6 may download the information processing program from an external device connected to the Internet 3 and store the downloaded information processing program in the storage device. In this case, the CPU 21 of the chat server 6 reads the information processing program downloaded from the external device and executes each process.

Explanation of Reference Numerals

[0213] 1 Information processing system, 2 User terminal, 2A Input device (of user terminal), 3 Internet, 4 Web server, 6 Chat server, 6A Chat control unit, 6B Display switching unit, 7 Camera, 8 Data server, 8A User data DB, 8B Classification data DB, 9 Microphone, 10 Chat room, 10A Button, 11 Speech bubble selection dialog, 13(13A, 13B, 13C) Mark, 14(14A, 14B, 14C, 14D, 14E, 14F, 14G, 14H) Icon, 16 Overhead view, 17 Group, 18 History view, 20 Computer, 21 CPU, 22 ROM, 23 RAM, 24 Non-volatile memory, 25 I / O, 26 Bus, 27 Communication unit, 28 Input unit, 29 Display unit, 30 Smartphone, 32 Desktop computer

Claims

1. An information processing apparatus comprising a processor, wherein the processor obtains user data posted in a chat room participated by a plurality of users, and by selecting an icon representing a reaction that a user who views the obtained user data can take, which is displayed together with the obtained user data, at least one piece of response information for the user data posted in the chat room is associated with the user who posted the user data and the posting time of the user data, and the associated information is switched and displayed in a plurality of display forms.

2. The processor classifies each of the user data into a plurality of categories using at least one of sentiment information added to the user data and response information for the user data, and displays the associated information in different display forms for each of the classified categories. The information processing apparatus according to claim 1.

3. In the category including the user data having the response information, the processor divides a series of response information starting from the common user data into a group forming one communication, and changes the display form for each group according to the degree of response of the response information to the user data serving as the starting point. The information processing apparatus according to claim 2.

4. The processor obtains the display attributes of the information devices used by the users participating in the chat room, and changes the display form of the associated information classified for each category according to the obtained display attributes of the information devices for each user. The information processing apparatus according to claim 2 or claim 3.

5. The processor extracts the user data posted in the chat room according to the conditions specified by the user, and uses the extracted user data to display the association information of each user. The information processing apparatus according to any one of claims 1 to 4.

6. The processor displays a list screen that displays the user data for each user extracted along the time series, and generates a report that records the activity status of the user in a specific period from the list screen for each user. The information processing apparatus according to claim 5.

7. In the list screen, the processor displays the user data so that the communication partner based on the user data can be understood. The information processing apparatus according to claim 6.

8. The processor estimates the user's emotion from the voice uttered by the user, divides the voice at the boundary where the type of the estimated emotion changes, and generates the user data with emotion information representing the estimated user emotion added to each of the divided voices. The information processing apparatus according to any one of claims 1 to 7.

9. The processor changes the emotion information added to the user data to the emotion information specified by the user. The information processing apparatus according to claim 8.

10. The processor re-divides the user data generated by dividing the voice at the division position specified by the user and displays it in the chat room. The information processing apparatus according to claim 8 or claim 9.

11. The processor estimates the user's emotion by combining the emotion of the user estimated from the voice and the emotion of the user estimated from the user image capturing the user who uttered the voice. The information processing apparatus according to any one of claims 8 to 10.

12. The processor uses a speech bubble associated with emotion information representing the estimated emotion of the user to display the user data in the chat room. The information processing apparatus according to any one of claims 8 to 11.

13. Causing a computer to acquire user data posted in a chat room in which a plurality of users participate, and execute a process of switching and displaying association information in which at least one of response information for the user data posted in the chat room, which is associated with the user who posted the user data and the posting time of the user data, is associated with a plurality of display forms, by selecting an icon representing a possible reaction of a user who has viewed the user data and the user data to be displayed together with the user data. An information processing program.

Images