Program, storage medium, information processing device, and control method

Abstract

To enable confirmation of content of a voice message even when the voice message within a chat room cannot be reproduced.SOLUTION: An information processing apparatus that comprises display means that can display a voice message which is transmitted and received within a chat room comprises: communication means that transmits and receives the voice message within the chat room in which chats are exchanged by a plurality of users; text-making means that converts content of the voice message into a text; summarization means that summarizes a character string in which text-making is performed by the text-making means; and display control means that associates summarization content summarized by the summarization means with the voice message to display the chat room on the display means.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present invention relates to an information processing device capable of displaying voice messages transmitted and received in a chat room, a control method thereof, and a program. [Background technology]

[0002] In recent years, communication between users using chat systems has become widespread. In such chat systems, a service is provided that enables communication by sending voice messages, not just characters, on the sending side and playing them on the receiving side. The technology disclosed in Patent Document 1 is a device that handles voice data by converting input voice data into text data using a recognition processing unit, and displays the text data as recognizable characters on a display unit (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-110911 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the voice message displayed in the chat window of the chat system is only an icon display for playback, and there is a problem that the content cannot be confirmed until the voice is played. For example, in a place where sound output is prohibited, the message cannot be confirmed immediately. On the other hand, there is also a problem that visibility is poor if all voice data is converted into characters and displayed in the chat system, as in the device disclosed in Patent Document 1.

[0005] SUMMARY OF THE PRESENTLY PREFERRED EMBODIMENT An object of the present invention is to enable a user to check the contents of a voice message in a chat room even when the message cannot be reproduced. [Means for solving the problem]

[0006] In order to achieve the above object, one aspect of the present invention is an information processing device having a display means capable of displaying voice messages transmitted and received in a chat room, A communication means for transmitting and receiving voice messages in a chat room where a plurality of users chat; a text conversion means for converting the contents of the voice message into text; A summarizing means for summarizing the character string converted into text by the text conversion means; The chat room display device further comprises a display control means for displaying the summary content summarized by the summarizing means on a display means for displaying the chat room, the summary content being linked to the voice message. Effect of the Invention

[0007] According to the present invention, even if a voice message in a chat room cannot be reproduced, the content of the message can be confirmed. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a system configuration diagram according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram of a chat application server according to the present embodiment. [Diagram 3] FIG. 2 is a block diagram of a mobile terminal according to the embodiment. [Figure 4] FIG. 2 is an explanatory diagram of an operation screen of a chat application in the present embodiment. [Diagram 5] 10 is a flowchart illustrating an operation of a chat server in the present embodiment. [Figure 6] 6 is a flowchart illustrating a receiving operation of a mobile terminal according to the present embodiment. [Figure 7] 10 is a flowchart illustrating a message display operation of the mobile terminal according to the embodiment. [Figure 8] 10 is a flowchart illustrating a read information receiving operation of the mobile terminal according to the present embodiment. [Figure 9] 11 is an explanatory diagram of a chat room management table in the present embodiment. [Figure 10] FIG. 2 is an explanatory diagram of the sequence of the entire chat system in this embodiment. [Figure 11] 5 is a flowchart illustrating the operation of the mobile terminal in the first embodiment. [Figure 12] FIG. 2 is an explanatory diagram of a display of a chat room in the first embodiment. [Figure 13] FIG. 4 is an explanatory diagram of a voice message management table in the first embodiment. [Figure 14] 10 is a flowchart illustrating the operation of a mobile terminal in the second embodiment. [Figure 15] FIG. 11 is an explanatory diagram of a display screen of a chat room in the second embodiment. [Figure 16] FIG. 11 is an explanatory diagram of a voice message management table in the second embodiment. [Figure 17] 13 is a flowchart illustrating an operation according to a heading setting of a mobile terminal in the second embodiment. [Figure 18] 13 is a flowchart illustrating a search operation of a mobile terminal in the third embodiment. [Figure 19] FIG. 13 is an explanatory diagram of a display screen of a chat room in the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments of the present invention will be described in detail with reference to the drawings. However, the configurations described in the following embodiments are merely examples, and the scope of the present invention is not limited to the configurations described in the embodiments. First, a first embodiment of the present invention will be described.

[0010] First Embodiment <System configuration> 1 is a diagram showing an example of a system configuration according to an embodiment of the present invention. This system has a communication base station 105 and a chat application server 107, which are connected to each other via the Internet 106 so that they can communicate with each other. The communication base station 105 communicates required information with a mobile terminal 102 owned by a sender 100 and a mobile terminal 104 owned by a receiver 103.

[0011] A sender 100 sends a "voice message" using a chat application installed in a mobile terminal 102. Next, the "voice message" sent by the sender 100 is transmitted to a chat application server 107 via a communication base station 105 and the Internet 106. Next, the chat application server 107 performs a predetermined process on the received "voice message" and transmits it to a destination. Then, the "voice message" sent by the chat application server 107 is transmitted to a mobile terminal 104 owned by a receiver 103 via the Internet 106 and the communication base station 105.

[0012] <Hardware configuration of chat application server 107> 2 is a diagram showing an example of the configuration of chat application server 107 according to this embodiment. Chat application server 107 includes control unit 201, storage unit 202, and network I / F 211, which are connected via system bus 210 so as to be able to transmit and receive required information to and from each other.

[0013] <Control unit 201> The control unit 201 reads out and executes a control program 203 stored in a non-volatile manner in the storage unit 202. This realizes various functions required for the chat application server 107. The control unit 201 is configured to have one or more processors (CPU, DSP, etc.). The control unit 201 also includes a chat processing unit 209, and performs overall control of each device connected to a system bus 210. The chat processing unit 209 interprets and responds to messages received from a chat application of the mobile terminal 102. In this way, the chat application server 107 has an automatic dialogue function and also functions as a "chatbot."

[0014] <Storage section 202> The storage unit 202 is used as an internal storage. The storage unit 202 stores a control program 203, text data 204, image data 205, audio data 206, video data 207, video data 207, registered user data 208, system software, etc. The storage unit 202 is configured with storage devices such as an HDD (hard disk drive), an SSD (solid state drive), a RAM, and a ROM.

[0015] Text data 204 is text data of a message posted by a user or a chatbot on the chat. Image data 205 is image data posted by a user on the chat. Audio data 206 is audio data of an audio message posted by a user on the chat. Video data 207 is video data of a video message posted by a user on the chat. Registered user data 208 is list information of combinations of user IDs and passwords required for each user to log in to the chat application.

[0016] <Network I / F> The network I / F 211 is an interface for performing network communication by being connected to the Internet 106 via a LAN cable or the like. For example, a well-known network card or the like can be used.

[0017] <Configurations of the Mobile Terminal 102 and the Mobile Terminal 104> 3 is a block diagram of the mobile terminals 102 and 104. The mobile terminals 102 and 104 are examples of information processing terminals with communication functions, and are terminals that can be used anywhere due to their wireless communication functions and other features. Examples of such terminals include smartphones, tablet terminals, and notebook PCs.

[0018] Both mobile terminals 102, 104 have the same configuration, and the configuration of one of them, mobile terminal 102, will be described. Mobile terminal 102 has a system bus 308. To system bus 308, a control unit 301, a storage unit 302, an input / output unit 303, a display unit (display means) 304, a microphone 305, a speaker 306, and a network I / F 307 are connected. Each device connected to system bus 308 is capable of transmitting and receiving required information to and from each other.

[0019] The control unit 301 reads and executes a control program stored in a non-volatile manner in the storage unit 302. This realizes various functions required for the mobile terminal 102. The control unit 201 is configured to have one or more processors (CPU, DSP, etc.). The control unit 301 also performs overall control of each device connected to a system bus 308.

[0020] The storage unit 302 is used as an internal storage. The storage unit 302 stores text data, audio data, image data, system software, etc. The storage unit 302 is configured with storage devices such as a hard disk drive (HDD), a solid state drive (SSD), a RAM, and a ROM.

[0021] The input / output unit 303 is composed of an LCD touch panel or the like, and acquires information operated by a user and sends the acquired information to the control unit 301. The input / output unit 303 also outputs the processing result of the control unit 301. Note that operation input from the user may be realized by a hardware input device such as a switch or a keyboard. The method for detecting input from the touch panel may be a general detection method such as a resistive film method, an infrared method, an electromagnetic induction method, or a capacitive method.

[0022] The display unit 304 performs display according to image data. The display unit 304 can also display an operation screen and provide a user interface (UI) to the user. The microphone 305 is for inputting voice data. The speaker 306 is for outputting voice data. In this embodiment, a speaker built into the mobile terminal 102 is used for voice output. Alternatively, the control unit 301 may send voice to a voice output device such as an external earphone or speaker connected from outside to the mobile terminal 102 and output the voice. The network I / F 307 is an interface for connecting to the Internet 106 and performing network communication.

[0023] <Operation screen within a chat application on a mobile device> Fig. 4(a) is an explanatory diagram of an operation screen of a chat application running on the mobile terminals 102 and 104. At this time, the chat application is performing the required communication with the chat application server 107. On the operation screen of Fig. 4(a), messages 401 and 402 of users A and B are displayed vertically downward on the screen in chronological order. In this embodiment, the message 402 of the operator of the mobile terminal is placed on the right side of the display screen, but the display position of the user's message may be placed on the left side of the display screen.

[0024] The display screen shown in FIG. 4(a) also displays the word "read" (403) to determine whether or not user B has viewed the message sent by user A. Also, as shown in FIG. 4(s), a message input field 404 is displayed at the bottom of the display screen. In the message input field 404, text can be input or image data stored in the storage unit 302 can be selected and sent to the chat server. This sends the message to user B.

[0025] FIG. 4(b) is an explanatory diagram of an operation screen of a chat application including a voice message.

[0026] The voice message 405 is displayed separately from the text message, and a button (button icon) 406 for playing the voice is displayed. When a voice playback instruction is received by a user pressing the button, the control unit 301 of the mobile terminal 102 or 104 controls the voice playback. As a result, the played voice is output from the speaker 306.

[0027] 4(c) is an explanatory diagram of an operation screen for inputting a voice message. When the voice message input field 407 is pressed, the chat application starts recording voice. The user inputs voice into the microphone 305 of the mobile terminal 102 or 104. To end input, the user presses the voice message input field again. The chat application stops recording voice and stores the voice message data in the storage unit 302. Thereafter, the voice message is sent to the recipient by sending it to the chat server at the user's instruction.

[0028] <Figure 5: Operation of chat application server 107> Next, the operation of message transmission / reception processing in chat application server 107 will be described with reference to Fig. 5. The operation of the flowchart shown in Fig. 5 is realized by control unit 201 (CPU) reading out control program 203 stored in storage unit 202 into RAM or the like and executing it. First, in S501, control unit 201 holds a chat room including a plurality of users. Next, in S502, control unit 201 receives a message from a chat application of mobile terminal 102 (message sending terminal).

[0029] Next, in S503, the control unit 201 stores the received message in the storage unit 202. Then, in S504, the control unit 201 transmits the received message to another user in the chat room. In this manner, the operation of the chat application server 107 is performed.

[0030] <Figure 6: Processing on a mobile device when receiving a message> Next, the operation of mobile terminal 104 when receiving a message will be described with reference to Fig. 6. The operation of the flowchart shown in Fig. 6 is realized by control unit 301 (CPU) reading a program stored in storage unit 302 into RAM or the like and executing it. First, in S601, control unit 301 receives a message from chat application server 107. Next, in S602, control unit 301 notifies the corresponding chat room of the message based on a chat room ID that identifies the chat room included in the message.

[0031] Next, in S603, the control unit 301 determines whether or not it is necessary to display on a UI (user interface) that the message has been received and to notify the user. Note that whether or not to notify the user is set in advance by the user, and the setting is stored and saved in the storage unit 302.

[0032] Then, in S603, if the control unit 301 determines that a notification is necessary (Yes), the process proceeds to S604. On the other hand, if the control unit 301 determines that a notification is not necessary (No), the process ends. Next, in S604, the control unit 301 creates a notification. Then, in S605, the control unit 301 displays the notification on the display unit 304, and notifies the user by sound using the speaker 306 according to the settings. In this manner, the mobile terminal 104 that has received the message operates.

[0033] <Figure 7: Processing on a mobile device when a message is displayed> Next, the operation of mobile terminal 104 when displaying a message in a chat room will be described with reference to Fig. 7. The operation of the flowchart shown in Fig. 7 is realized by control unit 301 reading a program stored in storage unit 302 into RAM or the like and executing it. First, in S701, control unit 301 accepts display of a chat room from a user. Next, in S702, control unit 301 displays the selected chat room on display unit 304. Specifically, the chat room is displayed as a UI.

[0034] Next, in S703, the control unit 301 stores the most recent of the displayed unconfirmed messages as a "read message" in the storage unit 302. Then, in S704, the control unit 301 transmits the read message stored in the storage unit 302 to the chat application server 107. In this manner, the processing of the mobile terminal when a message is displayed is executed.

[0035] <Figure 8: How a mobile device receives read information> Next, a read information receiving operation in the mobile terminal 102 will be described with reference to Fig. 8. The operation of the flowchart shown in Fig. 8 is realized by the control unit 301 reading a program stored in the storage unit 302 into a RAM or the like and executing the program.

[0036] First, in S801, control unit 301 receives read information from chat application server 107. Next, in S802, control unit 301 updates the read information of the corresponding chat room in chat management table 901 based on the chat room ID that identifies the chat room included in the read information. The read information receiving operation is performed by the above operations.

[0037] <Figure 9: Chat management table> 9 is an explanatory diagram of an example of the "chat management table" updated in S802. The chat management table 901 is saved in the storage unit 302. The chat management table 901 stores chat room IDs (chat room identifiers) and read information in association with each other.

[0038] That is, chat management table 901 manages information for each chat room, and holds read information 902 indicating how much of a message has been read by the recipient in each chat room. Based on this information, read information 403 is displayed when the chat room is displayed. For example, in chat room "A," read information 902 is "100," which indicates that "100" messages have been read.

[0039] <Figure 10: Sequence of the entire chat system> Next, the chat application server 107, the mobile terminal 104, and the sequence of the entire chat system using the mobile terminal 104 will be described with reference to Fig. 10. First, the sender 100 operates the mobile terminal 104 to send a "message" to the chat application server 107. In response to this, the chat application server 107 adds the "message" to the chat management table 901, and sends the "message" to the mobile terminal 104 of the receiver 103.

[0040] Next, the receiver 103 operates the mobile terminal 104 to display the chat room and check the "message." In response to this, the mobile terminal 104 transmits "read information" to the chat application server 107. The chat application server 107 then transmits the "read information" to the mobile terminal 102. This allows the sender 100 to recognize that the "message" has been checked. This is the sequence of the entire chat system.

[0041] <Figure 11: Voice message heading creation operation, etc.> Next, a process of automatically creating and displaying a heading for a voice message in a chat room of mobile terminal 104 will be described with reference to Fig. 11. The operation shown in the flowchart of Fig. 11 can be realized by control unit 301 reading a program stored in storage unit 302 into RAM or the like and executing it. First, in S1101, control unit 301 accepts a display of a chat room from a user. Next, in S1102, control unit 301 displays the selected chat room on display unit 304.

[0042] Next, in S1103, the control unit 301 stores the most recent of the displayed unconfirmed messages as a read message in the storage unit 302. Next, in S1104, the control unit 301 transmits the read messages stored in the storage unit 302 to the chat application server 107.

[0043] Next, in S1105, the control unit 301 performs voice analysis of the voice message to convert it into text. Next, in S1106, the control unit 301 summarizes the characters converted into text in S1105. Then, in S1107, the control unit 301 displays the "text" created in S1105 and the "summary characters" created in S1106 in association with the voice message in the chat room. At this time, as shown in FIG. 13, the control unit 301 stores it in the storage unit 302 as a voice message management table 1301, and displays a heading 1305 in association with the voice message 1303 on the display unit 304. Note that, whether the voice message is one that the user has sent or one that the user has received from the other party, it is possible to display the voice message in association with a "heading (summary content)" according to the operation of FIG. 11 (see FIG. 12 described later).

[0044] <Figure 13: Voice message management table 1301> 13, a voice message management table 1301 stores chat room IDs 1302, voice message IDs 1303, text 1304, and headings 1306 in association with each other. For example, chat room ID 1302 "A" is associated with voice message IDs "1" to "3." Each voice message ID "1" to "3" is linked with text 1304, which is the voice message converted to text, and its heading 1308.

[0045] For example, the text 1304 of the chat room ID 1302 "A" and the voice message ID 1303 "3" is "Got it," and the heading 1308 is "Got it."

[0046] <FIG. 12: Chat room display example in the first embodiment> FIG. 12 shows an example of the display of a chat room in which the above processing has been executed. The "voice message" sent by "A" is displayed with a heading 1201 linked to it. The manner of linking is such that the corresponding heading 1201 is displayed below the voice message, and there are no other text messages, voice messages, etc. between the two. The manner of linking is not limited to this, and the two may be displayed to be linked by a display such as a "string." Furthermore, the "voice message" sent by "B" is displayed with a heading 1203 attached. Furthermore, the transcribed text message (voice message converted into text) sent by "A" is displayed with a "text" display 1202 attached to it, indicating that the message has been transcribed, so that it can be distinguished from other text messages.

[0047] Also, by displaying the heading 1203 (summary) associated with the "voice message" in a manner different from that of a normal text message displayed in the chat room, it is possible to make the heading 1201 distinguishable. More specifically, the heading 1201 (summary) associated with the "voice message" may be displayed in a manner that highlights the heading 1201 (summary) in a manner that includes characters and colors different from those of a normal text message displayed in the chat room.

[0048] <Summary of the First Embodiment> In the first embodiment, the "text" to be displayed is explained as a "summary," but it is also possible to display only the first sentence of the "voice message" in association with it. In this embodiment, a case has been explained in which the mobile terminal 104 executes the processes subsequent to converting the voice message into text. The content of the "summary" can be, for example, a predetermined number of characters from the beginning of the converted text, or the date, time, and place described in the character string.

[0049] In addition, the transcribed text message may be easily distinguished from other text messages by displaying a display 1202 indicating that the message has been transcribed, or by highlighting the message in a manner that includes the text and color. As described above, by displaying a "summary" of the voice message, the content of the voice message can be confirmed even in situations where the voice message cannot be played back.

[0050] The above provides the following configuration. First, the control unit 301 (communication means) transmits and receives voice messages in a chat room where multiple users chat. Next, the control unit 301 (text conversion means, summarization means) converts the contents of the voice messages into text and summarizes the text strings. Then, the control unit 301 (display control means) links the summary contents to the voice messages and displays them in the chat room.

[0051] Alternatively, a request may be sent to have chat application server 107 process "text" or "summary", information on the "summary" may be received from chat application server 107, and the summarized characters may be displayed on mobile terminal 102 in S1107. That is, control unit 301 (request sending means) sends information indicating a request for summarization of a voice message to chat application server 107 (external server) capable of summarization that generates a summary of a character string obtained by converting a voice message into text. In response to this, control unit 301 (receiving means) receives the summarization information sent by chat application server 107 (external server). Then, control unit 301 (display control means) may be configured to link the summary content indicated by the received summarization information to the voice message and display it in the chat room. In this way, chat application server 107 with high processing power can generate a highly accurate summary and link it to the voice message.

[0052] <Second embodiment> In the first embodiment, a method for displaying a summarized heading for a voice message is described. In the second embodiment, a mode in which the "presence" or absence" of the heading for a voice message is switched depending on the setting at the time of sending the voice message is described.

[0053] <Figure 14: Setting whether to display headings> An operation of posting a "voice message" with a setting for whether or not to display a heading in a chat room of the sending mobile terminal 102 will be described with reference to Fig. 14. The operation of the flowchart shown in Fig. 14 is realized by the control unit 301 reading a program stored in the storage unit 302 into a RAM or the like and executing it.

[0054] First, in S1401, control unit 301 accepts display of a chat room from the user. Next, in S1402, control unit 301 displays the selected chat room on display unit 304. Next, in S1403, control unit 301 accepts a voice input instruction from the user.

[0055] Next, in S1404, the control unit 301 displays on the display unit 304 a voice input field 1501 in which a dedicated button such as the heading setting button 1502 (see FIG. 15) is displayed. The state of the heading setting set by the heading setting button 1502 is held as indicated by reference numeral 1606 in the "voice message management table 1601" shown in FIG. 16, and is stored in the memory unit 302. Next, in S1405, the control unit 301 starts recording voice. Then, in S1406, the control unit 301 transmits the recorded voice message and the voice message management table 1601 to the chat application server 107.

[0056] <Figure 16: Voice message management table 1601> 16 is an explanatory diagram of a voice message management table 1601 in the second embodiment. The voice message management table 1601 stores a chat room ID 1602, a voice message 1603, a text 1604, a heading 1606, and a heading setting 1606 in association with each other. As can be seen by comparing with the voice message management table 1301 in the first embodiment in FIG. 13, the voice message management table 1601 associates a new "heading setting 1606" with a voice message in the voice message management table 1301. The heading setting 1606 can be set to either "ON" or "OFF", with "ON" being a setting for displaying a heading and "OFF" being a setting for not displaying a heading. For example, the heading setting 1606 is set to "ON" for a chat room ID 1602 of "A" and a voice message ID 1603 of "2".

[0057] <Figure 17: Behavior according to heading settings> Next, with reference to Fig. 17, an operation of displaying a heading of a voice message according to a heading setting in a chat room of the receiving mobile terminal 104 will be described. The operation shown in the flowchart of Fig. 17 can be realized by control unit 301 reading a program stored in storage unit 302 into RAM or the like and executing it. First, in S1701, control unit 301 accepts display of a chat room from a user. Next, in S1702, control unit 301 displays the selected chat room on display unit 304. Next, in S1703, control unit 301 stores the most recent of the displayed unconfirmed messages in storage unit 302 as a read message.

[0058] Next, in S1704, the control unit 301 transmits the read messages stored in the storage unit 302 to the chat application server 107. Next, in S1705, the control unit 301 determines whether or not a heading display setting is present from the voice message management table 1601 received from the chat application server 107. If the control unit 301 determines that the heading setting is "ON" (Yes), the process proceeds to S1706. If the control unit 301 determines that the heading setting is "OFF" (No), the operation ends.

[0059] Next, in S1706, the control unit 301 performs voice analysis on the voice message to convert it into text. Next, in S1707, the control unit 301 summarizes the text. Then, in S1708, the control unit displays the characters summarized in S1707 on the display unit 304 as a text message 1201 in the same way as a normal message.

[0060] <Summary of the second embodiment> To summarize the second embodiment, first, a heading setting button 1502 (heading presence / absence setting means) is further provided for setting whether or not a summary is to be attached to a voice message when the voice message is sent. Then, the control unit 301 performs display control of the summary linked to the voice message in the chat room based on the setting content of the heading setting button 1502 (heading presence / absence setting means).

[0061] As described above, the heading (summary) of a voice message can be displayed or not by using the "heading setting" which is information indicating the heading display setting mode added when the voice message is transmitted.

[0062] <Third embodiment> In the third embodiment, since there may be cases where the user wishes to search the summarized text later, an operation example will be described for the case where a search is performed on a character string that has been summarized and displayed.

[0063] An operation of searching for a voice message in the mobile terminal 102 will be described with reference to Fig. 18. The operation shown in the flowchart of Fig. 18 can be realized by the control unit 301 reading a program stored in the storage unit 302 into a RAM or the like and executing it.

[0064] First, in S1801, the control unit 301 receives a search instruction from the user. Next, in S1802, the control unit 301 displays a search bar 1901 shown in Fig. 19 on the display unit 304. As shown in Fig. 19, the search bar 1901 is a horizontally long area at the top of the display screen, but is not limited to this position.

[0065] Next, in S1803, the control unit 301 accepts an input of a search character string from the user. Next, in S1804, the control unit 301 searches the text 1304 of the voice message management table 1301 for a character string that matches the character string accepted in S1804.

[0066] Next, in S1805, the control unit 301 judges whether or not a matching character string exists in S1804. If the control unit 301 judges that a matching character string exists (YES), the process proceeds to S1806. On the other hand, if the control unit 301 judges that a matching character string does not exist (No), the operation ends. Then, in S1806, the control unit 301 displays the corresponding voice message 1902 on the display unit 304.

[0067] The search results displayed on the display unit 304 may be not only voice message 1902 but also text 1903, heading 1904, and voice message 1905 which is an extracted voice around the search string. Fig. 19(a) is an example in which voice message 1902 is displayed as a search result, and Fig. 19(b) is an example in which text 1903 is displayed as a search result. Fig. 19(c) is an example in which heading 1904 is displayed as a search result, and Fig. 19(d) is an example in which "voice message 1905 which is an extracted voice around the search string" is displayed as a search result. Note that a search is not limited to text only, and a search may be made of text and summary content, or text or summary content.

[0068] <Summary of the third embodiment> To summarize the third embodiment, first, the control unit 301 (reception means) accepts input of a search string in a search bar 1901 displayed in a chat room. Then, the control unit 301 (search means) searches for a text string obtained by converting a voice message into text and / or a summarized summary using the accepted string as a search key. As described above, when searching for a voice message, by converting the voice message into text (characterization), a character search of the voice message can be performed.

[0069] <Other> (1) In the above explanation, an example of chatting between mobile terminal devices has been explained, but the above-mentioned operation can be realized even when, for example, PCs such as desktop PCs and notebook PCs are used, or when a PC is used between a mobile terminal and a PC. In this case, the PC may be connected to a router by a LAN cable or may be connected to a wireless router. The notebook PC may be portable. However, the PC must have an application program (AP) according to the present invention installed, and must have a built-in or external microphone for voice input and speaker for voice output. (2) The chat room itself may be displayed as a three-dimensional image, and the text may also be displayed in three dimensions, so that the chat room and the text character strings can be recognized three-dimensionally by wearing special glasses or special goggles. In this case, the user's own avatar and the other party's avatar are set in advance and displayed at appropriate positions, so that the user can enjoy the chat even more. (3) The user decides a color for each chat content (for friends, for work, for a certain meeting, etc.). Then, keywords (date and time, place, etc.) are extracted from the text of the voice message, and the control unit 301 determines the chat content and displays the text character strings in the corresponding color. This allows the user to get a rough idea of ​​the content.

[0070] (4) By performing a specific operation such as a long press on the triangular play icon of the voice message, the text string can be displayed in the display area of ​​the voice message without outputting the voice. At this time, if the string does not fit in the display area, the string can be displayed by sliding the string in sequence so that it can be seen. (5) If a contract is made with the operator of the chat application server 107 to always convert voice messages into text, the various processes described above can be realized with high accuracy on a mobile terminal without having to request the chat application server 107 every time.

[0071] Although the preferred embodiment of the present invention has been described above, the present invention is not limited to the above-mentioned embodiment, and various modifications and changes are possible within the scope of the gist of the present invention. For example, the present invention can be realized by supplying a program that realizes one or more functions of the above-mentioned embodiment to a system or device via a network or a recording medium, and having a processor of a computer in the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0072] 102 Mobile devices 104 Mobile Devices 105 Telecommunications Base Station 106 Internet 107 Chat App Server 201 Control section 202 Storage section 203 Control Program 209 Chat Processing Unit 301 Control Unit 302 Storage section 303 Input / output section 304 Display section 305 Mike 306 Speaker 308 System Bus 404 Message input field 901 Chat Management Table 1301 Voice Message Management Table 1601 Voice Message Management Table

Claims

1. A program for causing a computer to execute a control method for an information processing device capable of receiving a voice message, comprising: The control method includes: a text conversion step of converting the content of the received voice message into text; a summarization step of summarizing the text converted by the text conversion step into a summary text; a first display step of displaying identification information corresponding to the sender of the voice message and an object that performs an operation of reproducing the voice message; a second display step of displaying an object indicating that the voice message has been converted into the summary text, and the summary text.

2. 2. The program according to claim 1, wherein the summary text displayed by the second display step is displayed in a format different from that of a normally displayed text message.

3. 3. The program according to claim 2, wherein the summary text displayed by the second display step is displayed in a display format including characters and colors different from those of a normally displayed text message.

4. a caption presence / absence setting step for setting whether or not a summary text obtained by the text conversion step and the summarization step is to be attached to the voice message when the voice message is transmitted; The second display step includes:

2. The program according to claim 1, wherein the summary text is displayed based on the setting contents of the heading setting step.

5. The summarizing step includes: The program according to claim 1, characterized in that the content of the voice message is summarized based on a predetermined number of character strings from the beginning of the text converted by the text conversion process, or based on the date, time, and location described in the character strings.

6. 5. The program according to claim 4, wherein an object for accepting the step of setting whether or not a heading is present is displayed.

7. a search reception step of receiving an input of a character string for searching the summary text summarized by the summarization step; 2. The program according to claim 1, further comprising a search step of searching the summary text based on the character string received in the search receiving step.

8. a sending step of sending the voice message to an external server and causing the external server to execute the text conversion step and the summarization step; a receiving step of receiving the summarized text obtained by the text generation step and the summarization step from the external server, The second display control step includes:

2. The program according to claim 1, wherein the summary text received in the receiving step is displayed.

9. The program described in Claim 1, characterized in that the information device is a mobile terminal including a smartphone or tablet terminal.

10. The program described in Claim 1, characterized in that the summary text is displayed below an object that performs the action of playing the audio message.

11. The program described in Claim 1, characterized in that the object indicating that the voice message has been converted into the summary text is a string.

12. A storage medium storing a program described in any one of claims 1 to 11.

13. An information processing device capable of receiving a voice message, a text conversion means for converting the content of the received voice message into text; a summarizing means for summarizing the text converted by the text conversion means into a summary text; a first display means for displaying identification information corresponding to the sender of the voice message and an object for performing an operation of reproducing the voice message; An information processing apparatus comprising: an object indicating that the voice message has been converted into the summary text; and second display means for displaying the summary text.

14. A method for controlling an information processing device capable of receiving a voice message, comprising: The control method includes: a text conversion step of converting the content of the received voice message into text; a summarization step of summarizing the text converted by the text conversion step into a summary text; a first display step of displaying identification information corresponding to the sender of the voice message and an object that performs an operation of reproducing the voice message; A method for controlling an information processing apparatus, comprising: a second display step of displaying an object indicating that the voice message has been converted into the summary text, and the summary text.

Images