Program, storage medium, control method for information processing device, and information processing device
Patent Information
- Application Number
- JP2023025959
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-02-22
- Publication Date
- 2025-10-22
Smart Images
Figure 00000000_0000_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device capable of displaying voice messages transmitted and received in a chat room, a control method thereof, and a program. [Background technology]
[0002] In recent years, communication between users using chat systems has become widespread. In such chat systems, a service is provided that enables communication by sending voice messages, not just characters, on the sending side and playing them on the receiving side. The technology disclosed in Patent Document 1 is a device that handles voice data by converting input voice data into text data using a recognition processing unit, and displays the text data as recognizable characters on a display unit (see Patent Document 1). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2021-110911 A Summary of the Invention [Problem to be solved by the invention]
[0004] However, the voice message displayed in the chat window of the chat system is only an icon display for playback, and there is a problem that the content cannot be confirmed until the voice is played. For example, in a place where sound output is prohibited, the message cannot be confirmed immediately. On the other hand, there is also a problem that visibility is poor if all voice data is converted into characters and displayed in the chat system, as in the device disclosed in Patent Document 1.
[0005] SUMMARY OF THE PRESENTLY PREFERRED EMBODIMENT An object of the present invention is to enable a user to check the contents of a voice message in a chat room even when the message cannot be reproduced. [Means for solving the problem]
[0006] In order to achieve the above-mentioned object, one aspect of the present invention is an information processing device having a display means capable of displaying voice messages transmitted and received within a chat room, and is characterized in that it comprises: a communication means for transmitting and receiving voice messages in a chat room where chats are exchanged between multiple users; a text conversion means for converting the contents of the voice message into text; an instruction means for giving an instruction to convert the contents of the voice message into text; and a control means for, when an instruction is given by operating the instruction means, displaying a string of characters resulting from the content of the voice message being converted into text by the text conversion means on the display means which displays the chat room in a manner different from that of normal text messages in the chat room. Effect of the Invention
[0007] According to the present invention, even if a voice message in a chat room cannot be reproduced, the content of the message can be confirmed. [Brief description of the drawings]
[0008] [Figure 1] FIG. 1 is a system configuration diagram according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram of a chat application server according to the present embodiment. [Diagram 3] FIG. 2 is a block diagram of a mobile terminal according to the embodiment. [Figure 4] FIG. 2 is an explanatory diagram of an operation screen of a chat application in the present embodiment. [Diagram 5] 10 is a flowchart illustrating an operation of a chat server in the present embodiment. [Figure 6] 6 is a flowchart illustrating a receiving operation of a mobile terminal according to the present embodiment. [Figure 7] 6 is a flowchart illustrating a message display operation of the mobile terminal according to the embodiment. [Figure 8] 10 is a flowchart illustrating a read information receiving operation of the mobile terminal according to the present embodiment. [Figure 9] FIG. 4 is an explanatory diagram of a chat room management table in the present embodiment. [Figure 10] FIG. 2 is an explanatory diagram of the sequence of the entire chat system in this embodiment. [Figure 11] 5 is a flowchart illustrating the operation of the mobile terminal in the first embodiment. [Figure 12] FIG. 2 is an explanatory diagram of a display of a chat room in the first embodiment. [Figure 13] FIG. 2 is an explanatory diagram of a display of a chat room in the first embodiment. [Figure 14] FIG. 2 is an explanatory diagram of a display of a chat room in the first embodiment. [Figure 15] 10 is a flowchart illustrating the operation of a mobile terminal in the second embodiment. [Figure 16] 10 is a flowchart illustrating the operation of a mobile terminal in the second embodiment. [Figure 17] FIG. 11 is an explanatory diagram of a chat room display in the second embodiment. [Figure 18] 10 is a flowchart illustrating the operation of a mobile terminal in the second embodiment. [Figure 19] 13 is a flowchart illustrating the operation of a mobile terminal in the third embodiment. [Figure 20] FIG. 13 is an explanatory diagram of a chat room display in the third embodiment. [Figure 21] 13 is a flowchart illustrating the operation of a mobile terminal in the fourth embodiment. [Figure 22] FIG. 13 is an explanatory diagram of a chat room display in the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Hereinafter, the embodiments of the present invention will be described in detail with reference to the drawings. However, the configurations described in the following embodiments are merely examples, and the scope of the present invention is not limited to the configurations described in the embodiments. First, a first embodiment of the present invention will be described.
[0010] First Embodiment <System configuration> 1 is a diagram showing an example of a system configuration according to an embodiment of the present invention. This system has a communication base station 105 and a chat application server 107, which are connected to each other via the Internet 106 so that they can communicate with each other. The communication base station 105 communicates required information with a mobile terminal 102 owned by a sender 100 and a mobile terminal 104 owned by a receiver 103.
[0011] A sender 100 sends a "voice message" using a chat application installed in a mobile terminal 102. Next, the "voice message" sent by the sender 100 is transmitted to a chat application server 107 via a communication base station 105 and the Internet 106. Next, the chat application server 107 performs a predetermined process on the received "voice message" and transmits it to a destination. Then, the "voice message" sent by the chat application server 107 is transmitted to a mobile terminal 104 owned by a receiver 103 via the Internet 106 and the communication base station 105.
[0012] <Hardware configuration of chat application server 107> 2 is a diagram showing an example of the configuration of chat application server 107 according to this embodiment. Chat application server 107 includes control unit 201, storage unit 202, and network I / F 211, which are connected via system bus 210 so as to be able to transmit and receive required information to and from each other.
[0013] <Control unit 201> The control unit 201 reads out and executes a control program 203 stored in a non-volatile manner in the storage unit 202. This realizes various functions required for the chat application server 107. The control unit 201 is configured to have one or more processors (CPU, DSP, etc.). The control unit 201 also includes a chat processing unit 209, and performs overall control of each device connected to a system bus 210. The chat processing unit 209 interprets and responds to messages received from a chat application of the mobile terminal 102. In this way, the chat application server 107 has an automatic dialogue function and also functions as a "chatbot."
[0014] <Storage section 202> The storage unit 202 (storage means) is used as an internal storage. The storage unit 202 stores a control program 203, text data 204, image data 205, audio data 206, video data 207, video data 207, registered user data 208, system software, etc. The storage unit 202 is configured with storage devices such as an HDD (hard disk drive), an SSD (solid state drive), a RAM, and a ROM.
[0015] Text data 204 is text data of a message posted by a user or a chatbot on the chat. Image data 205 is image data posted by a user on the chat. Audio data 206 is audio data of an audio message posted by a user on the chat. Video data 207 is video data of a video message posted by a user on the chat. Registered user data 208 is list information of combinations of user IDs and passwords required for each user to log in to the chat application.
[0016] <Network I / F> The network I / F 211 is an interface for performing network communication by being connected to the Internet 106 via a LAN cable or the like. For example, a well-known network card or the like can be used.
[0017] <Configurations of the Mobile Terminal 102 and the Mobile Terminal 104> 3 is a block diagram of the mobile terminals 102 and 104. The mobile terminals 102 and 104 are examples of information processing terminals with communication functions, and are terminals that can be used anywhere due to their wireless communication functions and other features. Examples of such terminals include smartphones, tablet terminals, and notebook PCs.
[0018] Both mobile terminals 102, 104 have the same configuration, and the configuration of one of them, mobile terminal 102, will be described. Mobile terminal 102 has a system bus 308. To system bus 308, a control unit 301, a storage unit 302, an input / output unit 303, a display unit 304 (display means), a microphone 305, a speaker 306, and a network I / F 307 are connected. Each device connected to system bus 308 is capable of transmitting and receiving required information to and from each other.
[0019] The control unit 301 reads and executes a control program stored in a non-volatile manner in the storage unit 302. This realizes various functions required for the mobile terminal 102. The control unit 201 is configured to have one or more processors (CPU, DSP, etc.). The control unit 301 also performs overall control of each device connected to a system bus 308.
[0020] The storage unit 302 is used as an internal storage. The storage unit 302 stores text data, audio data, image data, system software, etc. The storage unit 302 is configured with storage devices such as a hard disk drive (HDD), a solid state drive (SSD), a RAM, and a ROM.
[0021] The input / output unit 303 is composed of an LCD touch panel or the like, and acquires information operated by a user and sends the acquired information to the control unit 301. The input / output unit 303 also outputs the processing result of the control unit 301. Note that operation input from the user may be realized by a hardware input device such as a switch or a keyboard. The method for detecting input from the touch panel may be a general detection method such as a resistive film method, an infrared method, an electromagnetic induction method, or a capacitive method.
[0022] The display unit 304 performs display according to image data. The display unit 304 can also display an operation screen and provide a user interface (UI) to the user. The microphone 305 is for inputting voice data. The speaker 306 is for outputting voice data. In this embodiment, a speaker built into the mobile terminal 102 is used for voice output. Alternatively, the control unit 301 may send voice to a voice output device such as an external earphone or speaker connected from outside to the mobile terminal 102 and output the voice. The network I / F 307 is an interface for connecting to the Internet 106 and performing network communication.
[0023] <Operation screen within a chat application on a mobile device> Fig. 4(a) is an explanatory diagram of an operation screen of a chat application running on the mobile terminals 102 and 104. At this time, the chat application is performing the required communication with the chat application server 107. On the operation screen of Fig. 4(a), messages 401 and 402 of users A and B are displayed vertically downward on the screen in chronological order. In this embodiment, the message 402 of the operator of the mobile terminal is placed on the right side of the display screen, but the display position of the user's message may be placed on the left side of the display screen.
[0024] The display screen shown in FIG. 4(a) also displays the word "read" (403) to determine whether or not user B has viewed the message sent by user A. In addition, a message input field 404 is displayed at the bottom of the display screen shown in FIG. 4(s). In the message input field 404, text can be input or image data stored in the storage unit 302 can be selected and sent to the chat server. This sends the message to user B.
[0025] 4(b) is an explanatory diagram of an operation screen of a chat application including a voice message. The voice message 405 is displayed separately from the text message, and a button (button icon) 406 for playing the voice is displayed. When a voice playback instruction is received by a user pressing the button, the control unit 301 of the mobile terminal 102 and the mobile terminal 104 controls the voice playback. As a result, the played voice is output from the speaker 306.
[0026] 4(c) is an explanatory diagram of an operation screen for inputting a voice message. When the voice message input field 407 is pressed, the chat application starts recording voice. The user inputs voice into the microphone 305 of the mobile terminal 102 or 104. To end input, the user presses the voice message input field again. The chat application stops recording voice and stores the voice message data in the storage unit 302. Thereafter, the voice message is sent to the recipient by sending it to the chat server at the user's instruction.
[0027] <Figure 5: Operation of chat application server 107> Next, the operation of message transmission / reception processing in chat application server 107 will be described with reference to Fig. 5. The operation of the flowchart shown in Fig. 5 is realized by control unit 201 (CPU) reading out control program 203 stored in storage unit 202 into RAM or the like and executing it. First, in S501, control unit 201 holds a chat room including a plurality of users. Next, in S502, control unit 201 receives a message from a chat application of mobile terminal 102 (message sending terminal).
[0028] Next, in S503, the control unit 201 stores the received message in the storage unit 202. Then, in S504, the control unit 201 transmits the received message to another user in the chat room. In this manner, the operation of the chat application server 107 is performed.
[0029] <Figure 6: Processing on a mobile device when receiving a message> Next, the operation of mobile terminal 104 when receiving a message will be described with reference to Fig. 6. The operation of the flowchart shown in Fig. 6 is realized by control unit 301 (CPU) reading a program stored in storage unit 302 into RAM or the like and executing it. First, in S601, control unit 301 receives a message from chat application server 107. Next, in S602, control unit 301 notifies the corresponding chat room of the message based on a chat rule ID that identifies the chat room included in the message.
[0030] Next, in S603, the control unit 301 determines whether or not it is necessary to display on a UI (user interface) that the message has been received and to notify the user. Note that whether or not to notify the user is set in advance by the user, and the setting is stored and saved in the storage unit 302.
[0031] Then, in S603, if the control unit 301 determines that a notification is necessary (Yes), the process proceeds to S604. On the other hand, if the control unit 301 determines that a notification is not necessary (No), the process ends. Next, in S604, the control unit 301 creates a notification. Then, in S605, the control unit 301 displays the notification on the display unit 304, and notifies the user by sound using the speaker 306 according to the settings. In this manner, the mobile terminal 104 that has received the message operates.
[0032] <Figure 7: Processing on a mobile device when a message is displayed> Next, the operation of mobile terminal 104 when displaying a message in a chat room will be described with reference to Fig. 7. The operation of the flowchart shown in Fig. 7 is realized by control unit 301 reading a program stored in storage unit 302 into RAM or the like and executing it. First, in S701, control unit 301 accepts display of a chat room from a user. Next, in S702, control unit 301 displays the selected chat room on display unit 304. Specifically, the chat room is displayed as a UI.
[0033] Next, in S703, the control unit 301 stores the most recent of the displayed unconfirmed messages as a "read message" in the storage unit 302. Then, in S704, the control unit 301 transmits the read message stored in the storage unit 302 to the chat application server 107. In this manner, the processing of the mobile terminal when a message is displayed is executed.
[0034] <Figure 8: How a mobile device receives read information> Next, a read information receiving operation in the mobile terminal 102 will be described with reference to Fig. 8. The operation of the flowchart shown in Fig. 8 is realized by the control unit 301 reading a program stored in the storage unit 302 into a RAM or the like and executing the program.
[0035] First, in S801, control unit 301 receives read information from chat application server 107. Next, in S802, control unit 301 updates the read information of the corresponding chat room in chat management table 901 based on the chat room ID that identifies the chat room included in the read information. The read information receiving operation is performed by the above operations.
[0036] <Figure 9: Chat management table> 9 is an explanatory diagram of an example of the "chat management table" updated in S802. The chat management table 901 is saved in the storage unit 302. The chat management table 901 stores chat room IDs (chat room identifiers) and read information in association with each other.
[0037] That is, chat management table 901 manages information for each chat room, and holds read information 902 indicating how much of a message has been read by the recipient in each chat room. Based on this information, read information 403 is displayed when the chat room is displayed. For example, in chat room "A," read information 902 is "100," which indicates that "100" messages have been read.
[0038] <Figure 10: Sequence of the entire chat system> Next, the chat application server 107, the mobile terminal 104, and the sequence of the entire chat system using the mobile terminal 104 will be described with reference to Fig. 10. First, the sender 100 operates the mobile terminal 104 to send a "message" to the chat application server 107. In response to this, the chat application server 107 adds the "message" to the chat management table 901, and sends the "message" to the mobile terminal 104 of the receiver 103.
[0039] Next, the receiver 103 operates the mobile terminal 104 to display the chat room and check the "message." In response to this, the mobile terminal 104 transmits "read information" to the chat application server 107. The chat application server 107 then transmits the "read information" to the mobile terminal 102. This allows the sender 100 to recognize that the "message" has been checked. This is the sequence of the entire chat system.
[0040] First Embodiment Next, referring to Fig. 11, a process of transcribing (converting to text) and displaying a voice message in a chat room of mobile terminal 104 at the instruction of a user will be described. The operation shown in the flowchart of Fig. 11 can be realized by control unit 301 reading a program stored in storage unit 302 into RAM or the like and executing it. First, in S1101, control unit 301 accepts display of a chat room from a user. Next, in S1102, control unit 301 displays the selected chat room on display unit 304.
[0041] Next, in S1103, the control unit 301 stores the most recent of the displayed unconfirmed messages as a read message in the storage unit 302. Next, in S1104, the control unit 301 transmits the read messages stored in the storage unit 302 to the chat application server 107.
[0042] Next, in S1105, the control unit 301 determines whether or not there is an instruction to transcribe the voice message (an instruction to convert to text). If the control unit 301 determines that there is an instruction to transcribe (Yes), the process proceeds to S1106. On the other hand, if the control unit 301 determines that there is no instruction to transcribe (No), the operation of FIG. 11 ends.
[0043] The transcription instruction (instruction means) in S1105 may be given when an action is performed on the touch panel, such as a long press or slide operation on the corresponding voice message. This is not limiting, and a dedicated instruction button may be displayed, such as the transcription button 1301 (FIG. 13), and the user may give an instruction to transcribe by performing an action on the button. FIG. 13 shows an example in which a dedicated transcription button 1301 (the word "text") for transcription is displayed within a display object showing a voice message on the mobile terminal 102. The dedicated button is not limited to the one shown in FIG. 13.
[0044] In this way, the control unit 301 (instruction means) can be configured to give an instruction to convert the contents of a voice message into text by performing an action including a long press operation or a slide operation on the voice message.
[0045] Next, in S1106, control unit 301 converts the selected voice message into text by performing voice analysis. Next, in S1107, control unit 301 displays display 1201 of the character string converted into text in S1106 on display unit 304 in a display mode different from normal message display, linked to the voice message in the chat room. Fig. 12 shows how the character string converted into text (1201) is displayed on the display screen of the chat room displayed on mobile terminal 102. In the linked display example shown in Fig. 12, a text display space is linked below an object showing the voice message, and the character string of text is displayed.
[0046] Next, in S1108, the control unit 301 determines whether or not there is an "erase instruction" from the erasure instruction unit (erase instruction means) for the transcription display 1201 displayed in S1107. If the control unit 301 determines that there is an "erase instruction" (Yes), the process proceeds to S1109. On the other hand, if the control unit 301 determines that there is no "erase instruction" (No), the process returns to S1108 and checks the transcription erasure instruction.
[0047] In this way, when an instruction to erase the text of the voice message is given, the control unit 301 (control means) erases the display of the text of the voice message displayed on the display unit 304. Note that the "erase instruction" for the transcription in S1108 may be configured to be performed by an action on the touch panel such as a long press or slide operation on the transcription display, just like the transcription instruction. In addition, without being limited to this, a dedicated erase instruction button may be displayed and the user may give the "erase instruction" by tapping the button, or by tapping an area other than the transcription display. Then, in S1109, the control unit erases the transcription display 1201.
[0048] In the first embodiment, the voice message is converted to text on the mobile terminal 102 side, but the conversion to text may be requested to the chat application server 107, and text information may be received from the chat application server 107 having a function of converting to text. Then, in S1107, the converted text string may be displayed. Note that the voice message can be displayed with a "headline" associated therewith according to the operation of FIG. 11, regardless of whether it is a message sent by the user or received from the other party.
[0049] If the voice message is a long message, the transcribed text may be long. In such a case, displaying the entire text may result in poor visibility. In such a case, as in the display of reference numeral 1401 in FIG. 14, only a part of the text may be displayed by sliding the text in the transcribed display to advance the text. FIG. 14 shows an example in which the text displayed in association with the voice message is long and does not fit in the text display space on the display screen of the chat room of the mobile terminal 102. In this case, if the text does not fit in the text display space, the non-displayed part is displayed in sequence by sliding the character string.
[0050] In this way, when the character string converted to text by the text conversion unit does not fit within the text display space, the control unit 301 moves the character string within the text display space so that it is visible in response to receipt of a slide operation (specific operation).
[0051] As described above, by transcribing and displaying a voice message, the contents of the voice message can be confirmed even in situations where the voice message cannot be played back. Furthermore, since the transcription is performed at the user's command, it is even more convenient for the user.
[0052] <Summary of the First Embodiment> From the above, the following configuration is provided. First, control unit 301 (communication means) transmits and receives voice messages in a chat room where multiple users chat. Next, control unit 310 (text conversion means) converts the content of the voice message into text. Then, when a transcription instruction is given, control unit 301 (control means) displays the text string obtained by converting the content of the voice message into text by control unit 301 (text conversion means) in the chat room in a format different from that of normal text messages in the chat room.
[0053] Furthermore, the control unit 301 (control means) can also be configured to link the character string converted into text by the text conversion unit to a voice message and display it on the display unit 304 (display means) that displays the chat room.
[0054] In addition, a configuration may be adopted in which a request is sent to have chat application server 107 process the "text conversion", information on the "text data" is received from chat application server 107, and the information is displayed on mobile terminal 102. That is, control unit 301 (request sending means) sends information indicating a request for converting a voice message to text to chat application server 107 (external server) capable of generating a character string obtained by converting a voice message to text. In response to this, control unit 301 (receiving means) receives the text conversion information sent by chat application server 107 (external server). Then, control unit 301 (control means) may be configured to link the text conversion character string indicated by the received text conversion information to the voice message and display it in the chat room. In this way, chat application server 107 with high processing power can generate text data with high accuracy and link it to the voice message.
[0055] <Second embodiment> In the first embodiment, a character string obtained by converting a voice message into text is displayed in a different manner from the manner in which a normal message is displayed. In the second embodiment, an operation of displaying a transcribed character string in the same manner as an existing message will be described.
[0056] <Figure 15: Transcribed text displayed as a normal message> An operation of transcribing a voice message in a chat room of mobile terminal 102 at the user's instruction and posting the transcribed message in the message field will be described with reference to Fig. 15. The operation of the flowchart shown in Fig. 15 is realized by control unit 301 reading a program stored in storage unit 302 into RAM or the like and executing the program. First, in S1501, control unit 301 accepts a display of a chat room from the user. Next, in S1502, control unit 301 displays the selected chat room on display unit 304.
[0057] Next, in S1503, control unit 301 stores the most recent of the displayed unconfirmed messages as a "read message" in memory unit 302. Next, in S1504, control unit 301 transmits the "read message" stored in memory unit 302 to chat application server 107. Next, in S1505, control unit 301 determines whether or not there is an instruction to transcribe the voice message. If control unit 301 determines that there is an instruction to transcribe (Yes), the process proceeds to S1506. On the other hand, if control unit 301 determines that there is no instruction to transcribe (No), the operation ends.
[0058] Next, in S1506, the control unit 301 performs voice analysis on the selected voice message to convert it into text. Then, in S1507, the control unit 301 displays the character string converted into text in S1506 as a text message 1701 on the display screen of the chat room in the same manner as a normal message. Fig. 17 shows the display screen of the display unit 304 of the mobile terminal 102, and the character string indicated by the reference numeral 1701 is displayed like a normal chat message (post) without being linked to a voice message.
[0059] Note that a message with a character string that is a transcribed text may be displayed with a display 1702 indicating that the message has been transcribed (converted into text) so that the message can be distinguished from other text messages. The transcribed message may be highlighted by changing the color, shape, etc., of the transcribed message so that it can be easily distinguished from character strings posted in normal chat. In this way, the control unit 301 (control means) can display an indication on the display unit 304 indicating that the corresponding voice message has been converted into text for the converted text character string.
[0060] <Figure 16: When entering a text string into the message field> In the operation of the flowchart shown in Fig. 15, the transcribed text is directly displayed in the message field, but the operation of inputting text into the message input field will be described with reference to Fig. 16. The operation of the flowchart shown in Fig. 16 can be realized by control unit 301 reading a program stored in storage unit 302 into RAM or the like and executing it. First, in S1601, control unit 301 accepts the display of a chat room from the user. Next, in S1602, control unit 301 displays the selected chat room on display unit 304.
[0061] Next, in S1603, control unit 301 stores the most recent of the displayed unconfirmed messages as a read message in memory unit 302. Next, in S1604, control unit 301 transmits the "read message" stored in memory unit 302 to chat application server 107. Next, in S1605, control unit 301 determines whether or not there is an instruction to transcribe the voice message. If control unit 301 determines that there is an instruction to transcribe (Yes), the process proceeds to S1606. On the other hand, if control unit 301 determines that there is no instruction to transcribe (No), the operation ends.
[0062] Next, in S1606, control unit 301 performs voice analysis on the selected voice message and converts it into text. Next, in S1607, control unit 301 inputs the character string converted into text in S1606 into message input field 404 (posting field: see FIG. 4). By inputting the character string into message input field 404, the content of the voice message can be confirmed as text.
[0063] Next, in S1608, the control unit 301 determines whether or not there is an instruction to post a message. If the control unit 301 determines that there is an instruction to post (Yes), the process proceeds to S1609. On the other hand, if the control unit 301 determines that there is no instruction to post (No), the process ends. Then, in S1609, the control unit 301 transmits the message to the chat application server 107. After the message is posted, other users who receive the message can also check the transcribed text.
[0064] In the operation of the flowcharts shown in Figures 15 and 16, the transcribed text is displayed in the message field, but after the transcription is performed, the sender may delete the original voice message. In such a case, the voice message itself is deleted from the chat room, but the transcribed message remains, resulting in unnecessary messages or erroneous messages remaining.
[0065] <Figure 18: Voice message deletion operation> An operation of erasing a transcribed text message in association with erasing an original voice message will now be described with reference to Fig. 18. The operation of the flowchart shown in Fig. 18 is realized by control unit 301 reading a program stored in storage unit 302 into RAM 302 or the like and executing the program. First, in S1801, control unit 301 accepts display of a chat room from a user. Next, in S1802, control unit 301 displays the selected chat room on display unit 304.
[0066] Next, in S1803, the control unit 301 stores the most recent of the displayed unconfirmed messages as a "read message" in the storage unit 302. Next, in S1804, the control unit 301 transmits the "read message" stored in the storage unit 302 to the chat application server 107. Next, in S1805, the control unit 301 judges whether or not there is a "delete instruction" for the voice message. If the control unit 301 judges that there is a "delete instruction" (Yes), the process proceeds to S1806. On the other hand, if the control unit 301 judges that there is no "delete instruction" (No), the operation ends. Note that the form of the "delete instruction means" that gives the "delete instruction" for the voice message may be, but is not limited to, an action such as a touch operation of a dedicated button displayed on the display unit 304, a multiple tap operation on the displayed voice message, a long press operation, etc.
[0067] Next, in S1806, the control unit 301 determines whether or not a text message transcribed from the voice message for which a "deletion instruction" was given exists. If the control unit 301 determines that a text message exists (Yes), the process proceeds to S1809. On the other hand, if the control unit 301 determines that a text message does not exist (No), the process proceeds to S1807.
[0068] Next, in S1807, control unit 301 deletes the selected voice message, and displays the chat room from which the voice message has been deleted on display unit 304. Next, in S1808, control unit 301 transmits information about the deleted voice message to chat application server 107.
[0069] On the other hand, in S1809, the control unit 301 deletes the “text message” transcribed from the selected voice message, and displays the chat room from which the transcribed message has been deleted on the display unit 304. Then, in S1810, the control unit 301 transmits information about the deleted transcribed message to the chat application server 107.
[0070] <Summary of the second embodiment> As described above, the chat room is provided with a message input field 404 (input means) for inputting normal text and displaying it in the chat room, and the control unit 301 (control means) automatically inputs a character string obtained by converting a voice message into text into the message input field 404. This allows a character string of text corresponding to a voice message to be automatically posted in the chat room, enabling automatic posting of both voice and text. Furthermore, when a deletion instruction is given by operating a deletion instruction unit (deletion instruction means) that gives an instruction to delete a voice message, the control unit 301 (control means) deletes the voice message and also deletes the character string converted into text corresponding to the voice message. This allows the transcribed text message to be deleted in conjunction with the deletion of the original voice message.
[0071] <Third embodiment> In the second embodiment, a method for converting a voice message into text and displaying it at once was described. However, if the voice message is long, it may take a long time to convert it into text, or a large amount of text may be displayed, resulting in poor visibility. Therefore, in the third embodiment, an operation for performing transcription for a predetermined time or for a predetermined number of characters when an instruction to transcribe a voice message is given will be described.
[0072] <Figure 19: Transcription operation for the set amount> With reference to Fig. 19, an operation of converting a predetermined time period or a predetermined number of characters into text when a "transcription instruction" is given to mobile terminal 104 will be described. The operation of the flowchart shown in Fig. 19 can be realized by control unit 301 reading a program stored in storage unit 302 into RAM or the like and executing it. First, in S1901, control unit 301 accepts a display of a chat room from a user. Next, in S1902, control unit 301 displays the selected chat room on display unit 304.
[0073] Next, in S1903, control unit 301 stores the most recent of the displayed unconfirmed messages as a "read message" in memory unit 302. Next, in S1904, control unit 301 transmits the "read message" stored in memory unit 302 to chat application server 107. Next, in S1905, control unit 301 determines whether or not there is an instruction to transcribe the voice message. If control unit 301 determines that there is an instruction to transcribe (Yes), the process proceeds to S1906. On the other hand, if control unit 301 determines that there is no instruction to transcribe (No), the process ends the operation.
[0074] Next, in S1906, the control unit 301 performs voice analysis on the selected voice message and converts it into text for a set amount. Note that the amount of transcription to be performed in S1906 is roughly a preset value stored in the storage unit 302. For example, the time may be set to "10(%): a set amount" of the message, or "10(seconds): a set amount of time." Also, the number of characters may be set, such as "30 characters." In this way, the storage unit 302 stores information for setting the set amount in a changeable manner.
[0075] Next, in S1907, the control unit 301 displays a transcription display of the characters converted to text in S1906 on the display unit 304 in a display mode different from the normal message display, linked to the voice message in the chat room. Next, in S1908, the control unit 301 determines whether or not there is an instruction to delete the transcription display (text string) displayed in S1907. If the control unit 301 determines that there is an instruction to delete the transcription (Yes), the process proceeds to S1910. If the control unit 301 determines that there is no instruction to delete the transcription (No), the process proceeds to S1909.
[0076] Next, in S1909, the control unit 301 determines whether or not there is a "transcription update instruction." If the control unit 301 determines that there is a "transcription update instruction" (Yes), the process proceeds to S1906. On the other hand, if the control unit 301 determines that there is no "transcription update instruction" (No), the process proceeds to S1908.
[0077] The "transcription update instruction" in S1909 may be an action on the touch panel, such as a long press or slide operation on the transcription display. Alternatively, a dedicated instruction button may be displayed, such as the "transcription update button 2001 (update instruction unit)" shown in FIG. 20, and the user may input the instruction. FIG. 20 is an explanatory diagram of the display screen of the display unit 304 of the mobile terminal 102, and shows a triangular display object (2001) on the right side of the text display space on the display screen, and the display object serves as the "transcription update button 2001." Then, in S1910, the control unit 301 erases the transcribed text from the display unit 304.
[0078] <Summary of the third embodiment> As described above, the control unit 301 (text conversion means) converts the contents of the voice message into text for a preset amount. Furthermore, when an update instruction is given by operating the transcription update button 2001 (update instruction means), the control unit 301 (text conversion means) converts the contents of the voice message into text again for the preset amount. This makes it possible to transcribe the voice message for the set amount desired by the user. Furthermore, if there is time to spare, the voice message is transcribed again for the set amount, which is very convenient. Therefore, by displaying the transcription for the set amount of the voice message, the contents of the voice message can be confirmed even if the entire voice message cannot be played back.
[0079] <Fourth embodiment> There are cases where the transcribed text is desired to be left as a memo, or to be copied and used in another process, etc. In the fourth embodiment, an operation of executing another action on a transcribed and displayed character string will be described.
[0080] <Figure 21: Editing action> The operation of the mobile terminal 102 when an action instruction is given for the transcription display will be described with reference to Fig. 21. The operation of the flowchart shown in Fig. 21 can be realized by the control unit 301 reading a program stored in the storage unit 302 into a RAM or the like and executing it.
[0081] First, in S2101, control unit 301 accepts display of a chat room from a user. Next, in S2102, control unit 301 displays the selected chat room on display unit 304. Next, in S2103, control unit 301 stores the most recent of the displayed unconfirmed messages as a "read message" in storage unit 302. Next, in S2104, control unit 301 transmits the "read message" stored in storage unit 302 to chat application server 107.
[0082] Next, in S2105, the control unit 301 determines whether or not there is an instruction to transcribe the voice message. If the control unit 301 determines that there is an instruction to transcribe (Yes), the process proceeds to S2106. On the other hand, if the control unit 301 determines that there is no instruction to transcribe (No), the operation ends. Next, in S2106, the control unit 301 performs a voice analysis of the selected voice message and converts it into text.
[0083] Next, in S2107, the control unit displays a transcription display 1201 (see FIG. 12) of the characters converted to text in S2106 on the display unit 304 in a display mode different from the normal message display, linked to the voice message in the chat room. Next, in S2108, the control unit 301 determines whether or not there is an "instruction to delete" the transcription display 1201 displayed in S2107. If the control unit 301 determines that there is an "instruction to delete" the transcription (Yes), the process proceeds to S2110. On the other hand, if the control unit 301 determines that there is no "instruction to delete" the transcription (No), the process proceeds to S2109.
[0084] Next, in S2109, the control unit 301 determines whether or not there is an "editing instruction" for the transcription display 1201. If the control unit 301 determines that there is an "editing instruction" (Yes), the process proceeds to S2111. On the other hand, if the control unit 301 determines that there is no "editing instruction" (No), the process proceeds to S2108.
[0085] The "edit instruction" in S2109 may be given by performing an action on the touch panel such as a long press or a slide operation on the corresponding transcription display. However, the present invention is not limited to this, and a dedicated instruction button may be displayed and the user may tap the dedicated instruction button to give the "edit instruction."
[0086] Next, in S2110, the control unit 301 erases the transcription from the display unit 304. Next, in S2111, the control unit 301 displays a transcription edit pop-up 2201 on the display unit 304.
[0087] There are various editing actions for the transcription display. For example, the edit popup 2201 in Fig. 22 includes copy 2202, memo 2203, and post message 2204. Copy 2202 is for copying the transcribed text, and memo 2203 is for inputting into the memo area in the chat room. Furthermore, post message 2204 is an action for posting the transcribed character string as a normal message. In this way, popup displays are performed for the action modes.
[0088] Next, in S2112, the control unit 301 determines whether or not an action has been selected for the edit popup 2201. If the control unit 301 determines that an action has been selected (Yes), the process proceeds to S2113. On the other hand, if the control unit 301 determines that no action has been selected (No), the process proceeds to S2108. Then, in S2113, the control unit 301 executes the selected action.
[0089] As described above, by performing an editing action on the transcription display, the transcribed text can be used for other tasks. Note that, although copy 2202, memo 2203, and message posting 2204 are displayed in FIG. 22, other actions may be added, or the action modes displayed in FIG. 22 may be replaced with other actions. For example, "call" for calling up all memos stored in the memo area in the past, or "summary" for summarizing a voice message that has been converted into text, etc. may be used. Note that, in order to make the summary of the text string simple, it is possible to extract only keywords such as the location, date and time, or to extract a predetermined number of characters from the beginning of the text string.
[0090] <Summary of the Fourth Embodiment> As described above, when an editing instruction is given by operating an editing instruction unit (editing instruction unit) for giving an instruction to edit the text character string, the control unit 301 (editing unit) edits the text character string. As shown in FIG. 22, the editing of the character string performed by the control unit 301 (editing unit) includes "copy" for copying the character string, "memo" for storing the character string in a storage unit, and "post" for making the character string into a text message in a chat room. Note that the editing modes are not limited to these. In this way, editing using the text character string for the voice message is provided for the convenience of the user.
[0091] <Other> (1) In the above explanation, an example of chatting between mobile terminal devices has been described, but the above-described operations can be realized, for example, between PCs such as desktop PCs or notebook PCs, or between a PC and a mobile terminal. In this case, the PC may be connected to a router via a LAN cable or may be connected to a wireless router. The notebook PC may also be portable. However, the PC must have the application program (AP) according to the present invention installed and must have a built-in or external microphone for audio input and speaker for audio output. (2) The chat room itself may be displayed as a three-dimensional image, and the text may also be displayed in three dimensions, so that the chat room and the textual characters can be recognized in three dimensions by wearing special glasses, special goggles, etc. In this case, the user's own avatar and the other party's avatar may be set in advance and displayed in appropriate positions, making chatting even more enjoyable. (3) The user determines a color for each chat content (for friends, for work, for a certain meeting, etc.). Then, keywords (date, time, place, etc.) are extracted from the text of the voice message, and the control unit 301 determines the chat content and displays the text string with the corresponding color. This allows the user to roughly understand the content. (4) By performing a specific operation such as a long press on the triangular play icon of the voice message, the text string can be displayed in the display area of the voice message without the sound being output. In this case, if the string does not fit in the display area, the string can be displayed by sliding the string in sequence so that it can be seen. (5) If a contract is concluded with the operator of chat application server 107 to always convert voice messages to text, the various processes described above can be realized with a high degree of accuracy on a mobile terminal, without having to request chat application server 107 each time.
[0092] Although the preferred embodiment of the present invention has been described above, the present invention is not limited to the above-mentioned embodiment, and various modifications and changes are possible within the scope of the gist of the present invention. For example, the present invention can be realized by supplying a program that realizes one or more functions of the above-mentioned embodiment to a system or device via a network or a recording medium, and having a computer processor of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]
[0093] 102 Mobile devices 104 Mobile Devices 105 Telecommunications Base Station 106 Internet 107 Chat App Server 201 Control section 202 Storage section 203 Control Program 209 Chat Processing Unit 301 Control Unit 302 Storage section 303 Input / output section 304 Display section 305 Mike 306 Speaker 404 Message input field 901 Chat Management Table
Claims
1. A program for causing a computer to execute a control method for an information processing device capable of receiving voice messages via the Internet, comprising: The control method includes: a text conversion step of converting the content of the received voice message into text; a display step of displaying the character string converted into text by the conversion step; and a control step of controlling display of an object that performs processing for posting information about the voice message.
2. In the control step, 2. The program according to claim 1, wherein the display form of the character string converted into text in the converting step and the display form of the message different from the voice message are displayed in a distinguishable manner.
3. In the control step, 2. The program according to claim 1, further comprising: displaying, for the character string converted into text in the conversion step, an indication that the corresponding voice message has been converted into text.
4. The program described in claim 1 or 2, further comprising an instruction process for giving an instruction to convert the contents of the voice message into text upon receiving a long press or slide operation on the voice message.
5. In the control step, The program according to claim 1 or 2, characterized in that if the character string converted into text by the text conversion process does not fit in the display area, the display is updated so that the character string becomes visible by accepting a specific operation.
6. The method further comprises a deletion instruction step for giving an instruction to delete the character string converted into text by the text conversion step, The control step further comprises:
3. The program according to claim 1, wherein when a delete instruction is given by operating the delete instruction step, the displayed character string is deleted.
7. In the text conversion step, 3. The program according to claim 1, wherein the content of the voice message is converted into a character string based on a preset time.
8. In the control step, The program described in claim 1 or 2, characterized in that it receives a usage operation instruction for performing a usage operation using a string converted into text by the text conversion process through a predetermined operation for the object that performs processing to post information about the voice message, and executes the usage operation based on the received usage operation instruction.
9. 9. The program according to claim 8, wherein the use action includes copying the character string converted into text by the conversion step and posting the character string converted into text by the conversion step as a text message.
10. In the text conversion step, 3. The program according to claim 1, wherein the program causes an external server to execute a process of converting the content of the voice message into a character string, and receives the converted character string.
11. The program described in claim 1 or 2, characterized in that the information processing device is a mobile terminal including a smartphone or tablet terminal.
12. A storage medium storing the program described in claim 1.
13. A method for controlling an information processing device capable of receiving a voice message via the Internet, comprising: a text conversion step of converting the content of the received voice message into text; a display step of displaying the character string converted into text by the conversion step; a control step of controlling display of an object that performs processing for posting information about the voice message.
14. An information processing device capable of receiving voice messages via the Internet, comprising: a text conversion means for converting the content of the received voice message into text; a display means for displaying the character string converted into text by the text conversion means; and a control means for displaying an object that performs processing for posting information about the voice message.
15. A program for causing a computer to execute a control method for an information processing device capable of receiving voice messages via the Internet, comprising: The control method includes: a text conversion step of converting the content of the received voice message into text; a display step of displaying the character string converted into text by the conversion step; a summarizing step of summarizing the character string converted into text by the text conversion step; a control step of controlling display of the results summarized by the summarizing step.
16. In the summarizing step, 16. The program according to claim 15, wherein the character string converted into text by the converting step is summarized in response to a user instruction.