Information processing device, information processing terminal, information processing method, and information processing program

Abstract

An information processing device is provided with: a reception unit that receives voice data transmitted from a first terminal or a second terminal; a text conversion unit that converts the sound data received by the reception unit into text data; and a transmission unit that transmits the text data converted by the conversion unit to the second terminal or the first terminal.

Description

Technical Field

[0001] This invention relates to an information processing apparatus, an information processing terminal, an information processing method, and an information processing program that can be used in a system for transmitting and receiving sound data. Background Technology

[0002] In the past, GPS terminals capable of exchanging voice data were proposed for use in childcare.

[0003] Existing technical documents Patent documents Patent Document 1: Japanese Re-registration 2020 / 202449 Summary of the Invention

[0004] The problem that the invention aims to solve However, in the past, GPS terminals only provided users with location information and voice information, which could be improved in terms of convenience.

[0005] The present invention was made in view of the above-mentioned problems, and its purpose is to provide information processing devices, information processing terminals, information processing methods and information processing programs with greater convenience.

[0006] Technical solutions for solving technical problems To address the aforementioned issues, the information processing apparatus of the present invention comprises: a receiving unit that receives sound data transmitted from a first terminal or a second terminal; a text conversion unit that converts the sound data received by the receiving unit into text data; and a sending unit that sends the text data converted by the conversion unit to the second terminal or the first terminal.

[0007] Invention Effects According to the present invention, more convenient information processing devices, information processing terminals, information processing methods, and information processing programs can be provided. Attached Figure Description

[0008] Figure 1 This is a diagram illustrating an example of the configuration of an information processing system involved in the implementation method.

[0009] Figure 2 This is a diagram illustrating an example of the structure of an information processing server (information processing device) according to an embodiment.

[0010] Figure 3 This is a diagram illustrating an example of the structure of an information processing server involved in an implementation method.

[0011] Figure 4 This is a diagram illustrating an example of the structure of the first terminal (information processing terminal) involved in the implementation method.

[0012] Figure 5 This is a diagram illustrating an example of the structure of the first terminal involved in the implementation method.

[0013] Figure 6 This is a diagram illustrating an example of the structure of the second terminal (information processing terminal) involved in the implementation method.

[0014] Figure 7 This is a diagram illustrating an example of the structure of the second terminal involved in the implementation method.

[0015] Figure 8 This is a diagram illustrating an example of a screen displayed by the display device of the first terminal according to the embodiment.

[0016] Figure 9 This is a diagram illustrating an example of a screen displayed by the display device of the first terminal according to the embodiment.

[0017] Figure 10 This is a diagram illustrating an example of a screen displayed by the display device of the first terminal according to the embodiment.

[0018] Figure 11 This is a diagram illustrating an example of a screen displayed by the display device of the first terminal according to the embodiment.

[0019] Figure 12 This is a flowchart illustrating an example of the processing of the information processing system involved in the implementation method.

[0020] Figure 13 This is a flowchart illustrating an example of the processing of the information processing system involved in the implementation method.

[0021] Figure 14 This is a flowchart illustrating an example of the processing of the information processing system involved in the implementation method.

[0022] Figure 15 This is a flowchart illustrating an example of the processing of the information processing system involved in the implementation method.

[0023] Figure 16 This is a flowchart illustrating an example of the processing of the information processing system involved in the implementation method.

[0024] Figure 17 This is a flowchart illustrating an example of the processing of the information processing system involved in the implementation method.

[0025] Figure 18 This is a diagram illustrating an example of a screen displayed by a display device on a first terminal in a variation of the embodiment.

[0026] Figure 19This is a diagram illustrating an example of a screen displayed by a display device on a first terminal in a variation of the embodiment.

[0027] Figure 20 This is a diagram illustrating an example of a screen displayed by a display device on a first terminal in a variation of the embodiment.

[0028] Figure 21 This is a diagram illustrating an example of a screen displayed by a display device on a first terminal in a variation of the embodiment. Detailed Implementation

[0029] [Implementation Method] Hereinafter, the information processing system 1 of the embodiment will be described with reference to the accompanying drawings.

[0030] The information processing system 1 described in this embodiment is a so-called monitoring system. It determines the location of the second terminal 4 based on data uploaded to a server 2 at given time intervals, such as 1.5 minutes, from a second terminal 4 carried and used by the person being cared for (e.g., a child). The determined location is then notified from the server 2 to the first terminal 3 carried or used by the caregiver (e.g., a family member, relative, such as a parent or grandparent). Furthermore, in the information processing system 1 described in this embodiment, both the first terminal 3 and the second terminal 4 are equipped with microphones, speakers, etc., and are capable of sending and receiving voice messages (hereinafter also referred to as voice). That is, it is configured to enable voice-based message exchange between the caregiver and the person being cared for. Additionally, the first terminal 3 is configured to include a display device and is capable of converting voice into text and displaying it on the display device. It should be noted that in the following description, the caregiver will also be referred to as the first user. The person being cared for will also be referred to as the second user.

[0031] like Figure 1 As shown, the information processing system 1 includes a server 2 (information processing device) and one or more first terminals 3 (information processing terminals) and second terminals 4 (information processing terminals) connected to the server 2 via a network 5. Figure 1 In the example shown, although the information processing system 1 is configured to have a server 2, a first terminal 3 and a second terminal 4 respectively, the number of servers 2, first terminals 3 and second terminals 4 in the information processing system 1 is arbitrary.

[0032] (Server 2) Figure 2 and Figure 3 This is the structure diagram of server 2. Figure 2This describes the main hardware structure of server 2, which includes a communication IF200A, a storage device 200B (storage unit), and a CPU 200C. It should be noted that, although in... Figure 2 Although not shown in the figure, server 2 may also have input devices (such as mouse, keyboard, touch panel, etc.) and display devices (CRT (Cathode Ray Tube: CRT monitor), liquid crystal display, organic EL monitor, etc.).

[0033] The IF200A communication interface is used to communicate with other devices (e.g., first terminal 3, second terminal 4, etc.).

[0034] Storage device 200B is, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive). Various data and information processing programs are stored in storage device 200B. It should be noted that some or all of the various data stored in storage device 200B may also be stored on external storage devices such as USB (Universal Serial Bus) memory, external HDDs, or storage devices of other information processing devices connected via network 5. In this case, server 2 refers to or retrieves various data stored on external storage devices or storage devices of other information processing devices.

[0035] The storage device 200B stores account information of the first terminal 3, such as the identification number (terminal ID) of the first terminal 3, name, contact address (email address, telephone number), and the identification number (terminal ID) of the second terminal 4 held by the second user (e.g., one's own child).

[0036] Additionally, the storage device 200B stores account information of the second terminal 4, such as the terminal ID and name of the second terminal 4, and the terminal ID of the first terminal 3 held by the first user (e.g., family members such as one's parents or grandparents).

[0037] Furthermore, in storage device 200B, various data sent and received by the first terminal 3 and the second terminal 4 (including metadata attached to the data) are stored as logs along with date and time data and associated with terminal IDs. It should be noted that these various data can also be stored as logs along with date and time data in a manner associated with account IDs.

[0038] Here, the various data mentioned above include, for example, the following data.

[0039] (1) Data used to determine location (e.g., location information sent from the second terminal 4). (2) Audio data (including audio data converted from text data) (3) Text data (including data after converting audio data) In addition, the aforementioned data includes metadata such as the acquisition date and time (which can also be the recording date and time, or the conversion date and time), the sending date and time, the receiving date and time, the terminal ID of the sending first or second terminal, and the terminal ID of the receiving first or second terminal. Furthermore, data used to determine location is also referred to as location information.

[0040] The CPU200C controls the server 2 involved in this embodiment and has ROM and RAM (not shown).

[0041] Figure 3 This is a functional block diagram of server 2. (Example) Figure 3 As shown, server 2 has the functions of a receiving unit 201, a sending unit 202, a storage device control unit 203 (association establishment unit), a text conversion unit 204, a voice conversion unit 205, a file format conversion unit 206, a removal unit 207, a volume adjustment unit 208, and an extraction unit 209. It should be noted that... Figure 3 The functions shown are implemented by the CPU 200C executing the information processing program stored in the storage device 200B.

[0042] The receiving unit 201 receives data (including metadata) sent from the first terminal 3 and the second terminal 4. The data received by the receiving unit 201 may include, for example, data for determining location, audio data, and text data. It should be noted that the data described above is an example and does not prevent the receiving unit 201 from receiving other data.

[0043] The transmitting unit 202 sends data (including metadata) to the first terminal 3 and the second terminal 4. The data transmitted by the transmitting unit 202 may include, for example, data for determining location, audio data, and text data. It should be noted that the data described above is an example and does not preclude the transmitting unit 202 from transmitting other data.

[0044] The storage device control unit 203 stores the data (including metadata) sent and received by the first terminal 3 and the second terminal 4 in the storage device 200B in a manner that establishes a correspondence with the terminal ID of the terminal that sent and received the data. In addition, the storage device control unit 203 stores various types of data in the storage device 200B in a manner that establishes a correspondence with the terminal ID.

[0045] The text conversion unit 204 converts the audio data received by the receiving unit 201 into text data. Therefore, even when reproducing audio data is not preferred, messages can be confirmed via text, such as in public transportation like trams and buses, or in public facilities like libraries, offering high convenience.

[0046] The voice conversion unit 205 converts the text data received by the receiving unit 201 into sound data. Therefore, even when visual recognition of text data is not preferred, messages can be confirmed by voice while driving (including on a bicycle), walking, running, exercising, cooking, etc., offering high convenience. Furthermore, even when the second terminal 4 lacks a display device or the display device cannot display text, messages can still be confirmed by voice, further enhancing convenience.

[0047] The file format conversion unit 206 converts the file format of the audio data. For example, the file format conversion unit 206 converts the audio data file format based on the storage capacity of the storage device and the speaker of the second terminal 4. The file format conversion unit 206 compresses the file to reduce the data volume. This reduces the amount of data sent to the second terminal 4, thereby reducing communication costs. Furthermore, it reduces the storage capacity of the storage device of the second terminal 4, thereby lowering the cost of the second terminal 4. Alternatively, the storage device 200B can store data that establishes a correspondence between the type of the second terminal 4 and a terminal ID, as well as data that establishes a correspondence between the file format and the type. The file format conversion unit 206 determines the type of the second terminal 4 based on the terminal ID and converts the audio data into a file format corresponding to the determined type. In addition, the file format conversion unit 206 can also convert the audio data file format based on the storage capacity of the storage device and the speaker of the first terminal 3. This reduces the amount of data sent to the first terminal 3, thereby reducing communication costs. Furthermore, it reduces the storage capacity of the storage device of the first terminal 3. It should be noted that the configuration may also be as follows: the storage device 200B stores data that establishes a correspondence between the category of the first terminal 3 and the terminal ID, and data that establishes a correspondence between the file format and the category. The file format conversion unit 206 determines the category of the first terminal 3 based on the terminal ID and converts the sound data into a file format that corresponds to the determined category.

[0048] The removal unit 207 processes the audio data by removing or reducing data in frequency bands outside the audible range, outside the speech band, or that constitute noise, or by emphasizing only the speech band. This reduces the amount of data transmitted to the second terminal 4, thereby reducing communication costs. It should be noted that the removal unit 207 can also process the audio data by removing or reducing data in frequency bands outside the audible range, outside the speech band, or that constitute noise, or by emphasizing only the speech band. This reduces the amount of data transmitted to the second terminal 4, thereby reducing communication costs.

[0049] The volume adjustment unit 208 adjusts the playback volume when the second terminal 4 reproduces sound data. This allows sound data to be reproduced according to the level of the voice output device, such as the speaker, provided by the second terminal 4, thereby suppressing sound distortion. It should be noted that the configuration may also include storing data in the storage device 200B that establishes a correspondence between the category and terminal ID of the second terminal 4, and data that establishes a correspondence between the playback volume and the category. The volume adjustment unit 208 then determines the category of the second terminal 4 based on the terminal ID and adjusts the volume of the sound data to a playback volume corresponding to the determined category. It should also be noted that the volume adjustment unit 208 may adjust the playback volume when the first terminal 3 reproduces sound data. This allows sound data to be reproduced according to the level of the voice output device, such as the speaker, provided by the first terminal 3. It should be noted that the configuration may also include storing data in the storage device 200B that establishes a correspondence between the category and terminal ID of the first terminal 3, and data that establishes a correspondence between the playback volume and the category. The volume adjustment unit 208 then determines the category of the first terminal 3 based on the terminal ID and adjusts the volume of the sound data to a playback volume corresponding to the determined category.

[0050] The extraction unit 209 extracts data from the storage device 200B based on sorting conditions. For example, the extraction unit 209 extracts data specified by the sorting conditions.

[0051] In addition, the extraction unit 209 extracts data associated with the location information history (via the terminal ID), such as voice data.

[0052] (First Terminal 3) The first terminal 3 is a terminal held by the first user, such as a smartphone with application software installed to enable the first terminal 3 to function as a terminal with the functions shown in this embodiment. The first user can exchange voice and text data with a second user (e.g., their child) by sending and receiving voice and text data with the second terminal 4 registered using the first terminal 3. Figure 4 The main hardware structure of the first terminal 3 includes a communication IF300A, a storage device 300B, an input device 300C, a display device 300D, a CPU 300E, a microphone 300F, and a speaker 300G.

[0053] The communication IF300A is an interface used for communicating with other devices (server 2 in this embodiment).

[0054] Storage device 300B is, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive). Storage device 300B stores terminal IDs, information processing programs (application software), etc. The terminal ID is a number used to identify the first terminal 3. By assigning a terminal ID to data sent from the first terminal 3, server 2 can determine which first terminal 3 the received data originated from. It should be noted that the terminal ID can utilize an IP (Internet Protocol) address, MAC (Media Access Control) address, etc., or it can be assigned by server 2 to the first terminal 3.

[0055] Additionally, the storage device 300B may also store data transmitted and received between the first terminal 3 and the second terminal 4. The storage device 300B may also store, for example, transmitted and received data (e.g., voice data, text data).

[0056] The input device 300C can be, for example, a keyboard, mouse, touch panel, or other input devices, but it can also be any other device or equipment that can input. Additionally, it can be a voice input device.

[0057] The display device 300D is, for example, a liquid crystal display, a plasma display, an organic EL display, etc., but as long as it can display, it can also be other devices or equipment (e.g., CRT: CRT display).

[0058] The CPU300E controls the first terminal 3 involved in this embodiment and has ROM and RAM (not shown).

[0059] Microphone 300F is an audio device that converts sound into electrical signals. The user of the first terminal 3 can use microphone 300F to input voice. The input voice is sent to server 2 by the transmitting unit 302, described later.

[0060] The loudspeaker 300G is an audio device that converts electrical signals into sound. For example, the loudspeaker 300G reproduces sound data that is transmitted from the second terminal 4 via the server 2 and stored in the storage device 300B.

[0061] It should be noted that, similar to the second terminal 4 described later, it may also have a position sensor that determines its current position based on signals from GNSS satellites and outputs the current position determined by the position sensor. It should be noted that as long as the position of the first terminal 3 can be determined, the current position can also be output using methods other than a position sensor. Methods other than a position sensor include, for example, utilizing wireless access point information, utilizing portable network tower information, utilizing accelerometers and magnetic sensors to measure the movement and direction of travel of the first terminal 3. It should be noted that, of course, methods other than those illustrated are also possible.

[0062] Figure 5 This diagram illustrates the functions of the first terminal 3. The first terminal 3 includes a receiving unit 301, a transmitting unit 302, a storage device control unit 303, an input receiving unit 304 (receiving unit), a display device control unit 305 (display control unit), and a voice output control unit 306. Further explanation is needed. Figure 5 The functions shown are implemented by CPU 300E executing information processing programs stored in storage device 300B.

[0063] For example, receiving unit 301 receives data sent from server 2.

[0064] For example, the sending unit 302 sends data to the server 2 based on the input operation accepted by the input receiving unit 304.

[0065] The storage device control unit 303 controls the storage device 300B. For example, the storage device control unit 303 stores data sent and received by the first terminal 3 and the second terminal 4 in the storage device 300B in a manner corresponding to the terminal IDs of the sending and receiving data. Additionally, the storage device control unit 303 stores transmitted and received voice data and text data in the storage device 300B.

[0066] The input receiving unit 304 accepts input operations from the input device 300C. For example, the input receiving unit 304 accepts message input.

[0067] The display device control unit 305 controls the display device 300D to display data received by the receiving unit 301. The display device control unit 305 functions, for example, as a notification control unit, and has a push notification function that causes the display device 300D to display a message indicating that sound data has been received if sound data is received by the receiving unit 301.

[0068] The voice output control unit 306 controls the speaker 300G. The speaker 300G outputs voice based on the control of the voice output control unit 306. The voice output control unit 406 functions, for example, as a notification control unit, notifying the receiver of the received voice data by reproducing at least a portion of the voice data. It should be noted that the voice output control unit 406 may also notify the receiver of the received voice data by generating sound in a given pattern if the receiver 301 receives the voice data.

[0069] (Second Terminal 4) The second terminal 4 is a terminal used by a second user of this information processing system 1. The second user can exchange voice data and text data with the first terminal 3 registered using the second terminal 4, thereby enabling the second user (e.g., a family member) to conduct voice-based communication.

[0070] Figure 6 This indicates the main hardware structure of the second terminal 4, which includes a communication IF 400A, a storage device 400B, an input device 400C, a display device 400D, a CPU 400E, a microphone 400F, a speaker 400G, and a position sensor 400H.

[0071] The IF400A communication interface is used to communicate with other devices (server 2 in this embodiment).

[0072] Storage device 400B is, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive). Storage device 400B stores terminal IDs, information processing programs, and voice data transmitted from the first terminal 3. The terminal ID is a number used to identify the second terminal 4. By assigning a terminal ID to data transmitted from the second terminal 4, the server 2 can determine which second terminal 4 the received data originated from. It should be noted that the terminal ID can utilize an IP (Internet Protocol) address, a MAC (Media Access Control) address, etc., or it can be assigned by the server 2 to the second terminal 4.

[0073] Alternatively, the storage device 400B may store transmitted and received data (e.g., voice data).

[0074] The input device 400C can be an input device such as an operation button, but it can also be other devices or equipment, such as a keyboard, mouse, or touch panel, as long as it can input. Alternatively, it can be a voice input device. The second user can operate the input device 400C to input voice and send it to the server 2, or to reproduce sound data sent from the server 2.

[0075] The display device 400D is, for example, an LED. The display device 400D notifies the receipt of voice by lighting up or flashing in a given pattern.

[0076] The CPU400E controls the second terminal 4 involved in this embodiment and has ROM and RAM (not shown).

[0077] Microphone 400F is an audio device that converts sound into electrical signals. The user of the second terminal 4 can use microphone 400F to input voice. The input voice is sent to server 2 by the transmitting unit 402, described later.

[0078] The speaker 400G is an audio device that converts electrical signals into sound. For example, the speaker 400G reproduces sound data transmitted from the first terminal 3 via the server 2 and stored in the storage device 400B. Furthermore, the speaker 400G notifies the reception of voice by generating sound in a given pattern.

[0079] The position sensor 400H determines its current position based on signals from GNSS satellites. Furthermore, the position sensor 400H outputs the determined current position. It should be noted that as long as the position of the second terminal 4 can be determined, the current position can also be output using methods other than the position sensor 400H. Methods other than the position sensor 400H include, for example, utilizing wireless access point information, utilizing portable network tower information, and utilizing accelerometers and magnetometers to measure the movement and direction of travel of the second terminal 4. It should be noted that methods other than those illustrated are also possible.

[0080] Figure 7 This is a functional block diagram of the second terminal 4. The second terminal 4 has the functions of a receiving unit 401, a transmitting unit 402, a storage device control unit 403, an input receiving unit 404, a display device control unit 405 (display control unit), and a voice output control unit 406. It should be explained that... Figure 7 The functions shown are implemented by CPU 400E executing information processing programs stored in storage device 400B.

[0081] The receiving unit 401 receives, for example, data sent from the server 2, such as audio data.

[0082] The sending unit 402 sends, for example, voice data to the server 2 based on the input operation received by the input receiving unit 304.

[0083] The storage device control unit 403 controls the storage device 400B. For example, the storage device control unit 403 controls the storage device 400B to write and read data. For example, the storage device control unit 403 stores the data received by the receiving unit 401 into the storage device 400B.

[0084] The input receiving unit 404 accepts input operations from the input device 400C. For example, the input receiving unit 404 accepts the playback operation of sound data stored in the storage device 400B.

[0085] The display device control unit 405 controls the display device 400D. For example, the display device control unit 405 functions as a notification control unit. If sound data is received by the receiving unit 401, it causes the display device 400D (LED) to light up or flash in a given mode.

[0086] The voice output control unit 406 controls the speaker 400G. The speaker 400G outputs voice based on the control of the voice output control unit 406. The voice output control unit 406 functions, for example, as a notification control unit; if sound data is received by the receiving unit 401, it notifies the receiver of the received voice by generating sound in a given pattern. It should be noted that the voice output control unit 406 can also notify the receiver of the received sound data by reproducing at least a portion of the sound data.

[0087] (Screenshot) Figures 8-11 This is a diagram showing an example of screen G1 displayed on the display device 300D of the first terminal 3. Hereinafter, refer to... Figures 8-11 An example of screen G1 displayed on the display device 300D of the first terminal 3 will be described. It should be noted that, compared with the reference... Figures 1-7 Structures with the same structure should be labeled with the same number and repeated descriptions should be omitted.

[0088] ( Figure 8 (Image description) like Figure 8 As shown, in the display device 300D, the sound data transmitted and received between the first terminal 3 and the second terminal 4 is converted into text data of characters and displayed in chronological order as a unit of sound data files (hereinafter also referred to as timeline display).

[0089] exist Figure 8In the example shown, the name 11 (or pseudonym) of the second user is displayed at the top of screen G1. Additionally, on the left side of screen G1, text data 12B, converted from the audio data (the second user's voice file) sent from the second terminal 4, is displayed along with the sending date and time 12D (using timestamp information) and an icon 12A. Furthermore, selecting the play button 12C allows playback of the audio data (voice file) corresponding to the displayed text data 12B, and the user can listen to the audio. On the right side of screen G1, text data 13A, converted from the audio data (the first user's voice file) sent from the first terminal 3, is displayed along with the sending date and time 13C (using timestamp information). Additionally, each piece of text data sent from the first terminal 3 includes a status 13D (e.g., whether the audio data was played back on the second terminal 4). Selecting the play button 13B allows playback of the audio data (voice file) corresponding to the displayed text data 13A, and the user can listen to the audio.

[0090] It should be noted that, Figure 8 In the example shown, the text data converted from the audio data is displayed in chronological order, unit by file of audio data. However, it is also possible to configure the text data 12B converted from the audio data not to be displayed. In this case, for example, the playback time of the audio data could be displayed instead of the text data 12B, but it is not limited to this example.

[0091] In addition, Figure 8 In the example shown, all the characters after the sound data is converted are displayed on the screen, but it is not necessary to display all the characters. For example, the initial few characters (e.g., 10 characters) can also be displayed.

[0092] ( Figure 9 (Image description) Figure 9 The screen G2 shown is for selecting (clicking) references. Figure 8 The icon 15 indicates the screen that migrates. (For example...) Figure 9 As shown, if in Figure 8When icon 15 is selected, the display box 16 for sending messages to the second terminal 4 appears raised from the bottom of screen G2. The first user can select the microphone icon 16A displayed in the display box 16 of screen G2 by operating the input device 300C, such as a touch panel, to record voice and send it to the second terminal 4. It should be noted that if microphone icon 16A is selected, the display mode of microphone icon 16A (e.g., the color of the icon) changes to indicate that recording has started. If the first user finishes saying a message (e.g., "I'm coming to pick you up now") and selects microphone icon 16A again, the display mode of microphone icon 16A (e.g., the color of the icon) returns to normal, indicating that recording has ended.

[0093] In addition, below the microphone icon 16A, there are multiple messages 16B preset in the storage device 300B, messages 16B input by the first user, messages 16B downloaded from the server 2, etc. The first user can select any one of the multiple messages 16B displayed by operating the input device 300C such as the touch panel, and can send the selected message 16B to the second terminal 4.

[0094] Additionally, it is preferred to display in Figure 9 Message 16B on screen G2 is displayed in an order corresponding to the message usage frequency of the first user or the settings of the first user. For example, when displaying messages based on usage frequency, it is preferable to display messages in descending order of usage frequency. Additionally, messages displayed on... Figure 9 Message 16B of screen G2 can also be displayed in the order corresponding to the settings of the first user. In this case, it can also be displayed in accordance with the settings described later. Figure 10 Similarly, as described above, the first user can manually rearrange and set the display order of messages 16B by operating the input device 300C. Alternatively, the settings can be configured such that messages 16B are displayed in descending order of usage frequency (pre-set), or that messages 16B input by the first user are displayed in descending order of usage frequency. It should be noted that when displaying messages in descending order of usage frequency, it is preferable to consider inconsistencies in message descriptions (e.g., message similarity) when calculating message usage frequency.

[0095] ( Figure 10 (Image description) It should be noted that the message input by the first user, as already described, includes voice messages recorded by the first user using microphone 300F and character messages input by the first user via text input device 300C. Here, the voice messages are converted into text and displayed as the title of the voice file.

[0096] Figure 10This is an example of screen G3 that converts voice messages into characters (text) and displays them as the title of the voice file. For example... Figure 10 As illustrated, on screen G3, a message containing a voice file obtained by converting a voice message into text is displayed as title 16B. Additionally, the first terminal 3 accesses server 2 to query the first user's purchase information in the voice stamp shop (described later). If a purchase history exists, the voice stamp and its title (described later) are also displayed on screen G3. It should be noted that the voice stamp title can also be displayed differently from the message entered by the first user (e.g., changing the font, adding an asterisk or other symbol before the title). The first user selects the voice stamp shop 17A on screen G3 by operating an input device 300C such as a touch panel, thereby displaying a migration or voice stamp shop on a voice stamp shop screen (not shown), allowing them to purchase or download voice stamps (voice files) different from the voice message entered by the first user free of charge. For example, if a voice stamp purchase is made in the voice stamp store, server 2 records the first user's purchase information and can use the voice file corresponding to the purchased voice stamp stored in server 2 (e.g., voice files of messages recorded by voice actors, actors, idols, etc., voice files of messages based on voice synthesis engines such as VOCALOID (registered trademark)).

[0097] In addition, the first user can select (long press or click, etc.) the replacement button 17B located on the left side of the title 16B by operating the input device 300C such as the touch panel, thereby changing the display order of the title 16B.

[0098] Additionally, the first user can select (by long-pressing or clicking) the edit button 17C located on the right side of title 16B via an input device 300C such as a touch panel, thereby editing the volume, playback speed, pitch, playback start time, playback end time, etc., of the voice file that forms the basis of title 16B. However, if the voice file is a voice stamp, the edit button 17C may not be displayed.

[0099] In addition, the first user can select (long press or click, etc.) the playback button 17D located on the right side of title 16B by operating the input device 300C such as the touch panel, thereby playing back (listening to) the audio file that is the source of title 16B.

[0100] In addition, the first user can select (long press or click, etc.) the recording button 17E located on the right side of title 16B by operating the input device 300C such as the touch panel, thereby re-recording the voice file that became the source of title 16B.

[0101] It should be noted that the title 16B of the audio files can also be set to display in descending order of usage frequency. It should also be noted that when displaying in descending order of usage frequency, it is preferable to consider inconsistencies in message expression (e.g., message similarity) when calculating message usage frequency.

[0102] ( Figure 11 (Image description) like Figure 11 As shown, in this embodiment, the history of the location P of the second terminal 4 can be displayed on the display device 300D of the first terminal 3, overlapping with the map. Furthermore, the current time T and the username U of the second terminal 4 are displayed at the top of the display screen 300D. A calendar C and a slider S are displayed at the bottom of the display screen 300D. The first user selects a date by clicking the calendar C, thereby displaying the history of the location P of the second terminal 4 for the selected date. Additionally, the first user can zoom in or out of the map by moving the slider S left or right. It should be noted that when data associated with location P exists, such as audio data, a dialog box F is displayed at the corresponding location P, indicating the existence of associated data (e.g., audio data). Furthermore, in this embodiment, the status of the associated data (whether it has been confirmed, etc.) is shown. For example, if the associated data is audio data, and the audio data has been completely reproduced in the first terminal 3, it is marked as "read". In this way, not only can the history of the location be confirmed, but also what kind of information was associated at which location, thus providing high convenience.

[0103] It should be noted that the second terminal 4 may also be configured to include a display device such as a liquid crystal display, a plasma display, or an organic EL display, capable of displaying... Figures 8-11 The illustrated screen is displayed on the first terminal 3. In this case, the relationship between the first terminal 3 and the second terminal 4 is as follows: Figures 8-11 The opposite is true in the illustrated screen.

[0104] (Information Processing) Figures 12-17 This is a flowchart illustrating an example of information processing in information processing system 1. The following refers to... Figures 12-17 The information processing of information processing system 1 shall be described. It should be explained that, in relation to reference... Figures 1-11 Structures with the same structure should be labeled with the same number and repeated descriptions should be omitted.

[0105] (Location information transmission and processing) Figure 12 This is a flowchart illustrating an example of location information transmission and processing in information processing system 1. The following refers to... Figure 12 An example of location information transmission and processing in information processing system 1 will be described.

[0106] (Step S101) The transmitting unit 402 of the second terminal 4 determines whether it is a given time interval (e.g., every 1.5 minutes). If it is a given time interval (yes), the server 2 executes the processing in step S102. If it is not a given time interval (no), the server 2 ends the processing.

[0107] (Step S102) The transmitting unit 402 of the second terminal 4 sends the location information output from the position sensor 400H to the server 2. It should be noted that the data sent from the second terminal 4 is assigned data such as the terminal ID of the second terminal 4 and a timestamp (sending date and time, location information positioning date and time).

[0108] (Step S103) The receiving unit 201 of server 2 receives location information sent from the second terminal 4.

[0109] (Step S104) The storage device control unit 203 of server 2 stores the location information (including metadata) sent and received by the second terminal 4 in storage device 200B in a manner that establishes a correspondence with the terminal ID of the terminal that sent and received the data.

[0110] (Message sending and processing 1) Figure 13 This is a flowchart illustrating an example of message sending process 1 in information processing system 1. The following refers to... Figure 13 An example of message sending processing 1 in information processing system 1 will be described. It should be noted that, in Figure 13 The following section describes the process of sending messages from the first terminal 3 to the second terminal 4.

[0111] (Step S201) The first user operates the input device 400C of the first terminal 3 to input messages via voice or text.

[0112] (Step S202) When voice input is received, it is converted into an electrical signal by microphone 400F, then converted into digital data by an A / D converter (not shown), and sent as audio data from transmitter 302 to server 2. Similarly, when text input is received, it is sent as text data from transmitter 302 to server 2. It should be noted that data sent from first terminal 3 is assigned data such as terminal ID and timestamp of first terminal 3.

[0113] (Step S203) The receiving unit 201 of server 2 receives data sent from the first terminal 3. The voice conversion unit 205 of server 2 determines whether the receiving unit 201 has received audio data. If audio data has been received (yes), server 2 executes the process of step S205. If no audio data has been received (no), server 2 executes the process of step S204.

[0114] (Step S204) The speech conversion unit 205 of server 2 converts the received text data into sound data. It should be noted that when converting the received text data into speech, it can also be configured to reproduce sound data from which the original language has been changed to another language via a language conversion unit (not shown), or sound data from which one or more other languages ​​different from the original language have been changed. In this case, it is preset by the first terminal 3.

[0115] (Step S205) When the storage device control unit 203 of server 2 receives audio data, it stores the audio data sent by the first terminal 3 in the storage device 200B in a manner that establishes a correspondence with the terminal ID of the data sender and receiver. Furthermore, when the storage device control unit 203 receives text data, it stores the text data sent by the first terminal 3 and the audio data converted from the text data in the storage device 200B in a manner that establishes a correspondence with the terminal ID of the data sender and receiver.

[0116] (Step S206) Server 2 performs audio data conversion processing. (For further explanation, please refer to...) Figure 15 The details of audio data conversion and processing will be described later.

[0117] (Step S207) The transmitting unit 202 of server 2 refers to storage device 200B, determines the terminal ID of the second terminal 4 that corresponds to the terminal ID assigned to the voice data received by receiving unit 201, and sends the voice data sent from first terminal 3 or the voice conversion unit 205 converted to the determined second terminal 4.

[0118] (Step S208) The receiving unit 401 of the second terminal 4 receives audio data sent from the server 2. The second terminal 4 notifies the recipient of the received message. This notification can be made, for example, by the display device control unit 405 controlling the display device 400D, such as by lighting an LED or making a sound, to notify the recipient of the received message. Alternatively, the voice output control unit 406 can control the speaker 400G to output sound or voice, thereby notifying the recipient of the received message.

[0119] (Message sending and processing 2) Figure 14 This is a flowchart illustrating an example of message sending and processing 2 in information processing system 1. The following refers to... Figure 14 An example of message sending processing 2 in information processing system 1 will be described. It should be noted that... Figure 14 The following section describes the situation where a message is sent from the second terminal 4 to the first terminal 3.

[0120] (Step S301) The second user operates the input device 400C of the second terminal 4 to input voice. The input voice is converted into an electrical signal by the microphone 400F.

[0121] (Step S302) The transmitting unit 402 of the second terminal 4 sends the input audio data to the server 2. It should be noted that the data sent from the second terminal 4 is assigned data such as the terminal ID and timestamp of the second terminal 4.

[0122] (Step S303) The receiving unit 201 of server 2 receives the audio data sent from the second terminal 4.

[0123] (Step S304) The text conversion unit 204 of server 2 converts the received audio data into text data. It should be noted that when converting the received audio data into text data, it can also be configured to simultaneously display text data whose original language has been changed to another language by a language conversion unit (not shown), and text data changed to one or more other languages ​​different from the original language. In this case, it is preset by the first terminal 3.

[0124] (Step S305) The storage device control unit 203 of server 2 stores the voice data sent by the second terminal 4 and the text data converted by the text conversion unit 204 in the storage device 200B in a manner that establishes a correspondence with the terminal ID of the sending and receiving data.

[0125] (Step S306) The sending unit 202 of server 2 refers to storage device 200B, determines the terminal ID of the first terminal 3 that corresponds to the terminal ID assigned to the sound data received by receiving unit 201, and sends the sound data sent from second terminal 4 and the text data converted by text conversion unit 204 to the determined first terminal 3.

[0126] (Step S307) The receiving unit 301 of the first terminal 3 receives audio data and text data sent from the server 2.

[0127] (Step S308) The receiving unit 301 of the first terminal 3 receives audio data and text data sent from the server 2. The first terminal 3 notifies the recipient of the message. This notification can be made, for example, by the display device control unit 305 controlling the display device 300D to notify the recipient of the message via a push notification, or by the voice output control unit 306 controlling the speaker 300G to output audio or voice, thereby notifying the recipient of the message.

[0128] (Audio data conversion and processing) Figure 15 This is a flowchart illustrating an example of sound data conversion and processing in information processing system 1. The following refers to... Figure 15 An example of sound data conversion and processing in information processing system 1 will be described.

[0129] (Step S401) The removal unit 207 of server 2 processes the sound data by removing or reducing data in frequency bands outside the audible region, outside the speech band, or that become noise in the sound data received by receiving unit 201, or by emphasizing only the speech band.

[0130] (Step S402) The volume adjustment unit 208 of server 2 adjusts the playback volume when the sound data is reproduced on the second terminal 4.

[0131] (Step S403) The file format conversion unit 206 of server 2 converts the file format of the sound data.

[0132] It should be explained, see reference. Figure 15 The steps of the audio data conversion process described do not necessarily need to be performed in the order of steps S401 to S403. The order can be reversed. Alternatively, any one or more of the steps S401 to S403 can be performed in any order.

[0133] It should be noted that, as described above, when sending voice data from the second terminal 4 to the first terminal 3, the reference can also be executed. Figure 15 The audio data conversion and processing described herein.

[0134] (TL display processing) Figure 16 This is a flowchart illustrating an example of TL display processing in information processing system 1. The following refers to... Figure 16 An example of TL display processing in information processing system 1 will be described.

[0135] (Step S501) A first user operates the input device 300C of the first terminal 3 to input sorting criteria for the data to be displayed. The input receiving unit 304 of the first terminal 3 accepts the input sorting criteria, and the sending unit 302 sends the sorting criteria accepted by the input receiving unit 304 to the server 2. The receiving unit 201 of the server 2 receives the sorting criteria sent from the first terminal 3.

[0136] (Step S502) The extraction unit 209 of server 2 extracts data from storage device 200B based on sorting criteria. For example, the extraction unit 209 extracts data specified by the sorting criteria (e.g., location information, resume data, and voice data).

[0137] (Step S503) The sending unit 202 of server 2, referring to storage device 200B, determines the terminal ID of the first terminal 3 that corresponds to the terminal ID assigned to the data received by receiving unit 201, and sends the data extracted by extraction unit 209 to the determined first terminal 3. The receiving unit 301 of the first terminal 3 receives the data sent from server 2. The display device control unit 305 of the first terminal 3 displays the data received by receiving unit 301 on display device 300D.

[0138] It should be noted that, as mentioned above, the second terminal 4 may also include a display device such as a liquid crystal display, a plasma display, or an organic EL display, to perform the reference... Figure 16 The TL display processing is explained.

[0139] (Map display processing) Figure 17 This is a flowchart illustrating an example of map display processing in information processing system 1. The following refers to... Figure 17 An example of map display processing in information processing system 1 will be described.

[0140] (Step S601) A first user operates the input device 300C of the first terminal 3 to instruct map display. The input receiving unit 304 of the first terminal 3 receives the map display instruction. The sending unit 302 of the first terminal 3 sends the map display instruction to the server 2. The receiving unit 201 of the server 2 receives the map display instruction sent from the first terminal 3.

[0141] (Step S602) The extraction unit 209 of server 2 refers to storage device 200B, determines the terminal ID of the second terminal 4 that corresponds to the terminal ID assigned to the instruction displayed on the map, and extracts the history of the determined location information of the second terminal 4.

[0142] (Step S603) In addition, the extraction unit 209 of server 2 extracts data associated with the history of location information (via terminal ID), such as voice data.

[0143] (Step S604) The sending unit 202 of server 2 refers to storage device 200B, determines the terminal ID of the first terminal 3 that corresponds to the terminal ID of the data received by receiving unit 201, and sends data, such as voice data, that is associated with the history of the location information extracted by extraction unit 209 and the history of the location information to the determined first terminal 3.

[0144] (Step S605) The receiving unit 301 of the first terminal 3 receives the history of the location information sent from the server 2 and the data associated with the history of the location information.

[0145] (Step S606) The display device control unit 305 of the first terminal 3 causes the display device 300D to display the history of the location information received by the receiving unit 301 on a map (for example, referring to...). Figure 11 Additionally, the display device control unit 305 of the first terminal 3 enables the display device 300D to display data associated with the location information displayed on the map in or near the resume.

[0146] It should be noted that, although in Figure 17 In the illustrated map display process, server 2 sends the extracted location information history and the audio data associated with the location information history (via terminal ID) to the first terminal 3. However, the following structure can also be adopted: send the extracted location information history and data indicating the existence of audio data associated with the location information history (via terminal ID) to the first terminal 3 (e.g., sign information, etc.), and download and reproduce the audio data that the user instructed to reproduce.

[0147] It should be noted that, as described above, the second terminal 4 may also include a display device such as a liquid crystal display, a plasma display, or an organic EL display, and perform the reference... Figure 17 The map display process is explained.

[0148] According to the information processing system 1 of this embodiment, it is capable of converting sound data into text data and sending or receiving it, or converting text data into sound data and sending or receiving it. Therefore, in various scenarios where one of the first user and the second user (or both) has difficulty performing text or voice input (e.g., children, the elderly, or users with speech impairments who have difficulty operating complex terminals), terminals that have difficulty performing text or voice input (e.g., small devices that are difficult to install keyboards, or situations where displays are difficult to install due to cost constraints), scenarios where text input is difficult due to hands being occupied, or scenarios where voice recording or reproduction is difficult in public spaces, communication between the two (the first user and the second user) is possible.

[0149] [Variations on the implementation method] It should be noted that in the above embodiment, text can also be displayed using the LEDs provided by the display device 400D of the second terminal 4. Alternatively, a liquid crystal panel or an organic EL panel can be used as the display device 400D of the second terminal 4, and configured to display text. With this configuration, audio data sent from the first terminal 3 and the server 2 can be converted into text data and confirmed via text. Furthermore, text data sent from the first terminal 3 and the server 2 can also be confirmed.

[0150] It should be noted that, in the above embodiments, at least one of the first terminal 3 and the second terminal 4 may also have Figure 3 The server 2 shown may have some or all of the functions it possesses. For example, at least one of the first terminal 3 and the second terminal 4 may also have... Figure 3 The server 2 shown has at least one of the following functions: storage device control unit 203 (association unit), text conversion unit 204, voice conversion unit 205, file format conversion unit 206, removal unit 207, and volume adjustment unit 208. It should be noted that in this case, information stored in storage device 200B can be obtained or referenced by at least one of the first terminal 3 and the second terminal 4, or the information stored in storage device 200B can be stored in at least one of storage devices 300B and 400B.

[0151] In addition, in the above embodiments, the second terminal 4 can have at least one or more of the functions of the first terminal 3, and the first terminal 3 can have at least one or more of the functions of the second terminal 4.

[0152] Alternatively, the reference can be... Figure 8 The screen G1 is set to [specify]. Figure 18 The image shown is G4. Figure 18 In the screen G4 shown, there is an input box 18A for inputting text messages and a microphone icon 18B for inputting voice messages, which allows the first user to choose whether to input a message via text or voice.

[0153] It should be noted that in the input text message, characters without sound (such as labels) or combinations thereof (such as surrounding characters with sound with the aforementioned symbols) can also be used to change the sound source (the tone of speech, the timbre of an instrument) or to specify the volume, interval, etc. in detail.

[0154] in addition, Figure 19 The screen shown in G5 is the screen that appears after a voice or text message has been entered.

[0155] like Figure 19 As shown, the first user can select icon 19A on screen G5 by operating input device 300C such as a touch panel, thereby listening to and confirming the entered voice or text message. Additionally, the first user can select icon 19B on screen G5 by operating input device 300C such as a touch panel, and can re-enter the voice or text message.

[0156] Additionally, the first user can set a notification mode when notifying the second user terminal 4 of a message received from the first user terminal 3. Figure 19 In the example shown, the first user can select a notification mode from "Incoming call tone on" 20A, "Incoming call tone off" 20B, and "Set this voice as the incoming call tone" 20C. It should be noted that in... Figure 19 In the example shown, the notification pattern is selected from three options, but the notification pattern is not limited to these three. Figure 19 The example shown.

[0157] Additionally, in screen G5, the tone and emotion of the voice message can be specified for reproduction. If the first user selects a tone from option 21 and an emotion from option 22, the voice message will be reproduced using the tone and emotion selected by the second terminal 4 that sent the voice message. It should be noted that the expression of tone and emotion can be achieved by learning human voices through AI machine learning and performing speech synthesis.

[0158] Figure 20 and Figure 21 This diagram illustrates the meaning of receiving sound or text data on the display device 300D via the PUSH notification function.

[0159] It is possible Figure 20As shown, a push notification can be sent only to indicate the meaning of the received message and the username of the user who sent the message. Alternatively, it can be done as follows: Figure 21 As shown, a push notification is sent to the content of the received message (in the case of a long message, such as when it exceeds a given number of characters, either only the initial given number of characters of the message content are displayed, or a summary of the content is displayed) and the username of the user who sent the message. It should be noted that in... Figure 21 In the example shown, the method of reproducing the message notified by PUSH ("hands-free reproduction", "reproducing by holding it close to the ear", "opening the application", etc.) and responding to the message (selecting a pre-registered message (preset message, user-registered message, etc.) and sending it) can be performed without opening the application, which is highly convenient.

[0160] (Processing when a new voice is received before the voice received by the second terminal 4 is heard) After receiving audio data sent from server 2 by second terminal 4, there may be a situation where new audio data is sent from server 2 before being reproduced by second terminal 4. In this case, regarding the holding of audio data, second terminal 4 performs any of the processes illustrated in (1) to (3) below. It should be noted that the following processes are performed by second terminal 4 because the storage capacity of storage device 400B of second terminal 4 is small.

[0161] (1) Upon receiving new audio data, delete the previously received audio data from the storage device 400B.

[0162] (2) Store audio data in an audio data buffer (e.g., set in a given area of ​​storage device 400B) that can store any number of data or storage capacity, and delete old data in a FIFO (first in first out) manner when the audio data buffer is about to overflow.

[0163] (3) Store audio data in an audio data buffer (e.g., set in a given area of ​​storage device 400B) that can store any number of data or storage capacity. When the audio data buffer is about to overflow, send a buffer full message (indicating that audio data cannot be further stored in the second terminal 4) to the first terminal 3 through the server 2, and display the message that the message cannot be sent on the first terminal 3.

[0164] In addition, when the sound data is reproduced in the second terminal 4, the second terminal 4 reproduces the sound data on the buffer in the order of the saved data or the order of the new sound data. However, the sound data on the sound data buffer can be deleted afterward, or the data other than the last received sound data can be deleted, or the above (2) process can be performed without deletion.

[0165] It should be noted that these settings can be changed from the first terminal 3.

[0166] (Dialogue Group) In the above embodiments and variations, one first terminal 3 and one second terminal 4 were described as a pair. However, multiple first terminals 3 and / or second terminals 4 can also be configured as a dialog group. The dialog group is managed in the server 2 by a unique ID (dialogue group ID), and the identification numbers (terminal IDs) of all first terminals 3 and second terminals 4 belonging to the same dialog group are associated. It should be noted that each terminal can also be given a name to make it easy for users belonging to the dialog group to understand. In this case, the terminal ID is also associated with the terminal name.

[0167] Furthermore, terminals within the same chat group can send messages simultaneously, or messages can be sent only to specific terminals. For example, when sending from the first terminal 3, one or more terminals can be selected to send messages in the text or voice message sending screen. Additionally, when sending a voice message from the second terminal 4, the user can speak the name of the terminal to which they wish to send the message in the voice message, or select the terminal by pressing the button for sending a voice message to the second terminal 4 at any time, or have the second terminal 4's speaker 400G prompt "Who are you sending this to?" by pressing the button for sending a voice message at any time. If the second user inputs the names of one or more terminals to which they wish to send the message via voice, the message to the second user can be processed after confirming the name of the terminal to which the message is intended, and then sent via the server 2 to the target terminal within the same chat group.

[0168] Furthermore, the above-described embodiments and variations are merely illustrative examples of implementing the present invention and should not be construed as limiting the scope of the invention. That is, the present invention can be implemented in various forms without departing from its spirit or main features.

[0169] Based on the above, consider, for example, the following approach. [1] An information processing device comprising: A receiving unit that receives audio data transmitted from a first terminal or a second terminal; A text conversion unit that converts the audio data received by the receiving unit into text data; and The sending unit sends the text data converted by the conversion unit to the second terminal or the first terminal. [2] An information processing device comprising: The receiving unit receives text data sent from the first terminal or the second terminal; A speech conversion unit that converts the text data received by the receiving unit into sound data; and The transmitting unit sends the audio data converted by the conversion unit to the second terminal or the first terminal. [3] Based on the information processing device described in [1], it has the following features: An association unit is established, which associates the sound data with information used to determine the location of the first terminal or the second terminal. The receiving unit receives location information sent from the first terminal or the second terminal. The association establishment unit establishes an association between the sound data received by the receiving unit and the information used to determine the location. The transmitting unit sends the sound data and the location information associated with the sound data to the second terminal or the first terminal. [4] Based on the information processing device described in [2], it has the following features: An association unit is established, which associates the text data with information used to determine the location of the first terminal or the second terminal. The receiving unit receives location information sent from the first terminal or the second terminal. The association establishment unit establishes an association between the text data received by the receiving unit and the information used to determine the location. The transmitting unit sends the text data and the location-determining information associated with the text data to the second terminal or the first terminal. [5] Based on the information processing apparatus described in [3], the association establishment unit associates the information used to determine the location of the first terminal or the second terminal with the sound data based on the information used to determine the location, at least one of the location date and time, the sending date and time, and the receiving date and time of the information used to determine the location, and at least one of the recording date and time, the sending date and time, and the receiving date and time of the sound data. [6] Based on the information processing device described in [1], it has the following features: The format conversion unit converts the file format of the sound data. [7] Based on the information processing device described in [1], it has the following features: The removal section removes or reduces data in the audio data that is outside the audible region, outside the speech band, or constitutes noise, or emphasizes only the speech band. The transmitting unit sends to the second terminal or the first terminal data that has been removed or reduced by the removing unit, data outside the audible region, data outside the speech band, or data that has become noise, or data that has only emphasized the speech band. [8] Based on the information processing device described in [1], it has the following features: The volume adjustment unit adjusts the reproduction volume of the sound data transmitted by the transmitting unit to the second terminal or the first terminal. [9] An information processing terminal, comprising: A receiving unit that receives audio data transmitted from a first terminal or a second terminal; A text conversion unit that converts the audio data received by the receiving unit into text data; and The output unit outputs the text data converted by the conversion unit.

[10] An information processing terminal, comprising: The receiving unit receives text data; A speech conversion unit that converts the text data received by the receiving unit into sound data; and The output unit outputs the audio data converted by the conversion unit.

[11] An information processing terminal, comprising: The reception department accepts input of voice data; A text conversion unit converts the audio data received by the receiving unit into text data; and The sending unit transmits the text data converted by the conversion unit.

[12] An information processing terminal, comprising: The receiving department is responsible for accepting and inputting text data. A speech conversion unit that converts the text data received by the receiving unit into sound data; and The transmitting unit transmits the audio data converted by the conversion unit.

[13] Based on the information processing terminal described in [9], it includes: an association establishment unit that associates the sound data with information used to determine the location. The receiving unit receives the information used to determine the location. The association establishment unit establishes an association between the sound data received by the receiving unit and the information used to determine the location.

[14] Based on the information processing terminal described in

[10] , it has the following features: An association unit is established, which associates the text data with information used to determine the location. The receiving unit receives the information used to determine the location. The association establishment unit establishes an association between the text data received by the receiving unit and the information used to determine the location.

[15] Based on the information processing terminal described in

[11] , it has the following features: The volume adjustment unit adjusts the reproduction volume of the sound data transmitted by the transmitting unit.

[16] Based on the information processing terminal described in [9] or

[10] , it has the following features: The control unit is notified that when the receiving unit receives the audio data or the text data, it notifies the control unit that the audio data or the text data has been received.

[17] Based on the information processing terminal described in

[16] , the notification control unit notifies that the sound data or the text data has been received by reproducing at least a portion of the sound data or the text data.

[18] Based on the information processing terminal described in

[13] , the association establishment unit associates the location information of other information processing terminals that sent the sound data with the sound data based on the location information used to determine the location, at least one of the location date and time, the sending date and time, and the receiving date and time of the sound data, and at least one of the recording date and time, the sending date and time, and the receiving date and time of the sound data.

[19] Based on the information processing terminal described in [9], it has the following features: The format conversion unit converts the file format of the sound data.

[20] Based on the information processing terminal described in

[11] , it has the following features: The removal section removes or reduces data in frequency bands outside the audible range, outside the speech band, or that constitute noise, or emphasizes only the speech band. The transmitting unit sends data outside the audible area or outside the audio band that has been removed or reduced by the removing unit, or audio data that only emphasizes the audio band, to other information processing terminals. [twenty one] Based on the information processing terminal described in

[11] , it has the following features: The volume adjustment unit adjusts the reproduction volume of the sound data transmitted by the transmitting unit to other information processing terminals. [twenty two] An information processing terminal, comprising: A receiving unit, configured to receive audio data and text data converted from the audio data; and The display control unit displays the presence of the sound data received by the receiving unit in correspondence with at least a portion of the text data. [twenty three] An information processing terminal, comprising: The receiving unit receives information for determining the location of other information processing terminals, as well as sound data or text data obtained by converting the sound data; and The display control unit displays the information received by the receiving unit for determining the location of other information processing terminals in correspondence with the presence of the sound data or the text data. [twenty four] Based on the information processing terminal described in

[22] , the display control unit displays the information received by the receiving unit for determining the location of other information processing terminals on a map in correspondence with the presence of the sound data or the text data.

[25] Based on the information processing terminal described in

[23] , the display control unit causes one or more audio or text data to be displayed. The information processing terminal has the following features: The receiving department accepts the designation of the displayed audio data or text data; and The sending unit transmits the designated audio data or text data received by the receiving unit.

[26] Based on the information processing terminal described in any of [9] to

[12] , the information processing terminal is a terminal used to monitor the actions of the holder of the information processing terminal by outputting information for determining the location of the information processing terminal at given time intervals.

[27] An information processing method, comprising: The receiving process involves the receiving unit receiving audio data transmitted from either the first or second terminal. In the text conversion process, the text conversion unit converts the audio data received by the receiving unit into text data; and In the sending process, the sending unit sends the text data converted by the conversion unit to the second terminal or the first terminal.

[28] An information processing program that enables a computer to function as a receiver, a text converter, and a transmitter: A receiving unit that receives audio data transmitted from a first terminal or a second terminal; A text conversion unit that converts the audio data received by the receiving unit into text data; and The sending unit sends the text data converted by the conversion unit to the second terminal or the first terminal.

[0196] Based on the information processing device described in [2] or [4], the association establishment unit is based on The information used to determine the location, The location information used to determine the location includes at least one of the location date and time, the sending date and time, and the receiving date and time. The text data is obtained from at least one of the following dates and times: the date and time of acquisition, the date and time of transmission, and the date and time of receipt. The information used to determine the location of the first terminal is associated with the text data.

[0197] Based on the information processing terminal described in

[14] , the establishment of the association unit is based on The information used to determine the location, The location information used to determine the location includes at least one of the location date and time, the sending date and time, and the receiving date and time. The text data is at least one of the recording date and time, the sending date and time, and the receiving date and time. The information used to determine the location of other information processing terminals that sent the text data is associated with the text data.

[0198] Label Explanation 1. Information Processing System 2. Server (Information Processing Device) 200A Communication IF 200B storage device 200C CPU 201 Receiving Department 202 Sending Department 203 Storage Device Control Unit (Association Establishment Unit) 204 Text Conversion Department 205 Voice Conversion Department 206 File Format Conversion Department 207 Removal Section 208 Volume Adjustment Section 209 Extraction Section 3. First Terminal (Information Processing Terminal) 300A Communication IF 300B storage device 300C Input Device 300D display device 300E CPU 300F microphone 300G speaker 301 Receiving Department 302 Sending Department 303 Storage Device Control Unit 304 Input Acceptance Department (Acceptance Department) 305 Display Device Control Unit (Display Control Unit) 306 Voice Output Control Unit 4 Second Terminal 400A Communication IF 400B storage device 400C Input Device 400D Display Device (LED) 400E CPU 400F microphone 400G speaker 400H position sensor 401 Receiving Department 402 Sending Department 403 Storage Device Control Unit 404 Input Reception Department 405 Display Device Control Unit (Display Control Unit) 406 Voice Output Control Unit 5. Network.

Claims

1. An information processing device, comprising: The receiving unit receives text data sent from the first terminal; A speech conversion unit that converts the text data received by the receiving unit into sound data; and The transmitting unit sends the audio data converted by the conversion unit to the second terminal.

2. The information processing apparatus according to claim 1, wherein, The information processing device includes an association establishment unit that associates the text data with information used to determine the location of the first terminal. The receiving unit receives information sent from the first terminal for determining location. The association establishment unit establishes an association between the text data received by the receiving unit and the information used to determine the location. The transmitting unit sends the text data and the location information associated with the text data to the second terminal.

3. The information processing apparatus according to claim 2, wherein, The establishment of the association is based on The information used to determine the location, The location information used to determine the location includes at least one of the location date and time, the sending date and time, and the receiving date and time. The text data is obtained from at least one of the following dates and times: the date and time of acquisition, the date and time of transmission, and the date and time of receipt. The information used to determine the location of the first terminal is associated with the text data.

4. An information processing terminal, comprising: The receiving unit receives text data; A speech conversion unit that converts the text data received by the receiving unit into sound data; and The output unit outputs the audio data converted by the conversion unit.

5. The information processing terminal according to claim 4, wherein, The information processing terminal includes an association establishment unit, which associates the text data with information used to determine location. The receiving unit receives the information used to determine the location. The association establishment unit establishes an association between the text data received by the receiving unit and the information used to determine the location.

6. The information processing terminal according to claim 5, wherein, The establishment of the association is based on The information used to determine the location, The location information used to determine the location includes at least one of the location date and time, the sending date and time, and the receiving date and time. The text data is at least one of the recording date and time, the sending date and time, and the receiving date and time. The information used to determine the location of other information processing terminals that sent the text data is associated with the text data.

7. The information processing terminal according to any one of claims 4 to 6, wherein, The information processing terminal includes a notification control unit, which notifies the receiving unit of the receipt of the text data when the receiving unit receives the text data.

8. The information processing terminal according to claim 7, wherein, The notification control unit notifies the recipient that the audio data or text data has been received by reproducing at least a portion of the audio data or text data.

9. An information processing terminal, comprising: The receiving department is responsible for accepting and inputting text data. A speech conversion unit that converts the text data received by the receiving unit into sound data; and The transmitting unit transmits the audio data converted by the conversion unit.

10. An information processing terminal, comprising: A receiving unit, configured to receive audio data and text data converted from the audio data; and The display control unit displays the presence of the sound data received by the receiving unit in correspondence with at least a portion of the text data.

11. An information processing terminal, comprising: The receiving unit receives information for determining the location of other information processing terminals, as well as sound data or text data obtained by converting the sound data; and The display control unit displays the information received by the receiving unit for determining the location of other information processing terminals in correspondence with the presence of the sound data or the text data.

12. The information processing terminal according to claim 11, wherein, The display control unit displays the information received by the receiving unit for determining the location of other information processing terminals on the map in correspondence with the presence of the sound data or the text data.

13. The information processing terminal according to any one of claims 10 to 12, wherein, The display control unit enables the display of more than one audio or text data. The information processing terminal has the following features: The receiving department accepts the designation of the displayed audio data or text data; and The sending unit transmits the designated audio data or text data received by the receiving unit.

14. The information processing terminal according to any one of claims 4 to 13, wherein, The information processing terminal is a terminal used to monitor the actions of the holder of the information processing terminal by outputting information at given time intervals to determine the location of the information processing terminal.

Images