Voice chat device, voice chat method and program

Abstract

Provided are a voice chat device, a voice chat method, and a program capable of appropriately controlling whether to provide text as a result of voice recognition of voice in voice chat. According to the present invention, a voice receiving unit (44) receives voice in voice chat. A text acquisition unit (46) acquires text as a result of voice recognition of the voice received by the voice receiving unit (44). A transmission control unit (52) controls whether text data including the text acquired by the text acquisition unit (46) is transmitted to the communication partner according to whether display of the voice recognition result is performed in the voice chat system as the communication partner.

Description

Technical Field

[0001] The present invention relates to a voice chat device, a voice chat method and a program. Background Art

[0002] In recent years, for example, a user plays a video game while having a voice chat with other users who are playing the video game with the user at a distance, or who are watching a moving image depicting a situation in the video game. Summary of the invention

[0003] [Technical issues]

[0004] In voice chat, some users want to understand the content of the voice chat as text obtained as a result of voice recognition of the voice in the voice chat. It is hoped that such users can obtain the text obtained as a result of voice recognition of the voice in the voice chat.

[0005] However, in voice chat, some users do not need the text obtained as a result of voice recognition of the voice in the voice chat, such as users who do not want to understand the text. Providing text to such users will only increase data traffic unnecessarily.

[0006] The present invention is made in view of the above situation, and an object of the present invention is to provide a voice chat device, a voice chat method and a program, which realize appropriate control of whether to provide text obtained as a result of voice recognition of voice in the voice chat in the voice chat.

[0007] [Problem Solution]

[0008] In order to solve the above-mentioned problem, according to the present invention, there is provided a voice chat device, which is included in a voice chat system among a plurality of voice chat systems, the voice chat system being configured to enable voice chat, the voice chat device comprising: a voice receiving unit configured to receive voice in the voice chat; a text acquiring unit configured to acquire text obtained as a result of voice recognition of the voice; and a sending control unit configured to control whether to send text data including the text to the communication destination based on whether display of the voice recognition result is performed in the voice chat system as the communication destination.

[0009] In an aspect of the present invention, when display of the voice recognition result is performed in any one of the plurality of voice chat systems, the text acquisition unit starts acquiring the text.

[0010] In this aspect, when display of the voice recognition result is not performed in the plurality of voice chat systems, the text acquisition unit stops acquiring the text.

[0011] Furthermore, in an aspect of the present invention, the transmission control unit controls whether to transmit the text data to the communication destination based on whether the voice chat system as the communication destination includes an auxiliary device configured to display a voice recognition result.

[0012] In this aspect, when the auxiliary device is included in any one of the plurality of voice chat systems, the text acquisition unit starts acquiring the text.

[0013] Furthermore, when the auxiliary device is not included in any of the plurality of voice chatting systems, the text acquiring unit may stop acquiring the text.

[0014] In addition, in one aspect of the present invention, the voice chat device further includes: a text receiving unit configured to receive text; and a voice acquiring unit configured to acquire voice obtained as a result of voice synthesis of the text. The transmission control unit controls whether to transmit text data including the text received by the text receiving unit to the communication destination based on whether display of the voice recognition result is performed in the voice chat system as the communication destination.

[0015] In this aspect, the text receiving unit receives the text input to the auxiliary device connected to the voice chatting device.

[0016] In addition, in one aspect of the present invention, the text acquisition unit sends voice data indicating the voice to a server capable of communicating with the above-mentioned voice chat device, and the text acquisition unit receives text obtained as a result of voice recognition of the voice indicated by the voice data from the server.

[0017] In addition, according to the present invention, there is provided a voice chat method, the method comprising the following steps: receiving voice in a voice chat; acquiring text obtained as a result of voice recognition of the voice; and controlling whether text data including the text is sent to the communication destination based on whether display of the voice recognition result is performed in the voice chat system as the communication destination.

[0018] In addition, according to the present invention, a program is provided for causing a computer to execute the following process: receiving voice in a voice chat; acquiring text obtained as a result of voice recognition of the voice; and controlling whether text data including the text is sent to the communication destination based on whether display of the voice recognition result is performed in the voice chat system as the communication destination. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a diagram showing an exemplary overall configuration of a computer network according to an embodiment of the present invention.

[0020] Figure 2A is a diagram showing an exemplary configuration of a voice chat system according to an embodiment of the present invention.

[0021] Figure 2B is a diagram showing an exemplary configuration of a voice chat system according to an embodiment of the present invention.

[0022] Figure 3 is a diagram illustrating exemplary participant management data.

[0023] Figure 4 is a diagram illustrating an exemplary process performed when a voice for voice chatting is input.

[0024] Figure 5 is a diagram illustrating exemplary participant management data.

[0025] Figure 6 is a diagram illustrating an exemplary process performed when text to be converted into speech for voice chatting is input.

[0026] Figure 7 is a diagram illustrating an exemplary process performed when a voice for voice chatting is input.

[0027] Figure 8 is a diagram showing an exemplary assistance screen.

[0028] Fig. 9 is a functional block diagram illustrating exemplary functions implemented in a voice chat system according to an embodiment of the present invention.

[0029] Fig.10 Detailed description is a flowchart showing an exemplary process flow executed in the voice chat device according to the embodiment of the present invention.

[0030] Fig.11 Detailed description is a flowchart showing an exemplary process flow executed in the voice chat device according to the embodiment of the present invention.

[0031] Fig.12Detailed description is a flowchart showing an exemplary process flow executed in the voice chat device according to the embodiment of the present invention. DETAILED DESCRIPTION

[0032] Figure 1 1 is a diagram showing an exemplary overall configuration of a computer network according to an embodiment of the present invention. Figure 1 As shown, voice chat system 10 (10-1, 10-2, ..., and 10-n), voice proxy server 12, and management server 14 (each of which mainly includes a computer) are connected to a computer network 16 such as the Internet. Voice chat system 10, voice proxy server 12, and management server 14 can communicate with each other.

[0033] The management server 14 is, for example, a computer such as a server configured to manage account information about users using the voice chat system 10. The management server 14 stores, for example, a plurality of account data associated with each user. The account data includes, for example, a user identification (ID) as identification information about the user, real name data indicating the real name of the user, and email address data indicating the email address of the user.

[0034] The voice proxy server 12 of the present embodiment is a server computer, for example, configured to generate text indicating the result of performing voice recognition processing on the received voice, and generate voice indicating the result of performing voice synthesis processing on the received text. The voice proxy server 12 may implement a voice recognition engine and a voice synthesis engine, the voice recognition engine being configured to generate text indicating the result of performing voice recognition processing on the received voice, and the voice synthesis engine being configured to generate voice indicating the result of performing voice synthesis processing on the received text.

[0035] like Figure 2A As shown, the voice chat system 10 includes a voice chat device 20 and a router 22 .

[0036] The voice chat apparatus 20 is a computer capable of inputting or outputting voice in voice chat, such as a video game console, a portable video game device, a smart phone, or a personal computer.

[0037] like Figure 2A As shown, the voice chat device 20 includes, for example, a processor 20a, a storage unit 20b, a communication unit 20c, a display unit 20d, an operation unit 20e, a microphone 20f, a speaker 20g, and an encoding / decoding unit 20h. Note that the voice chat device 20 may include a camera.

[0038] The processor 20a is, for example, a program-controlled device such as a central processing unit (CPU), and executes various types of information processing based on a program stored in the storage unit 20b.

[0039] The storage unit 20 b is, for example, a storage element such as a read-only memory (ROM) or a random access memory (RAM), or a hard disk drive.

[0040] The communication unit 20 c is, for example, a communication interface for transmitting or receiving data to or from a computer such as other voice chat systems 10 , a voice proxy server 12 , or a management server 14 via the router 22 and the computer network 16 .

[0041] The display unit 20d is, for example, a liquid crystal display, and displays a screen generated by the processor 20a or a moving image indicated by moving image data received via the communication unit 20c.

[0042] The operation unit 20e is, for example, an operation member for inputting an operation to the processor 20a. Note that the operation unit 20e may be a video game controller.

[0043] The microphone 20f is a voice input device used for voice input in voice chatting, for example.

[0044] The speaker 20g is a voice output device used for voice output in voice chatting, for example.

[0045] The encoding / decoding unit 20h includes, for example, an encoder and a decoder. The encoding / decoding unit 20h encodes input speech to generate speech data indicating the speech. Furthermore, the encoding / decoding unit 20h decodes the input speech data to output speech indicated by the speech data.

[0046] Furthermore, by performing a predetermined pairing process, such as Figure 2B As shown, in addition to the voice chat device 20, an auxiliary device 24 configured to assist voice chat may be added to the voice chat system 10 according to the present embodiment.

[0047] The auxiliary device 24 is, for example, a portable computer such as a smartphone or a tablet device. Note that the auxiliary device 24 may be a stationary computer.

[0048] The auxiliary device 24 according to the present embodiment includes, for example, a processor 24 a , a storage unit 24 b , a communication unit 24 c , and a touch panel 24 d .

[0049] The processor 24 a is, for example, a program control device such as a CPU, and executes various types of information processing based on a program stored in the storage unit 24 b .

[0050] The storage unit 24 b is, for example, a storage element such as a ROM or a RAM or a hard disk drive.

[0051] The communication unit 24c is, for example, a communication interface for transmitting or receiving data to or from a computer such as the voice chat device 20 via the router 22. Note that the communication unit 24c can transmit or receive data to or from a computer such as other voice chat systems 10, the voice proxy server 12, or the management server 14 via the router 22 and the computer network 16, for example.

[0052] The touch panel 24d includes, for example, a touch sensor and a display such as a liquid crystal display integrated with each other. The touch panel 24d displays a screen generated by the processor 24a. In addition, the user performs various types of operations on the touch panel 24d, such as tapping the touch panel 24d, thereby being able to perform operation input to the processor 24a.

[0053] The voice chat device 20 and the auxiliary device 24 are connected to the router 22 by cables or wirelessly, and the router 22 is connected to the computer network 16. The voice chat device 20 and the auxiliary device 24 communicate with other voice chat systems 10, voice proxy servers 12 or management servers 14 via the router 22.

[0054] In the present embodiment, a plurality of voice chat systems 10 (10-1 to 10-n) support voice chat. Therefore, the present embodiment allows a plurality of users who use respective voice chat systems 10 to enjoy voice chat. Here, for example, the users can perform voice chat while sharing a moving image depicting a situation in a video game being played by some or all of the users participating in the voice chat.

[0055] In the present embodiment, a plurality of users participating in the voice chat belong to a group called a “party.” In addition, the user of the voice chat system 10 according to the present embodiment performs a predetermined operation, thereby being able to create a new party or participate in an already created party.

[0056] Furthermore, in the present embodiment, the user of the voice chat system 10 to which the auxiliary device 24 and the voice chat device 20 have been paired performs a predetermined operation, thereby being able to use the voice chat auxiliary service in the voice chat system 10 .

[0057] In the voice chat system 10 in which the voice chat auxiliary service is available, the result of voice recognition of the voice in the voice chat can be displayed on the touch panel 24d of the auxiliary device 24, or text can be input for the voice chat instead of the voice. In addition, the user who uses the voice chat auxiliary service performs a predetermined operation, thereby being able to stop using the voice chat auxiliary service.

[0058] In this embodiment, using Figure 3 The participant management data shown is used to manage information associated with the participants. For example, the participant management data is stored in the management server 14. Figure 3 As shown, the participant management data includes a participant ID as identification information about the participant and user data associated with a user participating in the participant. The user data includes a user ID, connection destination address data, type data, auxiliary service use flag, and the like.

[0059] The user ID is, for example, identification information about the user. The connection destination address data is, for example, data indicating the address of the voice chat device 20 used by the user. The type data is, for example, data indicating the type of the voice chat device 20 used by the user. The auxiliary service use flag is, for example, a flag indicating whether the voice chat auxiliary service is available in the voice chat system 10 used by the user. Here, for example, in the case where the voice chat auxiliary service is provided in the voice chat system 10, the auxiliary service use flag with a value of 1 is set. In addition, for example, in the case where the voice chat auxiliary service is not available in the voice chat system 10, the auxiliary service use flag with a value of 0 is set.

[0060] Figure 3 The participant management data of a party in which five users participate and whose participant ID is 001 is illustrated. Figure 3 The participant management data shown includes five pieces of user data associated with each user participating in the participant. In the following, a user with user ID aaa, a user with user ID bbb, a user with user ID ccc, a user with user ID ddd, and a user with user ID eee are respectively referred to as "user A", "user B", "user C", "user D", and "user E". In addition, user A, user B, user C, user D, and user E use respective voice chat systems 10-1, 10-2, 10-3, 10-4, and 10-5. In addition, the voice chat systems 10-1, 10-2, 10-3, 10-4, and 10-5 include respective voice chat devices 20-1, 20-2, 20-3, 204, and 20-5.

[0061] Figure 3 The participant management data shown indicates that voice chat assistance services are not available in the voice chat system 10.

[0062] Furthermore, in the present embodiment, a copy of the party management data stored in the management server 14 is sent to the voice chat device 20 used by the user participating in the party associated with the party management data. The storage unit 20b of the voice chat device 20 stores a copy of the party management data stored in the management server 14. Therefore, the voice chat device 20 used by the user participating in the party can recognize the address of the voice chat device 20 used by the user participating in the party.

[0063] Furthermore, in the present embodiment, for example, when a user performs an operation of participating in a party, an operation of starting to use a voice chat auxiliary service, or an operation of stopping to use a voice chat auxiliary service, the party management data stored in the management server 14 is updated. Each time the party management data stored in the management server 14 is updated, a copy of the updated party management data is sent to the voice chat device 20 used by the user participating in the party associated with the party management data. Then, the copy of the party management data stored in the storage unit 20b of the voice chat device 20 is updated. In this way, in the present embodiment, the latest information described in the party management data is shared between the voice chat devices 20 used by the users participating in the party associated with the party management data.

[0064] The following description assumes that Figure 3 The five users described in the party management data for have voice chats.

[0065] Figure 4 It is shown in Figure 3 FIG. 1 is a diagram of an exemplary process performed when user A inputs a voice for voice chat in the case of participant management data shown. In this case, in each of voice chat systems 10-1 to 10-5, the voice chat device 20 included in the corresponding voice chat system 10 performs participant processing 30.

[0066] When user A inputs a voice through microphone 20f of voice chat device 20-1, voice data indicating the voice is input to participant processing 30 (see Figure 4 Then, the participant processing 30 sends the input voice data to the participant processing 30 of the voice chat device 20 used by other users participating in the same party as user A (see Figure 4 Here, for example, voice data associated with the user ID of user A may be sent. Figure 4 , the voice chat device 20-2 is illustrated as an exemplary transmission destination of the voice data, but similar voice data is transmitted to the voice chat devices 20-3 to 20-5. Then, the party processing 30 receiving the voice data outputs the voice indicated by the voice data from the speaker 20g (see Figure 4 (3) in the figure.

[0067] In a similar manner, the voice input by each of the user B to the user E is output from the voice chat device 20 used by other users participating in the same party as the user.

[0068] Figure 5 is a diagram showing other exemplary party management data. Figure 5The illustrated party management data indicates that the voice chat assistance service is available in voice chat systems 10-1 and 10-2, but is not available in voice chat systems 10-3 through 10-5.

[0069] Note that the following description assumes that the voice chat system 10-1 includes the voice chat device 20-1 and the auxiliary device 24-1, and the voice chat system 10-2 includes the voice chat device 20-2 and the auxiliary device 24-2.

[0070] In the present embodiment, for example, when user A performs a predetermined operation to enable the voice chat auxiliary service in the voice chat system 10-1, the party management data stored in the management server 14 is updated. Here, for example, the value of the auxiliary service use flag of the user data with the user ID aaa is updated from 0 to 1. Then, in response, the party management data stored in the voice chat devices 20-1 to 20-5 are also updated.

[0071] In addition, in a similar manner, when user B performs a predetermined operation to enable the voice chat auxiliary service in the voice chat system 10-2, the party management data stored in the management server 14 and the voice chat devices 20-1 to 20-5 are updated. Here, for example, the value of the auxiliary service use flag of the user data with the user ID bbb is updated from 0 to 1.

[0072] Figure 6 It is shown in Figure 5 FIG. 1 is a diagram of an exemplary process performed when user A inputs text to be converted into speech for voice chat in the case of participant management data shown. Also in this example, in each of the voice chat systems 10-1 to 10-5, the voice chat device 20 included in the corresponding voice chat system 10 performs the participant process 30.

[0073] Furthermore, in this example, the auxiliary device 24 of the voice chat system 10, in which the voice chat auxiliary service is available, executes the companion application process 32. Then, the voice chat device 20 of the voice chat system 10 executes the proxy process 34 for communicating with the companion application process 32. Here, for example, the auxiliary devices 24-1 and 24-2 execute the companion application process 32. Then, the voice chat device 20-1 executes the proxy process 34 to communicate with the companion application process 32 of the auxiliary device 24-1. Furthermore, the voice chat device 20-2 executes the proxy process 34 for communicating with the companion application process 32 of the auxiliary device 24-2.

[0074] In addition, in this embodiment, Figure 5As shown, the voice chat auxiliary service is available in at least one of the voice chat systems 10 used by the users participating in the party. In this case, all the voice chat devices 20 used by the users participating in the party perform the voice agent processing 36.

[0075] For example, when the value of the auxiliary service use flag of any user data included in the party management data stored in the voice chat device 20 is updated to 1, the voice chat device 20 enables the voice agent process 36.

[0076] Here, for example, the voice chat devices 20-1 to 20-5 execute the voice agent process 36. Note that in the present embodiment, the voice chat devices 20-3 to 20-5 execute the voice agent process 36 although there is no voice chat auxiliary service in the voice chat systems 10-3 to 10-5.

[0077] Here, for example, user A inputs text to the touch panel 24d of the auxiliary device 24-1 (see Figure 6 Then, the companion application processing 32 of the auxiliary device 24-1 sends text data including text to the proxy processing 34 of the voice chat device 20-1 (see Figure 6 Then, in the voice chat device 20-1, the agent process 34 outputs the text data to the voice agent process 36 and the participant process 30 (see Figure 6 (3) and (4) in the text).

[0078] Then, the voice agent processing 36 of the voice chat device 20-1 sends the text data to the voice agent server 12 (see Figure 6 Then, the voice agent server 12 performs speech synthesis processing on the text data, and sends the speech data obtained as a result of the processing to the voice agent processing 36 of the voice chat device 20-1 (see Figure 6 Then, the voice agent process 36 outputs the voice data to the participant process 30 (see Figure 6 (7) in the figure.

[0079] Then, the participant process 30 of the voice chat device 20-1 identifies other voice chat systems 10 for which the voice chat auxiliary service is available. Here, for example, the voice chat system 10-2 is identified. Then, the participant process 30 of the voice chat device 20-1 sends the above-mentioned voice data and text data to the participant process 30 of the voice chat device 20 included in the identified voice chat system 10 (see Figure 6 Here, for example, voice data and text data associated with the user ID of user A may be sent.

[0080] Then, the participant process 30 of the voice chat device 20-2 outputs the received text data to the agent process 34 (see Figure 6 Then, the proxy process 34 of the voice chat device 20-2 sends the text data to the companion application process 32 of the auxiliary device 24-2 (see Figure 6 Then, the accompanying application processing 32 of the auxiliary device 24-2 displays the text included in the text data on the touch panel 24d (see Figure 6 In addition, the party processing 30 of the voice chat device 20-2 may output the voice indicated by the voice data received from the speaker 20g (see Figure 6 (12) in ).

[0081] In addition, the participant process 30 of the voice chat device 20-1 identifies other voice chat systems 10 for which the voice chat auxiliary service is not available. Here, for example, the voice chat systems 10-3 to 10-5 are identified. Then, the participant process 30 of the voice chat device 20-1 sends the above-mentioned voice data only to the participant process 30 of the voice chat device 20 included in the identified voice chat system 10 (see Figure 6 Here, for example, voice data associated with the user ID of user A may be sent. The above text data is not sent to the participant processing 30 of the voice chat system 10 where the voice chat auxiliary service is not available. Figure 6 In the example, the voice data is sent to the participant processing 30 of the voice chat device 20-3 as a representative. Then, the participant processing 30 of the voice chat device 20-3 outputs the voice indicated by the received voice data from the speaker 20g (see Figure 6 Note that in the present embodiment, the voice indicated by the above-mentioned voice data is output from the speakers 20g of the voice chat devices 20-4 and 20-5 in a similar participating manner.

[0082] Figure 7 It is shown in Figure 5 FIG. 1 is a diagram of exemplary processing performed when user C inputs voice for voice chat in the case of participant management data shown.

[0083] When the user C inputs a voice through the microphone 20f of the voice chat device 20-3, voice data indicating the voice is input to the participant process 30 and the voice agent process 36 (see Figure 7 (1) in the figure.

[0084] Then, the voice agent processing 36 of the voice chat device 20-3 sends the voice data to the voice agent server 12 (see Figure 7Then, the voice agent server 12 performs voice recognition processing on the voice data and sends the text data obtained as a result of the processing to the voice agent processing 36 (see Figure 7 Then, the voice agent process 36 outputs the text data to the participant process 30 (see Figure 7 (4) in the figure.

[0085] Then, the participant process 30 of the voice chat device 20-3 identifies other voice chat systems 10 for which the voice chat auxiliary service is available. Here, for example, the voice chat systems 10-1 and 10-2 are identified. Then, the participant process 30 of the voice chat device 20-3 sends the above-mentioned voice data and text data to the participant process 30 of the voice chat device 20 included in the identified voice chat system 10 (see Figure 7 Here, for example, voice data and text data associated with the user ID of user C may be sent. Figure 7 In the process, the voice data and the text data are sent to the participant process 30 of the voice chat device 20-1 as a representative.

[0086] Then, the participant process 30 of the voice chat device 20-1 outputs the received text data to the agent process 34 (see Figure 7 Then, the proxy process 34 of the voice chat device 20-1 sends the text data to the companion application process 32 of the auxiliary device 24-1 (see Figure 7 Then, the accompanying application processing 32 of the auxiliary device 24-1 displays the text included in the text data on the touch panel 24d (see Figure 7 In addition, the party processing 30 of the voice chat device 20-1 may output the voice indicated by the voice data received from the speaker 20g (see Figure 7 (9) in FIG. 20 ). Note that in the present embodiment, the auxiliary device 24-2 displays the text included in the text data on the touch panel 24d in a similar participating manner. Here, the voice indicated by the above-mentioned voice data can be output from the speaker 20g of the voice chat device 20-2.

[0087] In addition, the participant process 30 of the voice chat device 20-3 identifies other voice chat systems 10 for which the voice chat auxiliary service is not available. Here, for example, the voice chat systems 10-4 and 10-5 are identified. Then, the participant process 30 of the voice chat device 20-3 sends the above-mentioned voice data only to the participant processes 30 of the voice chat devices 20 included in the identified voice chat systems 10 (see Figure 7Here, for example, voice data associated with the user ID of user C may be sent. The above text data is not sent to the participant processing 30 of the voice chat system 10 where the voice chat auxiliary service is not available. Figure 7 In the example, the voice data is sent to the participant processing 30 of the voice chat device 20-4 as a representative. Then, the participant processing 30 of the voice chat device 20-4 outputs the voice indicated by the received voice data from the speaker 20g (see Figure 7 Note that in the present embodiment, the voice indicated by the above-mentioned voice data is output from the speaker 20g of the voice chat device 20-5 in a similar participating manner.

[0088] Figure 8 is a diagram showing an exemplary assistance screen displayed on the touch panel 24d of the assistance device 24 according to the present embodiment. Figure 8 An auxiliary screen displayed on the touch panel 24d of the auxiliary device 24-1 is shown, in which the voice chat auxiliary service is available, and the user A uses the auxiliary screen.

[0089] exist Figure 8 On the auxiliary screen shown, text obtained as a result of speech recognition of a speech input by a user other than user A is displayed. Figure 8 On the auxiliary screen shown, text S1 obtained as a result of recognizing the voice input by user B is displayed in association with a character string S2 representing the user ID of user B. For example, the user ID of user B can be identified based on voice data and text data transmitted from the voice chat device 20-2 associated with the user ID of user B.

[0090] In addition, Figure 8 On the auxiliary screen shown, text S3 obtained as a result of voice recognition of user C's input is displayed in association with a character string S4 representing the user ID of user C. For example, the user ID of user C can be identified based on voice data and text data transmitted from the voice chat device 20-3 associated with the user ID of user C.

[0091] Furthermore, on the auxiliary screen, a form F for text input and a send button SB for publishing the text input in the form F are displayed. For example, user A inputs text in the form F and clicks the send button SB to send text data including the text to the voice chat device 20-1. Furthermore, on the auxiliary screen, text S5 input by user A in this manner is displayed in association with a character string S6 indicating the user ID of user A.

[0092] In this embodiment, when the voice chat auxiliary service is not available in the voice chat system 10 used by the user participating in the party, the voice agent process 36 is terminated in all the voice chat devices 20 used by the user participating in the party.

[0093] For example, when the values ​​of the auxiliary service use flags of all user data included in the party management data stored in the voice chat apparatus 20 are set to 0, the voice chat apparatus 20 ends the voice agent process 36 .

[0094] As described above, in the present embodiment, the text obtained as a result of recognizing the voice in the voice chat is not provided to the voice chat system 10 for which the voice chat auxiliary service is not available. Therefore, according to the present embodiment, the amount of data communication for the voice chat is reduced compared to the case where the text obtained as a result of voice recognition on the voice in the voice chat is provided to all the voice chat systems 10. In this way, according to the present embodiment, whether to provide the text obtained as a result of voice recognition on the voice in the voice chat can be appropriately controlled.

[0095] Now, the functions implemented in the voice chat system 10 according to the present embodiment are described in more detail.

[0096] Fig. 9 1 is a functional block diagram showing exemplary functions implemented in the voice chat system 10 according to the present embodiment. Note that in the voice chat system 10 according to the present embodiment, it is not necessary to implement Fig. 9 All functions shown can be realized except Fig. 9 Functions other than those shown.

[0097] like Fig. 9 As shown, the voice chat device 20 according to the present embodiment functionally includes, for example, a participant management data storage unit 40, a participant management unit 42, a voice receiving unit 44, a text acquisition unit 46, a text receiving unit 48, a voice acquisition unit 50, a sending control unit 52, a data receiving unit 54, a voice output unit 56 and an auxiliary sending unit 58.

[0098] The participant management data storage unit 40 is mainly implemented by the storage unit 20b. The participant management unit 42 and the transmission control unit 52 are mainly implemented by the processor 20a and the communication unit 20c. The voice receiving unit 44 is mainly implemented by the microphone 20f and the encoding / decoding unit 20h. The text acquisition unit 46, the text receiving unit 48, the voice acquisition unit 50, the data receiving unit 54 and the auxiliary transmission unit 58 are mainly implemented by the communication unit 20c. The voice output unit 56 is mainly implemented by the speaker 20g and the encoding / decoding unit 20h.

[0099] The above functions are realized by the processor 20a executing a program including instructions corresponding to the above functions, which has been installed on the voice chat device 20 as a computer. For example, the program is provided to the voice chat device 20 through a computer-readable information storage medium such as an optical disk, a magnetic disk, a magnetic tape, a magneto-optical disk, or a flash memory, or via the Internet.

[0100] In addition, if Fig. 9 As shown, the auxiliary device 24 according to the present embodiment functionally includes, for example, a text receiving unit 60, a text sending unit 62, an auxiliary receiving unit 64, and a display control unit 66. The text receiving unit 60 and the display control unit 66 are mainly implemented by the processor 24a and the touch panel 24d. The text sending unit 62 and the auxiliary receiving unit 64 are mainly implemented by the communication unit 24c.

[0101] The above functions are realized by the processor 24a executing a program including instructions corresponding to the above functions, which has been installed on the auxiliary device 24 as a computer. For example, the program is provided to the auxiliary device 24 through a computer-readable information storage medium such as an optical disk, a magnetic disk, a magnetic tape, a magneto-optical disk, or a flash memory, or via the Internet.

[0102] The participant management data storage unit 40 of this embodiment stores, for example, Figure 3 and Figure 5 The party management data shown.

[0103] For example, when receiving the participant management data transmitted from the management server 14, the participant management unit 42 of the present embodiment updates the participant management data stored in the participant management data storage unit 40 to the received participant management data.

[0104] In the present embodiment, for example, when a user performs an operation of starting to use the voice chat auxiliary service or an operation of stopping to use the voice chat auxiliary service, the value of the auxiliary service use flag in the participant management data stored in the management server 14 is updated. Then, based on the update, the management server 14 sends the updated participant management data to the voice chat system 10 used by the user who participates in the party managed by the participant management data. Then, as described above, when the participant management data sent from the management server 14 is received, the participant management unit 42 updates the participant management data stored in the participant management data storage unit 40 to the received participant management data.

[0105] In addition, the participant management unit 42 can detect, based on the updated participant management data, that the display of the speech recognition result is enabled in any voice chat system 10. This detection includes, for example, detecting that at least one of the values ​​of the auxiliary service use flags that were once all 0 is changed to 1.

[0106] In addition, the participant management unit 42 can detect, based on the updated participant management data, that the display of the voice recognition results is disabled in all voice chat systems 10. This detection includes, for example, detecting that at least one of the values ​​of the auxiliary service use flags that were once 1 is changed so that the values ​​of all auxiliary service use flags are 0.

[0107] The voice receiving unit 44 of this embodiment receives voice in voice chat, for example. The voice receiving unit 44 may encode the voice to generate voice data indicating the voice.

[0108] The text acquisition unit 46 of the present embodiment acquires text obtained as a result of, for example, speech recognition of the speech received by the speech receiving unit 44. Here, for example, the text acquisition unit 46 may send speech data indicating the speech to the speech proxy server 12 capable of communicating with the voice chat device 20. Then, the text acquisition unit 46 may receive text data including the text obtained as a result of speech recognition processing of the speech indicated by the speech data from the speech proxy server 12. This function corresponds to Figure 6 and Figure 7 The functionality of the voice agent process 36 is shown.

[0109] In addition, when the display of the speech recognition result is enabled in any one of the at least one voice chat systems 10, the text acquisition unit 46 may start acquiring text. In addition, when the auxiliary device 24 is included in any one of the at least one voice chat systems 10, the text acquisition unit 46 may start acquiring text. For example, when the participant management unit 42 detects that the display of the text is enabled in any voice chat system 10, the text acquisition unit 46 may start the voice agent process 36.

[0110] In addition, when the display of the speech recognition result is prohibited in all of the at least one voice chat system 10, the text acquisition unit 46 may stop acquiring the text. In addition, when the auxiliary device 24 is not included in at least one voice chat system 10, the text acquisition unit 46 may stop acquiring the text. For example, when the participant management unit 42 detects that the display of the text is prohibited in all voice chat systems 10, the text acquisition unit 46 may end the voice agent process 36.

[0111] The text receiving unit 48 of the voice chat device 20 of this embodiment receives, for example, text to be subjected to speech synthesis processing. Here, the text receiving unit 48 may receive, for example, text input to the auxiliary device 24 connected to the voice chat device 20. This function corresponds to Figure 6 and Figure 7 The functionality of the proxy process 34 is shown.

[0112] The speech acquisition unit 50 of the present embodiment acquires, for example, speech obtained as a result of speech synthesis of the text received by the text receiving unit 48. Here, for example, the speech acquisition unit 50 may send text data indicating the text to the voice proxy server 12 capable of communicating with the voice chat device 20. Then, the speech acquisition unit 50 may receive speech obtained as a result of speech synthesis of the text included in the text data from the voice proxy server 12. This function corresponds to Figure 6 and Figure 7 The functionality of the voice agent process 36 is shown.

[0113] The sending control unit 52 of this embodiment controls whether to send text data to the communication destination, for example, based on whether the display of the voice recognition result is enabled in the voice chat system 10 as the communication destination. Here, the sending control unit 52 can control whether to send voice data or voice data and text data to the communication destination based on whether the display of the voice recognition result is enabled in the voice chat system 10 as the communication destination. For example, the sending control unit 52 can control whether to send voice data indicating the voice received by the voice receiving unit 44, or whether to send text data acquired by the text acquiring unit 46 together with the voice data. In addition, for example, the sending control unit 52 can control whether to send voice data indicating the voice acquired by the voice acquiring unit 50, or whether to send text data including the text received by the text receiving unit 48 together with the voice data. This function corresponds to Figures 5 to 7 The functionality of the participant process 30 is shown.

[0114] Here, the transmission control unit 52 can control whether to transmit text data to the communication destination, for example, based on whether the voice chat system 10 as the communication destination includes an auxiliary device 24 configured to display the voice recognition result. In addition, the transmission control unit 52 can control whether to transmit voice data or voice data and text data, for example, based on whether the voice chat system 10 as the communication destination includes an auxiliary device 24 configured to display the voice recognition result.

[0115] In addition, for example, whether to send text data to the communication destination may be controlled based on the value of the auxiliary service use flag in the party management data stored in the party management data storage unit 40. For example, voice data and text data may be sent to the voice chat system 10 whose value of the auxiliary service use flag is 1. At the same time, only voice data may be sent to the voice chat system 10 whose value of the auxiliary service use flag is 0.

[0116] The data receiving unit 54 of this embodiment receives, for example, voice data transmitted from the voice chat device 20 as a communication destination. In addition, the data receiving unit 54 of this embodiment receives, for example, text data transmitted from the voice chat device 20 as a communication destination. This function corresponds to Figures 5 to 7 The functionality of the participant process 30 is shown.

[0117] The voice output unit 56 of this embodiment outputs the voice in the voice chat, for example. For example, the voice output unit 56 outputs the voice indicated by the voice data received by the data receiving unit 54. The voice output unit 56 can decode the voice data received by the data receiving unit 54 to output the voice indicated by the voice data.

[0118] The auxiliary transmission unit 58 of this embodiment transmits the text data received by the data receiving unit 54 to the auxiliary device 24, for example. Figure 6 and Figure 7 The functions of the proxy process 34 shown correspond.

[0119] The text receiving unit 60 of the auxiliary device 24 of the present embodiment receives text input to the touch panel 24d, for example.

[0120] The text transmission unit 62 of the present embodiment transmits text data including the text received by the text reception unit 60 to the voice chat device 20 , for example.

[0121] The auxiliary receiving unit 64 of this embodiment receives text data transmitted from the voice chat device 20, for example.

[0122] The display control unit 66 of this embodiment displays, for example, text included in the text data received by the auxiliary receiving unit 64 or text received by the text receiving unit 60 on the touch panel 24d. The display control unit 66 may display on the touch panel 24d Figure 8 Auxiliary screen shown.

[0123] The functions of the text receiving unit 60, the text sending unit 62, the auxiliary receiving unit 64 and the display control unit 66 of the auxiliary device 24 correspond to Figure 6 and Figure 7 The functionality of the companion application process 32 is shown.

[0124] Here, refer to Fig.10 The flowchart of FIG. 1 describes an example flow of processing executed in the voice chat device 20 in which the voice agent processing 36 according to the present embodiment has been stopped. Fig.10 The processing from S101 to S102 is shown.

[0125] First, the speech receiving unit 44 encodes the speech received during this cycle to generate speech data (S101).

[0126] Then, the transmission control unit 52 transmits the voice data generated in the process in S101 to the voice chat device 20 used by the user participating in the same party (S102), and the process returns to the process in S101. Note that the voice data is not transmitted to the voice chat device 20 executing the process in S102.

[0127] The voice chat device 20 that has received the transmitted voice data in the process of S102 outputs the voice indicated by the voice data.

[0128] Next, refer to Fig.11 The flowchart of FIG. 1 describes an example flow of processing executed in the voice chat device 20 in which the voice agent processing 36 according to the present embodiment has been operated. Fig.11 The processing in S201 to S207 is shown.

[0129] First, the speech receiving unit 44 encodes the speech received during the cycle to generate speech data (S201).

[0130] Then, the text acquisition unit 46 transmits the voice data generated in the process in S201 to the voice proxy server 12 ( S202 ).

[0131] Then, the text acquisition unit 46 receives the text data transmitted from the voice agent server 12 (S203).

[0132] Then, the transmission control unit 52 identifies the voice chat device 20 associated with the user data whose value of the auxiliary service use flag is 1 based on the participant management data stored in the participant management data storage unit 40 (S204).

[0133] Then, the transmission control unit 52 transmits the voice data generated in the process of S201 and the text data received in the process of S203 to the voice chat device 20 identified in the process of S204 (S205). Note that the voice data and text data are not transmitted to the voice chat device 20 executing the process in S205.

[0134] Then, the transmission control unit 52 identifies the voice chat device 20 associated with the user data whose value of the auxiliary service use flag is 0 based on the participant management data stored in the participant management data storage unit 40 (S206).

[0135] Then, the transmission control unit 52 transmits the voice data generated in the process in S201 to the voice chat device 20 identified in the process in S206 (S207), and the process returns to the process in S201. Note that the voice data is not transmitted to the voice chat device 20 executing the process in S207.

[0136] The voice chat device 20 that has received the transmitted voice data in the process of S205 or S207 outputs the voice indicated by the voice data.

[0137] The voice chat device 20 having received the transmitted text data in the process of S205 transmits the text data to the auxiliary device 24 connected to the voice chat device 20. Then, the auxiliary device 24 having received the text data displays the text included in the text data on the touch panel 24d of the auxiliary device 24.

[0138] Note that, in the process of S205, only the text data received in the process of S203 may be transmitted. In this case, the voice chat device 20 that has received the text data may not output the voice indicated by the voice data generated in the process of S201.

[0139] Next, refer to Fig.12 The flowchart of FIG. 1 describes an example flow of processing based on input text executed in the voice chat device 20 in which the voice agent processing 36 according to the present embodiment has been operated. Fig.12 The processing from S301 to S307 is shown.

[0140] First, the text receiving unit 48 receives text data transmitted from the auxiliary device 24 during this cycle (S301).

[0141] Then, the voice acquisition unit 50 transmits the text data generated in the process in S301 to the voice proxy server 12 ( S302 ).

[0142] Then, the voice acquisition unit 50 receives the voice data transmitted from the voice agent server 12 (S303).

[0143] Then, the transmission control unit 52 identifies the voice chat device 20 associated with the user data whose value of the auxiliary service use flag is 1 based on the participant management data stored in the participant management data storage unit 40 (S304).

[0144] Then, the transmission control unit 52 transmits the voice data received in the process of S303 and the text data received in the process of S301 to the voice chat device 20 identified in the process of S304 (S305). Note that the voice data and the text data are not transmitted to the voice chat device 20 that performs the process of S305.

[0145] Then, the transmission control unit 52 identifies the voice chat device 20 associated with the user data whose value of the auxiliary service use flag is 0 based on the participant management data stored in the participant management data storage unit 40 (S306).

[0146] Then, the transmission control unit 52 transmits the voice data received in the process of S303 to the voice chat device 20 identified in the process of S306 (S307), and the process returns to the process of S301. Note that in S307, the voice data is not transmitted to the voice chat device 20 executing the process in S307.

[0147] The voice chat device 20 that has received the transmitted voice data in the process of S305 or S307 outputs the voice indicated by the voice data.

[0148] The voice chat device 20 having received the transmitted text data in the process of S305 transmits the text data to the auxiliary device 24 connected to the voice chat device 20. Then, the auxiliary device 24 having received the text data displays the text included in the text data on the touch panel 24d of the auxiliary device 24.

[0149] Note that, in the process of S305, only the text data received in the process of S301 may be transmitted. In this case, the voice chat device 20 that has received the text data may not output the voice indicated by the voice data generated in the process of S303.

[0150] Note that the present invention is not limited to the above-described embodiments.

[0151] For example, the role division of the voice chat device 20 and the auxiliary device 24 is not limited to the above. For example, the auxiliary device 24 can implement Fig. 9 The voice chat device 20 may be configured to provide a portion or all of the functions of the voice chat device 20 shown. Fig. 9 Some or all of the functions of the auxiliary device 24 are shown.

[0152] Furthermore, the specific character strings and numerical values ​​described above and those in the drawings are illustrative, and the present invention is not limited to these character strings and numerical values.

Claims

1. A communication system, comprising a plurality of voice chat systems, wherein the plurality of voice chat systems comprises a first voice chat system and a second voice chat system: The first voice chat system comprises: A voice receiving unit, configured to receive voice in voice chat; a text acquisition unit configured to acquire text obtained as a result of performing speech recognition on the speech; a transmission control unit configured to control whether to transmit text data including the text to the communication destination based on whether display of a voice recognition result is performed by the auxiliary device in the second voice chat system as the communication destination; as well as a sending unit configured to send the voice data and send the text data in response to control of the sending control unit; and The second voice chat system comprises: a voice chat device, comprising a first communication unit configured to receive the voice data and the text data from the first voice chat system and to output the received voice via a speaker, and a second communication unit configured to send the text data to the auxiliary device; as well as The auxiliary device is configured to output the received text data for display, wherein the auxiliary device is a portable device that executes a companion application process.

2. The communication system according to claim 1, wherein: When display of the voice recognition result is performed in any one of the plurality of voice chat systems, the text acquisition unit starts acquiring the text.

3. The communication system according to claim 2, wherein: When display of the voice recognition result is not performed in the plurality of voice chat systems, the text acquisition unit stops acquiring the text.

4. The communication system according to any one of claims 1 to 3, in, The text acquisition unit sends voice data indicating the voice to a server capable of communicating with the first voice chat system, and The text acquisition unit receives, from the server, text obtained as a result of performing voice recognition on the voice indicated by the voice data.

5. A voice chat method, comprising the following steps: Receiving, at the first voice chat system, voice in the voice chat; Acquiring, at the first voice chat system, text obtained as a result of performing voice recognition on the voice; controlling whether to transmit text data including the text to the communication destination based on whether display of a voice recognition result is performed by the auxiliary device in a second voice chat system as the communication destination; transmitting the voice data from the first voice chat system and transmitting the text data in response to control of a transmission control unit included in the first voice chat system; receiving the voice data and the text data at a voice chat device associated with the second voice chat system; outputting the received speech by the voice chat device at the voice chat device associated with the second voice chat system; sending the text data to the auxiliary device by the voice chat device; as well as The received text data is output for display at a secondary device associated with the second voice chat system, wherein the secondary device is a portable device executing a companion application process.

6. A computer provided with a program, wherein the program causes the computer to execute the voice chatting method according to claim 5.

Images