Information processing device and information processing system

Abstract

The present technology relates to an information processing device and an information processing system capable of performing different processing for a spontaneous speech section and a section other than the spontaneous speech section by using voice recognition data. A spontaneous speech detection unit detects a spontaneous speech section, which is a section of spontaneous speech, i.e., a user's speech, from among an audio signal. A data generation unit generates voice recognition data in which a spontaneous speech flag, which represents the spontaneous speech section detected by the spontaneous speech detection unit, is associated with the audio signal. The present technology can be applied to, for example, smart glasses that display character information corresponding to voice recognition data, a portable terminal device that transmits and receives voice recognition data, and an information processing system that is constituted by a voice recognition server for performing voice recognition processing on voice recognition data.

Description

Information processing device and information processing system

[0001] The present technology relates to an information processing device and an information processing system, and more particularly to an information processing device and an information processing system that are capable of performing different processing on self-speech sections and sections other than the self-speech section using speech recognition data.

[0002] In recent years, technologies for detecting audio signals of specific sounds from audio signals acquired by microphones have been developed. For example, there is a specific sound detector that detects specific sounds based on multiple audio signals obtained by collecting sounds using multiple microphones provided in a wearable device (see, for example, Patent Document 1).

[0003] International Publication No. 2020 / 137585

[0004] However, it has not been considered to generate speech recognition data in which self-speech information representing self-speech sections, which are sections of the audio signal where the user speaks (voice), is associated with the audio signal. Therefore, it has been difficult to identify the self-speech sections of the audio signal using speech recognition data and to perform different processing on the self-speech sections and sections other than the self-speech sections.

[0005] Therefore, there is a demand for a method that can perform different processing on self-speech sections and sections other than self-speech sections using speech recognition data, but such a demand has not yet been fully met.

[0006] The present technology has been made in view of such circumstances, and makes it possible to perform different processing on self-speech sections and sections other than the self-speech section using speech recognition data.

[0007] An information processing device according to a first aspect of the present technology is an information processing device including: a self-speech detection unit that detects a self-speech section, which is a section of an audio signal where the self-speech is a user's speech; and a data generation unit that generates data for speech recognition in which the audio signal is associated with self-speech information that represents the self-speech section detected by the self-speech detection unit.

[0008] In a first aspect of the present technology, a self-speech section, which is a section of a user's self-speech that is speech, is detected from an audio signal, and data for voice recognition is generated in which self-speech information representing the detected self-speech section is associated with the audio signal.

[0009] An information processing system according to a second aspect of the present technology is an information processing system including smart glasses having a self-speech detection unit that detects a self-speech section, which is a section of an audio signal where the user speaks, and a data generation unit that generates voice recognition data that associates the audio signal with self-speech information that represents the self-speech section detected by the self-speech detection unit, and a neckband-type battery having multiple battery boxes into which batteries can be inserted.

[0010] In a second aspect of the present technology, smart glasses are provided that include a self-speech detection unit that detects a self-speech section, which is a section of an audio signal that is a user's self-speech, and a data generation unit that generates voice recognition data that associates the audio signal with self-speech information that represents the self-speech section detected by the self-speech detection unit, and a neckband-type battery that has multiple battery boxes into which batteries can be inserted.

[0011] 17 is a diagram illustrating a configuration example of a first embodiment of an information processing system to which the present technology is applied. FIG. 18 is a diagram illustrating a hardware configuration example of smart glasses 11. FIG. 19 is a block diagram illustrating a functional configuration example of a character conversion processing unit in the first embodiment. FIG. 20 is a block diagram illustrating a hardware configuration example of a mobile terminal device. FIG. 21 is a block diagram illustrating a functional configuration example of a voice recognition processing unit, a classic connection unit, and a call unit in the first embodiment. FIG. 22 is a diagram illustrating a GATT connection method. FIG. 23 is a diagram illustrating an example of a self-speech flag. FIG. 24 is a diagram illustrating an example of a speaking direction. FIG. 25 is a diagram illustrating processing by an information processing system for an audio signal. FIG. 26 is a diagram illustrating example character information in the first embodiment. FIG. 27 is a block diagram illustrating a configuration example of the data generation unit of FIG. 3. FIG. 28 is a block diagram illustrating a configuration example of the processing unit of FIG. 11. FIG. 29 is a diagram illustrating an example of speeding up an audio signal. FIG. 30 is a diagram illustrating an example of a silent section of an audio signal. FIG. 31 is a diagram illustrating a method of deleting a silent section. FIG. 32 is a flowchart illustrating HFP connection processing. FIG. 33 is a flowchart illustrating BLE connection processing. FIG. 34 is a flowchart illustrating the determination processing of FIG. 17. FIG. 35 is a flowchart illustrating character conversion processing in the first embodiment. FIG. 36 is a flowchart illustrating the data generation processing of FIG. 19. FIG. 37 is a flowchart illustrating the processing processing of FIG. 20. FIG. 38 is a flowchart illustrating the character data processing of FIG. 19. FIG. 39 is a block diagram illustrating a functional configuration example of a character conversion processing unit in a second embodiment. 24 is a block diagram showing an example of the configuration of the data generation unit of FIG. 23. FIG. 25 is a block diagram showing an example of the configuration of the processing unit of FIG. 24. FIG. 26 is a diagram showing examples of self-speech intervals and silent intervals. FIG. 27 is a diagram showing an example of character information in a second embodiment. FIG. 28 is a flowchart explaining processing by the processing unit of FIG. 25. FIG. 29 is a flowchart explaining character data processing by the information processing unit of FIG. 23. FIG. 29 is a block diagram showing an example of the functional configuration of a character conversion processing unit in a third embodiment. FIG. 30 is a diagram showing an example of detection of speaker identification information. FIG. 31 is a block diagram showing an example of the functional configuration of a character conversion processing unit in a fourth embodiment. FIG. 32 is a block diagram showing examples of the functional configuration of a speech recognition processing unit, a classic connection unit, and a call unit in the fourth embodiment. FIG. 33 is a block diagram showing an example of the configuration of the data generation unit of FIG. 33. FIG. 34 is a flowchart explaining character conversion processing in the fourth embodiment.40 is a diagram illustrating an example of a configuration of a fifth embodiment of an information processing system to which the present technology is applied. FIG. 41 is a diagram illustrating a first other example of an exterior configuration of smart glasses of the information processing system to which the present technology is applied. FIG. 42 is a diagram illustrating an example of a normal direction to a sound acquisition surface of the microphone of FIG. 40. FIG. 43 is a diagram illustrating a second other example of an exterior configuration of smart glasses of the information processing system to which the present technology is applied. FIG. 44 is a diagram illustrating a third other example of an exterior configuration of smart glasses of the information processing system to which the present technology is applied. FIG. 45 is a diagram illustrating a fourth other example of an exterior configuration of smart glasses of the information processing system to which the present technology is applied. FIG. 46 is a diagram illustrating a fifth other example of an exterior configuration of smart glasses of the information processing system to which the present technology is applied. FIG. 47 is a diagram illustrating a sixth other example of an exterior configuration of smart glasses of the information processing system to which the present technology is applied. FIG. 48 is a diagram illustrating a seventh other example of an exterior configuration of smart glasses of the information processing system to which the present technology is applied. FIG. 49 is a diagram illustrating an eighth other example of an exterior configuration of smart glasses of the information processing system to which the present technology is applied. FIG. 49 is a diagram illustrating an example of an exterior configuration of a mobile battery.

[0012] Hereinafter, modes for implementing the present technology (hereinafter referred to as embodiments) will be described. The description will be made in the following order: 1. First embodiment (information processing system in which smart glasses transmit audio signals for all sections) 2. Second embodiment (information processing system in which smart glasses transmit audio signals only for sections in which other people are speaking) 3. Third embodiment (information processing system in which speaker identification information is detected) 4. Fourth embodiment (information processing system in which a mobile terminal device generates data for voice recognition) 5. Fifth embodiment (information processing system in which a mobile terminal device performs voice recognition processing) 6. Other examples of smart glasses 7. Example of a mobile battery for smart glasses

[0013] 1. First Embodiment Configuration Example of Information Processing System FIG. 1 is a diagram showing a configuration example of a first embodiment of an information processing system to which the present technology is applied.

[0014] 1 includes smart glasses 11 (first information processing device), a mobile terminal device 12 (second information processing device), and a voice recognition server 13. In the information processing system 10, voice recognition processing is performed on an audio signal acquired by the smart glasses 11, and the resulting text information is displayed on the smart glasses 11.

[0015] Specifically, the smart glasses 11 are so-called scouter-type smart glasses (single-eye smart glasses) that have five microphones 21-1 to 21-5 that function as beamforming microphones and a display 22 at one end.

[0016] The microphones 21-1 to 21-5 each acquire (collect) audio signals of surrounding sounds, including the user's own speech, which is the speech of the user (wearer) wearing the smart glasses 11, and other speech, which is the speech of a conversation partner other than the user. Note that speech typically includes silent intervals. In this specification, unless otherwise specified (unless otherwise explained in detail), it is assumed that the user's own speech and other speech include silent intervals. The display 22 displays text information obtained as a result of speech recognition processing of the audio signals.

[0017] The smart glasses 11 detect the self-speech section from the audio signals acquired by the microphones 21-1 to 21-5, and detect the speaking direction, which is the direction of the speaker of the audio signal relative to the user. The smart glasses 11 process the audio signals acquired by the microphones 21-1 to 21-5.

[0018] The smart glasses 11 generate voice recognition data in which the processed audio signal is associated with a self-speech flag voice, which is an audio signal of a self-speech flag (self-speech information) indicating a self-speech section, and a speech direction voice, which is an audio signal indicating a speech direction. The smart glasses 11 transmit this voice recognition data to the mobile terminal device 12 via wireless communication (hereinafter referred to as classic communication) of BT Classic (Bluetooth (registered trademark) Classic) (first wireless communication standard).

[0019] The smart glasses 11 receive character data (character codes) obtained as a result of voice recognition processing of voice recognition data from the mobile terminal device 12 via wireless communication based on Bluetooth Low Energy (BLE) (a second wireless communication standard) (hereinafter referred to as BLE communication). The smart glasses 11 extract character data for the audio signal, the self-speech flag voice, and the speaking direction voice from the character data. The smart glasses 11 display character information consisting of characters, symbols, etc. on the display 22 based on the character data for the audio signal, the self-speech flag voice, and the speaking direction voice. The smart glasses 11 also receive commands, etc. transmitted via BLE communication from the mobile terminal device 12 as needed.

[0020] In the following description, when there is no need to particularly distinguish between the microphones 21-1 to 21-5, they will be collectively referred to as the microphones 21.

[0021] The mobile terminal device 12 establishes a Hands-Free Profile (HFP) connection via classic communication with the smart glasses 11. The mobile terminal device 12 executes, for example, an application for a voice recognition service.

[0022] As a result, the mobile terminal device 12 establishes a GATT (Generic attribute profile) connection with the smart glasses 11 via BLE communication. The mobile terminal device 12 receives voice recognition data from the smart glasses 11 via classic communication and transmits the data to the voice recognition server 13 via wireless LAN (Local Area Network) communication. The mobile terminal device 12 receives character data obtained as a result of voice recognition processing on the voice recognition data from the voice recognition server 13 via wireless LAN communication. The mobile terminal device 12 transmits the character data to the smart glasses 11 via BLE communication. The mobile terminal device 12 also transmits commands to the smart glasses 11 via BLE communication as necessary.

[0023] The voice recognition server 13 is a server that provides a voice recognition service in the cloud. The voice recognition server 13 performs voice recognition processing on the voice recognition data transmitted from the mobile terminal device 12, converts the voice recognition data into character data, and transmits the character data to the mobile terminal device 12 via wireless LAN communication.

[0024] In the example of FIG. 1, the mobile terminal device 12 and the voice recognition server 13 are connected via wireless LAN communication, but they may also be connected via wireless communication of a mobile communication standard such as LTE (Long Term Evolution) (registered trademark).

[0025] In the information processing system 10 configured as described above, by wearing the smart glasses 11, the user can have a conversation while recognizing the content of the conversation in text.

[0026] <Example of Hardware Configuration of Smart Glasses> FIG. 2 is a diagram showing an example of the hardware configuration of the smart glasses 11 in FIG.

[0027] 2, a main processor 51 is connected via wires to microphones 21-1 to 21-5, a ROM (Read Only Memory) 52, a RAM (Random Access Memory) 53, an oscillator circuit 54, a power supply circuit 55, an operation unit 56, and an audio output unit 57. The main processor 51 is also connected via wires to a display control unit 58 and a coprocessor 59.

[0028] The main processor 51 is configured by an SoC (System on a Chip) and includes a CPU (Central Processing Unit) 71, a DSP (Digital Signal Processor) 72, and a power supply control unit 73. In the main processor 51, the CPU 71 and the DSP 72 execute programs stored in the ROM 52 to perform various processes.

[0029] For example, the main processor 51 generates voice recognition data from five-channel audio signals acquired by the microphones 21-1 to 21-5 and supplies the data to the coprocessor 59. The main processor 51 generates display data (font data) for displaying predetermined character information based on character data obtained as a result of performing voice recognition processing on the voice recognition data supplied from the coprocessor 59, and supplies the display data to the display control unit 58. The power supply control unit 73 controls the power supplied from the power supply circuit 55 and supplies it to each unit of the main processor 51.

[0030] The ROM 52 stores programs and the like executed by the CPU 71 and the DSP 72. The RAM 53 stores, as needed, data on intermediate and final results of processing performed by the CPU 71 and the DSP 72. The data stored in the RAM 53 is read out by the CPU 71 and the DSP 72 as needed.

[0031] The oscillator circuit 54 generates a clock signal for operating the main processor 51 and supplies it to the main processor 51. The power supply circuit 55 is connected to a battery 60 and supplies power from the battery 60 to the main processor 51. The operation unit 56 accepts operations by the user and supplies an operation signal representing the operation to the main processor 51. This operation signal is used by the CPU 71 etc. as necessary.

[0032] The audio output unit 57 is configured by a speaker, an audio output earphone, a bone conduction earphone, etc. The audio output unit 57 outputs audio corresponding to the audio signal generated by the CPU 71 and the DSP 72.

[0033] The display control unit 58 displays character information on the display 22 (display unit) based on the display data generated by the CPU 71 and the DSP 72 .

[0034] The coprocessor 59 is configured by an SoC, and includes a sub-CPU 91, a wireless communication unit 92, a ROM 93, and a RAM 94. In the coprocessor 59, the sub-CPU 91 executes a program stored in the ROM 93 to perform various processes.

[0035] For example, the sub-CPU 91 controls the wireless communication unit 92 to transmit voice recognition data supplied from the main processor 51 to the mobile terminal device 12 by classic communication. The sub-CPU 91 controls the wireless communication unit 92 to receive character data transmitted from the mobile terminal device 12 by BLE communication and supplies the character data to the main processor 51.

[0036] The wireless communication unit 92 is connected to the 2.4 GHz band Bluetooth antenna 61, and performs BLE communication and classic communication with the mobile terminal device 12 under the control of the sub-CPU 91. The ROM 93 stores programs and the like executed by the sub-CPU 91. The RAM 94 stores intermediate and final results of processing performed by the sub-CPU 91 as needed. The data stored in the RAM 93 is read by the sub-CPU 91 as needed.

[0037] The coprocessor 59 is also connected to an acceleration sensor 62 and an illuminance sensor 63. The acceleration sensor 62 detects the acceleration of the smart glasses 11 and supplies an acceleration signal representing the acceleration to the coprocessor 59. The illuminance sensor 63 detects the ambient illuminance and supplies an illuminance signal representing the illuminance to the coprocessor 59.

[0038] The CPU 71, DSP 72, and sub-CPU 91 configured as described above function as a character conversion processing unit that executes a predetermined program to display on the display 22 character information corresponding to the audio signal acquired by the microphone 21.

[0039] <Example of Functional Configuration of Character Conversion Processing Unit> FIG. 3 is a block diagram showing an example of the functional configuration of such a character conversion processing unit.

[0040] The character conversion processing unit 111 includes a conversion unit 151, an amplification unit 152, a self-speech detection unit 153, a division unit 154, an amplification unit 155, a specific direction amplification unit 156, an amplification unit 157, a direction detection unit 158, and a data generation unit 159. The smart glasses 11 also include a communication unit 160, an information processing unit 161, and an amplification unit 162.

[0041] The conversion unit 151 converts the format of the five-channel audio signals supplied from the microphones 21-1 to 21-5. The conversion unit 151 supplies audio signals of two or more channels out of the converted five-channel audio signals to the amplification unit 152. The conversion unit 151 supplies the converted five-channel audio signals to the division unit 154.

[0042] Amplification unit 152 amplifies the audio signal supplied from conversion unit 151 and supplies the amplified signal to self-speech detection unit 153. Self-speech detection unit 153 detects a self-speech section in the audio signal supplied from amplification unit 152, and generates a self-speech flag that is turned on only during the self-speech section. Self-speech detection unit 153 supplies the self-speech flag to data generation unit 159.

[0043] The dividing unit 154 divides the four-channel audio signals acquired by the microphones 21-1 to 21-4 from the five-channel audio signals supplied from the conversion unit 151. The dividing unit 154 supplies the four-channel audio signals to amplification units 155 and 157.

[0044] The amplifier 155 amplifies the audio signal supplied from the splitter 154 and supplies the amplified signal to the specific direction amplifier 156. The specific direction amplifier 156 performs beamforming on the audio signal supplied from the amplifier 155 and amplifies the audio signal of a sound source in a specific direction. The specific direction amplifier 156 supplies the resulting audio signal to the data generator 159.

[0045] The amplifier 157 amplifies the audio signal supplied from the divider 154 and supplies the amplified signal to the direction detector 158. The direction detector 158 (speaker information detector) detects the speaking direction based on the audio signal supplied from the amplifier 157 as speaker information, which is information related to the speaker, and supplies the detected information to the data generator 159.

[0046] Data generation unit 159 processes the audio signal supplied from specific direction amplification unit 156. Data generation unit 159 generates self-speech flag audio based on the self-speech flag supplied from self-speech detection unit 153. Data generation unit 159 generates speech direction audio based on the speech direction supplied from direction detection unit 158. Data generation unit 159 generates speech recognition data in which the self-speech flag audio and speech direction audio are associated with the processed audio signal, and supplies the data to communication unit 160.

[0047] The communication unit 160 is composed of a generation unit 160a, a transmission unit 160b, and a reception unit 160c.

[0048] The generation unit 160a generates classic device information (first information processing device information) used when connecting to the mobile terminal device 12 via classic communication. The classic device information is composed of an identifier unique to the smart glasses 11 and compatible device information indicating that the device is compatible with an application for the voice recognition service. The identifier of the smart glasses 11 may be a Bluetooth address generated by the smart glasses 11 using random numbers, a device name, a serial number, a product ID, a random number generated by the smart glasses 11, etc.

[0049] Note that the processing executed by the application for the voice recognition service includes processing for establishing a GATT connection with the smart glasses 11 that have established an HFP connection. Therefore, the compatible device information can also be said to be dual connection capability information indicating that the smart glasses 11 can connect via both classic communication and BLE communication.

[0050] Based on the classic device information, the generation unit 160a generates BLE device information (second information processing device information) to be used when connecting to the mobile terminal device 12 via BLE communication. The BLE device information is configured from an identifier of the smart glasses 11, BLE information indicating that the smart glasses 11 are BLE, and compatible device information.

[0051] The transmitter 160b controls the wireless communication unit 92 in FIG. 2 to perform pairing with the mobile terminal device 12. The transmitter 160b controls the wireless communication unit 92 to transmit the Classic device information generated by the generator 160a as DeviceName to the mobile terminal device 12 via the antenna 61, thereby establishing an HFP connection by Classic communication. The transmitter 160b controls the wireless communication unit 92 to transmit the BLE device information generated by the generator 160a as DeviceName to the mobile terminal device 12 via the antenna 61, thereby establishing a GATT connection with the mobile terminal device 12 by BLE communication. The transmitter 160b controls the wireless communication unit 92 to transmit the voice recognition data supplied from the data generator 159 to the mobile terminal device 12 via the antenna 61 by Classic communication.

[0052] The receiving unit 160c (character data acquiring unit) controls the wireless communication unit 92 to receive (acquire) character data transmitted from the mobile terminal device 12 by BLE communication in accordance with the voice recognition data, via the antenna 61. The receiving unit 160c supplies the character data to the information processing unit 161. The receiving unit 160c controls the wireless communication unit 92 to receive, via the antenna 61, an audio signal of voice related to a call, such as a ringtone, transmitted from the mobile terminal device 12 by classic communication, and supplies the audio signal to the amplifier 162.

[0053] The information processing unit 161 extracts character data corresponding to the audio signal, the self-speech flag voice, and the speech direction voice from the character data supplied from the receiving unit 160c. The information processing unit 161 identifies the self-speech section of the character data corresponding to the audio signal based on the character data corresponding to the extracted self-speech flag voice. The information processing unit 161 recognizes the speech direction corresponding to the character data of the other-speech section, which is a section other than the self-speech section, based on the character data corresponding to the extracted speech direction voice. The information processing unit 161 generates display data for displaying character information corresponding to the character data of the audio signal based on the self-speech section and the speech direction, and supplies the display data to the display control unit 58 of FIG. 2 .

[0054] The amplifier 162 amplifies the audio signal supplied from the receiver 160c, and causes the audio output unit 57 in FIG. 2 to output audio based on the amplified audio signal.

[0055] <Example of Hardware Configuration of Portable Terminal Device> FIG. 4 is a block diagram showing an example of the hardware configuration of the portable terminal device 12. As shown in FIG.

[0056] In the mobile terminal device 12 , a CPU 181 , a ROM 182 , and a RAM 183 are interconnected by a bus 184 .

[0057] An input / output interface 185 is further connected to the bus 184. An input unit 186, an output unit 187, a storage unit 188, and a communication unit 189 are connected to the input / output interface 185.

[0058] The input unit 186 includes operation buttons, a microphone, a camera, etc. The output unit 187 includes a display 187a and an audio output unit 187b. The audio output unit 187b includes a speaker, an output terminal for audio output earphones, and a bone conduction earphone, etc. The storage unit 188 includes a hard disk, a non-volatile memory, etc. The communication unit 189 includes a network interface, etc., and is connected to an antenna 190. The antenna 190 includes a 2.4 GHz band Bluetooth antenna 190a, a 2.4 GHz band or 5 GHz band wireless LAN antenna 190b, and a mobile antenna 190c.

[0059] The communication unit 189 performs classic communication or BLE communication with the smart glasses 11 via a Bluetooth antenna 190a. The communication unit 189 performs wireless LAN communication with the voice recognition server 13 via a wireless LAN antenna 190b. The communication unit 189 performs wireless communication according to the mobile communication standard with a base station (not shown) via a mobile antenna 190c, and transmits and receives audio signals for calls.

[0060] In the mobile terminal device 12 configured as described above, the CPU 181 performs various processes by, for example, loading a program stored in the memory unit 188 into the RAM 183 via the input / output interface 185 and the bus 184 and executing the program.

[0061] For example, the CPU 181 executes a predetermined program such as an OS (Operating System) to function as a classic connection unit that establishes an HFP connection with the smart glasses 11 and as a call unit that makes calls. The CPU 181 also executes an application for a voice recognition service. As a result, the CPU 181 functions as a voice recognition processing unit that transmits voice recognition data transmitted from the smart glasses 11 to the voice recognition server 13 and transmits the resulting text data to the smart glasses 11.

[0062] <Example of Functional Configuration of Speech Recognition Processing Unit, Classic Connection Unit, and Call Unit> FIG. 5 is a block diagram showing an example of the functional configuration of such a speech recognition processing unit, classic connection unit, and call unit.

[0063] The voice recognition processing unit 211 is composed of a communication unit 221 , a storage control unit 222 , an information processing unit 223 , and a display control unit 224 .

[0064] The communication unit 221 is composed of a generation unit 221a, a transmission unit 221b, and a reception unit 221c.

[0065] The generation unit 221a supplies the classic device information of the smart glasses 11 connected to the mobile terminal device 12 via classic communication, which is supplied from the classic connection unit 212, to the storage control unit 222. The generation unit 221a generates BLE device information similar to the generation unit 160a in FIG. 3 based on the classic device information supplied from the storage control unit 222. The generation unit 221a supplies the BLE device information to the storage control unit 222.

[0066] 4, and transmits the voice recognition data received by the receiving unit 221c to the voice recognition server 13 by wireless LAN communication via the wireless LAN antenna 190b. The transmitting unit 221b controls the communication unit 189, and transmits the character data received from the voice recognition server 13 by the receiving unit 221c to the smart glasses 11 by BLE communication via the Bluetooth antenna 190a.

[0067] The receiving unit 221c controls the communication unit 189 to receive, via the Bluetooth antenna 190a, BLE device information transmitted (advertised) from devices (peripherals) that can be connected via BLE communication, including the smart glasses 11. The receiving unit 221c controls the communication unit 189 to connect, via BLE communication, to the smart glasses 11 that has transmitted the BLE device information supplied from the storage control unit 222, among the received BLE device information.

[0068] The receiving unit 221c controls the communication unit 189 to receive the voice recognition data transmitted by the transmitting unit 160b in Fig. 3 via the Bluetooth antenna 190. The receiving unit 221c supplies the voice recognition data to the transmitting unit 221b and also to the storage control unit 222.

[0069] The receiving unit 221c controls the communication unit 189 to receive character data transmitted from the voice recognition server 13 via wireless LAN communication via the wireless LAN antenna 190b, and supplies the data to the transmitting unit 221b and the memory control unit 222.

[0070] The storage control unit 222 stores the voice recognition data supplied from the receiving unit 221c in the storage unit 188 of Fig. 4. The storage control unit 222 associates the character data supplied from the receiving unit 221c with the voice recognition data corresponding to the character data and stores the associated data as a log in the storage unit 188. The storage control unit 222 reads out the character data stored in the storage unit 188 and supplies it to the information processing unit 223.

[0071] The storage control unit 222 stores the BLE device information and the Classic device information supplied from the generation unit 221a in the storage unit 188. The storage control unit 222 reads out the Classic device information stored in the storage unit 188 and supplies it to the generation unit 221a. The storage control unit 222 reads out the BLE device information stored in the storage unit 188 and supplies it to the reception unit 221c.

[0072] 3 based on the character data supplied from the storage control unit 222, the information processing unit 223 generates display data in the same manner as the information processing unit 161 in Fig. 3 and supplies the display data to the display control unit 224. The display control unit 224 displays character information on the display 187a in Fig. 4 based on the display data.

[0073] The classic connection unit 212 controls the communication unit 189 and performs pairing with the smart glasses 11 via the Bluetooth antenna 190a. By controlling the communication unit 189, the classic connection unit 212 receives classic device information transmitted by the transmission unit 160b via the Bluetooth antenna 190 and performs an HFP connection with the smart glasses 11. The classic connection unit 212 supplies the classic device information to the generation unit 221a.

[0074] The call unit 213 controls the communication unit 189 and performs call processing by communicating audio signals of the call and the like via a base station (not shown) and a mobile antenna 190c. In the call processing, the call unit 213 generates an audio signal of the voice related to the call and transmits the audio signal to the smart glasses 11 by classic communication via the Bluetooth antenna 190. For example, when the call unit 213 recognizes an incoming call in the call processing, it generates an audio signal of a ringtone. The call unit 213 transmits the audio signal of the ringtone to the smart glasses 11 by classic communication via the Bluetooth antenna 190.

[0075] <Explanation of GATT Connection Method> FIG. 6 is a diagram illustrating a method for establishing a GATT connection between the smart glasses 11 and the mobile terminal device 12.

[0076] In the example of Fig. 6 , multiple users are individually wearing smart glasses 11 at a school, a company, or the like, and two smart glasses 11 are present within a range where BLE communication is possible with the mobile terminal device 12. In Fig. 6 , the smart glasses 11 that should not communicate with the mobile terminal device 12 are referred to as smart glasses 241 to distinguish them from the smart glasses 11 that should communicate with the mobile terminal device 12.

[0077] First, the smart glasses 11 and the mobile terminal device 12 perform pairing. Next, the transmitter 160b of the smart glasses 11 transmits classic device information including the identifier A of the smart glasses 11 to the mobile terminal device 12, and the classic connection unit 212 of the mobile terminal device 12 receives the classic device information. Then, the smart glasses 11 and the mobile terminal device 12 perform an HFP connection. The generator 221a generates BLE device information including the identifier A based on the identifier A included in the received classic device information.

[0078] Next, the transmission unit 160b of the smart glass 11 advertises the BLE device information including the identifier A generated by the generation unit 160a. At this time, in the example of Fig. 6, the smart glass 241 also advertises the BLE device information including the identifier B.

[0079] The receiving unit 221c of the mobile terminal device 12 receives (scans) both the BLE device information including the identifier A transmitted from the transmitting unit 160b of the smart glass 11 and the BLE device information including the identifier B transmitted from the smart glass 241. Then, the receiving unit 221c establishes a GATT connection with the smart glass 11 that has transmitted the BLE device information generated by the generating unit 221a. On the other hand, the receiving unit 221c does not establish a GATT connection with the smart glass 241 that has transmitted BLE device information different from the BLE device information.

[0080] As described above, the smart glasses 11 transmit Classic device information including the identifier A during an HFP connection, and transmit BLE device information including the identifier A during a GATT connection. Therefore, the mobile terminal device 12 can infer the BLE device information of the HFP-connected smart glasses 11 based on the identifier A included in the Classic device information transmitted during an HFP connection. As a result, the mobile terminal device 12 can establish a GATT connection with the HFP-connected smart glasses 11 by establishing a GATT connection with the smart glasses 11 that transmitted the BLE device information. In other words, the mobile terminal device 12 can prevent a GATT connection from being established with the smart glasses 241 that are not HFP-connected.

[0081] <Example of Self-Speech Flag> FIG. 7 is a diagram showing an example of a self-speech flag.

[0082] 7, the horizontal axis represents time [s]. The upper part of Fig. 7 represents the audio signal [dB] input to self-speech detection section 153 in Fig. 3, and the lower part represents the self-speech flag.

[0083] In the audio signal shown in the upper part of Fig. 7, the sections from time t0 to time t1, the section from time t2 to time t3, etc. are other-utterance sections, and the section from time t4 to time t5, etc. is the self-utterance section.

[0084] When this audio signal is input, self-speech detection unit 153 detects the section from time t4 to time t5, etc. as a self-speech section. As a result, self-speech detection unit 153 generates a self-speech flag that is turned on only for the section detected as the self-speech section, as shown in the lower part of Fig. 7 .

[0085] <Examples of Speech Direction> FIG. 8 is a diagram showing examples of speech directions detected by the direction detection unit 158 ​​in FIG.

[0086] 8, the direction detection unit 158 ​​detects one of four directions, namely, rightward, forward, leftward, and backward, as the speaking direction. Speaker A is on the right side of the user, speaker B is in front, speaker C is on the left, and speaker D is in the back.

[0087] In this case, the direction detection unit 158 ​​detects the right direction as the speaking direction based on the audio signal of the speech section of speaker A, and detects the forward direction as the speaking direction based on the audio signal of the speech section of speaker B. The direction detection unit 158 ​​detects the left direction as the speaking direction based on the audio signal of the speech section of speaker C, and detects the backward direction as the speaking direction based on the audio signal of the speech section of speaker D.

[0088] For example, based on an audio signal in which speaker A says "Hello," the direction detection unit 158 ​​detects the right direction as the speaking direction of the audio signal. Based on an audio signal in which speaker B says "Good morning. It's a nice day today," the direction detection unit 158 ​​detects the forward direction as the speaking direction of the audio signal.

[0089] It should be noted that the number of speaking directions that can be detected by the direction detection unit 158 ​​is not limited to four.

[0090] <Explanation of Processing of Audio Signal in Speech Section of Speaker A> FIG. 9 is a diagram illustrating processing by the information processing system 10 on the audio signal in which speaker A in FIG. 8 speaks "Hello."

[0091] 9, the first row from the top represents the items in each column, and the first column from the left represents the items in each row. The second and subsequent rows correspond to the executors of the processes, and the second and subsequent columns correspond to the process targets.

[0092] As shown in the second row from the top and the second column from the left in Fig. 9 , when speaker A utters "Hello," the microphone 21 of the smart glasses 11 acquires an audio signal of the utterance "Hello," as shown in the third row from the top and the second column from the left in Fig. 9 . Since this utterance "Hello" is an utterance from another person, the self-speech detection unit 153 does not detect a self-speech section and turns off the self-speech flag. Since speaker A is located to the right of the user, the direction detection unit 158 ​​detects the right direction as the direction of speech.

[0093] Because the self-speech flag is off, the data generation unit 159 does not generate any sound as the self-speech flag sound, for example. The data generation unit 159 generates, as the speech direction sound, an audio signal for the speech "right," which indicates the right direction detected as the speech direction. Then, as shown in the third row from the top and the third column from the left in FIG. 9 , the data generation unit 159 generates speech recognition data "Right hello," in which the audio signal for the speech "right" is added as the speech direction sound before the audio signal for the speech "Hello." The transmission unit 160b transmits the speech recognition data "Right hello" to the mobile terminal device 12.

[0094] As shown in the fourth row from the top and third column from the left in Figure 9, the receiving unit 221c of the mobile terminal device 12 receives this voice recognition data ``Hello, turn right,'' and the transmitting unit 221b transmits this voice recognition data ``Hello, turn right'' to the voice recognition server 13.

[0095] As shown in the fifth row from the top and third column from the left in Fig. 9 , the voice recognition server 13 receives this voice recognition data "Migiri Konnichiwa." Then, as shown in the fifth row from the top and fourth column from the left in Fig. 9 , the voice recognition server 13 performs voice recognition processing on this voice recognition data "Migiri Konnichiwa" and converts it into character data. Specifically, the voice recognition server 13 converts "Migiri" to the character code "U+53F3," "Ko" to the character code "U+3053," and "N" to the character code "U+3093." The voice recognition server 13 converts "Ni" to the character code "U+306B," "Chi" to the character code "U+3061," and "Wa" to the character code "U+306F." The voice recognition server 13 transmits the character codes “U+53F3”, “U+3053”, “U+3093”, “U+306B”, “U+3061”, and “U+306F” to the mobile terminal device 12 .

[0096] 9 , the receiving unit 221c of the mobile terminal device 12 receives the character codes "U+53F3", "U+3053", "U+3093", "U+306B", "U+3061", and "U+306F". Then, the transmitting unit 221b transmits the character codes "U+53F3", "U+3053", "U+3093", "U+306B", "U+3061", and "U+306F" to the smart glasses 11.

[0097] As shown in the third row from the top and fourth column from the left in Figure 9, the receiving unit 160c of the smart glasses 11 receives these character codes "U+53F3", "U+3053", "U+3093", "U+306B", "U+3061", and "U+306F".

[0098] The information processing unit 161 extracts the character code "U+53F3" from the character codes "U+53F3," "U+3053," "U+3093," "U+306B," "U+3061," and "U+306F" as character data corresponding to speech direction audio. The information processing unit 161 also extracts the character codes "U+3053," "U+3093," "U+306B," "U+3061," and "U+306F" from the character codes "U+53F3," "U+3053," "U+3093," "U+306B," "U+3061," and "U+306F" as character data corresponding to an audio signal.

[0099] As shown in the third line from the top and the fifth line from the left in FIG. 9 , the information processing unit 161 deletes the character code "U+53F3" from the character codes "U+53F3," "U+3053," "U+3093," "U+306B," "U+3061," and "U+306F" to generate character data for generating display data. Based on the character code "U+53F3" corresponding to the utterance "right" extracted as character data corresponding to the speech direction voice, the information processing unit 161 recognizes that the speaking direction is rightward. Based on the rightward direction, the information processing unit 161 then generates display data for displaying an arrow pointing from right to left, indicating the direction from speaker A on the right toward the user, before the character information corresponding to the character data for generating display data.

[0100] 9, the display control unit 58 displays the text information "←Hello" on the display 22 based on this display data. As a result, as shown in the sixth line from the top and the sixth line from the left in Fig. 9, the user wearing the smart glasses 11 can visually recognize the text information "←Hello." This allows the user to know that speaker A on the right has spoken "Hello."

[0101] In the example of Figure 9, the speech direction audio was an audio signal for the utterance "right." However, in order to improve the accuracy of extracting character data of the speech direction audio, it is desirable that the speech direction audio be an audio signal of an utterance (word) that is not frequently used in normal speech and that includes the speech direction audio so that the context is as natural as possible. For example, the speech direction audio can be an onomatopoeic audio signal. Furthermore, in the example of Figure 9, the speech direction audio is added before the audio signal, but it may also be added after the audio signal, or both before and after.

[0102] <Example of Text Information> FIG. 10 is a diagram showing an example of text information displayed on the display 22. As shown in FIG.

[0103] In the example of Figure 10, speaker A in Figure 8 utters "Hello," then speaker B utters "Good morning. It's a nice day today," and then the user utters "Hello."

[0104] In this case, as described in Fig. 9, the direction detection unit 158 ​​detects the rightward direction as the speaking direction based on the audio signal of the utterance "Hello" by the speaker A. As a result, "←Hello" is displayed as text information on the first line from the top of the display screen of the display 22, as shown in Fig. 10.

[0105] The direction detection unit 158 ​​detects the forward direction as the speaking direction based on the audio signal of the utterance "Good morning" by the speaker B. As a result, as shown in Fig. 10, an arrow pointing from top to bottom, indicating the direction from the front speaker A toward the user, is displayed before the text information "Good morning" on the second line from the top of the display screen of the display 22.

[0106] The direction detection unit 158 ​​detects the forward direction as the speaking direction based on the audio signal of the utterance "It's a nice day today, isn't it?" by the speaker B. As a result, as shown in Fig. 10, an arrow pointing from top to bottom, indicating the direction from the front speaker A toward the user, is displayed before the text information "It's a nice day today, isn't it?" on the third line from the top of the display screen of the display 22.

[0107] The self-speech detection unit 153 detects the audio signal of the user's utterance "Hello" as a self-speech section and turns on the self-speech flag. The data generation unit 159 generates, for example, the audio signal of the utterance "Hello" that represents the self-speech section as a self-speech flag audio. Then, the data generation unit 159 generates speech recognition data "Hello" by adding the audio signal of the self-speech flag audio before the audio signal of the utterance "Hello."

[0108] As a result, the information processing unit 161 extracts the character data of the utterance "self" from the character data transmitted by the transmission unit 221b as character data of the self-speech flag voice, and extracts the character data of the utterance "hello" as character data of the audio signal. The information processing unit 161 recognizes that it is a self-speech section based on the extracted character data of the self-speech flag voice. The information processing unit 161 then generates display data for displaying the character information "hello" corresponding to the extracted character data of the audio signal with a black circle indicating the self-speech section added before it. As a result, as shown in FIG. 10 , the black circle indicating the self-speech section is displayed added before the character information "hello" on the fourth line from the top of the display screen of the display 22.

[0109] Note that a period may be displayed at the end of each line in Fig. 10, i.e., at the end of each utterance. In the example of Fig. 10, the character information of a plurality of utterances is displayed on the display 22 from top to bottom in the order of utterance, but the character information of each utterance may also be displayed on the display 22 in the order of utterance.

[0110] In the example of Figure 10, the self-speech section is represented by text information, but the self-speech section may also be represented by displaying the text information of the audio signal of the self-speech section in a display format such as a color or font that is different from the text information of the audio signal of other speech sections.

[0111] <Configuration Example of Data Generator> FIG. 11 is a block diagram showing a configuration example of the data generator 159 in FIG.

[0112] The data generating unit 159 in FIG. 11 is made up of a processing unit 261 , a flag voice unit 262 , a directional voice unit 263 , and an adding unit 264 .

[0113] The processing unit 261 processes the audio signal supplied from the specific direction amplifier 156 in Fig. 3 to shorten the time of the audio signal. The shortened time is determined depending on the performance of the speech recognition server 13, such as the recognition accuracy and real-time performance. The processing unit 261 supplies the processed audio signal to the adding unit 264.

[0114] Flag audio unit 262 generates a self-speech flag audio based on the self-speech flag supplied from self-speech detection unit 153 in Fig. 3 and supplies the generated audio to addition unit 264. Direction audio unit 263 generates a speech direction audio based on the speech direction supplied from direction detection unit 158 ​​in Fig. 3 and supplies the generated audio to addition unit 264. Addition unit 264 generates voice recognition data by associating the self-speech flag audio supplied from flag audio unit 262 and the speech direction audio supplied from direction audio unit 263 with the audio signal supplied from processing unit 261, and supplies the generated audio to transmission unit 160b in Fig. 3.

[0115] <Configuration Example of Processing Unit> FIG. 12 is a block diagram showing a configuration example of the processing unit 261 in FIG.

[0116] The processing section 261 in FIG. 12 is made up of a noise removal section 281 , a normalization section 282 , an equalizer 283 , a speed change section 284 , a silence deletion section 285 , and a filter 286 .

[0117] The noise removal unit 281 removes noise from the audio signal supplied from the specific direction amplification unit 156 in Fig. 3 and supplies the result to a normalization unit 282. The normalization unit 282 performs normalization processing on the audio signal supplied from the noise removal unit 281 and supplies the result to an equalizer 283. The equalizer 283 adjusts the frequency characteristics of the audio signal supplied from the normalization unit 282 and supplies the result to a speed change unit 284.

[0118] The speed change unit 284 speeds up the audio signal supplied from the equalizer 283 without changing the pitch (musical interval), thereby shortening the time of the audio signal. The speed change unit 284 supplies the resulting audio signal to a silence deletion unit 285.

[0119] The silence deletion unit 285 detects, from the audio signal supplied from the speed change unit 284, a section in which an audio signal level below a threshold continues for a time period equal to or longer than the threshold, as a silence section. The threshold used to detect a silence section is determined, for example, based on the performance of the speech recognition server 13. The time threshold can be set to, for example, 0.1 seconds. The silence deletion unit 285 shortens the time of the audio signal by deleting the detected silence section, and supplies the resulting audio signal to the filter 286.

[0120] The filter 286 is configured by an LPF (Low Pass filter) or the like, and performs filtering on the audio signal supplied from the silence deletion unit 285. The filter 286 supplies the resulting audio signal to the addition unit 264 in FIG.

[0121] 12, the processing unit 261 is configured to perform the processes of the speed change unit 284 and the silence deletion unit 285 in that order, but the order of the processes may be reversed, or the processes may be performed in parallel with the processing results being synthesized at a later stage. The processing unit 261 may also be configured to include only one of the speed change unit 284 and the silence deletion unit 285.

[0122] <Example of Speeding Up Audio Signal> FIG. 13 is a diagram showing an example of speeding up an audio signal by the speed change unit 284 in FIG.

[0123] In Fig. 13, the horizontal axis represents time [s]. Fig. 13A shows the audio signal [dB] before speed-up supplied to the speed change unit 284, and Fig. 13B shows the audio signal [dB] after speed-up by the speed change unit 284.

[0124] In the example of Fig. 13, the duration (total playback time) of the audio signal before speed-up shown in Fig. 13A is 33.252 seconds, while the duration of the audio signal after speed-up shown in Fig. 13B is 29.926 seconds. Therefore, speed-up has reduced the duration of the audio signal by approximately 10%.

[0125] Since the speed change unit 284 speeds up the audio signal without changing its pitch, the speech corresponding to the sped-up audio signal is easier for the human ear to hear, and deterioration in the accuracy of the speech recognition processing results can be suppressed. In order to make the speech corresponding to the sped-up audio signal audible to the human ear and to prevent deterioration in the accuracy of the speech recognition processing results, it is desirable that the time reduction rate of the audio signal due to speed-up be, for example, 50% or less, particularly about 30%. Note that this reduction rate, i.e., the degree of speed-up by the speed change unit 284, is optimized based on the performance of the speech recognition server 13.

[0126] <Example of Silence Section> FIG. 14 is a diagram showing an example of a silence section of an audio signal detected by the silence deletion unit 285 of FIG.

[0127] In Fig. 14, the horizontal axis represents time [s]. In the example of Fig. 14, the audio signal supplied from the speed change unit 284 is made up of speech sections A1 to A5 of one speaker.

[0128] The speech section A1 is an audio signal of the utterance "Yes, / Well, / We / Regarding / N30's / sales strategy / We / think / we should / focus / on / advertising strategy / ." This speech section A1 includes silent sections a1 to a5.

[0129] Speech section A2 is an audio signal of the utterance "Please take a look at the document in front of you." This utterance section A2 includes a silent section a6. Speech section A3 is an audio signal of the utterance "Sales figures for the previous model N20 smartphone are as shown on page 3." This utterance section A3 includes silent sections a7 and a8.

[0130] Speech section A4 is an audio signal of the utterance "In terms of sales strategy, the advertising campaign using television commercials can be said to have been successful, but as you can see on page 4, it cost a lot of money." This utterance section A4 includes silent sections a9 to a13.

[0131] The speech section A5 is an audio signal of "This time, I would like to propose advertising in magazines, which can reduce costs." This speech section A5 includes silent sections a14 to a16.

[0132] When an audio signal consisting of speech sections A1 to A5 including the above-described silent sections a1 to a16 is supplied from the speed change section 284, the silent deletion section 285 deletes the silent sections a1 to a16.

[0133] Experiments have shown that the results of speech recognition processing of an audio signal from which silent sections of 0.1 seconds or more have been deleted are substantially unchanged from the results of speech recognition processing of the audio signal before deletion.

[0134] If there is a silent section at the beginning or end of the audio signal, the silence deletion unit 285 does not need to delete the silent section. The length and number of silent sections contained in the audio signal differ depending on the type of speech. For example, the number of silent sections contained in an audio signal during normal conversation is small, but the number of silent sections contained in an audio signal during reading a document is large.

[0135] <Description of Silence Section Deletion Method> FIG. 15 is a diagram illustrating a silence section deletion method performed by the silence deletion unit 285. In FIG.

[0136] In FIG. 15, the horizontal axis represents time [seconds].

[0137] When the audio signal [dB] shown in A of Fig. 15 is supplied from the speed change unit 284, the silence deletion unit 285 detects a silent section a31 from time t31 to t32 of the audio signal. Then, as shown in B of Fig. 15, the silence deletion unit 285 deletes the silent section a31 of the audio signal, and combines the remaining audio signal up to time t31 with the audio signal from time t32 onwards, as shown in C of Fig. 15. As a result, the duration of the audio signal after deletion of the silent section a31 is shortened by the duration of the silent section a31.

[0138] <Explanation of HFP Connection Processing> FIG. 16 is a flowchart illustrating HFP connection processing for establishing an HFP connection between the character conversion processing unit 111 and the classic connection unit 212.

[0139] In step S11 of Fig. 16, the generation unit 160a of the character conversion processing unit 111 generates classic device information and BLE device information and supplies them to the transmission unit 160b. In the example of Fig. 16, the identifier of the smart glasses 11 is a six-digit random number "nnnnnn" (n is an arbitrary number) generated by the smart glasses 11 at the time of initial startup or factory reset, the compatible device information is "SG", and the BLE information is "LE". Therefore, the classic device information is "SG-nnnnnn", and the BLE device information is "LE-SG-nnnnnn".

[0140] In step S12, the transmitting section 160b transmits the classical music device information "SG-nnnnnn" generated in the process of step S11 to the mobile terminal device 12.

[0141] In step S13, the classic connection unit 212 receives the classic device information "SG-nnnnnn" transmitted by the processing in step S12, and supplies it to the generation unit 221a.

[0142] In step S14, the transmission unit 160b and the classic connection unit 212 perform pairing.

[0143] In step S15, the transmitter 160b and the classic connection unit 212 establish an HFP connection, and the HFP connection process then ends.

[0144] <Explanation of BLE Connection Processing> FIG. 17 is a flowchart illustrating the BLE connection processing performed between the character conversion processing unit 111 that performed the HFP connection processing of FIG. 16 and the voice recognition processing unit 211 when an application for a voice recognition service is started.

[0145] In step S31 of FIG. 17, the transmission unit 160b of the character conversion processing unit 111 advertises the BLE device information "LE-SG-nnnnnn" generated by the processing in step S12 of FIG.

[0146] In step S32, the receiving unit 221c of the voice recognition processing unit 211 receives the BLE device information “LE-SG-nnnnnn” transmitted by the processing in step S31. At this time, if another smart glass 11 is present within the range in which BLE communication is possible with the mobile terminal device 12, the receiving unit 221c also receives BLE device information from that smart glass 11.

[0147] In step S33, the voice recognition processing unit 211 performs a determination process to determine the smart glasses 11 to be connected to by GATT. Details of this determination process will be described later with reference to Fig. 18. Through the determination process, the smart glasses 11 that have transmitted the BLE device information "LE-SG-nnnnnn" in the process of step S31 are determined to be the target for GATT connection.

[0148] In step S34, the transmitter 160b and receiver 221c of the smart glasses 11 determined as GATT connection targets in the determination process of step S33 perform a GATT connection. At this time, the transmitter 160b and receiver 221c of the smart glasses 11 not determined as GATT connection targets in the determination process of step S33 do not perform a GATT connection. Then, the BLE connection process ends.

[0149] <Description of Determination Process> FIG. 18 is a flowchart illustrating the determination process in step S33 of FIG.

[0150] 18, the generation unit 221a determines whether or not the HFP connection processing of Fig. 16 is being performed, that is, whether or not classic device information has been supplied from the classic connection unit 212. If it is determined in step S51 that the HFP connection processing is being performed, the processing proceeds to step S52.

[0151] In step S52, the generation unit 221a determines whether or not compatible device information is included in the classic device information supplied from the classic connection unit 212. If it is determined in step S52 that compatible device information is included, the generation unit 221a supplies the classic device information to the storage control unit 222, and the process proceeds to step S53.

[0152] In step S53, the storage control unit 222 determines whether or not classic device information is stored in the storage unit 188. If it is determined in step S53 that classic device information is stored, the process proceeds to step S54.

[0153] In step S54, the storage control unit 222 determines whether or not the classic device information received in the processing of step S14 in Fig. 16 is already stored in the storage unit 188. That is, the storage control unit 222 determines whether or not the classic device information supplied from the generation unit 221a is the same as the classic device information stored in the storage unit 188. If it is determined in step S54 that the classic device information received is already stored in the storage unit 188, the processing proceeds to step S58.

[0154] On the other hand, if it is determined in step S54 that the received classic device information has not yet been stored in the storage unit 188, the process proceeds to step S55. In step S55, the storage control unit 222 supplies the classic device information supplied from the generation unit 221a to the storage control unit 222, thereby updating the classic device information stored in the storage unit 188 with the classic device information. As a result, the most recently received classic device information is stored in the storage unit 188. The process then proceeds to step S58.

[0155] If it is determined in step S53 that the classic device information has not yet been stored in the storage unit 188, the process proceeds to step S56. In step S56, the storage control unit 222 stores the classic device information supplied from the generation unit 221a in the storage unit 188. Then, the process proceeds to step S58.

[0156] If it is determined in step S51 that the HFP connection process has not been performed, or if it is determined in step S52 that the compatible device information is not included, the process proceeds to step S57. In step S57, the storage control unit 222 performs a process similar to that of step S53. If it is determined in step S57 that the classic device information is stored in the storage unit 188, the process proceeds to step S58.

[0157] In step S58, the storage control unit 222 reads out the Classic device information from the storage unit 188 and supplies it to the generation unit 221a. In step S59, the generation unit 221a generates (estimates) BLE device information corresponding to the Classic device information based on the Classic device information read out by the processing of step S58. For example, the generation unit 221a generates BLE device information "LE-SG-nnnnnn" that includes the identifier "nnnnnn" included in the Classic device information "SG-nnnnnn." The generation unit 221a supplies the BLE device information to the storage control unit 222, which stores it in the storage unit 188. The storage control unit 222 reads out the BLE device information stored in the storage unit 188 and supplies it to the receiving unit 221c.

[0158] In step S60, the receiving unit 221c starts searching for the BLE device information supplied from the storage control unit 222 from the received BLE device information. In step S61, the receiving unit 221c determines whether the BLE device information supplied from the storage control unit 222 is found as a result of the search. If it is determined in step S61 that the BLE device information is found, the processing proceeds to step S62. In step S62, the receiving unit 221c determines the smart glasses 11 that have transmitted the found BLE device information as a GATT connection target. Then, the determination processing ends.

[0159] On the other hand, if it is determined in step S61 that the search has not been found, the process proceeds to step S63. In step S63, the receiving unit 221c determines whether or not an instruction to end the search has been given, for example, by the user. If it is determined in step S63 that an instruction to end the search has not been given, the process returns to step S60, and the subsequent processes are repeated.

[0160] On the other hand, if it is determined in step S63 that an instruction to end the search has been issued, the receiving unit 221c ends the search and proceeds to step S64. If it is determined in step S57 that the classic device information has not yet been stored in the storage unit 188, the processing proceeds to step S64. In step S64, the display control unit 224 displays an error message on the display 187a. The determination processing then ends.

[0161] <Description of Character Conversion Process> Fig. 19 is a flowchart illustrating the character conversion process performed between the character conversion processing unit 111, the voice recognition processing unit 211, and the voice recognition server 13 after the GATT connection process in Fig. 17. This character conversion process is started, for example, when the user starts a dialogue.

[0162] 19, the character conversion processing unit 111 performs a data generation process to generate data for speech recognition. Details of this data generation process will be described later with reference to FIG. 20. In step S82, the transmission unit 160b transmits the data for speech recognition generated by the process of step S81 to the mobile terminal device 12 by classic communication.

[0163] In step S91, the receiving unit 221c of the voice recognition processing unit 211 receives the voice recognition data transmitted by the processing of step S82. The receiving unit 221c supplies the voice recognition data to the transmitting unit 221b and also supplies the voice recognition data to the storage control unit 222 for storage in the storage unit 188. In step S92, the transmitting unit 221b transmits the voice recognition data supplied from the receiving unit 221c to the voice recognition server 13.

[0164] In step S101, the voice recognition server 13 receives the voice recognition data transmitted in step S92. In step S102, the voice recognition server 13 performs voice recognition processing on the voice recognition data received in step S101 and converts it into character data. In step S103, the voice recognition server 13 transmits the character data to the mobile terminal device 12.

[0165] In step S93, the receiving unit 221c receives the character data transmitted by the processing of step S103. The receiving unit 221c supplies the character data to the transmitting unit 221b and the storage control unit 222. In step S94, the storage control unit 222 associates the character data with the voice recognition data received and stored by the processing of step S91, and stores the character data in the storage unit 188. In step S95, the transmitting unit 221b transmits the character data supplied from the receiving unit 221c to the smart glasses 11.

[0166] In step S83, receiving unit 160c receives the character data transmitted in step S95. In step S84, character conversion processing unit 111 performs character data processing using the character data received in step S83. Details of this character data processing will be described later with reference to FIG. 22. After character data processing, the character conversion processing ends.

[0167] <Description of Data Generation Processing> FIG. 20 is a flowchart illustrating the data generation processing in step S81 of FIG.

[0168] 20 , the conversion unit 151 acquires five-channel audio signals supplied from the microphones 21-1 to 21-5. In step S122, the conversion unit 151 converts the format of the audio signals acquired by the processing of step S121. The conversion unit 151 supplies audio signals of two or more channels of the converted five-channel audio signals to the amplification unit 152. The conversion unit 151 supplies the converted five-channel audio signals to the division unit 154.

[0169] In step S123, amplification unit 152 amplifies two or more channel audio signals from the audio signals whose formats have been converted by the processing of step S122, and supplies the amplified signals to self-speech detection unit 153. In step S124, self-speech detection unit 153 detects a self-speech section from the audio signals amplified by the processing of step S123, and generates a self-speech flag. Self-speech detection unit 153 supplies the self-speech flag to data generation unit 159.

[0170] In step S125, the dividing unit 154 divides the five-channel audio signals whose format has been converted by the processing in step S122 into four-channel audio signals acquired by the microphones 21-1 to 21-4. The dividing unit 154 supplies the four-channel audio signals to the amplifiers 155 and 157.

[0171] In step S126, the amplifier 155 amplifies the four-channel audio signals obtained as a result of the division in the processing of step S125, and supplies the amplified signals to the specific direction amplifier 156. In step S127, the specific direction amplifier 156 performs beamforming on the audio signals amplified in the processing of step S126, and amplifies the audio signals of the sound source in the specific direction. The specific direction amplifier 156 supplies the resulting audio signals to the data generator 159.

[0172] In step S128, the amplifier 157 amplifies the four-channel audio signals obtained as a result of the division in the processing of step S125, and supplies the amplified signals to the direction detector 158. In step S129, the direction detector 158 detects the speaking direction based on the audio signals amplified in the processing of step S128, and supplies the detected direction to the data generator 159.

[0173] In step S130, processing unit 261 of data generation unit 159 performs processing to process the audio signal obtained by the processing of step S127. Details of this processing will be described later with reference to FIG. 21. In step S131, flag audio unit 262 generates self-speech flag audio based on the self-speech flag supplied from self-speech detection unit 153, and supplies the generated audio to addition unit 264. In step S132, direction audio unit 263 generates speech direction audio based on the speech direction detected by the processing of step S129, and supplies the generated audio to addition unit 264.

[0174] In step S133, the adding unit 264 generates data for voice recognition. Specifically, the adding unit 264 generates data for voice recognition by associating the self-speech flag voice generated in the process of step S131 and the speaking direction voice generated in the process of step S132 with the audio signal obtained in the process of step S130. The adding unit 264 supplies the data for voice recognition to the transmitting unit 160b, and the process returns to step S81 in Fig. 19. Then, the process proceeds to step S82.

[0175] <Explanation of Processing> FIG. 21 is a flowchart illustrating the processing in step S130 of FIG.

[0176] 21 , the noise removal unit 281 of the processing unit 261 obtains the audio signal obtained as a result of the processing of step S127 of Fig. 20 from the specific direction amplification unit 156. In step S152, the noise removal unit 281 removes noise from the audio signal obtained by the processing of step S151, and supplies the audio signal to the normalization unit 282.

[0177] In step S153, the normalization unit 282 performs normalization processing on the audio signal from which noise has been removed by the processing in step S152, and supplies the normalized audio signal to the equalizer 283. In step S154, the equalizer 283 adjusts the frequency characteristics of the audio signal obtained as a result of the processing in step S153, and supplies the adjusted audio signal to the speed change unit 284.

[0178] In step S155, the speed change unit 284 speeds up the audio signal whose frequency characteristics have been adjusted by the process in step S154 without changing the pitch, and supplies the audio signal to the silence deletion unit 285.

[0179] In step S156, the silence deletion unit 285 detects and deletes silence intervals from the audio signal whose speed has been increased by the processing in step S155. The silence deletion unit 285 supplies the resulting audio signal to the filter 286.

[0180] In step S157, the filter 286 performs filtering on the audio signal from which the silent intervals have been deleted in the processing of step S156, and supplies the resulting audio signal to the adding unit 264. After the processing of step S157, the processing returns to step S130 in Fig. 20, and the processing proceeds to step S131.

[0181] <Description of Character Data Processing> FIG. 22 is a flowchart illustrating the character data processing in step S84 of FIG.

[0182] In step S181 of FIG. 22, the information processing unit 161 extracts the character data of the audio signal, the character data of the self-speech flag voice, and the character data of the speaking direction voice from the character data received in the process of step S83 of FIG.

[0183] In step S182, the information processing unit 161 identifies the self-speech section in the character data of the audio signal extracted in the process of step S181, based on the character data of the self-speech flag voice extracted in the process of step S181.

[0184] In step S183, the information processing unit 161 recognizes the speaking direction corresponding to the character data of the other speech section based on the character data of the speaking direction voice extracted by the process of step S181.

[0185] In step S184, information processing unit 161 generates display data. Specifically, information processing unit 161 generates display data for displaying text information corresponding to the text data in the audio signal, based on the self-speech section identified in the process of step S182 and the speaking direction recognized in the process of step S183. Information processing unit 161 supplies the display data to display control unit 58.

[0186] In step S185, the display control unit 58 displays character information on the display 22 based on the display data generated in step S184. Then, the character data processing ends, and the process returns to step S84 in Fig. 19, where the character conversion processing ends.

[0187] The transmitting unit 160b may transmit the self-speech flag and the speaking direction together with the voice recognition data to the mobile terminal device 12. In this case, the storage unit 188 stores the voice recognition data together with the self-speech flag and the speaking direction.

[0188] The information processing unit 161 may generate display data for displaying only the text information corresponding to the text data of the audio signal in the self-speech section or the other-speech section, based on the identified self-speech section.

[0189] The smart glasses 11 may have a conversation partner display different from the display 22, for example, a display screen of which is arranged to face the conversation partner. In this case, the smart glasses 11 display text information corresponding to the audio signal of the other person's speech section on the conversation partner display, for example, based on the identified user's speech section. This allows the conversation partner of the user wearing the smart glasses 11 to check how their speech is converted into text information.

[0190] As described above, in the character conversion processing unit 111, the self-speech detection unit 153 detects a self-speech section from the audio signal acquired by the microphone 21, and the data generation unit 159 generates speech recognition data in which the audio signal is associated with a self-speech flag sound, etc. Therefore, the character conversion processing unit 111 can use the speech recognition data to perform different processing on the self-speech section and sections other than the self-speech section.

[0191] For example, based on the character data obtained as a result of performing a speech recognition process on the speech recognition data, the character conversion processing unit 111 can add a black circle representing the self-speech section to the character information corresponding to the character data of the self-speech section and display it as shown in Fig. 10. This allows the user to distinguish between the character information of the self-speech section and the character information of the other-speech section.

[0192] In the character conversion processing unit 111, the direction detection unit 158 ​​detects the speaking direction based on the audio signal acquired by the microphone 21, and the data generation unit 159 generates data for speech recognition in which the audio signal is associated with a speech direction voice, etc. Therefore, the information processing unit 161 can recognize the speaking direction corresponding to the character data of the audio signal based on the character data obtained as a result of speech recognition processing of the data for speech recognition.

[0193] 10, the display control unit 58 can add an arrow indicating the speaking direction to the text information corresponding to the audio signal of the speaking direction and display it on the display 22. As a result, the user can recognize the text information of the utterance and the direction of the speaker who made the utterance relative to the user, i.e., the direction from which the utterance is coming. This effect is particularly useful when the user is conversing with multiple speakers.

[0194] The data generation unit 159 converts the self-speech flag into self-speech flag audio and converts the speaking direction into speaking direction audio, and includes the converted audio in the voice recognition data. Therefore, even if the mobile terminal device 12 or the voice recognition server 13 does not support the self-speech flag or the speaking direction, i.e., supports only audio signals, the voice recognition process can be performed on the voice recognition data. As a result, the information processing unit 161 can recognize the self-speech flag and the speaking direction from the character data obtained as a result of the voice recognition process.

[0195] On the other hand, when the self-speech flag and the speaking direction are directly associated with the audio signal to generate voice recognition data, the mobile terminal device 12 and the voice recognition server 13, which do not correspond to the self-speech flag or the speaking direction, ignore the self-speech flag or the speaking direction, for example. Therefore, only the text data of the audio signal is transmitted from the voice recognition server 13 to the smart glasses 11 via the mobile terminal device 12. As a result, it is difficult for the information processing unit 161 to identify the self-speech section or recognize the speaking direction from the text data of the received audio signal.

[0196] The processing unit 261 processes the audio signal to shorten the duration of the audio signal. Therefore, if the usage fee for the voice recognition service (voice recognition processing) provided by the voice recognition server 13 is a pay-per-use system, the usage fee can be reduced.

[0197] The generation unit 160a generates the BLE device information so as to include an identifier included in the Classic device information. Specifically, the connection methods are different between the HFP connection via Classic communication and the GATT connection via BLE communication. Therefore, in order to perform both connections between the smart glasses 11 and the mobile terminal device 12, the mobile terminal device 12 needs to associate the smart glasses 11 that perform the HFP connection with the smart glasses 11 that perform the GATT connection. Therefore, the generation unit 160a includes an identifier included in the Classic device information transmitted during the HFP connection in the BLE device information transmitted during the GATT connection.

[0198] This allows the mobile terminal device 12 to associate the smart glasses 11 that perform the HFP connection with the smart glasses 11 that perform the GATT connection. As a result, it is possible to prevent the mobile terminal device 12 from performing a GATT connection with a smart glass 11 that is different from the smart glass 11 that performed the HFP connection.

[0199] 2. Second embodiment Example of functional configuration of character conversion processing unit A second embodiment of an information processing system to which the present technology is applied differs from the information processing system 10 in Fig. 1 in that a character conversion processing unit generates data for speech recognition in which speaking direction voice is added to an audio signal of another utterance section, but is otherwise configured in the same way as the information processing system 10. Therefore, the following description will focus on the character conversion processing unit.

[0200] FIG. 23 is a block diagram showing an example of the functional configuration of the character conversion processing unit in the second embodiment.

[0201] In the character conversion processing unit 340 in Fig. 23, parts corresponding to those in the character conversion processing unit 111 in Fig. 3 are assigned the same reference numerals. Therefore, explanations of those parts will be omitted as appropriate, and the following explanation will focus on parts that differ from the character conversion processing unit 111. The character conversion processing unit 340 differs from the character conversion processing unit 111 in that it includes a data generation unit 341 and an information processing unit 342 instead of the data generation unit 159 and the information processing unit 161, but is otherwise configured in the same way as the character conversion processing unit 111.

[0202] Data generator 341 generates speech direction audio based on the speech direction supplied from direction detector 158. Data generator 341 processes the audio signal supplied from specific direction amplifier 156 based on the own speech flag supplied from own speech detector 153. As a result, data generator 341 deletes silent intervals and own speech intervals and generates a sped-up audio signal, i.e., a sped-up audio signal of other speech intervals. Data generator 341 generates speech recognition data in which speech direction audio is associated with the audio signal of other speech intervals, and supplies the data to transmitter 160b.

[0203] The information processing unit 342 extracts character data corresponding to the audio signal of the other speech section and character data corresponding to the speech direction voice from the character data supplied from the receiving unit 160c. The information processing unit 342 recognizes the speech direction corresponding to the character data of the other speech section based on the extracted character data corresponding to the speech direction voice. The information processing unit 342 generates display data for displaying character information corresponding to the character data of the other speech section based on the speech direction, and supplies the display data to the display control unit 58.

[0204] <Configuration Example of Data Generator> FIG. 24 is a block diagram showing a configuration example of the data generator 341 of FIG.

[0205] In data generation unit 341 in Fig. 24, parts corresponding to those in data generation unit 159 in Fig. 11 are assigned the same reference numerals. Therefore, explanation of those parts will be omitted as appropriate, and the explanation will focus on parts that differ from data generation unit 159. Data generation unit 341 differs from data generation unit 159 in that it has a processing unit 361 and an addition unit 364 instead of processing unit 261 and addition unit 264, and in that it does not have a flag audio unit 262, but is otherwise configured similarly to data generation unit 159.

[0206] A self-speech flag is supplied to processing unit 361 from self-speech detection unit 153 in FIG. 23 . Based on the self-speech flag, processing unit 361 processes the audio signal supplied from specific direction amplification unit 156 by deleting self-speech sections and silent sections and speeding up the audio signal, thereby shortening the time of the audio signal. The shortened time is determined by the performance of speech recognition server 13, such as recognition accuracy and real-time performance. Processing unit 361 supplies the processed audio signal to adding unit 364.

[0207] The adding unit 364 adds the speaking direction voice supplied from the directional voice unit 263 to the audio signal supplied from the processing unit 361 to generate voice recognition data, and supplies the data to the transmitting unit 160b.

[0208] <Configuration Example of Processing Unit> FIG. 25 is a block diagram showing a configuration example of the processing unit 361 in FIG.

[0209] In processing unit 361 in Fig. 25, parts corresponding to those in processing unit 261 in Fig. 12 are assigned the same reference numerals. Therefore, the description of those parts will be omitted as appropriate, and the following description will focus on parts that differ from processing unit 261. Processing unit 361 differs from processing unit 261 in that it includes a self-speech deletion unit 384, but is otherwise configured the same as processing unit 261.

[0210] Own-speech deletion unit 384 deletes the self-speech section from the audio signal supplied from equalizer 283, based on the self-speech flag supplied from self-speech detection unit 153. Own-speech deletion unit 384 supplies the resulting audio signal of the other-speech section to speed change unit 284.

[0211] 25, processing by self-speech deletion unit 384, speed change unit 284, and silence deletion unit 285 is performed in that order, but the order of processing may be different. At least two of the processes by self-speech deletion unit 384, speed change unit 284, and silence deletion unit 285 may be performed in parallel, and the processing results may be combined at a later stage.

[0212] The voice recognition processing unit in the second embodiment is the same as the voice recognition processing unit 211 in FIG. 5 except that the information processing unit 223 is configured in the same manner as the information processing unit 342 in FIG. 23, and therefore is not shown in the figure.

[0213] <Examples of Self-Speech Sections and Silence Sections> FIG. 26 is a diagram showing examples of self-speech sections and silence sections deleted by processing unit 361 in FIG.

[0214] In Fig. 26, the horizontal axis represents time [s]. The upper part of Fig. 26 represents the audio signal [dB] input to self-speech deletion section 384 in Fig. 25, and the lower part represents the self-speech flag.

[0215] 26 are similar to those in the upper and lower parts of Fig. 7, and therefore description thereof will be omitted. When the audio signal shown in the upper part of Fig. 26 and the self-speech flag shown in the lower part of Fig. 26 are input, self-speech deletion unit 384 deletes, as a self-speech section, for example, a section from time t4 to time t6 in which sections in which the self-speech flag is set to ON continue at intervals of a predetermined time or less. Silence deletion unit 285 deletes, as a silent section, a section from time t1 to time t2 in which an audio signal at a level below the threshold continues for a time equal to or longer than the threshold.

[0216] As a result, the speech recognition data includes an audio signal in which other speech sections, such as the section from time t0 to t1, which is neither a self-speech section nor a silent section, have been sped up.

[0217] Note that self-speech deletion section 384 may delete only the section for which the self-speech flag is set to ON.

[0218] <Example of Character Information> FIG. 27 is a diagram showing an example of character information displayed on the display 22 in the second embodiment.

[0219] In the example of Fig. 27, the same speech is being made as in the example of Fig. 10. In this case, as shown in Fig. 27, the character information displayed on the display 22 is the same as the character information in the first to third lines from the top in Fig. 10.

[0220] That is, since the voice recognition data does not include the audio signal of the self-speech section and the self-speech flag voice, the character data transmitted from the mobile terminal device 12 does not include the character data of the utterance "myself" and the user's utterance "hello." Therefore, the character information "hello" in the fourth line from the top in Figure 10 is not displayed on the display 22.

[0221] <Explanation of Processing> FIG. 28 is a flowchart illustrating the processing performed by the processing unit 361 in FIG.

[0222] The processing of steps S251 to S254 in Fig. 28 is the same as the processing of steps S151 to S154 in Fig. 21. After the processing of step S254, in step S255, own-speech deletion unit 384 deletes the own-speech section from the audio signal whose frequency characteristics have been adjusted by the processing of step S254, based on the own-speech flag supplied from own-speech detection unit 153. Own-speech deletion unit 384 supplies the resulting audio signal of the other-speech section to speed change unit 284.

[0223] In step S256, the speed change unit 284 speeds up the audio signal of the other speech section obtained by the process in step S255 without changing the pitch, and supplies the speeded up audio signal to the silence deletion unit 285.

[0224] The processing in steps S257 and S258 is the same as the processing in steps S156 and S157 in Fig. 21, and therefore a description thereof will be omitted. After the processing in step S258, the processing ends.

[0225] <Description of Character Data Processing> FIG. 29 is a flowchart illustrating character data processing by the information processing unit 342 in FIG.

[0226] In step S281 of FIG. 29, the information processing unit 342 extracts character data of the audio signal in the other speech section and character data of the speech direction voice from the character data received by the receiving unit 160c of FIG.

[0227] In step S282, the information processing unit 342 recognizes the speaking direction corresponding to the character data of the other speech section based on the character data of the speaking direction voice extracted by the process of step S281.

[0228] In step S283, the information processing unit 342 generates display data for displaying character information corresponding to the character data of the other utterance section extracted in step S281, based on the speech direction recognized in step S282. The information processing unit 342 supplies the display data to the display control unit 58.

[0229] In step S284, the display control unit 58 displays character information on the display 22 based on the display data generated in step S283, and then the character data processing ends.

[0230] In the character conversion processing unit 340, the transmission unit 160b may transmit the speaking direction together with the voice recognition data to the mobile terminal device 12. In this case, the storage unit 188 stores the voice recognition data together with the speaking direction.

[0231] The smart glasses 11 in the second embodiment may also have a display for the conversation partner, as in the first embodiment. In this case, the smart glasses 11 also display, for example, text information corresponding to the audio signal of the other user's speech section on the display for the conversation partner. This allows the conversation partner of the user wearing the smart glasses 11 to check how their speech is converted into text information.

[0232] As described above, in character conversion processing unit 340, self-speech deletion unit 384 deletes the self-speech section of the audio signal based on the self-speech flag, thereby shortening the duration of the audio signal. Therefore, if the usage fee for the speech recognition service provided by speech recognition server 13 is a pay-per-use system, the usage fee can be further reduced.

[0233] 3. Third embodiment Example of functional configuration of character conversion processing unit A third embodiment of an information processing system to which the present technology is applied differs from the information processing system 10 in Fig. 1 in that the character conversion processing unit detects speaker identification information that identifies a speaker instead of a speaking direction, but is otherwise configured in the same manner as the information processing system 10. Therefore, the following description will focus on the character conversion processing unit.

[0234] FIG. 30 is a block diagram showing an example of the functional configuration of the character conversion processing unit in the third embodiment.

[0235] In the character conversion processing unit 440 in Fig. 30, parts corresponding to the character conversion processing unit 111 in Fig. 3 are assigned the same reference numerals. Therefore, explanation of those parts will be omitted as appropriate, and the following explanation will focus on parts that differ from the character conversion processing unit 111. The character conversion processing unit 440 differs from the character conversion processing unit 111 in that it includes a speaker detection unit 441, a data generation unit 442, and an information processing unit 443 instead of the direction detection unit 158, the data generation unit 159, and the information processing unit 161. Other than that, it is configured in the same way as the character conversion processing unit 111.

[0236] The speaker detection unit 441 detects speaker identification information such as the name of the speaker based on the audio signal supplied from the amplifier 157 as speaker information, and supplies the information to the data generation unit 442 .

[0237] Data generation unit 442 processes the audio signal supplied from specific direction amplification unit 156. Data generation unit 442 generates a self-speech flag sound based on the self-speech flag supplied from self-speech detection unit 153. Data generation unit 442 generates a speaker identification sound, which is an audio signal representing the speaker identification information supplied from speaker detection unit 441, based on the speaker identification information. Data generation unit 442 generates voice recognition data in which the self-speech flag sound and the speaker identification sound are associated with the audio signal, and supplies the data to transmission unit 160b.

[0238] The information processing unit 443 extracts character data corresponding to the audio signal, the self-speech flag voice, and the speaker identification voice from the character data supplied from the receiving unit 160c. The information processing unit 443 identifies the self-speech section of the character data corresponding to the audio signal based on the character data corresponding to the extracted self-speech flag voice. The information processing unit 443 recognizes speaker identification information corresponding to the character data of the other-speech section based on the character data corresponding to the extracted speaker identification voice.

[0239] The information processing unit 443 generates display data for displaying text information corresponding to the text data of the audio signal based on the self-speech section and the speaker identification information. For example, the information processing unit 443 generates display data for displaying text information of the audio signal in the other-speech section by adding text information indicating the speaker identification information of the audio signal to the text information of the other-speech section. Alternatively, for example, the information processing unit 443 generates display data for displaying the text information of the audio signal in the other-speech section in a display format that differs for each piece of speaker identification information corresponding to the speaker identification information of the audio signal. The information processing unit 443 supplies the generated display data to the display control unit 58.

[0240] <Example of Detection of Speaker Identifying Information> FIG. 31 is a diagram showing an example of detection of speaker identifying information by the speaker detecting unit 441 of FIG.

[0241] In the example of FIG. 31 , a user 460 wearing smart glasses 11 is having a conversation with five speakers 461 to 465 around a table 471.

[0242] In this case, the speaker detection unit 441 performs predetermined audio signal processing on the audio signal to detect the name of any of the speakers 461 to 465 as speaker identification information of the audio signal.

[0243] The data generation process and character data process in the third embodiment are similar to the data generation process in FIG. 20 and the character data process in FIG. 22 except that the speaking direction and speaking direction voice are replaced by speaker identifying information and speaker identifying voice, respectively, and therefore are not shown in the figures.

[0244] The information processing unit 443 may generate display data for displaying only the text information corresponding to the text data of the audio signal of the self-speech section or the other-speech section based on the identified self-speech section.

[0245] The smart glasses 11 in the third embodiment also have a display for the dialogue partner, as in the first embodiment, and may be configured to display text information corresponding to the audio signal of the other speech section on the display for the dialogue partner.

[0246] As described above, in the character conversion processing unit 440, the speaker detection unit 441 detects speaker identification information based on the audio signal acquired by the microphone 21, and the data generation unit 159 generates voice recognition data in which the audio signal is associated with speaker identification voice, etc. Therefore, the information processing unit 443 can recognize speaker identification information corresponding to the character data of the audio signal based on the character data obtained as a result of performing voice recognition processing on the voice recognition data.

[0247] As a result, the display control unit 58 can add, for example, text information representing the speaker identification information to text information corresponding to the audio signal of the speaker identified by the speaker identification information, and display the added text information on the display 22. As a result, the user can recognize the text information of the utterance and the speaker who made the utterance. This effect is particularly beneficial when the user is conversing with multiple speakers.

[0248] The data generator 442 converts the self-speech flag into a self-speech flag sound and converts the speaker identification information into a speaker identification sound to be included in the voice recognition data. Therefore, as in the first embodiment, even if the mobile terminal device 12 and the voice recognition server 13 only support audio signals, the information processor 443 can recognize the self-speech flag and the speaker identification information.

[0249] 4. Fourth Embodiment Example of Functional Configuration of Character Conversion Processing Unit A fourth embodiment of an information processing system to which the present technology is applied differs from the information processing system of the second embodiment in that a voice recognition processing unit 611 generates voice recognition data instead of a character conversion processing unit, and that the voice recognition processing unit 611 reproduces an audio signal. The rest of the configuration is the same as that of the second embodiment. Therefore, the following description will focus on these points.

[0250] FIG. 32 is a block diagram showing an example of the functional configuration of the character conversion processing unit in the fourth embodiment.

[0251] In the character conversion processing unit 540 in Fig. 32, parts corresponding to those in the character conversion processing unit 340 in Fig. 23 are assigned the same reference numerals. Therefore, explanations of those parts will be omitted as appropriate, and the following explanation will focus on parts that differ from the character conversion processing unit 340. The character conversion processing unit 540 differs from the character conversion processing unit 340 in that it includes a signal generation unit 541 and a communication unit 542 instead of the data generation unit 341 and the communication unit 160, but is otherwise configured in the same way as the character conversion processing unit 340.

[0252] 12 . Signal generation unit 541 thereby performs noise removal, normalization processing, and frequency characteristic adjustment on the audio signal supplied from specific direction amplification unit 156. Signal generation unit 541 generates a transmission signal in which the resulting audio signal is associated with the own-speech flag supplied from own-speech detection unit 153 and the speech direction supplied from direction detection unit 158. Signal generation unit 541 supplies the transmission signal to communication unit 542.

[0253] The communication unit 542 differs from the communication unit 160 of FIG. 23 in that it includes a transmission unit 542b instead of the transmission unit 160b, but is otherwise configured in the same manner as the communication unit 160.

[0254] Similar to the transmitter 160b, the transmitter 542b performs pairing with the mobile terminal device 12, transmits Classic device information to the mobile terminal device 12, and establishes an HFP connection. Similar to the transmitter 160b, the transmitter 542b performs a GATT connection with the mobile terminal device 12 by transmitting BLE device information to the mobile terminal device 12. The transmitter 542b controls the wireless communication unit 92 and transmits the transmission signal supplied from the signal generation unit 541 to the mobile terminal device 12 via the antenna 61 by Classic communication.

[0255] <Example of Functional Configuration of Speech Recognition Processing Unit, Classic Connection Unit, and Call Unit> FIG. 33 is a block diagram showing an example of functional configuration of the speech recognition processing unit, classic connection unit, and call unit in the fourth embodiment.

[0256] 5 in that the speech recognition processing unit 611 includes a communication unit 621, a storage control unit 622, and an information processing unit 624 instead of the communication unit 221, the storage control unit 222, and the information processing unit 223. The speech recognition processing unit 611 also includes a data generation unit 623 and a playback unit 625. Apart from these points, the speech recognition processing unit 611 is configured in the same manner as the speech recognition processing unit 211.

[0257] The communication unit 621 differs from the communication unit 221 in FIG. 5 in that it includes a transmission unit 621b and a reception unit 621c instead of the transmission unit 221b and the reception unit 221c, but is otherwise configured in the same manner as the communication unit 221.

[0258] The transmitter 621b controls the communication unit 189 and transmits the voice recognition data supplied from the storage control unit 622 to the voice recognition server 13 by wireless LAN communication via the wireless LAN antenna 190b. Similar to the transmitter 221b, the transmitter 621b transmits character data received from the voice recognition server 13 by the receiver 221c to the smart glasses 11.

[0259] Similar to the receiving unit 221c, the receiving unit 621c receives BLE device information transmitted from devices that can be connected via BLE communication, including the smart glasses 11. Similar to the receiving unit 221c, the receiving unit 621c connects, via BLE communication, to the smart glasses 11 that have transmitted the BLE device information supplied from the storage control unit 622, among the received BLE device information.

[0260] The receiving unit 621c controls the communication unit 189 to receive, via the Bluetooth antenna 190, a transmission signal transmitted by the transmitting unit 542b in Fig. 32 via classic communication. The receiving unit 621c supplies the transmission signal to the storage control unit 622. Similar to the receiving unit 221c, the receiving unit 621c receives character data transmitted from the voice recognition server 13 and supplies the character data to the transmitting unit 621b and the storage control unit 622.

[0261] The storage control unit 622 stores the transmission signal supplied from the receiving unit 621c in the storage unit 188. The storage control unit 622 reads out the transmission signal stored in the storage unit 188 and supplies it to the data generating unit 623. The storage control unit 622 stores the intermediate processing results and voice recognition data supplied from the data generating unit 623 in the storage unit 188. The storage control unit 622 supplies the voice recognition data to the transmitting unit 621b.

[0262] The storage control unit 622 associates the character data supplied from the receiving unit 621c with the voice recognition data corresponding to the character data and stores the same as a log in the storage unit 188. The storage control unit 622 reads out the character data stored in the storage unit 188 and supplies it to the information processing unit 624. The storage control unit 622 reads out the transmission signal stored in the storage unit 188 and supplies it to the playback unit 625.

[0263] The storage control unit 622 stores the BLE device information and Classic device information supplied from the generation unit 221a in the storage unit 188. The storage control unit 622 reads out the Classic device information stored in the storage unit 188 and supplies it to the generation unit 221a. The storage control unit 622 reads out the BLE device information stored in the storage unit 188 and supplies it to the reception unit 621c.

[0264] The data generator 623 generates speech direction audio based on the speech direction included in the transmission signal supplied from the storage controller 622. The data generator 623 processes the audio signal included in the transmission signal based on the user's own speech flag included in the transmission signal. As a result, the data generator 341 generates an audio signal of the other user's speech section at a higher speed. The data generator 623 generates speech recognition data in which the audio signal is associated with the speech direction audio, and supplies the data to the storage controller 622.

[0265] 32 based on the character data supplied from the storage control unit 622, the information processing unit 624 generates display data and supplies it to the display control unit 224. The display control unit 224 displays character information on the display 187a based on the display data.

[0266] Reproducing unit 625 reproduces the audio signal included in the transmission signal supplied from storage control unit 622 based on the own-speech flag included in the transmission signal. Specifically, reproducing unit 625 extracts and reproduces the own-speech section or other-speech section from the audio signal included in the transmission signal based on the own-speech flag included in the transmission signal. Reproducing unit 625 outputs the audio obtained as a result of the reproduction from audio output unit 187b.

[0267] <Configuration Example of Data Generator> FIG. 34 is a block diagram showing a configuration example of the data generator 623 in FIG.

[0268] In data generation unit 623 in Fig. 34, parts corresponding to those in data generation unit 341 in Fig. 24 are assigned the same reference numerals. Therefore, explanation of those parts will be omitted as appropriate, and the explanation will focus on parts that differ from data generation unit 341. Data generation unit 623 differs from data generation unit 341 in that it includes processing unit 640 instead of processing unit 361, but is otherwise configured similarly to data generation unit 341.

[0269] Processing section 640 is made up of a self-speech deletion section 641 , a silence deletion section 642 , a speed change section 643 , and a filter 644 .

[0270] Own speech deletion section 641 deletes the self-speech section from the audio signal in the transmission signal based on the self-speech flag in the transmission signal supplied from storage control section 622 in Fig. 33. Own speech deletion section 384 supplies the resulting audio signal of the other-speech section to storage control section 622, which stores it in storage section 188.

[0271] 12 from the audio signal of the other speech section read from the storage unit 188 by the storage control unit 622. The silence deletion unit 642 supplies the audio signal of the other speech section from which the silence sections have been deleted to the storage control unit 622, which stores the audio signal in the storage unit 188.

[0272] The speed change unit 643 speeds up the audio signal of the other speech sections, from which the silent sections have been deleted and which has been read out from the storage unit 188 by the storage control unit 622, without changing the pitch, in the same way as the speed change unit 284 in Fig. 12. The speed change unit 643 supplies the resulting audio signal to the filter 644.

[0273] 12, and performs filtering on the audio signal supplied from the speed change unit 643. The filter 644 supplies the resulting audio signal to the addition unit 364.

[0274] 34, processing unit 640 performs the processes of self-speech deletion unit 641, silence deletion unit 642, and speed change unit 643 in that order, but the order of processing may be different. The processes of at least two of self-speech deletion unit 641, silence deletion unit 642, and speed change unit 643 may be performed in parallel, and the processing results may be synthesized at a later stage. Furthermore, processing unit 640 may include only one of self-speech deletion unit 641, silence deletion unit 642, and speed change unit 643.

[0275] 35 is a flowchart illustrating the character conversion process performed between the character conversion processing unit 540, the voice recognition processing unit 611, and the voice recognition server 13. This character conversion process is started, for example, when the user starts a dialogue.

[0276] In step S381 of Fig. 35, the character conversion processing unit 540 performs a transmission signal generation process to generate a transmission signal. Details of this transmission signal generation process will be described later with reference to Fig. 36. In step S382, the transmission unit 542b transmits the transmission signal generated by the process of step S381 to the mobile terminal device 12 by classic communication.

[0277] In step S391, the receiving unit 621c of the voice recognition processing unit 611 receives the transmission signal transmitted in the process of step S382. In step S392, the data generating unit 623 performs a data generating process to generate voice recognition data from the transmission signal. Details of this data generating process will be described later with reference to FIG. 38. In step S393, the transmitting unit 621b transmits the voice recognition data generated in the process of step S391 to the voice recognition server 13.

[0278] The processes of steps S401 to S403 are similar to the processes of steps S101 to S103 in Fig. 19, and therefore their explanations are omitted. The processes of steps S394 to S396 are similar to the processes of steps S93 to S95 in Fig. 19, except that the speech recognition data to be associated with the character data is generated by the process of step S392, and therefore their explanations are omitted. The processes of steps S383 and S384 are similar to the processes of steps S83 and S84 in Fig. 19, and therefore their explanations are omitted.

[0279] <Description of Transmission Signal Generation Processing> FIG. 36 is a flowchart illustrating the transmission signal generation processing in step S381 of FIG.

[0280] The processes of steps S421 to S429 in Fig. 36 are similar to the processes of steps S121 to S129 in Fig. 20, and therefore descriptions thereof will be omitted. The processes of steps S430 to S433 are similar to the processes of steps S151 to S154 in Fig. 21, and therefore descriptions thereof will be omitted.

[0281] In step S434, signal generation unit 541 generates a transmission signal by associating the audio signal obtained as a result of the processing in step S433 with the own-speech flag supplied from own-speech detection unit 153 and the speaking direction supplied from direction detection unit 158. Signal generation unit 541 supplies the transmission signal to transmission unit 542b, and the process returns to step S381 in Fig. 35. Then, the process proceeds to step S382.

[0282] <Description of Data Generation Processing> FIG. 37 is a flowchart illustrating the data generation processing in step S392 of FIG.

[0283] In step S450 of Fig. 37, the processing unit 640 of the data generation unit 623 performs processing processing to process the audio signal included in the transmission signal received by the processing of step S391 of Fig. 35. Details of this processing processing will be described later with reference to Fig. 38.

[0284] In step S 451 , the direction voice unit 263 generates speech direction voice based on the speech direction included in the transmission signal received in the process of step S 391 , and supplies the generated speech direction voice to the adding unit 264 .

[0285] In step S452, the adding unit 264 generates data for voice recognition by associating the audio signal obtained in the process of step S450 with the speaking direction voice generated in the process of step S451. The adding unit 264 notifies the storage control unit 622 of the completion of generation of the data for voice recognition by supplying the data for voice recognition to the storage control unit 622. The storage control unit 622 stores the data for voice recognition supplied from the adding unit 264 in the storage unit 188 and supplies it to the transmission unit 621b. Then, the process returns to step S392 in FIG. 35 and proceeds to step S393.

[0286] <Explanation of Processing> FIG. 38 is a flowchart illustrating the processing in step S450 of FIG.

[0287] In step S471 of FIG. 38, the receiving unit 621c supplies the transmission signal received in the process of step S391 of FIG.

[0288] In step S481, storage control unit 622 acquires the transmission signal supplied by the processing of step S471. In step S482, storage control unit 622 stores the transmission signal acquired by the processing of step S481 in storage unit 188. In step S483, storage control unit 622 reads out the transmission signal stored in storage unit 188, and supplies the audio signal and own-speech flag included in the transmission signal to own-speech deletion unit 641.

[0289] In step S491, self-speech deletion unit 641 acquires the audio signal and self-speech flag supplied by the processing of step S482. In step S492, self-speech deletion unit 641 determines, based on the self-speech flag acquired by the processing of step S491, whether or not a self-speech section is present in the audio signal acquired together with the self-speech flag.

[0290] If it is determined in step S492 that there is a self-speech section, i.e., if there is a section where the self-speech flag is set to on, the process proceeds to step S493. In step S493, self-speech deletion unit 641 deletes the audio signal of the self-speech section, i.e., the section where the self-speech flag is set to on, and the process proceeds to step S494.

[0291] On the other hand, if it is determined in step S492 that there is no self-speech section, that is, if the self-speech flag is off for all sections, the process skips step S493 and proceeds to step S494.

[0292] In step S494, own speech deletion unit 641 supplies the audio signal from which the own speech section has been deleted by the processing of step S493, or the audio signal from which it has been determined by the processing of step S492 that there is no own speech section, to storage control unit 622. That is, own speech deletion unit 641 supplies the audio signal of the other-speech section to storage control unit 622. As a result, own speech deletion unit 641 notifies storage control unit 622 of the completion of processing.

[0293] In step S484, the storage control unit 622 acquires the audio signal of the other speech section supplied by the processing in step S494, and overwrites the audio signal of the transmission signal stored in the storage unit 188 by the processing in step S482. In step S485, the storage control unit 622 reads out the audio signal of the other speech section from the storage unit 188, and supplies it to the silence deletion unit 642.

[0294] In step S501, the silence deletion unit 642 acquires the audio signal of the other speech section supplied by the process of step S485. In step S502, the silence deletion unit 642 determines whether or not the audio signal acquired by the process of step S501 includes a silence section. If it is determined in step S502 that a silence section exists, the process proceeds to step S503.

[0295] In step S503, the silence deletion unit 642 deletes the silence intervals from the audio signal of the other speech interval acquired in the process of step S501, and the process proceeds to step S504.

[0296] On the other hand, if it is determined in step S502 that there is no silent interval, the process skips step S503 and proceeds to step S504.

[0297] In step S504, the silence deletion unit 642 supplies the audio signal of the other speech section from which the silence section has been deleted in the processing of step S503 or the audio signal of the other speech section from which it has been determined in the processing of step S502 that there is no silent section to the storage control unit 622. In response to this, the silence deletion unit 642 notifies the storage control unit 622 of the completion of the processing.

[0298] In step S486, the storage control unit 622 acquires the audio signal supplied in the process of step S504, and overwrites the audio signal of the other speech section stored in the storage unit 188 in the process of step S484. In step S487, the storage control unit 622 reads out the audio signal stored in the storage unit 188 in the process of step S486, and supplies it to the speed change unit 643.

[0299] In step S511, the speed change unit 643 acquires the audio signal supplied by the processing of step S487. In step S512, the speed change unit 643 speeds up the audio signal acquired by the processing of step S511 without changing the pitch. In step S513, the speed change unit 643 supplies the resulting audio signal to the filter 644.

[0300] In step S521, the filter 644 acquires the audio signal supplied by the processing in step S513. In step S522, the filter 644 performs filtering on the audio signal acquired by the processing in step S521, and supplies the resulting audio signal to the adding unit 364. Then, the processing returns to step S450 in Fig. 37 and proceeds to step S451.

[0301] 38 of the data generation process in Fig. 37 is performed on a transmission signal every predetermined time (n seconds). That is, the storage control unit 622 reads out the transmission signal and generates voice recognition data every time the transmission signal for the predetermined time is stored in the storage unit 188.

[0302] At least some of the self-speech detection unit 153 and direction detection unit 158 ​​that are located before the signal generation unit 541 may be provided in the voice recognition processing unit 611 instead of the character conversion processing unit 440 .

[0303] The smart glasses 11 in the fourth embodiment also have a display for the conversation partner, as in the first embodiment, and may also display text information corresponding to the audio signal of the other speech section on the display for the conversation partner.

[0304] As described above, in character conversion processing unit 540, self-speech detection unit 153 detects a self-speech section from the audio signal acquired by microphone 21, and signal generation unit 541 generates a transmission signal in which a self-speech flag, etc. is associated with the audio signal. Therefore, using the voice recognition data generated by voice recognition processing unit 611 from the transmission signal, character conversion processing unit 540 can perform different processing on the self-speech section and sections other than the self-speech section, as in the first embodiment.

[0305] In the character conversion processing unit 540, the direction detection unit 158 ​​detects the speaking direction based on the audio signal acquired by the microphone 21, and the signal generation unit 541 generates a transmission signal in which the speaking direction, etc. is associated with the audio signal. Therefore, the character conversion processing unit 540 can recognize the speaking direction corresponding to the character data of the audio signal, similar to the first embodiment, based on the character data corresponding to the voice recognition data generated from the transmission signal by the voice recognition processing unit 611.

[0306] The data generator 623 converts the speaking direction into speaking direction audio and includes it in the speech recognition data. Therefore, even if the speech recognition server 13 does not support speaking direction, i.e., supports only audio signals, the data generator 623 can perform speech recognition processing on the speech recognition data.

[0307] The processing unit 640 processes the audio signal to shorten the duration of the audio signal. Therefore, in the fourth embodiment, as in the second embodiment, if the usage fee for the speech recognition service provided by the speech recognition server 13 is a pay-per-use system, the usage fee can be reduced.

[0308] 5. Fifth Embodiment Configuration Example of Information Processing System FIG. 39 is a diagram showing a configuration example of a fifth embodiment of an information processing system to which the present technology is applied.

[0309] In the information processing system 710 of Fig. 39, parts corresponding to those in the information processing system 10 of Fig. 1 are assigned the same reference numerals. Therefore, the description of those parts will be omitted as appropriate, and the description will focus on parts that differ from the information processing system 10. The information processing system 710 differs from the information processing system 10 in that the mobile terminal device performs voice recognition processing, but is otherwise configured in the same way as the information processing system 10.

[0310] Specifically, the information processing system 710 is composed of smart glasses 11 and a mobile terminal device 712.

[0311] The mobile terminal device 712 establishes an HFP connection with the smart glasses 11 via Classic communication. The mobile terminal device 712 executes, for example, an application for a voice recognition service. As a result, the mobile terminal device 712 establishes a GATT connection with the smart glasses 11 via BLE communication. The mobile terminal device 712 receives voice recognition data transmitted from the smart glasses 11 via Classic communication. The mobile terminal device 712 performs voice recognition processing by, for example, executing an embedded voice recognition application on this voice recognition data. The mobile terminal device 712 transmits character data obtained as a result of the voice recognition processing to the smart glasses 11 via BLE communication. The mobile terminal device 712 also transmits commands to the smart glasses 11 via BLE communication as necessary.

[0312] Note that, as the identifier of the smart glasses 11, information unique to the smart glasses 11 is used, rather than a model ID common to each model of the smart glasses 11. Therefore, even if multiple smart glasses 11 of the same model are present within a range where they can communicate with the mobile terminal device 12 using BLE, it is possible to reliably identify the smart glasses 11 that have established an HFP connection from the multiple smart glasses 11 and make them a target for GATT connection.

[0313] If the voice recognition server 13 (mobile terminal device 712) can perform voice recognition processing on an audio signal associated with a self-speech flag and speaker information, the voice recognition data can be an audio signal associated with a self-speech flag and a speaking direction. In this case, the voice recognition processing result for the voice recognition data is character data corresponding to the audio signal associated with the self-speech flag and speaker information.

[0314] Self-speech detection unit 153 may detect not only self-speech sections but also other-speech sections and associate other-speech flags indicating other-speech sections with the audio signal. In this case, for example, other-speech sections in the audio signal can be accurately identified based on the other-speech flags.

[0315] The smart glasses 11 may be provided with a bone conduction microphone, an acceleration sensor, a camera, or the like instead of the microphone 22-5. In this case, the self-speech detection unit 153 detects the self-speech section based on the user's vibration detected by the bone conduction microphone, the user's vibration detected as acceleration by the acceleration sensor, an image of the user's mouth captured by the camera, and the like, rather than on an audio signal. Specifically, the self-speech detection unit 153 detects, based on the detected user's vibration, a section in which vibration occurs during speech as the self-speech section. The self-speech detection unit 153 determines whether the user is speaking based on the image of the user, and detects, when it is determined that the user is speaking, a section of the audio signal acquired as the self-speech section.

[0316] Note that self-speech detection section 153 may detect the self-speech section based on a combination of at least two or more of an audio signal, a user's vibration, and an image of the user.

[0317] The number of microphones 21 included in the smart glasses 11 is not limited to five. The number of channels of the audio signals used by the self-speech detection unit 153, the specific direction amplification unit 156, and the direction detection unit 158 ​​is not limited to the above-mentioned numbers. If the number of channels of the audio signals used by the specific direction amplification unit 156 and the direction detection unit 158 ​​is the same as the number of channels of the audio signals input to the character conversion processing unit 111 (340, 540), the division unit 154 does not need to be provided.

[0318] The smart glasses 11 do not need to transmit classic device information and BLE device information to the mobile terminal device 12 (712). In this case, the mobile terminal device 12 (712) establishes a GATT connection with smart glasses 11 that are located at the same distance as the distance between the smart glasses 11 that have established the HFP connection and the mobile terminal device 12 (712), for example. Alternatively, the mobile terminal device 12 (712) establishes a GATT connection with each smart glass 11 that is located within a range where the smart glasses 11 can establish a BLE connection in a time-division manner, and identifies the smart glasses 11 that have established the HFP connection.

[0319] The voice recognition data and the transmission signal may be transmitted via wireless communication of another wireless communication standard, such as BLE communication or wireless LAN communication, instead of classic communication. The text data may be transmitted via wireless communication of another wireless communication standard, such as classic communication or wireless LAN communication, instead of BLE communication. The voice recognition data, the transmission signal, and the text data may be transmitted via wireless communication of the same wireless communication standard. The profile for the connection between the smart glasses 11 and the mobile terminal device 12 via classic communication is not limited to HFP, and may be another profile, such as SPP (Serial Port Profile).

[0320] The present technology can also be applied to wearable devices other than single-lens smart glasses. For example, the present technology can be applied to eyewear devices such as binocular smart glasses with displays on both ends. The present technology can also be applied to head-mounted displays with displays for AR (Augmented Reality), VR (Virtual Reality), and MR (Mixed Reality).

[0321] 6. Other Examples of Smart Glasses Another First Exterior Configuration Example of Smart Glasses FIG. 40 is a diagram showing another first exterior configuration example of smart glasses of an information processing system to which the present technology is applied.

[0322] 40 is a so-called scouter type smart glass. The smart glass 811 includes five microphones 821-1 to 821-5, an optical engine block 822, and a light guide plate 823.

[0323] The five microphones 821-1 to 821-5 (audio acquisition units) function as beamforming microphones in the same way as the microphones 21-1 to 21-5. The four microphones 821-1 to 821-4 are arranged on the same xy plane. The audio acquisition surfaces (microphone surfaces) of the microphones 821-1 to 821-4 face the outside of the smart glasses 811 (the side opposite to the side worn by the user), and the normal direction of the audio acquisition surfaces indicated by the arrows in FIG. 40 is, for example, a direction parallel to the xy plane. The microphone 821-5 is arranged on the bottom surface of the optical engine block 822, which is not on the xy plane. In the example of FIG. 40, the normal direction of the audio acquisition surface of the microphone 821-5 is the z direction, which is perpendicular to the xy plane.

[0324] In the following description, when there is no need to particularly distinguish between the microphones 821-1 to 821-4, they will be collectively referred to as microphone 821.

[0325] The optical engine block 822 is provided at the right end of the user when the user wears the smart glasses 811, and is connected to the light guide plate 823. The optical engine block 822 and the light guide plate 823 function as a display unit, and perform display in the same manner as the display 22 in FIG.

[0326] Specifically, the optical engine block 822 is configured to incorporate a light source, a microdisplay, a projection lens, etc., and uses a light guide plate 823 to project image display light to show images such as text information to the user.

[0327] The light guide plate 823 is disposed in front of the user's right eyeball when the user wears the smart glasses 811, and guides the image display light projected from the optical engine block 822 to reach the right eyeball. When transparent glass or resin is used for the light guide plate 823, the user can see an image generated by the image display light superimposed on the surrounding environment.

[0328] As described above, in the smart glasses 811, the four microphones 821 are arranged on the same xy plane. Therefore, the character conversion processing unit 111 (340, 440, 540) can separate sound sources and recognize the directions of the sound sources with high accuracy using audio signals of two or more channels among the audio signals acquired by the four microphones 821. As a result, the accuracy of beamforming, detection of speaking direction, and the like in the character conversion processing unit 111 (340, 440, 540) can be improved.

[0329] Furthermore, in the smart glasses 811, the microphone 821-5 is not placed on the xy plane. Therefore, the character conversion processing unit 111 (340, 440, 540) can detect the self-speech section with high accuracy by using the audio signals of the microphone 821-5 and at least one microphone 821 placed on the xy plane.

[0330] <Example of Normal Direction to Audio Acquisition Surface of Microphones> FIG. 41 is a diagram illustrating an example of the normal direction to the audio acquisition surface of the microphones 821-1 to 821-4 in FIG.

[0331] Specifically, A and B of FIG. 41 are schematic diagrams of smart glasses 811 worn on a user's head 824, viewed from above the user's head.

[0332] 41A and 41B, when the smart glasses 811 are worn on the head 824, the microphones 821-1 and 821-4 and the microphones 821-1 and 821-3 are arranged in mirror symmetry with respect to a plane 825 that divides the head 824 into left and right halves. That is, the four microphones 821 are configured by two pairs of microphones 821 that are arranged in mirror symmetry with respect to the plane 825.

[0333] Therefore, as shown in A of Figure 41, when the smart glasses 811 are curved with a predetermined curvature to fit the shape of the head 824, the normal directions of the sound acquisition surfaces of the four microphones 821 are radial, centered on a point on the surface 825.

[0334] On the other hand, as shown in B of Figure 41, when the smart glasses 811 have a flat portion parallel to the surface 825 and four microphones 821 are placed on that flat portion, the normal direction of the sound acquisition surface of the microphones 821 is perpendicular to the flat portion.

[0335] The distance between microphones 821-1 and 821-2 and the distance between microphones 821-3 and 821-4 are the same, and it is desirable that this distance be in the range of, for example, 25 mm to 60 mm.

[0336] <Second Other Exterior Configuration Example of Smart Glasses> FIG. 42 is a diagram showing a second other exterior configuration example of smart glasses of an information processing system to which the present technology is applied.

[0337] In the smart glasses 830 in Fig. 42, parts corresponding to those of the smart glasses 811 in Fig. 40 are denoted by the same reference numerals. Therefore, the description of those parts will be omitted as appropriate, and the description will focus on parts that differ from the smart glasses 811. The smart glasses 830 differ from the smart glasses 811 in that the optical engine block 822 has a built-in camera 831, but are otherwise configured in the same way as the smart glasses 811.

[0338] The camera 831 (imaging unit) captures an image in the front direction of the user wearing the smart glasses 830. The image captured by the camera 831 is used, for example, to identify the speaker.

[0339] <Third Another Example of the Appearance of Smart Glasses> FIG. 43 is a diagram showing a third another example of the appearance of smart glasses of an information processing system to which the present technology is applied.

[0340] In the smart glasses 850 in Fig. 43, parts corresponding to those of the smart glasses 811 in Fig. 40 are denoted by the same reference numerals. Therefore, the description of those parts will be omitted as appropriate, and the description will focus on parts that differ from the smart glasses 811. The smart glasses 850 differ from the smart glasses 811 in that they include a camera module box 851, but are otherwise configured in the same way as the smart glasses 811.

[0341] The camera module box 851 (imaging unit) is disposed on the outer surface of the optical engine block 822, and captures an image in the front direction of the user wearing the smart glasses 850. The image captured by the camera module box 851 is used, for example, to identify the speaker.

[0342] <Fourth Other Exterior Configuration Example of Smart Glasses> FIG. 44 is a diagram showing a fourth other exterior configuration example of smart glasses of an information processing system to which the present technology is applied.

[0343] The left-hand figure in A(B) of Figure 44 is an oblique view of the smart glasses 911 (912) when a user wearing the smart glasses 911 (912) is viewed from the right side facing the user, and the right-hand figure is an oblique view of the smart glasses 911 (912) when viewed from the left side.

[0344] 44A is a pair of eyeglass-type smart glasses 911. The smart glasses 911 include five microphones 921-1 to 921-5, an optical engine block 922, and a light guide plate 923.

[0345] Five microphones 921-1 to 921-5 (voice acquisition units) function as beamforming microphones in the same way as microphones 21-1 to 21-5. Four microphones 921-1 to 921-4 are arranged on the temples of the smart glasses 911 in the same positional relationship and orientation as microphones 821-1 to 821-4 in Fig. 40. Microphone 921-5 is arranged on the bottom surface of the optical engine block 922 facing downward.

[0346] In the following description, when there is no need to particularly distinguish between the microphones 921-1 to 921-4, they will be collectively referred to as microphone 921.

[0347] The optical engine block 922 is provided on the inside of the smart glasses 911 near the joint between the rim for the right eye and the temple, and is connected to a light guide plate 923. The light guide plate 923 is disposed within the rim for the right eye and is held by the rim for the right eye. The optical engine block 922 and the light guide plate 923 perform display in the same manner as the optical engine block 822 and the light guide plate 823 in FIG. 40 .

[0348] In the smart glasses 912 in Fig. 44B, parts corresponding to those in the smart glasses 911 in Fig. 44A are assigned the same reference numerals. Therefore, the description of those parts will be omitted as appropriate, and the description will focus on parts that are different from the smart glasses 911. The smart glasses 912 are configured in the same way as the smart glasses 911, except for the positions of the optical engine block 922 and the light guide plate 923.

[0349] Specifically, the optical engine block 922 of the smart glasses 912 is provided inside near the joint between the rim and temple for the left eye of the smart glasses 912. The light guide plate 923 of the smart glasses 912 is disposed within the rim for the left eye and is held by the rim for the left eye.

[0350] As described above, in the smart glasses 911 and 912, the four microphones 921 are arranged on the same xy plane, and the microphone 921-5 is not arranged on the xy plane. Therefore, similar to the smart glasses 811 in Fig. 40, the accuracy of beamforming, detection of speaking direction, detection of the user's own speech section, etc. in the character conversion processing unit 111 (340, 440, 540) can be improved.

[0351] <Fifth Other Exterior Configuration Example of Smart Glasses> FIG. 45 is a diagram showing a fifth other exterior configuration example of smart glasses of an information processing system to which the present technology is applied.

[0352] The left-hand figure in A(B) of Figure 45 is an oblique view of the smart glasses 971 (972) when a user wearing the smart glasses 971 (972) is viewed from the right side facing the user, and the right-hand figure is an oblique view of the smart glasses 971 (972) when viewed from the left side.

[0353] In the smart glasses 971 of A in Fig. 45, parts corresponding to those of the smart glasses 911 of A in Fig. 44 are assigned the same reference numerals. Therefore, the description of those parts will be omitted as appropriate, and the description will focus on parts that differ from the smart glasses 911. The smart glasses 971 differ from the smart glasses 911 in that they include a camera module box 981 and a microphone 982, but are otherwise configured in the same way as the smart glasses 911.

[0354] The camera module box 981 (imaging unit) is provided inside the smart glasses 971 near the joint between the rim for the left eye and the temple, and captures an image in the front direction of the user wearing the smart glasses 971. The image captured by the camera module box 981 is used, for example, to identify the speaker. A microphone 982 is arranged facing downward on the bottom surface of the camera module box 981, which is not on the xy plane.

[0355] In the smart glasses 972 in Fig. 45B, parts corresponding to those in the smart glasses 971 in Fig. 45A are assigned the same reference numerals. Therefore, the description of those parts will be omitted as appropriate, and the description will focus on parts that are different from the smart glasses 971. The smart glasses 972 are configured in the same way as the smart glasses 971, except for the positions of the optical engine block 922, the light guide plate 923, and the camera module box 981.

[0356] Specifically, the optical engine block 922 of the smart glasses 972 is provided inside and near the joint between the rim and temple for the left eye of the smart glasses 972. The light guide plate 923 of the smart glasses 972 is disposed within and held by the rim for the left eye. The camera module box 981 of the smart glasses 972 is provided inside and near the joint between the rim and temple for the right eye.

[0357] As described above, the four microphones 921 are arranged on the same xy plane in the smart glasses 971 and 972. Therefore, similar to the smart glasses 911 and 912 in Fig. 44, the accuracy of beamforming and speech direction detection in the character conversion processing unit 111 (340, 440, 540) can be improved.

[0358] Furthermore, in the smart glasses 971 and 972, the microphones 921-5 and 982 are not arranged on the xy plane. Therefore, the character conversion processing unit 111 (340, 440, 540) can detect the self-speech section with high accuracy by using the audio signals of at least one of the microphones 921-5 and 982 and one or more microphones 921.

[0359] It should be noted that the smart glasses 971 and 972 do not necessarily need to be provided with both microphones 921-5 and 982 as long as they are provided with at least one of them.

[0360] <Sixth Other Exterior Configuration Example of Smart Glasses> FIG. 46 is a diagram showing a sixth other exterior configuration example of smart glasses of an information processing system to which the present technology is applied.

[0361] The left-hand figure in A(B) of Figure 46 is an oblique view of the smart glasses 991 (992) when a user wearing the smart glasses 991 (992) is viewed from the right side facing the user, and the right-hand figure is an oblique view of the smart glasses 991 (992) when viewed from the left side.

[0362] In the smart glasses 991 of A in Fig. 46, the parts corresponding to those of the smart glasses 911 of A in Fig. 44 are denoted by the same reference numerals. Therefore, the description of those parts will be omitted as appropriate, and the description will focus on the parts that differ from the smart glasses 911. The smart glasses 991 differ from the smart glasses 911 in that they display to both of the user's eyes, but are otherwise configured in the same way as the smart glasses 911.

[0363] Specifically, the smart glasses 991 in Fig. 46A not only include an optical engine block 922 on the inside near the joint between the rim and temple for the right eye, but also include an optical engine block 1001 on the inside near the joint between the rim and temple for the left eye. The smart glasses 991 also include a light guide plate 923 in the rim for the right eye, and a light guide plate 1002 in the rim for the left eye. The light guide plate 1002 is connected to the optical engine block 1001. The optical engine block 922 and the light guide plate 923 provide display for the right eye, and the optical engine block 1001 and the light guide plate 1002 provide display for the left eye.

[0364] In the example of FIG. 46A, microphone 921-5 is provided on the bottom surface of optical engine block 922, but microphone 921-5 may be provided on the bottom surface of optical engine block 1001.

[0365] As described above, in the smart glasses 991, the four microphones 921 are arranged on the same xy plane, and the microphone 921-5 is not arranged on the xy plane. Therefore, similar to the smart glasses 911 and 912 in Fig. 44, the accuracy of beamforming, detection of speaking direction, detection of self-speech section, etc. in the character conversion processing unit 111 (340, 440, 540) can be improved.

[0366] In the smart glasses 992 in Fig. 46B, parts corresponding to those of the smart glasses 991 in Fig. 46A are assigned the same reference numerals. Therefore, the description of those parts will be omitted as appropriate, and the description will focus on parts that differ from the smart glasses 991. The smart glasses 992 differ from the smart glasses 991 in that they include a microphone 1003, but are otherwise configured in the same way as the smart glasses 991.

[0367] Specifically, microphone 1003 is positioned facing downwards on the bottom surface of optical engine block 922, which is not on the xy plane.

[0368] As described above, in the smart glasses 992, the four microphones 921 are arranged on the same xy plane. Therefore, similar to the smart glasses 991, the accuracy of beamforming and speech direction detection in the character conversion processing unit 111 (340, 440, 540) can be improved.

[0369] Furthermore, in the smart glasses 992, the microphones 921-5 and 1003 are not arranged on the xy plane. Therefore, the character conversion processing unit 111 (340, 440, 540) can detect the self-speech section with high accuracy by using the audio signals of at least one of the microphones 921-5 and 1003 and one or more microphones 921.

[0370] <Seventh Other Exterior Configuration Example of Smart Glasses> FIG. 47 is a diagram showing a seventh other exterior configuration example of smart glasses of an information processing system to which the present technology is applied.

[0371] The figure on the left side of Figure 47 is a perspective view of the smart glasses 1021 when a user wearing the smart glasses 1021 is viewed from the right side facing the user, and the figure on the right side is a perspective view of the smart glasses 1021 when viewed from the left side.

[0372] In the smart glasses 1021 in A of Fig. 47, parts corresponding to those in the smart glasses 992 in B of Fig. 46 are denoted by the same reference numerals. Therefore, the description of those parts will be omitted as appropriate, and the description will focus on parts that differ from the smart glasses 992. The smart glasses 1021 differ from the smart glasses 992 in that the optical engine block 1001 has a built-in camera 1031, but are otherwise configured in the same way as the smart glasses 992.

[0373] The camera 1031 captures an image in the front direction of the user wearing the smart glasses 1021. The image captured by the camera 1031 is used, for example, to identify the speaker.

[0374] In addition, the smart glasses 1021 do not necessarily need to be provided with both microphones 921-5 and 1003 as long as at least one of them is provided.

[0375] <Another eighth external configuration example of smartglasses> FIG. 48 is a diagram showing another eighth external configuration example of smartglasses of an information processing system to which the present technology is applied.

[0376] The figure on the left side of Figure 48 is an oblique view of the smart glasses 1051 when a user wearing the smart glasses 1051 is viewed from the right side facing the user, and the figure on the right side is an oblique view of the smart glasses 1051 when viewed from the left side.

[0377] In the smart glasses 1051 in Fig. 48, parts corresponding to those of the smart glasses 992 in Fig. 46B are assigned the same reference numerals. Therefore, the description of those parts will be omitted as appropriate, and the description will focus on parts that differ from the smart glasses 992. The smart glasses 1051 differ from the smart glasses 992 in that they include camera module boxes 1061a and 1061b and in the positions of the microphones 921-5 and 1003, but are otherwise configured in the same way as the smart glasses 992.

[0378] The camera module box 1061a is disposed outside near the joint between the rim and temple for the right eye, and captures an image in the front direction of the user wearing the smart glasses 1051. The camera module box 1061b is disposed outside near the joint between the rim and temple for the left eye, and captures an image in the front direction of the user wearing the smart glasses 1051. The images captured by the camera module boxes 1061a and 1061b are used, for example, to identify the speaker.

[0379] In the smart glasses 1051, the microphone 921-5 is disposed on the bottom surface of the camera module box 1061a, and the microphone 1003 is disposed on the bottom surface of the camera module box 1061b. Note that the position of the microphone 921-5 (1003) in the smart glasses 1051 may be the optical engine block 922 (1001), similar to the smart glasses 992.

[0380] The number of microphones 821 (921) arranged on the same xy plane is not limited to four, and can be any number equal to or greater than 2. The number of microphones 821-5 (921-5, 982, 1003) not arranged on the xy plane can also be any number equal to or greater than 1. Microphones 821-5 (921-5, 982, 1003) can be arranged at any position not on the xy plane.

[0381] The positional relationships such as "on the same plane," "mirror symmetry," "parallel," and "same distance" in the above description include not only perfect positional relationships but also substantial positional relationships within a certain error range.

[0382] The above-described smart glasses 11 (811, 830, 850, 911, 912, 971, 972, 991, 992, 1021, 1051) have a built-in battery 60, but an external mobile battery may be used instead of the battery 60.

[0383] 7. Example of a Mobile Battery for Smart Glasses FIG. 49 is a diagram showing an example of the external configuration of a mobile battery for the smart glasses 811 in this case.

[0384] A in FIG. 49 is a diagram of a user wearing smart glasses 811 as seen from the front, and B in FIG. 49 is a diagram of a user wearing smart glasses 811 as seen from the left side of the user.

[0385] 49A and 49B, the mobile battery 1081 of the smart glasses 811 is a neckband-type mobile battery configured to be worn around the user's neck. Therefore, the length of the USB (Universal Serial Bus) cable 1082 connecting the smart glasses 811 and the mobile battery 1081 can be short, which can prevent the USB cable 1082 from getting caught on objects such as surrounding devices. By providing the battery of the smart glasses 811 externally as the mobile battery 1081, the weight and size of the smart glasses 811 can be reduced compared to when the battery is built into the smart glasses 811, and comfort when wearing the smart glasses 811 can be improved.

[0386] The mobile battery 1081 has two battery boxes 1081a and 1081b into which batteries (rechargeable batteries) can be inserted. This allows the user to replace a dead battery (rechargeable battery) inserted in one of the battery boxes 1081a and 1081b with a charged spare battery without turning off the power while using the smart glasses 811. As a result, the user can extend the battery life of the smart glasses 811.

[0387] One end of the USB cable 1082 may be a magnetic adhesive type connector. In this case, if the USB cable 1082 gets caught on a surrounding object, the connection by the USB cable 1082 is immediately released, thereby ensuring the safety of the user. The number of battery boxes included in the mobile battery 1081 may be two or more.

[0388] At least a part of the above-described series of processes can also be executed by hardware.

[0389] The programs executed by the smart glasses 11 or the mobile terminal device 12 (712) can be provided by being recorded on removable media such as package media, etc. The programs can also be provided via wired or wireless transmission media such as a local area network, the Internet, or digital satellite broadcasting.

[0390] When the program is provided by being recorded on removable media, the removable media can be attached to a drive (not shown) of the smart glasses 11 or the mobile terminal device 12 (712) to install the program in the RAM 53 or 94 or the storage unit 188. When the program is provided via a wired or wireless transmission medium, the program can be received by the wireless communication unit 92 or the communication unit 189 and installed in the RAM 53 or 94 or the storage unit 188.

[0391] The programs executed by the smart glasses 11 and the mobile terminal device 12 (712) may be programs that are processed chronologically in the order described in this specification, or may be programs that are processed in parallel or at the required timing, such as when a call is made.

[0392] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0393] For example, it is possible to adopt a configuration in which all or part of the above-described embodiments are combined.

[0394] For example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0395] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

[0396] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0397] The effects described in this specification are merely examples and are not limiting, and there may be effects other than those described in this specification.

[0398] The present technology may have the following configurations. (1) An information processing device comprising: a self-speech detection unit that detects a self-speech section, which is a section of self-speech uttered by a user, from an audio signal; and a data generation unit that generates data for speech recognition in which self-speech information representing the self-speech section is associated with the audio signal. (2) The information processing device described in (1), wherein the self-speech detection unit is configured to detect the self-speech section based on at least one of the audio signal, vibration of the user, and an image of the user. (3) The information processing device described in (1) or (2), further comprising: a display control unit that causes a display unit to display text information corresponding to the audio signal, based on the self-speech information associated with the audio signal. (4) The information processing device described in (3), wherein the display control unit is configured to cause the display unit to display the text information for a section of the audio signal other than the self-speech section, based on the self-speech information. (5) The information processing device according to (3) or (4), further comprising: a character data acquisition unit that acquires character data corresponding to the character information of at least the audio signal from the data for voice recognition generated by the data generation unit. (6) The information processing device according to (5), further comprising: a transmission unit that transmits the data for voice recognition generated by the data generation unit to another information processing device, wherein the character data acquisition unit is configured to acquire the character data of at least the audio signal from the data for voice recognition transmitted from the other information processing device in response to the data for voice recognition transmitted by the transmission unit.(7) The information processing device according to (6), further comprising: a playback unit that plays back the audio signal based on the self-speech information associated with the audio signal; (9) The information processing device according to (8), further comprising: a playback unit that plays back a section of the audio signal other than the self-speech section based on the self-speech information associated with the audio signal; (10) The information processing device according to (1) or (2), further comprising: a speaker information detection unit that detects, based on the audio signal, a speaking direction, which is a direction of a speaker of the audio signal relative to the user, or speaker identification information that identifies the speaker, as speaker information, wherein the data generation unit is configured to generate the data for speech recognition in which the self-speech information and the speaker information are associated with the audio signal. (11) The information processing device according to (10), further comprising: a display control unit that displays, on a display unit, character information corresponding to the audio signal, based on the self-speech information and the speaker information associated with the audio signal. (12) The information processing device according to (11), further comprising: a character data acquisition unit that acquires character data of at least the audio signal from the data for speech recognition generated by the data generation unit.(13) The information processing device according to (12), further comprising: a transmitting unit that transmits the data for speech recognition generated by the data generating unit to another information processing device, wherein the character data acquiring unit is configured to acquire character data of at least the audio signal from the data for speech recognition transmitted from the other information processing device in accordance with the data for speech recognition transmitted by the transmitting unit. (14) The data for speech recognition is voice data in which the audio signal is associated with self-speech information voice that is voice corresponding to the self-speech information and speaker information voice that is voice corresponding to the speaker information, wherein the character data acquiring unit acquires the character data of the data for speech recognition transmitted from the other information processing device in accordance with the data for speech recognition transmitted by the transmitting unit, and the display control unit is configured to display character information corresponding to the character data of the audio signal included in the character data of the data for speech recognition acquired by the character data acquiring unit on the display unit, based on the character data of the self-speech information voice and the speaker information voice included in the character data of the data for speech recognition. (15) The information processing device according to any of (1) to (14), further comprising: a storage control unit that stores the data for speech recognition generated by the data generation unit in a storage unit. (16) The information processing device according to any of (1) to (5), (8) to (12), and (15), wherein the data generation unit comprises: a processing unit that processes the audio signal; and an adding unit that generates the data for speech recognition by associating the self-speech information with the audio signal processed by the processing unit. (17) The information processing device according to (16), further comprising: a transmitting unit that transmits the data for speech recognition to another information processing device. (18) The information processing device according to (16) or (17), wherein the processing unit is configured to process the audio signal by deleting the self-speech section of the audio signal based on the self-speech section detected by the self-speech detection unit.(19) The information processing device according to any of (16) to (18), wherein the processing unit processes the audio signal by speeding up the audio signal without changing its pitch. (20) The information processing device according to any of (16) to (19), wherein the processing unit processes the audio signal by deleting silent sections of the audio signal. (21) The information processing device according to any of (1) to (5), (8) to (12), (15), and (16), further including: a transmitting unit that transmits the voice recognition data generated by the data generating unit to another information processing device via a first wireless communication of a first wireless communication standard; and a receiving unit that receives character data corresponding to the voice recognition data transmitted from the other information processing device via a second wireless communication of a second wireless communication standard. (22) The information processing device according to (21), wherein the first wireless communication standard is Bluetooth Classic, and the second wireless communication standard is Bluetooth Low Energy (BLE). (23) The information processing device according to (21) or (22), further comprising: a generation unit that generates second information processing device information including information unique to the information processing device and used when connecting to the other information processing device via the second wireless communication, based on first information processing device information including information unique to the information processing device and used when connecting to the other information processing device via the first wireless communication. (24) The information processing device according to (23), wherein the transmission unit is also configured to transmit the first information processing device information and the second information processing device information to the other information processing device. (25) The information processing device according to (23) or (24), wherein the first information processing device information and the second information processing device information include dual connection availability information indicating that connection via both the first wireless communication and the second wireless communication is possible.(26) The information processing device according to any of (1) to (5), (8) to (12), (15), and (16), further comprising: a receiving unit that receives the voice recognition data transmitted from another information processing device via first wireless communication of a first wireless communication standard; and a transmitting unit that transmits character data corresponding to the voice recognition data received by the receiving unit to the other information processing device via second wireless communication of a second wireless communication standard. (27) The information processing device according to (26), configured such that the first wireless communication standard is Bluetooth Classic, and the second wireless communication standard is Bluetooth Low Energy (BLE). (28) The information processing device according to (26) or (27), further comprising: a generating unit that generates second information processing device information including information unique to the other information processing device and used when connecting to the other information processing device via the second wireless communication, based on first information processing device information transmitted from the other information processing device and including information unique to the other information processing device and used when connecting to the other information processing device via the first wireless communication. (29) The information processing device according to (28), wherein the receiving unit is also configured to connect, via the second wireless communication, to the other information processing device that has transmitted the second information processing device information generated by the generating unit. (30) The information processing device according to (28) or (29), wherein the first information processing device information and the second information processing device information include dual connection capability information indicating that connection via both the first wireless communication and the second wireless communication is possible. (31) The information processing device according to any of (1), (2), (4) to (6), and (8) to (13), further comprising three or more audio acquisition units that acquire the audio signal, wherein two or more of the audio acquisition units are arranged on the same plane, and the other one or more audio acquisition units are not arranged on the plane. (32) The information processing device according to (31), wherein the two or more audio acquisition units are configured by one or more pairs of the audio acquisition units arranged in mirror symmetry with each other.(33) The information processing device according to (30) or (31), wherein four of the voice acquisition units are arranged on the plane, and wherein a distance between two of the four voice acquisition units is the same as a distance between the other two of the voice acquisition units. (34) The information processing device according to any of (30) to (33), further comprising a display unit, and wherein the one or more voice acquisition units are arranged on the display unit. (35) The information processing device according to (34), further comprising an imaging unit, and wherein the one or more voice acquisition units are arranged on at least one of the display unit and the imaging unit. (36) An information processing method, the information processing device including: a self-speech detection step of detecting a self-speech section, which is a section of self-speech uttered by a user, from an audio signal; and a data generation step of generating voice recognition data in which self-speech information representing the self-speech section is associated with the audio signal. (37) A program for causing a computer to function as an information processing device including: a self-speech detection unit that detects a self-speech section, which is a section of an audio signal where the self-speech section is a user's speech; and a data generation unit that generates data for speech recognition in which the audio signal is associated with self-speech information representing the self-speech section. (38) An information processing system comprising: a first information processing device having: a self-speech detection unit that detects a self-speech section, which is a section of self-speech uttered by a user, from an audio signal; a data generation unit that generates data for speech recognition in which self-speech information representing the self-speech section is associated with the audio signal; a first transmission unit that transmits the data for speech recognition to a second information processing device; and a first reception unit that receives character data corresponding to the data for speech recognition transmitted by the first transmission unit from the second information processing device; and the second information processing device having: a second reception unit that receives the data for speech recognition transmitted by the first transmission unit; and a second transmission unit that transmits the character data corresponding to the data for speech recognition received by the second reception unit to the first information processing device.(39) An information processing system comprising: smart glasses including: a self-speech detection unit that detects a self-speech section, which is a section of self-speech uttered by a user, from an audio signal; and a data generation unit that generates data for voice recognition in which self-speech information representing the self-speech section detected by the self-speech detection unit is associated with the audio signal; and a neckband-type battery having multiple battery boxes into which batteries can be inserted.

[0399] REFERENCE SIGNS LIST 10 Information processing system, 11 Smart glasses, 12 Portable terminal device, 22 Display, 58 Display control unit, 153 Self-speech detection unit, 158 Direction detection unit, 159 Data generation unit, 160a Generation unit, 160b Transmission unit, 160c Reception unit, 221a Generation unit, 221b Transmission unit, 221c Reception unit, 261 Processing unit, 341 Data generation unit, 361 Processing unit, 364 Addition unit, 441 Speaker detection unit, 442 Data generation unit, 541 Signal generation unit, 542b Transmission unit, 625 Reproduction unit, 621b Transmission unit, 621c Reception unit, 623 Data generation unit, 640 Processing unit, 710 Information processing system, 712 Mobile terminal device, 811 smart glasses, 821-1 to 821-5 microphones, 822 optical engine block, 823 light guide plate, 830 smart glasses, 831 camera, 850 smart glasses, 851 camera module box, 911, 912 smart glasses, 921-1 to 921-5 microphones, 922 optical engine block, 923 light guide plate, 971, 972 smart glasses, 981 camera module box, 982 microphone, 991, 992 smart glasses, 1001 optical engine block, 1002 light guide plate, 1003 microphone, 1021 smart glasses, 1031 camera, 1051 smart glasses, 1061a, 1061b camera module box

Claims

1. An information processing device comprising: a self-speech detection unit that detects self-speech sections, which are sections of self-speech uttered by a user, from an audio signal; and a data generation unit that generates data for voice recognition in which the audio signal is associated with self-speech information representing the self-speech sections detected by the self-speech detection unit.

2. The information processing device according to claim 1, further comprising a display control unit that causes text information corresponding to the audio signal to be displayed on a display unit based on the self-utterance information associated with the audio signal.

3. The information processing device according to claim 1, further comprising: a playback unit that plays back the audio signal based on the self-utterance information associated with the audio signal.

4. The information processing device of claim 1, further comprising a speaker information detection unit that detects, based on the audio signal, a speaking direction, which is the direction of the speaker of the audio signal relative to the user, or speaker identification information that identifies the speaker, as speaker information, and the data generation unit is configured to generate the voice recognition data in which the self-speech information and the speaker information are associated with the audio signal.

5. The information processing device according to claim 4, further comprising a display control unit that causes text information corresponding to the audio signal to be displayed on a display unit based on the self-speech information and the speaker information associated with the audio signal.

6. The information processing device according to claim 1, wherein the data generation unit comprises: a processing unit that processes the audio signal; and an addition unit that generates the data for voice recognition by associating the self-utterance information with the audio signal processed by the processing unit.

7. The information processing device according to claim 6, wherein the processing unit is configured to process the audio signal by deleting the self-speech section from the audio signal based on the self-speech section detected by the self-speech detection unit.

8. The information processing device according to claim 6, wherein said processing unit is configured to process said audio signal by deleting silent periods from said audio signal.

9. The information processing device according to claim 1, further comprising: a transmitting unit that transmits the voice recognition data generated by the data generating unit to another information processing device via a first wireless communication of a first wireless communication standard; and a receiving unit that receives character data corresponding to the voice recognition data transmitted from the other information processing device via a second wireless communication of a second wireless communication standard.

10. The information processing device according to claim 9, wherein the first wireless communication standard is Bluetooth Classic, and the second wireless communication standard is BLE (Bluetooth Low Energy).

11. The information processing device according to claim 9, further comprising a generation unit that generates second information processing device information including information specific to the information processing device and used when connecting to the other information processing device via the second wireless communication, based on first information processing device information including information specific to the information processing device and used when connecting to the other information processing device via the first wireless communication.

12. The information processing device according to claim 1, further comprising: a receiving unit that receives the voice recognition data transmitted from another information processing device via a first wireless communication of a first wireless communication standard; and a transmitting unit that transmits character data corresponding to the voice recognition data received by the receiving unit to the other information processing device via a second wireless communication of a second wireless communication standard.

13. The information processing device according to claim 12, wherein the first wireless communication standard is Bluetooth Classic, and the second wireless communication standard is BLE (Bluetooth Low Energy).

14. The information processing device according to claim 12, further comprising a generation unit that generates second information processing device information including information specific to the other information processing device and used when connecting to the other information processing device via the second wireless communication, based on first information processing device information including information specific to the other information processing device transmitted from the other information processing device and used when connecting to the other information processing device via the first wireless communication.

15. An information processing device as described in claim 1, further comprising three or more audio acquisition units that acquire the audio signal, wherein two or more of the three or more audio acquisition units are arranged on the same plane, and the other one or more audio acquisition units are not arranged on the plane.

16. The information processing device according to claim 15, wherein the two or more voice acquisition units are configured by one or more pairs of the voice acquisition units arranged in mirror symmetry with respect to each other.

17. The information processing device of claim 15, wherein four of the voice acquisition units are arranged on the plane, and the distance between two of the four voice acquisition units is the same as the distance between the other two of the voice acquisition units.

18. The information processing device according to claim 15, further comprising a display unit, wherein the one or more voice acquisition units are configured to be arranged on the display unit.

19. An information processing device as described in claim 18, further comprising an imaging unit, wherein the one or more audio acquisition units are configured to be arranged in at least one of the display unit and the imaging unit.

20. An information processing system comprising: smart glasses having a self-speech detection unit that detects self-speech sections, which are sections of self-speech uttered by a user, from an audio signal; and a data generation unit that generates data for voice recognition in which self-speech information representing the self-speech sections detected by the self-speech detection unit is associated with the audio signal; and a neckband-type battery having multiple battery boxes into which batteries can be inserted.

Images