Audio Processing Device

Abstract

An audio processing device is disclosed. [Solution] The audio processing device includes a processor configured to perform sound source separation of audio signals associated with speakers' voices based on the sound source position of each voice, and a memory. The processor is configured to generate sound source position information indicating the sound source position of each voice using the audio signals associated with the voices, generate separated audio signals associated with each speaker's voice from the audio signals based on the sound source position information, and match the separated audio signals and the sound source position information with each other and store them in memory.

Description

[Technical field]

[0001] The present invention relates to an apparatus for processing audio and a method of operation thereof. [Background technology]

[0002] A microphone is a device that converts voice into an audio signal, which is an electrical signal. When a microphone is placed in a space where multiple speakers are located, such as a conference room or a classroom, the microphone receives all the voices from the multiple speakers and generates an audio signal associated with the voices of the multiple speakers. On the other hand, when multiple speakers speak simultaneously, the voices of the multiple speakers are all mixed together, and it is then necessary to separate a voice signal representing a specific speaker's voice from the voices of the multiple speakers. Summary of the Invention [Problem to be solved by the invention]

[0003] The problem to be solved by the present invention is to provide an apparatus and an operating method thereof that can determine the position of a speaker using a plurality of input voice signals and separate and recognize the plurality of voice signals for each speaker. The problem that the present invention seeks to solve is to provide an apparatus and method of operation that is capable of generating, in response to the speech of speakers, separate speech signals associated with each of the speakers' speech. The problem to be solved by the present invention is to provide an apparatus and an operating method thereof that can generate a translation result for each speaker's voice using separated voice signals associated with the respective speaker's voice and output the generated translation result. [Means for solving the problem]

[0004] The voice processing device of the present invention includes a processor configured to perform sound source separation of voice signals associated with the voices of speakers based on the sound source position of each voice, and a memory, and the processor is configured to generate sound source position information indicating the sound source position of each voice using the voice signals associated with the voices, generate separated voice signals associated with the voices of each speaker from the voice signals based on the sound source position information, match the separated voice signals and the sound source position information with each other and store them in the memory. Effect of the Invention

[0005] The device of the present invention can grasp the speaker's position using the voice signal, and can distinguish which speaker's voice the voice signal corresponds to based on the speaker's position. As a result, even if multiple speakers speak at the same time, the voice separation device can separate the voices by speaker. The audio processing device of the present invention is capable of generating a separated audio signal associated with audio from a specific sound source position based on the sound source position of the audio, and thus has the effect of generating an audio signal in which the influence of surrounding noise is minimized. The speech processing device of the present invention is not only capable of extracting the speech of each speaker from the transmitted speech signal, but also has the advantage of being able to determine the starting language, which is the language the speech is to be translated into, based on the source position of the speech, and to translate the speech based on the determined starting language to provide a translation result. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 illustrates an audio processing environment in accordance with an embodiment of the present invention. [Diagram 2] FIG. 1 illustrates a voice processing device according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram for explaining the operation of the voice processing device according to the embodiment of the present invention. [Figure 4] FIG. 2 is a diagram for explaining the operation of the voice processing device according to the embodiment of the present invention. [Diagram 5]FIG. 2 is a diagram for explaining the operation of the voice processing device according to the embodiment of the present invention. [Figure 6] 4 is a flowchart illustrating a speech separation method using the speech processing device according to the embodiment of the present invention. [Figure 7] FIG. 2 is a diagram for explaining a translation function of the speech processing device according to the embodiment of the present invention. [Figure 8] FIG. 2 is a diagram for explaining a translation function of the speech processing device according to the embodiment of the present invention. [Figure 9] 4 is a flowchart illustrating a method for providing a translation result by a speech processing device according to an embodiment of the present invention. [Figure 10] FIG. 2 is a diagram for explaining the operation of the voice processing device according to the embodiment of the present invention. [Figure 11] FIG. 2 is a diagram for explaining the operation of the voice processing device according to the embodiment of the present invention. [Figure 12] FIG. 2 is a diagram illustrating the operation of a sound processing device according to an embodiment of the present invention. [Figure 13] FIG. 1 illustrates a voice processing device according to an embodiment of the present invention. [Figure 14] FIG. 13 is a diagram for explaining a speaker movement mode according to an embodiment of the present invention. [Figure 15] FIG. 13 is a diagram for explaining a speaker movement mode according to an embodiment of the present invention. [Figure 16] FIG. 1 illustrates a voice processing device according to an embodiment of the present invention. [Figure 17] FIG. 2 is a diagram illustrating the operation of a sound processing device according to an embodiment of the present invention. [Figure 18] FIG. 2 is a diagram illustrating the operation of a sound processing device according to an embodiment of the present invention. [Figure 19] 4 is a flowchart illustrating a method of operating an audio processing device according to an embodiment of the present invention. [Figure 20] FIG. 1 illustrates a voice processing device according to an embodiment of the present invention. [Figure 21] FIG. 1 illustrates a voice processing device according to an embodiment of the present invention. [Figure 22] FIG. 2 is a diagram for explaining the operation of the voice processing device according to the embodiment of the present invention. [Figure 23] FIG. 2 is a diagram for explaining the operation of the voice processing device according to the embodiment of the present invention. [Figure 24] 4 is a flowchart illustrating a method of operating an audio processing device according to an embodiment of the present invention. [Diagram 25] FIG. 2 is a diagram for explaining the operation of the voice processing device according to the embodiment of the present invention. [Figure 26] FIG. 2 is a diagram illustrating the operation of a sound processing device according to an embodiment of the present invention. [Figure 27] FIG. 2 is a diagram for explaining the operation of the voice processing device according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Fig. 1 is a diagram showing a voice processing environment according to an embodiment of the present invention. Referring to Fig. 1, speakers SPK1 to SPK4 can be located in a space (e.g., a conference room, a vehicle, a lecture hall, etc.) and can pronounce voices. In the embodiment, a first speaker SPK1 can pronounce voice at a first position P1, a second speaker SPK2 can pronounce voice at a second position P2, a third speaker SPK3 can pronounce voice at a third position P3, and a fourth speaker SPK4 can pronounce voice at a fourth position P4.

[0008] The voice processing device 100 may be an electronic device having a computing function. For example, the voice processing device 100 may be a smartphone, a laptop, a personal digital assistance (PDA), a wearable device, a smart watch, or a tablet computer, but the embodiment of the present invention is not limited thereto.

[0009] The voice processing device 100 processes the voice signals associated with the voices of the speakers SPK1 to SPK4, thereby making it possible to perform voice processing on the voices of the speakers SPK1 to SPK4. The voice processing device 100 can generate voice signals associated with the voices of the speakers SPK1 to SPK4 in response to the voices of the speakers SPK1 to SPK4. The voice signals are signals associated with voices uttered during a specific period of time, and may be signals indicating the voices of multiple speakers.

[0010] In an embodiment, the voice processing device 100 uses voice signals associated with the voices of speakers SPK1 to SPK4 to determine the sound source position of each of the voices of speakers SPK1 to SPK4, and performs sound source separation based on the sound source position, thereby extracting (or generating) separated voice signals associated with the voices of each of speakers SPK1 to SPK4 from the voice signals. That is, the voice processing device 100 can generate separated voice signals associated with the voices of the speakers SPK1 to SPK4 located at the positions P1 to P4 based on the sound source positions of the voices corresponding to the voice signals. For example, the voice processing device 100 can generate a first separated voice signal associated with the voice of the first speaker SPK1 who spoke at the first position P1 based on the voice signals. At this time, the first separated voice signal may be a voice signal having the highest degree of association with the voice of the first speaker SPK1 among the voices of the speakers SPK1 to SPK4. In other words, the voice components of the first speaker SPK1 have the highest weight among the voice components included in the first separated voice signal.

[0011] Furthermore, the speech processing device 100 can provide translations for the speech of each of the speakers SPK1 to SPK4. For example, the speech processing device 100 can determine a source language (a language to be translated) and a target language (a language after translation) for translating the speech of each of the speakers SPK1 to SPK4, and provide translations for each language of the speakers using the separated speech signals.

[0012] In an embodiment, the speech processing device 100 can output a translation result for each speech. The translation result may be text data or a speech signal associated with the speech of each of the speakers SPK1 to SPK4 expressed in the arrival language. In other words, the speech processing device 100 according to an embodiment of the present invention determines the departure language and the destination language according to the sound source position of each of the speakers SPK1 to SPK4, and has the advantage of being able to provide translation for the speaker's speech with little time and few resources, without the need to identify the language of the speaker's speech.

[0013] 2 shows a voice processing device according to an embodiment of the present invention. Referring to FIG. 2, the voice processing device 100 may include a microphone 110, a communication circuit 120, a processor 130, and a memory 140. In an embodiment, the voice processing device 100 may further include a speaker 150.

[0014] The microphone 110 can generate an audio signal in response to generated sound. In an embodiment, the microphone 110 can detect vibrations in the air caused by sound and generate an audio signal, which is an electrical signal corresponding to the vibrations, in accordance with the detection result.

[0015] In an embodiment, there may be a plurality of microphones 110, and each of the plurality of microphones 110 may generate an audio signal in response to a voice. In this case, since the positions at which the plurality of microphones 110 are arranged may be different from each other, the audio signals generated from each of the microphones 110 may have a phase difference (or a time delay) from each other.

[0016] For example, the microphone 110 can receive the voices of the speakers SPK1 to SPK4 located at the positions P1 to P4 and convert the voices of the speakers SPK1 to SPK4 into audio signals, which are electrical signals. The communication circuit 120 can exchange data with an external device by a wireless communication method. In the embodiment, the communication circuit 120 can exchange data with an external device by using radio waves of various frequencies. For example, the communication circuit 120 can exchange data with an external device by at least one wireless communication method of short-range wireless communication, medium-range wireless communication, and long-range wireless communication.

[0017] The processor 130 can control the overall operation of the audio processing device 100. In an embodiment, the processor 130 can include a processor having a calculation function. For example, the processor 130 can include, but is not limited to, a central processing unit (CPU), a micro controller unit (MCU), a graphics processing unit (GPU), a digital signal processor (DSP), an analog to digital converter (ADC converter), or a digital to analog converter (DAC converter).

[0018] The processor 130 can process the audio signal generated by the microphone 110. For example, the processor 130 can convert the analog type audio signal generated by the microphone 110 into a digital type audio signal and process the converted digital type audio signal. In this case, since the type of the signal (analog or digital) changes, the digital type audio signal and the analog type audio signal are used together in the description of the embodiment of the present invention.

[0019] In an embodiment, the processor 130 can extract (or generate) a separated audio signal associated with the voice of each of the speakers SPK1 to SPK4 using the audio signal generated by the microphone 110. In an embodiment, the processor 130 can generate a separated audio signal associated with the voice of the speakers SPK1 to SPK4 located at each of the positions P1 to P4.

[0020] The processor 130 can determine the location of the sound source (i.e., the location of the speakers SPK1 to SPK4) of the sound using the time delay (or phase delay) between the sound signals. For example, the processor 130 can determine the relative location of the sound source (i.e., the speakers SPK1 to SPK4) with respect to the sound processing device 100.

[0021] The processor 130 can generate a separated voice signal associated with the voice of each of the speakers SPK1 to SPK4 based on the determined sound source position. For example, the processor 130 can generate a first separated voice signal associated with the voice of the first speaker SPK1 based on the sound source position of the voice.

[0022] In an embodiment, the processor 130 can match the sound source position information indicating the determined sound source position with the separated voice signal and store it. For example, the processor 130 can match the first separated voice signal associated with the voice of the first speaker SPK1 with the first sound source position information indicating the sound source position of the voice of the first speaker SPK1 and store it in the memory 140. That is, since the positions of the sound sources correspond to the positions of the speakers SPK1 to SPK4, the sound source position information can also function as speaker position information for identifying the positions of the speakers SPK1 to SPK4.

[0023] The operations of the processor 130 or the audio processing device 100 described herein may be implemented in the form of a program executable by a computing device. For example, the processor 130 may execute an application stored in the memory 140 and perform an operation corresponding to an instruction command that instructs a particular operation by executing the application.

[0024] The memory 140 can store data necessary for the operation of the audio processing device 100. For example, the memory 140 can include at least one of a non-volatile memory and a volatile memory.

[0025] In an embodiment, the memory 140 can store identifiers corresponding to the respective positions P1 to P4 in space. The identifiers may be data for distinguishing the positions P1 to P4. Since the speakers SPK1 to SPK4 are located at the positions P1 to P4, respectively, the speakers SPK1 to SPK4 can be distinguished from each other using the identifiers corresponding to the positions P1 to P4. For example, the first identifier indicating the first position P1 can indicate the first speaker SPK1. From this perspective, the identifiers corresponding to the respective positions P1 to P4 in space can also function as speaker identifiers for identifying the speakers SPK1 to SPK4, respectively. The identifier is input via an input device of the audio processing device 100 (eg, a touchpad).

[0026] In an embodiment, the memory 140 may store sound source position information associated with the position of each of the speakers SPK1-SPK4 and separated speech signals associated with the speech of each of the speakers SPK1-SPK4.

[0027] Speaker 150 may vibrate under the control of processor 130, the vibrations generating sound. In an embodiment, speaker 150 may reproduce a sound associated with an audio signal by producing vibrations corresponding to the audio signal.

[0028] Meanwhile, in this specification, it is described that the voice processing device 100 includes the microphone 110 and directly generates voice signals associated with the voices of the speakers SPK1 to SPK4 using the microphone 110, but in an embodiment, the microphone is configured externally and separate from the voice processing device 100, and the voice processing device 100 can receive voice signals from the separate microphone and process or use the received voice signals. For example, the voice processing device 100 can generate a separated voice signal from the voice signal received from the separate microphone. However, for ease of explanation, the following description will be given assuming that the audio processing device 100 includes a microphone 110 unless otherwise specified.

[0029] 3 to 5 are diagrams for explaining the operation of the speech processing device according to the embodiment of the present invention. Referring to Fig. 3 to Fig. 5, speakers SPK1 to SPK4 located at positions P1 to P4 can each speak. A voice processing device 100 according to an embodiment of the present invention can generate separated voice signals associated with the voice of each speaker SPK1 to SPK4 from the voices of the speakers SPK1 to SPK4, and can store the separated voice signals and position information indicating the position of each of the speakers SPK1 to SPK4.

[0030] In an embodiment, the speech processing device 100 can determine the position of the speech source (i.e., the position of the speakers SPK1 to SPK4) using the time delay (or phase delay) between the speech signals. For example, the speech processing device 100 can determine the relative position of the speech source (i.e., the speakers SPK1 to SPK4) with respect to the speech processing device 100.

[0031] The voice processing device 100 can generate separated voice signals associated with the voices of the speakers SPK1 to SPK4, based on the determined sound source positions.

[0032] As shown in Fig. 3, the first speaker SPK1 speaks the voice "AAA". When the voice "AAA" is spoken, the voice processing device 100 can generate a voice signal associated with the voice "AAA" in response to the voice "AAA". In the embodiment, the voice signal associated with the voice "AAA" also includes components related to noise other than the voice "AAA".

[0033] In the embodiment, the voice processing device 100 can generate a separated voice signal associated with the voice "AAA" of the first speaker SPK1 using the generated voice signal. At this time, the voice processing device 100 can store the first separated voice signal associated with the voice "AAA" of the first speaker SPK1 and first sound source position information indicating a first position P1 which is the position of the first speaker SPK1 in the memory 140. For example, as shown in Fig. 3, the first separated voice signal and the first sound source position information are matched with each other and stored.

[0034] 4, the second speaker SPK2 speaks the voice "BBB." When the voice "BBB" is spoken, the voice processing device 100 can generate a voice signal associated with the voice "BBB" in response to the voice "BBB."

[0035] In the embodiment, the voice processing device 100 can generate a second separated voice signal associated with the voice "BBB" of the second speaker SPK2 using the generated voice signal. At this time, the voice processing device 100 can store the second separated voice signal associated with the voice "BBB" of the second speaker SPK2 and second sound source position information indicating a second position P2 that is the position of the second speaker SPK2 in the memory 140. For example, as shown in FIG. 4, the second separated voice signal and the second sound source position information are matched with each other and stored.

[0036] 5, a third speaker SPK3 speaks a voice "CCC" and a fourth speaker SPK4 speaks a voice "DDD." In response to the voice "CCC" and the voice "DDD," the voice processing device 100 can generate voice signals associated with the voices "CCC" and "DDD." That is, the voice signal is a voice signal including components associated with the voices "CCC" and "DDD." In an embodiment, the voice processing device 100 can use the generated voice signal to generate a third separated voice signal associated with the voice "CCC" of the third speaker SPK3, and a fourth separated voice signal associated with the voice "DDD" of the fourth speaker SPK4.

[0037] At this time, the voice processing device 100 can store in the memory 140 a third separated voice signal associated with the voice "CCC" of the third speaker SPK3 and third position information indicating a third position P3 which is the position of the third speaker SPK3. In addition, the voice processing device 100 can store in the memory 140 a fourth separated voice signal associated with the voice "DDD" of the fourth speaker SPK4 and fourth position information indicating a fourth position P4 which is the position of the fourth speaker SPK4.

[0038] For example, as shown in FIG. 5, the third separated sound signal and the third sound source position information are matched with each other and stored, and the fourth separated sound signal and the fourth sound source position information are matched with each other and stored.

[0039] In other words, the speech processing device 100 according to an embodiment of the present invention can generate separated speech signals associated with the speech of each speaker SPK1 to SPK4 from the speech of the speakers SPK1 to SPK4, and can store the separated speech signals and position information indicating the position of each speaker SPK1 to SPK4.

[0040] 6 is a flowchart showing a voice separation method by a voice processing device according to an embodiment of the present invention. The operation method of the voice processing device described with reference to FIG. 6 is realized as an application (e.g., a voice separation application) stored in a non-transitory storage medium and executable by a computing device. For example, the processor 130 can execute the application stored in the memory 140 and perform an operation corresponding to an instruction word that instructs a specific operation by executing the application.

[0041] 6, the audio processing device 100 can generate an audio signal in response to an audio (S110). In an embodiment, the audio processing device 100 can convert an audio detected in a space into an audio signal, which is an electrical signal.

[0042] The voice processing device 100 can determine the sound source position for each of the voices (i.e., the position of the speakers SPK1 to SPK4) by using the voice signal associated with the voices of the speakers SPK1 to SPK4 (S120). In the embodiment, the voice processing device 100 can generate sound source position information indicating the sound source position for each of the voices of the speakers SPK1 to SPK4 (i.e., the position of the speakers SPK1 to SPK4).

[0043] The speech processing device 100 can generate separated speech signals associated with the speech of each of the speakers SPK1 to SPK4 based on the sound source position for each of the speeches (S130). In the embodiment, the speech processing device 100 can generate separated speech signals associated with the speech of each of the speakers SPK1 to SPK4 by separating the generated speech signals based on the sound source position for each of the speeches. For example, the speech processing device 100 can generate separated speech signals associated with the speech of each of the speakers SPK1 to SPK4 by separating components included in the speech signals based on the sound source position.

[0044] The voice processing device 100 can store sound source position information indicating the position of the sound source and the separated sound signals (S140). In the embodiment, the voice processing device 100 can match and store the sound source position information indicating the position of the sound source and the separated sound signals associated with the voices of the speakers SPK1 to SPK4. For example, the voice processing device 100 can match and store data corresponding to the separated sound signals associated with the voices of the speakers SPK1 to SPK4 and the sound source position information.

[0045] In an embodiment, the voice processing device 100 (or the processor 130) according to an embodiment of the present invention can generate (or separate) separated voice signals associated with the voices of each of the speakers SPK1 to SPK4 from the voice signals associated with the voices of the speakers SPK1 to SPK4 by executing an application (e.g., a voice separation application) stored in the memory 140.

[0046] Generally, to process a voice signal, hardware such as a microphone and a processor configured to process the voice signal is required. On the other hand, since a mobile terminal such as a smartphone basically includes a speaker and a processor, a user can perform a method according to an embodiment of the present invention using the voice processing device 100 to separate the voice of a speaker without having to provide additional hardware. For example, the processor 130 of the voice processing device 100 can execute a voice separation application and perform voice separation using hardware (e.g., a speaker) included in the voice processing device 100.

[0047] 7 is a diagram for explaining a translation function of a speech processing device according to an embodiment of the present invention. Referring to FIG. 7, a first speaker SPK1 speaks the voice "AAA" in Korean (KR), a second speaker SPK2 speaks the voice "BBB" in English (EN), a third speaker SPK3 speaks the voice "CCC" in Chinese (CN), and a fourth speaker SPK4 speaks the voice "DDD" in Japanese (JP).

[0048] The speech processing device 100 according to the embodiment of the present invention can generate separated speech signals associated with the speech of each of the speakers SPK1 to SPK4 from the speech of each of the speakers SPK1 to SPK4, and can provide a translation for the speech of each of the speakers SPK1 to SPK4 using the separated speech signals. At this time, the speech processing device 100 can determine the starting language of the speech of each of the speakers SPK1 to SPK4 using starting language information corresponding to the position of each of the speakers SPK1 to SPK4, and can provide a translation for the speech.

[0049] 7, the speech processing device 100 can store separated speech signals associated with the speech of each of the speakers SPK1 to SPK4, sound source position information indicating the positions of the speakers SPK1 to SPK4, and starting language information indicating the starting language of the speech of the speakers SPK1 to SPK4. At this time, the starting language is determined in advance for each sound source position and stored.

[0050] For example, the voice processing device 100 can store in the memory 140 first starting language information indicating that the starting language corresponding to the first position P1 is "KR". In addition, the voice processing device 100 can store in the memory 140 a first separated voice signal associated with the voice "AAA" of the first speaker SPK1, first sound source position information indicating the first position P1 which is the position of the first speaker SPK1, and first starting language information indicating "KR" which is the starting language of the voice "AAA(KR)" of the first speaker SPK1.

[0051] In an embodiment, when speakers SPK1 to SPK4 speak, the voice processing device 100 can generate separated voice signals associated with the voices of each of the speakers SPK1 to SPK4 and sound source position information indicating the positions of the speakers SPK1 to SPK4 in response to the voices of the speakers SPK1 to SPK4.

[0052] The speech processing device 100 can use the start language information to determine the start language corresponding to each separated speech signal, and provide a translation for the speech of the speakers SPK1 to SPK4 based on the determined start language. In an embodiment, the speech processing device 100 can use the sound source position information corresponding to each separated speech signal to determine the start language corresponding to the sound source position of each speech, and generate a translation result for the separated speech signal based on the determined start language.

[0053] For example, the speech processing device 100 can convert the separated speech signals into text data (e.g., Speech-To-Text (STT) conversion), generate a translation result from the converted text data from the departure language to the arrival language, and convert the translation result into a speech signal (e.g., Text-to-Speech (TTS) conversion). That is, the translation result referred to in this specification can mean all of the text data or speech signals associated with the speech of each of the speakers SPK1 to SPK4 expressed in the arrival language.

[0054] In an embodiment, the speech processing device 100 may output the generated translation result. For example, the speech processing device 100 may output the generated translation result via a speaker 150 or transmit it to another external device.

[0055] Fig. 8 is a diagram for explaining a translation function of a speech processing device according to an embodiment of the present invention. Referring to Fig. 8, the speech processing device 100 can generate separated speech signals associated with the speech of each of the speakers SPK1 to SPK4, and output a translation result for the speech of each of the speakers SPK1 to SPK4 using the separated speech signals. At this time, the translation result indicates the result of converting the language of the speech of the speakers SPK1 to SPK4 from the source language to another language (e.g., the destination language).

[0056] As shown in Fig. 8, the first speaker SPK1 speaks the voice "AAA" in Korean (KR), and the second speaker SPK2 speaks the voice "BBB" in English (EN). In this case, the starting language of the voice "AAA" of the first speaker SPK1 is Korean (KR), and the starting language of the voice "BBB" of the second speaker SPK2 is English (EN).

[0057] The voice processing device 100 can determine a sound source position (e.g., P1) of the first speaker SPK1 in response to the voice "AAA (KR)" of the first speaker SPK1, and generate a first separated voice signal associated with the voice "AAA (KR)" of the first speaker SPK1 based on the sound source position. Similarly, the voice processing device 100 can determine a sound source position (e.g., P2) of the second speaker SPK2 in response to the voice "BBB (EN)" of the second speaker SPK2, and generate a second separated voice signal associated with the voice "BBB (EN)" of the second speaker SPK2 based on the sound source position.

[0058] The speech processing device 100 can provide a translation from the start language to the destination language for the language of the speech of the speakers SPK1 to SPK4 using the generated separated speech signals. According to the embodiment, the speech processing device 100 can determine a start language that is determined according to the sound source position of the speech of the speakers SPK1 to SPK4 using the start language information stored in the memory 140, and output a translation result from the start language to the destination language for the language of the speech of each of the speakers SPK1 to SPK4 according to the determined start language.

[0059] In the embodiment, the speech processing device 100 can store arrival language information indicating the arrival language for each position, and can determine the arrival language corresponding to the sound source position of the speech of each of the speakers SPK1 to SPK4 using the stored arrival language information. Also, in the embodiment, the speech processing device 100 can determine the arrival language for the speech of each of the speakers SPK1 to SPK4 based on an input from a user.

[0060] For example, the voice processing device 100 can use first sound source position information indicating a first position P1 that is the sound source position of the voice "AAA (KR)" of the first speaker SPK1 to read out first start language information corresponding to the first position P1 from the memory 140. The read out first start language information can indicate that the start language of the voice "AAA" of the first speaker SPK1 is Korean (KR).

[0061] The translation result may be output via speaker 150, stored in memory 140, or transmitted to an external device via communication circuitry 120.

[0062] In this specification, the translation result output by the speech processing device 100 may be, but is not limited to, text data expressed in the destination language or a speech signal associated with speech spoken in the destination language.

[0063] In this specification, when the speech processing device 100 generates a translation result, it includes not only generating a translation result by translating a language through calculations by the processor 130 of the speech processing device 100 itself, but also generating a translation result by the speech processing device 100 communicating with a server having a translation function and receiving the translation result from the server.

[0064] For example, the processor 130 can execute a translation application stored in the memory 140 to generate translation results for the speech of each of the speakers SPK1 to SPK4.

[0065] For example, the voice processing device 100 may transmit the separated voice signal, the starting language information, and the destination language information to a translator and receive a translation result for the separated voice signal from the translator. The translator may refer to an environment or system that provides translation for a language. In the embodiment, the translator may output a translation result for each of the voices of the speakers SPK1 to SPK4 using the separated voice signal, the starting language information, and the destination language information.

[0066] For example, as shown in FIG. 8, the speech processing device 100 can determine the starting language (i.e., Korean (KR)) and the destination language (i.e., English (EN)) for the speech "AAA (KR)" of the first speaker SPK1, and output the translation result for the speech "AAA (KR)" of the first speaker SPK1 according to the determined starting language and destination language. For example, the translation result for the speech "AAA (KR)" may be data (e.g., speech data or text data, etc.) associated with the speech "AAA (EN)" expressed in English (EN). Meanwhile, while FIG. 8 describes that the destination language for the speech "AAA (KR)" is English (EN), the embodiment of the present invention is not limited thereto.

[0067] As described above, the speech processing device 100 provides translation based on separated speech signals associated with the speech of speakers SPK1 to SPK4, and therefore has the advantage of being able to output a translation result for the speech of a specific speaker.

[0068] Similarly, the speech processing device 100 can determine the starting language (i.e., English (EN)) and the destination language (i.e., Korean (KR)) for the speech "BBB (EN)" of the second speaker SPK2, and output the translation result for the speech "BBB (EN)" of the second speaker SPK2 according to the determined starting language and destination language. The speech processing device 100 can also output the translation result for the speech "CCC (CN)" of the third speaker SPK3 and the speech "DDD (CN)" of the fourth speaker SPK4.

[0069] 9 is a flowchart illustrating a method for providing a translation result by a speech processing device according to an embodiment of the present invention. The operation method of the speech processing device described with reference to FIG. 9 is realized as an application (e.g., a translation application) stored in a non-transitory storage medium and executable by a computing device. For example, the processor 130 may execute the application stored in the memory 140 and perform an operation corresponding to a command instructing a particular operation by executing the application.

[0070] 9, the voice processing device 100 can generate separated voice signals associated with the voices of the speakers SPK1 to SPK4 (S210). In the embodiment, the voice processing device 100 can generate the separated voice signals by separating the voice signals generated in response to the voices based on the sound source positions of the voices.

[0071] The speech processing device 100 can determine a starting language for translating the speech of each of the speakers SPK1 to SPK4 (S220). In an embodiment, the speech processing device 100 can determine a starting language for translating the speech of each of the speakers SPK1 to SPK4 based on the voice source position of the speech of each of the speakers SPK1 to SPK4. Also, the speech processing device 100 can determine a destination language into which the speech of each of the speakers SPK1 to SPK4 is translated based on the voice source position of the speech of each of the speakers SPK1 to SPK4.

[0072] The speech processing device 100 can output a translation result for the speech of each of the speakers SPK1 to SPK4 according to the departure language using the separated speech signal (S230). In the embodiment, the speech processing device 100 can output a translation result from the departure language to the arrival language for the speech of each of the speakers SPK1 to SPK4 based on the determined departure language (and arrival language).

[0073] A voice processing device 100 according to an embodiment of the present invention generates voice signals associated with the voices of speakers SPK1 to SPK4, and processes the voice signals to generate separated voice signals associated with the voices of each of the speakers SPK1 to SPK4.

[0074] Furthermore, the speech processing device 100 according to the embodiment of the present invention can translate the speech of the speakers SPK1 to SPK4 using the separated speech signals and output the translation results. This has the effect that even if the speakers SPK1 to SPK4 use different languages, each of the speakers SPK1 to SPK4 can speak in his / her own language, and the speech of a speaker who speaks another language can be translated and heard in his / her own language.

[0075] Generally, in order to process a voice signal, hardware such as a microphone and a processor configured to process the voice signal is required. On the other hand, since a mobile terminal such as a smartphone basically includes a speaker and a processor, if the voice processing device 100 is implemented in a mobile terminal such as a smartphone, a user can perform the method according to the embodiment of the present invention using the voice processing device 100 to separate the voice of the speaker without having to provide additional hardware, and can provide a translation for the voice using these.

[0076] 10 and 11 are diagrams for explaining the operation of the voice processing device according to the embodiment of the present invention. Referring to FIG. 10 and FIG. 11, the voice processing device 100 can operate in a position registration mode (or a speaker registration mode). The position registration mode means a mode in which the sound source positions of the voices of the speakers SPK1 to SPK4 are stored as reference sound source positions in the voice processing device 100. Thereafter, the voice processing device 100 can identify the speakers SPK1 to SPK4 using the stored reference sound source positions to generate separated voice signals, or selectively process only separated voice signals associated with voices spoken at specific positions.

[0077] The processor 130 can operate in the location registration mode in response to an input from the outside. In the embodiment, the processor 130 can operate in the location registration mode in response to a voice signal including a specific phrase, or in response to an input via an input unit (e.g., a button or a touch panel) formed in the voice processing device 100.

[0078] The voice processing device 100 can determine the voice source position for each of the voices of the speakers SPK1 to SPK4 in response to the voices of the speakers SPK1 to SPK4, and generate voice source position information indicating the voice source position.

[0079] In the position registration mode, the voice processing device 100 can store the generated sound source position information in the memory 140 as reference sound source position information.

[0080] For example, as shown in Fig. 10, in the position registration mode, if a first speaker SPK1 utters "I'm Alice," the voice processing device 100 can generate a voice signal in response to the voice of the first speaker SPK1 and determine a first position P1, which is the position of the first speaker SPK1, from the voice signal. The voice processing device 100 can generate first position information indicating the first position P1 and store the first sound source position information as reference sound source position information.

[0081] Similarly, for example, as shown in Fig. 11, the speech processing device 100 can determine the sound source positions P2 to P4 of the speech of the remaining speakers SPK2 to SPK4 in response to the speech of the remaining speakers SPK2 to SPK3. On the other hand, the speech processing device 100 according to the embodiment of the present invention can calculate the sound source position of each of the speeches of the speakers SPK1 to SPK4 even if the speeches of the speakers SPK1 to SPK4 are uttered with overlapping in time.

[0082] The voice processing device 100 can generate second sound source position information indicating a second position P2 which is the position of the second speaker SPK2 and store the second sound source position information as reference sound source position information, generate third sound source position information indicating a third position P3 which is the position of the third speaker SPK3 and store the third sound source position information as reference sound source position information, and generate fourth sound source position information indicating a fourth position P4 which is the position of the fourth speaker SPK4 and store the fourth sound source position information as reference sound source position information.

[0083] In an embodiment, the voice processing device 100 can store sound source position information and an identifier corresponding to the sound source position information. The identifier is data for distinguishing the sound source position, and may be, for example, data (e.g., a name) indicating a speaker located at the corresponding sound source position.

[0084] For example, as shown in FIG. 10, the voice processing device 100 may generate a first identifier SID1 indicating the first speaker SPK1 in response to the voice of the first speaker SPK1, and may match and store the generated first identifier SID1 together with the first sound source position information. That is, the first identifier SID1 may be a means for identifying the first speaker SPK1. For example, the voice processing device 100 may convert at least a part of the voice of the first speaker SPK1 into text, and generate a first identifier SID1 corresponding to the converted text. For example, the voice processing device 100 may convert at least a part of a wording included in the voice of the first speaker SPK1 as the first identifier SID1.

[0085] For example, as shown in FIG. 11, the voice processing device 100 can generate identifiers SID2 to SID4 indicating the remaining speakers SPK2 to SPK4 in response to the voices of the remaining speakers SPK2 to SPK4, and match and store the generated identifiers SID2 to SID4 together with the sound source position information of the speakers SPK2 to SPK4.

[0086] 12 is a diagram illustrating the operation of the audio processing device according to an embodiment of the present invention. Referring to FIG 12, the audio processing device 100 can operate in an audio separation mode.

[0087] In an embodiment, the processor 130 may operate in the voice separation mode in response to an external input. In an embodiment, the processor 130 may operate in the voice separation mode in response to a voice signal including a specific phrase, or in response to an input via an input unit (e.g., a button or a touch panel) formed in the voice processing device 100.

[0088] In the voice separation mode, the voice processing device 100 can generate separated voice signals related to the voices of the speakers SPK1 to SPK4 by separating the voice signals associated with the voices of the speakers SPK1 to SPK4 based on the sound source positions of the voices, and store the generated separated voice signals.

[0089] In an embodiment, the audio processing device 100 can store a separated audio signal associated with audio corresponding to a sound source position corresponding to a reference sound source position stored (or registered) in advance. For example, the audio processing device 100 can store a separated audio signal associated with audio corresponding to a sound source position within a reference range from the reference sound source position, among the separated audio signals separated from the audio signal.

[0090] In an embodiment, in the voice separation mode, the processor 130 may operate in a location registration mode if the sound source position of the voice to be recognized does not correspond to a pre-stored (or registered) reference sound source position. For example, the processor 130 may operate in the location registration mode if the sound source position of the voice to be recognized differs from the pre-stored reference sound source position, and thus may register a new sound source position.

[0091] The voice processing device 100 can match the separated voice signals associated with the voices of the speakers SPK1 to SPK4 with the corresponding identifiers and store them. For example, as shown in Fig. 12, the voice processing device 100 can match the first separated voice signal with the first identifier SID1 according to the first sound source position information indicating the first position P1 which is the position of the first speaker SPK1 and store them. For example, the voice processing device 100 can refer to the reference sound source position information stored in the memory 140 and match the first identifier SID1 corresponding to the first sound source position information with the first separated voice signal and store them.

[0092] In addition, the voice processing device 100 may additionally match and store the separated voice signal with a time point at which the voice corresponding to the separated voice signal was received.

[0093] 13 shows a voice processing device according to an embodiment of the present invention. Referring to FIG. 13, the voice processing device 100A may include a microphone 110, a communication circuit 120, a processor 130, a memory 140, and a trigger signal generating circuit 151.

[0094] 2, the sound processing device 100A in Fig. 13 is different in that it additionally includes a trigger signal generating circuit 151. For the sake of convenience, the following description will be limited to the differences and overlapping parts will be omitted.

[0095] The trigger signal generating circuit 151 can generate a trigger signal in response to an external input. The trigger signal may be a signal that causes the processor 130 to perform a specific operation. The trigger signal may include a speaker enrollment trigger signal and a speaker movement trigger signal. In this case, the input conditions for generating the speaker enrollment trigger signal and the speaker movement trigger signal may be different.

[0096] In an embodiment, the trigger signal generating circuit 151 includes an input unit capable of detecting an external physical input such as a touch panel or a button, and can generate a trigger signal in response to the physical input. For example, the trigger signal generating circuit 151 can generate a trigger signal when a user's touch is detected.

[0097] In an embodiment, the trigger signal generating circuit 151 can generate a trigger signal by recognizing an activation language included in a voice signal received by the voice processing device 100A. For example, the trigger signal generating circuit 151 can generate a trigger signal when a voice signal including a specific phrase such as "speaker registration" is received. The trigger signal generating circuit 151 can transmit the generated trigger signal to the processor 130 .

[0098] In an embodiment, the processor 130 may proceed to a speaker enrollment mode (or a location enrollment mode) in response to a speaker identification trigger signal. In an embodiment, the speaker enrollment mode is defined as a predetermined period from the time when the speaker enrollment trigger signal is received, or as a period during which the speaker enrollment trigger signal is received, but is not limited thereto.

[0099] As described with reference to Figures 10 and 11, in the speaker enrollment mode, the voice processing device can generate reference sound source position information and an identifier using a received voice signal, generate a separated voice signal, and match the reference sound source position information, the identifier and the separated voice signal with each other and store them.

[0100] 14 and 15 are diagrams for explaining the speaker movement mode according to an embodiment of the present invention. The speaker movement mode explained with reference to Fig. 14 and Fig. 15 is performed by the voice processing device 100A of Fig. 13.

[0101] 14 and 15, Alice's position before movement is "P1", and after the speaker registration mode, an identifier SID for identifying Alice is stored in the memory 140 of the voice processing device 100A, and Alice's position "P1" is stored as the reference sound source position.

[0102] After the movement, Alice utters the voice "I am Alice" at position P5. In the speaker movement mode, the voice processing device 100A can newly generate sound source position information indicating Alice's position "P5" after the movement by using a voice signal associated with Alice's voice.

[0103] The voice processing device 100A can update the reference sound source position information stored by matching with the speaker identifier SID with reference to the memory 140. For example, the voice processing device 100A can update the reference sound source position information "P1" already matched with the speaker identifier SID "Alice" and stored to "P5" which is the reference sound source position information for the position after movement.

[0104] As a result, the voice processing device 100A according to an embodiment of the present invention has the advantage that even if the speaker's position changes due to the speaker's movement, the speaker position information stored by matching with the speaker identifier can be updated to the changed speaker position information.

[0105] 16 shows a sound processing device according to an embodiment of the present invention. Referring to FIG. 16, the sound processing device 100B may include a microphone 110, a communication circuit 120, a processor 130, a memory 140, and a motion sensor 153.

[0106] 2, the sound processing device 100B in Fig. 16 is different in that it additionally includes a motion sensor 153. For the sake of convenience, the following description will be limited to the differences and overlapping parts will be omitted.

[0107] The motion sensor 153 can measure a physical quantity related to the audio processing device 100B and generate a detection signal corresponding to the measured physical quantity. For example, the motion sensor 153 can measure a position or movement of the audio processing device 100B and generate and output a detection signal corresponding to the measured position or movement.

[0108] In an embodiment, the motion sensor 153 can measure the position of the audio processing device 100B and output a detection signal indicating the position of the audio processing device 100B. For example, the motion sensor 153 may be a GPS sensor, a LIDAR (Light Detection And Ranging) sensor, a Radar (Radio Detection And Ranging) sensor, or a UWB (Ultra Wide Band) sensor, but the embodiment of the present invention is not limited thereto.

[0109] In an embodiment, the motion sensor 153 can measure the movement of the audio processing device 100B and output a detection signal indicating the movement of the audio processing device 100B. For example, the motion sensor 153 can be a gyro sensor, a speed sensor, or an acceleration sensor, but the embodiment of the present invention is not limited thereto.

[0110] Meanwhile, in this specification, a configuration is described in which the motion sensor 153 measures the position or movement of the audio processing device 100B, but in an embodiment, the position or movement of the audio processing device 100B can also be measured by the processor 130 and the motion sensor 153. For example, the motion sensor 153 can generate and output a signal related to the position or movement of the audio processing device 100B, and the processor 130 can generate a value related to the position or movement of the audio processing device 100B based on the signal output from the motion sensor 153.

[0111] 17 and 18 are diagrams showing the operation of the audio processing device according to the embodiment of the present invention. The operation described with reference to Fig. 17 and Fig. 18 is performed by the audio processing device 100B described with reference to Fig. 16.

[0112] 17 and 18, when the movement of the voice processing device 100B is detected, the voice processing device 100B can store the changed voice source position for the voices of the speakers SPK1 to SPK4 as reference voice source position information.

[0113] 17, when the position of the speech processing device 100B changes due to the movement of the speech processing device 100B, the relative positions of the speakers SPK1 to SPK4 with respect to the speech processing device 100B may change. Furthermore, even if the position of the speech processing device 100B does not change, when the speech processing device 100B moves (rotates, vibrates, moves, etc.), the relative positions of the speakers SPK1 to SPK4 with respect to the speech processing device 100B may change. In other words, the sound source positions of the speech of the speakers SPK1 to SPK4 may change.

[0114] For example, the position of the first speaker SPK1 can vary from P1 to P5, the position of the second speaker SPK2 can vary from P2 to P6, the position of the third speaker SPK3 can vary from P3 to P7, and the position of the fourth speaker SPK4 can vary from P4 to P8.

[0115] The voice processing device 100B according to the embodiment of the present invention includes a motion sensor 153 capable of detecting a movement of the voice processing device 100B, and can detect a change in position of the voice processing device 100B through a detection result of the motion sensor 153. In addition, the voice processing device 100B has an effect of determining a sound source position changed due to the movement of the voice processing device 100B, and storing the changed sound source position as reference sound source position information.

[0116] 18, the audio processing device 100B can operate in the location registration mode if the motion of the audio processing device 100B is detected. In the embodiment, the processor 130 can detect the motion of the audio processing device 100B using the detection result of the motion sensor 153, and can determine whether to operate in the location registration mode.

[0117] In other words, even after the sound source positions of each of the speakers SPK1 to SPK4 (i.e., the positions of the speakers SPK1 to SPK4) have been registered as reference sound source position information in the position registration mode, the voice processing device 100B can operate in the position registration mode again if movement of the voice processing device 100B is detected.

[0118] 18, in the position update mode, if the first speaker SPK1 speaks "I'm Alice" at the changed position "P5", the voice processing device 100B generates a voice signal in response to the voice of the first speaker SPK1 and can determine "P5", which is the changed sound source position (i.e., the changed position of the first speaker SPK1), from the voice signal. The voice processing device 100B can generate sound source position information indicating the changed position "P5" and store the sound source position information as reference sound source position information.

[0119] In an embodiment, the voice processing device 100B can store new sound source position information indicating the changed positions of each of the speakers SPK1 to SPK4 as reference sound source position information, or can replace already stored sound source position information with sound source position information indicating the changed position.

[0120] 19 is a flowchart showing a method for operating a sound processing device according to an embodiment of the present invention. The method for operating a sound processing device described with reference to FIG. 19 is realized as a program stored in a non-transitory storage medium and executable by a computing device.

[0121] The operating method described with reference to FIG. 19 is performed by the audio processing device 100B described with reference to FIG.

[0122] The voice processing device 100B can generate voice source position information indicating the voice source position for each of the voices of the speakers SPK1 to SPK4 in response to the voices of the speakers SPK1 to SPK4 (S310). In the embodiment, the voice processing device 100B can generate a voice signal in response to the voices of the speakers SPK1 to SPK4, and generate voice source position information indicating the voice source position for each of the voices of the speakers SPK1 to SPK4 from the voice signal. At this time, the voice source position indicates the position of each of the speakers SPK1 to SPK4.

[0123] The sound processing device 100B can store the generated sound source position information as reference sound source position information (S320). In an embodiment, the sound processing device 100B can store the generated sound source position information in the memory 140 as reference sound source position information.

[0124] The audio processing device 100B can detect the movement of the audio processing device 100B (S330). In an embodiment, the audio processing device 100B can detect the movement of the audio processing device 100B using the motion sensor 153. For example, the audio processing device 100B can detect a change in position, an angle, or a change in speed and acceleration of the audio processing device 100B using the motion sensor 153.

[0125] The audio processing device 100B can determine whether the detected motion exceeds a reference motion (S340). In an embodiment, the audio processing device 100B can determine whether a physical quantity detected using the motion sensor 153 exceeds a pre-specified reference physical quantity. For example, the audio processing device 100B can determine whether the motion exceeds the reference motion by determining whether a periodically measured change in position of the audio processing device 100B exceeds a reference value or by determining whether an acceleration of the audio processing device 100B exceeds a reference value.

[0126] When the detected movement exceeds the reference movement (Y in S340), the voice processing device 100B can generate sound source position information in response to the voices of the speakers SPK1 to SPK4 and store the generated sound source position information as reference sound source position information. That is, when the detected movement exceeds the reference movement, the voice processing device 100B can re-determine the sound source positions of the voices of the speakers SPK1 to SPK4 and re-store the sound source position information indicating the changed sound source position as reference sound source position information. As a result, even if the relative positions of the speakers SPK1 to SPK4 change due to the movement of the voice processing device 100B, the reference sound source position information is updated. As a result, the error due to the change in the relative positions of the speakers SPK1 to SPK4 due to the movement of the voice processing device 100B can be minimized.

[0127] 20 shows an audio processing device according to an embodiment of the present invention. Referring to FIG. 20, the audio processing device 100C may include a microphone 110, a communication circuit 120, a processor 130, a memory 140, and a light emitting device 155. 2, the sound processing device 100C of FIG 20 is different in that it additionally includes a light emitting device 155. Hereinafter, for convenience of explanation, the explanation of the overlapping parts will be omitted and only the differences will be explained.

[0128] The light emitting device 155 can emit light under the control of the processor 130. In an embodiment, the light emitting device 155 includes a light emitting element, and the light emitting element can emit light of a specific wavelength according to an electrical signal. For example, the light emitting device 155 can be a light emitting diode, a liquid crystal display (LCD), an organic lighting emitting diode (OLED) light emitting device, a flexible light emitting device, a micro LED light emitting device, or a quantum dot light emitting device, but the embodiment of the present invention is not limited thereto.

[0129] In an embodiment, the light emitting device 155 may be operated under the control of the processor 130. For example, the light emitting device 155 may display a particular visual pattern based on a control signal transmitted from the processor 130.

[0130] 21 shows an audio processing device according to an embodiment of the present invention. Referring to FIG. 21, the audio processing device 100C may include a light emitting device 155. According to an embodiment of the present invention, the light emitting device 155 may include a plurality of light emitting elements LED1 to LEDn (n is a natural number equal to or greater than 2). In the embodiment, the plurality of light emitting elements LED1 to LEDn are arranged on a surface of the audio processing device 100C, but the embodiment of the present invention is not limited thereto, and the plurality of light emitting elements LED1 to LEDn may be arranged in a portion of the audio processing device 100 that is visible to the naked eye.

[0131] For example, as shown in FIG. 21, the audio processing device 100C is realized in a form having a circular cross-section, and multiple light-emitting elements LED1 to LEDn are continuously arranged around the surface of the audio processing device 100C, but is not limited to this. The multiple light emitting elements LED1 to LEDn are disposed at different positions from each other.

[0132] As described later, the voice processing device 100C can determine the positions of the speakers SPK1 to SPK4 in response to the voices of the speakers SPK1 to SPK4, and can display a visual pattern corresponding to the position of the speaker who is speaking via the light emitting device 155. For example, the voice processing device 100C can turn on a light emitting element corresponding to the position of the speaker who is speaking among the multiple light emitting elements LED1 to LEDn. This has the effect of allowing the user to grasp the position of the speaker SPK1 to SPK4 who is currently speaking through the position of the light emitting element that is emitting light among the light emitting elements LED1 to LEDn arranged in the light emitting device 100C. For example, each of the light emitting elements LED1 to LEDn can indicate a specific position.

[0133] 22 and 23 are diagrams for explaining the operation of the voice processing device according to the embodiment of the present invention. Referring to Fig. 22 and Fig. 23, the voice processing device 100C according to the embodiment of the present invention can determine the position of the voice of each of the speakers SPK1 to SPK4 in response to the voices of the speakers SPK1 to SPK4, and output a visual pattern corresponding to each position according to the determined position.

[0134] 22 and 23, it is assumed that the speech processing device 100C outputs visual patterns corresponding to the positions of the speakers SPK1 to SPK4 using a plurality of light-emitting elements LED1 to LED8. However, in the embodiment, the speech processing device 100C can output visual patterns corresponding to the positions of the speakers SPK1 to SPK4 using other visual expression methods.

[0135] The voice processing device 100C, according to the embodiment described with reference to FIGS. 3 to 5, can determine the voice source positions (that is, the positions of the speakers SPK1 to SPK4) from the voices of the speakers SPK1 to SPK4.

[0136] The sound processing device 100C can store identifiers for distinguishing the light-emitting elements LED1 to LED8 and information on the positions corresponding to the light-emitting elements LED1 to LED8. For example, as shown in Fig. 22 and Fig. 23, the position corresponding to the second light-emitting element LED2 is the second position P2. At this time, the positions corresponding to the light-emitting elements LED1 to LED8 may be the actual positions of the light-emitting elements LED1 to LED8, or may be pre-specified positions that are unrelated to the actual positions.

[0137] According to an embodiment of the present invention, the voice processing device 100C can determine a sound source position associated with the voice of each of the speakers SPK1 to SPK4 in response to the voices of the speakers SPK1 to SPK4, and activate a light-emitting element among the light-emitting elements LED1 to LED8 that is arranged at a position corresponding to the determined sound source position.

[0138] For example, as shown in Fig. 22, when a first speaker SPK1 located at a first position P1 speaks, the voice processing device 100C can determine the position of the first speaker SPK1 (i.e., the position of the sound source) from the voice of the first speaker SPK1 and activate the light-emitting element corresponding to the first position P1, which is the position of the first speaker SPK1. Since the light-emitting element corresponding to the first position P1 is the eighth light-emitting element LED8, the voice processing device 100C can turn on the eighth light-emitting element LED8. For example, the processor 130 can output a control signal for turning on the eighth light-emitting element LED8.

[0139] Similarly, for example, as shown in Fig. 23, when a second speaker SPK2 located at a second position P2 speaks, the voice processing device 100C can turn on the second light-emitting element LED2. For example, the processor 130 can output a control signal for turning on the second light-emitting element LED2.

[0140] The voice processing device 100C can turn on light-emitting elements corresponding to the positions of the speakers SPK1 to SPK4 when the voice of each of the speakers SPK1 to SPK4 is recognized. In the embodiment, the voice processing device 100C can turn on the light-emitting elements while the voice of each of the speakers SPK1 to SPK4 is being recognized, and can turn off the light-emitting elements when the voice of each of the speakers SPK1 to SPK4 is not being recognized.

[0141] The voice processing device 100C according to the embodiment of the present invention can turn on a light-emitting element corresponding to the position of a speaker among the plurality of light-emitting elements LED1 to LEDn. This has the effect of allowing the user to grasp the position of the speaker SPK1 to SPK4 currently speaking through the position of the light-emitting element that is emitting light among the light-emitting elements LED1 to LEDn arranged in the light-emitting device 100C.

[0142] 24 is a flowchart showing a method for operating an audio processing device according to an embodiment of the present invention. The method for operating an audio processing device described with reference to FIG. 24 is realized as a program stored in a non-transitory storage medium and executable by a computing device.

[0143] 24, the audio processing device 100C can generate an audio signal in response to an audio (S410). In an embodiment, the audio processing device 100C can convert an audio detected in a space into an audio signal, which is an electrical signal.

[0144] The voice processing device 100C can determine the sound source position for each of the voices (i.e., the position of the speakers SPK1 to SPK4) by using the voice signals associated with the voices of the speakers SPK1 to SPK4 (S420). In the embodiment, the voice processing device 100C can generate sound source position information indicating the sound source position for each of the voices of the speakers SPK1 to SPK4 (i.e., the position of the speakers SPK1 to SPK4).

[0145] Based on the sound source positions for each of the sounds, the sound processing device 100C can display visual patterns corresponding to the sound source positions (S430). In the embodiment, the sound processing device 100C includes a light emitting device 155 including a plurality of light emitting elements LED1 to LEDn, and can turn on a light emitting element among the plurality of light emitting elements LED1 to LEDn that corresponds to the position of a sound source.

[0146] Furthermore, in an embodiment, the voice processing device 100C may include a light-emitting device 155 realized by a display device, and the light-emitting device 155 may display a visual pattern pointing to the sound source positions of the speakers SPK1 to SPK4. For example, the light-emitting device 155 may display a graphic such as an arrow, a straight line, or a finger to display a visual pattern pointing to the sound source positions of the speakers SPK1 to SPK4.

[0147] 25 is a diagram for explaining the operation of the voice processing device according to the embodiment of the present invention. Referring to FIG 25, the voice processing device 100C can operate in a location registration mode.

[0148] As described with reference to FIGS. 10 and 11, in the position registration mode, the sound processing device 100C can store the generated sound source position information in the memory 140 as reference sound source position information.

[0149] In the position registration mode, the voice processing device 100C can determine the sound source position of each of the voices of the speakers SPK1 to SPK4 in response to the voices of the speakers SPK1 to SPK4, and output a visual pattern corresponding to the determined sound source position.

[0150] In the embodiment, when the sound source position information is completely stored in the memory 140 as the reference sound source position information, the sound processing device 100C can output a visual pattern corresponding to the completely stored sound source position information.

[0151] 25, if the first sound source position information indicating the first position P1 is stored as the reference sound source position information, the voice processing device 100C can turn on the eighth light-emitting element LED8 corresponding to the first position P1 among the plurality of light-emitting elements LED1 to LED8. Also, if the sound source position information indicating the positions P2 to P4 of the remaining speakers SPK2 to SPK4 is stored as the reference sound source position information, the voice processing device 100C can turn on the second light-emitting element LED2 corresponding to the second position P2, the sixth light-emitting element LED6 corresponding to the third position P3, and the fourth light-emitting element LED4 corresponding to the fourth position P4 among the plurality of light-emitting elements LED1 to LED8.

[0152] This has the effect of allowing the speakers SPK1 to SPK4 to easily grasp where the sound source position registered as the reference position is.

[0153] 26 is a diagram showing the operation of the audio processing device according to the embodiment of the present invention. Referring to FIG 26, the audio processing device 100C can operate in an audio separation mode.

[0154] As described with reference to Figure 12, in the voice separation mode, the voice processing device 100C can generate separated voice signals related to the voices of speakers SPK1 to SPK4 by separating the voice signals associated with the voices of speakers SPK1 to SPK4 based on the sound source positions of the voices, and store the generated separated voice signals.

[0155] In the voice separation mode, the voice processing device 100C can determine the voice source position of each of the speakers SPK1 to SPK4 in response to the voices of the speakers SPK1 to SPK4 and output a visual pattern corresponding to the voice source position of the recognized voice. For example, as shown in Fig. 26, the voice processing device 100C can turn on the light-emitting elements (the eighth light-emitting element LED8, the second light-emitting element LED2, and the sixth light-emitting element LED6) corresponding to the position of the speaker who is speaking among the multiple light-emitting elements LED1 to LED8.

[0156] This has the effect of allowing the speakers SPK1 to SPK4 to easily grasp the location of the sound source of the currently spoken voice.

[0157] In an embodiment, the voice processing device 100C may output a visual pattern corresponding to a sound source position in a voice separation mode using a display method different from that in the position registration mode. For example, the voice processing device 100C may output a visual pattern corresponding to a sound source position using a first display method in the position registration mode, and output a visual pattern corresponding to a sound source position using a second display method different from the first display method in the voice separation mode. The display method may refer to an output hue, an output time, an output period, etc. of the visual pattern.

[0158] For example, in the position registration mode, once the sound source position information has been completely stored in the memory 140 as reference sound source position information, the sound processing device 100C can output a visual pattern corresponding to the sound source position information that has been completely stored, and in the voice separation mode, while the voices of the speakers SPK1 to SPK4 are being recognized, the sound processing device 100C can output a visual pattern corresponding to the sound source position information of the recognized voice.

[0159] 27 is a diagram for explaining the operation of the voice processing device according to the embodiment of the present invention. The operation method of FIG. 27 is performed by the voice processing devices 100, 100A, 100B, and 100C.

[0160] The voice processing device 100 can generate the meeting record MIN using the separated voice signals associated with the voices of the speakers SPK1 to SPK4. The generated meeting record MIN is stored in the form of, but is not limited to, a document file, an image file, or a voice file.

[0161] The voice processing device 100 can generate data representing the voice of each of the speakers SPK1 to SPK4 based on the separated voice signals that have been matched and stored, and can generate the meeting minutes MIN using the generated data representing the voice of each of the speakers SPK1 to SPK4. In an embodiment, the voice processing device 100 can generate the meeting minutes MIN by arranging data associated with the voice of each speaker in chronological order according to the time point when the voice of each of the speakers SPK1 to SPK4 was recognized.

[0162] In the embodiment, the voice processing device 100 can display an identifier indicating the speaker SPK1-SPK4 who uttered a specific voice in the meeting record MIN by using an identifier for identifying the speaker SPK1-SPK4. This allows the utterances in the meeting record MIN to be distinguished by speaker.

[0163] As shown in Fig. 27, speakers SPK1 to SPK4 sequentially utter "AAA1", "BBB2", "AAA3", "CCC4", "DDD5", "CCC6" and "BBB7". As described above, the speech processing device 100 can match and store the first separated speech signal corresponding to "AAA1" and "AAA3" with the first identifier SID1 indicating the first speaker SPK1, match and store the second separated speech signal corresponding to "BBB2" and "BBB7" with the second identifier SID2, match and store the third separated speech signal corresponding to "CCC4" and "CCC6" with the third identifier SID3, and match and store the fourth separated speech signal corresponding to "DDD5" with the fourth identifier SID4.

[0164] Although the embodiments have been described above with reference to limited examples and drawings, those skilled in the art may make various modifications and variations from the above description, for example, the techniques described may be performed in a different order than described, and / or the components of the systems, structures, devices, circuits, etc. described may be combined or combined in a different manner than described, or may be replaced or substituted by other components or equivalents, while still achieving suitable results. As such, other implementations, other embodiments, and equivalents of the claims are intended to be within the scope of the following claims. [Industrial Applicability]

[0165] An embodiment of the present invention relates to an apparatus and method of operation for processing audio. [Explanation of symbols]

[0166] 100, 100A, 100B, 100C Audio processing device 110 Mike 120 Communication Circuit 130 processors 140 Memory 150 Speaker 151 Trigger signal generating circuit 153 Motion Sensor 155 Light emitting device

Claims

1. a processor configured to perform source separation of audio signals associated with the speakers' voices based on source locations of each of the voices; A memory, The processor, generating sound source position information indicative of a sound source position of each of the sounds using a sound signal associated with the sounds; generating, from the speech signal, a separated speech signal associated with a speech of each of the speakers based on the sound source position information; The separated audio signal and the sound source position information are matched with each other and stored in the memory; The processor, In the location registration mode, generating sound source position information indicating a sound source position of each of the sounds using the sound signals; generating an identifier for identifying each of said speakers in response to the speech of said speakers; The sound source position information is stored in the memory as reference sound source position information; In audio separation mode, storing a separated sound signal associated with a sound corresponding to a sound source position within a reference range from the reference sound source position indicated by the reference sound source position information in the memory; The memory stores the identifier; The voice processing device according to claim 1, wherein the identifier is stored in a manner matched with the reference sound source position information and the separated voice signal.

2. The audio processing device includes: The audio processing apparatus of claim 1 , further comprising a microphone configured to generate the audio signal in response to a voice of the speaker.

3. The microphone includes a plurality of microphones. The audio processing device of claim 2 , wherein the plurality of microphones are configured to generate the audio signals in response to the sound.

4. The processor, determining a source location of each of the sounds based on a time delay between a plurality of audio signals generated from the plurality of microphones; The audio processing apparatus according to claim 3 , further comprising: generating the separated audio signals based on the determined sound source positions.

5. the memory stores starting language information indicating a starting language which is a spoken language of the speaker's voice; The speech processing device according to claim 1 , wherein the processor outputs a translation result in an arrival language, the arrival language being a language into which the language of the speaker's speech is translated from the departure language, based on the departure language information and the separated speech signal.

6. 6. The speech processing device according to claim 5, wherein the processor determines a start language corresponding to the position of the speech depending on a sound source position of each of the speeches based on the start language information, and outputs a translation result for each of the speeches depending on the determined start language.

7. The audio processing device includes: further comprising a trigger signal generating circuit configured to generate a speaker enrollment trigger signal; 2. The speech processing apparatus of claim 1, wherein the processor operates in the location registration mode in response to the speaker registration trigger signal.

8. the trigger signal generating circuit generates a speaker movement trigger signal; The processor, 8. The speech processing device according to claim 7, further comprising: in response to the speaker movement trigger signal, generating speaker position information and a speaker identifier using the speech signal; determining a reference speaker identifier matched with the speaker identifier; and updating the reference speaker position information stored in association with the reference speaker identifier to the generated speaker position information.

9. the audio processing device further includes a motion sensor configured to detect movement of the audio processing device; The processor, determining whether the motion of the audio processing device detected by the motion sensor exceeds a reference motion; The voice processing device according to claim 7, characterized in that, when the movement of the voice processing device exceeds a reference movement, sound source position information indicating a changed sound source position of the speaker's voice is generated based on the speaker's voice, and the sound source position information indicating the changed sound source position is stored in the memory as the reference sound source position information.

10. The audio processing device further includes a light emitting device configured to emit light under control of the processor; The processor, The audio processing device according to claim 1 , further comprising: an output of a light emission control signal for controlling the light emitting device so that a visual pattern corresponding to the sound source position is displayed via the light emitting device.

11. the light emitting device includes a plurality of light emitting elements, each configured to emit light; The processor, The sound processing device according to claim 10, further comprising: an output unit for outputting the light emission control signal for selectively turning on a light emitting element corresponding to the determined sound source position among the plurality of light emitting elements.

12. The memory stores information indicating an identifier and a position of each of the light-emitting elements; The processor, The sound processing device according to claim 11, further comprising: referring to the memory, reading out an identifier of a light-emitting element corresponding to a determined sound source position among the plurality of light-emitting elements; and outputting the light-emitting control signal for selectively turning on the light-emitting element corresponding to the determined sound source position using the read identifier.

Images