Audio processing device, audio processing system, and audio processing method for processing audio

Abstract

A speech processing apparatus is provided for generating and processing separated speech signals associated with speech of multiple speakers. [Solution] The audio processing device according to the present invention includes a microphone configured to generate audio signals in response to the voices of multiple speakers, a communication circuit configured to transmit and receive data, a memory, and a processor, and the processor performs source separation of the audio signal based on the source positions of each of the voices based on instruction words stored in the memory, generates separated audio signals related to each of the voices through the source separation, determines an output mode corresponding to the source position of each of the voices, and outputs the separated audio signals according to the determined output mode using the communication circuit.

Description

[Technical field]

[0001] The present invention relates to a voice processing device, a voice processing system, and a voice processing method for processing voice. [Background technology]

[0002] A microphone is a device that converts sounds into audio signals, which are electrical signals. When a microphone is placed in a space where multiple speakers are located, such as a conference room or classroom, the microphone receives all the sounds from the multiple speakers and generates audio signals related to the sounds of the multiple speakers.

[0003] On the other hand, when multiple speakers speak simultaneously, their voices may all be mixed together, and it becomes necessary to separate the audio signals representing a specific speaker's voice among the multiple speakers' voices. Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention has been made in view of the above-mentioned prior art, and an object of the present invention is to provide a speech processing device and a speech processing method capable of generating separated speech signals related to the speech of each speaker in response to the speech of each speaker.

[0005] Another object of the present invention is to provide a voice processing device, a voice processing system, and a voice processing method that can output the voice of each speaker in an output mode set for each speaker using separated voice signals related to the voice of each speaker. [Means for solving the problem]

[0006] In order to achieve the above object, according to one aspect of the present invention, an audio processing device for processing audio from multiple speakers includes a microphone configured to generate audio signals in response to the audio from the multiple speakers, a communication circuit configured to transmit and receive data, a memory, and a processor, wherein the processor performs sound source separation of the audio signal based on a sound source position of each of the audio based on an instruction word stored in the memory, generates separated audio signals related to each of the audio based on the sound source position of each of the audio, determines an output mode corresponding to the sound source position of each of the audio, and outputs the separated audio signals according to the determined output mode using the communication circuit.

[0007] In order to achieve the above-mentioned object, one aspect of the present invention provides a voice processing method for processing voices of multiple speakers, comprising the steps of generating voice signals in response to the voices of the multiple speakers, source separating the voice signals based on source positions of each of the voices, generating separated voice signals related to each of the voices, determining an output mode for the source positions of each of the voices, and outputting each of the voices of the multiple speakers using the separated voice signals in the determined output mode. Effect of the Invention

[0008] According to the present invention, a separated audio signal related to a sound from a specific sound source position can be generated based on the sound source position of the sound, thereby having the effect of generating an audio signal in which the influence of ambient noise is minimized.

[0009] In addition, according to the present invention, the voices of multiple speakers can be separated according to the sound source position and output according to an output mode set for each speaker, allowing a user to listen (auditory) or view (visual) only the voices of some of the speakers, thereby enabling the user to selectively listen to or view a conversation depending on the importance of each speaker. [Brief description of the drawings]

[0010] [Figure 1] 1 illustrates an audio processing system according to an embodiment of the present invention. [Diagram 2] 1 illustrates an audio processing device according to an embodiment of the present invention. [Diagram 3] 2 illustrates a user terminal according to an embodiment of the present invention. [Figure 4] 4 is a diagram illustrating an operation of a mobile terminal according to an embodiment of the present invention; [Diagram 5] 4 is a diagram illustrating an operation of a mobile terminal according to an embodiment of the present invention; [Figure 6] 4 is a diagram illustrating an operation of a mobile terminal according to an embodiment of the present invention; [Figure 7] 2 is a flowchart illustrating a method for audio separation according to an embodiment of the present invention. [Figure 8] FIG. 1 is a diagram for explaining a voice processing system according to an embodiment of the present invention. [Figure 9] FIG. 1 is a diagram for explaining a voice processing system according to an embodiment of the present invention. [Figure 10] FIG. 4 is a diagram for explaining an output mode according to an embodiment of the present invention. [Figure 11] FIG. 4 is a diagram for explaining an output mode according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0012] 1 shows a voice processing system according to an embodiment of the present invention. Referring to FIG 1, the voice processing system 10 includes a voice processing device 100 that generates voice signals corresponding to the voices of speakers SPK1 to SPK4 and outputs the generated voice signals, and a user terminal 200 that receives the voice signals output from the voice processing device 100.

[0013] The speakers SPK1 to SPK4 are located in a space (for example, a conference room, a vehicle, a lecture hall, etc.) and pronounce sounds. In some embodiments, the first speaker SPK1 pronounces sounds at a first position P1, the second speaker SPK2 pronounces sounds at a second position P2, the third speaker SPK3 pronounces sounds at a third position P3, and the fourth speaker SPK4 pronounces sounds at a fourth position P4.

[0014] The voice processing device 100 is a device that processes the voices of the speakers SPK1 to SPK4, and includes a microphone that converts the voices into electrical signals.

[0015] The voice processing device 100 generates a voice signal related to the voice of the speakers SPK1 to SPK4 in response to each of the voices of the speakers SPK1 to SPK4. The voice signal is a signal related to voices uttered during a specific period of time, and is a signal indicating the voices of multiple speakers.

[0016] According to an embodiment, the voice processing device 100 uses voice signals related to the voices of the speakers SPK1 to SPK4 to determine the sound source positions of each of the voices of the speakers SPK1 to SPK4, and performs sound source separation based on the sound source positions, thereby extracting (or generating) separated voice signals related to the voices of the speakers SPK1 to SPK4 from the voice signals.

[0017] The speech processing device 100 stores the separated speech signals and sound source position information indicating the sound source positions.

[0018] That is, the voice processing device 100 generates a separated voice signal related to the voice of the speakers SPK1 to SPK4 located at each position P1 to P4 based on the sound source position of the voice corresponding to the voice signal. For example, the voice processing device 100 generates a first separated voice signal related to the voice of the first speaker SPK1 who spoke at the first position P1 based on the voice signal. At this time, the first separated voice signal is a voice signal having the highest degree of relevance to the voice of the first speaker SPK1 among the voices of the speakers SPK1 to SPK4. In other words, the voice component of the first speaker SPK1 has the highest weight among the voice components included in the first separated voice signal.

[0019] The voice processing device 100 transmits the separated voice signals associated with each of the voices of the speakers SPK1 to SPK4 to the user terminal 200. According to an embodiment, the voice processing device 100 transmits to the user terminal 200 the separated voice signals associated with each of the voices of the speakers SPK1 to SPK4 and sound source position information indicating the corresponding sound source positions.

[0020] Furthermore, the speech processing device 100 can provide translations for the speech of each of the speakers SPK1 to SPK4. For example, the speech processing device 100 determines a source language (a language to be translated) and a target language (a language after translation) for translating the speech of each of the speakers SPK1 to SPK4, and provides a translation for each of the speakers' languages ​​using the separated speech signals.

[0021] Depending on the embodiment, the speech processing device 100 outputs a translation result for each of the speeches. The translation result is text data or a speech signal related to the speech of each of the speakers SPK1 to SPK4 expressed in the target language.

[0022] The user terminal 200 is an electronic device having a communication function and a processing function. For example, the user terminal 200 may be a smartphone, a laptop, a PDA (personal digital assistance), a wearable device, a smart watch, a tablet computer, a TV (television), etc., but the embodiment of the present invention is not limited thereto.

[0023] The user terminal 200 receives separated audio signals associated with the respective audio of the speakers SPK1 to SPK4 from the audio processing device 100, and outputs the received separated audio signals. For example, the user terminal 200 can output the separated audio signals audibly (i.e., sound) or visually (i.e., subtitles).

[0024] 2 shows an audio processing device according to an embodiment of the present invention. Referring to FIG. 2, the audio processing device 100 may include a microphone 110, a communication device 120, a processor 130, a memory 140, and a display 150. Depending on the embodiment, the audio processing device 100 may further include a speaker 160.

[0025] The microphone 110 generates an audio signal in response to a generated sound. According to an embodiment, the microphone 110 detects vibrations in the air caused by the sound and generates an audio signal, which is an electrical signal corresponding to the vibrations, based on the detection result.

[0026] According to an embodiment, there may be a plurality of microphones 110, each of which generates an audio signal in response to a voice. Since the positions at which the plurality of microphones 110 are arranged may be different from each other, the audio signals generated from the respective microphones 110 have a phase difference (or a time delay) from each other.

[0027] The communication device 120 exchanges data with an external device through a wireless communication method. According to an embodiment, the communication device 120 exchanges data with an external device using radio waves of various frequencies. For example, the communication device 120 exchanges data with an external device through at least one wireless communication method of short-range wireless communication, medium-range wireless communication, and long-range wireless communication.

[0028] The processor 130 controls the overall operation of the audio processing device 100. Depending on the embodiment, the processor 130 may include a processor having a calculation function. For example, the processor 130 may include, but is not limited to, a central processing unit (CPU), a micro controller unit (MCU), a graphics processing unit (GPU), a digital signal processor (DSP), an analog to digital converter (ADC converter), or a digital to analog converter (DAC converter).

[0029] The processor 130 processes the audio signal generated by the microphone 110. For example, the processor 130 converts the analog type audio signal generated by the microphone 110 into a digital type audio signal, and processes the converted digital type audio signal. In this case, since the type of signal (analog or digital) changes, the digital type audio signal and the analog type audio signal are used together in the description of the embodiment of the present invention.

[0030] According to an embodiment, the processor 130 extracts (or generates) a separated audio signal associated with the voice of each of the speakers SPK1 to SPK4 using the audio signal generated by the microphone 110. According to an embodiment, the processor 130 generates a separated audio signal associated with the voice of the speakers SPK1 to SPK4 located at each of the positions P1 to P4.

[0031] The processor 130 determines the positions of the sound sources (i.e., the positions of the speakers SPK1 to SPK4) of the sounds using the time delay (or phase delay) between the sound signals. For example, the processor 130 determines the relative positions of the sound sources (i.e., the speakers SPK1 to SPK4) with respect to the sound processing device 100.

[0032] The processor 130 generates a separated audio signal related to the voice of each of the speakers SPK1 to SPK4 based on the determined sound source position. For example, the processor 130 generates a first separated audio signal related to the voice of the first speaker SPK1 based on the sound source position of the voice.

[0033] According to an embodiment, the processor 130 matches the sound source position information indicating the determined sound source position with the separated voice signal and stores the matched sound source position information. For example, the processor 130 matches the first separated voice signal related to the voice of the first speaker SPK1 with the first sound source position information indicating the sound source position of the voice of the first speaker SPK1 and stores the matched sound source position information in the memory 140.

[0034] The operations of the processor 130 or the voice processing device 100 described herein may be implemented in the form of a program executable by a computer device. For example, the processor 130 executes an application stored in the memory 140, and performs an operation corresponding to a command that instructs a particular operation by executing the application.

[0035] The memory 140 stores data necessary for the operation of the audio processing device 100. For example, the memory 140 includes at least one of a non-volatile memory and a volatile memory.

[0036] According to an embodiment, the memory 140 stores identifiers corresponding to the respective positions P1 to P4 in space. The identifiers are data for distinguishing the positions P1 to P4. Since the speakers SPK1 to SPK4 are located at the positions P1 to P4, respectively, the speakers SPK1 to SPK4 can be distinguished from each other using the identifiers corresponding to the positions P1 to P4. For example, the first identifier indicating the first position P1 indicates the first speaker SPK1.

[0037] The identifier may be input through an input device (eg, a touchpad) of the audio processing device 100.

[0038] According to an embodiment, the memory 140 stores sound source position information associated with the position of each of the speakers SPK1 to SPK4 and separated speech signals associated with the speech of each of the speakers SPK1 to SPK4.

[0039] 3 shows a user terminal according to an embodiment of the present invention. Referring to FIG. 3, a user terminal 200 includes an input unit 210, a control unit 220, a storage unit 230, and an output unit 240.

[0040] The input unit 210 generates an input signal in response to an operation of a user of the user terminal 200. According to an embodiment, the input unit 210 generates the input signal, which is an electrical signal, based on the user's operation on the user terminal 200. For example, the input unit 210 may be, but is not limited to, a keyboard, a touch panel, or a touch display.

[0041] The control unit 220 controls the overall operation of the user terminal 200. According to an embodiment, the control unit 220 includes a processor having a calculation function. For example, the control unit 220 may be, but is not limited to, a central processing unit (CPU), a micro controller unit (MCU), or an application processor (AP).

[0042] The control unit 220 controls the user terminal 200 to output the separated voice signal transmitted from the voice processing device 100. According to an embodiment, the control unit 220 executes a program (or application) stored in the storage unit 230, and controls the user terminal 200 to output the separated voice signal transmitted from the voice processing device 100 through the output unit 240 according to the execution result.

[0043] According to an embodiment, the control unit 220 controls the user terminal 200 to perform a specific operation by executing a program stored in the storage unit 230. That is, the operation of the user terminal 200 described in this specification can be understood as an operation caused by the control unit 220 executing a stored program.

[0044] The storage unit 230 stores data necessary for the operation of the user terminal 200. According to an embodiment, the storage unit 230 may be, but is not limited to, a non-volatile memory or a volatile memory.

[0045] The output unit 240 outputs data stored in the user terminal 200. According to an embodiment, the output unit 240 outputs data stored in the user terminal 200 audibly or visually under the control of the control unit 220. For example, the output unit 240 may be a speaker or a display device, but the embodiment of the present invention is not limited thereto.

[0046] 4 to 6 are diagrams illustrating the operation of a mobile terminal according to an embodiment of the present invention, in which speakers SPK1 to SPK4 located at positions P1 to P4 respectively make speeches.

[0047] The speech processing device 100 according to the embodiment of the present invention generates separated speech signals related to the speech of each of the speakers SPK1 to SPK4 from the speech of the speakers SPK1 to SPK4, and stores the separated speech signals and position information indicating the positions of each of the speakers SPK1 to SPK4.

[0048] Depending on the embodiment, the speech processing device 100 uses the time delay (or phase delay) between the speech signals to determine the position of the speech source (i.e., the position of the speakers SPK1 to SPK4). For example, the speech processing device 100 determines the relative position of the speech source (i.e., the speakers SPK1 to SPK4) with respect to the speech processing device 100.

[0049] The voice processing device 100 generates separated voice signals related to the voices of the speakers SPK1 to SPK4, based on the determined sound source positions.

[0050] As shown in Fig. 4, the first speaker SPK1 speaks the voice "AAA". When the voice "AAA" is spoken, the voice processing device 100 generates a voice signal related to the voice "AAA" in response to the voice "AAA". According to an embodiment, the voice signal related to the voice "AAA" also includes components related to noise other than the voice "AAA".

[0051] According to the embodiment, the voice processing device 100 generates a separated voice signal related to the voice "AAA" of the first speaker SPK1 using the generated voice signal. At this time, the voice processing device 100 stores the first separated voice signal related to the voice "AAA" of the first speaker SPK1 and first sound source position information indicating a first position P1 which is the position of the first speaker SPK1 in the memory 140. For example, as shown in FIG. 4, the first separated voice signal and the first sound source position information are matched with each other and stored.

[0052] 5, the second speaker SPK2 utters the voice "BBB." When the voice "BBB" is uttered, the voice processing device 100 generates a voice signal related to the voice "BBB" in response to the voice "BBB."

[0053] According to the embodiment, the voice processing device 100 generates a second separated voice signal related to the voice "BBB" of the second speaker SPK2 using the generated voice signal. At this time, the voice processing device 100 stores the second separated voice signal related to the voice "BBB" of the second speaker SPK2 and second sound source position information indicating a second position P2, which is the position of the second speaker SPK2, in the memory 140. For example, as shown in FIG. 5, the second separated voice signal and the second sound source position information are matched with each other and stored.

[0054] 6, a third speaker SPK3 speaks a voice "CCC" and a fourth speaker SPK4 speaks a voice "DDD." In response to the voice "CCC" and the voice "DDD," the voice processing device 100 generates a voice signal related to the voice "CCC" and the voice "DDD." That is, the voice signal is a voice signal including components related to the voice "CCC" and the voice "DDD."

[0055] In an embodiment, the audio processing device 100 uses the generated audio signals to generate a third separated audio signal associated with the audio "CCC" of the third speaker SPK3 and a fourth separated audio signal associated with the audio "DDD" of the fourth speaker SPK4.

[0056] At this time, the voice processing device 100 stores in the memory 140 a third separated voice signal related to the voice "CCC" of the third speaker SPK3 and third position information indicating the third position P3 which is the position of the third speaker SPK3. In addition, the voice processing device 100 stores in the memory 140 a fourth separated voice signal related to the voice "DDD" of the fourth speaker SPK4 and fourth position information indicating the fourth position P4 which is the position of the fourth speaker SPK4.

[0057] For example, as shown in FIG. 6, the third separated voice signal and the third sound source position information are matched with each other and stored, and the fourth separated voice signal and the fourth sound source position information are matched with each other and stored.

[0058] That is, the voice processing device 100 according to an embodiment of the present invention generates separated voice signals related to the voice of each speaker SPK1 to SPK4 from the voices of the speakers SPK1 to SPK4, and stores the separated voice signals and position information indicating the respective positions of the speakers SPK1 to SPK4.

[0059] 7 is a flowchart illustrating a voice separation method by a voice processing device according to an embodiment of the present invention. The operating method of the mobile terminal described with reference to FIG. 7 is implemented as an application (e.g., a voice separation application) stored in a non-transitory storage medium and executable by a computing device. For example, the processor 130 executes the application stored in the memory 140, and performs an operation corresponding to a command that instructs a particular operation by executing the application.

[0060] 7, the sound processing device 100 generates a sound signal in response to a sound (S110). According to an embodiment, the sound processing device 100 converts a sound sensed in a space into a sound signal, which is an electrical signal.

[0061] The voice processing device 100 uses a voice signal related to the voices of the speakers SPK1 to SPK4 to determine the voice source position for each of the voices (i.e., the position of the speakers SPK1 to SPK4) (S120). According to an embodiment, the voice processing device 100 generates voice source position information indicating the voice source position for each of the voices of the speakers SPK1 to SPK4 (i.e., the position of the speakers SPK1 to SPK4).

[0062] The speech processing device 100 generates separated speech signals related to each of the speeches of the speakers SPK1 to SPK4 based on the sound source positions for each of the speeches (S130). According to an embodiment, the speech processing device 100 separates the generated speech signals based on the sound source positions for each of the speeches, thereby generating separated speech signals related to each of the speeches of the speakers SPK1 to SPK4. For example, the speech processing device 100 generates separated speech signals related to each of the speeches of the speakers SPK1 to SPK4 by separating components included in the speech signals based on the sound source positions.

[0063] The voice processing device 100 stores sound source position information indicating the position of the sound source and the separated sound signals (S140). According to an embodiment, the voice processing device 100 matches the sound source position information indicating the position of the sound source with the separated sound signals related to the respective voices of the speakers SPK1 to SPK4 and stores them. For example, the voice processing device 100 matches data corresponding to the separated sound signals related to the respective voices of the speakers SPK1 to SPK4 with the sound source position information and stores them.

[0064] According to an embodiment, the voice processing device 100 (or the processor 130) according to an embodiment of the present invention generates (or separates) separated voice signals related to the voices of each of the speakers SPK1 to SPK4 from the voice signals related to the voices of the speakers SPK1 to SPK4 by executing an application (e.g., a voice separation application) stored in the memory 140.

[0065] 8 and 9 are diagrams for explaining a speech processing system according to an embodiment of the present invention. Referring to Fig. 8 and Fig. 9, a first speaker SPK1 speaks "Hello", a second speaker SPK2 speaks "Nice to meet you", and a fourth speaker SPK4 speaks "Welcome".

[0066] 4 to 7, the voice processing device 100 matches the separated voice signals related to the voices of the speakers SPK1, SPK3, and SPK4 ("Hello," "Nice to meet you," and "Welcome"), respectively, with the sound source position information indicating the positions of the speakers SPK1, SPK3, and SPK4, and stores them in the memory 140. That is, the voice processing device 100 separates the voices uttered by the speakers SPK1, SPK3, and SPK4, and generates separated voice signals related to each of them.

[0067] Referring to FIG. 9, the user terminal 200 receives separated speech signals associated with the speech of each of the speakers SPK1, SPK3, and SPK4 from the speech processing device 100, and outputs the received separated speech signals visually or audibly.

[0068] According to a voice processing system according to an embodiment of the present invention, user terminal 200 sets an output mode for each of speakers SPK1-SPK4 (or for each of sound source positions P1-P4), and outputs the voice of each of speakers SPK1-SPK4 according to the set output mode. According to an embodiment, user terminal 200 outputs the voice of a first speaker among speakers SPK1-SPK4 in a first mode, and outputs the voice of a second speaker in a second mode different from the first mode. For example, user terminal 200 outputs the voice of the first speaker audibly in the first mode, and outputs the voice of the second speaker visually in the second mode.

[0069] As shown in FIG. 9, the user terminal 200 receives from the speech processing device 100 separated speech signals associated with the speech of speakers SPK1, SPK3, and SPK4, respectively ("Hello," "Nice to meet you," and "Welcome").

[0070] Depending on the output mode set for each of the speakers SPK1, SPK3, and SPK4, the user terminal 200 outputs the voice "Hello" of the first speaker SPK1 as subtitles (i.e., visually) through a display device (e.g., a display) of the user terminal 200, and outputs the voice "Nice to meet you" of the third speaker SPK3 and the voice "Welcome" of the fourth speaker SPK4 as sounds (i.e., audibly) through the speaker of the user terminal 200. Thus, the user can check the voice of a particular speaker in a desired mode.

[0071] Meanwhile, if necessary, the user terminal 200 can convert the format of the received voice signal into a format suitable for the output mode, for example, to output the voice visually, the received voice signal can be converted into text data (Speech to Text).

[0072] Fig. 10 is a diagram for explaining output modes according to an embodiment of the present invention. Referring to Fig. 10, the user terminal 200 stores output mode information regarding output modes corresponding to each of the speakers SPK1 to SPK4 (or sound source positions P1 to P4). According to the embodiment, the user terminal 200 stores a table 231 including the output mode information. According to the embodiment, the table 231 is generated according to an input signal input by a user of the user terminal 200 and stored in the storage unit 230, but the embodiment of the present invention is not limited thereto.

[0073] The user terminal 200 determines an output mode for each of the speakers SPK1 to SPK4 by referring to the table 231 stored in the storage unit 230, and outputs the voice of each of the speakers SPK1 to SPK4 according to the output mode determined using the separated voice signal received from the voice processing device 100.

[0074] According to an embodiment, the user terminal 200 receives separated voice signals from the voice processing device 100, determines a sound source position corresponding to the separated voice signals, determines an output mode corresponding to each of the separated voice signals from the table 231 using the determined sound source position, and outputs the voices of the speakers SPK1 to SPK4 according to the determined output mode.

[0075] For example, as shown in FIG. 10, the user terminal 200 visually outputs the voice of a first speaker SPK1 at a first position P1 in a first mode M1, and audibly outputs the voices of speakers SPK2, SPK3, and SPK4 at positions P2, P3, and P4 in a second mode M2.

[0076] Meanwhile, in the present specification, the audio output mode has been described as being either a visual or an auditory mode, but the present invention is not limited thereto, and the output mode according to the present invention may include various modes of different modes. Also, not outputting audio should be understood as one output mode.

[0077] Fig. 11 is a diagram for explaining output modes according to an embodiment of the present invention. Referring to Fig. 11, the voice processing device 100 stores output mode information regarding output modes corresponding to each of the speakers SPK1 to SPK4 (or sound source positions P1 to P4). According to the embodiment, the voice processing device 100 stores a table 141 including the output mode information. According to the embodiment, the table 141 is generated according to an input signal input by a user of the voice processing device 100 and stored in the memory 140, but the embodiment of the present invention is not limited thereto.

[0078] According to an embodiment, the voice processing device 100 stores output mode information indicating an output mode for each sound source position (or each speaker) for each of a plurality of user terminals. For example, as shown in FIG 11, output mode information for terminal 1 and output mode information for terminal 2 are stored separately.

[0079] The voice processing device 100 refers to the table 141 stored in the memory 140 to determine an output mode for each of the speakers SPK1 to SPK4, and transmits the separated voice signals to the user terminal 200 according to the determined output mode. According to an embodiment, the voice processing device 100 transmits the separated voice signals in a format corresponding to the determined output mode to the user terminal 200. On the other hand, if the output mode is "not output", the voice processing device 100 does not transmit the separated voice signals.

[0080] According to an embodiment, the voice processing device 100 converts the stored separated voice signals into a format corresponding to the determined output mode, and transmits the separated voice signals having the converted format to the user terminal 200. For example, the voice processing device 100 converts the separated voice signal for the first speaker SPK1 of the terminal 1 into a text data format and transmits it, and converts the separated voice signal for the second speaker SPK2 of the terminal 1 into a voice signal format and transmits it, but is not limited thereto.

[0081] The speech processing device 100 according to the embodiment of the present invention generates speech signals related to the speech of the speakers SPK1 to SPK4, and processes the speech signals to generate separated speech signals related to the speech of each of the speakers SPK1 to SPK4.

[0082] According to an embodiment of the present invention, the user terminal 200 outputs the separated voice signals related to the voices of the speakers SPK1 to SPK4 transmitted from the voice processing device 100 according to the output mode set for each of the speakers SPK1 to SPK4. This allows the user to listen (auditory) or view (visual) only the voices of some speakers, and has the effect of being able to selectively listen to or view the dialogue depending on the importance of the speakers.

[0083] Although the present invention has been described with reference to limited embodiments and drawings, those skilled in the art will appreciate that various modifications and variations may be made from the above description, for example, the described techniques may be performed in a different manner than described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different manner than described, or may be replaced or substituted by other components or equivalents, while still achieving suitable results.

[0084] Accordingly, other implementations, embodiments and equivalents are intended to fall within the scope of the present invention. [Industrial Applicability]

[0085] The present invention relates to a voice processing device, a voice processing system, and a voice processing method for processing voice. [Explanation of symbols]

[0086] 100 Audio processing device 110 Mike 120 Communication Equipment 130 processors 140 Memory 200 User terminals (user terminals) 210 Input section 220 Control section 230 Preservation Department 240 Output section

Claims

1. 1. A speech processing apparatus for processing speech from multiple speakers, comprising: a microphone configured to generate audio signals in response to voices of the plurality of speakers; a communications circuit configured to transmit and receive data; Memory and a processor; The processor, based on the instructions stored in the memory, separating the audio signal into audio sources based on the positions of the respective audio sources; generating separated audio signals associated with each of said sounds by audio source separation; determining an output mode corresponding to each source position of the sounds; outputting the separated audio signal in the determined output mode using the communication circuit; the memory stores output mode information indicating an output mode corresponding to each sound source position of the sounds; The processor, determining an output mode corresponding to each of the sound source positions by referring to the stored output mode information; generating a second separated audio signal by converting the first separated audio signal generated by the sound source separation into a format corresponding to the output mode; The output modes include visual, auditory, and non-output, and when the output mode is visual, the second separated audio signal is output in text data format using the communication circuit, when the output mode is auditory, the second separated audio signal is output in audio data format, and when the output mode is non-output, the second separated audio signal is not output.

2. The microphone is 2. The audio processing device of claim 1, comprising a plurality of microphones arranged in an array.

3. The processor, determining a source location of each of the sounds based on a time delay between a plurality of audio signals generated from the plurality of microphones; The apparatus according to claim 2 , further comprising: a processor for generating the separated audio signals based on the determined sound source positions.

4. The processor, generating sound source position information indicating a sound source position of each of the sounds based on a time delay between a plurality of sound signals generated from the plurality of microphones; The audio processing apparatus according to claim 3 , wherein the sound source position information for the audio and the separated audio signal for the audio are matched with each other and stored.

5. 1. A speech processing method for processing speech from multiple speakers, comprising: generating audio signals in response to the speech of the plurality of speakers; source separating the audio signal based on source positions of each of the sounds; generating a separated audio signal associated with each of said sounds; determining an output mode for each source position of said sounds; outputting the separated audio signal according to the determined output mode; The step of determining an output mode includes: storing output mode information indicating an output mode for each sound source position of the sounds; determining an output mode for each of the sound source positions by referring to the stored output mode information; The step of outputting the separated audio signal in the determined output mode includes: generating a second separated audio signal by converting a first separated audio signal generated by the sound source separation into a format corresponding to the output mode; the output modes include visual, auditory, and no output, and outputting the second separated audio signal using a communication circuit; A voice processing method characterized in that, when the output mode is visual, the second separated voice signal is converted into a text data format, when the output mode is auditory, the second separated voice signal is converted into a voice data format, and when the output mode is non-output, the second separated voice signal is not generated.

6. The step of generating an audio signal comprises:

6. The method of claim 5, further comprising the step of generating the audio signal using a plurality of microphones arranged in an array.

7. The step of separating the sound sources includes: determining a source location of each of the sounds based on a time delay between a plurality of audio signals generated from the plurality of microphones; and generating the separated audio signals based on the determined sound source locations.

8. The step of separating the sound sources includes: generating sound source position information indicating a sound source position of each of the sounds based on a time delay between a plurality of sound signals generated from the plurality of microphones; The method of claim 7, further comprising: matching the sound source position information for the voice and the separated voice signal for the voice with each other and storing the matched information.

Images