Electronic device for outputing sound and method of operating the same

Abstract

An electronic device according to various embodiments includes a vibration sensor, a microphone, and a processor, wherein the processor receives a voice signal including an echo uttered by a user through the microphone, receives a vibration signal associated with the voice signal transmitted through at least a part of the user's body through the vibration sensor, predicts echo information based on the voice signal and the vibration signal, and may be configured to remove the echo included in the voice signal based on the predicted echo information.

Description

Technology Field

[0001] Various embodiments of the present invention relate to an electronic device that outputs sound and a method of operating the same. Background Technology

[0002] With the advancement of wireless communication technology, electronic devices can communicate with other electronic devices through various wireless communication technologies. Bluetooth communication technology refers to a short-range wireless communication technology that allows electronic devices to connect with each other and exchange data or information. Furthermore, Bluetooth communication technology may include Bluetooth legacy (or classic) network technology or BLE (Bluetooth Low Energy) networks, and can have various connection topologies such as piconets and scatternets. Electronic devices can share data with each other at low power consumption by utilizing Bluetooth communication technology. By using this Bluetooth technology, external wireless communication devices can be connected, and audio data regarding content running on the electronic device can be transmitted to the external wireless communication device, which then processes the audio data and outputs it to the user. Recently, wireless earphones utilizing Bluetooth communication technology have become widely used. Additionally, to improve the performance of wireless earphones, wireless earphones equipped with multiple microphones are also widely used. The problem to be solved

[0003] TWS (true wireless stereo) based wireless earphones can capture the user's spoken voice using a microphone included in the earphones. However, because the microphone included in the wireless earphones is physically separated from the user's mouth, the user's voice may be damaged by reverberation or ambient noise.

[0004] Recent wireless earphones utilize beamforming signal processing technology employing multiple microphones to acquire high-quality audio. In this process, wireless earphones are designed to maximize the distance between microphones and position them close to the user's mouth. However, since the primary purpose of beamforming signal processing is to eliminate ambient noise, it may not be effective in removing echoes from the user's voice.

[0005] Various embodiments may provide an electronic device and a method of operation thereof that can predict echoes in a user's voice obtained through a microphone and remove echoes included in the user's voice using the predicted echo information. means of solving the problem

[0006] An electronic device according to various embodiments includes a vibration sensor, a microphone, and a processor, wherein the processor receives a voice signal including an echo uttered by a user through the microphone, receives a vibration signal associated with the voice signal transmitted through at least a part of the user's body through the vibration sensor, predicts echo information based on the voice signal and the vibration signal, and may be configured to remove the echo included in the voice signal based on the predicted echo information.

[0007] A method of operating an electronic device according to various embodiments may include receiving a voice signal including an echo uttered by a user through a microphone included in the electronic device, receiving a vibration signal related to the voice signal transmitted through at least a part of the user's body through a vibration sensor included in the electronic device, predicting echo information based on the voice signal and the vibration signal, and removing the echo included in the voice signal based on the predicted echo information.

[0008] A non-transient recording medium according to various embodiments may store a program capable of performing the following operations: receiving a voice signal including an echo uttered by a user through a microphone included in an electronic device; receiving a vibration signal related to the voice signal transmitted through at least a part of the user's body through a vibration sensor included in the electronic device; predicting echo information based on the voice signal and the vibration signal; and removing an echo included in the voice signal based on the predicted echo information. Effects of the invention

[0009] An electronic device according to various embodiments can predict echoes in a user's voice obtained through a microphone and remove echoes included in the user's voice using the predicted echo information, thereby reducing the degradation of sound quality caused by echoes. Brief explanation of the drawing

[0010] FIG. 1 is a block diagram of an electronic device in a network environment according to various embodiments. FIG. 2a is a drawing of an electronic system according to various embodiments. FIG. 2b is a block diagram of an electronic device according to various embodiments. FIG. 3 is a flowchart illustrating the operation of an electronic device according to various embodiments to remove echoes of a voice signal. FIG. 4 is a diagram illustrating the operation of an electronic device according to various embodiments to remove echoes of a voice signal. FIG. 5 is a diagram illustrating the operation of an electronic device according to various embodiments acquiring an impulse response based on a voice signal and a vibration signal. FIG. 6 is a diagram illustrating the operation of an electronic device according to various embodiments acquiring echo information based on an impulse response. FIG. 7 is a flowchart illustrating the operation of an electronic device according to various embodiments removing reverberation included in a voice signal using reverberation time. FIG. 8 is a flowchart illustrating the operation of an electronic device according to various embodiments to remove echoes of a voice signal based on noise intensity. FIG. 9 is a flowchart illustrating the operation of an electronic device according to various embodiments to remove echoes of a voice signal based on echo intensity. FIGS. 10a through 10c are drawings illustrating the operation of an electronic device according to various embodiments to remove echoes included in a voice signal. Specific details for implementing the invention

[0011] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108). According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display module (160), audio module (170), sensor module (176), interface (177), connection terminal (178), haptic module (179), camera module (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor module (176), camera module (180), or antenna module (197)) may be integrated into a single component (e.g., display module (160)).

[0012] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor module (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0013] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display module (160), sensor module (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera module (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0014] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor module (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related commands. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0015] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0016] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0017] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0018] The display module (160) can visually provide information to an external (e.g., user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0019] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0020] The sensor module (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor module (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0021] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0022] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0023] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0024] The camera module (180) can capture still images and video. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0025] The power management module (188) can manage the power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0026] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0027] The communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication processors that operate independently of the processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local area network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data association)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0028] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna.

[0029] The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., electronic device (104)), or a network system (e.g., a second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for realizing eMBB, loss coverage (e.g., 164 dB or less) for realizing mMTC, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for realizing URLLC.

[0030] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0031] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0032] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0033] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199). Each of the external electronic devices (102, or 104) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104, or 108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the above request may execute at least part of the requested function or service, or additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device (101) may provide ultra-low latency services, for example, using distributed computing or mobile edge computing. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server using machine learning and / or neural networks. According to one embodiment, the external electronic device (104) or the server (108) may be included within a second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0034] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0035] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as “coupled” or “connected” to another (e.g., 2nd) component, with or without the terms “functionally” or “communicationly,” it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0036] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0037] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0038] According to one embodiment, the method according to the various embodiments disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or an application store (e.g., Play Store). TM It can be distributed online (e.g., downloaded or uploaded) through ) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0039] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0040] FIG. 2a is a drawing of an electronic system according to various embodiments.

[0041] Referring to FIG. 2a, according to various embodiments, the electronic device (201) may be implemented identically or similarly to the electronic device (101) of FIG. 1. The electronic device (201) may be implemented in a form that can be worn on the user's right or left ear. For example, the electronic device (201) may be implemented as an earphone that outputs sound wirelessly. For example, the electronic device (201) may be implemented as a TWS (true wireless stereo) based wireless earphone.

[0042] According to various embodiments, the electronic device (201) may form a communication link (e.g., a communication link using Bluetooth communication technology) with an external electronic device (202) (e.g., the electronic device (102 or 104) of FIG. 1). The electronic device (201) may transmit and receive data related to sound with the external electronic device (202) through the communication link. For example, the external electronic device (202) may be implemented as a smartphone.

[0043] According to various embodiments, the electronic device (201) can convert data received from an external electronic device (202) into sound and output the converted sound (e.g., audio, music, ambient sound, or telephone sound) through a speaker (e.g., speaker (270) of FIG. 2b). The electronic device (201) can acquire external sound (e.g., user voice or ambient sound) through a microphone (250) and transmit data corresponding to the acquired sound to the external electronic device (202). For example, the electronic device (201) can perform noise removal and reverberation removal operations on the sound acquired through the microphone (250). Additionally, the electronic device (201) can transmit data corresponding to the sound after noise removal and reverberation removal has been performed to the external electronic device (202).

[0044] According to various embodiments, the electronic device (101) can receive a voice signal spoken by the user through a microphone while worn on the user's ear. Additionally, the electronic device (101) can receive a vibration signal corresponding to vibration (e.g., vocal cord vibration) caused by the user's speech through a vibration sensor (260). For example, the vibration signal may be transmitted by at least a part of the user's body. For example, the voice signal may include an echo generated by reflection from the space surrounding the user. The vibration signal may not include or may include almost no such echo.

[0045] According to various embodiments, the electronic device (201) can remove echoes included in the voice signal based on the voice signal and the vibration signal. The operation of the electronic device (201) removing echoes included in the voice signal will be described in detail below.

[0046] FIG. 2b is a block diagram of an electronic device according to various embodiments.

[0047] Referring to FIG. 2b, the electronic device (201) may include a processor (220), memory (230), microphone (250), vibration sensor (260), speaker (270), and communication module (280).

[0048] According to various embodiments, the processor (220) can control the overall operation of the electronic device (201). The processor (220) may be implemented identically or similarly to the processor (120) of FIG. 1.

[0049] According to various embodiments, the processor (220) may receive voice signals including echoes spoken by a user through a microphone (250). For example, the first microphone (250) may refer to a microphone connected to an outer hole while the electronic device (201) is worn on the user's ear. Although the electronic device (201) is illustrated in FIG. 2b as including a single microphone (250), the technical concept of the present invention may not be limited thereto. For example, the electronic device (201) may include a plurality of microphones. Additionally, the processor (220) may receive voice signals including echoes spoken by a user from a plurality of microphones.

[0050] According to various embodiments, the processor (220) may receive a vibration signal associated with a voice signal transmitted through at least a part of the user's body via a vibration sensor (260). For example, the vibration signal may be generated by the vibration of the user's vocal cords when the user speaks. For example, the vibration sensor (260) may include an accelerometer, an in-ear microphone, and / or a bone conduction microphone.

[0051] According to various embodiments, the processor (220) can predict reverberation information based on voice signals and vibration signals resulting from a user's speech. For example, the reverberation information may include the reverberation time of a transfer function (e.g., impulse response) between a voice signal corresponding to voice input to the microphone (250) and a vibration signal corresponding to vibration input to the vibration sensor (260), and / or an early-to-late reverberation ratio between early reverberation and late reverberation included in the voice information. For example, the reverberation time may mean the time it takes for the impulse response signal to decrease by a specified intensity (e.g., 60 dB).

[0052] According to various embodiments, the processor (220) can remove echoes included in the voice signal based on predicted echo information. For example, the processor (220) can determine the intensity or power of each of the initial reflection and reverberation included in the voice signal using the echo information. The processor (220) can remove echoes included in the voice signal by subtracting the intensity or power of the reverberation from the intensity or power of the voice signal.

[0053] According to various embodiments, the processor (220) may output an echo-removed voice signal through a speaker (270) (e.g., the acoustic output module (155) of FIG. 1). Alternatively, the processor (220) may transmit audio data corresponding to the echo-removed voice signal to an external electronic device (201) through a communication module (280) (e.g., the communication module (190) of FIG. 1). According to various embodiments, the processor (220) may store audio data corresponding to the echo-removed voice signal in a memory (230) (e.g., the memory (130) of FIG. 1).

[0054] According to various embodiments, the electronic device (201) may be implemented as a first-direction earphone (e.g., an earphone worn on the left ear). The electronic device (201) may be paired with a second-direction earphone (e.g., an earphone worn on the left ear). Meanwhile, for convenience of explanation, only the electronic device (201) is described in this specification, but the technical features of the electronic device (201) may be equally applied to the second-direction earphone.

[0055] The operation of the electronic device (201) described below can be controlled by a processor (220). Meanwhile, for the convenience of explanation, the electronic device (201) is described as performing the following operations, but the technical features of the present invention may be performed by a second-direction earphone paired with the electronic device (201).

[0056] FIG. 3 is a flowchart illustrating the operation of an electronic device according to various embodiments to remove echoes of a voice signal.

[0057] Referring to FIG. 3, according to various embodiments, in operation 301, the electronic device (201) can acquire a voice signal spoken by a user through a microphone (250). For example, if the electronic device (201) includes a plurality of microphones, the electronic device (201) can acquire a plurality of voice signals from the plurality of microphones. In this case, the electronic device (201) may use a signal that is the average of the plurality of voice signals to predict echo information. Alternatively, the electronic device (201) may use any one of the plurality of voice signals to predict echo information. For example, x(t), representing the voice signal acquired through the microphone (250), may be s(t)*h(t). In this case, s(t) is a voice signal spoken by a user, and h(t) may be a room impulse response.

[0058] According to various embodiments, in operation 303, the electronic device (201) may obtain a vibration signal associated with a voice signal through a vibration sensor (260). For example, y(t), representing the vibration signal obtained through the vibration sensor (260), may be s(t)*i(t). Here, s(t) is a vibration signal resulting from the user's speech, and i(t) may be a transmission path function from the user's vocal cords to the vibration sensor (260).

[0059] According to various embodiments, in operation 305, the electronic device (201) can predict echo information contained in the voice signal based on the voice signal and the vibration signal. For example, the electronic device (201) can identify the impulse response (h(t), hereinafter IR response) between the voice signal and the vibration signal. For example, the electronic device (201) can obtain the IR response (e.g., h(t)) using a normalized least mean square (NLMS) algorithm. The electronic device (201) can predict or identify echo information contained in the voice signal based on the IR response. For example, the echo information may include the reverberation time of the IR response and the ratio between the initial reflection and the reverberation.

[0060] According to various embodiments, in operation 307, the electronic device (201) can remove echoes included in the voice signal based on predicted echo information. For example, the electronic device (201) can determine the intensity (or power) of the reverberation based on the reverberation time. The electronic device (201) can remove echoes included in the voice signal by subtracting the intensity (or power) of the reverberation from the intensity (or power) of the voice signal.

[0061] FIG. 4 is a diagram illustrating the operation of an electronic device according to various embodiments to remove echoes of a voice signal.

[0062] Referring to FIG. 4, according to various embodiments, the electronic device (201) may perform an echo cancellation operation (401) to remove echoes included in a voice signal. For example, the echo cancellation operation (401) may be performed by a processor (220).

[0063] According to various embodiments, in operation 410, the electronic device (201) may filter a voice signal corresponding to voice received through the microphone (250) through a band pass filter (BPF). For example, the electronic device (201) may filter the voice signal into a specified frequency band through the BPF. According to various embodiments, the electronic device (201) may also filter a voice signal corresponding to voice received through the microphone (250) through a low pass filter (LPF) or a high pass filter (HPF).

[0064] According to various embodiments, in operation 420, the electronic device (201) may filter a vibration signal corresponding to a vibration received through a vibration sensor (260) through a band pass filter (BPF). For example, the electronic device (201) may filter the vibration signal into a specified frequency band through the BPF. According to various embodiments, the electronic device (201) may also filter a voice signal corresponding to a voice received through a microphone (250) through a low pass filter (LPF) or a high pass filter (HPF).

[0065] According to various embodiments, in operation 430, the electronic device (201) may perform pre-equalization on the vibration signal filtered by the BPF. For example, the vibration signal may have different characteristics from the voice signal. Accordingly, the electronic device (201) may perform pre-equalization to equalize the vibration signal so as to match the characteristics of the voice signal.

[0066] According to various embodiments, in operation 440, the electronic device (201) can acquire or predict an IR signal based on a voice signal and a pre-equalized vibration signal. For example, the electronic device (201) can acquire an IR signal between the voice signal and the pre-equalized vibration signal through a normalized least mean square (NLMS) adaptive filter. Additionally, the electronic device (201) can determine the reverberation time based on the IR signal. For example, the electronic device (201) can determine the time when the energy magnitude decreases by 60 dB with respect to the root mean square (RMS) graph of the IR signal as the reverberation time (e.g., RT). 60 It can be decided as ).

[0067] According to various embodiments, in operation 450, the electronic device (201) can identify reverberation (or reverberation components) in a voice signal based on reverberation time. For example, the electronic device (201) can identify the initial reflection components and reverberation components included in the voice signal based on the root mean square (RMS) graph of the IR signal.

[0068] According to various embodiments, in operation 460, the electronic device (201) can remove the reverberation included in the voice signal by subtracting the intensity (or power) of the reverberation component from the intensity (or power) of the voice signal. The electronic device (201) can output the voice signal with the reverberation removed. For example, the electronic device (201) can output the voice signal with the reverberation removed through a speaker (270). Alternatively, the electronic device (201) can transmit data corresponding to the voice signal with the reverberation removed to an external electronic device (202).

[0069] FIG. 5 is a diagram illustrating the operation of an electronic device according to various embodiments acquiring an impulse response based on a voice signal and a vibration signal.

[0070] Referring to FIG. 5, according to various embodiments, in operation 410, the electronic device (201) can filter a voice signal (e.g., x(t)=s(t)*h(t)) corresponding to voice received through the microphone (250) through a band pass filter (BPF). For example, the signal filtered by the BPF may be s_b(t)*h(t).

[0071] According to various embodiments, in operation 420, the electronic device (201) can filter a vibration signal corresponding to a vibration received through a vibration sensor (260) (e.g., y(t)=s(t)*i(t)) through a band pass filter (BPF) (e.g., output y_b(t).

[0072] According to various embodiments, in operation 430, the electronic device (201) may perform pre-equalization on a vibration signal (e.g., y_b(t)) filtered by a BPF. A path transfer function component (i(t)) included in the filtered vibration signal (e.g., y_b(t)) may be removed through pre-equalization. For example, a signal filtered by a BPF and pre-equalized may be s_b(t).

[0073] According to various embodiments, in operation 550, the electronic device (201) can predict the IR signal through an adaptive filter. For example, the adaptive filter can be implemented as an NLMS adaptive filter. In operation 560, the electronic device (201) can output an error signal (e(t)) by subtracting the magnitude of the signal output through the adaptive filter from the power of the sound source signal filtered by the BFP. The electronic device (201) can predict (or confirm) the IR signal between the voice signal and the vibration signal by controlling the error signal (e(t)) to become zero or converge to zero.

[0074] According to various embodiments, the adaptive filter may be efficiently implemented in the frequency axis using a fast Fourier transform (FFT). The electronic device (201) can efficiently predict the IR signal (h(t)) in the frequency axis by using an adaptive filter that combines an NLMS adaptive filter and a fast Fourier transform (FFT) filter. Meanwhile, in FIG. 6 below, the predicted IR signal will be defined as h'(t).

[0075] FIG. 6 is a diagram illustrating the operation of an electronic device according to various embodiments acquiring echo information based on an impulse response.

[0076] Referring to FIG. 6, according to various embodiments, an electronic device (201) can predict (or verify) an IR signal (h'(t)) through an adaptive filter (550) (e.g., an NLMS adaptive filter). The electronic device (201) can verify the root mean square (RMS) value of the IR signal (h'(t)).

[0077] According to various embodiments, the electronic device (201) can determine the reverberation time based on the RMS value (RMS) of the IR signal (h'(t)). For example, the reverberation time is the time (RT) required for the intensity of the RMS value to decrease by a specified level (e.g., 60 dB). 60 , the unit may mean seconds. However, the value of the specified level may be changed by the user or processor.

[0078] According to various embodiments, the IR signal (h'(t)) may include an initial reflection component and a reverberation component. For example, the initial reflection (or initial reflection component) may refer to a reflection (or reverberation) of the original sound that is reflected into the surrounding space and applied to the microphone before a specified time (e.g., 15 msec). The reverberation (or reverberation component) may refer to a reflection (or reverberation) of the original sound that is reflected into the surrounding space and applied to the microphone after a specified time. For example, the reverberation (or reverberation component) may cause an echo of the voice signal.

[0079] According to various embodiments, the electronic device (201) can remove the reverberation of the voice signal by subtracting the reverberation or reverberation component from the voice signal.

[0080] FIG. 7 is a flowchart illustrating the operation of an electronic device according to various embodiments removing reverberation included in a voice signal using reverberation time.

[0081] Referring to FIG. 7, according to various embodiments, in operation 701, the electronic device (201) based on the predicted IR signal between the voice signal and the vibration signal, the reverberation time (RT) 60 ) can be obtained (or predicted). The electronic device (201) can obtain (or predict) the reverberation time (RT) in the "Polack's model" of Equation 1. 60 h(t) can be checked by applying ).

[0083]

[0085] According to various embodiments, in operation 703, the electronic device (201) can identify the initial reflection and reverberation in the IR signal (h(t)). For example, the electronic device (201) can separate the IR signal (h(t)) obtained through the Pollack model of Equation 1 into an initial reflection component (h_e(t)) and a reverberation component (h_l(t)). For example, h(t) = h_e(t) + h_l(t). For example, the criterion for distinguishing the initial reflection and reverberation may be 50ms in the case of speech. For example, the criterion for distinguishing the initial reflection and reverberation may be 80ms in the case of music.

[0086] According to various embodiments, the electronic device (201) can separate the voice signal (x(t)) input to the microphone (250) into an initial reflection component (x_e(t)) and a reverberation component (x_l(t)). For example, the electronic device (201) can remove the reverberation by a spectral subtraction technique, which subtracts the power of x_l(t) from the power of x(t), because the uncorrelation between x_e(t) and x_l(t) is established by the Pollack model. At this time, the electronic device (201) can determine the power of x_l(t) using Equation 2 (e.g., PSD, power spectral density).

[0088]

[0090] According to various embodiments, the electronic device (201) can determine the intensity (or power) of the reverberation using Equation 3. For example, the electronic device (201) indicates the intensity (or power) of an IR signal (e.g., h(t) of Equation 1). In, indicating the intensity (or power) of the initial reflection and indicating the intensity (or power) of the reverberation It can be classified as such. Accordingly, the electronic device (201) indicates the intensity (or power) of the reverberation. ...can be verified. For example, the electronic device (201) is, By reflecting the reverberation time (t) in the reverberation, the intensity (or power) of the reverberation can be checked (or predicted). For example, N1 may mean the number of frames, T1 may be 50 msec, and f2 may mean the sampling frequency, which may be 8 kHz or 16 kHz.

[0092]

[0094] According to various embodiments, in operation 705, the electronic device (201) can remove reverberation from the voice signal. For example, the electronic device (201) can subtract the intensity (or power) of the reverberation from the intensity (or power) of the voice signal through spectral subtraction. For example, the electronic device (201) can remove the reverberation by a technique (spectral subtraction) that subtracts the power of x_l(t) from the power of x(t), since x_e(t) and x_l(t) are uncorrelation according to the Pollack model. Through this, the electronic device (201) can remove the reverberation contained in the voice signal.

[0095] FIG. 8 is a flowchart illustrating the operation of an electronic device according to various embodiments to remove echoes of a voice signal based on noise intensity.

[0096] Referring to FIG. 8, according to various embodiments, in operation 801, the electronic device (201) can acquire a voice signal and a vibration sensor (through a microphone and a vibration sensor) when a voice is spoken by a user.

[0097] According to various embodiments, in operation 803, the electronic device (201) can detect a voice segment of a vibration signal. For example, the electronic device (201) can detect a voice segment of a vibration signal by performing a voice activity detection (VAD) operation. For example, the electronic device (201) can detect a voice segment of a vibration signal through pre-equalizing.

[0098] According to various embodiments, the electronic device (201) can check the noise intensity of the voice signal. For example, in operation 805, the electronic device (805) can check whether the noise intensity is greater than a specified first value. For example, the specified first value may be a value indicating a noise intensity such that the reverberation of the voice signal can be ignored. For example, the specified first value may be a value indicating a noise intensity in a state where the SNR is 0dB or less. For example, the specified first value may be determined by a user or automatically determined by a processor (220).

[0099] According to various embodiments, if the noise intensity is greater than a specified first value (e.g., operation 805), in operation 807, the electronic device (201) may not remove the echo included in the voice signal.

[0100] According to various embodiments, if the noise intensity is not greater than a specified first value (No in operation 805), in operation 809, the electronic device (201) can determine whether the reverberation intensity is greater than a specified second value. For example, the reverberation intensity may refer to the intensity (or power) of the reverberation contained in the IR signal. For example, the specified second value may be a value representing a reverberation intensity such that the reverberation of the voice signal can be ignored. For example, the specified second value may be determined by the user or automatically determined by the processor (220).

[0101] According to various embodiments, if the reverberation intensity is not greater than a specified second value (no in operation 809), in operation 807, the electronic device (201) may not remove the reverberation included in the voice signal.

[0102] According to various embodiments, if the reverberation intensity is greater than a specified second value (e.g., in operation 809), the electronic device (201) may remove the reverberation included in the voice signal in operation 811. For example, the electronic device (201) may remove the reverberation included in the voice signal by subtracting the intensity (or power) of the reverberation from the intensity (or power) of the voice signal. According to one embodiment, the electronic device (201) may determine the amount of reverberation included in the voice signal to be removed based on the noise intensity. For example, if the noise intensity is relatively large (e.g., less than a specified first value but greater than a specified third value), the reverberation component tends to be removed by the noise, thereby reducing the amount of reverberation removed. For example, the electronic device (201) may apply a certain weight (e.g., the weight may be greater than 0 and less than 1) to the intensity of the reverberation subtracted from the intensity of the voice signal.

[0103] According to various embodiments, in operation 813, the electronic device (201) can remove noise included in the voice signal.

[0104] According to various embodiments, in operation 815, the electronic device (201) may output a voice corresponding to a voice signal after removing noise. For example, the electronic device (201) may output a voice through a speaker (270). Alternatively, the electronic device (201) may transmit data corresponding to a voice signal to an external electronic device (202) through a communication module (280). At this time, the external electronic device (202) may output a voice corresponding to a voice signal through a speaker included in the external electronic device (202). Alternatively, the external electronic device (202) may transmit data corresponding to a voice signal to the electronic device of the other party communicating with the external electronic device (202).

[0105] FIG. 9 is a flowchart illustrating the operation of an electronic device according to various embodiments to remove echoes of a voice signal based on echo intensity.

[0106] Referring to FIG. 9, according to various embodiments, in operation 901, the electronic device (201) can acquire a voice signal and a vibration sensor (through a microphone and a vibration sensor) when a voice is spoken by a user.

[0107] According to various embodiments, in operation 903, the electronic device (201) can detect a voice segment of a vibration signal. For example, the electronic device (201) can detect a voice segment of a vibration signal by performing a voice activity detection (VAD) operation. For example, the electronic device (201) can detect a voice segment of a vibration signal through pre-equalizing.

[0108] According to various embodiments, the electronic device (201) can check the reverberation strength of the voice signal. For example, the electronic device (201) can check whether the reverberation strength is greater than a specified second value in operation 905. For example, the reverberation strength may refer to the strength (or power) of the reverberation included in the IR signal.

[0109] According to various embodiments, if the noise intensity is not greater than the specified second value (no in operation 905), in operation 907, the electronic device (201) may not remove the echo included in the voice signal.

[0110] According to various embodiments, if the reverberation intensity is greater than a specified second value (e.g., operation 905), in operation 909, the electronic device (201) can remove the reverberation included in the voice signal. For example, the electronic device (201) can remove the reverberation included in the voice signal by subtracting the intensity (or power) of the reverberation from the intensity (or power) of the voice signal.

[0111] According to various embodiments, in operation 911, the electronic device (201) can remove noise included in the voice signal.

[0112] According to various embodiments, in operation 913, the electronic device (201) may output a voice corresponding to a voice signal after removing noise. For example, the electronic device (201) may output a voice through a speaker (270). Alternatively, the electronic device (201) may transmit data corresponding to a voice signal to an external electronic device (202) through a communication module (280). At this time, the external electronic device (202) may output a voice corresponding to a voice signal through a speaker included in the external electronic device (202). Alternatively, the external electronic device (202) may transmit data corresponding to a voice signal to the electronic device of the other party communicating with the external electronic device (202).

[0113] FIGS. 10a through 10c are drawings illustrating the operation of an electronic device according to various embodiments to remove echoes included in a voice signal.

[0114] Referring to FIG. 10a, an electronic device (201) can acquire a voice signal (1010) corresponding to a voice generated by a user's speech in a specific space through a microphone (250). For example, the voice signal (1010) may include an echo (or echo component) reflected in the specific space. For example, the shape of the voice signal may change as the specific space changes.

[0115] Referring to FIG. 10b, the electronic device (201) can acquire a vibration signal (1020) corresponding to voice generated by a user's speech in a specific space through a vibration sensor (260). For example, the vibration signal (1020) may not include or may hardly include any reflection (or reflection component) reflected in the specific space. For example, even if the specific space changes, the form of the voice signal may not change significantly.

[0116] Referring to FIG. 10c, the electronic device (201) can remove echoes included in the voice signal (1010) based on the voice signal (1010) and the vibration signal (1020). The electronic device (201) can obtain a signal (1030) from which echoes have been removed from the voice signal (1010).

[0117] According to various embodiments, the removed signal (1030) may be used in an application for voice calls and / or voice recognition. By doing so, the electronic device (201) can improve the sound quality of the user's voice in an echoing environment.

[0119] An electronic device (201) according to various embodiments includes a vibration sensor (260), a microphone (250), and a processor (220), wherein the processor may be configured to receive a voice signal including an echo uttered by a user through the microphone, receive a vibration signal associated with the voice signal transmitted through at least a part of the user's body through the vibration sensor, predict echo information based on the voice signal and the vibration signal, and remove the echo included in the voice signal based on the predicted echo information.

[0120] The processor may be configured to check an impulse response signal between the voice signal and the vibration signal, and to predict the echo information based on the impulse response.

[0121] The above reverberation information may include the reverberation time of the impulse response and / or the early-to-late reverberation ratio between the early reverberation and the late reverberation included in the speech information.

[0122] The above reverberation time may be the time it takes for the impulse response signal based on the voice signal and the vibration signal to decrease by a specified intensity.

[0123] The processor may be configured to check the intensity of the initial reflection and reverberation included in the voice signal, respectively, using the reverberation information, and to remove the reverberation included in the voice signal by subtracting the intensity of the reverberation from the intensity of the voice signal.

[0124] The processor may be configured to check the noise intensity of the voice signal, and if the noise intensity is not greater than a specified value, to remove the echo included in the voice signal.

[0125] The processor may be configured to check the strength of the echo included in the voice signal, and if the strength of the echo is greater than a specified value, to remove the echo included in the voice signal.

[0126] The above processor may be configured to remove noise included in the voice signal without removing the echo included in the voice signal if the intensity of the echo is not greater than a specified value.

[0127] The processor may be configured to remove noise included in the voice signal without removing the echo included in the voice signal if the noise intensity is greater than a specified value.

[0128] The processor may be configured to determine the amount of reverberation included in the voice signal based on the noise intensity.

[0129] A method of operation of an electronic device (201) according to various embodiments may include receiving a voice signal including an echo uttered by a user through a microphone (250) included in the electronic device, receiving a vibration signal related to the voice signal transmitted through at least a part of the user's body through a vibration sensor (260) included in the electronic device, predicting echo information based on the voice signal and the vibration signal, and removing the echo included in the voice signal based on the predicted echo information.

[0130] The operation of predicting the above-mentioned echo information may include the operation of confirming an impulse response signal between the voice signal and the vibration signal, and the operation of predicting the echo information based on the impulse response.

[0131] The above reverberation information may include the reverberation time of the impulse response and / or the early-to-late reverberation ratio between the early reverberation and the late reverberation included in the speech information.

[0132] The above reverberation time may be the time it takes for the impulse response signal based on the voice signal and the vibration signal to decrease by a specified intensity.

[0133] The operation of removing the above-mentioned echo may include the operation of checking the intensity of the initial reflection and the reverberation included in the voice signal using the echo information, and the operation of removing the echo included in the voice signal by subtracting the intensity of the reverberation from the intensity of the voice signal.

[0134] The operation of removing the above echo may include an operation of checking the noise intensity of the voice signal and, if the noise intensity is not greater than a specified value, an operation of removing the echo included in the voice signal.

[0135] The operation of removing the above echo may include an operation of checking the strength of the echo included in the voice signal and, if the strength of the echo is greater than a specified value, an operation of removing the echo included in the voice signal.

[0136] The method of operation of the electronic device may further include, if the intensity of the reverberation is not greater than a specified value, an operation of removing noise included in the voice signal without removing the reverberation included in the voice signal.

[0137] The method of operation of the electronic device may further include, if the noise intensity is greater than a specified value, an operation of removing noise included in the voice signal without removing the echo included in the voice signal.

[0138] A non-transient recording medium according to various embodiments may store a program capable of performing the following operations: receiving a voice signal including an echo uttered by a user through a microphone included in an electronic device; receiving a vibration signal related to the voice signal transmitted through at least a part of the user's body through a vibration sensor included in the electronic device; predicting echo information based on the voice signal and the vibration signal; and removing an echo included in the voice signal based on the predicted echo information. Explanation of the symbols

[0140] 201: Electronic device 202: External electronic device 220: Processor 230: Memory 250: Microphone 260: Vibration sensor 270: Speaker 280: Communication module

Claims

Claim 1 An electronic device comprising: a vibration sensor; a microphone; and a processor, wherein the processor receives a voice signal including an echo uttered by a user through the microphone, and receives a vibration signal associated with the voice signal transmitted through at least a part of the user's body through the vibration sensor, and determines an impulse response signal between the voice signal and the vibration signal, and predicts reverberation information based on the impulse response signal, respectively determines the intensities of an early reverberation and a late reverberation included in the voice signal based on the predicted reverberation information, and is configured to remove the reverberation included in the voice signal by subtracting the intensity of the reverberation from the intensity of the voice signal, wherein the reverberation information includes the reverberation time of the impulse response and / or the ratio between the early reverberation and the reverberation included in the voice signal. Claim 2 delete Claim 3 delete Claim 4 An electronic device according to claim 1, wherein the reverberation time is the time required for an impulse response signal based on the voice signal and the vibration signal to decrease by a specified intensity. Claim 5 delete Claim 6 In claim 1, the processor is an electronic device configured to check the noise intensity of the voice signal and, if the noise intensity is not greater than a specified value, to remove the echo included in the voice signal. Claim 7 In claim 6, the processor is an electronic device configured to check the strength of the echo included in the voice signal and, if the strength of the echo is greater than a specified value, to remove the echo included in the voice signal. Claim 8 In claim 7, the processor is an electronic device configured to remove noise included in the voice signal without removing the echo included in the voice signal if the intensity of the echo is not greater than a specified value. Claim 9 In claim 6, the processor is an electronic device configured to remove noise included in the voice signal without removing the echo included in the voice signal when the noise intensity is greater than a specified value. Claim 10 In paragraph 6, the processor is an electronic device configured to determine the amount of reverberation included in the voice signal based on the noise intensity. Claim 11 A method of operating an electronic device, comprising: receiving a voice signal including an echo uttered by a user through a microphone included in the electronic device; receiving a vibration signal related to the voice signal transmitted through at least a part of the user's body through a vibration sensor included in the electronic device; identifying an impulse response signal between the voice signal and the vibration signal; predicting reverberation information based on the impulse response signal; identifying the intensities of an early reverberation and a late reverberation included in the voice signal, respectively, based on the predicted reverberation information; and removing the reverberation included in the voice signal by subtracting the intensity of the reverberation from the intensity of the voice signal, wherein the reverberation information includes the reverberation time of the impulse response and / or the early-to-late reverberation ratio between the early reverberation and the reverberation included in the voice signal. Claim 12 delete Claim 13 delete Claim 14 An electronic device according to claim 11, wherein the reverberation time is the time required for an impulse response signal based on the voice signal and the vibration signal to decrease by a specified intensity. Claim 15 delete Claim 16 A method of operation of an electronic device according to claim 11, wherein the operation of removing the echo comprises: an operation of checking the noise intensity of the voice signal; and an operation of removing the echo included in the voice signal if the noise intensity is not greater than a specified value. Claim 17 A method of operation of an electronic device according to claim 16, wherein the operation of removing the echo comprises: an operation of checking the strength of the echo included in the voice signal; and an operation of removing the echo included in the voice signal if the strength of the echo is greater than a specified value. Claim 18 A method of operation of an electronic device according to claim 17, wherein if the intensity of the reverberation is not greater than a specified value, the reverberation included in the voice signal is not removed, and noise included in the voice signal is further included. Claim 19 A method of operation of an electronic device according to claim 16, wherein if the noise intensity is greater than a specified value, the operation of removing noise included in the voice signal without removing the echo included in the voice signal. Claim 20 A recording medium that stores a program capable of performing the following operations: receiving a voice signal including an echo uttered by a user through a microphone included in an electronic device; receiving a vibration signal related to the voice signal transmitted through at least a part of the user's body through a vibration sensor included in the electronic device; identifying an impulse response signal between the voice signal and the vibration signal; predicting reverberation information based on the impulse response signal; identifying the intensities of early reverberation and late reverberation included in the voice signal, respectively, based on the predicted reverberation information; and removing the reverberation included in the voice signal by subtracting the intensity of the reverberation from the intensity of the voice signal, wherein the reverberation information includes the reverberation time of the impulse response and / or the ratio between the early reverberation and the reverberation included in the voice signal (early-to-late reverberation ratio).

Images