Electronic device for improving voice recognition rate and operation method thereof

Abstract

Disclosed are an electronic device for improving a voice recognition rate and an operation method thereof. The electronic device may comprise: a memory for storing instructions; and at least one processor for executing the instructions, wherein when the at least one processor individually and / or collectively executes the instructions, the instructions cause the electronic device to: receive an audio signal from the outside of the electronic device; identify a target voice signal and an interference signal from the audio signal; insert a mask signal into the audio signal; and provide a service using the target voice signal on the basis of the audio signal having the mask signal inserted therein.

Description

Electronic device for improving speech recognition rate and method of operation thereof

[0001] An electronic device for improving voice recognition rate and an operating method thereof are disclosed.

[0002] Automatic Speech Recognition (ASR) technology allows computers to recognize human speech and convert it into text. ASR technology can be applied in a variety of fields. For example, it can be used in services provided by electronic devices, such as automatic subtitle generation, natural language processing, or voice control. Accurately converting human speech into text requires accurate recognition of the human voice. Therefore, it can be crucial to recognize only the target speech signal from the audio signal.

[0003] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art related to the present disclosure.

[0004] According to one embodiment, an electronic device may include a memory that stores instructions. The electronic device may include at least one processor that executes the instructions. When the at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to receive an audio signal from an external source of the electronic device. When the at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to identify the target voice signal and an interference signal from the audio signal. When the at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to insert a mask signal into the audio signal. When the at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to provide a service that utilizes the target voice signal based on the audio signal into which the mask signal has been inserted.

[0005] According to one embodiment, an electronic device may include a memory that stores instructions. The electronic device may include at least one processor that executes the instructions. When the at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to provide a voice call service with a second speaker to a first speaker who is a user of the electronic device. When the at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to receive an audio signal from an external electronic device of the second speaker while providing the voice call service. When the at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to identify the target voice signal and the interference signal from the audio signal. When the at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to insert a masking signal into the audio signal. When said at least one processor individually and / or collectively executes said instructions, said instructions may cause said electronic device to generate text corresponding to said target speech signal using said audio signal into which said mask signal has been inserted.

[0006] According to one embodiment, a method of operating an electronic device may include receiving an audio signal from an external source of the electronic device. The method of operating the electronic device may include identifying the target audio signal and the interference signal from the audio signal. The method of operating the electronic device may include inserting a mask signal into the audio signal. The method of operating the electronic device may include providing a service using the target audio signal based on the audio signal into which the mask signal has been inserted.

[0007] According to one embodiment, a method of operating an electronic device may include providing a voice call service with a second speaker to a first speaker who is a user of the electronic device. The method of operating the electronic device may include receiving an audio signal from an external electronic device of the second speaker while providing the voice call service. The method of operating the electronic device may include identifying the target voice signal and the interference signal from the audio signal. The method of operating the electronic device may include inserting a mask signal into the audio signal. The method of operating the electronic device may include generating a text corresponding to the target voice signal using the audio signal into which the mask signal has been inserted.

[0008] According to one embodiment, a non-transitory computer-readable recording medium may store one or more computer programs including instructions. When at least one processor individually and / or collectively executes the instructions, the instructions may cause an electronic device to receive an audio signal from an external source of the electronic device. When at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to identify the target audio signal and the interference signal from the audio signal. When at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to insert a mask signal into the audio signal. When at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to provide a service using the target audio signal based on the audio signal into which the mask signal has been inserted.

[0009] According to one embodiment, a non-transitory computer-readable recording medium may store one or more computer programs including instructions. When at least one processor individually and / or collectively executes the instructions, the instructions may cause an electronic device to provide a voice call service with a second speaker to a first speaker who is a user of the electronic device. When the at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to receive an audio signal from an external electronic device of the second speaker while providing the voice call service. When the at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to identify the target voice signal and the interference signal from the audio signal. When the at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to insert a masking signal into the audio signal. When said at least one processor individually and / or collectively executes said instructions, said instructions may cause said electronic device to generate text corresponding to said target speech signal using said audio signal into which said mask signal has been inserted.

[0010] FIG. 1 is a block diagram of an electronic device within a network environment according to various embodiments.

[0011] FIG. 2 is a drawing for explaining a voice call service provided by an electronic device according to one embodiment of the present disclosure.

[0012] FIG. 3 and FIG. 4 are flowcharts for explaining an operating method of an electronic device according to one embodiment of the present disclosure.

[0013] FIG. 5 and FIG. 6 are diagrams for explaining identification of a target voice signal according to one embodiment of the present disclosure.

[0014] FIG. 7 is a diagram for explaining insertion of a mask signal according to one embodiment of the present disclosure.

[0015] FIG. 8 is a block diagram illustrating the operation of an electronic device according to one embodiment of the present disclosure.

[0016] FIG. 9 is a flowchart for explaining an operating method of an electronic device according to one embodiment of the present disclosure.

[0017] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are assigned the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted.

[0018] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with the electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of the electronic device (104) or the server (108) via a second network (199) (e.g., a long-range wireless communication network). In one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0019] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134).

[0020] According to one embodiment, the processor (120) may be implemented as a circuit (e.g., a processing circuit) such as a system on chip (SoC) or an integrated circuit (IC). The processor (120) may include one or more processors. For example, the processor (120) may include a combination of one or more processors such as a CPU, a GPU, an MPU, an AP, and a CP.

[0021] According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0022] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, in the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include a plurality of artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0023] The memory (130) can store various data used by at least one component (e.g., the processor (120) or the sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., the program (140)) and input data or output data for commands related thereto. According to one embodiment, the memory (130) can include one or more memories. The instructions stored in the memory (130) can be stored in one memory. The instructions stored in the memory (130) can be divided and stored in multiple memories. The memory (130) can include a volatile memory (132) or a nonvolatile memory (134).

[0024] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0025] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0026] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0027] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. According to one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0028] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0029] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0030] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0031] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0032] The haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. According to one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0033] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0034] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

[0035] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0036] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0037] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0038] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas, for example, by the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device via the at least one selected antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197). According to various embodiments, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0039] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0040] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0041] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0042] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0043] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0044] Various embodiments of the present document may be implemented as software (e.g., a program (140)) including one or more instructions stored in a storage medium (e.g., an internal memory (136) or an external memory (138)) readable by a machine (e.g., an electronic device (101)). For example, a processor (e.g., a processor (120)) of the machine (e.g., an electronic device (101)) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0045] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as a computer program product. The computer program product may be traded between sellers and buyers as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or may be provided through an application store (e.g., Play Store). TM ) or directly between two user devices (e.g., smart phones), online distribution (e.g., downloading or uploading). In the case of online distribution, at least a portion of the computer program product may be at least temporarily stored or temporarily created in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0046] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0047]

[0048] FIG. 2 is a drawing for explaining a voice call service provided by an electronic device according to one embodiment of the present disclosure.

[0049] ASR technology can recognize human speech and convert it into text. ASR technology can be utilized in a variety of fields. For example, ASR technology can be utilized in services such as voice assistants, automatic subtitle generation, natural language processing, voice control, and simultaneous interpretation. For example, text corresponding to human speech acquired using ASR technology can serve as the basis for various services. Therefore, when an audio signal includes various signals in addition to the human speech signal, accurately recognizing only the human speech signal can be crucial.

[0050] For convenience of explanation, the following description assumes that the electronic device (200) simultaneously provides voice call services and simultaneous interpretation services. To provide simultaneous interpretation services, the conversation between the first speaker (220) and the second speaker (230) may first need to be converted into text. For example, ASR technology may be utilized.

[0051] However, it is obvious to those skilled in the art that the following description is not necessarily applicable only to audio signals received from an external electronic device (210), but can also be equally applied to audio signals directly acquired by an electronic device (200) (e.g., electronic device (101) of FIG. 1) to provide services such as the above-described voice assistant, automatic subtitle generation, natural language processing, and voice control.

[0052] Referring to FIG. 2, an electronic device (200) and an external electronic device (210) (e.g., electronic devices (102, 104) of FIG. 1) are illustrated. The electronic device (200) and the external electronic device (210) can communicate via a network (240) (e.g., the first network (198) of FIG. 1 and the second network (199) of FIG. 1). The first speaker (220) can be a user on the electronic device (200) side. The second speaker (230) can be a user on the external electronic device (210) side.

[0053] According to one embodiment, the electronic device (200) can provide a voice call service to a first speaker (220). For example, the electronic device (200) can provide a voice call service to a second speaker (230) via a network (240) to the first speaker (220). With respect to the electronic device (200), the first speaker (220) side can be referred to as the near-end, and the second speaker (230) side can be referred to as the far-end.

[0054] The electronic device (200) can obtain an audio signal from an external electronic device (210) via a network (240) in order to provide a voice call service. The electronic device (200) can provide the audio signal to a first speaker (220) via a speaker (e.g., audio module (170) of FIG. 1).

[0055] In one embodiment, the audio signal may include a voice signal of a second speaker (230) collected by an external electronic device (210), as well as a interference signal that interferes with a voice call service. The voice signal may be the voice of the second speaker related to a conversation between the first speaker (220) and the second speaker (230). The interference signal may be a signal unrelated to the conversation between the first speaker (220) and the second speaker (230) that interferes with the conversation.

[0056] In one embodiment, the interference signal may include a signal collected from an environment in which the external electronic device (210) is providing a voice call service and / or a signal collected from a third speaker unrelated to the voice call service. For example, the interference signal may include a signal collected from an environment in which the external electronic device (210) is providing a voice call service and / or a signal collected from a third speaker unrelated to a conversation between the first speaker (220) and the second speaker (230) and unrelated to the conversation and located adjacent to the external electronic device (210).

[0057] According to one embodiment, if the audio signal includes a voice signal of a third speaker, the electronic device (200) may convert even the voice signal of the third speaker into text. Therefore, the quality of the service provided based on the converted text may be degraded. Therefore, in order to accurately obtain the text corresponding to the voice of the second speaker (230) from the audio signal received by the electronic device (200), the electronic device (200) may need to be prevented from recognizing the interference signal. A method for preventing the recognition of the interference signal will be described later with reference to FIGS. 3 to 7 .

[0058] According to one embodiment, the electronic device (200) may provide services that do not require the reception of audio signals from an external electronic device (210), such as voice assistants and voice control. For example, unlike simultaneous interpretation services, the electronic device (200) may provide a service to the user based on an audio signal obtained directly from the user (e.g., the first speaker (220)). The audio signal may include the user's voice signal and / or a interference signal. The description of the interference signal is omitted as described above. In this case, the electronic device (200) may need to be prevented from recognizing the interference signal in order to accurately obtain text corresponding to the user's voice signal. It will be apparent to those skilled in the art that the method for preventing the recognition of the interference signal described in FIGS. 3 to 7 can be equally applied in this case.

[0059] According to one embodiment, the electronic device (200) is a device that provides a service based on a voice signal to a user (e.g., a first speaker (220)), and may include, for example, various computing devices such as a mobile phone, a smart phone, a tablet, an e-book device, a laptop, a personal computer, a desktop, a workstation, or a server, various wearable devices such as a smart watch, smart glasses, or a head-mounted display (HMD), various home appliances such as a smart speaker, a smart TV, or a smart refrigerator, a smart car, a smart kiosk, an IoT (internet of things) device, a WAD (walking assist device), a drone, or a robot.

[0060] According to one embodiment, the external electronic device (210) may represent a terminal on the other side (e.g., the second speaker (230)) of the first speaker (220) in a voice call. The external electronic device (210) may acquire an audio signal through a microphone. The external electronic device (210) may transmit the audio signal to the electronic device (200) through a network (240). A description of the audio signal is omitted as described above.

[0061] According to one embodiment, the network (240) can connect the electronic device (200) and the external electronic device (210) to enable voice calls. For example, the network (240) can provide a voice call service between the electronic device (200) and the external electronic device (210). The network (240) can take the form of a wired communication network and / or a wireless communication network, depending on the embodiment. For example, the network (240) can provide a voice call service using various voice call providing methods such as voice over internet protocol (VoIP), voice over LTE (VoLTE), voice over new radio (VoNR), public switched telephone network (PSTN), and Wi-Fi calling.

[0062] Below, the operating method of the electronic device (200) will be described.

[0063]

[0064] FIG. 3 and FIG. 4 are flowcharts for explaining an operating method of an electronic device according to one embodiment of the present disclosure.

[0065] Referring to FIG. 3, a flowchart illustrating a method of operating an electronic device is illustrated.

[0066] The operations described below may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. Furthermore, some operations may be omitted according to some embodiments. Operations (310) to (340) may be performed by at least one component (e.g., the processor (120) of FIG. 1) of an electronic device (e.g., the electronic device (101) of FIG. 1 and the electronic device (200) of FIG. 2). For example, instructions stored in a memory (e.g., the memory (130) of FIG. 1) by at least one processor may be individually and / or collectively executed, and the instructions may cause the electronic device to perform operations (310) to (340) below.

[0067] In operation (310), the electronic device can receive an audio signal from outside the electronic device.

[0068] According to one embodiment, an electronic device can provide a voice call service to a first speaker (e.g., the first speaker (220) of FIG. 2) with a second speaker (e.g., the second speaker (230) of FIG. 2). The electronic device can receive an audio signal from an external electronic device (e.g., the electronic devices (102, 104) of FIG. 1 and the external electronic device (210) of FIG. 2), which is a terminal of the second speaker. A description of the audio signal received from the external electronic device is omitted, as described above with reference to FIG. 2.

[0069] According to one embodiment, an electronic device can receive a digital signal from an external electronic device. For example, the digital signal may be an audio signal acquired by the external electronic device, converted into a digital signal, and then encoded using a vocoder. The electronic device can receive the encoded digital signal. The electronic device can decode the encoded digital signal using the vocoder to obtain a digital signal. The electronic device can obtain an audio signal by converting the digital signal into an analog signal.

[0070] In one embodiment, an audio signal received by an electronic device from an external electronic device may have undergone noise reduction processing. For example, the external electronic device may initially perform noise reduction processing on an audio signal acquired for a voice call service. This may be to improve call quality. Therefore, even if noise reduction processing has been performed, the audio signal may still contain the target voice signal and interference signals.

[0071] According to one embodiment, an electronic device can acquire an audio signal containing a voice signal of a user of the electronic device (e.g., a first speaker). For example, if the electronic device provides a service that does not require reception of an audio signal from an external electronic device, the electronic device can directly acquire the audio signal. The description of the electronic device directly acquiring the audio signal is omitted, as described above in FIG. 2.

[0072] In operation (320), the electronic device can identify a target voice signal and a interference signal from an audio signal.

[0073] In one embodiment, an electronic device can identify a target voice signal and a nuisance signal from an audio signal. The target voice signal may be a signal used for a service. For example, the target voice signal may be the voice of a first speaker or a second speaker. For example, if the electronic device receives an audio signal from an external electronic device, the target voice signal may be the voice of the second speaker. For example, if the electronic device directly collects the audio signal, the target voice signal may be the voice of the first speaker.

[0074] The method for identifying target voice signals and interference signals will be described later in Fig. 5.

[0075] In operation (330), the electronic device can insert a mask signal into the audio signal.

[0076] Electronic devices can mask interference signals by inserting a masking signal. For example, an electronic device can mask an interference signal by inserting a masking signal. By masking the interference signal, the electronic device can recognize only the target voice signal. By masking the interference signal, the recognition rate of the target voice signal can be increased. A method for masking the interference signal will be described later in Figure 7.

[0077] In operation (340), the electronic device can provide a service using a target voice signal based on an audio signal into which a mask signal has been inserted.

[0078] According to one embodiment, an electronic device can recognize a target speech signal based on an audio signal into which a mask signal has been inserted. The electronic device can convert the target speech signal into text. The electronic device can provide various services using the text converted from the target speech signal. For example, the electronic device can provide a simultaneous interpretation service along with a voice call service using the text converted from the target speech signal. For example, if the target speech signal includes a control command for the electronic device, the electronic device can control the electronic device based on the converted text.

[0079] By inserting a masking signal into an audio signal, the recognition rate of the target speech signal in an electronic device can be increased, and the accuracy of the text converted from the target speech signal can be improved accordingly. Consequently, the electronic device can provide more accurate services.

[0080] Below, we will explain how to identify a target voice signal from an audio signal.

[0081] Referring to FIG. 4, a flowchart is shown for explaining a method for identifying a target voice signal and a interference signal according to one embodiment of the present disclosure.

[0082] The operations described below may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. Furthermore, some operations may be omitted according to some embodiments. Operations (410) to (440) may be performed by at least one component (e.g., the processor (120) of FIG. 1) of an electronic device (e.g., the electronic device (101) of FIG. 1 and the electronic device (200) of FIG. 2). For example, instructions stored in a memory (e.g., the memory (130) of FIG. 1) by at least one processor may be individually and / or collectively executed, and the instructions may cause the electronic device to perform operations (410) to (440) below.

[0083] In operation (410), the electronic device can perform independent component analysis on the audio signal to determine a target speech signal and an interference signal from the audio signal.

[0084] According to one embodiment, an electronic device can perform independent component analysis (ICA) on an audio signal. The electronic device can extract at least one component from the audio signal through ICA. The electronic device can determine a target speech signal or an interference signal for each of the at least one component.

[0085] According to one embodiment, the electronic device may determine a component among at least one component that has a relatively high level compared to other components as the target speech signal. For example, the electronic device may determine the component among at least one component that has the highest level as the target speech signal. For example, the electronic device may determine the component among at least one component that has the highest average level as the target speech signal. However, this is merely an example, and the present disclosure is not limited thereto. The electronic device may determine the remaining components, excluding the component determined as the target speech signal, as interference signals. The target speech signal is a speech signal of a second speaker (e.g., the second speaker (230) of FIG. 2), and an external electronic device (e.g., the electronic devices (102, 104) of FIG. 1 and the external electronic device (210) of FIG. 2) may receive the target speech signal generated at a location adjacent to the external electronic device. The interference signal may include a signal collected from an environment in which the external electronic device is providing a voice call service and / or a signal collected from a third speaker unrelated to the voice call service. Therefore, external electronic devices can receive interference signals generated at a greater distance than the target voice signal. For example, the level (e.g., loudness) of the target voice signal may be greater than that of the interference signal, so the electronic device can determine the component with a relatively large level as the target voice signal.

[0086] In operation (420), the electronic device can determine whether the target voice signal and the interference signal are properly determined.

[0087] According to one embodiment, the electronic device may determine whether the target speech signal extracted in operation (410) includes a speech signal of a second speaker (e.g., the second speaker (230) of FIG. 2 ). If the target speech signal and the interference signal are properly extracted, the electronic device may perform operation (440). If the target speech signal and the interference signal are not properly determined, the electronic device may perform operation (430).

[0088] According to one embodiment, if the audio signal includes a signal having a higher level than the second speaker's speech signal, the electronic device may determine the signal having a higher level than the second speaker's speech signal as the target speech signal in operation (410). For example, the target speech signal may not be properly extracted. Therefore, to properly extract the target speech signal, the electronic device may perform operation (430).

[0089] In operation (430), the electronic device can input an audio signal to the learned artificial intelligence model to determine a target voice signal and a nuisance signal.

[0090] The trained AI model may be trained to extract the target speech signal (the second speaker's speech signal) and the interference signal when receiving an audio signal. The method of utilizing the AI ​​model will be described later in Figure 6.

[0091] In operation (440), the electronic device can identify a target voice signal and a interference signal in an audio signal.

[0092] If the target voice signal and the interference signal are properly extracted in operation (420), the electronic device can finally identify the extracted target voice signal and the interference signal as the target voice signal and the interference signal.

[0093] If the target voice signal and the interference signal are not properly extracted in operation (420), the electronic device can finally identify the target voice signal and the interference signal output from the learned artificial intelligence model as the target voice signal and the interference signal in operation (430).

[0094] When the electronic device identifies a target voice signal and an interference signal in an audio signal, it can insert a mask signal into the audio signal. The insertion of the mask signal will be described later in FIG. 7.

[0095] Below, we will explain the identification of the target voice signal using drawings.

[0096]

[0097] FIG. 5 and FIG. 6 are diagrams for explaining identification of a target voice signal according to one embodiment of the present disclosure.

[0098] Referring to FIG. 5, an audio signal (500) received from the outside by an electronic device (e.g., the electronic device (101) of FIG. 1 and the electronic device (200) of FIG. 2) is illustrated. The audio signal (500) may be a signal collected by an external electronic device (e.g., the electronic devices (102, 104) of FIG. 1 and the external electronic device (210) of FIG. 2) or a signal collected by the electronic device. The audio signal (500) may include a interference signal and a target voice signal. The description of the audio signal (500) will be omitted as described above with reference to FIGS. 2 and 3.

[0099] According to one embodiment, an electronic device can perform independent component analysis on an audio signal (500). For convenience of explanation, in this disclosure, it is assumed that two components (e.g., signals) are extracted when independent component analysis is performed on an audio signal (500). However, this is merely an example, and the present disclosure is not limited thereto.

[0100] In one embodiment, the electronic device can determine a target speech signal and a interference signal for two components (e.g., a first component (510) and a second component (520)). In one embodiment, the electronic device can determine the target speech signal and the interference signal based on the levels of the components.

[0101] According to one embodiment, the electronic device can analyze the levels of the components. According to one embodiment, the electronic device can determine the average level of the peak values ​​included in each component. According to one embodiment, the electronic device can determine the maximum peak value of each component. According to one embodiment, the electronic device can determine the average level of the valley values ​​included in each component. According to one embodiment, the electronic device can determine the maximum valley value of each component. However, this is merely an example and the present disclosure is not limited thereto.

[0102] According to one embodiment, the electronic device may analyze the levels of components and determine the component with the largest level among the components as the target speech signal. The electronic device may determine the remaining components other than the component determined as the target speech signal as interference signals. According to one embodiment, the electronic device may determine the component with the largest average level of peak values ​​as the target speech signal. According to one embodiment, the electronic device may determine the component with the largest maximum peak value as the target speech signal. According to one embodiment, the electronic device may determine the component with the largest average level of valley values ​​as the target speech signal. According to one embodiment, the electronic device may determine the component with the largest maximum valley value as the target speech signal. However, this is merely an example and the present disclosure is not limited thereto.

[0103] Accordingly, the electronic device can identify the first component (510), which is the component with the relatively largest level among the first component (510) and the second component (520), as the target voice signal. The electronic device can identify the remaining component, the second component (520), as the interference signal.

[0104] Below, we will explain the case where the target voice signal is not properly determined among the extracted components.

[0105] Referring to FIG. 6, an audio signal (600) (e.g., an audio signal (500) of FIG. 5) received from an external source by an electronic device (e.g., an electronic device (101) of FIG. 1 and an electronic device (200) of FIG. 2) is illustrated. The audio signal (600) may be a signal collected by an external electronic device (e.g., an electronic device (102, 104) of FIG. 1 and an external electronic device (210) of FIG. 2) or a signal collected by the electronic device. The audio signal (600) may include a interference signal and a target voice signal. The description of the audio signal (600) as described above with reference to FIGS. 2 and 3 will be omitted.

[0106] According to one embodiment, the electronic device can perform independent component analysis on an audio signal (600). In the present disclosure, for convenience of explanation, it will be assumed that when independent component analysis is performed on an audio signal (600), three components (e.g., a first component (610) (e.g., the first component (510) of FIG. 5 ), a second component (620) (e.g., the second component (520) of FIG. 5 ), or a third component (630)) are extracted. However, this is merely an example, and the present disclosure is not limited thereto.

[0107] The electronic device can determine the target voice signal and interference signal based on the extracted components. The method for determining the target voice signal and interference signal is described above in FIG. 5, so a detailed description thereof will be omitted.

[0108] According to one embodiment, the third component (630) may be determined as the target speech signal according to the method described in FIG. 5. However, since the third component (630) is a signal collected one-time within the audio signal (600), it may not be a speech signal of a first speaker (e.g., the first speaker (220) of FIG. 2) or a second speaker (e.g., the second speaker (230) of FIG. 2). For example, the third component (630) may be a signal collected from an external electronic device, such as a car horn, or an environment in which the electronic device is providing a service. For example, the third component (630) may be a signal collected from an external electronic device or a third speaker located adjacent to the electronic device, such as a call sound of a third speaker passing by the external electronic device or the electronic device. For example, the target speech signal may not be appropriately determined.

[0109] In one embodiment, the electronic device can determine whether the target speech signal and the interference signal have been properly determined. For example, the electronic device can determine whether the target speech signal has been properly determined based on the frequency with which the component determined as the target speech signal appears within the audio signal (600). For example, the electronic device can determine whether the target speech signal has been properly determined based on the duration (640) of the component determined as the target speech signal.

[0110] According to one embodiment, the electronic device can re-determine the target speech signal from among the plurality of components if the target speech signal and the interference signal are not properly determined.

[0111] According to one embodiment, the electronic device can re-determine the target voice signal based on the learned artificial intelligence model. The electronic device can train the artificial intelligence model based on the voice signal of the speaker acquired while providing the service. For example, the electronic device can train the artificial intelligence model to identify the voice signal of the second speaker (e.g., the second speaker (230) of FIG. 2) within the audio signal (600) based on the voice signal of the second speaker collected while providing the voice call service. For example, when the electronic device provides a standalone service that does not require communication with an external electronic device, the electronic device can train the artificial intelligence model to identify the voice signal of the first speaker within the audio signal (600) based on the voice signal of the first speaker acquired while providing the standalone service. The artificial intelligence model can include a neural network-based model such as a convolution neural network (CNN), a deep neural network (DNN), and a recurrent neural network (RNN).

[0112] According to one embodiment, the electronic device can input an audio signal (600) into a learned artificial intelligence model to determine a target speech signal and a nuisance signal.

[0113] According to one embodiment, an electronic device can perform voice activity detection (VAD) to distinguish a voice signal within an audio signal (600). The electronic device can perform formant analysis on the distinguished voice signal. The electronic device can analyze the voice characteristics of each speaker through the formant analysis. Based on the result of performing the formant analysis, the electronic device can distinguish different speakers from the voice signals included in the audio signal (600). The electronic device can determine a target voice signal and a nuisance signal based on the result of performing the formant analysis. For example, the electronic device can perform VAD to distinguish a first component (610) and a second component (620) within the audio signal (600) as voice signals. The electronic device can perform formant analysis on the first component (610) and the second component (620) and determine the target voice signal and the nuisance signal based on the result of the formant analysis.

[0114] In one embodiment, an electronic device can analyze the context of a segmented speech signal. The electronic device can analyze the context of the segmented speech signal to determine which speech signal is relevant to the service currently being provided by the electronic device. The electronic device can analyze the context and determine the speech signal relevant to the service being provided by the electronic device as the target speech signal.

[0115] The electronic device can identify the first component (610) as a target voice signal based on the methods described above. The electronic device can identify the second component (620) and the third component (630) as interference signals.

[0116] Below, we will explain how to insert a mask signal.

[0117]

[0118] FIG. 7 is a diagram for explaining insertion of a mask signal according to one embodiment of the present disclosure.

[0119] Referring to FIG. 7, an audio signal (700) (e.g., audio signal (500) of FIG. 5 and audio signal (600) of FIG. 6) received from an external source by an electronic device (e.g., electronic device (101) of FIG. 1 and electronic device (200) of FIG. 2) is illustrated. In FIG. 7, the audio signal (700) may be in a state where a target voice signal and an interference signal are identified.

[0120] In one embodiment, the electronic device may insert a masking signal into the audio signal (700). To increase the ASR recognition rate of the target speech signal within the audio signal (700), the electronic device should not recognize interference signals. For example, to increase the ASR recognition rate, false recognition may need to be reduced. The electronic device may insert a masking signal to mask the interference signal, thereby preventing recognition (e.g., false recognition) of the interference signal.

[0121] According to one embodiment, the electronic device may insert a noise signal as a mask signal. For example, the electronic device may insert at least one of a pink noise signal, a brown noise signal, a white noise signal, a blue noise signal, a green noise signal, and a maximum length sequence signal (MLS) into the audio signal (700).

[0122] In one embodiment, the electronic device can determine the level of the mask signal to be inserted. The electronic device can determine the level of the mask signal based on the highest level (710) of the interference signal. For example, the electronic device can insert the mask signal having the highest level (710) of the interference signal into the audio signal (700).

[0123] According to one embodiment, the electronic device can mask a nuisance signal by applying a filter to the audio signal (700). The electronic device can apply a filter, such as a band pass filter, a low pass filter, and a high pass filter, to the audio signal (700) so that only the nuisance signal is masked.

[0124] In one embodiment, the electronic device can determine the largest level (710) of the interference signal and remove interference signals smaller than the level (710) using software methods such as a noise gate and dynamic range compression (DRC).

[0125] In one embodiment, when the call path and the ASR path for processing audio signals are separated, a mask signal may be inserted only for the ASR path. For example, the method for removing interference signals described above may be applied only to the ASR path.

[0126] Below, a block diagram of an electronic device performing the methods of FIGS. 2 to 7 will be described.

[0127]

[0128] FIG. 8 is a block diagram illustrating the operation of an electronic device according to one embodiment of the present disclosure.

[0129] Referring to FIG. 8, an electronic device (800) (e.g., the electronic device (101) of FIG. 1 or the electronic device (200) of FIG. 2) is illustrated. For example, the electronic device (800) may include a vocoder (810), a jamming signal identification module (820), a mask signal insertion module (830), an ASR module (840), and / or a voice enhancement module (850). It will be apparent to those skilled in the art that each module included in the electronic device (800) may be implemented in hardware and / or software.

[0130] According to one embodiment, the electronic device (800) can receive an audio signal (e.g., an audio signal (500) of FIG. 5, an audio signal (600) of FIG. 6, and / or an audio signal (700) of FIG. 7) from an external source. When the electronic device (800) provides a service (e.g., a voice call service, etc.) that utilizes an external electronic device (e.g., an electronic device (102, 104) of FIG. 1 and / or an external electronic device (210) of FIG. 2), the electronic device (800) can receive the audio signal from the external electronic device (210). For example, when the electronic device (800) provides a service (e.g., voice control, etc.) on its own, the electronic device (800) can directly obtain the audio signal from the user.

[0131] In one embodiment, the electronic device (800) can decode the audio signal with a vocoder (810). For example, the vocoder (810) can be omitted if the electronic device (800) obtains the audio signal directly from the user.

[0132] According to one embodiment, the electronic device (800) can identify a target voice signal and a interference signal from an audio signal using a interference signal identification module (820). The electronic device (800) can identify the target voice signal and the interference signal from the audio signal according to the method described above with reference to FIGS. 4 to 6.

[0133] According to one embodiment, the electronic device (800) can insert a mask signal into an audio signal using a mask signal insertion module (830). The mask signal insertion module (830) can insert a mask signal into an audio signal according to the method described above in FIG. 7.

[0134] According to one embodiment, the electronic device (800) can generate text corresponding to a target speech signal from an audio signal into which a mask signal has been inserted using the ASR module (840). The mask signal can act as a mask to obscure interference signals in the audio signal. For example, the insertion of the mask signal can increase the recognition rate for the target speech signal.

[0135] According to one embodiment, the electronic device (800) may provide a service to a user based on text corresponding to a target voice signal. For example, the electronic device (800) may provide a simultaneous interpretation service to a first speaker (e.g., the first speaker (220) of FIG. 2 ) based on text corresponding to the target voice signal. For example, the electronic device (800) may provide a voice control service based on text corresponding to the target voice signal.

[0136] According to one embodiment, when an electronic device (800) provides a voice call service and the call path and the ASR path are separated, the electronic device (800) may insert a mask signal only for the ASR path. The call path may include a vocoder (810) and a voice enhancement module (850). The ASR path may include a vocoder (810), a interference signal identification module (820), a mask signal insertion module (830), and an ASR module (840). For example, the above-described method for removing interference signals may be applied only to the path for ASR.

[0137] In one embodiment, the electronic device (800) can enhance an audio signal using a voice enhancement module (850). The electronic device (800) can enhance call quality by providing the enhanced audio signal to a first speaker (e.g., the first speaker (220) of FIG. 2).

[0138] Hereinafter, an operating method of an electronic device according to one embodiment of the present disclosure will be described.

[0139]

[0140] FIG. 9 is a flowchart for explaining an operating method of an electronic device according to one embodiment of the present disclosure.

[0141] The operations described below may be performed sequentially, but are not necessarily performed sequentially. For example, the order of the operations may be changed, and at least two operations may be performed in parallel. Furthermore, some operations may be omitted according to some embodiments. Operations (910) to (940) may be performed by at least one component (e.g., the processor (120) of FIG. 1) of an electronic device (e.g., the electronic device (101) of FIG. 1, the electronic device (200) of FIG. 2, and / or the electronic device (800) of FIG. 8). For example, instructions stored in a memory (e.g., the memory (130) of FIG. 1) by at least one processor may be individually and / or collectively executed, and the instructions may cause the electronic device to perform operations (910) to (950) below.

[0142] In operation (910), the electronic device can provide a voice call service with a second speaker to a first speaker who is a user of the electronic device.

[0143] In operation (920), the electronic device may receive an audio signal (e.g., audio signal (500) of FIG. 5, audio signal (600) of FIG. 6, and / or audio signal (700) of FIG. 7) from an external electronic device of a second speaker (e.g., electronic device (102, 104) of FIG. 1 and external electronic device (210) of FIG. 2) while providing a voice call service.

[0144] In operation (930), the electronic device can identify a target voice signal and a interference signal from an audio signal.

[0145] In operation (940), the electronic device can insert a mask signal into the audio signal.

[0146] In operation (950), the electronic device can generate text corresponding to a target speech signal using an audio signal into which a mask signal has been inserted.

[0147] Electronic devices can provide various services using text corresponding to a target voice signal.

[0148] Since the matters described above through FIGS. 1 to 8 are applied to each operation illustrated in FIG. 9, a more detailed description is omitted.

[0149] According to one embodiment, an electronic device (e.g., electronic device 101 of FIG. 1, electronic device 200 of FIG. 2, and electronic device 800 of FIG. 8) may include a memory (e.g., memory 130 of FIG. 1) that stores instructions. The electronic device may include at least one processor (e.g., processor 120 of FIG. 1) that executes the instructions. When the at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to receive an audio signal (e.g., audio signal 500 of FIG. 5, audio signal 600 of FIG. 6, and audio signal 700 of FIG. 7) from outside the electronic device. When the at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to identify a target speech signal and a nuisance signal from the audio signal. When the at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to insert a mask signal into the audio signal. When at least one processor individually and / or collectively executes instructions, the instructions may cause the electronic device to provide a service utilizing a target speech signal based on an audio signal into which a mask signal has been inserted.

[0150] According to one embodiment, when at least one processor individually and / or collectively executes instructions, the instructions may cause the electronic device to determine a level of a mask signal based on a largest level of a interference signal (e.g., level (710) of FIG. 7).

[0151] According to one embodiment, when at least one processor individually and / or collectively executes instructions, the instructions may cause the electronic device to insert at least one of a pink noise signal, a brown noise signal, a white noise signal, a blue noise signal, a green noise signal, and a maximum length sequence signal (MLS) signal having a level into an audio signal.

[0152] According to one embodiment, when at least one processor individually and / or collectively executes instructions, the instructions may cause the electronic device to perform independent component analysis on an audio signal to extract at least one component. When at least one processor individually and / or collectively executes instructions, the instructions may cause the electronic device to extract a nuisance signal and a target speech signal from the audio signal based on the at least one component.

[0153] According to one embodiment, when at least one processor individually and / or collectively executes instructions, the instructions may cause the electronic device to identify a target speech signal and a nuisance signal based on an artificial intelligence model learned to identify a target speech signal and a nuisance signal in an audio signal.

[0154] According to one embodiment, when at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to provide a voice call service to a first speaker (e.g., the first speaker (220) of FIG. 2) of the electronic device, which provides a voice call with a second speaker (e.g., the second speaker (230) of FIG. 2). When at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to receive an audio signal from an external electronic device of the second speaker (e.g., the electronic devices (102, 104) of FIG. 1 and the external electronic device (210) of FIG. 2) via the voice call service.

[0155] According to one embodiment, the audio signal is a signal collected through an external electronic device, noise reduction processing is performed on the signal by the external electronic device, and may include a target speech signal corresponding to the speech of a second speaker.

[0156] In one embodiment, the interference signal may include a signal collected from an environment where an external electronic device is providing a voice call service and / or a signal collected from a third party speaker unrelated to the voice call service.

[0157] According to one embodiment, when at least one processor individually and / or collectively executes instructions, the instructions may cause the electronic device to provide a simultaneous interpretation service for a target voice signal together with a voice call service.

[0158] According to one embodiment, a method of operating an electronic device may include receiving an audio signal from an external source of the electronic device. The method of operating the electronic device may include identifying a target audio signal and a interference signal from the audio signal. The method of operating the electronic device may include inserting a mask signal into the audio signal. The method of operating the electronic device may include providing a service utilizing the target audio signal based on the audio signal into which the mask signal has been inserted.

[0159] According to one embodiment, the act of inserting a mask signal may further include the act of determining a level of the mask signal based on a largest level of the interference signal.

[0160] According to one embodiment, the operation of inserting a mask signal may insert at least one of a pink noise signal, a brown noise signal, a white noise signal, a blue noise signal, a green noise signal, and a maximum length sequence signal (MLS) signal having a level into the audio signal.

[0161] In one embodiment, the operation of identifying the target speech signal and the interference signal may include performing an independent component analysis (ICA) on the audio signal to extract at least one component. The operation of identifying the target speech signal and the interference signal may include identifying the interference signal and the target speech signal based on the at least one component.

[0162] According to one embodiment, the operation of identifying a target speech signal and a interference signal may identify the target speech signal and the interference signal based on an artificial intelligence model learned to identify the target speech signal and the interference signal in an audio signal.

[0163] In one embodiment, the act of receiving an audio signal external to the electronic device may further include providing a voice call service to the first speaker of the electronic device, which provides a voice call with a second speaker. The act of receiving an audio signal external to the electronic device may further include receiving an audio signal from an external electronic device of the second speaker via the voice call service.

[0164] According to one embodiment, the audio signal is a signal collected through an external electronic device, noise reduction processing is performed on the signal by the external electronic device, and may include a target speech signal corresponding to the speech of a second speaker.

[0165] In one embodiment, the interference signal may include a signal collected from an environment where an external electronic device is providing a voice call service and / or a signal collected from a third party speaker unrelated to the voice call service.

[0166] According to one embodiment, the operation of providing a service using a target voice signal may provide a simultaneous interpretation service for the target voice signal together with a voice call service.

[0167] According to one embodiment, a non-transitory computer-readable recording medium may store one or more computer programs including instructions. When at least one processor individually and / or collectively executes the instructions, the instructions may cause an electronic device to receive an audio signal from outside the electronic device. When at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to identify a target audio signal and a interference signal from an audio signal. When at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to insert a mask signal into the audio signal. When at least one processor individually and / or collectively executes the instructions, the instructions may cause the electronic device to provide a service using the target audio signal based on the audio signal into which the mask signal has been inserted.

[0168]

[0169] And the embodiments of the present invention disclosed in this specification and drawings are only specific examples to easily explain the technical contents according to the embodiments of the present invention and to help understand the embodiments of the present invention, and are not intended to limit the scope of the embodiments of the present invention. Therefore, the scope of the various embodiments of the present invention should be interpreted as including all changes or modified forms derived based on the technical ideas of the various embodiments of the present invention in addition to the embodiments disclosed herein.

Claims

1. In electronic devices (101, 200, 800), Memory (130) for storing commands; and At least one processor (120) executing the above instructions Including, When the at least one processor (120) individually and / or collectively executes the instructions, the instructions cause the electronic device (101, 200, 800) to: An audio signal (500, 600, 700) is received from outside of the electronic device (101, 200, 800), a target voice signal and an interference signal are identified from the audio signal (500, 600, 700), a mask signal is inserted into the audio signal (500, 600, 700), and a service using the target voice signal is provided based on the audio signal (500, 600, 700) into which the mask signal is inserted. Electronic devices (101, 200, 800).

2. In paragraph 1, When the at least one processor (120) individually and / or collectively executes the instructions, the instructions cause the electronic device (101, 200, 800) to: Determine the level of the mask signal based on the largest level (710) of the above interference signal. Electronic devices (101, 200, 800).

3. In either of paragraphs 1 and 2, When the at least one processor (120) individually and / or collectively executes the instructions, the instructions cause the electronic device (101, 200, 800) to: Inserting at least one of a pink noise signal, a brown noise signal, a white noise signal, a blue noise signal, a green noise signal and a maximum length sequence signal (MLS) having the above level into the audio signal (500, 600, 700). Electronic devices (101, 200, 800).

4. In any one of paragraphs 1 to 3, When the at least one processor (120) individually and / or collectively executes the instructions, the instructions cause the electronic device (101, 200, 800) to: Performing independent component analysis on the above audio signal (500, 600, 700) to extract at least one component, and identifying the interference signal and the target voice signal based on the at least one component. Electronic devices (101, 200, 800).

5. In any one of paragraphs 1 to 4, When the at least one processor (120) individually and / or collectively executes the instructions, the instructions cause the electronic device (101, 200, 800) to: Identifying the target voice signal and the interference signal based on an artificial intelligence model learned to identify the target voice signal and the interference signal in the audio signal (500, 600, 700). Electronic devices (101, 200, 800).

6. In any one of paragraphs 1 to 5, When the at least one processor (120) individually and / or collectively executes the instructions, the instructions cause the electronic device (101, 200, 800) to: Providing a voice call service that provides a voice call with a second speaker (230) to the first speaker (220) of the electronic device (101, 200, 800), and receiving the audio signal (500, 600, 700) from an external electronic device (102, 104, 210) of the second speaker (230) through the voice call service. Electronic devices (101, 200, 800).

7. In any one of paragraphs 1 to 6, The above audio signals (500, 600, 700) are A signal collected through the external electronic device (102, 104, 210) and subjected to noise reduction processing by the external electronic device (102, 104, 210), and including the target voice signal corresponding to the speech of the second speaker (230). Electronic devices (101, 200, 800).

8. In any one of paragraphs 1 to 7, The above interference signal is, The external electronic device (102, 104, 210) includes a signal collected from an environment in which the voice call service is being provided and / or a signal collected from a third party speaker not related to the voice call service. Electronic devices (101, 200, 800).

9. In any one of paragraphs 1 to 8, When the at least one processor (120) individually and / or collectively executes the instructions, the instructions cause the electronic device (101, 200, 800) to: To provide simultaneous interpretation service for the target voice signal together with the above voice call service, Electronic devices (101, 200, 800).

10. In the operating method of an electronic device (101, 200, 800), An operation of receiving an audio signal (500, 600, 700) from outside the electronic device (101, 200, 800); An operation of identifying a target voice signal and an interference signal from the above audio signal (500, 600, 700); An operation of inserting a mask signal into the above audio signal (500, 600, 700); and An operation for providing a service using the target voice signal based on the audio signal (500, 600, 700) into which the mask signal is inserted. including, How it works.

11. In paragraph 10, The operation of inserting the above mask signal is: An operation for determining the level of the mask signal based on the largest level (710) of the above interference signal. Including more, How it works.

12. In any one of paragraphs 10 and 11, The operation of inserting the above mask signal is: Inserting at least one of a pink noise signal, a brown noise signal, a white noise signal, a blue noise signal, a green noise signal and a maximum length sequence signal (MLS) having the above level into the audio signal (500, 600, 700). How it works.

13. In any one of paragraphs 10 to 12, The operation of identifying the target voice signal and the interference signal is as follows: An operation of performing independent component analysis on the above audio signal (500, 600, 700) to extract at least one component; and An operation of extracting the interference signal and the target voice signal based on at least one component. including, How it works.

14. In any one of paragraphs 10 to 13, The operation of identifying the target voice signal and the interference signal is as follows: Identifying the target voice signal and the interference signal based on an artificial intelligence model learned to identify the target voice signal and the interference signal in the audio signal (500, 600, 700). How it works.

15. In any one of paragraphs 10 to 14, The operation of receiving the above audio signal (500, 600, 700) from outside of the electronic device (101, 200, 800) is as follows: An operation of providing a voice call service to a first speaker (220) of the electronic device (101, 200, 800) that provides a voice call with a second speaker (230); and An operation of receiving the audio signal (500, 600, 700) from the external electronic device (102, 104, 210) of the second speaker (230) through the voice call service. including more, How it works.

Images