Electronic device and control method therefor

Abstract

This electronic device may comprise: a memory; a communication interface for communicating with an external server; and at least one processor including a processing circuit, wherein the at least one processor can obtain audio data for ambient sound, obtain a preset type of target audio signal from the audio data, obtain sound quality information about the target audio signal, transmit the target audio signal to the external server through the communication interface if the sound quality information is not greater than or equal to a threshold grade, receive, from the external server through the communication interface, a source audio signal corresponding to the target audio signal, identify a playback time point of the source audio signal on the basis of the target audio signal, and provide, on the basis of the playback time point, a synthesized audio signal obtained by synthesizing the source audio signal and an inversion signal corresponding to the audio data.

Description

Electronic device and method of controlling the same

[0001] The present disclosure relates to an electronic device and a control method thereof, and for example, to an electronic device providing a noise canceling function and a control method thereof.

[0002] Noise cancellation is a feature that eliminates (or reduces) unwanted ambient noise or background noise. Users can configure which signals are considered noise. Depending on the user's settings or preset method, sounds considered noise can be attenuated (or eliminated) by the noise cancellation feature.

[0003] When attempting to hear only the target portion of a sound measured in the surrounding environment, excluding the noise portion, there is a problem with poor sound quality. This is because noise may not be completely removed depending on the performance of the noise filtering function.

[0004] If the target portion is output without completely removing the noise, the user may feel that he or she is listening to low-quality audio.

[0005] An electronic device (100) can recognize audio collected from the surrounding environment and provide audio search results. The search results may include information indicating which music or content the collected audio corresponds to.

[0006] When playing searched content while using the noise-canceling function, the user may experience discomfort as ambient sounds that are not completely removed are heard along with the content.

[0007] The present disclosure is designed to improve the above-described problems, and an object of the present disclosure is to provide an electronic device and a control method thereof that measure ambient sound to obtain a target audio signal desired by a user and determine whether to receive a source audio signal from an external server by considering the quality of the target audio signal.

[0008] According to one embodiment, an electronic device may include a memory, a communication circuit, a communication interface for communicating with an external server, and at least one processor, wherein the at least one processor individually and / or collectively obtains audio data for ambient sound, obtains a target audio signal of a preset type from the audio data, obtains sound quality information of the target audio signal, and if the sound quality information is not higher than a threshold level, transmits the target audio signal to the external server through the communication interface, receives a source audio signal corresponding to the target audio signal from the external server through the communication interface, identifies a reproduction time of the source audio signal based on the target audio signal, and provides a synthesized audio signal obtained by synthesizing an inverted signal corresponding to the audio data and the source audio signal based on the reproduction time.

[0009] The at least one processor can obtain the target audio signal and the noise signal from the audio data, obtain a signal-to-noise ratio (SNR) based on the target audio signal and the noise signal, and obtain the sound quality information based on the signal-to-noise ratio (SNR).

[0010] The at least one processor may transmit the target audio signal to the external server via the communication interface if the signal-to-noise ratio (SNR) is less than a threshold value.

[0011] The at least one processor can determine a sound quality grade of the first audio signal based on the signal-to-noise ratio (SNR) and a grade table for sound quality stored in the memory, and obtain the sound quality information including the sound quality grade.

[0012] The at least one processor can obtain the inverted signal by inverting the phase of a signal included in the audio data to attenuate the ambient sound.

[0013] The at least one processor may compare a waveform of the target audio signal with a waveform of the source audio signal to determine the playback point of the source audio signal, synthesize the inverted signal and a region corresponding to the playback point among the entire region of the source audio signal to obtain the synthesized audio signal, and provide the synthesized audio signal.

[0014] The at least one processor can transmit the synthesized audio signal to an external sound device via the communication interface.

[0015] The external sound device is a wireless earphone or a wireless headphone, and the at least one processor includes a communication circuit and can transmit the synthesized audio signal to the external sound device through a Bluetooth communication module included in the communication interface.

[0016] The above-mentioned synthetic audio signal is a first synthetic audio signal, and the at least one processor can obtain a second synthetic audio signal by synthesizing the inverted signal corresponding to the audio data and the target audio signal when the sound quality information is above a threshold level, and provide the second synthetic audio signal.

[0017] The at least one processor may obtain a conversion signal by converting the target audio signal into a stereo type when the output method of the target audio signal is a mono type, and obtain the second synthesized audio signal by synthesizing the conversion signal and the inverted signal.

[0018] According to one embodiment, a method for controlling an electronic device communicating with an external server may include the steps of: obtaining audio data for ambient sound; obtaining a target audio signal of a preset type from the audio data; obtaining sound quality information of the target audio signal; transmitting the target audio signal to the external server if the sound quality information is not higher than a threshold level; receiving a source audio signal corresponding to the target audio signal from the external server; identifying a playback time of the source audio signal based on the target audio signal; and providing a synthesized audio signal obtained by synthesizing an inverted signal corresponding to the audio data and the source audio signal based on the playback time.

[0019] The step of obtaining the above sound quality information may include obtaining the target audio signal and the noise signal from the audio data, obtaining a signal-to-noise ratio (SNR) based on the target audio signal and the noise signal, and obtaining the sound quality information based on the signal-to-noise ratio (SNR).

[0020] The step of transmitting the target audio signal to the external server may transmit the target audio signal to the external server if the signal-to-noise ratio (SNR) is less than a threshold value.

[0021] The step of obtaining the above sound quality information may determine a sound quality grade of the first audio signal based on the signal-to-noise ratio (SNR) and a grade table for sound quality stored in the electronic device, and obtain the sound quality information including the sound quality grade.

[0022] The above control method may further include a step of obtaining the inverted signal by inverting the phase of a signal included in the audio data to attenuate the ambient sound.

[0023] The step of providing the above-mentioned synthetic audio signal may include comparing a waveform of the target audio signal with a waveform of the source audio signal to determine the playback point of the source audio signal, synthesizing a region corresponding to the playback point of the entire region of the inverted signal and the source audio signal to obtain the synthetic audio signal, and providing the synthetic audio signal.

[0024] The above control method may further include a step of transmitting the synthesized audio signal to an external sound device.

[0025] The external sound device is a wireless earphone or a wireless headphone, and the step of transmitting the synthesized audio signal to the external sound device may transmit the synthesized audio signal to the external sound device via a Bluetooth communication module included in the electronic device.

[0026] The above-mentioned synthetic audio signal is a first synthetic audio signal, and the control method may further include, if the sound quality information is above a threshold level, a step of obtaining a second synthetic audio signal by synthesizing the inverted signal corresponding to the audio data and the target audio signal, and a step of providing the second synthetic audio signal.

[0027] The above control method further includes a step of obtaining a conversion signal by converting the target audio signal into a stereo type when the output method of the target audio signal is a mono type, and the step of obtaining the second synthesized audio signal may obtain the second synthesized audio signal by synthesizing the conversion signal and the inverted signal.

[0028] FIG. 1 is a drawing for explaining a noise canceling function according to one embodiment.

[0029] FIG. 2 is a block diagram illustrating an electronic device according to one embodiment.

[0030] FIG. 3 is a block diagram illustrating a specific configuration of the electronic device of FIG. 2, according to one embodiment.

[0031] FIG. 4 is a drawing for explaining a mode performed in an electronic device according to one embodiment.

[0032] FIG. 5 is a diagram for explaining an operation of obtaining an audio signal according to one embodiment.

[0033] FIG. 6 is a diagram for explaining a method of providing content using sound quality information, according to one embodiment.

[0034] FIG. 7 is a diagram for explaining an operation of determining a different content provision method using sound quality information according to one embodiment.

[0035] FIG. 8 is a diagram for explaining audio data according to one embodiment.

[0036] FIG. 9 is a diagram for explaining a grading standard of sound quality information according to one embodiment.

[0037] FIG. 10 is a diagram for explaining an operation of obtaining sound quality information using a signal-to-noise ratio (SNR), according to one embodiment.

[0038] FIG. 11 is a diagram for explaining an operation of determining a content provision method using a signal-to-noise ratio (SNR), according to one embodiment.

[0039] FIG. 12 is a diagram for explaining an operation of obtaining sound quality information using a recognition rate according to one embodiment.

[0040] FIG. 13 is a diagram for explaining an operation of determining a content provision method using a recognition rate according to one embodiment.

[0041] FIG. 14 is a diagram for explaining an operation of determining a playback point of a third audio signal according to one embodiment.

[0042] FIG. 15 is a diagram for explaining an operation of converting a first audio signal according to one embodiment.

[0043] FIG. 16 is a diagram for explaining an operation of filtering only noise, according to one embodiment.

[0044] FIG. 17 is a drawing for explaining a UI (User Interface) related to a content provision service according to one embodiment.

[0045] FIG. 18 is a drawing for explaining an operation of providing an audio signal using a sound device according to one embodiment.

[0046] FIG. 19 is a drawing for explaining an operation of providing an audio signal using a sound device according to one embodiment.

[0047] FIG. 20 is a drawing for explaining an operation of providing an audio signal using a sound device according to one embodiment.

[0048] Figure 21 is a drawing for explanation according to one embodiment.

[0049] Hereinafter, the present disclosure will be described in detail with reference to the attached drawings.

[0050] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0051] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.

[0052] The expression "at least one of A and / or B" should be understood to mean either "A" or "B" or "A and B".

[0053] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0054] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component can be directly coupled to at least one other component, or can be connected through another component (e.g., a third component).

[0055] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0056] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Furthermore, multiple "modules" or multiple "parts" may be integrated into at least one module and implemented as at least one processor (not shown), excluding any "modules" or "parts" that need to be implemented as specific hardware. Accordingly, each "module" herein may include a circuit.

[0057] In this specification, the term user may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0058] An embodiment of the present disclosure will be described in more detail with reference to the attached drawings below.

[0059] FIG. 1 is a drawing for explaining a noise canceling function according to one embodiment.

[0060] Referring to FIG. 1, the electronic device (100) can perform a noise-canceling mode. The noise-canceling mode may be a mode that attenuates (or filters) a signal set as a target noise signal. The electronic device (100) can receive ambient sounds (noise, music). The electronic device (100) may include a filtering module (121). The electronic device (100) can remove noise from the ambient sounds through the filtering module (121). The electronic device (100) can provide an audio signal from which noise has been removed.

[0061] A filtering module (121) including a filtering circuit may be described as a noise filtering module, a sound filtering module, etc.

[0062] In one embodiment, the electronic device (100) can remove only noise from ambient sound. The electronic device (100) can provide music contained in the ambient sound as is.

[0063] According to one embodiment, the electronic device (100) can obtain high-quality music data corresponding to music (low-quality) from ambient sounds. The electronic device (100) can filter (or block, attenuate, or cancel) all ambient sounds (low-quality music, noise) and provide high-quality music data.

[0064] FIG. 2 is a block diagram illustrating an electronic device (100) according to one embodiment.

[0065] Referring to FIG. 2, the electronic device (100) may include at least one of a memory (110), at least one processor (120), or a communication interface (130).

[0066] The memory (110) can store a rating table for sound quality. A description related to this is described in Fig. 9.

[0067] The communication interface (130) can be directly or indirectly connected to an external sound device (200) or an external server (300) (see, for example, 200 and 300 of FIGS. 18 to 20).

[0068] The electronic device (100) may be an electronic whiteboard, TV, desktop PC, laptop, smartphone, tablet PC, server, etc. The above-described examples are merely examples for describing the electronic device and are not necessarily limited to the above-described devices.

[0069] At least one processor (120) can perform overall control operations of the electronic device (100). At least one processor (120) can perform a function of controlling overall operations of the electronic device (100).

[0070] At least one processor (120) may obtain audio data (first audio data) for ambient sound, obtain a target audio signal (first audio signal) of a preset type from the audio data, obtain sound quality information of the target audio signal, and if the sound quality information is not higher than a threshold level, transmit the target audio signal to an external server through a communication interface (130), receive a source audio signal (third audio signal) corresponding to the target audio signal from the external server through the communication interface (130), identify a reproduction time of the source audio signal based on the target audio signal, and provide a synthesized audio signal (fourth audio signal) obtained by synthesizing an inverted signal corresponding to the audio data and the source audio signal based on the reproduction time.

[0071] At least one processor (120) can obtain audio data for ambient sound.

[0072] For example, at least one processor (120) may receive audio data recording ambient sound. At least one processor (120) may receive audio data from an external sound device (200) via a communication interface (130). A description related to the external sound device (200) is provided in FIGS. 18 to 20 .

[0073] For example, the electronic device (100) may include a microphone (180), and at least one processor (120) may record ambient sound through the microphone (180). At least one processor (120) may obtain audio data about the ambient sound through the microphone (180).

[0074] At least one processor (120) can determine whether audio data includes a target audio signal of a preset type. At least one processor (120) can identify a target audio signal of a preset type by analyzing the audio data.

[0075] At least one processor (120) can obtain an audio signal included in audio data. At least one processor (120) can obtain characteristic information of the audio data and determine whether an audio signal of a preset type is included in the audio data based on the characteristic information of the audio data.

[0076] Audio data may include data in which multiple signals are mixed. At least one processor (120) may obtain characteristic information including at least one of a frequency spectrum, frequency band energy, spectrum average, spectrum analysis, energy change information, amplitude change information, and period change information of the audio data. At least one processor (120) may determine whether a signal of a preset type exists among the multiple signals included in the audio data based on the characteristic information.

[0077] Predefined types can include content types that include audio. For example, predefined types can include video content with audio, music content, etc. Predefined types can be described as target types, target categories, predefined categories, etc.

[0078] At least one processor (120) can obtain a target audio signal of a preset type from audio data. The audio data can include a plurality of audio signals. At least one processor (120) can identify (or extract) a target audio signal of a preset type from among the plurality of audio signals.

[0079] At least one processor (120) can obtain sound quality information of a target audio signal. This information can include information indicating the accuracy and clarity of the audio signal. The sound quality information can include sound quality data indicating sound quality or a sound quality grade indicating sound quality. At least one processor (120) can obtain sound quality data based on audio data. At least one processor (120) can obtain a sound quality grade based on audio data.

[0080] The sound quality rating may include multiple ratings categorized according to a preset method. The sound quality rating may indicate a relative value of the sound quality of audio data. At least one processor (120) may compare the sound quality rating with a threshold rating. The threshold rating may be changed according to user settings.

[0081] According to one embodiment, the audio quality data may include a signal-to-noise ratio (SNR).

[0082] At least one processor (120) can obtain a target audio signal and a noise signal from audio data, obtain a signal-to-noise ratio (SNR) based on the target audio signal and the noise signal, and obtain sound quality information based on the signal-to-noise ratio (SNR).

[0083] At least one processor (120) can transmit a target audio signal to an external server via a communication interface (130) if the signal-to-noise ratio (SNR) is below a threshold value.

[0084] At least one processor (120) can determine a sound quality grade of a first audio signal based on a signal-to-noise ratio (SNR) and a grade table for sound quality stored in a memory, and obtain sound quality information including the sound quality grade.

[0085] Specific descriptions related to signal-to-noise ratio (SNR) are described in FIGS. 9 to 11.

[0086] According to one embodiment, at least one processor (120) may transmit a target audio signal to an external server (300). Since at least one processor (120) has already extracted the target audio signal that the user wishes to hear, it may transmit only the target audio signal to the external server (300).

[0087] According to one embodiment, at least one processor (120) may transmit audio data to an external server (300). At least one processor (120) may transmit audio data including both a target audio signal and a noise signal to the external server (300). The external server (300) may transmit a source audio signal (third audio signal) to the electronic device (100) based on the received audio data.

[0088] At least one processor (120) may transmit a control command (or control signal) requesting that the external server (300) provide search results while transmitting a target audio signal (or audio data). The external server (300) may transmit search results corresponding to the received target audio signal (or audio data) to the electronic device (100) based on the received control command.

[0089] At least one processor (120) can obtain an inverted signal by inverting the phase of a signal included in the audio data to attenuate ambient sound.

[0090] At least one processor (120) may operate in one of a plurality of modes. The plurality of modes may include at least one of an ambient sound collection mode or a noise canceling mode.

[0091] The ambient sound collection mode may be a mode for acquiring audio data. The noise-canceling mode may be a mode for filtering out target noise sounds from among ambient sounds. At least one processor (120) may identify the target noise sounds as noise and filter out the target noise sounds identified as noise from the audio data.

[0092] At least one processor (120) may perform an operation to reduce the size of a target noise sound while performing a noise canceling mode. At least one processor (120) may attenuate (or filter) the target noise sound.

[0093] At least one processor (120) can obtain a target noise signal corresponding to the target noise sound to reduce the size of the target noise sound. At least one processor (120) can obtain an inverted signal (or inverted audio signal) having a phase opposite to that of the target noise signal.

[0094] At least one processor (120) can perform a noise canceling function by outputting an inverted signal. When an inverted signal having an opposite phase to the target noise signal is output, the magnitude of the target noise signal can be attenuated.

[0095] At least one processor (120) can set a target noise signal according to a preset method when performing a noise canceling mode.

[0096] According to one embodiment, at least one processor (120) may set all sounds (or signals) of the ambient sound as target noise sounds (or target noise signals). When the noise-cancelling mode is started, at least one processor (120) may attenuate (or filter) all sounds included in the ambient sound. At least one processor (120) may determine all sounds included in the ambient sound as target noise sounds.

[0097] According to one embodiment, at least one processor (120) may set only a noise signal among all sounds (or signals) of the ambient sound as a target noise sound (or target noise signal). The noise signal may include a signal that the user does not want. The noise signal may include noise or static. The noise signal may represent a signal that does not correspond to voice, music, etc. For example, the noise signal may include a mechanical sound, white noise, etc. In noise-canceling mode, at least one processor (120) may provide only the audio (voice of the caller or music) that the user wants while attenuating the level of the ambient environmental noise (or noise). An embodiment related to this is described in FIG. 16.

[0098] At least one processor (120) can compare the waveform of the target audio signal with the waveform of the source audio signal to determine the playback point of the source audio signal, synthesize (or combine) an inverted signal and a region corresponding to the playback point among the entire region of the source audio signal to obtain a synthesized audio signal, and provide the synthesized audio signal.

[0099] Areas can be described as bands, parts, segments, clips, etc.

[0100] The target audio signal included in the audio data may be an audio signal output based on a specific playback point in time. The source audio signal may be a signal received as the entire data received from an external server (300). The recorded (or collected) audio data may only include signals corresponding to a specific playback point in time. The source audio signal received by the external server (300) may include signals for the initial region or all regions.

[0101] At least one processor (120) needs to determine which point in time to provide a signal from the entire range of the source audio signal. The at least one processor (120) can compare the waveform of the target audio signal with the entire waveform of the source audio signal to determine a playback point in time that is identical to the waveform of the target audio signal from the entire range of the source audio. The at least one processor (120) can extract (or obtain) a region corresponding to the playback point in time from the entire range of the source audio signal. The at least one processor (120) can obtain a synthesized audio signal by synthesizing an inverted signal and the region corresponding to the playback point in time.

[0102] For example, assuming that the entire content of the target audio signal is a 3 minute and 30 second piece of music, and that the currently playing portion (target audio signal) is 1 minute and 10 seconds long, at least one processor (120) may determine 1 minute and 10 seconds as the playback point and provide the portion corresponding to 1 minute and 10 seconds of the source audio signal. A description related to the playback point is described in FIG. 14.

[0103] The playback point can be described as a playback area, a playback part, an output point, an output area, an output part, etc.

[0104] At least one processor (120) can transmit a synthesized audio signal to an external sound device (200) via the communication interface (130).

[0105] The external sound device (200) is a wireless earphone or wireless headphone, and at least one processor (120) can transmit a synthesized audio signal to the external sound device (200) via a Bluetooth communication module included in the communication interface (130). A description of the external sound device (200) is provided in FIGS. 18 to 20.

[0106] The synthetic audio signal is a first synthetic audio signal, and at least one processor (120) can obtain a second synthetic audio signal (second audio signal) by synthesizing an inverted signal corresponding to audio data and a target audio signal when the sound quality information is above a threshold level, and provide the second synthetic audio signal (second audio signal).

[0107] At least one processor (120) may determine that there is no need to acquire a source audio signal from an external server (300) if the sound quality information is above a threshold level. This is because the sound quality of the target audio signal has been determined to be high quality. At least one processor (120) may provide a target audio signal if the sound quality information of the audio data is above a threshold level.

[0108] At least one processor (120) can perform a noise-canceling mode while simultaneously providing only a target audio signal. The noise-canceling mode may be a function that blocks (or attenuates) at least some of the surrounding sounds. At least one processor (120) can provide the extracted target audio signal while performing the noise-canceling mode. A detailed description of this is provided in FIG. 6.

[0109] At least one processor (120) can obtain a conversion signal by converting the target audio signal into a stereo type when the output method of the target audio signal is mono type, and obtain a second synthesized audio signal (second audio signal) by synthesizing the conversion signal and the inverted signal. A description related to this is described in FIG. 15.

[0110] An operation of providing an audio signal or audio data may include an output operation or a transmission operation.

[0111] According to one embodiment, at least one processor (120) may output an audio signal through a speaker (170) included in the electronic device (100).

[0112] According to one embodiment, at least one processor (120) may transmit an audio signal to an external server (300).

[0113] The actions of recording audio data, analyzing audio data, and outputting audio signals can be performed separately in each of the electronic device (100) and the sound device (200).

[0114] The audio signal providing method of the electronic device (100) according to various embodiments can selectively (or intensively) provide a target audio signal of a preset type to a user.

[0115] FIG. 3 is a block diagram for explaining a specific configuration of the electronic device (100) of FIG. 2, according to one embodiment.

[0116] Referring to FIG. 3, the electronic device (100) may include at least one of a memory (110), at least one processor (120), a communication interface (130), a display (140), an operation interface (150), an input / output interface (160), a speaker (170), a microphone (180), and a camera (190).

[0117] The memory (110) may be implemented as an internal memory such as a ROM (e.g., an electrically erasable programmable read-only memory (EEPROM)) or RAM included in at least one processor (120), or may be implemented as a separate memory from at least one processor (120). The memory (110) may be implemented as a memory embedded in the electronic device (100) or as a memory detachable from the electronic device (100) depending on the purpose of data storage. For example, data for driving the electronic device (100) may be stored in a memory embedded in the electronic device (100), and data for expanding functions of the electronic device (100) may be stored in a memory detachable from the electronic device (100).

[0118] In the case of memory embedded in the electronic device (100), it may be implemented as at least one of volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), etc.), hard drive, or solid state drive (SSD), and in the case of memory that can be attached or detached to the electronic device (100), it may be implemented in the form of a memory card (e.g., compact flash (CF), secure digital (SD), micro secure digital (Micro-SD), mini secure digital (Mini-SD), extreme digital (xD), multi-media card (MMC), etc.), external memory that can be connected to a USB port (e.g., USB memory), etc.

[0119] The memory (110) can store at least one instruction. Based on the instruction stored in the memory (110), at least one processor (120) can perform various operations.

[0120] At least one processor (120) may be implemented as a digital signal processor (DSP), a microprocessor, or a time controller (TCON) that processes digital signals. However, the present invention is not limited thereto, and may include one or more of a central processing unit (CPU), a micro controller unit (MCU), a micro processing unit (MPU), a controller, an application processor (AP), a graphics-processing unit (GPU), a communication processor (CP), or an advanced reduced instruction set computer (RISC) machines (ARM) processor, or may be defined by the relevant terminology. At least one processor (120) may be implemented as a system on chip (SoC) having a built-in processing algorithm, a large scale integration (LSI), or may be implemented in the form of a field programmable gate array (FPGA). At least one processor (120) may perform various functions by executing computer executable instructions stored in a memory.

[0121] The communication interface (130) is a component that performs communication with various types of external devices according to various types of communication methods. The communication interface (130) may include a wireless communication module or a wired communication module. Each communication module may be implemented in the form of at least one hardware chip.

[0122] A wireless communication module may be a module that wirelessly communicates with an external device. For example, the wireless communication module may include at least one of a Wi-Fi module, a Bluetooth module, an infrared communication module, or other communication modules that include communication circuitry.

[0123] Wi-Fi and Bluetooth modules can communicate via Wi-Fi and Bluetooth, respectively. When using a Wi-Fi or Bluetooth module, various connection information, such as the service set identifier (SSID) and session key, is first transmitted and received. This information is then used to establish a communication connection before various other information can be transmitted and received.

[0124] Infrared communication modules perform communication based on infrared communication (IrDA, infrared Data Association) technology, which transmits data wirelessly over short distances using infrared light, which is between visible light and millimeter waves.

[0125] In addition to the above-described communication method, other communication modules may include at least one communication chip that performs communication according to various wireless communication standards such as zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc.

[0126] A wired communication module may be a module that communicates with an external device via a wire. For example, the wired communication module may include at least one of a Local Area Network (LAN) module, an Ethernet module, a paired cable, a coaxial cable, a fiber optic cable, or an Ultra Wide-Band (UWB) module.

[0127] According to various embodiments, the communication interface (130) includes communication circuitry and may utilize the same communication module (e.g., a Wi-Fi module) to communicate with external devices such as a remote control device and an external server.

[0128] According to various embodiments, the communication interface (130) may include communication circuitry and utilize different communication modules to communicate with external devices, such as remote control devices, and external servers. For example, the communication interface (130) may utilize at least one of an Ethernet module or a Wi-Fi module to communicate with an external server, and may also utilize a Bluetooth module to communicate with an external device, such as a remote control device. However, this is merely an embodiment, and the communication interface (130) may utilize at least one of various communication modules when communicating with multiple external devices or external servers.

[0129] The display (140) may be implemented as a variety of displays such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display panel (PDP), etc. The display (140) may also include a driving circuit, a backlight unit, etc., which may be implemented as a form such as an a-si TFT (amorphous silicon thin film transistor), an LTPS (low temperature poly silicon) TFT, an OTFT (organic TFT), etc. The display (140) may be implemented as a touch screen combined with a touch sensor, a flexible display, a three-dimensional display (3D display, three-dimensional dispaly), etc. According to an embodiment of the present disclosure, the display (140) may include not only a display panel that outputs an image, but also a bezel that houses the display panel. In particular, according to an embodiment of the present disclosure, the bezel may include a touch sensor for detecting user interaction.

[0130] The operating interface (150) may be implemented as a device such as a button, a touch pad, a mouse, and a keyboard, or as a touch screen capable of performing the above-described display function and operating input function. The button may be a mechanical button, a touch pad, a wheel, or any other type of button formed in any area of ​​the front, side, or back of the main body of the electronic device (100).

[0131] The input / output interface (160) may be any one of HDMI (High Definition Multimedia Interface), MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (D-subminiature), and DVI (Digital Visual Interface). The input / output interface (160) may input / output at least one of audio and video signals. Depending on the implementation example, the input / output interface (160) may include a port that inputs / outputs only audio signals and a port that inputs / outputs only video signals as separate ports, or may be implemented as a single port that inputs / outputs both audio signals and video signals. The electronic device (100) may transmit at least one of the audio and video signals to an external device (e.g., an external display device or an external speaker) through the input / output interface (160). An output port included in the input / output interface (160) can be connected to an external device, and the electronic device (100) can transmit at least one of an audio and video signal to the external device through the output port.

[0132] The input / output interface (160) may be directly or indirectly connected to a communication interface. The input / output interface (160) may transmit information received from an external device to the communication interface or transmit information received through the communication interface to the external device.

[0133] The speaker (170) may be a component that outputs various audio data as well as various notification sounds or voice messages.

[0134] The microphone (180) is a component that receives a user's voice or other sounds and converts them into audio data. The microphone (180) can receive the user's voice in an activated state. For example, the microphone (180) can be formed integrally on the upper side, the front side, the side side, etc. of the electronic device (100). The microphone (180) can include various components such as a microphone that collects the user's voice in analog form, an amplifier circuit that amplifies the collected user's voice, an A / D conversion circuit that samples the amplified user's voice and converts it into a digital signal, and a filter circuit that removes noise components from the converted digital signal.

[0135] The camera (190) is a device configured to capture a subject and generate a captured image, and the captured image includes both moving images and still images. The camera (190) can acquire images for at least one external device and can be implemented with a camera, lens, infrared sensor, or the like.

[0136] The camera (190) may include a lens and an image sensor. The type of lens may include a general-purpose lens, a wide-angle lens, a zoom lens, etc., and may be determined according to the type, characteristics, usage environment, etc. of the electronic device (100). The image sensor may include a complementary metal oxide semiconductor (CMOS) and a charge-coupled device (CCD).

[0137] FIG. 4 is a drawing for explaining a mode performed in an electronic device (100) according to one embodiment.

[0138] Referring to the embodiment (400) of FIG. 4, the electronic device (100) may operate in one of a plurality of modes. The plurality of modes may include at least one of an ambient sound collection mode or a noise canceling mode.

[0139] The ambient sound collection mode may be described as the first mode. When operating in the first mode, the electronic device (100) can measure ambient sound using a microphone included in the electronic device (100). The electronic device (100) can obtain digital data regarding the ambient sound acquired through the microphone.

[0140] The noise canceling mode may be described as a second mode. When operating in the second mode, the electronic device (100) may reduce a target noise signal from ambient sound. The target noise signal may include a cancellation target. The operation of reducing the target noise signal may include at least one of an operation of reducing the size of the target noise signal, an operation of attenuating the target noise signal, an operation of removing the target noise signal, and an operation of canceling the target noise signal.

[0141] For example, the target noise signal may be any signal included in the ambient sound. The target noise signal may include any signal of the ambient sound measured by the electronic device (100). The electronic device (100) may set the target noise signal to any signal included in the ambient sound, and the electronic device (100) may reduce any signal included in the ambient sound.

[0142] For example, the target noise signal may include noise signals among the signals included in the ambient sound. The target noise signal may only include signals related to noise. The electronic device (100) may set the target noise signal as a noise signal, and the electronic device (100) may reduce only signals corresponding to noise among the ambient sound.

[0143] The electronic device (100) can operate in a first mode or a second mode according to a user's command. When a first user input for changing to a second mode is received while operating in the first mode, the electronic device (100) can change the first mode to the second mode.

[0144] When a second user input is received to change to the first mode while operating in the second mode, the electronic device (100) can change the second mode to the first mode.

[0145] When a third user input is received to change to the first mode while operating in the second mode, the electronic device (100) can change the first mode to the second mode.

[0146] The electronic device (100) can change modes based on user input or preset control commands.

[0147] FIG. 5 is a diagram for explaining an operation of obtaining an audio signal according to one embodiment.

[0148] Referring to FIG. 5, the electronic device (100) may acquire first audio data regarding ambient sound (S505). The audio data may include signals measuring the ambient sound. The ambient sound may include at least one audio signal measured in the space where the electronic device (100) is located. The ambient sound may represent at least one of analog audio data or digital audio data.

[0149] The electronic device (100) can determine whether an audio signal of a preset type is identified in the first audio data. The electronic device (100) can determine whether an audio signal of a preset type is identified among a plurality of audio signals included in the first audio data (S510). The preset type may be a type indicating music content. For example, the preset type may include a music type.

[0150] If an audio signal of a preset type is not identified in the first audio data (S510-N), the electronic device (100) can perform steps S505 and S510.

[0151] When an audio signal of a preset type is identified in the first audio data (S510-Y), the electronic device (100) can obtain a first audio signal from the first audio data (S515). The first audio signal can represent an audio signal corresponding to the preset type. The electronic device (100) can obtain (or extract) a first audio signal corresponding to the preset type among at least one audio signal included in the first audio data.

[0152] The electronic device (100) can obtain sound quality information of the first audio signal (S520). The sound quality information can include information indicating the accuracy and clarity of the audio signal.

[0153] The electronic device (100) may include various methods to obtain sound quality information of the first audio signal.

[0154] According to one embodiment, the electronic device (100) can obtain sound quality information using a signal-to-noise ratio (SNR). The electronic device (100) can distinguish (or extract or separate) a first audio signal and a noise signal from first audio data. The electronic device (100) can obtain the energy of the first audio signal and the energy of the noise signal. The electronic device (100) can obtain the signal-to-noise ratio (SNR) using 10*log(energy of the first audio signal / energy of the noise signal). The electronic device (100) can obtain sound quality information (or a value representing the sound quality) corresponding to the first audio signal using the signal-to-noise ratio. The electronic device (100) can determine that the sound quality is higher (better) as the signal-to-noise ratio (SNR) increases. Specific descriptions related to this are described in FIGS. 10 and 11.

[0155] According to one embodiment, the electronic device (100) may obtain sound quality information using a recognition rate. The recognition rate may indicate at least one of the probability of recognizing the result of an audio signal or the accuracy associated with the result of the audio signal. The electronic device (100) may obtain a recognition rate for a first audio signal. The electronic device (100) may determine that a higher recognition rate indicates higher sound quality. The recognition rate may be described as a recognition probability, a recognition ratio, a recognition accuracy, etc. Specific descriptions related thereto are provided in FIGS. 12 and 13 .

[0156] According to one embodiment, the electronic device (100) can obtain sound quality information using dynamic range. The dynamic range can represent the difference between the lowest and highest volumes. The electronic device (100) can determine that the sound quality is higher as the dynamic range increases.

[0157] According to one embodiment, the electronic device (100) can obtain sound quality information using a frequency response. The frequency response may include a frequency bandwidth. The electronic device (100) can determine that the wider (larger) the frequency bandwidth, the higher the sound quality.

[0158] According to one embodiment, the electronic device (100) can obtain sound quality information using bit depth or sampling rate. The electronic device (100) can determine that the higher the bit depth or sampling rate, the higher the sound quality.

[0159] According to one embodiment, the electronic device (100) can obtain sound quality information using a bit rate. The electronic device (100) can determine that the higher the bit rate, the higher the sound quality.

[0160] The electronic device (100) can determine a content provision method based on sound quality information (S525). The content may include content related to a first audio signal of a preset type. The content may represent content acquired using the first audio signal.

[0161] FIG. 6 is a diagram for explaining a method of providing content using sound quality information, according to one embodiment.

[0162] Referring to FIG. 6, the electronic device (100) can obtain sound quality information of the first audio signal (S620). Step S620 may correspond to step S520 of FIG. 5.

[0163] The electronic device (100) can determine whether the sound quality information is above a threshold level (S625). The electronic device (100) can determine a level for the sound quality of the first audio signal. The sound quality information can include a sound quality level. The electronic device (100) can obtain a sound quality level for the sound quality of the first audio signal. The electronic device (100) can compare the sound quality level with the threshold level.

[0164] If the sound quality information is above a threshold level, the electronic device (100) can determine that the first audio signal is high quality. The electronic device (100) can provide a high quality first audio signal. If the sound quality information is below a threshold level, the electronic device (100) can determine that the first audio signal is low quality. The electronic device (100) can provide a third audio signal received through an external server instead of providing a low quality first audio signal.

[0165] The sound quality rating can be determined based on a sound quality rating table. A description thereof is provided in Fig. 9.

[0166] If the sound quality information is above a threshold level (S625-Y), the electronic device (100) can obtain an inverted signal corresponding to the first audio data (S630). The inverted signal may represent a signal for offsetting multiple signals included in the first audio data. The inverted signal may be described as an inverted signal, an offsetting signal, an attenuated signal, an inverted waveform, an inverse waveform, an offsetting waveform, an attenuated waveform, etc.

[0167] The electronic device (100) can obtain a second audio signal by synthesizing (or combining) the first audio signal and the inverted signal (S635). The electronic device (100) can provide the second audio signal (S640).

[0168] The operation of providing a second audio signal may indicate performing a noise canceling operation. In the embodiment of Fig. 6, the target noise signal of the noise canceling function may be the entire ambient sound. In the embodiment of Fig. 6, all signals may be canceled (or filtered) without distinguishing between a first audio signal of a preset type and a noise signal. The electronic device (100) may cancel all ambient sounds, but may provide a first audio signal having a threshold level or higher by combining already extracted first audio signals. The electronic device (100) may filter all sounds heard in the ambient environment and obtain a second audio signal by synthesizing the first audio signal and an inverted signal to provide only the extracted first audio signal.

[0169] If the sound quality information is not above a threshold level (S625-N), the electronic device (100) can obtain a third audio signal corresponding to the first audio signal through an external server (S645). The external server may be a server that provides source data for the first audio signal. For example, the external server may include an OTT (Over The Top) server or a content provider server that provides content.

[0170] The electronic device (100) may transmit information including a first audio signal to an external server. The external server may search the first audio signal and identify a third audio signal corresponding to the first audio signal. The external server may transmit the third audio signal to the electronic device (100). The third audio signal may be included in the original source data. For example, the third audio signal may represent a high-quality audio signal provided by a content providing server.

[0171] The electronic device (100) can obtain an inverted signal corresponding to the first audio data (S650). Step S650 may correspond to step S630.

[0172] The electronic device (100) can obtain a fourth audio signal by synthesizing the third audio signal and the inverted signal (S655). The electronic device (100) can provide the fourth audio signal (S660).

[0173] The action of providing a fourth audio signal may indicate performing a noise-cancelling action. The electronic device (100) may cancel all ambient sounds, but instead of the first audio signal determined to be low-quality, receive and provide a high-quality third audio signal from an external server.

[0174] The operation of providing the second audio signal may include an operation of the electronic device (100) directly extracting the first audio signal from the first audio data and providing the first audio signal with the surrounding sound removed.

[0175] The operation of providing the fourth audio signal may include an operation of the electronic device (100) obtaining a third audio signal received from an external server rather than the first audio data, and providing the third audio signal (high-quality signal) with the surrounding sound removed.

[0176] According to various embodiments, the electronic device (100) can obtain a third audio signal corresponding to the first audio signal by using a database included in the electronic device (100) without using an external server.

[0177] FIG. 7 is a diagram for explaining an operation of determining a different content provision method using sound quality information according to one embodiment.

[0178] Steps S720, S725, S730, S745, and S750 of FIG. 7 may correspond to steps S620, S625, S630, S645, and S650 of FIG. 6. For simplicity, redundant descriptions are omitted.

[0179] If the sound quality information is above a threshold level (S725-Y), the electronic device (100) can obtain an inverted signal corresponding to the first audio data (S730). The electronic device (100) can provide the inverted signal (S735). The electronic device (100) can perform a noise-canceling function by providing the inverted signal. The electronic device (100) can provide the first audio signal (S740).

[0180] The electronic device (100) can perform an operation of providing an inverted signal and an operation of providing a first audio signal. By providing the inverted signal, the ambient sound included in the first audio data is attenuated, so the electronic device (100) can provide a high-quality first audio signal while performing a noise-canceling function.

[0181] If the sound quality information is not above a threshold level (S725-N), the electronic device (100) can obtain a third audio signal (S745). The electronic device (100) can obtain an inverted signal corresponding to the first audio data (S750). The electronic device (100) can provide the inverted signal (S755). The electronic device (100) can perform a noise canceling function by providing the inverted signal. The electronic device (100) can provide the third audio signal (S760).

[0182] The electronic device (100) can perform an operation of providing an inverted signal and an operation of providing a third audio signal. By providing an inverted signal, the ambient sound included in the first audio data is attenuated, so the electronic device (100) can provide a high-quality third audio signal while performing a noise-canceling function.

[0183] FIG. 8 is a diagram for explaining audio data according to one embodiment.

[0184] Steps S820, S825, S830, S835, S840, S845, S850, S855, and S860 of FIG. 8 may correspond to steps S620, S625, S630, S635, S640, S645, S650, S655, and S660 of FIG. 6. For simplicity, redundant descriptions are omitted.

[0185] After acquiring the second audio signal, the electronic device (100) can provide second audio data including the second audio signal (S840).

[0186] If the sound quality information is not above a threshold level (S825-N), the electronic device (100) can obtain third audio data including a third audio signal corresponding to the first audio signal through an external server (S845).

[0187] After acquiring the fourth audio signal, the electronic device (100) can provide fourth audio data including the fourth audio signal (S860).

[0188] An audio signal can represent an audio waveform. Audio data can include an audio signal and metadata related to the audio signal. The metadata can include at least one of the following: title, singer, lyricist, composer, lyrics, duration, category, album name, and album release date. Audio data can include various additional information to describe the audio signal, in addition to simply representing the audio waveform.

[0189] FIG. 9 is a drawing for explaining sound quality grading criteria according to one embodiment.

[0190] Referring to FIG. 9, the electronic device (100) may display a sound quality rating table (900) indicating sound quality rating criteria. The sound quality ratings may be categorized into first, second, and third ratings. A higher rating may indicate a relatively higher quality audio signal.

[0191] For example, if the signal-to-noise ratio (SNR) is 10 dB or less, the electronic device (100) may determine that the sound quality information of the first audio signal is level 3. If the signal-to-noise ratio (SNR) is greater than 10 dB and less than or equal to 20 dB, the electronic device (100) may determine that the sound quality information of the first audio signal is level 2. If the signal-to-noise ratio (SNR) exceeds 20 dB, the electronic device (100) may determine that the sound quality information of the first audio signal is level 1.

[0192] For example, if the recognition rate is 70% or less, the electronic device (100) may determine that the sound quality information of the first audio signal is level 3. If the recognition rate is greater than 70% and less than or equal to 90%, the electronic device (100) may determine that the sound quality information of the first audio signal is level 2. If the recognition rate exceeds 90%, the electronic device (100) may determine that the sound quality information of the first audio signal is level 1.

[0193] The number of grades and the criteria for dividing the grades (thresholds, etc.) are only examples and may be changed according to the user's settings.

[0194] FIG. 10 is a diagram for explaining an operation of obtaining sound quality information using a signal-to-noise ratio (SNR), according to one embodiment.

[0195] Referring to FIG. 10, the electronic device (100) can obtain a first audio signal and a noise signal from the first audio data (S1021). The electronic device (100) can extract the first audio signal and the noise signal from the first audio data, respectively.

[0196] The electronic device (100) can obtain a signal-to-noise ratio (SNR) based on the first audio signal and the noise signal (S1022). The electronic device (100) can obtain the energy of the first audio signal and the energy of the noise signal. The electronic device (100) can obtain the signal-to-noise ratio (SNR) using 10*log(energy of the first audio signal / energy of the noise signal). The electronic device (100) can obtain sound quality information (or a value representing sound quality) corresponding to the first audio signal using the signal-to-noise ratio. The electronic device (100) can determine that the sound quality is higher (better) as the signal-to-noise ratio (SNR) increases.

[0197] The electronic device (100) can obtain sound quality information corresponding to the signal-to-noise ratio (SNR) (S1023). The electronic device (100) can obtain (or determine) the sound quality grade of the first audio signal based on the signal-to-noise ratio (SNR) and the sound quality grade table (900) of FIG. 9. The sound quality information can include a sound quality grade related to the sound quality of the first audio signal. The electronic device (100) can obtain the sound quality information based on the signal-to-noise ratio (SNR).

[0198] FIG. 11 is a diagram for explaining an operation of determining a content provision method using a signal-to-noise ratio (SNR), according to one embodiment.

[0199] Steps S1121 and S1122 of FIG. 11 may correspond to steps S1021 and S1022 of FIG. 10. Steps S1130, S1135, S1140, S1145, S1150, S1155, and S1160 of FIG. 11 may correspond to steps S630, S635, S640, S645, S650, S655, and S660 of FIG. 6. For simplicity, redundant descriptions are omitted.

[0200] The electronic device (100) can obtain a first audio signal and a noise signal from the first audio data (S1121). The electronic device (100) can obtain a signal-to-noise ratio (SNR) based on the first audio signal and the noise signal (S1122).

[0201] The electronic device (100) can determine whether the signal-to-noise ratio (SNR) is greater than or equal to a first threshold value (S1125). If the signal-to-noise ratio (SNR) is greater than or equal to the first threshold value (S1125-Y), the electronic device (100) can determine that the first audio signal is of relatively high sound quality. If the signal-to-noise ratio (SNR) is not greater than or equal to the first threshold value (S1125-N), the electronic device (100) can determine that the first audio signal is of relatively low sound quality.

[0202] If the signal-to-noise ratio (SNR) is greater than or equal to the first threshold (S1125-Y), the electronic device (100) may perform steps S1130, S1135, and S1140.

[0203] If the signal-to-noise ratio (SNR) is not greater than the first threshold value (S1125-N), the electronic device (100) may perform steps S1145, S1150, S1155, and S1160 of FIG. 11.

[0204] FIG. 12 is a diagram for explaining an operation of obtaining sound quality information using a recognition rate according to one embodiment.

[0205] Referring to FIG. 12, the electronic device (100) can obtain a first audio signal from the first audio data (S1221). The electronic device (100) can extract (or separate) the first audio signal from the first audio data.

[0206] The electronic device (100) can obtain a recognition rate corresponding to the first audio signal (S1222). The recognition rate may indicate at least one of the probability of recognizing the result of the audio signal or the accuracy associated with the result of the audio signal. The electronic device (100) can obtain a recognition rate for the first audio signal. The electronic device (100) can determine that a higher recognition rate indicates higher sound quality. The recognition rate may be described as a recognition probability, a recognition ratio, or recognition accuracy.

[0207] The electronic device (100) can receive an audio signal as input data and use an artificial intelligence model to provide analysis results and accuracy of the analysis results.

[0208] For example, an artificial intelligence model may be stored in an electronic device (100). The electronic device (100) may input a first audio signal into the artificial intelligence model to obtain analysis results and analysis result accuracy corresponding to the first audio signal. The electronic device (100) may obtain a recognition rate based on the analysis result accuracy. The higher the analysis result accuracy, the higher the recognition rate can be determined by the electronic device (100).

[0209] For example, an artificial intelligence model may be stored in a server connected to an electronic device (100). The electronic device (100) may transmit a first audio signal to the server. The server may input the first audio signal into the artificial intelligence model to obtain an analysis result and an analysis result accuracy corresponding to the first audio signal. The server may transmit the analysis result accuracy to the electronic device (100). The electronic device (100) may obtain a recognition rate based on the analysis result accuracy.

[0210] The electronic device (100) can obtain sound quality information corresponding to the recognition rate (S1223). The electronic device (100) can obtain (or determine) the sound quality grade of the first audio signal based on the recognition rate and the sound quality grade table (900) of FIG. 9. The sound quality information can include a sound quality grade related to the sound quality of the first audio signal. The electronic device (100) can obtain the sound quality information based on the recognition rate.

[0211] FIG. 13 is a diagram for explaining an operation of determining a content provision method using a recognition rate according to one embodiment.

[0212] Steps S1321 and S1322 of FIG. 13 may correspond to steps S1221 and S1222 of FIG. 12. Steps S1330, S1335, S1340, S1345, S1350, S1355, and S1360 of FIG. 13 may correspond to steps S630, S635, S640, S645, S650, S655, and S660 of FIG. 6. Duplicate explanations are omitted.

[0213] The electronic device (100) can obtain a first audio signal from the first audio data (S1321). The electronic device (100) can obtain a recognition rate corresponding to the first audio signal (S1322).

[0214] The electronic device (100) can determine whether the recognition rate is greater than or equal to a second threshold (S1325). If the recognition rate is greater than or equal to the second threshold (S1325-Y), this may indicate that the first audio signal is of relatively high quality. If the recognition rate is not greater than or equal to the second threshold (S1325-N), this may indicate that the first audio signal is of relatively low quality.

[0215] If the recognition rate is greater than or equal to the second threshold value (S1325-Y), the electronic device (100) can perform steps S1330, S1335, and S1340 of FIG. 13.

[0216] If the recognition rate is not higher than the second threshold value (S1325-N), the electronic device (100) can perform steps S1345, S1350, S1355, and S1360 of FIG. 13.

[0217] FIG. 14 is a diagram for explaining an operation of determining a playback point of a third audio signal according to one embodiment.

[0218] Steps S1420, S1425, S1430, S1435, S1440, S1445, S1450, and S1460 of FIG. 14 may correspond to steps S620, S625, S630, S635, S640, S645, S650, and S660 of FIG. 6. Duplicate explanations are omitted.

[0219] The electronic device (100) can obtain a third audio signal corresponding to the first audio signal through an external server (S1445). The electronic device (100) can obtain an inverted signal corresponding to the first audio data (S1450).

[0220] The electronic device (100) can determine a playback point by comparing a first audio signal and a third audio signal (S1456). The first audio signal may be included in the sound output from the surrounding environment of the electronic device (100) in the space where the electronic device (100) is placed. The third audio signal may be audio data acquired through an external server.

[0221] It's necessary to determine which portion of the third audio signal is currently being output from the ambient sound. This is because even when noise filtering is performed, subtle ambient sounds can still be audible to the user. To minimize (or reduce) user discomfort, the third audio signal should be output at the same playback time as the first audio signal, as much as possible.

[0222] Therefore, the electronic device (100) needs to analyze the first audio signal to identify the same playback point as the first audio signal currently being played, and output a third audio signal based on the playback point.

[0223] For example, assuming that the content representing the first audio signal is a music piece lasting a total of 3 minutes and 30 seconds, and that the currently playing portion is 1 minute and 10 seconds, the electronic device (100) may determine 1 minute and 10 seconds as the playback point and provide the portion corresponding to 1 minute and 10 seconds of the third audio signal.

[0224] The electronic device (100) can obtain a fourth audio signal by synthesizing the third audio signal and the inverted signal based on the playback time (S1457). The electronic device (100) can provide the fourth audio signal.

[0225] FIG. 15 is a diagram for explaining an operation of converting a first audio signal according to one embodiment.

[0226] Steps S1520, S1525, S1530, S1540, S1545, S1550, S1555, and S1560 of FIG. 15 may correspond to steps S620, S625, S630, S640, S645, S650, S655, and S660 of FIG. 6. Duplicate explanations are omitted.

[0227] If the sound quality information is above a threshold level (S1525-Y), the electronic device (100) can obtain an inverted signal corresponding to the first audio data (S1530). The electronic device (100) can obtain a converted signal by converting the first audio signal in a preset manner (S1531).

[0228] The preset method may include at least one of converting a mono audio signal to a stereo audio signal or performing 360-degree rendering.

[0229] For example, the electronic device (100) can obtain a conversion signal by converting the first audio signal by changing the audio output format (mono, stereo).

[0230] The electronic device (100) can identify the output format of the first audio signal. The output format may include at least one of a mono type or a stereo type. The mono type may include a single audio channel. The stereo type may include two audio channels. The stereo type may refer to a left (L) channel and a right (R) channel.

[0231] If the output format of the first audio signal is mono, the electronic device (100) can convert the output format of the first audio signal to stereo. The electronic device (100) can obtain a conversion signal as a result of the conversion operation.

[0232] For example, the electronic device (100) can obtain a conversion signal by performing 360-degree rendering on the first audio signal.

[0233] 360-degree rendering may include rendering for playback from various spatial orientations. 360-degree rendering may include audio rendering provided in a VR or AR environment. 360-degree rendering may include at least one of the following: isolating (or identifying) audio objects, determining the spatial location of audio objects, performing 3D audio rendering, determining the user's location, and dynamically adjusting audio based on the user's location.

[0234] The electronic device (100) can obtain a second audio signal by synthesizing a conversion signal and an inversion signal (S1535). The electronic device (100) can provide the second audio signal (S1540).

[0235] FIG. 16 is a diagram for explaining an operation of filtering only noise, according to one embodiment.

[0236] Steps S1620, S1625, S1645, S1650, S1655, and S1660 of FIG. 16 may correspond to steps S620, S625, S645, S650, S655, and S660 of FIG. 6. Duplicate explanations are omitted.

[0237] If the sound quality information is above a threshold level (S1625-Y), the electronic device (100) can obtain (or extract) a noise signal from the first audio data (S1630). The electronic device (100) can obtain a noise inversion signal based on the noise signal (S1635).

[0238] The inversion signal described in Fig. 6 may be a signal for attenuating the entire surrounding sound included in the first audio data.

[0239] The noise inversion signal described in Fig. 16 may be a signal for attenuating only noise among the entire surrounding sound included in the first audio data.

[0240] The electronic device (100) can provide a noise inversion signal (S1640). When providing the noise inversion signal, the electronic device (100) can attenuate noise in ambient sound. The user can selectively listen to a first audio signal with noise removed from among various sounds included in the ambient sound.

[0241] FIG. 17 is a drawing for explaining a UI (User Interface) related to a content provision service according to one embodiment.

[0242] The electronic device (100) may provide a screen (1710) indicating whether a function for providing content included in ambient sound is performed. The screen (1710) may include at least one of a UI (1711) for determining whether a function is performed and a UI (1712) for explaining an operation performed by the electronic device (100).

[0243] The electronic device (100) may provide a screen (1720) that provides information about content included in the ambient sound. The screen (1720) may include at least one of a UI (1721) for determining whether to perform a function and a UI (1722) for describing the searched content.

[0244] FIG. 18 is a drawing for explaining an operation of providing an audio signal using a sound device (200), according to one embodiment.

[0245] Steps S1805 and S1815 of FIG. 18 may correspond to steps S505 and S515 of FIG. 5. Steps S1820, S1825, S1830, S1835, and S1840 of FIG. 18 may correspond to steps S620, S625, S630, S635, and S640 of FIG. 6. Duplicate descriptions are omitted. Some of the operations performed in FIGS. 5 and 6 may be performed in the sound device (200).

[0246] Referring to FIG. 18, the sound device (200) can obtain first audio data for ambient sound (S1805). The sound device (200) can transmit the first audio data to the electronic device (100) (S1806).

[0247] The electronic device (100) can receive first audio data from the sound device (200). The electronic device (100) can obtain a first audio signal from the first audio data (S1815). The electronic device (100) can obtain sound quality information of the first audio signal (S1820).

[0248] The electronic device (100) can determine whether the sound quality information of the first audio signal is above a threshold level (S1825).

[0249] If the sound quality information of the first audio signal is above a threshold level (S1825-Y), the electronic device (100) can obtain an inverted signal corresponding to the first audio data (S1830). The electronic device (100) can obtain a second audio signal by synthesizing the first audio signal and the inverted signal (S1835).

[0250] The electronic device (100) can transmit a second audio signal to the sound device (200) (S1836). The sound device (200) can receive the second audio signal from the electronic device (100). The sound device (200) can provide the second audio signal (S1840).

[0251] If the sound quality information of the first audio signal is not above a threshold level (S1825-N), the electronic device (100) may transmit the first audio signal to an external server (300) (S1841). The electronic device (100) may request high-quality data (or signal) corresponding to the first audio signal from the external server (300). Operations related thereto are described in FIG. 19.

[0252] FIG. 19 is a drawing for explaining an operation of providing an audio signal using a sound device (200), according to one embodiment.

[0253] Steps S1905 and S1915 of FIG. 19 may correspond to steps S505 and S515 of FIG. 5. Steps S1920, S1925, S1945, S1950, S1955, and S1960 of FIG. 19 may correspond to steps S620, S625, S645, S650, S655, and S660 of FIG. 6. Duplicate descriptions are omitted. Some of the operations performed in FIGS. 5 and 6 may be performed in the sound device (200).

[0254] Steps S1905, S1906, S1915, S1920, S1925, and S1941 of FIG. 19 may correspond to steps S1805, S1806, S1815, S1820, S1825, and S1841 of FIG. 18. Duplicate explanations are omitted.

[0255] If the sound quality information is not above a threshold level (S1925-N), the electronic device (100) may transmit the first audio signal to the external server (300) (S1941). The electronic device (100) may request original data or high-quality data corresponding to the first audio signal from the external server (300). The electronic device (100) may transmit a data packet including the first audio signal and a control signal requesting data corresponding to the first audio signal to the external server (300).

[0256] An external server (300) can receive a first audio signal from an electronic device (100). The external server (300) can obtain a third audio signal corresponding to the first audio signal (S1945). The external server (300) can transmit the third audio signal to the electronic device (100) (S1946).

[0257] The electronic device (100) can receive a third audio signal from an external server (300). The electronic device (100) can obtain an inverted signal corresponding to the first audio signal (S1950). The electronic device (100) can synthesize the third audio signal and the inverted signal to obtain a fourth audio signal (S1955). The electronic device (100) can transmit the fourth audio signal to the sound device (200) (S1956).

[0258] The sound device (200) can receive a fourth audio signal from the electronic device (100). The sound device (200) can provide the fourth audio signal (S1960).

[0259] FIG. 20 is a drawing for explaining an operation of providing an audio signal using a sound device (200), according to one embodiment.

[0260] Steps S2005 and S2015 of FIG. 20 may correspond to steps S505 and S515 of FIG. 5. Steps S2020, S2025, S2030, S2035, S2040, S2045, S2050, S2055, and S2060 of FIG. 20 may correspond to steps S620, S625, S630, S635, S640, S645, S650, S655, and S660. Duplicate descriptions are omitted. Some of the operations of FIG. 6 may be performed by the sound device (200).

[0261] The sound device (200) can obtain first audio data for ambient sound (S2005). The sound device (200) can obtain a first audio signal from the first audio data (S2015). The sound device (200) can obtain sound quality information of the first audio signal (S2020). The sound device (200) can determine whether the sound quality information is above a threshold level (S2025).

[0262] If the sound quality information is above a threshold level (S2025-Y), the sound device (200) can obtain an inverted signal corresponding to the first audio data (S2030). The sound device (200) can obtain a second audio signal by synthesizing the first audio signal and the inverted signal (S2035). The sound device (200) can provide the second audio signal (S2040).

[0263] If the sound quality information is not above a threshold level (S2025-N), the sound device (200) can transmit the first audio signal to the external server (300) (S2041).

[0264] An external server (300) can receive a first audio signal from a sound device (200). The external server (300) can obtain a third audio signal corresponding to the first audio signal (S2045). The external server (300) can transmit the third audio signal to the sound device (200) (S2046).

[0265] The sound device (200) can receive a third audio signal from an external server (300). The sound device (200) can obtain an inverted signal corresponding to the first audio data (S2050). The sound device (200) can obtain a fourth audio signal by synthesizing the third audio signal and the inverted signal (S2055). The sound device (200) can provide the fourth audio signal (S2060).

[0266] FIG. 21 is a drawing for explanation according to one embodiment. The embodiment of the present disclosure can be used in combination with other embodiments.

[0267] Referring to FIG. 21, a method for controlling an electronic device communicating with an external server includes a step of obtaining audio data for ambient sound (S2105), a step of obtaining a target audio signal of a preset type from the audio data (S2110), a step of obtaining sound quality information of the target audio signal (S2115), a step of transmitting the target audio signal to an external server if the sound quality information is not higher than a threshold level (S2120), a step of receiving a source audio signal corresponding to the target audio signal from the external server (S2125), a step of identifying a reproduction time of the source audio signal based on the target audio signal (S2130), and a step of providing a synthesized audio signal obtained by synthesizing an inverted signal corresponding to the audio data and the source audio signal based on the reproduction time (S2135).

[0268] The step of obtaining sound quality information (S2115) may include obtaining a target audio signal and a noise signal from audio data, obtaining a signal-to-noise ratio (SNR) based on the target audio signal and the noise signal, and obtaining sound quality information based on the signal-to-noise ratio (SNR).

[0269] The step of transmitting a target audio signal to an external server (S2120) may transmit the target audio signal to the external server if the signal-to-noise ratio (SNR) is less than a threshold value.

[0270] The step of obtaining sound quality information (S2115) may determine a sound quality grade of the first audio signal based on a signal-to-noise ratio (SNR) and a grade table for sound quality stored in an electronic device, and obtain sound quality information including the sound quality grade. “Based on” may be used as “based on at least one.”

[0271] The control method may further include a step of obtaining an inverted signal by inverting the phase of a signal included in the audio data to attenuate ambient sound.

[0272] The step (S2135) of providing a synthetic audio signal may be performed by comparing the waveform of the target audio signal with the waveform of the source audio signal to determine the playback point of the source audio signal, synthesizing an area corresponding to the playback point among the entire area of ​​the inverted signal and the source audio signal to obtain a synthetic audio signal, and providing the synthetic audio signal.

[0273] The control method may further comprise the step of transmitting the synthesized audio signal to an external sound device.

[0274] The external sound device is a wireless earphone or wireless headphone, and the step (S2120) of transmitting a synthesized audio signal to the external sound device includes a communication circuit and can transmit the synthesized audio signal to the external sound device through a Bluetooth communication module included in the electronic device.

[0275] The synthetic audio signal is a first synthetic audio signal, and the control method may further include a step of obtaining a second synthetic audio signal by synthesizing an inverted signal corresponding to audio data and a target audio signal when sound quality information is above a threshold level, and a step of providing the second synthetic audio signal.

[0276] The control method further includes a step of obtaining a conversion signal by converting the target audio signal into a stereo type when the output method of the target audio signal is a mono type, and the step of obtaining a second synthesized audio signal may obtain a second synthesized audio signal by synthesizing the conversion signal and the inverted signal.

[0277] The methods according to the various embodiments of the present disclosure described above can be implemented in the form of an application that can be installed on an existing electronic device.

[0278] The methods according to the various embodiments of the present disclosure described above can be implemented only with a software upgrade or a hardware upgrade for an existing electronic device.

[0279] The various embodiments of the present disclosure described above may also be performed through an embedded server provided in an electronic device, or an external server of at least one of the electronic device and the display device.

[0280] According to an example embodiment of the present disclosure, the various embodiments described above may be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device may include an electronic device according to the disclosed embodiments, which is a device that can call instructions stored in the storage medium and operate according to the called instructions. When the instructions are executed by a processor, the processor may directly or under the control of the processor use other components to perform a function corresponding to the instructions. The instructions may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not contain signals and is tangible, but does not distinguish between whether data is stored semi-permanently or temporarily in the storage medium.

[0281] Each "processor" may include processing circuitry and / or multiple processors. For example, a "processor" as used herein may include various processing circuitry including at least one processor, wherein one or more of said at least one processor may be individually and / or collectively distributed and configured to perform various functions. In this disclosure, when "processor," "at least one processor," "one or more processors," etc. are described as being configured to perform multiple functions, these terms do not limit the case where one processor performs some functions and another processor performs some functions. The processor(s) perform different functions among the described functions, and also perform situations where a single processor can perform all of the described functions. Additionally, the at least one processor may include a combination of processors that perform various functions, for example, in a distributed manner. At least one processor may execute program instructions to achieve or perform various functions.

[0282] According to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0283] Each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of a single or multiple entities, and some of the sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the respective components prior to integration. Operations performed by modules, programs or other components according to various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0284] While the present disclosure has been illustrated and described with respect to preferred embodiments thereof, it is to be understood that the present disclosure is not limited to the specific embodiments described above, and that various modifications and variations may be made by those skilled in the art to which the present disclosure pertains without departing from the scope of the present disclosure as claimed in the claims. Furthermore, such modifications should not be individually understood from the technical spirit of the present disclosure. While the present disclosure has been described with reference to various implementation examples, it should be understood that these are intended to be illustrative rather than limiting. Those skilled in the art will appreciate that various changes in form and detail are possible, as long as such changes do not depart from the overall scope of the invention. It will also be understood that any implementation example described herein can be used in conjunction with several other implementation examples.

Claims

1. In electronic devices, memory; A communication interface including a communication circuit and communicating with an external server; At least one processor comprising a processing circuit; At least one processor of the above, Obtain audio data about ambient sounds, Control to obtain a target audio signal of a preset type from the above audio data, Obtaining sound quality information of the above target audio signal, If the above sound quality information is not above a threshold level, the target audio signal is transmitted to the external server through the communication interface, Through the above communication interface, a source audio signal corresponding to the target audio signal is received from the external server, Identifying the playback point of the source audio signal based on the target audio signal, An electronic device that provides a synthesized audio signal that synthesizes an inverted signal corresponding to the audio data and the source audio signal based on the playback point in time.

2. In paragraph 1, At least one of the processors, individually and / or collectively, Obtaining the target audio signal and noise signal from the above audio data, Obtaining a signal-to-noise ratio (SNR) based on the target audio signal and the noise signal, An electronic device that obtains sound quality information based on the signal-to-noise ratio (SNR).

3. In paragraph 2, At least one of the processors, individually and / or collectively, An electronic device that controls transmission of the target audio signal to the external server through the communication interface when the signal-to-noise ratio (SNR) is less than a threshold value.

4. In paragraph 2, At least one of the processors, individually and / or collectively, Determine the sound quality grade of the first audio signal based on the signal-to-noise ratio (SNR) and the sound quality grade table stored in the memory, An electronic device that obtains sound quality information including the above sound quality grade.

5. In paragraph 1, At least one of the processors, individually and / or collectively, An electronic device that obtains the inverted signal by inverting the phase of a signal included in the audio data to attenuate the above-described ambient sound.

6. In paragraph 1, At least one of the processors, individually and / or collectively, Comparing the waveform of the target audio signal with the waveform of the source audio signal to determine the playback point of the source audio signal, The synthesized audio signal is obtained by synthesizing the area corresponding to the playback time among the entire area of ​​the inverted signal and the source audio signal, An electronic device providing the above-mentioned synthetic audio signal.

7. In paragraph 1, At least one of the processors, individually and / or collectively, An electronic device that controls the transmission of the synthesized audio signal to an external sound device through the communication interface.

8. In paragraph 7, The above external sound device, Wireless earphones and / or wireless headphones, At least one of the processors, individually and / or collectively, An electronic device comprising a communication circuit, and controlling the transmission of the synthesized audio signal to the external sound device through a Bluetooth communication module included in the communication interface.

9. In paragraph 1, The above synthetic audio signal is a first synthetic audio signal, At least one of the processors, individually and / or collectively, If the above sound quality information is above a threshold level, a second synthesized audio signal is obtained by synthesizing the inverted signal corresponding to the audio data and the target audio signal, An electronic device providing the second synthetic audio signal.

10. In paragraph 9, At least one of the processors, individually and / or collectively, If the output method of the above target audio signal is mono type, a conversion signal is obtained by converting the above target audio signal to stereo type, An electronic device that obtains the second synthesized audio signal by synthesizing the above-described converted signal and the above-described inverted signal.

11. A method for controlling an electronic device that communicates with an external server, A step of acquiring audio data about ambient sound; A step of obtaining a target audio signal of a preset type from the above audio data; A step of obtaining sound quality information of the above target audio signal; If the above sound quality information is not above a threshold level, a step of transmitting the target audio signal to the external server; A step of receiving a source audio signal corresponding to the target audio signal from the external server; A step of identifying a playback point of the source audio signal based on the target audio signal; and A control method, comprising: providing a synthesized audio signal by synthesizing an inverted signal corresponding to the audio data and the source audio signal based on the playback time; 12. In paragraph 11, The step of obtaining the above sound quality information is: Obtaining the target audio signal and noise signal from the above audio data, Obtaining a signal-to-noise ratio (SNR) based on the target audio signal and the noise signal, A control method for obtaining sound quality information based on the signal-to-noise ratio (SNR).

13. In paragraph 12, The step of transmitting the target audio signal to the external server is: A control method for transmitting the target audio signal to the external server when the signal-to-noise ratio (SNR) is less than a threshold value.

14. In paragraph 12, The step of obtaining the above sound quality information is: Determine the sound quality grade of the first audio signal based on the signal-to-noise ratio (SNR) and the sound quality grade table stored in the electronic device, A control method for obtaining sound quality information including the above sound quality grade.

15. In paragraph 11, The above control method is, A control method further comprising: a step of obtaining the inverted signal by inverting the phase of a signal included in the audio data to attenuate the ambient sound;

Images