Electronic device for performing active noise canceling and control method therefor

Abstract

An electronic device is disclosed. The electronic device comprises: a speaker; a first microphone; a second microphone; a memory for storing one or more computer programs; and one or more processors communicatively connected to the speaker, the first microphone, the second microphone, and the memory, wherein the one or more computer programs include computer-executable instructions and, when individually or collectively executed by the one or more processors, may control an external speaker to output sound of a first frequency, obtain a first sound signal corresponding to the sound of the first frequency through the first microphone, obtain a second sound signal including the first sound signal through the second microphone, obtain a first noise canceling signal on the basis of the first sound signal and the second sound signal, output, through the speaker, a second noise canceling signal, in which at least one of the phase or the gain of the first noise canceling signal has been changed, and a third sound signal of a second frequency, obtain the second sound signal by changing at least one of the phase or the gain, and identify at least one of a target phase or a target gain on the basis of a third frequency component that is related to distortion product otoacoustic emission (DPOAE) of the first frequency and the second frequency and is included in the second sound signal.

Description

Electronic device for performing active noise cancellation and method for controlling the same

[0001] The present disclosure relates to an electronic device and a control method thereof. More specifically, the present disclosure relates to an electronic device that performs active noise cancellation and a control method thereof.

[0002] Advances in electronic technology have led to the development of electronic devices offering a variety of functions. In particular, earphones equipped with various features, such as augmented listening, head tracking, audio recognition, and spatial sound, have recently become widespread.

[0003] In particular, recently released earphones are improving user convenience by blocking out external noise through noise cancellation functions.

[0004] The above information is provided solely as background information to aid in understanding the disclosure. No determination has been made, nor is any assertion made, as to whether any of the above information constitutes prior art in connection with the disclosure.

[0005] An aspect of the present disclosure is to address at least the problems and / or disadvantages mentioned above and to provide at least the advantages described below. Accordingly, an aspect of the present disclosure is to provide an electronic device that performs active noise cancellation and a method for controlling the same.

[0006] Additional aspects will be presented in part in the description that follows, and in part will be obvious from the description or may be learned through practice of the embodiments presented.

[0007] According to one embodiment of the present disclosure to achieve the above purpose, an electronic device is provided. An electronic device includes a speaker, a first microphone, a second microphone, a memory storing one or more computer programs, and one or more processors communicatively connected to the speaker, the first microphone, the second microphone, and the memory, wherein the one or more computer programs include computer-executable instructions, and when individually or collectively executed by the one or more processors, control an external speaker to output a sound of a first frequency, obtain a first sound signal corresponding to the sound of the first frequency through the first microphone, obtain a second sound signal including the first sound signal through the second microphone, obtain a first noise-canceling signal based on the first sound signal and the second sound signal, output a second noise-canceling signal in which at least one of a phase or a gain of the first noise-canceling signal is changed, and obtain the second sound signal by changing at least one of the phase or the gain, and obtain the second sound signal, and change the phase or the gain of the first frequency and the second frequency included in the second sound signal. At least one of the target phase or target gain can be identified based on a third frequency component associated with the displaced positive output acoustic emission (DPOAE).

[0008] In addition, the one or more computer programs further include computer-executable instructions, which, when individually or collectively executed by the one or more processors, can identify a phase section in which the modulation-induced acoustic reflection does not occur by gradually changing the phase, and identify the target phase based on the phase section.

[0009] And, the one or more computer programs further include computer-executable instructions, which, when individually or collectively executed by the one or more processors, can identify the center of the phase interval as the target phase.

[0010] In addition, the one or more computer programs further include computer-executable instructions, which, when individually or collectively executed by the one or more processors, can identify a minimum amplitude at which the modulation otoacoustic reflection occurs by changing the amplitude of the sound of the first frequency when the gain is 0, and identify the target gain based on the minimum amplitude.

[0011] And, the one or more computer programs further include computer-executable instructions, which, when individually or collectively executed by the one or more processors, identify half of the minimum amplitude as the amplitude of the sound of the first frequency, and identify the minimum gain at which the modulation otoacoustic reflection occurs by changing the gain when the amplitude of the sound of the first frequency is half of the minimum amplitude, and identify the minimum gain as the target gain.

[0012] In addition, the one or more computer programs further include computer-executable instructions, and when individually or collectively executed by the one or more processors, change the first frequency and the second frequency, repeat the target phase identification operation and the target gain identification operation, and map a plurality of target phases and a plurality of target gains obtained according to the repeating operation to the corresponding first frequency and the second frequency and store them in a memory.

[0013] And, the one or more computer programs further include computer-executable instructions, and when individually or collectively executed by the one or more processors, acquire an external sound signal corresponding to an external sound through the first microphone, acquire a fourth sound signal including the external sound signal through the second microphone, obtain a third noise-canceling signal by actively noise-canceling the external sound signal and the fourth sound signal, and output a fourth noise-canceling signal in which the phase and gain of the third noise-canceling signal are changed based on a target phase and a target gain corresponding to a frequency of the external sound among the information stored in the memory, through the speaker.

[0014] Additionally, the one or more computer programs further include computer-executable instructions, which, when individually or collectively executed by the one or more processors, can control the communication interface to transmit a control signal to the external speaker to output sound of the first frequency.

[0015] And, the one or more computer programs further include computer-executable instructions, and when individually or collectively executed by the one or more processors, can receive the second sound signal including the first sound signal, the second noise canceling signal, the third sound signal, and a sound signal corresponding to the third frequency component through the second microphone.

[0016] In addition, the one or more computer programs further include computer-executable instructions, and when individually or collectively executed by the one or more processors, can obtain an antiphase signal of a signal including the first sound signal and the second sound signal as the first noise-cancelling signal.

[0017] Meanwhile, according to one embodiment of the present disclosure, a method for controlling an electronic device is provided. The control method may include: controlling an external speaker to output a sound of a first frequency; obtaining a first sound signal corresponding to the sound of the first frequency through a first microphone of the electronic device; obtaining a second sound signal including the first sound signal through a second microphone of the electronic device; obtaining a first noise-canceling signal based on the first sound signal and the second sound signal; outputting a second noise-canceling signal in which at least one of a phase or a gain of the first noise-canceling signal is changed and a third sound signal of a second frequency through the speaker of the electronic device; obtaining the second sound signal by changing at least one of the phase or the gain; and identifying at least one of a target phase or a target gain based on a third frequency component related to a modulation over-the-ear acoustic emission (DPOAE) of the first frequency and the second frequency included in the second sound signal.

[0018] In addition, the identifying step can identify a phase section in which the modulation acoustic reflection does not occur by gradually changing the phase, and identify the target phase based on the phase section.

[0019] And, the identifying step can identify the center of the phase section as the target phase.

[0020] In addition, the identifying step may identify the minimum amplitude at which the modulation-induced acoustic reflection occurs by changing the amplitude of the sound of the first frequency when the gain is 0, and identify the target gain based on the minimum amplitude.

[0021] And, the identifying step may identify a half value of the minimum amplitude as the amplitude of the sound of the first frequency, and when the amplitude of the sound of the first frequency is half value of the minimum amplitude, identify a minimum gain at which the modulation induced acoustic reflection occurs by changing the gain, and identify the minimum gain as the target gain.

[0022] In addition, the method may further include a step of changing the first frequency and the second frequency, repeating the target phase identification operation and the target gain identification operation, and a step of mapping a plurality of target phases and a plurality of target gains obtained according to the repeating operation to corresponding first frequencies and second frequencies and storing the same in a memory of the electronic device.

[0023] And, the method may further include a step of obtaining an external sound signal corresponding to an external sound through the first microphone, obtaining a fourth sound signal including the external sound signal through the second microphone, a step of obtaining a third noise canceling signal by actively noise canceling the external sound signal and the fourth sound signal, and a step of outputting a fourth noise canceling signal in which the phase and gain of the third noise canceling signal are changed based on a target phase and a target gain corresponding to the frequency of the external sound among the information stored in the memory through the speaker.

[0024] Additionally, the controlling step can transmit a control signal to the external speaker to output sound of the first frequency.

[0025] And, the step of obtaining the first sound signal and the second sound signal may include receiving the second sound signal including the first sound signal, the second noise canceling signal, the third sound signal, and a sound signal corresponding to the third frequency component through the second microphone.

[0026] In addition, the step of obtaining the first noise canceling signal may obtain an inverse phase signal of a signal including the first sound signal and the second sound signal as the first noise canceling signal.

[0027] In accordance with one aspect of the disclosure, one or more non-transitory computer-readable storage media are provided that store computer-executable instructions that, when executed individually or collectively by one or more processors, cause an electronic device to perform a task. Such an operation may include the steps of controlling an external speaker to output a sound of a first frequency, obtaining a first sound signal corresponding to the sound of the first frequency through a first microphone of the electronic device, obtaining a second sound signal including the first sound signal through a second microphone of the electronic device, obtaining a first noise canceling signal based on the first sound signal and the second sound signal, outputting a second noise canceling signal in which at least one of a phase or a gain of the first noise canceling signal is changed and a third sound signal of a second frequency through the speaker of the electronic device, obtaining the second sound signal by changing at least one of the phase or the gain, and identifying at least one of a target phase or a target gain based on a third frequency component related to a modulation over-the-ear acoustic emission (DPOAE) of the first frequency and the second frequency included in the second sound signal.

[0028] Other aspects, advantages and important features of the disclosure will become apparent to those skilled in the art from the following detailed description, which discloses various embodiments of the disclosure together with the accompanying drawings.

[0029] The above and other aspects, features and advantages of specific embodiments of the disclosure will become more apparent in the following description taken in conjunction with the accompanying drawings, in which:

[0030] FIGS. 1A to 1C are drawings for explaining noise canceling to help understanding according to various embodiments of the present disclosure.

[0031] FIG. 2 is a block diagram showing the configuration of an electronic device according to an embodiment of the present disclosure.

[0032] FIG. 3 is a block diagram showing a detailed configuration of an electronic device according to an embodiment of the present disclosure.

[0033] FIG. 4 is a diagram for explaining distortion product otoacoustic emissions (DPOAE) according to one embodiment of the present disclosure.

[0034] FIGS. 5 and 6 are drawings for explaining noise canceling according to various embodiments of the present disclosure.

[0035] FIG. 7 is a diagram illustrating a method for identifying a target phase according to an embodiment of the present disclosure.

[0036] FIG. 8 is a diagram illustrating a method for identifying a target gain according to an embodiment of the present disclosure.

[0037] FIGS. 9, 10, 11, and 12 are drawings for explaining the effect of noise canceling according to various embodiments of the present disclosure.

[0038] FIG. 13 is a flowchart for explaining a method for controlling an electronic device according to an embodiment of the present disclosure.

[0039] It should be noted that the same reference numbers are used throughout the drawings to describe identical or similar elements, features and structures.

[0040] The following description, with reference to the accompanying drawings, is provided to facilitate a comprehensive understanding of various embodiments of the disclosure as defined by the claims and their equivalents. While it includes numerous specific details to aid understanding, these should be considered illustrative only. Accordingly, those skilled in the art will recognize that various changes and modifications to the various embodiments described herein can be made without departing from the scope and spirit of the disclosure. Furthermore, descriptions of well-known functions and configurations may be omitted for clarity and brevity.

[0041] The terms and words used in the following description and claims are not intended to be limited to their bibliographic meanings, but are merely used to ensure a clear and consistent understanding of the present disclosure by the inventors. Therefore, it will be apparent to those skilled in the art that the following description of various embodiments of the present disclosure is provided for illustrative purposes only and is not intended to limit the disclosure defined by the appended claims and their equivalents.

[0042] It should be understood that the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to "a component surface" includes reference to one or more of these surfaces.

[0043] The purpose of the present disclosure is to provide an electronic device and a control method thereof for performing noise cancellation with the ear membrane of a user, rather than an electronic device, as a reference position.

[0044] Hereinafter, the present disclosure will be described with reference to the attached drawings.

[0045] The terms used in the embodiments of this disclosure are selected from widely used, current terms based on the functions described herein. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, in which case their meanings will be described in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of the disclosure.

[0046] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.

[0047] The expression "at least one of A and / or B" should be understood to mean either "A" or "B" or "A and B".

[0048] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0049] In this application, terms such as “comprise” or “consist of” are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but should be understood not to exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0050] In this specification, the term user may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0051] Various embodiments of the present disclosure are described below with reference to the attached drawings.

[0052] It should be recognized that each block in the flowchart and the combination of flowcharts can be performed by one or more computer programs containing computer-executable instructions. The entirety of one or more computer programs may be stored in a single memory device, or the one or more computer programs may be divided into different parts stored in different memory devices.

[0053] The functions or operations described herein may be performed by a single processor or a combination of processors. A single processor or a combination of processors is a circuit that performs processing and includes an application processor (AP, e.g., a central processing unit (CPU)), a communication processor (CP, e.g., a modem), a graphics processing unit (GPU), a neural processing unit (NPU) (e.g., an artificial intelligence (AI) chip), a wireless fidelity (Wi-Fi) chip, a Bluetooth™ chip, a global positioning system (GPS) chip, a near-field communication (NFC) chip, a connection chip, a sensor controller, a touch controller, a fingerprint sensor controller, a display driver integrated circuit (IC), an audio codec chip, a universal serial bus (USB) controller, a camera controller, an image processing IC, a microprocessor unit (MPU), a system on a chip (SoC), an IC, or similar circuits.

[0054] FIGS. 1A, 1B, and 1C are drawings illustrating noise canceling to aid understanding according to various embodiments of the present disclosure.

[0055] According to FIGS. 1A, 1B, and 1C, an electronic device performing noise cancellation may include one speaker and two microphones. For example, as illustrated in FIG. 1A, the electronic device may include a speaker for outputting sound Y(z) to the external auditory canal of a user wearing the electronic device, an external microphone for receiving external sound K(z), and an in-ear microphone for receiving external sound L(z) and sound inside the external auditory canal of the user.

[0056] Here, the sound path of the electronic device may include a primary path and an auxiliary path. For example, the primary path is the transmittance between the external microphone and the in-ear microphone, which represents how external sound changes when entering the ear, and can be expressed as P(z)=L(z) / K(z). The auxiliary path is the transmittance between the speaker and the in-ear microphone, and can be expressed as S(z).

[0057] In noise-canceling algorithms, auxiliary paths are measured at initialization and simulated within the noise-canceling algorithm, allowing their response to a given output (S^(z)) to be predicted. This allows for the use of an at least partially adaptive internal model control (IMC) approach in real-time simulated acoustic systems.

[0058] Noise cancellation may include calculating the in-ear signal via D(z)=P(z)X(z) and calculating the speaker signal via Y(z)=D(z) / S(z). However, Y(z) cannot be perfectly calculated because S(z) has a longer delay time than P(z). This limitation may also affect the convergence of the adaptive filter used for estimation.

[0059] That is, as illustrated in Fig. 1b, noise-cancelling performance may deteriorate due to the gap between the in-ear microphone and the ear membrane. To address this, equalization (EQ) is required. In the absence of equalization, the electronic device serves as the reference point for noise-cancelling. However, in the presence of equalization, the user's ear membrane may serve as the reference point for noise-cancelling.

[0060] In this case, as illustrated in Fig. 1c, noise is reduced in both the case without equalization and the case with equalization compared to the case without wearing the electronic device, and noise canceling performance can be improved in the case with equalization compared to the case without equalization. In other words, the equalization can fill the gap between the in-ear microphone and the ear membrane.

[0061] FIG. 2 is a block diagram showing the configuration of an electronic device (100) according to one embodiment of the present disclosure.

[0062] According to FIG. 2, the electronic device (100) is a device worn by a user to output sound, and can be implemented as an earphone, a headset, etc.

[0063] A user wearing an electronic device (100) can hear not only the sound output from the electronic device (100) but also external noise of the electronic device (100). The electronic device (100) may be a device equipped with a noise cancellation function that removes external noise as noise.

[0064] Noise-cancelling technology can be a technology that blocks or cancels out external noise, helping you hear clearly without any noise. For example, noise-cancelling can include active noise-cancelling (ANC), which generates an opposite waveform to cancel out external noise, and passive noise-cancelling (PNC), which blocks noise by physically covering the ear.

[0065] The electronic device (100) is worn by the user and outputs sound, and any device equipped with a noise-canceling function may be used.

[0066] According to FIG. 2, the electronic device (100) includes a speaker (110), a first microphone (120), a second microphone (130), and a processor (140).

[0067] The speaker (110) is a component that outputs various audio data processed by the processor (140) as well as various notification sounds or voice messages. For example, the processor (140) can output signals of some channels included in the sound through the speaker (110).

[0068] The speaker (110) may be implemented in multiple ways. For example, if the electronic device (100) includes a first body and a second body, the speaker (110) may include a first speaker provided in the first body and a second speaker provided in the second body. However, this is not limited thereto, and the implementation method of the speaker (110) may vary.

[0069] The speaker (110) may be provided in a direction in which sound can be output from the electronic device (100) to the user's ear membrane. For example, when the electronic device (100) is worn on the user's ear, the speaker (110) may be implemented to be positioned in a direction in which sound is output to the ear membrane through the user's external auditory canal. That is, when the electronic device (100) is worn on the user, the entrance to the user's external auditory canal may be blocked by the electronic device (100), and the sound output by the speaker (110) may be provided from the entrance to the external auditory canal to the ear membrane. For the convenience of explanation, the configuration exposed to the outside of the electronic device (100) after the electronic device (100) is worn on the user is expressed as being implemented in the first region of the electronic device (100), and the configuration provided in the direction of the entrance to the external auditory canal of the electronic device (100) is expressed as being implemented in the second region of the electronic device (100). In this case, the speaker (110) can be expressed as being implemented in the second area of ​​the electronic device (100).

[0070] Each of the first microphone (120) and the second microphone (130) is configured to receive sound and convert it into an audio signal. Each of the first microphone (120) and the second microphone (130) is electrically connected to a processor (140) and can receive sound under the control of the processor (140).

[0071] For example, a first microphone (120) may be implemented in a first area of ​​an electronic device (100) to receive external noise of the electronic device (100). A second microphone (130) may be implemented in a second area of ​​the electronic device (100) to receive sound reflected from the external auditory canal and ear membrane, etc., output through a speaker (110).

[0072] Each of the first microphone (120) and the second microphone (130) may include various configurations such as a microphone that collects sound in analog form, an amplifier circuit that amplifies the collected sound, an A / D conversion circuit that samples the amplified sound and converts it into a digital signal, and a filter circuit that removes noise components from the converted digital signal.

[0073] Meanwhile, each of the first microphone (120) and the second microphone (130) may be implemented in the form of a sound sensor, and any method may be used as long as it is configured to collect sound.

[0074] The processor (140) controls the overall operation of the electronic device (100). Specifically, the processor (140) is connected to each component of the electronic device (100) and can control the overall operation of the electronic device (100). For example, the processor (140) is connected to components such as a speaker (110), a communication interface (150), a memory (not shown), and the like and can control the operation of the electronic device (100).

[0075] The one or more processors (140) may include one or more of a CPU, a GPU (Graphics Processing Unit), an APU (Accelerated Processing Unit), a MIC (Many Integrated Core), an NPU (Neural Processing Unit), a hardware accelerator, or a machine learning accelerator. The one or more processors (140) may control one or any combination of other components of the electronic device (100) and perform operations related to communication or data processing. The one or more processors (140) may execute one or more programs or instructions stored in a memory. For example, the one or more processors (140) may perform a method according to an embodiment of the present disclosure by executing one or more instructions stored in a memory.

[0076] When a method according to an embodiment of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-specific processor).

[0077] One or more processors (140) may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When one or more processors (140) are implemented as a multicore processor, each of the multiple cores included in the multicore processor may include an internal processor memory, such as a cache memory or an on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to an embodiment of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to an embodiment of the present disclosure.

[0078] When a method according to an embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in a multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0079] In embodiments of the present disclosure, one or more processors (140) may refer to a system on a chip (SoC) in which one or more processors and other electronic components are integrated, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, an NPU, a hardware accelerator, or a machine learning accelerator, but the embodiments of the present disclosure are not limited thereto. However, for convenience of explanation, the operation of the electronic device (100) is described below using the expression processor (140).

[0080] First, a method for obtaining a target phase and target gain for performing noise cancellation based on the user's ear membrane is described, and then a method for noise-cancelling external sound using the target phase and target gain is described.

[0081] The processor (140) can control an external speaker to output sound of a first frequency. For example, the electronic device (100) further includes a communication interface, and the processor (140) can control the communication interface to transmit a control signal to the external speaker to output sound of the first frequency. Here, the sound of the first frequency can be considered as an external sound.

[0082] The processor (140) can obtain a first sound signal corresponding to a sound of a first frequency through a first microphone (120), and obtain a second sound signal including the first sound signal through a second microphone (130). Here, the second sound signal includes not only the first sound signal but also a signal obtained through an operation described below, which will be described later after further explaining the operation of the processor (140).

[0083] The processor (140) can obtain a first noise canceling signal based on the first sound signal and the second sound signal. For example, the processor (140) can obtain an antiphase signal of a signal including the first sound signal and the second sound signal as the first noise canceling signal. For example, the processor (140) can obtain an antiphase signal of a signal including the first sound signal and the second sound signal as the first noise canceling signal in a feed forward active noise canceling (AFNC) manner.

[0084] The processor (140) can output a second noise-canceling signal in which at least one of the phase or gain of the first noise-canceling signal is changed and a third sound signal of a second frequency through the speaker (110). The sound output through the speaker (110) can be received through the second microphone (130). Accordingly, the processor (140) can receive a signal including the first sound signal, the second noise-canceling signal, the third sound signal, and a sound signal corresponding to the third frequency component as the second sound signal through the second microphone (130). Here, the third frequency component is a frequency component related to distortion product otoacoustic emissions (DPOAE), and DPOAE means that sounds of two frequencies generate multiple modulated sounds different from the two frequencies in the outer hair cells of the cochlea, and the third frequency component may be a component having the highest intensity among the modulated components. In the above example, a sound of a first frequency of f1 and a sound of a second frequency of f2 can be modulated in the outer hair cells of the cochlea, and the third frequency component with the strongest intensity among the modulated frequency components can be a frequency of 2f1-f2. Here, f2 can be 1.2f1.

[0085] The processor (140) can obtain a second sound signal by changing at least one of the phase or the gain, and identify at least one of the target phase or the target gain based on a third frequency component related to the modulation over-acoustic emissions (DPOAE) of the first frequency and the second frequency included in the second sound signal. That is, when at least one of the phase or the gain is changed, the second noise-canceling signal is changed, and when the second noise-canceling signal is changed, the sound output through the speaker (110) is changed, and when the sound output through the speaker (110) is changed, the second sound signal obtained through the second microphone (130) is also changed, and accordingly, the third frequency component included in the second sound signal can also be changed. The processor (140) can repeatedly change at least one of the phase or the gain, and obtain the changed third frequency component to identify at least one of the target phase or the target gain.

[0086] For example, the processor (140) can gradually change the phase to identify a phase interval where no otoacoustic reflection occurs, and identify a target phase based on the phase interval. For example, the processor (140) can identify the center of the phase interval as the target phase. The absence of a otoacoustic reflection means that maximum attenuation has occurred in the ear membrane, which means that external sound has been attenuated to the maximum in the ear membrane.

[0087] The processor (140) can identify the minimum amplitude at which a modulation otoacoustic reflection occurs by changing the amplitude of the sound of the first frequency when the gain is 0, and can identify a target gain based on the minimum amplitude. For example, the processor (140) can identify half the minimum amplitude as the amplitude of the sound of the first frequency, and can identify the minimum gain at which a modulation otoacoustic reflection occurs by changing the gain when the amplitude of the sound of the first frequency is half the minimum amplitude, and can identify the minimum gain as the target gain.

[0088] As described above, the processor (140) can identify the minimum phase and minimum gain based on the first frequency. However, the identified minimum phase and minimum gain are for the first frequency, and noise canceling performance may be degraded at frequencies other than the first frequency.

[0089] Accordingly, the processor (140) can change the first frequency and the second frequency and repeat the target phase identification operation and the target gain identification operation. The processor (140) can map the plurality of target phases and the plurality of target gains obtained according to the repeated operation to the corresponding first frequency and second frequency and store them in the memory.

[0090] The above describes a method for acquiring multiple target phases and multiple target gains for noise cancellation based on the user's ear membrane. Below, a method for noise-cancelling external sounds after storing multiple target phases and multiple target gains in memory is described.

[0091] The processor (140) can obtain an external sound signal corresponding to an external sound through a first microphone (120), obtain a fourth sound signal including the external sound signal through a second microphone (130), obtain a third noise canceling signal by actively noise canceling the external sound signal and the fourth sound signal, and output a fourth noise canceling signal in which the phase and gain of the third noise canceling signal are changed based on a target phase and a target gain corresponding to the frequency of the external sound among the information stored in the memory through the speaker (110). This operation can enhance the canceling effect of the external sound.

[0092] FIG. 3 is a block diagram showing a detailed configuration of an electronic device (100) according to one embodiment of the present disclosure.

[0093] According to FIG. 3, the electronic device (100) may include a speaker (110), a first microphone (120), a second microphone (130), and a processor (140). In addition, according to FIG. 3, the electronic device (100) may further include a communication interface (150), a user interface (160), and a memory (170). Among the components illustrated in FIG. 3, a detailed description of the overlapping portions with the components illustrated in FIG. 2 will be omitted.

[0094] The communication interface (150) is a configuration that performs communication with various types of external devices according to various types of communication methods. For example, the electronic device (100) can perform communication with an external speaker through the communication interface (150).

[0095] The communication interface (150) may include a Wi-Fi module, a Bluetooth module, an infrared communication module, a wireless communication module, etc. Here, each communication module may be implemented in the form of at least one hardware chip.

[0096] Wi-Fi and Bluetooth modules communicate via Wi-Fi and Bluetooth, respectively. When using a Wi-Fi or Bluetooth module, connection information, such as the SSID and session key, is first transmitted and received. This information is then used to establish a communication connection before various other information can be transmitted and received. Infrared communication modules use infrared data association (IrDA) technology, which wirelessly transmits data over short distances using infrared light, which lies between visible light and millimeter waves.

[0097] In addition to the above-described communication method, the wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards such as zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), 5G (5th Generation), etc.

[0098] Alternatively, the communication interface (150) may include a wired communication interface such as HDMI, DP, Thunderbolt, USB, RGB, D-SUB, DVI, etc.

[0099] In addition, the communication interface (150) may include at least one of a LAN (Local Area Network) module, an Ethernet module, or a wired communication module that performs communication using a pair cable, a coaxial cable, or an optical fiber cable.

[0100] The user interface (160) may be implemented with buttons, a touch pad, a mouse, a keyboard, etc., or may be implemented with a touch screen capable of performing both display and operation input functions. Here, the buttons may be various types of buttons, such as mechanical buttons, touch pads, wheels, etc., formed on any area of ​​the front, side, or back of the main body of the electronic device (100).

[0101] Memory (170) may refer to hardware that stores information such as data in an electrical or magnetic form so that the processor (140) or the like can access it. To this end, the memory (170) may be implemented as at least one piece of hardware from among non-volatile memory, volatile memory, flash memory, hard disk drive (HDD), solid state drive (SSD), RAM, ROM, etc.

[0102] The memory (170) may store at least one instruction required for the operation of the electronic device (100) or the processor (140). Here, the instruction is a unit of code that instructs the operation of the electronic device (100) or the processor (140), and may be written in machine language, which is a language that a computer can understand. Alternatively, the memory (170) may store a plurality of instructions for performing a specific task of the electronic device (100) or the processor (140) as an instruction set.

[0103] The memory (170) may store data in bit or byte units that can represent characters, numbers, images, etc. For example, a module related to a target phase and a target gain may be stored in the memory (170).

[0104] The memory (170) is accessed by the processor (140), and reading / writing / modifying / deleting / updating instructions, instruction sets, or data can be performed by the processor (140).

[0105] As described above, the electronic device (100) can improve noise canceling performance by performing noise canceling with the user's ear membrane as a reference position.

[0106] Hereinafter, the operation of the electronic device (100) will be described with reference to FIGS. 4 to 12. For convenience of explanation, individual embodiments are described in FIGS. 4 to 12. However, the individual embodiments of FIGS. 4 to 12 may be implemented in any combination.

[0107] FIG. 4 is a diagram for explaining distortion product otoacoustic emissions (DPOAE) according to one embodiment of the present disclosure.

[0108] According to Fig. 4, modulated otoacoustic emissions mean that when two pure-tone stimuli with different frequencies are simultaneously applied, modulated sounds of multiple frequencies different from the two different frequencies are generated in the outer hair cells of the cochlea.

[0109] For example, as illustrated in Fig. 4, when a sound of a first frequency of f1 and a sound of a second frequency of f2 are applied, modulation may occur in the outer hair cells of the cochlea, generating modulated sounds containing various frequencies such as f2-f1, 2f1-f2, 2f2-f1, etc., and among the modulated frequency components, the frequency component of 2f1-f2 with the strongest intensity may be used for clinical examination. Here, f2 may be set to 1.2f1.

[0110] Therefore, the smaller the amplitude of the third frequency component of 2f1-f2 related to the modulated otoacoustic radiation, the more the sound of the first frequency of f1 is attenuated to the maximum before reaching the ear membrane from the entrance of the external auditory canal. As the amplitude of the third frequency component of 2f1-f2 is lowered through the method described in Fig. 2, the noise canceling effect of external sound can increase.

[0111] FIGS. 5 and 6 are drawings for explaining noise canceling according to one embodiment of the present disclosure.

[0112] According to Fig. 5, existing earphones use a method of hybrid active noise canceling (520) to output sound received from an external microphone (510-1) and sound received from an in-ear microphone (510-2) through a speaker (530). In this case, noise canceling is performed based on the earphone, so noise may not be completely removed from the user's ear membrane.

[0113] To perform noise cancellation based on the user's ear membrane, the processor (140) may change at least one of the phase or gain of the noise cancellation signal to perform noise cancellation based on the user's ear membrane. However, since the shape of the external auditory canal may vary from user to user, the phase or gain cannot be specified.

[0114] According to FIG. 6, the processor (140) controls an external speaker to output a sound of a first frequency of f1 through an EQ (equalization) module (620), obtains a first sound signal corresponding to the sound of the first frequency through a first microphone (120), and obtains a second sound signal including the first sound signal through a second microphone (130).

[0115] The processor (140) can obtain a first noise canceling signal from a first sound signal and a second sound signal through a feed-forward active noise canceling method (610). The first noise canceling signal can be a signal having an amplitude of g1, a first frequency of f1, and a phase of φ1.

[0116] The processor (140) can obtain a second noise canceling signal by changing at least one of the phase or gain of the first noise canceling signal through the phase shift / gain (630), and output the second noise canceling signal and a third sound signal of a second frequency through the speaker (110). The second noise canceling signal can be a signal having an amplitude of g1g2, a first frequency of f1, and a phase of φ1+φ2, and the sound output through the speaker (110) can include a signal having an amplitude of g, a first frequency of f1, a phase of φ, and a signal having a second frequency of f2.

[0117] In this case, a first sound having a first frequency of f1 and a sound output through a speaker (110) are applied to the user's ear membrane, and a modulated sound including various frequencies can be generated from this according to modulated acoustic radiation.

[0118] The second microphone (130) can receive not only the first sound having the first frequency of f1 and the sound output through the speaker (110), but also the modulated acoustically radiated sound, and the processor (140) can identify the magnitude of the frequency component of 2f1-f2 having the strongest intensity among the modulated frequency components.

[0119] The processor (140) can identify the magnitude of the frequency component of 2f1-f2 by changing at least one of the phase or the gain, and can identify the target phase and target gain having the highest noise canceling performance based on the magnitude of the frequency component of 2f1-f2.

[0120] In addition, the processor (140) can repeatedly perform the target phase identification operation and the target gain identification operation as described above while changing the first frequency and the second frequency.

[0121] FIG. 7 is a diagram illustrating a method for identifying a target phase according to an embodiment of the present disclosure.

[0122] According to FIG. 7, the processor (140) can gradually change the phase to identify a phase section in which no modulation acoustic reflection occurs, and identify a target phase based on the phase section.

[0123] For example, as illustrated in FIG. 7, the processor (140) can gradually change the phase from -3.5 to 3.5 to identify a phase range of -1.6 to 2 in which no modulation acoustic reflection occurs, and identify 0.2, which is the center of the phase range, as the target phase.

[0124] FIG. 8 is a diagram illustrating a method for identifying a target gain according to an embodiment of the present disclosure.

[0125] According to FIG. 8, the processor (140) can identify the minimum amplitude at which a modulation-induced acoustic reflection occurs by changing the amplitude of the sound of the first frequency when the gain is 0, and identify the target gain based on the minimum amplitude.

[0126] For example, as illustrated in FIG. 8, the processor (140) can identify the minimum amplitude gext at which a modulation otoacoustic reflection occurs by changing the amplitude of the sound of the first frequency when the gain is 0, identify gext / 2, which is half the minimum amplitude, as the amplitude of the sound of the first frequency, and identify the minimum gain 1.3 at which a modulation otoacoustic reflection occurs by changing the gain when the amplitude of the sound of the first frequency is gext / 2, and identify the minimum gain 1.3 as the target gain.

[0127] However, this is not limited to this, and the target gain can be identified through various methods that take into account the occurrence of modulation-induced acoustic reflections.

[0128] FIGS. 9, 10, 11, and 12 are drawings for explaining the effect of noise canceling according to various embodiments of the present disclosure.

[0129] Figures 9 and 10 show the attenuation at the second microphone, and Figures 11 and 12 show the attenuation at the user's ear membrane.

[0130] Comparing FIG. 9 and FIG. 11, it can be seen that the attenuation characteristics in the ear membrane are improved when an equalization operation that changes at least one of the phase or gain of the present disclosure is added.

[0131] In addition, comparing FIGS. 10 and 12, it can be confirmed that the highest attenuation value is obtained when an equalization operation that changes at least one of the phase or gain of the present disclosure is added.

[0132] FIG. 13 is a flowchart for explaining a method for controlling an electronic device according to an embodiment of the present disclosure.

[0133] According to FIG. 13, an external speaker is controlled to output a sound of a first frequency (S1310). Then, a first sound signal corresponding to the sound of the first frequency is acquired through a first microphone of an electronic device (S1320). Then, a second sound signal including the first sound signal is acquired through a second microphone of the electronic device (S1330). Then, a first noise-canceling signal is acquired based on the first sound signal and the second sound signal (S1340). Then, a second noise-canceling signal in which at least one of the phase or gain of the first noise-canceling signal is changed and a third sound signal of the second frequency are output through the speaker of the electronic device (S1350). Then, a step of acquiring a second sound signal by changing at least one of the phase or the gain and identifying at least one of the target phase or the target gain based on a third frequency component related to the modulated over-the-air acoustic emissions (DPOAE) of the first frequency and the second frequency included in the second sound signal (S1360).

[0134] In addition, the identifying step (S1360) can identify a phase section in which modulation-induced acoustic reflection does not occur by gradually changing the phase, and identify the target phase based on the phase section.

[0135] And, the identifying step (S1360) can identify the center of the phase section as the target phase.

[0136] In addition, the identifying step (S1360) can identify the minimum amplitude at which a modulation-induced acoustic reflection occurs by changing the amplitude of the sound of the first frequency when the gain is 0, and identify the target gain based on the minimum amplitude.

[0137] And, the identifying step (S1360) identifies the amplitude of the sound of the first frequency as half of the minimum amplitude, and when the amplitude of the sound of the first frequency is half of the minimum amplitude, the gain is changed to identify the minimum gain at which the modulation-induced acoustic reflection occurs, and the minimum gain can be identified as the target gain.

[0138] In addition, the method may further include a step of changing the first frequency and the second frequency, repeating the target phase identification operation and the target gain identification operation, and a step of mapping a plurality of target phases and a plurality of target gains obtained according to the repeating operation to the corresponding first frequency and the second frequency and storing the same in the memory of the electronic device.

[0139] And, the method may further include a step of obtaining an external sound signal corresponding to an external sound through a first microphone, obtaining a fourth sound signal including the external sound signal through a second microphone, a step of obtaining a third noise canceling signal by actively noise canceling the external sound signal and the fourth sound signal, and a step of outputting a fourth noise canceling signal in which the phase and gain of the third noise canceling signal are changed based on a target phase and a target gain corresponding to the frequency of the external sound among information stored in the memory through a speaker.

[0140] Additionally, the controlling step (S1310) can transmit a control signal to an external speaker to control the output of sound of the first frequency.

[0141] And, the step of obtaining the first sound signal and the second sound signal (S1320) can receive the second sound signal including the first sound signal, the second noise canceling signal, the third sound signal, and the sound signal corresponding to the third frequency component through the second microphone.

[0142] In addition, the step of obtaining the first noise canceling signal (S1330) can obtain an inverse phase signal of a signal including the first sound signal and the second sound signal as the first noise canceling signal.

[0143] According to various embodiments of the present disclosure as described above, an electronic device can improve noise canceling performance by performing noise canceling with a user's ear membrane as a reference position.

[0144] Meanwhile, according to a temporary example of the present disclosure, the various embodiments described above can be implemented as software including instructions stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device is a device that can call instructions stored from the storage medium and operate according to the called instructions, and may include an electronic device (e.g., electronic device (A)) according to the disclosed embodiments. When an instruction is executed by a processor, the processor can perform a function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' means that the storage medium does not contain a signal and is tangible, but does not distinguish between data being stored semi-permanently or temporarily in the storage medium.

[0145] Furthermore, according to one embodiment of the present disclosure, the method according to the various embodiments described above may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0146] Furthermore, according to one embodiment of the present disclosure, the various embodiments described above may be implemented in a computer-readable recording medium or a similar device using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented by the processor itself. In a software implementation, embodiments such as the procedures and functions described herein may be implemented as separate software. Each software may perform one or more functions and operations described herein.

[0147] Meanwhile, computer instructions for performing processing operations of a device according to the various embodiments described above may be stored in a non-transitory computer-readable medium. The computer instructions stored in such a non-transitory computer-readable medium, when executed by a processor of a specific device, cause the specific device to perform processing operations in the device according to the various embodiments described above. A non-transitory computer-readable medium refers to a medium that permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specific examples of non-transitory computer-readable media may include a CD, DVD, hard disk, Blu-ray disk, USB, memory card, or ROM.

[0148] In addition, each of the components (e.g., modules or programs) according to the various embodiments described above may be composed of a single or multiple entities, and some of the corresponding sub-components described above may be omitted, or other sub-components may be further included in various embodiments. Alternatively or additionally, some components (e.g., modules or programs) may be integrated into a single entity, which may perform the same or similar functions as those performed by each of the corresponding components prior to integration. Operations performed by modules, programs or other components according to various embodiments may be executed sequentially, in parallel, iteratively or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0149] It will be appreciated that the various embodiments of the present disclosure, as defined by the claims and description herein, may be implemented in the form of hardware, software, or a combination of hardware and software.

[0150] Such software may be stored on a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium stores one or more computer programs (software modules), and the one or more computer programs include computer-executable instructions that, when executed by one or more processors of the electronic device, cause the electronic device to perform the methods of the present disclosure.

[0151] Such software may be stored in a volatile or non-volatile storage form, such as, for example, a storage device such as read-only memory (ROM), whether erasable or rewritable, or in a memory form such as, for example, random access memory (RAM), a memory chip, device or integrated circuit, or on, for example, a compact disc (CD), a digital versatile disc (DVD), a magnetic disk or magnetic tape or similar optically or magnetically readable medium. It will be appreciated that the storage device and the storage medium are various embodiments of a computer program or non-transitory machine-readable storage suitable for storing a computer program comprising instructions for implementing various embodiments of the present disclosure when executed. Accordingly, various embodiments provide a program comprising code for implementing a device or method as claimed in one of the claims of the present disclosure, and a non-transitory machine-readable storage storing such a program.

[0152] While the present disclosure has been shown and described with reference to various embodiments, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.

Claims

1. In electronic devices, speaker; 1st microphone; 2nd microphone; A memory storing one or more computer programs; and comprising one or more processors communicatively connected to the speaker, the first microphone, the second microphone and the memory; The one or more computer programs contain computer-executable instructions and, when individually or collectively executed by the one or more processors, Controls the external speaker to output sound of the first frequency, Obtaining a first sound signal corresponding to the sound of the first frequency through the first microphone, and obtaining a second sound signal including the first sound signal through the second microphone, Obtaining a first noise canceling signal based on the first sound signal and the second sound signal, A second noise canceling signal in which at least one of the phase or gain of the first noise canceling signal is changed and a third sound signal of the second frequency are output through the speaker, Obtaining the second sound signal by changing at least one of the phase or the gain, An electronic device that identifies at least one of a target phase or a target gain based on a third frequency component related to a modulation propagation acoustic emission (DPOAE) of the first frequency and the second frequency included in the second sound signal.

2. In paragraph 1, The one or more computer programs further comprise computer-executable instructions, and when individually or collectively executed by the one or more processors, By gradually changing the above phase, a phase section in which the above modulation acoustic reflection does not occur is identified, An electronic device that identifies the target phase based on the phase interval.

3. In paragraph 2, The one or more computer programs further comprise computer-executable instructions, and when individually or collectively executed by the one or more processors, An electronic device that identifies the center of the above phase interval as the target phase.

4. In paragraph 2, The one or more computer programs further comprise computer-executable instructions, and when individually or collectively executed by the one or more processors, When the above gain is 0, the amplitude of the sound of the first frequency is changed to identify the minimum amplitude at which the modulation-induced acoustic reflection occurs, An electronic device that identifies the target gain based on the minimum amplitude.

5. In paragraph 4, The one or more computer programs further comprise computer-executable instructions, and when individually or collectively executed by the one or more processors, Identifying half the above minimum amplitude as the amplitude of the sound of the first frequency, When the amplitude of the sound of the first frequency is half the minimum amplitude, the gain is changed to identify the minimum gain at which the modulation induced acoustic reflection occurs, An electronic device that identifies the minimum gain as the target gain.

6. In paragraph 5, The one or more computer programs further comprise computer-executable instructions, and when individually or collectively executed by the one or more processors, By changing the first frequency and the second frequency, repeating the target phase identification operation and the target gain identification operation, An electronic device that stores in a memory a plurality of target phases and a plurality of target gains obtained by the above-mentioned repetitive operation by mapping them to corresponding first frequencies and second frequencies.

7. In paragraph 6, The one or more computer programs further comprise computer-executable instructions, and when individually or collectively executed by the one or more processors, Obtaining an external sound signal corresponding to an external sound through the first microphone, and obtaining a fourth sound signal including the external sound signal through the second microphone, Obtaining a third noise canceling signal by actively noise canceling the above external sound signal and the fourth sound signal, An electronic device that outputs a fourth noise-canceling signal, in which the phase and gain of the third noise-canceling signal are changed based on a target phase and target gain corresponding to the frequency of the external sound among the information stored in the memory, through the speaker.

8. In paragraph 1, further comprising a communication interface; The one or more computer programs further comprise computer-executable instructions, and when individually or collectively executed by the one or more processors, An electronic device that controls the communication interface to transmit a control signal to the external speaker to output sound of the first frequency.

9. In paragraph 1, The one or more computer programs further comprise computer-executable instructions, and when individually or collectively executed by the one or more processors, An electronic device that receives the second sound signal including the first sound signal, the second noise canceling signal, the third sound signal, and a sound signal corresponding to the third frequency component through the second microphone.

10. In paragraph 1, The one or more computer programs further comprise computer-executable instructions, and when individually or collectively executed by the one or more processors, An electronic device that obtains an inverse phase signal of a signal including the first sound signal and the second sound signal as the first noise canceling signal.

11. In a method for controlling an electronic device, A step of controlling an external speaker to output sound of the first frequency; A step of obtaining a first sound signal corresponding to the sound of the first frequency through a first microphone of the electronic device, and obtaining a second sound signal including the first sound signal through a second microphone of the electronic device; A step of obtaining a first noise canceling signal based on the first sound signal and the second sound signal; A step of outputting a second noise canceling signal in which at least one of the phase or gain of the first noise canceling signal is changed and a third sound signal of a second frequency through a speaker of the electronic device; A step of obtaining the second sound signal by changing at least one of the phase or the gain; and A control method comprising: a step of identifying at least one of a target phase or a target gain based on a third frequency component related to a differential polarity acoustic emission (DPOAE) of the first frequency and the second frequency included in the second sound signal.

12. In paragraph 11, The above identifying step is, By gradually changing the above phase, a phase section in which the above modulation acoustic reflection does not occur is identified, A control method for identifying the target phase based on the phase section.

13. In paragraph 12, The above identifying step is, A control method for identifying the center of the above phase section as the target phase.

14. In paragraph 12, The above identifying step is, When the above gain is 0, the amplitude of the sound of the first frequency is changed to identify the minimum amplitude at which the modulation-induced acoustic reflection occurs, A control method for identifying the target gain based on the minimum amplitude.

15. In paragraph 14, The above identifying step is, Identifying half the above minimum amplitude as the amplitude of the sound of the first frequency, When the amplitude of the sound of the first frequency is half the minimum amplitude, the gain is changed to identify the minimum gain at which the modulation induced acoustic reflection occurs, A control method, wherein the above minimum gain is identified as the target gain.

Images