Electronic device and control method thereof

Abstract

This electronic device for performing active noise cancellation includes: a speaker; an outer microphone disposed in a direction opposite to a direction in which the speaker emits sound; an inner microphone disposed in the direction in which the speaker emits sound; a memory; and at least one processor, wherein instructions, when executed collectively or individually by the at least one processor, cause the electronic device to: obtain a first acoustic cancellation signal on the basis of a noise signal obtained by the outer microphone; obtain a second acoustic cancellation signal on the basis of the noise signal obtained by the inner microphone; and obtain a parameter indicating a wearing state of the electronic device of a user on the basis of the first and second acoustic cancellation signals and a signal output by the speaker.

Description

Electronic device and method of controlling the same

[0001] The present disclosure relates to an electronic device and a control method thereof, and more particularly, to an electronic device capable of outputting sound that blocks external noise and a control method thereof.

[0002] Audio output devices are electronic devices that convert electrical signals into sound waves and output them. Among these audio output devices, earphones and headphones are portable. Because earphones and headphones can be used in both quiet and noisy environments, recent earphones and headphones feature noise-canceling features.

[0003] Active Noise Cancellation (ANC) is a function that blocks out surrounding noise by receiving surrounding sounds through a microphone and outputting sound sources with the phase of the waves reversed through the speakers.

[0004] However, since the audio output device has low resources due to its characteristics, there was difficulty in obtaining the filter coefficients when using an adaptive filter.

[0005] Therefore, when calculating the coefficients of an adaptive filter, a method to overcome low resources is required.

[0006] According to one embodiment of the present disclosure, an electronic device performing active noise cancellation comprises: a speaker; an outer microphone positioned opposite to a direction in which the speaker emits sound; an inner microphone positioned in a direction in which the speaker emits sound; a memory; and at least one processor; wherein the instructions, when collectively or individually executed by the at least one processor, cause the electronic device to obtain a first cancellation sound signal based on a noise signal obtained by the outer microphone, to obtain a second cancellation sound signal based on a noise signal obtained by the inner microphone, to obtain a parameter indicating a wearing state of a user of the electronic device based on the first and second cancellation sound signals and a signal output by the speaker, and to obtain coefficients of a feedforward filter, a feedback filter, and an echo cancellation filter based on the parameter.

[0007] The above first offset sound signal is a signal having an opposite wavelength to the noise signal acquired by the outer microphone by inputting the noise signal acquired by the outer microphone into a feedforward filter.

[0008] A second cancellation sound signal can be obtained based on an acoustic signal that has passed through an echo cancellation filter, the first cancellation sound signal, and a noise signal acquired by the inner microphone, and the second cancellation sound signal can be input to a feedback filter to obtain a third cancellation sound signal having an opposite wavelength to the noise signal acquired by the inner microphone.

[0009] The signal output by the above speaker is a signal that combines the first offset sound signal and the third offset sound signal.

[0010] According to the wearing state of the electronic device of the user, a first matrix including data on a signal acquired from the inner microphone and a second matrix including data on a signal output from the speaker acquired from the inner microphone may be obtained, and the first and second matrices approximated by using singular value decomposition (SVD) on the first and second matrices may be obtained.

[0011] The first and second weight matrices included in the approximated first and second matrices can be obtained, the signal transmitted to the inner microphone can be obtained using the first and second weight matrices, and the second weight matrix can be updated using an algorithm that minimizes the difference between the signal transmitted to the inner microphone and the signal actually transmitted to the inner microphone.

[0012] The first weight matrix can be updated based on the updated second weight matrix, and the parameters can be obtained based on the updated first and second weight matrices.

[0013] If the user's utterance is not detected, an algorithm that calculates whether the parameter is fixed can be used to identify whether the parameter is fixed.

[0014] When the user's speech is detected, an external noise signal can be re-acquired from the outer microphone, the inner microphone, and the speaker.

[0015] If the above parameters are identified as being fixed, the coefficients of the first and feedback filters and the coefficients of the filter for removing the echo of the external noise signal can be updated based on the fixed parameters.

[0016] A method for controlling an electronic device performing active noise cancellation according to one embodiment of the present disclosure may include: obtaining a first cancellation sound signal based on a noise signal obtained by the outer microphone; obtaining a second cancellation sound signal based on a noise signal obtained by the inner microphone; obtaining a parameter indicating a wearing state of the electronic device by a user based on the first and second cancellation sound signals and a signal output by the speaker; and obtaining coefficients of a feedforward filter, a feedback filter, and an echo cancellation filter based on the parameters.

[0017] The above first offset sound signal is a signal having an opposite wavelength to the noise signal acquired by the outer microphone by inputting the noise signal acquired by the outer microphone into a feedforward filter.

[0018] The method may include: obtaining a second cancellation sound signal based on an acoustic signal that has passed through an echo cancellation filter, the first cancellation sound signal, and a noise signal obtained by the inner microphone; and inputting the second cancellation sound signal to a feedback filter to obtain a third cancellation sound signal having an opposite wavelength to the noise signal obtained by the inner microphone.

[0019] The signal output by the above speaker is a signal that combines the first offset sound signal and the third offset sound signal.

[0020] The method comprises: obtaining a first matrix including data on a signal acquired from the inner microphone based on an external noise signal according to a wearing state of the electronic device of the user; and obtaining a second matrix including data on a signal acquired from the inner microphone based on a signal output from the speaker; and obtaining first and second matrices approximated by using singular value decomposition (SVD) on the first and second matrices.

[0021] The method may include: obtaining a first weight matrix and a second weight matrix included in the approximated first and second matrices; obtaining a signal transmitted to the inner microphone using the first and second weight matrices; and updating the second weight matrix using an algorithm that minimizes a difference between a signal transmitted to the inner microphone and a signal actually transmitted to the inner microphone.

[0022] The method may include: a step of updating the first weight matrix based on the updated second weight matrix; and a step of obtaining the parameters based on the updated first and second weight matrices.

[0023] If the user's speech is not detected, the step of identifying whether the parameter is fixed using an algorithm that calculates whether the parameter is fixed may be included.

[0024] The method may include a step of re-acquiring an external noise signal from the outer microphone, the inner microphone, and the speaker when the user's speech is detected.

[0025] FIG. 1 is a drawing for explaining an embodiment of an electronic device according to one embodiment of the present disclosure;

[0026] FIG. 2 is a block diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure.

[0027] Figures 3 to 9 are drawings for explaining the operation for obtaining the coefficients of the filter and the parameters indicating the state in which the user is wearing the electronic device for active noise canceling.

[0028] FIG. 10 is a flowchart illustrating a method for controlling an electronic device according to an embodiment of the present disclosure.

[0029] The present embodiments may be modified and have various embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope to specific embodiments, but should be understood to encompass various modifications, equivalents, and / or alternatives of the embodiments of the present disclosure. In connection with the description of the drawings, similar reference numerals may be used for similar components.

[0030] In describing the present disclosure, if it is determined that a specific description of a related known function or configuration may unnecessarily obscure the gist of the present disclosure, a detailed description thereof will be omitted.

[0031] Additionally, the following embodiments may be modified in various other forms, and the scope of the technical concepts of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided to further faithfully and completely convey the technical concepts of the present disclosure to those skilled in the art.

[0032] The terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the scope of the rights. Singular expressions include plural expressions unless the context clearly dictates otherwise.

[0033] In this disclosure, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a corresponding feature (e.g., a component such as a number, function, operation, or part), and do not exclude the presence of additional features.

[0034] In this disclosure, expressions such as “A or B,” “at least one of A and / or B,” or “one or more of A or / and B” can include all possible combinations of the listed items. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” can all refer to (1) including at least one A, (2) including at least one B, or (3) including both at least one A and at least one B.

[0035] The expressions “first,” “second,” “first,” or “second,” etc., used in this disclosure can describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0036] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that said component may be directly coupled to said other component, or may be coupled via another component (e.g., a third component).

[0037] On the other hand, when it is said that a component (e.g., a first component) is "directly connected" or "directly connected" to another component (e.g., a second component), it can be understood that no other component (e.g., a third component) exists between said component and said other component.

[0038] The expression "configured to" as used in the present disclosure may be used interchangeably with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" may not necessarily mean only "specifically designed to" in terms of hardware.

[0039] Instead, in some contexts, the phrase "a device configured to" may mean that the device, in conjunction with other devices or components, is "capable of" performing A, B, and C. For example, the phrase "a processor configured (or set) to perform A, B, and C" may refer to a dedicated processor (e.g., an embedded processor) for performing those operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform those operations by executing one or more software programs stored in a memory device.

[0040] In the embodiments, a 'module' or 'part' performs at least one function or operation, and may be implemented as hardware or software, or as a combination of hardware and software. Furthermore, a plurality of 'modules' or 'parts' may be integrated into at least one module and implemented as at least one processor, except for a 'module' or 'part' that needs to be implemented as a specific hardware.

[0041] Meanwhile, the various elements and areas in the drawings are schematically drawn. Therefore, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0042] Hereinafter, various embodiments of the present invention will be described in detail using the attached drawings.

[0043] FIG. 1 is a drawing for explaining an embodiment of an electronic device (100) according to one embodiment of the present disclosure.

[0044] Referring to FIG. 1, an electronic device (100) is an electronic device that converts an electrical signal into a sound wave and outputs it. This electronic device (100) may be an earphone or a headphone. For ease of explanation, the following description assumes that the electronic device (100) is an earphone that is inserted into a user's ear and operates. However, when implemented, the electronic device (100) may be a headphone, a device that only performs the function of removing noise without the function of reproducing the sound of content stored externally or internally, an audio device that is used without being worn by the user, or a combination of an earphone and a smartphone.

[0045] In addition, since the electronic device (100) is implemented as a wireless earphone, only one unit among the two units (left unit and right unit) that constitute the wireless earphone is illustrated, but this is only for convenience of explanation. When describing the configurations of the electronic device below, only one unit will be described, but this is also only for convenience of explanation, and it is obvious that the configurations of the electronic device described below can be included in each of the two units that constitute the wireless earphone.

[0046] At this time, the electronic device (100) may include a speaker (110). The speaker (110) may output an audio signal, and the audio signal output through the speaker (110) may be transmitted to the user's eardrum through the external auditory canal of the user's ear.

[0047] The electronic device may include an outer microphone (131) and an inner microphone (132). The outer microphone (131) may receive an audio signal corresponding to external noise. The inner microphone (132) may receive an audio signal corresponding to an audio signal output through the speaker (110).

[0048] For example, the inner microphone (132) can receive an audio signal corresponding to an audio signal output through the speaker (110). At this time, the inner microphone (132) can receive the audio signal using a path between the speaker (110) and the inner microphone (132).

[0049] Additionally, the inner microphone (132) can receive an audio signal corresponding to an audio signal received through the outer microphone (131). At this time, the audio signal received through the outer microphone (131) can refer to a path between the outer microphone (131) and the inner microphone (132).

[0050] At this time, when the user wears the electronic device (100) on his / her ears, the speaker of the electronic device (100) is directed toward the user's eardrums, and the sound output from the electronic device (100) is transmitted toward the user's eardrums. Accordingly, the user can hear the sound.

[0051] When the electronic device (100) is worn on the ear, the electronic device (100) shields the ear, thereby providing a certain level of noise-blocking function (or passive noise cancellation). However, loud or specific external noises can be transmitted to the user's eardrums, and such noises can interfere with the enjoyment of content.

[0052] Accordingly, the electronic device (100) can identify the type of noise transmitted to the user's eardrums and generate and output an offset sound signal corresponding to the identified type of noise. This offset sound signal has an opposite wavelength to the noise signal, so that the two signals are canceled out by the interference of the waves.

[0053] This type of operation is called active noise cancellation, and as explained above, proper noise cancellation is possible only when the type of external noise that will be transmitted to the user's eardrums is well estimated.

[0054] The fixed filter method uses a fixed filter value (or filters) and generates a canceling audio signal using only the fixed filter value, allowing for computational savings. However, the use of fixed filter values ​​has the disadvantage that it cannot adaptively respond to changes in the user's ear canal shape and / or wearing conditions, and thus noise-cancelling performance may vary depending on the ear canal shape and / or wearing conditions.

[0055] On the other hand, the adaptive filter method calculates filter values ​​through computation and uses these values ​​to generate a counterbalanced audio signal, maintaining noise-cancelling performance even when the device is worn. However, existing adaptive filter methods require significant resources for calculating the filters and their parameters. Consequently, adaptive filtering has been difficult to apply to earphones with limited resources.

[0056] Therefore, in order to utilize the adaptive filter method in earphones with low resources, the coefficients of the adaptive filter can be obtained after the user obtains parameters indicating the state of wearing the electronic device (100). This will be described in detail below.

[0057] Meanwhile, the 'wearing status information' may include information on the degree of looseness or tightness of the user's wearing status. The wearing status information may include information on the space where audio signals may leak between the electronic device (100) and the user's ear when the user wears the electronic device (100) on the user's ear.

[0058] FIG. 2 is a block diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure. Referring to FIG. 2, the electronic device (100) may include a speaker (110), a memory (120), a microphone (130), and a processor (140). Specifically, the microphone (130) may include an outer microphone (131) and an inner microphone (132). The configuration illustrated in FIG. 2 is merely an example, and other components (e.g., a battery, etc.) may be added, and some components may be omitted.

[0059] The speaker (110) has the function of outputting a signal processed by the processor (140).

[0060] The speaker (110) has the function of converting electric pulses into sound waves, and can be implemented as a dynamic type, i.e., a coin type, which is distinguished according to the principle and method of converting electric signals into sound waves. However, it is not limited thereto, and can also be implemented as an electrostatic type, a dielectric type, a magnetostrictive type, etc. within the scope to which the present disclosure is applied.

[0061] According to one embodiment, the speaker (110) may be implemented as a full-range speaker designed to reproduce almost the entire frequency range of the home sound. For example, the full-range speaker may reproduce a frequency range of 0 to 22 kHz.

[0062] According to another embodiment, the speaker (110) may be implemented in a form that includes at least one of a tweeter speaker for reproducing a high frequency sound range, a midrange speaker for reproducing a middle frequency sound range, or a woofer speaker for reproducing a low frequency sound range.

[0063] In particular, the speaker (110) can output signal-processed sound. Specifically, the speaker (110) can output an offset sound signal transmitted from the processor (140) or an offset sound signal mixed with a content sound signal.

[0064] The memory (120) can store an operating system (OS) for controlling the overall operation of components of the electronic device (100) and instructions or data related to components of the electronic device (100).

[0065] The memory (120) may be implemented in various forms, such as volatile memory (e.g., dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory (e.g., one time programmable ROM (OTPROM), programmable ROM (PROM), erasable and programmable ROM (EPROM), electrically erasable and programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD)).

[0066] In particular, the memory (120) may store data on a method for obtaining parameters indicating the wearing state of the user's electronic device (100), and data on a method for obtaining coefficients of a feedback filter, a feedforward filter, and an echo cancellation filter (or echo cancellation filter). In addition, the memory (120) may also store data on a first cancellation acoustic signal or a second cancellation acoustic signal.

[0067] A microphone (130) may refer to a module that acquires sound and converts it into an electrical signal, and may be a condenser microphone, a ribbon microphone, a moving coil microphone, a piezoelectric element microphone, a carbon microphone, or a MEMS (Micro Electro Mechanical System) microphone. In addition, it may be implemented in an omnidirectional, bidirectional, unidirectional, subcardioid, supercardioid, or hypercardioid manner.

[0068] In one embodiment, the microphone (130) can receive a voice command. In one example, the microphone (130) can acquire a sound corresponding to the voice command and convert it into an audio signal.

[0069] In particular, the microphone (130) may include an outer microphone (131) or an inner microphone (132).

[0070] The outer microphone (131) is positioned in the opposite direction to the inner microphone. Specifically, the outer microphone (110) is a microphone positioned in the opposite direction to the user when the user wears the electronic device (100), and can generate audio signals around the user.

[0071] The inner microphone (132) and outer microphone (131) are electronic devices that convert sound signals into electrical signals. These microphones can be implemented as dynamic microphones, condenser microphones, etc. While the following illustrates and describes the use of two microphones, three or more microphones may be implemented.

[0072] The inner microphone (132) is positioned in the direction in which the speaker emits sound. Specifically, the inner microphone (132) is a microphone positioned toward the eardrum of a user when the user wears the electronic device (100), and can collect sound from the corresponding area to generate an audio signal.

[0073] The memory (140) can store an operating system (OS) for controlling the overall operation of components of the electronic device (100) and instructions or data related to components of the electronic device (100).

[0074] The processor (140) may include one or more processors. Specifically, the one or more processors may include one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Accelerated Processing Unit (APU), a Many Integrated Core (MIC), a Digital Signal Processor (DSP), a Neural Processing Unit (NPU), a hardware accelerator, or a machine learning accelerator. The processor (130) may control one or any combination of other components of the electronic device, and may perform operations related to communication or data processing. The one or more processors may execute one or more programs or instructions stored in a memory. For example, the one or more processors may perform a method according to an embodiment of the present disclosure by executing one or more instructions stored in a memory.

[0075] One or more processors may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When one or more processors are implemented as a multicore processor, each of the multiple cores included in the multicore processor may include internal processor memory, such as cache memory or on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to an embodiment of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to an embodiment of the present disclosure.

[0076] Meanwhile, the processor (140) can control each component of the electronic device (100). For example, the processor (140) can control the outer microphone (131) and the inner microphone (132) to receive external sound signals, and can control the speaker (110) to output processed sound signals.

[0077] In particular, the processor (140) can obtain a first offset sound signal based on a first noise signal obtained by the outer microphone (131), obtain a second offset sound signal based on a second noise signal obtained by the inner microphone, and obtain an audio signal output by the speaker (110) using the first offset sound signal and the second offset sound signal. At this time, the processor (140) can obtain a parameter indicating a wearing state of the user's electronic device based on the audio signal output by the speaker (110). In addition, the processor (140) can process an audio signal output by the speaker based on the parameter.

[0078] Hereinafter, the present disclosure will be described in more detail with reference to the drawings.

[0079] FIG. 3 is a drawing for explaining the configuration of a transmission path and filter of external noise included in an electronic device (100) according to one embodiment of the present disclosure.

[0080] The term 'primary path' may refer to a path between the outer microphone (131) and the inner microphone (132), and may be replaced with terms such as 'primary path'. Specifically, an audio signal corresponding to external noise (X(z)) received through the outer microphone (131) may be transmitted to the inner microphone (132) through the primary path. That is, external noise may remain in the inner microphone through the primary path. A transfer function representing the primary path may exhibit a characteristic in which a signal is attenuated depending on the state in which the user wears the electronic device (100).

[0081] The term 'secondary path' may refer to a path between the speaker (110) and the inner microphone (132), and may be replaced with terms such as 'secondary path'. Specifically, an audio signal (Y(z)) output through the speaker (110) may be transmitted to the user's eardrum through the user's external auditory canal, and may be transmitted to the inner microphone (132). A transfer function representing the secondary path may represent the speaker response of the electronic device and the characteristics of the user's external auditory canal.

[0082] Active noise cancellation methods can be categorized into feedforward (FF) ANC and feedback (FB) ANC depending on where the noise is collected, and can be categorized into fixed filter and adaptive filter depending on how and what filter value is applied to the collected noise.

[0083] In this disclosure, a method for implementing an adaptive filter using a feedforward filter, a feedback filter, and an echo cancellation filter is described below.

[0084] Meanwhile, the feedforward ANC method may refer to a method in which an outer microphone (131) detects noise generated in an external environment and converts the detected noise into an opposite phase to generate a signal that can cancel out the noise.

[0085] Additionally, the feedback ANC method may be a filter that adjusts a noise cancellation signal based on noise detected by the inner microphone (132). Specifically, the inner microphone (132) may measure noise reaching the ear. At this time, the noise acquired by the inner microphone (132) may include both external noise and a noise cancellation signal output from a speaker. At this time, the feedback ANC method may adjust the coefficient of the feedback filter based on the noise acquired by the inner microphone (132).

[0086] At this time, the outer microphone (131) can input the noise signal acquired by the outer microphone (131) into a feedforward filter to acquire a signal having an opposite wavelength to the noise signal acquired by the outer microphone (131). For the convenience of explanation, this will be defined as a first offset sound signal and described later.

[0087] The feedforward filter operates over a wide bandwidth and can remove noise up to approximately 3 kHz, allowing it to operate reliably in the high-frequency range corresponding to the voice band. The high-frequency components contribute to the clarity of the voice signal. The processor (140) can obtain a first offset sound signal by passing the signal acquired by the external microphone through the aforementioned feedforward filter.

[0088] In addition, a second offset sound signal can be obtained based on the sound signal that has passed through the echo removal filter, the first offset sound signal, and the noise signal acquired by the outer microphone, and the second offset sound signal can be input to a feedback filter to obtain a third offset sound signal having an opposite wavelength to the noise signal acquired by the inner microphone.

[0089] A feedback filter can cancel out external noise signals measured by the inner microphone.

[0090] Meanwhile, using the first and second offset sound signals obtained according to the above-described method, an audio signal output by the speaker can be obtained. In addition, the processor (140) can transmit a signal that combines the first and third offset sound signals to the speaker (110).

[0091] The signal output by the speaker may be a signal that combines the first offset sound signal and the third offset sound signal.

[0092] At this time, the electronic device (100) can obtain parameters indicating a state in which a user is wearing the electronic device (100) by using the first to third offset sound signals and the signal output from the speaker, and then obtain coefficients of a feedback filter, a feedforward filter, and an echo cancellation filter (or Echo cancellation filter).

[0093] Below, the operation of obtaining parameters and obtaining coefficients of the filter will be described in detail.

[0094] First, the processor (140) can obtain audio signals output from the inner microphone (132), outer microphone (131), and speaker (110) (S410).

[0095] After this, the processor (140) can obtain parameters indicating the state in which the user is wearing the electronic device (100) by using the audio signals output from the inner microphone (132), the outer microphone (131), and the speaker (110) (S420).

[0096] At this time, the process of obtaining parameters by the processor (140) will be described later with reference to FIGS. 5 to 10.

[0097] First, the processor (140) can obtain a first matrix including data on an external noise signal acquired from an inner microphone (132) according to the user's wearing state of the electronic device (100) and a second matrix including data on an acoustic signal output from a speaker (110) acquired from the inner microphone (132). In addition, the processor (140) can obtain approximated first and second matrices by using singular value decomposition (SVD) on the first and second matrices.

[0098] Specifically, the first matrix may include data on impulse responses in the primary path for each user's fitting state. The second matrix may include data on impulse responses in the secondary path for each user's fitting state.

[0099] The first matrix and the second matrix can be expressed as in <Mathematical Formula 1> below.

[0100] P = [p1, p2, … p N ] (L * N)

[0101] S = [s1, s2, … s N ] (L * N)

[0102] <Mathematical Formula 1>

[0103] Here, N is the number of impulse responses measured in different fitting states, and L can be the number of taps of the FIR (Finite Impulse Response) filter. At this time, the FIR (Finite Impulse Response) filter is a type of filter that processes audio signals, and the number of taps can indicate how precisely the filter processes the signal.

[0104] The processor (140) can obtain an average center matrix corresponding to the data included in the first and second matrices obtained according to <Mathematical Formula 1>.

[0105] The processor (140) can obtain the average center matrix using <Mathematical Formula 2> below.

[0106] <Mathematical Formula 2>

[0107]

[0108] Specifically, the mean-centered matrix corresponding to the first and second matrices ( , ) may include data obtained by subtracting the mean value of each data included in the first matrix.

[0109] The processor (140) generates a mean-centered matrix corresponding to the first and second matrices. , ) can be used to perform singular value decomposition using the SVD (Singular Value Decomposition) method.

[0110] Specifically, the mean-centered matrix corresponding to the first and second matrices can be decomposed into the left singular vector, the singular value matrix, and the right singular vector of the matrix, respectively, using <Mathematical Formula 3> below.

[0111] <Mathematical Formula 3>

[0112]

[0113] At this time, the singular value matrix, which consists of only diagonal elements, can represent the importance of each singular vector. The larger the value in the diagonal elements, the greater the importance of the data.

[0114] FIG. 5 is a diagram illustrating data regarding the distribution of singular values ​​included in the first and second matrices according to one embodiment of the present disclosure. According to the graph illustrated in FIG. 5, the processor (140) can obtain the distribution of singular value energy of the first and second matrices.

[0115] The x-axis of the graph can represent the basis (index k) of the singular values. That is, the first singular value is the most important singular value, and the importance of the later singular values ​​decreases.

[0116] Additionally, the y-axis of the graph can represent the proportion of the total energy of each singular value (normalized energy) of the first and second matrices. Singular values ​​with high energy may contain key information about the first and second matrices. On the other hand, singular values ​​with low energy may not contribute significantly to explaining the information of the first and second matrices.

[0117] At this time, the processor (140) can identify that the first and second data are important because the sizes of the first and second bases (indexes) are large. Accordingly, the processor (140) can extract two data from among the data included in the mean-centered matrix corresponding to the first and second matrices. However, this is only one example, and the processor (140) can of course extract not only two but also three or four pieces of data.

[0118] The extracted matrix can be expressed according to <Mathematical Formula 4> below.

[0119] <Mathematical Formula 4>

[0120]

[0121] For convenience of explanation, the extracted matrix ( , ) are defined as the approximate first and second matrices, which will be described later.

[0122] The processor (140) approximates the first and second matrices ( , ) are obtained, and an acoustic signal transmitted to the inner microphone (132) is obtained using the first and second weight matrices, and the second weight matrix can be updated using an algorithm that minimizes the difference between the acoustic signal transmitted to the inner microphone (132) and the acoustic signal actually transmitted to the inner microphone (132).

[0123] Specifically, the processor (140) approximates the first and second matrices ( , ) using the SVD method to obtain the first weight matrix (W p ) and the second weight matrix (W s ) can be obtained, which can be expressed as follows.

[0124] <Mathematical Formula 5>

[0125]

[0126] Meanwhile, the processor (140) can obtain the approximated first and second matrices using <Mathematical Formula 6> below.

[0127] <Mathematical Formula 6>

[0128]

[0129] Specifically, since the primary and secondary paths are correlated with each other, a linear transformation method is used to obtain the second weight matrix (W s ) is obtained, the first weight matrix (W p ) can be obtained. This will be described later with reference to Fig. 6.

[0130] FIG. 6 is a graph illustrating weight values ​​corresponding to each of the primary and secondary paths according to one embodiment of the present disclosure. Here, the horizontal axis represents the base (index), and the vertical axis represents the weight value.

[0131] In Fig. 6, for convenience of explanation, four bases (indexes) are used as the standard, but this is only an example, and it is of course possible to obtain weight values ​​for two bases or weight values ​​for three bases.

[0132] According to Fig. 6, the first and second weight matrices of the primary and secondary paths have a correlation, so the primary path can be estimated using the secondary path. In other words, it can be identified that the first weight matrix can be obtained using the second weight matrix.

[0133] FIG. 7 is a diagram illustrating that a first weight matrix can be obtained using a second weight matrix through a correlation matrix (M) according to one embodiment of the present disclosure. That is, as illustrated in FIG. 7, it can be identified that the characteristics of the measured main path and the characteristics of the actual main path can be obtained.

[0134] The processor (140) uses the <Mathematical Formula 7> below to generate the second weight matrix (W s ) from the first weight matrix (W p ) can be obtained.

[0135] <Mathematical Formula 7>

[0136]

[0137] <Mathematical Formula 8>

[0138]

[0139] <Equation 9>

[0140]

[0141] The first and second weight values ​​may be functions that continuously change depending on the state in which the user wears the electronic device (100). Specifically, the first and second weight values ​​may refer to weight values ​​that change as a parameter indicating the state in which the user wears the electronic device (100) continuously changes.

[0142] As previously described, the characteristic that there is a correlation between the first weight matrix and the second weight matrix will be described below regarding a method for obtaining the first weight matrix and the second weight matrix.

[0143] After obtaining the first weight matrix and the second weight matrix, the processor (140) can obtain parameters indicating the wearing state of the user's electronic device (100). Specifically, the processor (140) can update the first weight matrix based on the updated second weight matrix, and obtain parameters based on the updated first and second weight matrices.

[0144] First, the first weight matrix and the second weight matrix can be obtained using <Mathematical Formula 10> below.

[0145] <Mathematical Formula 10>

[0146]

[0147] The processor (140) can obtain continuous first and second weight values ​​according to the fitting state of the user's electronic device (100).

[0148] Meanwhile, when the state in which the user wears the electronic device (100) changes, the primary and secondary paths change, so the processor (140) can estimate the primary and secondary paths in real time using parameter values.

[0149] e e (n) = p e (n)*x(n) +s e (n)*y(n)

[0150] (*: convolution operator)

[0151] At this time, e e (n) is the noise signal acquired by the inner microphone (132) in the time domain, and p e (n) is the data for the impulse response in the basic path, s e (n) can be data for an impulse response in an auxiliary path. In addition, x(n) can be a noise signal acquired by an outer microphone (131) in the time domain, and y(n) can be an audio signal output by a speaker (110).

[0152] The processor (140) is e e (n) can be obtained to update the first weight matrix and the second weight matrix in real time.

[0153] In addition, the processor (140) uses the <Mathematical Formula 11> below to generate a vector and e containing information about the output signal. e (n) can be obtained.

[0154] <Mathematical Formula 11>

[0155]

[0156] <Mathematical Formula 12>

[0157]

[0158] At this time, the processor (140) can obtain the second weight matrix in real time using the least mean squares (LMS) method.

[0159] <Mathematical formula 13>

[0160]

[0161] Here, r(n) means the difference value between the actual noise signal obtained from the inner microphone (132) and the calculated noise signal.

[0162] Also, the μ value is a value that can be updated per sample with the stepsize vector, and means the Hadamard product.

[0163] At this time, the processor (140) can obtain a parameter value (t) that estimates the state in which the user wears the electronic device (100) using <Mathematical Formula 14> below.

[0164] <Mathematical Formula 14>

[0165]

[0166] Returning to FIG. 4 again, the processor (140) can re-acquire audio signals output by the outer microphone (131), inner microphone (132), and speaker (110) when an internal audio signal is detected (S430-Y).

[0167] Additionally, the processor (140) can determine whether an internal audio signal is detected. That is, if an internal audio signal is not detected (S430-N), the processor (140) can determine whether the parameter values ​​converge (S440).

[0168] Specifically, if the user's utterance is not detected, the processor (140) can identify whether the parameter is fixed using an algorithm that calculates whether the parameter is fixed. In this case, the algorithm that calculates whether the parameter is fixed may be a change point detection algorithm.

[0169] Meanwhile, the processor (140) is described below on the premise that it uses CUSUM (Cumulative Sum Control Chart) among the change point detection algorithms, but this is only an example, and the processor may also use the EWMA (Exponentially Weighted Moving Average) algorithm, the Bayesian Change Point Detection algorithm, or the Kernel-based Methods algorithm to identify whether the parameters are fixed.

[0170] Specifically, the CUSUM algorithm is a method for detecting changes in the average, and can find points of change by using the accumulated difference from the previous point in time.

[0171] The parameters obtained by the processor (140) through the above-described method may be parameters that estimate the user's wearing state. At this time, the parameters are defined as variables having values ​​between 0 and 1, as described below. However, this is only an example, and it is obvious that the range of numbers may be numbers outside of 0 and 1.

[0172] Specifically, when the parameter is 0, it may mean that the user is wearing the electronic device (100) tightly, and when the parameter is 1, it may mean that the user is wearing the electronic device (100) loosely.

[0173] At this time, the processor (140) can obtain coefficient values ​​of the feedforward filter, feedback filter, and echo removal filter when the parameter values ​​converge (S440-Y) (S450).

[0174] However, if the processor (140) identifies that the parameter values ​​do not converge (S440-N), the noise signal acquired from the inner microphone and outer microphone and the signal output from the speaker can be acquired again (S410).

[0175] FIG. 8 is a diagram for explaining an embodiment in which parameter values ​​converge according to one embodiment of the present disclosure.

[0176] Specifically, the processor (140) can identify whether a parameter is fixed by using an algorithm that calculates whether the parameter is fixed when the user's speech is not detected.

[0177] At this time, the processor (140) can use a change point detection algorithm to identify whether the parameter is fixed.

[0178] For example, the processor (140) may use the CUSUM (Cumulative Sum Control Chart) algorithm as a change point detection algorithm. This can find change points through a moving average.

[0179] When the processor (140) identifies that the parameters are fixed, it can update the coefficients of the first and feedback filters and the coefficients of the filter for removing the echo of the external noise signal based on the fixed parameters.

[0180] FIG. 9 is a diagram for explaining that when configuring a plurality of filters based on parameters according to one embodiment of the present disclosure, noise signals can be more canceled out than with a fixed filter.

[0181] For example, the processor (140) can obtain the frequency response of the feedforward filter when a parameter indicating a state in which the user is wearing the electronic device (100) is set to 0.5 or 0.6.

[0182] The graph shown on the left can represent the frequency response of the feedforward filter when the parameter is set to 0.5 or 0.6.

[0183] Meanwhile, the graph shown on the right can show the change in the response of the feedforward filter through interpolation when the parameter is set between 0.5 and 0.6.

[0184] At this time, interpolation processing is a method for predicting a response between continuous parameter values. Accordingly, the processor (140) can be adjusted according to the state in which the user wears the electronic device (100).

[0185] FIG. 10 is a flowchart for explaining a control method of an electronic device according to one embodiment of the present disclosure.

[0186] First, the electronic device (100) can obtain a first cancellation sound signal based on a noise signal obtained by the outer microphone (S1010).

[0187] The electronic device (100) can obtain a second cancellation sound signal based on the noise signal obtained by the inner microphone (S1020).

[0188] The electronic device (100) can obtain parameters indicating the wearing status of the user's electronic device based on the first and second offset sound signals and the signal output by the speaker (S1030).

[0189] The electronic device (100) can output a signal that cancels out a noise signal after setting the coefficients of a feedback filter, a feedforward filter, and an echo cancellation filter (S1040).

[0190] In addition, the methods according to various embodiments of the present disclosure may be provided as included in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user (20) devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0191] The methods according to various embodiments of the present disclosure may be implemented as software including commands stored in a machine-readable storage medium that can be read by a machine (e.g., a computer). The device is a device that can call commands stored from the storage medium and operate according to the called commands, and may include a server device or an electronic device according to the disclosed embodiments.

[0192] Meanwhile, a device-readable storage medium may be provided in the form of a non-transitory readable recording medium. Here, the term "non-transitory readable recording medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is stored semi-permanently in the storage medium and cases where data is stored temporarily. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0193] When the above instruction is executed by the processor, the processor may perform the function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or interpreter.

[0194] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. In an electronic device that performs active noise cancellation, speaker; An outer microphone placed in a direction opposite to the direction in which the above speaker emits sound; An inner microphone positioned in the direction in which the above speaker emits sound; memory; and comprising at least one processor; The above instructions, when collectively or individually executed by the at least one processor, cause the electronic device to: Obtaining a first cancellation sound signal based on the noise signal obtained by the above outer microphone, A second canceling sound signal is acquired based on the noise signal acquired by the inner microphone, Obtaining a parameter indicating the wearing state of the user's electronic device based on the first and second offset sound signals and the signal output by the speaker, An electronic device for obtaining coefficients of a feedforward filter, a feedback filter, and an echo cancellation filter based on the above parameters.

2. In paragraph 1, The above instructions, when collectively or individually executed by the at least one processor, cause the electronic device to: An electronic device in which the first offset sound signal is a signal having an opposite wavelength to the noise signal acquired by the outer microphone by inputting the noise signal acquired by the outer microphone into a feedforward filter.

3. In paragraph 2, The above instructions, when collectively or individually executed by the at least one processor, cause the electronic device to: Acquire a second cancellation sound signal based on an acoustic signal that has passed through an echo cancellation filter, the first cancellation sound signal, and a noise signal acquired by the inner microphone, An electronic device that inputs the second offset sound signal into a feedback filter to obtain a third offset sound signal having an opposite wavelength to the noise signal obtained by the inner microphone.

4. In paragraph 3, The above instructions, when collectively or individually executed by the at least one processor, cause the electronic device to: An electronic device in which the signal output by the speaker is a signal that combines the first offset sound signal and the third offset sound signal.

5. In paragraph 2, The above instructions, when executed collectively or individually by the at least one processor, If the first data is image data, and if the third data and the first data are divided into a preset number of blocks, it is identified that the hash values ​​of some blocks do not match, An electronic device that identifies a block identified as not matching as being counterfeit.

6. In paragraph 1, The above instructions, when collectively or individually executed by the at least one processor, cause the electronic device to: According to the wearing state of the electronic device of the user, a first matrix including data on a signal obtained from the inner microphone as an external noise signal and a second matrix including data on a signal obtained from the inner microphone as a signal output from the speaker are obtained, An electronic device for obtaining approximated first and second matrices by using singular value decomposition (SVD) on the first and second matrices.

7. In paragraph 6, The above instructions, when collectively or individually executed by the at least one processor, cause the electronic device to: Obtain the first weight matrix and the second weight matrix included in the above approximated first and second matrices, Obtaining a signal transmitted to the inner microphone using the first and second weight matrices, An electronic device that updates the second weight matrix using an algorithm that minimizes the difference between the signal transmitted to the inner microphone and the signal actually transmitted to the inner microphone.

8. In paragraph 7, The above instructions, when collectively or individually executed by the at least one processor, cause the electronic device to: Update the first weight matrix based on the updated second weight matrix, An electronic device for obtaining the parameters based on the updated first and second weight matrices.

9. In paragraph 8, The above instructions, when collectively or individually executed by the at least one processor, cause the electronic device to: An electronic device that identifies whether the parameter is fixed by using an algorithm that calculates whether the parameter is fixed when the user's speech is not detected.

10. In paragraph 9, The above instructions, when collectively or individually executed by the at least one processor, cause the electronic device to: An electronic device that, when the user's speech is detected, re-acquires an external noise signal from the outer microphone, the inner microphone, and the speaker.

11. In paragraph 9, The above instructions, when collectively or individually executed by the at least one processor, cause the electronic device to: An electronic device that updates the coefficients of the first and feedback filters and the coefficients of the filter for removing the echo of the external noise signal based on the fixed parameters when the above parameters are identified as being fixed.

12. A method for controlling an electronic device that performs active noise cancellation, A step of obtaining a first cancellation sound signal based on a noise signal obtained by the outer microphone; A step of obtaining a second cancellation sound signal based on the noise signal obtained by the inner microphone; A step of obtaining a parameter indicating a wearing state of a user's electronic device based on the first and second offset sound signals and the signal output by the speaker; and A control method comprising: a step of obtaining coefficients of a feedforward filter, a feedback filter, and an echo cancellation filter based on the above parameters.

13. In paragraph 12, A control method in which the first offset sound signal is a signal having an opposite wavelength to the noise signal acquired by the outer microphone by inputting the noise signal acquired by the outer microphone into a feedforward filter.

14. In paragraph 13, A step of obtaining a second cancellation sound signal based on an acoustic signal that has passed through an echo cancellation filter, the first cancellation sound signal, and a noise signal obtained by the inner microphone; and A control method comprising: a step of inputting the second offset sound signal into a feedback filter to obtain a third offset sound signal having an opposite wavelength to the noise signal obtained by the inner microphone.

15. In paragraph 14, A control method in which the signal output by the above speaker is a signal that combines the first offset sound signal and the third offset sound signal.

Images