Audio acquisition method, audio acquisition device, electronic equipment and storage medium
By invoking a target filter in the audio acquisition device to reduce the impact of interference signals, the problem of strict environmental requirements during audio acquisition is solved, enabling efficient and accurate audio signal acquisition in non-dedicated environments.
Patent Information
- Application Number
- CN202411155675.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2026-03-03
AI Technical Summary
In existing technologies, the audio acquisition process has strict requirements for the testing environment, resulting in low acquisition efficiency and making it impossible to conduct effectively in environments such as non-anechoic chambers.
By controlling the audio acquisition device to obtain the audio signal emitted by the audio playback device, and calling the target filter to output the compensation signal, the influence of interference signals is reduced. The target filter is constructed to superimpose the compensation signal, thereby reducing the amplitude of the interference signal and improving the accuracy of the acquired signal.
It reduces the requirements for the testing environment, improves the efficiency and accuracy of audio acquisition, and enables high-quality audio signal acquisition in non-dedicated environments.
Smart Images
Figure CN121603840A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of information processing, and in particular to an audio acquisition method, an audio acquisition device, an electronic device, and a storage medium. Background Technology
[0002] Spatial audio refers to the simulation of three-dimensional spatial sound effects in audio, making the listener feel that the sound comes from different directions and distances, thereby providing users with a better auditory experience during audio playback.
[0003] To provide users with a better auditory experience, research is underway on personalized spatial audio technology. This research reveals that the primary factor influencing the user's perceived audio experience is their Head Related Transfer Functions (HRTF). Therefore, a key factor in achieving personalized spatial audio playback is confirming the user's HRTF. Summary of the Invention
[0004] To overcome the problems existing in related technologies, this disclosure provides an audio acquisition method, an audio acquisition device, an electronic device, and a storage medium.
[0005] According to a first aspect of the present disclosure, an audio acquisition method is provided, comprising: controlling an audio acquisition device to acquire an audio signal emitted by an audio playback device, wherein the audio signal is a superposition signal of an initial audio signal and an interference signal, the interference signal being generated based on the frequency response characteristics of the audio acquisition device and the audio playback device; calling a target filter to output a compensation signal based on the interference signal, and superimposing the compensation signal with the audio signal to obtain a target audio signal, wherein the amplitude of the interference signal in the target audio signal is less than or equal to the amplitude of the interference signal in the audio signal; and using the target audio signal as the acquired audio signal.
[0006] In one embodiment, the target filter is constructed as follows: A test audio signal is acquired, and a first test audio feature is determined, wherein the first test audio feature is the frequency domain variation characteristic of the test audio signal corresponding to each direction; a target frequency band is defined and a target frequency point is set in the first test audio feature, wherein the target frequency band and the target frequency point have a corresponding relationship; target parameters corresponding to the target frequency point are determined, wherein the target parameters are known parameters of the target frequency point in the first test audio feature; filter parameters of the target frequency band corresponding to the target frequency point are determined based on the target parameters, and the target filter is constructed based on the filter parameters.
[0007] In one embodiment, the target frequency point is the feature point corresponding to the center frequency of the target frequency band in the first test audio feature. Determining the filter parameters of the target frequency band corresponding to the target frequency point based on the target parameters includes: determining the amplitude corresponding to the target frequency point, the bandwidth of the target frequency band, and the center frequency of the target frequency band; constructing a target function by using the amplitude corresponding to the target frequency point as the dependent variable, and the bandwidth and center frequency of the target frequency band as independent variables; determining the numerator and denominator coefficients of the target function based on the target function, and using the numerator and denominator coefficients as the filter parameters; wherein the amplitude is determined based on the target frequency point and the first test audio feature, the bandwidth of the target frequency band is determined based on the center frequency of the target frequency band and the quality factor of the band, and the center frequency is determined based on the center frequency of the target frequency band and the sampling rate; wherein the target parameters include at least one of the following: the center frequency corresponding to the target frequency band; the gain corresponding to the target frequency point; and the sampling rate.
[0008] In one embodiment, determining the first test audio feature includes: determining a second test audio feature of the test audio signal in its propagation direction, wherein the second test audio feature is a feature of the test audio signal changing over time in its propagation direction; performing feature transformation on the second test audio feature; and determining the first test audio feature based on the feature-transformed audio feature.
[0009] In one embodiment, determining the second test audio feature of the test audio signal in its propagation direction includes: acquiring the test audio signal and determining initial test audio features corresponding to the test audio signal based on the test audio signal; determining a first feature point in the initial test audio features, wherein the first feature point is the point with the largest amplitude in the initial test audio features; calling a target window function to truncate the initial test audio features based on the first feature point to obtain a second feature point, wherein the second feature point is the intersection of the target window function and the initial test audio features; and taking the initial test audio feature located between the first feature point and the second feature point as the second test audio feature.
[0010] According to a second aspect of the present disclosure, an audio acquisition device is provided, comprising: an acquisition unit configured to control an audio acquisition device to acquire an audio signal emitted by an audio playback device, wherein the audio signal is a superimposed signal of an initial audio signal and an interference signal, the interference signal being generated based on the frequency response characteristics of the audio acquisition device and the audio playback device; a processing unit configured to invoke a target filter to output a compensation signal based on the interference signal, and superimpose the compensation signal with the audio signal to obtain a target audio signal, wherein the amplitude of the interference signal in the target audio signal is less than or equal to the amplitude of the interference signal in the audio signal; and an execution unit configured to use the target audio signal as the acquired audio signal.
[0011] In one embodiment, the target filter is constructed by the processing unit in the following manner: acquiring a test audio signal and determining a first test audio feature, wherein the first test audio feature is the frequency domain variation feature of the test audio signal corresponding to each direction; defining a target frequency band and setting a target frequency point in the first test audio feature, wherein the target frequency band and the target frequency point have a corresponding relationship; determining the target parameters corresponding to the target frequency point, wherein the target parameters are known parameters of the target frequency point in the first test audio feature; determining the filter parameters of the target frequency band corresponding to the target frequency point based on the target parameters, and constructing the target filter based on the filter parameters.
[0012] In one embodiment, the target frequency point is the feature point corresponding to the center frequency of the target frequency band in the first test audio feature. The processing unit determines the filter parameters of the target frequency band corresponding to the target frequency point based on the target parameters in the following manner: determining the amplitude corresponding to the target frequency point, the bandwidth of the target frequency band, and the center frequency of the target frequency band; constructing a target function by using the amplitude corresponding to the target frequency point as the dependent variable, and the bandwidth and center frequency of the target frequency band as independent variables; determining the numerator and denominator coefficients of the target function based on the target transfer function, and using the numerator and denominator coefficients as the filter parameters; wherein, the amplitude is determined based on the target frequency point and the first test audio feature, the bandwidth of the target frequency band is determined based on the center frequency of the target frequency band and the quality factor of the band, and the center frequency is determined based on the center frequency of the target frequency band and the sampling rate; wherein, the target parameters include at least one of the following: the center frequency corresponding to the target frequency band; the gain corresponding to the target frequency point; and the sampling rate.
[0013] In one embodiment, the processing unit determines the first test audio feature in the following manner: determining a second test audio feature of the test audio signal in its propagation direction, wherein the second test audio feature is a feature of the test audio signal changing over time in its propagation direction; performing feature transformation on the second test audio feature, and determining the first test audio feature based on the feature-transformed audio feature.
[0014] In one embodiment, the processing unit determines the second test audio feature based on the transformed audio features in the following manner: acquiring the test audio signal and determining the initial test audio feature corresponding to the test audio signal based on the test audio signal; determining a first feature point in the initial test audio feature, wherein the first feature point is the point with the largest amplitude in the initial test audio feature; calling a target window function to truncate the initial test audio feature based on the first feature point to obtain a second feature point, wherein the second feature point is the intersection of the target window function and the initial test audio feature; and taking the initial test audio feature located between the first feature point and the second feature point as the second test audio feature.
[0015] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to: execute the audio acquisition method described in the first aspect or any embodiment of the first aspect.
[0016] According to a fourth aspect of the present disclosure, a storage medium is provided, the storage medium storing instructions that, when executed by a processor, enable the processor to perform the audio acquisition method described in the first aspect or any embodiment of the first aspect.
[0017] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: By controlling an audio acquisition device to acquire an audio signal emitted by an audio playback device, calling a target filter to output a compensation signal based on interference signals in the audio signal, and superimposing the compensation signal with the audio signal to obtain a target audio signal. By superimposing the compensation signal with the audio signal, the influence of interference signals in the audio signal is reduced, thereby obtaining a target audio signal less affected by interference signals, which is then used as the acquired audio signal. By calling the target filter to perform corresponding noise reduction during the audio signal acquisition process, the influence of interference signals on the audio signal is reduced, thereby improving the accuracy of the acquired audio signal.
[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0019] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0020] Figure 1A This is a flowchart illustrating an audio method according to an exemplary embodiment.
[0021] Figure 1B This is a schematic diagram illustrating an application scenario of an audio method according to an exemplary embodiment.
[0022] Figure 2 This is a flowchart illustrating a method for constructing a target filter according to an exemplary embodiment.
[0023] Figure 3 This is a flowchart illustrating a method for determining a first test audio feature according to an exemplary embodiment.
[0024] Figure 4A This is a schematic diagram illustrating an initial test audio feature according to an exemplary embodiment.
[0025] Figure 4B This is a flowchart illustrating a method for determining a second test audio feature according to an exemplary embodiment.
[0026] Figure 4C This is a schematic diagram illustrating the determination of a second feature point according to an exemplary embodiment.
[0027] Figure 5A This is a schematic diagram illustrating target frequency band determination according to an exemplary embodiment.
[0028] Figure 5B This is a schematic diagram illustrating target frequency point determination according to an exemplary embodiment.
[0029] Figure 6 This is a flowchart illustrating a filter parameter determination method according to an exemplary embodiment.
[0030] Figure 7 This is a block diagram illustrating an audio acquisition device according to an exemplary embodiment.
[0031] Figure 8 This is a block diagram illustrating an apparatus for audio acquisition according to an exemplary embodiment.
[0032] Figure 9 This is a block diagram illustrating an apparatus for audio acquisition according to an exemplary embodiment. Detailed Implementation
[0033] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure.
[0034] Spatial audio refers to the simulation of three-dimensional spatial sound effects in audio, making the listener feel that the sound comes from different directions and distances, thereby providing users with a better auditory experience during audio playback.
[0035] To provide users with a better auditory experience, research is underway on personalized spatial audio technology. This research reveals that the primary factor influencing the user's perceived audio experience is their Head Related Transfer Functions (HRTF). Therefore, a key factor in achieving personalized spatial audio playback is confirming the user's HRTF.
[0036] In related technologies, to ensure the accuracy of HRTF verification and reduce the impact of interference signals (such as interference signals caused by the frequency response characteristics of test equipment) during HRTF transmission, high requirements are usually placed on the test environment, typically requiring the setup of a dedicated test environment (e.g., an anechoic chamber). In this test environment, the sound source needs to be controlled to play audio from different directions, and the audio signal is received through an in-ear microphone worn by the audio collector. Relevant features are then extracted from the received audio signal to complete the HRTF verification.
[0037] However, the stringent testing environment requirements of related technologies pose significant obstacles for users when conducting audio acquisition, resulting in low efficiency during the audio acquisition process.
[0038] Based on this, this disclosure proposes an audio acquisition method. The method involves controlling an audio acquisition device to obtain an audio signal emitted by an audio playback device, and calling a target filter to issue a compensation signal to the interference signal. By superimposing the audio signal and the compensation signal, the influence of interference signals in the acquired audio signal is reduced, and the superimposed audio signal is used as the acquired audio signal. Since the influence of interference signals can be reduced by calling a target filter, the requirements for the audio acquisition testing environment can be set lower compared to related technologies, thereby reducing the obstacles faced by users during audio acquisition and improving audio acquisition efficiency.
[0039] Figure 1A This is a flowchart illustrating an audio method according to an exemplary embodiment, such as... Figure 1AAs shown, the audio acquisition method used in the terminal includes the following steps.
[0040] In step S11, the audio acquisition device is controlled to acquire the audio signal emitted by the audio playback device, wherein the audio signal is a superposition signal of the initial audio signal and the interference signal.
[0041] In step S12, the target filter is invoked to output a compensation signal based on the interference signal, and the compensation signal is superimposed on the audio signal to obtain the target audio signal, wherein the amplitude of the interference signal in the target audio signal is less than or equal to the amplitude of the interference signal in the audio signal.
[0042] In step S13, the target audio signal is used as the acquired audio signal.
[0043] In this embodiment, an audio acquisition device acquires an audio signal emitted by an audio playback device. A target filter is invoked to output a compensation signal based on interference signals in the audio signal. This compensation signal is then superimposed on the audio signal to obtain a target audio signal. By superimposing the compensation signal on the audio signal, the influence of interference signals in the audio signal is reduced, resulting in a target audio signal less affected by interference signals. This target audio signal is then used as the acquired audio signal. By invoking the target filter for noise reduction during the audio signal acquisition process, the influence of interference signals on the audio signal is reduced, thereby improving the accuracy of the acquired audio signal. This, in turn, improves the accuracy of the HRTF (High-Resolution Tolerance) based on the acquired audio signal.
[0044] It should be noted that in the above embodiments, the audio acquisition method can reduce the influence of interference signals in the acquired audio signal by calling a target filter. Therefore, the audio acquisition method involved in the above embodiments can be implemented in environments with lower environmental requirements than related technologies. For example, related technologies require audio acquisition to be carried out in an anechoic chamber, while the audio acquisition method involved in the above embodiments does not require implementation in an anechoic chamber. Furthermore, related technologies require audio acquisition to be completed in an anechoic chamber using dedicated audio playback equipment (e.g., a denoised or specially calibrated audio playback device) and dedicated audio acquisition equipment (e.g., a denoised or specially calibrated audio acquisition device), while the audio playback device used in the above embodiments can be a non-dedicated audio acquisition device (i.e., an audio playback device without denoising or special calibration), and the audio acquisition device used can also be a non-dedicated audio acquisition device.
[0045] Continuing with the above embodiments, in some embodiments, the audio playback device mentioned in step S11 may be, for example, a terminal, and the audio acquisition device mentioned in step S11 may be, for example, headphones with an external microphone that can make an impulse response based on the audio signal received from the audio playback device. The acquired audio signal can be used to determine the HRTF corresponding to the audio signal in its propagation direction.
[0046] It should be noted that "terminal" includes, but is not limited to, at least one of the following: mobile phone, wearable device, Internet of Things device, car with communication function, smart car, tablet, computer with wireless transceiver function, virtual reality (VR) terminal device, augmented reality (AR) terminal device, wireless terminal device in industrial control, wireless terminal device in self-driving, wireless terminal device in remote medical surgery, wireless terminal device in smart grid, wireless terminal device in transportation safety, wireless terminal device in smart city, and wireless terminal device in smart home.
[0047] For ease of understanding, the following embodiments will be explained through... Figure 1B The application scenarios of the audio acquisition method described in the above embodiments are described. Figure 1B This is a schematic diagram illustrating an application scenario of an audio method according to an exemplary embodiment. For example... Figure 1B As shown in the diagram, a mobile phone is used as the audio playback device, and headphones with an external microphone are used as the audio acquisition device. The audio signal is output through the phone's speaker (the speaker icon in the diagram). This output audio signal is then transmitted and acquired by the headphones' external microphone, which obtains relevant feature information (such as time-domain or frequency-domain features). A target filter is invoked to output a corresponding compensation signal based on the relevant information in the acquired audio signal. This compensation signal is then superimposed on the acquired audio signal, and the superimposed signal is output through the headphones. The target filter can be a pre-built filter used to reduce the impact of interference signals on the audio signal within a specified frequency band by outputting a compensation signal.
[0048] And if in Figure 1BIn the application scenario corresponding to this embodiment, the target filter and its related components are removed. The audio signal is emitted through the audio playback device, and the propagation path is that the audio signal travels through the air to the audio acquisition device. It is understood that there is a difference between the audio signal received by the acquisition device and the audio signal emitted by the audio playback device (e.g., their impulse responses are different). The audio signal emitted by the audio playback device changes as it is received by the audio acquisition device, for example, its impulse response changes. The main factors affecting the change in its impulse response include: propagation losses (attenuation, reflection, etc.) and interference caused by the frequency response of the hardware devices (audio playback device and audio acquisition device) on the impulse response of the audio signal. Furthermore, the primary influencing factor is the interference caused by the frequency response of the hardware devices on the impulse response of the audio signal (i.e., the interference signal mentioned in the above embodiment).
[0049] Therefore, the compensation signal output by the target filter can improve the accuracy of the acquired audio signal. The compensation signal output by the target filter can satisfy characteristics such as reducing the influence of interference signals on the audio signal. For example, the compensation signal can enhance the audio signal in a specified frequency band, and / or the compensation signal can be superimposed on the interference signal in the specified frequency band, so that the amplitude of the superimposed interference signal is lower than the amplitude of the original interference signal (e.g., the amplitude of the superimposed interference signal is 0).
[0050] Therefore, the construction process of the target filter can be, for example, as follows: Figure 2 As shown, Figure 2 This is a flowchart illustrating a method for constructing a target filter according to an exemplary embodiment, such as... Figure 2 As shown, the method includes the following steps.
[0051] In step S21, a test audio signal is acquired, and a first test audio feature is determined. The first test audio feature is the frequency domain variation feature of the test audio signal corresponding to each direction.
[0052] In step S22, a target frequency band is defined and a target frequency point is set in the first test audio feature, wherein the target frequency band and the target frequency point have a corresponding relationship.
[0053] In step S23, the target parameters corresponding to the target frequency point are determined, wherein the target parameters are known parameters of the target frequency point in the first test audio feature.
[0054] In step S24, the filter parameters of the target frequency band corresponding to the target frequency point are determined based on the target parameters, and the target filter is constructed based on the filter parameters.
[0055] In this embodiment of the disclosure, a test audio signal is acquired and its frequency domain characteristics in the propagation direction are determined. Based on these frequency domain characteristics, the target frequency point and target frequency band for constructing the target filter are determined. The target filter is constructed using the target frequency point and target frequency band.
[0056] It is understandable that by acquiring the frequency domain characteristics of the test audio signal in each direction, and defining the frequency band range for constructing the target filter within the corresponding frequency domain characteristics, as well as the target frequency point for constructing the target filter, the process can be completed. Since the target frequency point is a feature point set on the first test audio feature, some parameters related to the target frequency point can be determined on the first test audio feature (i.e., target parameters). Combining the target frequency point and the corresponding target parameters, the filter parameters used in constructing the target filter can be determined, and thus the target filter can be constructed using these filter parameters.
[0057] In some embodiments, the first test audio feature may be a frequency domain representation of the HRTF of the test audio signal. For ease of understanding, the following embodiments will be illustrated using the frequency domain representation of the HRTF of the test audio as the first test audio feature.
[0058] For the first test audio, the characteristics of the first test audio can be obtained by acquiring the time-domain representation of the test audio signals from multiple directions and then performing time-frequency transformation. For example, it can be achieved through... Figure 3 This is achieved in the manner described. Figure 3 This is a flowchart illustrating a method for determining a first test audio feature according to an exemplary embodiment. Figure 3 As shown, the method includes the following steps.
[0059] In step S31, a second test audio feature of the test audio signal in its propagation direction is determined. The second test audio feature is the feature of the test audio signal changing with time in its propagation direction.
[0060] In step S32, the second test audio features are transformed, and the first test audio features are determined based on the transformed audio features.
[0061] In this embodiment, by acquiring the time-domain features of the test audio signal along its propagation direction and performing feature transformation on these features, a frequency-domain representation of the test audio signal, i.e., the first test audio feature, can be obtained. In this way, the frequency-domain representation of the test audio signal along its corresponding propagation direction can be obtained. Therefore, the target filter constructed based on this data can filter audio features along different propagation directions.
[0062] In some embodiments, for step S31, a second test audio feature corresponding to test audio from multiple directions can be determined.
[0063] For example, by the angle of the sound source The direction of audio propagation is represented by θ, where θ represents the pitch angle of the sound source. The horizontal angle represents the angle of the sound source. Therefore, the sound source angle... The corresponding second test audio feature can be recorded as
[0064] The second test audio features corresponding to the test audio from multiple directions can then be recorded as follows: Wherein, for multiple directions, M is the number of angles measured (if only one set of data is measured at the same sound source angle, it is 1), and N is the length of each impulse response.
[0065] However, it is known that during the acquisition of the test audio signal, the initial test audio characteristics may contain some interference features due to the propagation process of the test audio signal. For example... Figure 4A As shown, Figure 4A This is a schematic diagram illustrating an initial test audio feature according to an exemplary embodiment. Figure 4A In the diagram, the signal waveform curve represents the initial test audio characteristics. The delay and tail components can be interpreted as interference features within the initial test audio characteristics. Therefore, to prevent the construction of the target filter from introducing additional interference features, the initial test audio characteristics need to be removed before determining the first test audio characteristics to obtain the second test audio characteristics.
[0066] The method for determining the second test audio features can be as follows: Figure 4B As shown. Figure 4B This is a flowchart illustrating a method for determining a second test audio feature according to an exemplary embodiment, the method comprising the following steps.
[0067] In step S41, a test audio signal is acquired, and the initial test audio features corresponding to the test audio signal are determined based on the test audio signal.
[0068] In step S42, a first feature point is determined in the initial test audio features. The first feature point is the point with the largest amplitude in the initial test audio features.
[0069] In step S43, the target window function is called to truncate the initial test audio features based on the first feature point to obtain the second feature point, which is the intersection of the target window function and the initial test audio features.
[0070] In step S44, the initial test audio feature located between the first feature point and the second feature point is taken as the second test audio feature.
[0071] In this embodiment, interference features are removed from the measured audio features to obtain a second test audio feature with fewer interference features. Then, feature transformation is performed on the second test audio feature to obtain a first test audio feature with even fewer interference features, thereby reducing the impact of interference features during the construction of the target filter and improving the accuracy of the constructed target filter.
[0072] Continued Figure 4A In the described embodiment, the feature after removing interference features can be used as the minimum phase of the second test audio feature. Figure 4A The middle can be represented as
[0073] h minphase =h(n), n∈[n peak [N]
[0074] Among them, h minphase n represents the minimum phase of the second test audio feature. peak The frequency point corresponding to the peak amplitude in the initial test audio features is represented by N, and N represents the intersection point of the initial test audio features with the target window function.
[0075] In some embodiments, the selection of the target window function can be based on a window function with smaller side lobes and more concentrated energy in the main lobe on the spectrum, such as the Hamming Window function and / or the Hanning Window function.
[0076] To facilitate understanding, for determining the second feature point through the target window function and truncating the trailing artifacts in the initial test audio features, for example, it can be done as follows: Figure 4C As shown. Figure 4C This is a schematic diagram illustrating the determination of a second feature point according to an exemplary embodiment. For example... Figure 4C As shown, the signal waveform curve represents the initial test audio characteristics. The smooth curve can be understood as a representation of the target window function, and the intersection of the two is the second feature point. The truncated portion is the initial test audio characteristic curve between the first and second feature points.
[0077] Based on this, the first test audio features from multiple directions can be determined through the above embodiments.
[0078] Understandably, after obtaining the first test audio features suitable for constructing the target filter, spectral analysis needs to be performed based on these features. The target frequency band and corresponding filter parameters within that band are then determined based on the spectral analysis results. Therefore, before formally constructing the target filter, the selection of the target frequency band and the setting of the target frequency points are also necessary.
[0079] In some embodiments, the selection of the target frequency band and the setting of the target frequency point can be determined based on the user's requirements for constructing the target filter.
[0080] For example, regarding the selection of target frequency bands, the number of target frequency bands within the frequency band corresponding to the first test audio feature can be determined based on user needs. It's understandable that the more target frequency bands there are, the better the fit to the first test audio feature will be, and the more accurate the subsequently constructed filter will be. However, if the number of target frequency bands is set sufficiently, it will introduce a large amount of computation during filter construction, increasing the difficulty of filter construction and thus increasing obstacles for the user in audio acquisition.
[0081] For example, the number of target frequency bands is determined, for instance, by a target number, such that the target number of target frequency bands can cover the spectral cues of the corresponding first test audio feature. For ease of understanding, this will be explained through... Figure 5A The number of target frequency bands is explained. Figure 5A This is a schematic diagram illustrating target frequency band determination according to an exemplary embodiment. For example... Figure 5A As shown, the waveform curves in the figure can be understood as the curve representation of the first test audio feature (the horizontal axis is the frequency axis, and the vertical axis is the decibel axis). Figure 5A The frequency regions contained within the four boxes (box 1, box 2, box 3, and box 4) can be understood as the four corresponding target frequency bands. Furthermore... Figure 5A The four target frequency bands can cover the spectral clues corresponding to the first test audio features.
[0082] It should be noted that spectral clues can be understood as spectral trends corresponding to spectral features (e.g., Figure 5A (The curve trend corresponding to the first test audio feature curve).
[0083] For example, regarding the setting of target frequencies, based on the factors affecting binaural sound source localization, the HRTF differences of audio signals propagating in the same direction mainly lie in the frequency bands above the frequency threshold (which can be set by the user). Therefore, when setting frequency points, a larger number of frequency points can be set for the frequency bands with greater impact. Conversely, fewer frequency points can be set for frequency bands below the frequency threshold compared to those above. Alternatively, data from an existing HRTF library can be combined with frequency bands above the frequency threshold.
[0084] For example, the target frequency can be determined by selecting a feature point from the first test audio features as the target frequency. Figure 5B For example, Figure 5B This is a schematic diagram illustrating target frequency point determination according to an exemplary embodiment. Figure 5B In the diagram, the waveform curve can be understood as the curve representation of the first test audio feature (the horizontal axis is the frequency axis, and the vertical axis is the decibel axis), where point 1 is the selected target frequency. Furthermore, since point 1 is selected from the first test audio feature, the target parameters of point 1 can be determined through the first test audio feature. For example, point 1 in the diagram is the feature point of the center frequency of the target frequency band in the first test audio feature, and the center frequency is... Figure 5B This is represented by Δf, the gain, and the quality factor (Q) corresponding to the frequency band at point 1. It is understandable that... Figure 5B The data (e.g., 1000Hz, 3dB, etc.) are for illustrative purposes only and do not have any limiting meaning.
[0085] According to Figure 5A as well as Figure 5B It can be seen that there is a corresponding relationship between the selected target frequency band and the set target frequency point. For example... Figure 5A as well as Figure 5B As shown, the two can have a one-to-one correspondence (i.e., one target frequency band corresponds to one target frequency point). However, it is understandable that, in order to improve the fitting effect, multiple target frequency points can be set for one target frequency band. Other correspondence methods will not be elaborated here.
[0086] To determine the filter parameters corresponding to the target frequency band, for example, a filter system function can be constructed based on the relevant information of the first audio feature and the target parameters corresponding to the target frequency point, and the correlation coefficient of the system function can be solved to determine the filter parameters.
[0087] In some embodiments, taking the target frequency point as an example, where the center frequency of the corresponding target frequency band is the feature point corresponding to the first test audio feature, the filter parameters of the target frequency band corresponding to the target frequency point can be obtained by, for example... Figure 6 The illustrated embodiment is defined. Figure 6 This is a flowchart illustrating a filter parameter determination method according to an exemplary embodiment. Figure 6 As shown, the method includes the following steps.
[0088] In step S51, the amplitude corresponding to the target frequency point, the bandwidth of the target frequency band, and the center frequency of the target frequency band are determined.
[0089] In step S52, the amplitude corresponding to the target frequency point is used as the dependent variable, and the bandwidth of the target frequency band and the center frequency of the target frequency band are used as independent variables to construct the objective function.
[0090] In step S53, the numerator and denominator coefficients of the objective function are determined based on the objective function, and the numerator and denominator coefficients are used as filter parameters.
[0091] The amplitude is determined based on the target frequency and the first test audio characteristics. The bandwidth of the target frequency band is determined based on the center frequency of the target frequency band and the quality factor of the frequency band. The center frequency is determined based on the center frequency of the target frequency band and the sampling rate.
[0092] In some embodiments, the target parameters include at least one of the following: the center frequency corresponding to the target frequency band, the gain corresponding to the target frequency point, and the sampling rate.
[0093] In this embodiment of the disclosure, an objective function is constructed based on the amplitude and the target difference parameter corresponding to the target frequency point, thereby determining the numerator and denominator coefficients of the objective function. The filter parameters are then determined using the numerator and denominator coefficients, thus enabling the construction of the filter.
[0094] It is understandable that, based on the descriptions in 5A and 5B above, once the target frequency is determined, the target parameters corresponding to the target frequency can be known, such as the center frequency of the target frequency band, the gain of the target frequency, and the sampling rate.
[0095] In this case, the bandwidth of the target frequency band corresponding to the target frequency point can be determined by the following formula:
[0096] B = f c / Q
[0097] Where B is the bandwidth of the target frequency band (i.e., the frequency range in which the target filter operates), and f cThis is the center frequency of the frequency band (in this example, it can be understood as the frequency of the target frequency point).
[0098] The digital center frequency can be determined using the following formula:
[0099] w c =2πf c / f s
[0100] Among them, w c f is the digital center frequency. c f is the center frequency of the frequency band. s The sampling rate.
[0101] The amplitude corresponding to the target frequency can be obtained from the first audio test features. The objective function expression based on the amplitude can be as follows:
[0102]
[0103] Among them, H PN (z) represents the objective function expression in the complex plane, where the absolute value of its value corresponds to the amplitude at the target frequency. equal.
[0104] Based on the above expression, its numerator coefficient (taking a numerator coefficient of 3 as an example) can be expressed as follows:
[0105]
[0106] Among them, b 0，k b 1，k and b 2，k Let represent the numerator coefficients of the above expression, and G represent the gain at the target frequency.
[0107] Similarly, based on the expression, its denominator coefficient (taking a denominator coefficient of 3 as an example) can be represented as follows:
[0108]
[0109] Where, α 0，k a 1，k and a 2，k These represent the numerator coefficients in the above expressions.
[0110] Based on this, the numerator and denominator coefficients of the objective function can be determined, thereby enabling the determination of the filter parameters. Furthermore, it is possible to construct a target filter corresponding to the target frequency band based on the filter parameters.
[0111] Based on the same concept, this disclosure also provides an audio acquisition device 100.
[0112] It is understood that the audio acquisition device 100 provided in this disclosure includes hardware structures and / or software modules corresponding to each function in order to achieve the above-mentioned functions. In conjunction with the units and algorithm steps of the various examples disclosed in this disclosure, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of this disclosure.
[0113] Figure 7 This is a block diagram illustrating an audio acquisition device according to an exemplary embodiment. (Refer to...) Figure 7 The device 100 includes an acquisition unit 101, a processing unit 102, and an execution unit 103.
[0114] The acquisition unit 101 is used to control the audio acquisition device to acquire the audio signal emitted by the audio playback device. The audio signal is a superposition signal of the initial audio signal and the interference signal. The interference signal is generated based on the frequency response characteristics of the audio acquisition device and the audio playback device.
[0115] The processing unit 102 is used to call the target filter to output a compensation signal based on the interference signal, and to superimpose the compensation signal with the audio signal to obtain the target audio signal, wherein the amplitude of the interference signal in the target audio signal is less than or equal to the amplitude of the interference signal in the audio signal.
[0116] The execution unit 103 is used to use the target audio signal as the acquired audio signal.
[0117] In one embodiment, the target filter is constructed by the processing unit 102 in the following manner: A test audio signal is acquired, and a first test audio feature is determined, wherein the first test audio feature is the frequency domain variation characteristic of the test audio signal corresponding to each direction. A target frequency band is defined and a target frequency point is set in the first test audio feature, wherein the target frequency band and the target frequency point have a corresponding relationship. Target parameters corresponding to the target frequency point are determined, wherein the target parameters are known parameters of the target frequency point in the first test audio feature. Based on the target parameters, filter parameters of the target frequency band corresponding to the target frequency point are determined, and a target filter is constructed based on the filter parameters.
[0118] In one embodiment, the target frequency is the center frequency of the corresponding target frequency band, which is a feature point in the first test audio features. The processing unit 102 determines the filter parameters of the target frequency band based on the target parameters in the following manner: determining the amplitude corresponding to the target frequency, the bandwidth of the target frequency band, and the center frequency of the target frequency band. Using the amplitude corresponding to the target frequency as the dependent variable, and the bandwidth and center frequency of the target frequency band as independent variables, a target transfer function is constructed. The numerator and denominator coefficients of the target transfer function are determined based on the target transfer function, and these coefficients are used as filter parameters. The amplitude is determined based on the target frequency and the first test audio features; the bandwidth of the target frequency band is determined based on the center frequency and the quality factor of the band; and the center frequency is determined based on the center frequency and the sampling rate. The target parameters include at least one of the following: the center frequency corresponding to the target frequency band; the gain corresponding to the target frequency; and the sampling rate.
[0119] In one embodiment, the processing unit 102 determines the first test audio feature by: determining a second test audio feature of the test audio signal in its propagation direction, wherein the second test audio feature is a feature of the test audio signal changing over time in its propagation direction; performing feature transformation on the second test audio feature; and determining the first test audio feature based on the feature-transformed audio feature.
[0120] In one embodiment, the processing unit 102 determines the second test audio feature based on the transformed audio features in the following manner: acquiring a test audio signal and determining the initial test audio feature corresponding to the test audio signal based on the test audio signal; determining a first feature point in the initial test audio feature, the first feature point being the point with the largest amplitude in the initial test audio feature; calling a target window function to truncate the initial test audio feature based on the first feature point to obtain a second feature point, the second feature point being the intersection of the target window function and the initial test audio feature; and taking the initial test audio feature located between the first feature point and the second feature point as the second test audio feature.
[0121] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0122] Figure 8 This is a block diagram illustrating an audio acquisition device 200 according to an exemplary embodiment. For example, device 200 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.
[0123] Reference Figure 8The device 200 for audio acquisition may include one or more of the following components: processing component 202, memory 204, power component 206, multimedia component 208, audio component 210, input / output (I / O) interface 212, sensor component 214, and communication component 216.
[0124] Processing component 202 typically controls the overall operation of device 200, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 202 may include one or more modules to facilitate interaction between processing component 202 and other components. For example, processing component 202 may include a multimedia module to facilitate interaction between multimedia component 208 and processing component 202.
[0125] Memory 204 is configured to store various types of data to support the operation of device 200. Examples of such data include instructions for any application or method operating on device 200, contact data, phonebook data, messages, pictures, videos, etc. Memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
[0126] The power supply component 206 provides power to the various components of the device 200. The power supply component 206 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 200.
[0127] Multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 208 includes a front-facing camera and / or a rear-facing camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
[0128] Audio component 210 is configured to output and / or input audio signals. For example, audio component 210 includes a microphone (MIC) configured to receive external audio signals when device 200 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 204 or transmitted via communication component 216. In some embodiments, audio component 210 also includes a speaker for outputting audio signals.
[0129] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.
[0130] Sensor assembly 214 includes one or more sensors for providing status assessments of various aspects of device 200. For example, sensor assembly 214 may detect the on / off state of device 200, the relative positioning of components such as the display and keypad of device 200, changes in the position of device 200 or a component of device 200, the presence or absence of user contact with device 200, the orientation or acceleration / deceleration of device 200, and temperature changes of device 200. Sensor assembly 214 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 214 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 214 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.
[0131] Communication component 216 is configured to facilitate wired or wireless communication between device 200 and other devices. Device 200 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 216 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 216 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0132] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.
[0133] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, which can be executed by a processor 220 of the device 200 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.
[0134] Figure 9 This is a block diagram illustrating an audio acquisition device 300 according to an exemplary embodiment. For example, device 300 may be provided as a server. (Refer to...) Figure 9 The device 300 includes a processing component 322, which further includes one or more processors, and memory resources represented by memory 332 for storing instructions, such as application programs, that can be executed by the processing component 322. The application programs stored in memory 332 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 322 is configured to execute instructions to perform the aforementioned cell selection method.
[0135] Device 300 may also include a power supply component 326 configured to perform power management of device 300, a wired or wireless network interface 350 configured to connect device 300 to a network, and an input / output (I / O) interface 358. Device 300 may operate on an operating system stored in memory 332, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.
[0136] It is understood that in this disclosure, "multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.
[0137] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this disclosure, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.
[0138] It is further understood that the terms “center,” “longitudinal,” “lateral,” “front,” “rear,” “up,” “down,” “left,” “right,” “vertical,” “horizontal,” “top,” “bottom,” “inner,” and “outer,” etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this embodiment and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation.
[0139] It can be further understood that, unless otherwise specified, "connection" includes both direct connections where no other components exist between the two parties and indirect connections where other components exist between them.
[0140] It is further understood that although operations are described in a specific order in the accompanying drawings in the embodiments of this disclosure, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.
[0141] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.
Claims
1. An audio acquisition method, characterized in that, include: The audio acquisition device is controlled to acquire audio signals emitted by the audio playback device, wherein the audio signal is a superposition signal of an initial audio signal and an interference signal, and the interference signal is generated based on the frequency response characteristics of the audio acquisition device and the audio playback device; The target filter is invoked to output a compensation signal based on the interference signal, and the compensation signal is superimposed on the audio signal to obtain the target audio signal, wherein the amplitude of the interference signal in the target audio signal is less than or equal to the amplitude of the interference signal in the audio signal; The target audio signal is used as the acquired audio signal.
2. The method according to claim 1, characterized in that, The target filter is constructed in the following manner: Acquire a test audio signal and determine a first test audio feature, wherein the first test audio feature is the frequency domain variation characteristic of the test audio signal corresponding to each direction; In the first test audio feature, a target frequency band is defined and a target frequency point is set, wherein the target frequency band and the target frequency point have a corresponding relationship; Determine the target parameters corresponding to the target frequency point, wherein the target parameters are known parameters of the target frequency point in the first test audio feature; Based on the target parameters, the filter parameters of the target frequency band corresponding to the target frequency point are determined, and the target filter is constructed based on the filter parameters.
3. The method according to claim 2, characterized in that, The target frequency point is the feature point corresponding to the center frequency of the target frequency band in the first test audio feature. Determining the filter parameters of the target frequency band corresponding to the target frequency point based on the target parameters includes: Determine the amplitude corresponding to the target frequency point, the bandwidth of the target frequency band, and the center frequency of the target frequency band; The target function is constructed by taking the amplitude corresponding to the target frequency point as the dependent variable, and the bandwidth of the target frequency band and the center frequency of the target frequency band as independent variables. The numerator and denominator coefficients of the objective function are determined based on the objective function, and the numerator and denominator coefficients are used as the filter parameters. Wherein, the amplitude is determined based on the target frequency point and the first test audio feature, the bandwidth of the target frequency band is determined based on the center frequency of the target frequency band and the quality factor of the frequency band, and the center frequency is determined based on the center frequency of the target frequency band and the sampling rate; The target parameter includes at least one of the following: The center frequency corresponding to the target frequency band; The gain corresponding to the target frequency; Sampling rate.
4. The method according to claim 2, characterized in that, The determination of the first test audio features includes: Determine a second test audio feature of the test audio signal in its propagation direction, wherein the second test audio feature is the characteristic of the test audio signal changing over time in its propagation direction; The second test audio feature is transformed, and the first test audio feature is determined based on the transformed audio feature.
5. The method according to claim 4, characterized in that, The determination of the second test audio feature of the test audio signal in its propagation direction includes: Acquire the test audio signal, and determine the initial test audio features corresponding to the test audio signal based on the test audio signal; A first feature point is determined from the initial test audio features, and the first feature point is the point with the largest amplitude in the initial test audio features; The target window function is called to truncate the initial test audio feature based on the first feature point to obtain a second feature point, where the second feature point is the intersection of the target window function and the initial test audio feature. The initial test audio feature located between the first feature point and the second feature point is used as the second test audio feature.
6. An audio acquisition device, characterized in that, include: The acquisition unit is used to control the audio acquisition device to acquire the audio signal emitted by the audio playback device, wherein the audio signal is a superposition signal of the initial audio signal and the interference signal, and the interference signal is generated based on the frequency response characteristics of the audio acquisition device and the audio playback device. The processing unit is configured to call the target filter to output a compensation signal based on the interference signal, and to superimpose the compensation signal with the audio signal to obtain a target audio signal, wherein the amplitude of the interference signal in the target audio signal is less than or equal to the amplitude of the interference signal in the audio signal; An execution unit is used to treat the target audio signal as the acquired audio signal.
7. The apparatus according to claim 6, characterized in that, The target filter is constructed by the processing unit in the following manner: Acquire a test audio signal and determine a first test audio feature, wherein the first test audio feature is the frequency domain variation characteristic of the test audio signal corresponding to each direction; In the first test audio feature, a target frequency band is defined and a target frequency point is set, wherein the target frequency band and the target frequency point have a corresponding relationship; Determine the target parameters corresponding to the target frequency point, wherein the target parameters are known parameters of the target frequency point in the first test audio feature; Based on the target parameters, the filter parameters of the target frequency band corresponding to the target frequency point are determined, and the target filter is constructed based on the filter parameters.
8. The apparatus according to claim 7, characterized in that, The target frequency point is the feature point corresponding to the center frequency of the target frequency band in the first test audio feature. The processing unit determines the filter parameters of the target frequency band corresponding to the target frequency point based on the target parameters in the following manner: Determine the amplitude corresponding to the target frequency point, the bandwidth of the target frequency band, and the center frequency of the target frequency band; The target function is constructed by taking the amplitude corresponding to the target frequency point as the dependent variable, and the bandwidth of the target frequency band and the center frequency of the target frequency band as independent variables. The numerator and denominator coefficients of the objective function are determined based on the objective transfer function, and the numerator and denominator coefficients are used as the filter parameters. Wherein, the amplitude is determined based on the target frequency point and the first test audio feature, the bandwidth of the target frequency band is determined based on the center frequency of the target frequency band and the quality factor of the frequency band, and the center frequency is determined based on the center frequency of the target frequency band and the sampling rate; The target parameter includes at least one of the following: The center frequency corresponding to the target frequency band; The gain corresponding to the target frequency; Sampling rate.
9. The apparatus according to claim 7, characterized in that, The processing unit determines the first test audio feature in the following manner: Determine a second test audio feature of the test audio signal in its propagation direction, wherein the second test audio feature is the characteristic of the test audio signal changing over time in its propagation direction; The second test audio features are then transformed. The first test audio feature is determined based on the audio features after feature transformation.
10. The apparatus according to claim 9, characterized in that, The processing unit determines the second test audio feature based on the transformed audio features in the following manner: Acquire the test audio signal, and determine the initial test audio features corresponding to the test audio signal based on the test audio signal; A first feature point is determined from the initial test audio features, and the first feature point is the point with the largest amplitude in the initial test audio features; The target window function is called to truncate the initial test audio feature based on the first feature point to obtain a second feature point, where the second feature point is the intersection of the target window function and the initial test audio feature. The initial test audio feature located between the first feature point and the second feature point is used as the second test audio feature.
11. An electronic device, characterized in that, include: processor: Memory used to store processor-executable instructions; The processor is configured to execute the audio acquisition method according to any one of claims 1 to 5.
12. A storage medium, characterized in that, The storage medium stores instructions that, when executed by a processor, enable the processor to perform the audio acquisition method according to any one of claims 1 to 5.