Audio signal processing method, audio signal processing apparatus, storage medium, and program product

Abstract

The invention provides an audio signal processing method, audio signal processing equipment, a storage medium and a program product, which can be applied to the technical fields of computers and signal processing. The audio signal processing method comprises the following steps: acquiring a target transfer function for an object; and adjusting the first audio signal by using the target transfer function to obtain a playing signal so as to control the sound source to emit the playing signal, wherein the target transfer function is obtained by the following operation: according to multiple pieces of feedback azimuth information, an intermediate transfer function is determined from multiple first candidate transfer functions, and the feedback azimuth information is obtained by the object through sensing an intermediate audio signal obtained by adjusting the second audio signal by using the first candidate transfer functions; and determining a target transfer function based on the frequency distribution characteristics of the intermediate transfer function.

Description

Technical Field

[0001] This disclosure relates to the fields of computer technology and signal processing technology, and more specifically, to an audio signal processing method, an audio signal processing device, a storage medium, and a program product. Background Technology

[0002] The transfer function can refer to the head-related transfer function (HRTF). Because the HRTF contains information about sound source localization, it is of great significance for the study of binaural hearing. Therefore, obtaining personalized transfer functions to improve binaural hearing is an urgent problem to be solved. Summary of the Invention

[0003] In view of this, the present disclosure provides an audio signal processing method, an audio signal processing device, a storage medium, and a program product.

[0004] According to one aspect of this disclosure, an audio signal processing method is provided, comprising: acquiring a target transfer function for an object; and adjusting a first audio signal using the target transfer function to obtain a playback signal, thereby controlling an audio source to emit the playback signal; wherein the target transfer function is obtained by: determining an intermediate transfer function from a plurality of first candidate transfer functions based on a plurality of feedback location information, wherein the feedback location information is obtained by the object perceiving an intermediate audio signal obtained by adjusting a second audio signal using the first candidate transfer functions; and determining the target transfer function based on the frequency distribution characteristics of the intermediate transfer function.

[0005] According to another aspect of this disclosure, an audio signal processing apparatus is provided, comprising: a processor for acquiring a target transfer function for an object; and adjusting a first audio signal using the target transfer function to obtain a playback signal; and a sound source for emitting the playback signal; wherein the target transfer function is obtained by: determining an intermediate transfer function from a plurality of first candidate transfer functions based on a plurality of feedback location information, wherein the feedback location information is obtained by the object perceiving the intermediate audio signal obtained by adjusting a second audio signal using the first candidate transfer functions; and determining the target transfer function based on the frequency distribution characteristics of the intermediate transfer function.

[0006] According to another aspect of this disclosure, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more instructions, wherein, when executed by the one or more processors, the one or more processors cause the one or more processors to perform the method as described in this disclosure.

[0007] According to another aspect of this disclosure, a computer-readable storage medium is provided having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the methods described in this disclosure.

[0008] According to another aspect of this disclosure, a computer program product is provided, which includes computer-executable instructions that, when executed, are used to perform the methods described in this disclosure. Attached Figure Description

[0009] The above and other objects, features and advantages of this disclosure will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0010] Figure 1 The system architecture of an audio signal processing apparatus according to an embodiment of the present disclosure is illustrated schematically;

[0011] Figure 2 A flowchart illustrating an audio signal processing method according to an embodiment of the present disclosure is shown schematically.

[0012] Figure 3 This illustration schematically shows an example diagram of a process for obtaining a target transfer function according to an embodiment of the present disclosure;

[0013] Figure 4 This illustration schematically shows an example diagram of a clustering result acquisition process according to an embodiment of the present disclosure;

[0014] Figure 5 This schematically illustrates an example diagram of a feedback orientation information acquisition process according to an embodiment of the present disclosure; and

[0015] Figure 6 An example schematic diagram of a target transfer function according to an embodiment of the present disclosure is shown. Detailed Implementation

[0016] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0017] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0018] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0019] When using expressions such as "at least one of A, B, and C", they should generally be interpreted in accordance with the meaning that is commonly understood by a person skilled in the art (e.g., "a system having at least one of A, B, and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B, and C, etc.).

[0020] In the technical solution of this invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0021] Virtual Auditory Display (VAD) is an important application of transfer functions. In virtual auditory display, a single-channel audio signal is filtered in the frequency domain using a transfer function and then played back through headphones to produce an auditory effect in the corresponding spatial location.

[0022] In one example, due to the significant localization error of non-personalized transfer functions, virtual auditory playback needs to rely on personalized transfer functions to achieve a better auditory effect. However, the measurement cost of personalized transfer functions is high, and the testing process is cumbersome, making it difficult to effectively guarantee the testing efficiency and accuracy of personalized transfer functions.

[0023] Therefore, this disclosure provides an audio signal processing method, an audio signal processing device, a storage medium, and a program product, which can be applied to the fields of computer technology and signal processing technology. The audio signal processing method includes: obtaining a target transfer function for an object; and adjusting a first audio signal using the target transfer function to obtain a playback signal, so as to control a sound source to emit a playback signal; wherein the target transfer function is obtained by: determining an intermediate transfer function from a plurality of first candidate transfer functions based on multiple feedback location information, wherein the feedback location information is obtained by the object perceiving the intermediate audio signal obtained by adjusting a second audio signal using the first candidate transfer functions; and determining the target transfer function based on the frequency distribution characteristics of the intermediate transfer function.

[0024] Figure 1 The system architecture of an audio signal processing apparatus according to an embodiment of the present disclosure is illustrated schematically.

[0025] like Figure 1 As shown, the audio signal processing device 100 may include an audio source 110 and a processor 130.

[0026] The processor 130 is used to obtain the target transfer function 103 for the object and adjust the first audio signal 104 using the target transfer function 103 to obtain the playback signal 105.

[0027] Audio source 110 is used to emit playback signal 105.

[0028] In one example, pitch and azimuth angles can be used to describe the location of a sound source. See dashed box 101 for an example of a pitch angle. The pitch angle ranges from -90° to 90°. The pitch angle is defined as follows: 0 degrees is directly in front of object 120, 90 degrees is clockwise until it is vertically above object 120, and -90 degrees is counter-clockwise until it is vertically below object 120; similarly, 0 degrees is directly behind object 120, 90 degrees is counter-clockwise until it is vertically above object 120, and -90 degrees is clockwise until it is vertically below object 120. See dashed box 102 for an example of an azimuth angle. The azimuth angle ranges from 0° to 360°. The azimuth angle is defined as follows: 0 degrees is the north direction line of the object, and 360 degrees is clockwise until it returns to the north direction line.

[0029] The target transfer function 103 can be determined from a function database corresponding to the object 120. The function database can include multiple candidate transfer functions. In determining the target transfer function 103 for the object 120, for each candidate transfer function, the candidate transfer function and multiple original audio signals can be processed separately to obtain multiple intermediate audio signals. The intermediate audio signals are emitted at any position within the aforementioned circumference so that the object 120 can perceive them and return perception information through an interface. After receiving the perception information, the processor 130 can also process the perception information to obtain feedback orientation information. Based on the multiple feedback orientation information, the target transfer function is determined from the multiple candidate transfer functions.

[0030] It should be understood that Figure 1 The number of processors and audio sources shown is merely illustrative. Depending on implementation requirements, any number of processors and audio sources can be used.

[0031] It should be noted that the sequence numbers of the operations in the following methods are for descriptive purposes only and should not be considered as indicating the execution order of the operations. Unless explicitly stated otherwise, the method does not need to be executed in the exact order shown.

[0032] The system architecture of the audio signal processing equipment has been described above. The following will focus on... Figure 2 As an example, the audio signal processing process of this disclosure will be further explained.

[0033] Figure 2 A flowchart illustrating an audio signal processing method according to an embodiment of the present disclosure is shown schematically.

[0034] like Figure 2 As shown, the audio signal processing method 200 includes operations S210 to S220.

[0035] In operation S210, obtain the target transfer function for the object.

[0036] In operation S220, the first audio signal is adjusted using the target transfer function to obtain a playback signal, so as to control the sound source to emit a playback signal.

[0037] The head-related transfer function (HRF) describes the frequency response characteristics of sound waves emitted from a sound source, passing through the head, auricles, and torso of an object, and reaching both ears. The method for obtaining the HRF can be configured according to actual business needs and is not limited here. For example, the HRF can be directly obtained from a database of object-related functions based on the object's characteristics. Alternatively, a HRF can be adjusted to better suit the object by playing sounds from different directions in an anechoic chamber and recording the sound signals reaching both ears, using a transfer function directly obtained from the database as a base.

[0038] In one example, for each first candidate transfer function, the first candidate transfer function and the original audio signal can be processed to obtain an intermediate audio signal. The processing method can be configured according to actual business needs and is not limited here. For example, a Fourier transform can be performed on the original audio signal to convert it into a transformed audio signal in the frequency domain; each frequency component of the transformed audio signal is multiplied by its corresponding HRTF to simulate the frequency response change of the sound wave during propagation; then an inverse Fourier transform is performed on the processed frequency domain signal to obtain the intermediate audio signal. Alternatively, an inverse Fourier transform can be performed on the first candidate transfer function to obtain a processed time domain function; the original audio signal is then convolved with the processed time domain function to simulate the time delay and attenuation effect of the sound wave during propagation to obtain the intermediate audio signal.

[0039] For each intermediate audio signal, it can be emitted at the target location, allowing the object to respond and obtain feedback location information. This feedback location information represents the playback position of the intermediate audio signal perceived by the object. The method for obtaining this feedback location information can be configured according to actual business needs and is not limited here. For example, after receiving an intermediate audio signal, the object can perceive the location where the signal was emitted and return this perception information to the processor. The processor can then process this perception information to obtain the feedback location information.

[0040] After obtaining the feedback location information, for each intermediate audio signal, the positional error can be determined based on the actual target location where the intermediate audio signal was emitted and the playback location perceived by the object, resulting in multiple positional errors. Based on these multiple positional errors, an intermediate transfer function is determined from multiple first candidate transfer functions.

[0041] The frequency response characteristics of the intermediate transfer function (ITF) describe the frequency response curves of sound waves as they travel through structures such as the head and reach the ears at different frequencies. These curves reflect the attenuation and phase changes of different frequency components during propagation. The frequency response characteristics can include low-frequency, mid-frequency, and high-frequency ranges. The low-frequency range can refer to frequencies below 1.5 kHz, the mid-frequency range to 1.5 kHz, and the high-frequency range above 4 kHz. In one example, in the low-frequency range, the ITF's frequency response curve is typically relatively flat, indicating that low-frequency sound waves experience less attenuation and phase change during propagation. In the mid-frequency range, the ITF's frequency response curve exhibits some fluctuations and peaks / valleys, which are related to the interaction between the sound waves and structures such as the head and ears. In the high-frequency range, the ITF's frequency response curve is typically more complex, with multiple fluctuations and peaks / valleys; these variations are important for simulating the spatial and directional aspects of sound.

[0042] After obtaining the intermediate transfer function, based on its frequency distribution characteristics (i.e., the properties of different frequency ranges mentioned above), some peaks and troughs can be selected from multiple peaks and troughs to simplify the intermediate transfer function and obtain the target transfer function. Based on this, the first audio signal can be processed using the target transfer function to obtain a playback signal, which can then be used to control the sound source to emit the playback signal. This method is applicable not only to integrated head-mounted displays but also to discrete head-mounted displays.

[0043] According to embodiments of this disclosure, since the intermediate transfer function is determined based on feedback orientation information obtained by the object's perception of the intermediate audio signal, a personalized target transfer function suitable for the object can be obtained by considering the frequency distribution characteristics of the intermediate transfer function. Based on this, by adjusting the first audio signal using the personalized target transfer function, it can be ensured that the playback signal obtained by the object maximally matches its auditory preferences, improving the accuracy and targeting of audio signal processing, thereby enhancing the personalization of the audio experience.

[0044] The above text explained the process of audio signal processing. The following will combine... Figures 3-6 The process of obtaining the target transfer function of this disclosure will be further explained.

[0045] Figure 3 The illustration shows an example schematic diagram of the process for obtaining the target transfer function according to an embodiment of the present disclosure.

[0046] like Figure 3As shown in Figure 300, the process of obtaining the target transfer function may include an offline processing stage and an online processing stage. The offline processing stage can be used to classify the function database 301 to obtain multiple first candidate transfer functions. The online processing stage can utilize a sound source to emit an intermediate audio signal obtained by adjusting a second audio signal using the first candidate transfer functions, and acquire feedback orientation information obtained from the object's perception of the intermediate audio signal. It can also calculate the positioning error between the actual position of the intermediate audio signal emitted by the sound source and the position represented by the feedback orientation information, thereby filtering and correcting the multiple first candidate transfer functions to obtain a target transfer function suitable for the object.

[0047] During the offline processing phase, a function database 301 applicable to the object can be obtained. Function database 301 is a far-field database obtained from actual measurements of the real object. The measurement radius of the multiple second candidate transfer functions included in the far-field database is greater than a preset distance. The preset distance can be 1 meter. In one example, the function database 301 applicable to the object can be selected based on at least one of the following conditions: object type, body structure, head shape, head size, and ear structure. Function database 301 includes N object identifiers and a respective second candidate transfer function for each object identifier. For example, the function database may include second candidate transfer function 301_1, second candidate transfer function 301_2, ..., second candidate transfer function 301_N.

[0048] After obtaining the function database 301, similarity analysis can be performed on the second candidate transfer functions 301_1, 301_2, ..., 301_N to determine the similarities or differences among the multiple second candidate transfer functions in terms of frequency response, phase characteristics, etc., resulting in similarity results 302. The similarity results characterize the average similarity between the corresponding second candidate transfer function and the other (N-1) second candidate transfer functions in the function database for each object identifier.

[0049] The similarity analysis method can be configured according to actual business needs and is not limited here. Alternatively, at least one of the following can be performed on multiple second candidate transfer functions in function database 301: statistical indicator analysis and model fitting analysis. Statistical indicator analysis can refer to using statistical indicators such as correlation coefficients to measure the correlation between different second candidate transfer functions. Model fitting analysis can refer to using statistical methods such as regression analysis and principal component analysis to perform model fitting and dimensionality reduction on different second candidate transfer functions.

[0050] In one example, similarity analysis can be performed on the second candidate transfer function corresponding to the object identifier and each of the other (N-1) second candidate transfer functions to obtain the similarity score with each of the (N-1) second candidate transfer functions. Based on the (N-1) similarities, the mean similarity score of the second candidate transfer functions is determined.

[0051]

[0052] Where m represents the number of sound source locations, k represents the number of frequencies, s and s′ represent different object identifiers, and NCC α (m, s, s′) represents the normalized cross-correlation coefficient between the second candidate transfer function corresponding to s and the second candidate transfer function corresponding to s′, MNCC(s, s′) represents the similarity between the second candidate transfer function corresponding to s and the second candidate transfer function corresponding to s′, and AVEMNCC(s) represents the average similarity between the second candidate transfer function corresponding to s and other second candidate transfer functions.

[0053] After obtaining the similarity result 302, the similarity result 302 can be used to provide distance for subsequent clustering.

[0054] D(s, s′)={2[1-MNCC(s, s′)]} 1 / 2 (4)

[0055] D(s, s′) represents the magnitude difference between the second candidate transfer function corresponding to s and the second candidate transfer function corresponding to s′, and can be used as the basis for subsequent clustering.

[0056] According to embodiments of this disclosure, by performing similarity analysis on the second candidate transfer function corresponding to each object identifier and other (N-1) second candidate transfer functions in the database, the specific similarity between each second candidate transfer function and other functions can be obtained. Based on this, by calculating the average similarity value corresponding to the object identifier, the characteristics of each second candidate transfer function in the overall function database can be more comprehensively evaluated, achieving a holistic evaluation of each second candidate transfer function. This helps to more accurately determine the transfer function that has higher representativeness and universality overall.

[0057] Using the threshold 303 determined based on similarity result 302 as the number of cluster centers, the above N second candidate transfer functions are clustered to determine M clustering results 304. The following will combine... Figure 4 The process of obtaining the clustering results disclosed herein will be further explained.

[0058] Figure 4 The illustration shows an example schematic diagram of the process for obtaining clustering results according to an embodiment of the present disclosure.

[0059] like Figure 4 As shown in Figure 400, the horizontal axis represents the object identifier, and the vertical axis represents the average similarity between the second candidate transfer function corresponding to the object identifier and the other (N-1) second candidate transfer functions in the function database.

[0060] Taking a threshold of 7 determined based on similarity results as an example, that is, using 7 as the number of cluster centers, clustering the N second candidate transfer functions, and obtaining 7 candidate clustering results.

[0061] After obtaining 7 candidate clustering results, for each candidate clustering result, it can be determined whether to retain it based on the relationship between the number of second candidate transfer functions included in that candidate clustering result and a preset value. Candidate clustering results with fewer second candidate transfer functions included will be pruned. The preset value can be configured according to actual business needs and is not limited here. The preset value can be 2.

[0062] For example, if the number of second candidate transfer functions is less than or equal to a preset value, the candidate clustering results can be pruned. Taking the first candidate clustering result as an example, this candidate clustering result includes two second candidate transfer functions, namely the second candidate transfer functions corresponding to object identifier 5 and object identifier 21 respectively. Since the number of second candidate transfer functions included in the first candidate clustering result is equal to the preset value, the first candidate clustering result can be pruned.

[0063] For example, if the number of second candidate transfer functions is greater than a preset value, the candidate clustering result can be determined as the clustering result to be retained. Taking the 6th candidate clustering result as an example, this candidate clustering result includes 9 second candidate transfer functions, namely the second candidate transfer functions corresponding to object identifiers 26, 8, 12, 7, 14, 33, 9, 2, and 30. Since the number of second candidate transfer functions included in the 6th candidate clustering result is greater than the preset value, the 6th candidate clustering result can be determined as the clustering result.

[0064] See also Figure 3 The clustering results include multiple second candidate transfer functions. After obtaining clustering result 304, the cluster center of each clustering result 304 can be determined. The second candidate transfer function corresponding to the cluster center is used as the representative of that clustering result 304. Therefore, the cluster center of that clustering result 304 can be determined as the first candidate transfer function 305, resulting in M ​​first candidate transfer functions 305. The cluster center can be understood as the second candidate transfer function whose mean similarity is among the top M of the N second candidate transfer functions.

[0065] According to embodiments of this disclosure, by introducing a threshold as a guide for the number of cluster centers during the clustering process, and since the selection of the threshold is based on comprehensive similarity analysis, it can reflect the similarity distribution among the second candidate transfer functions, thus ensuring the accuracy and representativeness of the candidate clustering results. Furthermore, by using the relationship between the number of second candidate transfer functions in the candidate clustering results and a preset value to perform pruning on the candidate clustering results, the clustering results can be further optimized, improving the accuracy and reliability of the final clustering results.

[0066] During the online processing phase, for each first candidate transfer function 305, the first candidate transfer function 305 and the original audio signal 306 can be processed to obtain an intermediate audio signal 307. Multiple intermediate audio signals 307 are emitted so that the object can respond to the intermediate audio signal 307 to obtain feedback location information 308. The following will combine... Figure 5 The process of obtaining the feedback location information disclosed herein will be further explained.

[0067] Figure 5 The illustration shows an example schematic diagram of a feedback orientation information acquisition process according to an embodiment of the present disclosure.

[0068] like Figure 5 As shown in Figure 500, for each first candidate transfer function 305, the first candidate transfer function 305 and the original audio signal 306 can be processed to obtain an intermediate audio signal 307. For example, the original audio signal 306 can be subjected to a Fourier transform to obtain a transformed audio signal 502. The transformed audio signal 502 is then processed using the first candidate transfer function 305 to obtain the intermediate audio signal 307.

[0069] According to embodiments of this disclosure, by performing a Fourier transform on the original audio signal, the original audio signal can be converted from the time domain to the frequency domain, thereby allowing for a clearer observation of the spectral characteristics of the audio signal. Furthermore, by selecting a suitable first candidate transfer function, customized processing can be performed for specific needs of the audio signal. By performing a multiplication operation between the first candidate transfer function and the transformed audio signal, complex convolution operations in the time domain are avoided, improving the efficiency and real-time performance of signal processing.

[0070] In one example, the first candidate transfer function 305 may include two sub-transfer functions: one corresponding to the left ear and one corresponding to the right ear. The transformed audio signal 502 is processed to obtain two sub-signals: one corresponding to the left ear and one corresponding to the right ear. Based on this, the sub-transfer function corresponding to the left ear and the sub-signal corresponding to the left ear can be multiplied to obtain an intermediate sub-signal corresponding to the left ear; the sub-transfer function corresponding to the right ear and the sub-signal corresponding to the right ear can be multiplied to obtain an intermediate sub-signal corresponding to the right ear; the intermediate sub-signal corresponding to the left ear and the intermediate sub-signal corresponding to the right ear can be determined as the intermediate audio signal 307.

[0071] After obtaining the intermediate audio signal 307, the audio source 110 can emit the intermediate audio signal 307. After receiving the intermediate audio signal 307 emitted by the audio source, the object 120 can sense the location of the emitted intermediate audio signal 307 and return the sensed data 505 through the interface 140. The form of the interface 140 can be configured according to actual business needs and is not limited here. For example, the interface 140 may include a physical interface or a digital interface.

[0072] In one example, the physical interface can refer to buttons and knobs, etc. When interface 140 is a physical interface, the state of the buttons and knobs can be read by hardware devices such as a microcontroller (MCU) and converted into digital signals to obtain sensing data 505. In another example, the digital interface can refer to touchscreens and software interfaces, etc. When interface 140 is a digital interface, sensing data 505 can be obtained by capturing user actions such as clicks and swipes in the graphical user interface (GUI).

[0073] After receiving the sensing data 505 returned by the object 120, the processor 130 can process the sensing data 505 to obtain feedback orientation information 308.

[0074] According to embodiments of this disclosure, by processing a first candidate transfer function with multiple raw audio signals and emitting the resulting multiple intermediate audio signals for object perception, the performance of each first candidate transfer function in a real-world environment can be evaluated more accurately. Furthermore, since the feedback location information is obtained by processing the perception information returned by the object through an interface, the objectivity and accuracy of the perception process are improved. This facilitates further screening and optimization of the first candidate transfer functions, ensuring that the final selected target transfer function better meets the object's actual needs and preferences.

[0075] See also Figure 3After obtaining the feedback azimuth information 308, for each first candidate transfer function 305, the average angle error 309 can be determined based on the original audio signal 306 and the feedback azimuth information 308. The calculation method of the average angle error 309 can be configured according to actual business needs and is not limited here. For example, for each original audio signal 306 and its corresponding feedback azimuth information, the angle error is calculated, which represents the difference between the actual sound source azimuth and the feedback azimuth. Based on this, the average angle error 309 is calculated for multiple angle errors.

[0076] According to embodiments of this disclosure, by determining the average angular error of each first candidate transfer function based on multiple raw audio signals and corresponding feedback orientation information, the accuracy and stability of the first candidate transfer function in practical applications can be more accurately reflected. Furthermore, by selecting the first candidate transfer function corresponding to the minimum average angular error as an intermediate transfer function, optimized selection of the transfer function is achieved, which helps improve the subsequent audio signal processing effect.

[0077] In one example, azimuth and elevation angles represent angular parameters of a sound source's position relative to an object. The azimuth angle is the horizontal angle measured clockwise from a north-pointing line to the target direction line. In audio signal processing, the azimuth angle represents the relative position of the sound source to the object in the horizontal direction. The azimuth angle ranges from 0° to 360° to cover all directions on the horizontal plane. The elevation angle is the angle measured vertically upwards or downwards from the object's horizontal plane to the sound source direction line. In audio signal processing, the elevation angle represents the relative position of the sound source to the object in the vertical direction. The elevation angle ranges from -90° to 90°, with positive values ​​indicating the sound source is above the object and negative values ​​indicating the sound is below the object.

[0078] The raw audio signal 306 may include a first azimuth angle and a first elevation angle, and the feedback azimuth information 308 may include a second azimuth angle and a second elevation angle. For each raw audio signal 306, a first angular difference can be determined based on the first azimuth angle and the second azimuth angle. This first angular difference represents the difference between the azimuth angle of the actual sound source and the azimuth angle of the perceived sound source. Based on the first elevation angle and the second elevation angle, a second angular difference can be determined. This second angular difference represents the difference between the elevation angle of the actual sound source and the elevation angle of the perceived sound source. Based on this, the angular difference value corresponding to the raw audio signal 306 can be determined.

[0079] After obtaining multiple angle differences, the number of signals among the multiple raw audio signals 306 that meet predetermined conditions can be determined. These predetermined conditions can be configured according to actual business needs and are not limited here. For example, the predetermined conditions may include one of the following: the first azimuth angle is the same, the first elevation angle is the same, or both the first azimuth angle and the first elevation angle are the same.

[0080] After obtaining the number of signals that meet the predetermined conditions, the average angle error 309 can be determined based on multiple angle differences and the number of signals that meet the predetermined conditions from multiple original audio signals. Based on multiple average angle errors 309, an intermediate transfer function 310 is determined from multiple first candidate transfer functions 305. For example, the smaller the average angle error 309, the higher the accuracy of sound source localization. Therefore, the first candidate transfer function 305 corresponding to the smallest average angle error among multiple average angle errors 309 can be determined as the intermediate transfer function 310.

[0081] According to embodiments of this disclosure, by comparing the first azimuth and first pitch angles in the original audio signal with the second azimuth and second pitch angles in the feedback azimuth information, the calculated first angle difference can accurately measure the error of the sound source in the azimuth angle, and the second angle difference can accurately measure the error of the sound source in the pitch angle. Based on this, by determining the average angle error based on multiple angle differences and the number of signals satisfying predetermined conditions, it helps to select the optimal intermediate transfer function from multiple first candidate transfer functions, further ensuring the reliability and accuracy of the intermediate transfer function determination and improving the subsequent audio signal processing effect.

[0082] After obtaining the intermediate transfer function 310, the target transfer function 312 can be determined based on the frequency distribution characteristics 311 of the intermediate transfer function 310. The following will combine... Figure 6 The process of obtaining the target transfer function of this disclosure will be further explained.

[0083] Figure 6 An example schematic diagram of a target transfer function according to an embodiment of the present disclosure is shown.

[0084] like Figure 6 As shown, in step 600, disturbances in the intermediate transfer function 601 can be filtered out to reduce noise and errors introduced during measurement or calculation, resulting in a smoothed transfer function 602. Specific smoothing methods can be configured according to actual business needs and are not limited here. For example, smoothing methods may include at least one of the following: Gaussian filtering, mean filtering, and median filtering.

[0085] For example, when smoothing the intermediate transfer function 601 based on the Gaussian filtering method, the weights can be selected according to the shape of the Gaussian function, a weighted average can be performed on each sampling point, and Gaussian filtering can be applied to smooth these sampling points to obtain the smoothed transfer function 602. The following equations (5) and (6) can be used to smooth the intermediate transfer function 601 based on the Gaussian filtering method to obtain the smoothed transfer function 602.

[0086]

[0087] Where W(n) represents the Gaussian filter, k and n represent the discrete frequency, and σ represents the standard deviation.

[0088] Alternatively, when smoothing the intermediate transfer function 601 using a mean filtering method, the frequency response of the intermediate transfer function 601 can be used as a series of sampling points, and mean filtering can be applied to smooth these sampling points to obtain a smoothed transfer function 602. Alternatively, when smoothing the intermediate transfer function 601 using a median filtering method, the value of each sampling point can be replaced with the median of the values ​​of all sampling points in its neighborhood, and median filtering can be applied to smooth these sampling points to obtain a smoothed transfer function 602.

[0089] After obtaining the smooth transfer function 602, a first distribution feature characterizing the frequency distribution of peaks and a second distribution feature characterizing the frequency distribution of troughs in the spectrum can be determined based on the spectrum of the smooth transfer function 602. A peak can refer to the frequency corresponding to the part of the wave with the highest energy in the spectrum, i.e., the peak point in the spectrum. A trough can refer to the frequency corresponding to the part of the wave with lower energy in the spectrum, i.e., the valley point in the spectrum.

[0090] Taking 600 as an example, the peaks of the smooth transfer function 602 can include points P1, P2, P3, and P4, and the troughs can include points N1, N2, N3, and N4. Since the high-frequency characteristics in the spectrum are the main influencing factors of elevation perception under binaural hearing conditions, when simplifying the smooth transfer function 602, the points in the frequency range below 5kHz (i.e., point P1) can be ignored. Then, the first distribution feature can include the frequency distributions corresponding to points P2, P3, and P4, and the second distribution feature can include the frequency distributions corresponding to points N1, N2, N3, and N4.

[0091] Based on this, the smooth transfer function 602 can be simplified according to the first and second distribution features to obtain the target transfer function 603. The simplification can be understood as setting the sound pressure level of points other than the first and second distribution features to 0, and regenerating the target transfer function 603 along the smooth trend based on the sound pressure levels of each peak included in the first distribution feature and the sound pressure levels of each trough included in the second distribution feature.

[0092] According to embodiments of this disclosure, by filtering out disturbances in the intermediate transfer function, noise and unnecessary fluctuations in the intermediate transfer function can be reduced, resulting in a smoother and more stable transfer function with a smoother frequency response. Based on this, by simplifying the smooth transfer function using a first distribution feature characterizing the frequency distribution of peaks in the spectrum and a second distribution feature characterizing the frequency distribution of troughs in the spectrum obtained from analyzing the spectrum of the smooth transfer function, the frequency response characteristics of the transfer function can be adjusted, thereby more accurately determining the target transfer function.

[0093] By filtering the function database according to the method of classification and matching, intermediate transfer functions that approximate individual objects are obtained. Then, by modifying the intermediate transfer functions, personalized target transfer functions with improved elevation positioning accuracy are obtained. This reduces the complexity of personalized transfer functions and helps to improve the audio signal processing effect.

[0094] The above are merely exemplary embodiments, but are not limited thereto. Other audio signal processing methods known in the art may also be included, as long as they can obtain a personalized target transfer function to enhance the personalization of the audio experience.

[0095] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the audio signal processing method according to the embodiments of this disclosure.

[0096] In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0097] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of this disclosure. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the object detection method provided in the embodiments of this disclosure.

[0098] When the computer program is executed by a processor, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0099] According to embodiments of this disclosure, program code for executing computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages.

[0100] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features recited in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not expressly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure may be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0101] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. An audio signal processing method, comprising: Get the target pass function for the object; as well as The first audio signal is adjusted using the target transfer function to obtain a playback signal, so as to control the sound source to emit the playback signal; The target transfer function is obtained using the following operation: Based on multiple feedback location information, an intermediate transfer function is determined from multiple first candidate transfer functions, wherein the feedback location information is obtained by the object perceiving the intermediate audio signal obtained by adjusting the second audio signal using the first candidate transfer function; and The target transfer function is determined based on the frequency distribution characteristics of the intermediate transfer function.

2. The method according to claim 1, further comprising: A similarity analysis is performed on N second candidate transfer functions in the function database for the object to obtain similarity results. The function database includes N object identifiers and a second candidate transfer function for each object identifier, where N is a positive integer. Using a threshold determined based on the similarity results as the number of cluster centers, the N second candidate transfer functions are clustered to determine M clustering results, where M < N and is a positive integer; and The cluster center of each clustering result is determined as the first candidate transfer function, resulting in the M first candidate transfer functions, wherein the cluster center is the second candidate transfer function whose corresponding mean similarity is among the top M of the N second candidate transfer functions.

3. The method according to claim 2, wherein, The similarity result represents the average similarity between the second candidate transfer function and the other (N-1) second candidate transfer functions in the function database for each object identifier; The similarity analysis performed on the N second candidate transfer functions in the function database for the object yields the following similarity results: For each object identifier, A similarity analysis is performed on the second candidate transfer function corresponding to the object identifier and the other (N-1) second candidate transfer functions respectively to obtain the similarity score corresponding to each of the (N-1) second candidate transfer functions; and Based on the (N-1) similarities, determine the mean similarity of the second candidate transfer function.

4. The method according to claim 2, wherein, The step of clustering the N second candidate transfer functions using a threshold determined based on the similarity results as the number of cluster centers, and determining M clustering results, includes: Using the threshold as the number of cluster centers, cluster the N second candidate transfer functions to obtain the threshold number of candidate clustering results; For each candidate clustering result, in response to the number of second candidate transfer functions in the candidate clustering result being less than or equal to a preset value, the candidate clustering result is pruned.

5. The method according to any one of claims 1 to 4, further comprising, before determining the intermediate transfer function from the plurality of first candidate transfer functions based on the plurality of feedback azimuth information: For each of the first candidate transfer functions The first candidate transfer function is used to adjust the multiple second audio signals to obtain the multiple intermediate audio signals; Emit the plurality of intermediate audio signals; and In response to receiving the perception information returned by the object through the interface, the perception information is processed to obtain the feedback orientation information.

6. The method according to claim 5, wherein, The step of adjusting multiple second audio signals using the first candidate transfer function to obtain multiple intermediate audio signals includes: For each of the second audio signals, a Fourier transform is performed on the second audio signal to obtain the transformed audio signal; and The transformed audio signal is adjusted using the first candidate transfer function to obtain the intermediate audio signal.

7. The method according to any one of claims 1 to 4, wherein, The step of determining an intermediate transfer function from multiple first candidate transfer functions based on the multiple feedback azimuth information includes: For each of the first candidate transfer functions, based on the plurality of second audio signals and the plurality of feedback azimuth information, the average angular error is determined, resulting in a plurality of average angular errors; and The first candidate transfer function corresponding to the smallest average angle error among the plurality of average angle errors is determined as the intermediate transfer function.

8. The method according to claim 7, wherein, The second audio signal includes a first azimuth angle and a first pitch angle, and the feedback azimuth information includes a second azimuth angle and a second pitch angle; The determination of the average angle error based on the plurality of second audio signals and the plurality of feedback azimuth information includes: For each of the second audio signals, Based on the first angle difference between the first azimuth angle and the second azimuth angle, and the second angle difference between the first pitch angle and the second pitch angle, determine the angle difference value corresponding to the second audio signal; and The average angle error is determined based on the number of angle differences and the number of signals among the multiple second audio signals that meet predetermined conditions.

9. The method according to any one of claims 1 to 4, wherein, The determination of the target transfer function based on the frequency distribution characteristics of the intermediate transfer function includes: By filtering out the disturbances in the intermediate transfer function, a smooth transfer function is obtained; Based on the spectrum of the smoothed transfer function, a first distribution feature and a second distribution feature are determined. The first distribution feature characterizes the frequency distribution of peaks in the spectrum, and the second distribution feature characterizes the frequency distribution of troughs in the spectrum. Based on the first distribution feature and the second distribution feature, the smooth transfer function is simplified to obtain the target transfer function.

10. An audio signal processing device, comprising: The processor is used to obtain the target pass function for the object; Furthermore, the first audio signal is adjusted using the target transfer function to obtain a playback signal; The audio source is used to emit the playback signal; The target transfer function is obtained by the following operations: determining an intermediate transfer function from multiple first candidate transfer functions based on multiple feedback directional information, wherein the feedback directional information is obtained by the object perceiving an intermediate audio signal obtained by adjusting a second audio signal using the first candidate transfer function; and determining the target transfer function based on the frequency distribution characteristics of the intermediate transfer function.

11. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 9.

12. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 9.

Images