Audio processing method and device, equipment, storage medium and program product

By performing decorrelation processing and spatial rendering processing on the initial audio signal, the problem of insufficient spatial sense of audio signals in the prior art is solved, and the sound field expansion and spatial sense enhancement of the audio signal are realized.

CN118942476APending Publication Date: 2024-11-12GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310535503.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-11
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In the prior art, the spatial audio effect achieved by binaural room impulse response (BRIR) convolution results in poor spatial sense of the audio signal and it is difficult to expand the sound field.

Method used

By decorrelation processing on the initial audio signal, the correlation between sub-audio signals of different channels is reduced, candidate audio signals are obtained, and spatial rendering is performed to enhance the spatial sense of the audio signal.

Benefits of technology

Effectively expand the sound field and enhance the spatial sense of the audio signal, so that the sound source heard by the human ear has a sufficient spatial sense.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118942476A_ABST
    Figure CN118942476A_ABST
Patent Text Reader

Abstract

The invention relates to an audio processing method and device, equipment, a storage medium and a program product. The method comprises the following steps: performing de-correlation processing on an initial audio signal to reduce the correlation between sub-audio signals of different sound channels in the initial audio signal so as to obtain a candidate audio signal; performing spatial rendering processing on the candidate audio signal to obtain a target audio signal; and performing audio output according to the target audio signal. The method can effectively enhance the sense of space of the audio signal.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of audio technology, and in particular to an audio processing method, device, equipment, storage medium and program product. Background Art

[0002] Currently, users often need to connect headphones to terminals to play audio signals transmitted by the terminals through the headphones. To obtain better auditory effects, headphones can enhance the spatial sense of audio signals and achieve spatial audio effects.

[0003] In the related art, binaural spatial filtering is mainly achieved by convolving an audio signal with a preset binaural room impulse response (BRIR). However, the audio signal played in the related art has a poor spatial sense. Summary of the invention

[0004] Based on this, it is necessary to provide an audio processing method, device, equipment, storage medium and program product that can effectively enhance the spatial sense of audio signals in response to the above technical problems.

[0005] In a first aspect, the present application provides an audio processing method. The method comprises:

[0006] Performing decorrelation processing on the initial audio signal to reduce the correlation between sub-audio signals of different channels in the initial audio signal to obtain a candidate audio signal;

[0007] Performing spatial rendering processing on the candidate audio signal to obtain a target audio signal;

[0008] Audio output is performed according to the target audio signal.

[0009] In a second aspect, the present application also provides an audio processing device. The device comprises:

[0010] A processing module, configured to perform decorrelation processing on the initial audio signal to reduce the correlation between sub-audio signals of different channels in the initial audio signal to obtain a candidate audio signal;

[0011] A rendering module, used to perform spatial rendering processing on the candidate audio signal to obtain a target audio signal;

[0012] The output module is used to perform audio output according to the target audio signal.

[0013] In a third aspect, the present application further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described in the first aspect when executing the computer program.

[0014] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0015] In a fifth aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0016] In the above-mentioned audio processing method, device, equipment, storage medium and program product, the electronic device performs decorrelation processing on the initial audio signal to reduce the correlation between the sub-audio signals of different channels in the initial audio signal to obtain a candidate audio signal; and then performs spatial rendering processing on the candidate audio signal to obtain a target audio signal, so as to perform audio output according to the target audio signal. Among them, by reducing the correlation between the sub-audio signals of different channels in the initial audio signal, the sound field is expanded during audio playback, and then by performing spatial rendering processing, the spatial sense of the target audio signal obtained is greatly enhanced, and the audio output is performed according to the target audio signal, so that the sound source heard by the human ear has sufficient spatial sense. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is a flowchart of an audio processing method in one embodiment;

[0019] Figure 2 is a schematic diagram of a flow chart of a first decorrelation process in an embodiment;

[0020] Figure 3 is a schematic diagram of a flow chart of a second decorrelation process in one embodiment;

[0021] Figure 4 A schematic diagram of a process of obtaining an intermediate audio signal in one embodiment;

[0022] Figure 5 A schematic diagram of a process flow of decorrelation processing in one embodiment;

[0023] Figure 6 A schematic diagram of a process for obtaining space parameters in one embodiment;

[0024] Figure 7A schematic diagram of a spatial rendering process in one embodiment;

[0025] Figure 8 A schematic diagram of a process of front-end spatial rendering in an embodiment;

[0026] Fig. 9 A schematic diagram of a process of rear-position spatial rendering in one embodiment;

[0027] Fig.10 A schematic diagram of a process of spatial rendering of an audio signal in one embodiment;

[0028] Fig.11 A schematic diagram of a process of performing audio output in one embodiment;

[0029] Fig.12 is a schematic structural diagram of an all-pass filter in one embodiment;

[0030] Fig.13 A schematic diagram of a room impulse response of a reverberation model in one embodiment;

[0031] Fig.14 is a schematic diagram of the structure of a reverberation model in one embodiment;

[0032] Fig.15 A schematic diagram of a spatial rendering process of an audio signal by headphones in one embodiment;

[0033] Fig.16 is a structural block diagram of an audio processing device in one embodiment;

[0034] Fig.17 FIG. 4 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0036] When a sound source is transmitted to a person's ears, the auditory system locates a single sound source through the interaural time difference (ITD) and interaural level difference (ILD). There is also the spectral factor HRTF (Head-related Transfer Function) of the sound source incident from different directions, which describes the sound transmission process from the sound source through the environment to the listener's ears.

[0037] Currently, users often need to connect headphones to terminals, and the headphones will play the audio signals transmitted by the terminals as sound sources and transmit them to the ears of people. In order to obtain better hearing effects, headphones can enhance the spatial sense of audio signals and achieve spatial audio effects.

[0038] In the related art, binaural spatial filtering is mainly achieved by convolving the audio signal with a preset binaural room impulse response (BRIR). Among them, when the sound source is located 20° to the right in front of the person's head, the sound source is closer to the right ear during the process of being transmitted to the left and right ears of the person. The sound source first reaches the right ear and then the left ear. Based on this, the earphone end convolves the audio signal transmitted by the terminal with the preset BRIR to implement a binaural spatial sound filter to achieve a spatial audio effect. However, when rendering the spatial audio effect through BRIR, the sound field is difficult to expand, and the "head effect" is obvious, resulting in a poor sense of space for the audio signal.

[0039] In view of this, an embodiment of the present application provides an audio processing method for effectively improving the spatial sense of an audio signal to be played.

[0040] It should be noted that the audio processing method provided in the embodiment of the present application, its execution subject can be an audio processing device, and the audio processing device can be implemented as part or all of an electronic device through software, hardware, or a combination of software and hardware. Among them, the electronic device can be a terminal, for example, various headphones, personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices, and the Internet of Things devices can be smart speakers, smart TVs, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The headphones can be various types of headphones such as limited headphones, wireless headphones or headphones. The embodiment of the present application does not limit the type of the electronic device, as long as it can be used for audio output. In the following method embodiments, the execution subject is explained as an electronic device. For ease of understanding, the electronic device is explained as a headset in the following embodiments.

[0041] In one embodiment, Figure 1 As shown, an audio processing method is provided, comprising the following steps:

[0042] Step 101 : performing decorrelation processing on an initial audio signal to reduce the correlation between sub-audio signals of different channels in the initial audio signal to obtain a candidate audio signal.

[0043] The audio signal that needs to be played obtained by the electronic device is the initial audio signal. Taking the electronic device as a headset as an example, the headset can be connected to the terminal, receive the audio signal that needs to be played transmitted by the terminal and use it as the initial audio signal. The initial audio signal may include sub-audio signals of different channels. For example, the headset may include a left headset and a right headset. Correspondingly, the initial audio signal includes a sub-audio signal of the left channel corresponding to the left headset and a sub-audio signal of the right channel corresponding to the right headset.

[0044] The phase relationship between the sub-audio signals of different channels in the initial audio signal is close, for example, the phases of the sub-audio signals are the same, so the correlation between the sub-audio signals of different channels in the initial audio signal is strong. Playing the sub-audio signals at the same time makes the sound field more concentrated and the sense of space is poor. Therefore, in the embodiment of the present application, decorrelation processing is performed on the initial audio signal to reduce the correlation between the sub-audio signals to enhance the sense of space during audio playback.

[0045] Taking the electronic device as headphones as an example, the sub-audio signals of the above different channels can be the sub-audio signal of the left channel and the sub-audio signal of the right channel. Exemplarily, the electronic device can also be a set of speakers, in which case the sub-audio signals of the above different channels can be the sub-audio signal of the left channel and the sub-audio signal of the right channel, the sub-audio signal of the front channel and the sub-audio signal of the rear channel, the sub-audio signal of the left front channel and the sub-audio signal of the right rear channel, the sub-audio signal of the left front channel and the sub-audio signal of the right front channel, etc., which are not fully exemplified here.

[0046] The audio signal obtained by decorrelation processing the initial audio signal is the candidate audio signal. The candidate audio signal may include a plurality of candidate sub-audio signals, and each candidate sub-audio signal may be a candidate sub-audio signal from a different direction.

[0047] Optionally, after the electronic device acquires the initial audio signal, the electronic device may convert the initial audio signal in the time domain into the frequency domain and perform decorrelation processing.

[0048] Step 102: Perform spatial rendering processing on the candidate audio signal to obtain a target audio signal.

[0049] The spatial rendering processing in the embodiment of the present application can enhance the spatial sense of the candidate audio signal to simulate the process of real sound sources in different directions in the space reaching the human ear, so that the listener can feel the position of the sound sources in different directions in the space.

[0050] Exemplarily, the spatial parameters of different directions can be obtained by simulating the process of transmitting sound sources from different directions to the human ear, and the candidate audio signal can be spatially rendered according to each spatial parameter. Optionally, as mentioned above, the candidate audio signal can include candidate sub-audio signals from different directions. Therefore, the spatial parameters of the corresponding direction can be used to perform spatial rendering on the candidate sub-audio signals in each direction, thereby obtaining the target audio signal. It can be understood that, relative to the initial audio signal, the target audio signal obtained by performing spatial rendering again on the basis of decorrelation processing has a reduced correlation between sub-audio signals of different channels and an enhanced sense of space.

[0051] Step 103: Perform audio output according to the target audio signal.

[0052] Optionally, the electronic device may directly output and play the target audio signal.

[0053] Or, optionally, in order to further enhance the spatial sense of the audio signal played by the electronic device, the target audio signal may continue to be processed for spatial sense enhancement to output a playback-enhanced audio signal.

[0054] In the above audio processing method, the electronic device performs decorrelation processing on the initial audio signal to reduce the correlation between sub-audio signals of different channels in the initial audio signal to obtain a candidate audio signal; and then performs spatial rendering processing on the candidate audio signal to obtain a target audio signal, so as to perform audio output according to the target audio signal. Among them, by reducing the correlation between sub-audio signals of different channels in the initial audio signal, the sound field is expanded during audio playback, and then by performing spatial rendering processing, the spatial sense of the target audio signal obtained is greatly enhanced, and audio output is performed according to the target audio signal, so that the sound source heard by the human ear has sufficient spatial sense.

[0055] In one embodiment, performing decorrelation processing on the initial audio signal includes: performing a first decorrelation processing on the initial audio signal to obtain a forward candidate audio signal; and performing a second decorrelation processing on the initial audio signal to obtain a backward candidate audio signal.

[0056] In an embodiment of the present application, different decorrelation processing is performed on the initial audio signal to obtain audio signals in different directions. It can be understood that, taking the initial audio signal including the sub-audio signal of the left channel and the sub-audio signal of the right channel as an example, the initial audio signal is subjected to the first decorrelation processing to obtain the forward candidate audio signal, that is, the candidate audio signal of the front and left channel and the candidate audio signal of the front and right channel are obtained; the initial audio signal is subjected to the second decorrelation processing to obtain the backward candidate audio signal, that is, the candidate audio signal of the backward and left channel and the candidate audio signal of the backward and right channel are obtained. In this way, the left and right channels are combined with the front and back directions to obtain candidate audio signals in different directions in the space, so as to achieve sound field expansion and provide a basis for enhancing the sense of audio space. The following takes the initial audio signal including the first initial sub-audio signal of the left channel and the second initial sub-audio signal of the right channel as an example to explain the processes of the first decorrelation processing and the second decorrelation processing respectively.

[0057] In one embodiment, Figure 2 As shown, it shows a schematic diagram of a first decorrelation process provided by an embodiment of the present application. Performing a first decorrelation process on an initial audio signal to obtain a forward candidate audio signal includes:

[0058] Step 201: Perform random phase processing on one of a first initial sub audio signal and a second initial sub audio signal to obtain a first candidate sub audio signal, and use the other of the first initial sub audio signal and the second initial sub audio signal as a second candidate sub audio signal.

[0059] Step 202: Obtain a forward candidate audio signal based on the first candidate sub audio signal and the second candidate sub audio signal.

[0060] The sub audio signal corresponding to the left channel is the first initial sub audio signal, and the sub audio signal corresponding to the right channel is the second initial sub audio signal. In the embodiment of the present application, the first decorrelation processing is implemented by random phase processing.

[0061] Among them, the random phase processing is to generate a random phase according to a random value, and process the original audio signal according to the random phase, so as to obtain an audio signal with a different phase from the original audio signal. For example, by multiplying the random phase with the original signal, the signal phase is randomized, that is, a random phase offset is added to the original audio signal. It can be understood that the phase correlation between the first initial sub-audio signal and the second initial sub-audio signal is high. By performing random phase processing on only one of the first initial sub-audio signal and the second initial sub-audio signal, the phases of the two are no longer the same, and decorrelation processing is achieved.

[0062] Exemplarily, the electronic device converts the first initial sub audio signal and the second initial sub audio signal into a frequency domain signal L in , R in . in or R in The random phase processing is performed to reduce the correlation between the left and right channels. The first initial sub-audio signal is subjected to random phase processing to obtain a candidate sub-audio signal L′ in (f) or the second initial sub-audio signal is subjected to random phase processing to obtain a candidate sub-audio signal R′ in (f) as follows:

[0063] or

[0064] in, is a time-varying random value, is a random phase, Optional, in, f is the frequency of the low-frequency sine wave oscillator, fs is the sampling rate, and f can be selected to be less than 5 Hz; optionally, the oscillator is not limited to a sine wave, but can also be a triangle wave, etc., which are not fully illustrated here.

[0065] The first candidate sub audio signal and the second candidate sub audio signal are obtained by performing random phase processing on one of the first initial sub audio signal and the second initial sub audio signal and not processing the other, and the first candidate sub audio signal and the second candidate sub audio signal can be used as forward candidate audio signals, that is, the forward candidate audio signal includes candidate sub audio signals of the left channel and the right channel.

[0066] In one embodiment, Figure 3 As shown, it shows a schematic diagram of a second decorrelation process provided by an embodiment of the present application. Performing a second decorrelation process on the initial audio signal to obtain a backward candidate audio signal includes:

[0067] Step 301: Perform target signal processing on an initial audio signal to obtain an intermediate audio signal.

[0068] Step 302: Perform random phase processing on one of the first middle sub audio signal and the second middle sub audio signal to obtain a third candidate sub audio signal, and use the other of the first middle sub audio signal and the second middle sub audio signal as a fourth candidate sub audio signal.

[0069] In an embodiment of the present application, based on the initial audio signal, target signal processing is performed on the first initial sub-audio signal and the second initial sub-audio signal in the initial audio signal to obtain a rear intermediate audio signal, and further random phase processing is performed on the intermediate audio signal so that the phases of the intermediate sub-audio signals of different channels in the intermediate audio signal are no longer closely correlated, thereby achieving decorrelation processing.

[0070] In one embodiment, Figure 4 As shown, it shows a schematic diagram of a process of obtaining an intermediate audio signal provided by an embodiment of the present application. The initial audio signal is subjected to target signal processing to obtain the intermediate audio signal, including:

[0071] Step 401: Mix a first initial sub audio signal and a second initial sub audio signal to obtain a mixed audio signal.

[0072] Step 402: low-pass filter the mixed audio signal to obtain an intermediate audio signal.

[0073] The electronic device obtains an initial audio signal, determines a first initial sub-audio signal and a second initial sub-audio signal in the initial audio signal, and exemplarily, performs a superposition process on the first initial sub-audio signal and the second initial sub-audio signal to achieve a mixing process of the two to obtain a mixed audio signal. Then, the mixed audio signal is low-pass filtered to retain low-frequency components and ensure the quality of the mixed audio signal, and the signal obtained after the low-pass filtering process is used as the intermediate audio signal.

[0074] Optionally, the electronic device may convert the first initial sub audio signal and the second initial sub audio signal in the initial audio signal into frequency domain signals and then perform mixing processing. Alternatively, the electronic device may also perform mixing processing on the first initial sub audio signal and the second initial sub audio signal and then convert them into frequency domain signals for subsequent operations.

[0075] Among them, parameters such as the cutoff frequency of the low-pass filtering process can be determined according to requirements, for example, the cutoff frequency can be 6 kHz.

[0076] Step 303: Obtain a backward candidate audio signal based on the third candidate sub audio signal and the fourth candidate sub audio signal.

[0077] The intermediate audio signal includes a first intermediate sub audio signal of a left channel and a second intermediate sub audio signal of a right channel. Similar to the random processing process above, in the second decorrelation processing process, one of the first intermediate sub audio signal and the second intermediate sub audio signal is subjected to random phase processing to obtain a third candidate sub audio signal, while the other of the first intermediate sub audio signal and the second intermediate sub audio signal is not subjected to random phase processing and is directly used as a fourth candidate sub audio signal, thereby reducing the correlation between the left and right channels.

[0078] Optionally, one of the first intermediate sub audio signal and the second intermediate sub audio signal may be multiplied by a random phase To obtain the third candidate sub audio signal. It should be noted that the random value in the random phase used in the first decorrelation process above and the random value in the random phase used in the second decorrelation process here can be the same or different.

[0079] In an embodiment of the present application, by performing a first decorrelation process and a second decorrelation process, candidate audio signals in different directions are obtained. By performing a second decorrelation process, the sub-audio signals of the left and right channels are mixed and then subjected to a random phase decorrelation process to simulate the rear surround signal, thereby achieving sound field expansion of the initial audio signal and effectively reducing the correlation between the sub-audio signals of different channels, thereby enhancing the sense of audio space.

[0080] In conjunction with the above instructions, please refer to Figure 5 , taking the random phase processing of the right channel during the first decorrelation processing as an example, a schematic diagram of the process of decorrelation processing the first initial sub-audio signal corresponding to the left channel in the initial audio signal and the second initial sub-audio signal of the right channel to obtain the candidate audio signal is shown.

[0081] After the electronic device obtains the candidate audio signal, it can obtain the target audio signal by performing spatial rendering on the candidate audio signal. In the embodiment of the present application, spatial rendering in different directions can be performed on the candidate audio signal according to spatial parameters in different directions, so as to obtain a target audio signal with a certain sense of space. The process of spatial rendering is described below.

[0082] In one embodiment, spatial rendering is performed on the candidate audio signal to obtain a target audio signal, including: spatial rendering is performed on the candidate audio signal using spatial parameters to obtain a target audio signal; wherein the spatial parameters are used to characterize a virtual sound transmission process in which the sound of a virtual sound source is transmitted to the listener's ears through a virtual sound transmission environment.

[0083] When spatial rendering is required, the electronic device can obtain spatial parameters, which represent the virtual sound transmission process of the sound of the virtual sound source being transmitted to the listener's ears through the virtual sound transmission environment, that is, the spatial parameters represent the sound transmission process of the sound source being transmitted to the human ear through the preset environment.

[0084] Optionally, the spatial parameter may include multiple spatial parameters in different directions, that is, the spatial parameters in different directions represent the virtual sound transmission process of the virtual sound source from different directions through the virtual sound transmission environment to the listener's ears. In this way, the candidate audio signal is spatially rendered according to the spatial parameters in different directions to achieve spatial enhancement processing, and audio output is performed according to the rendered audio signal, which can effectively enhance the listener's spatial experience.

[0085] Exemplarily, the spatial parameter may be a head-related transfer function (HRTF), which may include a plurality of HRTFs in different orientations.

[0086] As mentioned above, the related art uses BRIR to convolve the audio signal to achieve binaural spatial filtering. However, BRIR includes direct sound and reflected sound, that is, it represents the sound transmission response of the sound source directly reaching the listener's ears and the sound transmission response of the sound source reaching the listener's ears after being reflected by the environment. The use of BRIR for convolution processing results in a large consumption of chip computing power of the electronic device. The HRTF used in the embodiment of the present application represents the virtual sound transmission process of the sound of the virtual sound source propagating to the listener's ears through the virtual sound transmission environment, and does not include the transmission process of reflected sound. Therefore, using it for spatial rendering processing can effectively reduce the computing power consumption of electronic devices, while achieving an enhanced sense of space, avoiding excessive power consumption.

[0087] In one embodiment, the electronic device can determine the spatial parameter according to the angle data. Figure 6 , which shows a schematic diagram of a process for obtaining space parameters provided by an embodiment of the present application. The method also includes:

[0088] Step 601: Acquire target angle data. The target angle data is used to represent the angle between the virtual sound source and a preset direction.

[0089] The virtual sound source refers to the sound source heard by the listener's ears. The preset direction can be the direction directly in front of the listener's head. Optionally, the target angle data can be user-defined, that is, the user can customize the listening direction of the virtual sound source composed of the audio signal ultimately played by the electronic device through the speaker. Optionally, the target angle data may include the front and rear speaker angles.

[0090] In one practicable manner, the electronic device may be connected to a terminal, a preset application may be installed in the terminal, and the user sets the angle between the listener's head and the speaker through the preset application in the terminal to be used as the target angle data.

[0091] In another achievable manner, a gyro acceleration sensor is provided in the electronic device, wherein the gyro acceleration sensor is generally used to determine the user's head tracking angle. The user can set the angle of the gyro acceleration sensor through the preset application to be used as the target angle data.

[0092] Optionally, the electronic device may also obtain a head tracking angle determined in real time by a gyroscope acceleration sensor as target angle data.

[0093] Step 602: Determine, according to the target angle data, a spatial parameter corresponding to the target angle data from a spatial parameter set.

[0094] The electronic device may store a spatial parameter set, which includes a correspondence between multiple sets of different angle data and spatial parameters. After the target angle data is determined, the spatial parameters corresponding to the target angle data are found from the spatial parameter set for subsequent spatial rendering.

[0095] Optionally, the spatial parameter corresponding to each angle data in the spatial parameter set may be an HRTF of different angles and orientations obtained in advance by free field measurement.

[0096] In the embodiment of the present application, the corresponding spatial parameters can be determined according to the user-defined target angle data. Thus, the sound field formed after the audio signal generated after spatial rendering according to the spatial parameters is played meets the user's hearing needs, and the flexibility of spatial enhancement is greatly improved.

[0097] In one embodiment, Figure 7 As shown, it shows a schematic diagram of a spatial rendering process provided by an embodiment of the present application. Using spatial parameters to perform spatial rendering processing on candidate audio signals to obtain a target audio signal includes:

[0098] Step 701 : Perform a forward spatial rendering process on the forward candidate audio signal using the spatial parameters, and perform a backward spatial rendering process on the backward candidate audio signal to obtain a forward target audio signal and a backward target audio signal.

[0099] As mentioned above, the candidate audio signals include forward candidate audio signals and backward candidate audio signals. In order to accurately perform spatial rendering, spatial parameters can be used to perform spatial rendering processing in different directions on the forward candidate audio signals and the backward candidate audio signals to expand the sound field. Among them, the forward candidate audio signals are processed by spatial parameters for forward spatial rendering to obtain the forward target audio signals; the backward candidate audio signals are processed by spatial parameters for backward spatial rendering to obtain the backward target audio signals.

[0100] In one embodiment, the forward candidate audio signal is subjected to forward spatial rendering processing using spatial parameters, and the backward candidate audio signal is subjected to backward spatial rendering processing, including: filtering the forward candidate audio signal according to the forward spatial parameters to obtain a forward target audio signal; filtering the backward candidate audio signal according to the backward spatial parameters to obtain a backward target audio signal.

[0101] The spatial parameters may include front azimuth spatial parameters and rear azimuth spatial parameters. Specifically, the front azimuth spatial parameters may include front azimuth spatial parameters of different channels, and the rear azimuth spatial parameters also include front azimuth spatial parameters of different channels. For each sub-candidate audio signal in the forward candidate audio signal, the front azimuth spatial parameters of different channels may be used for filtering processing, and for each sub-candidate audio signal in the backward candidate audio signal, the rear azimuth spatial parameters of different channels may also be used for filtering processing, thereby spatially rendering the front surround signal and the rear surround signal in all-round stereo, respectively, to achieve comprehensive spatial enhancement processing.

[0102] Step 702: Mix the forward target audio signal and the backward target audio signal to obtain a target audio signal.

[0103] Since the forward candidate audio signal and the backward candidate audio signal each include a plurality of candidate sub-audio signals, accordingly, the forward target audio signal and the backward target audio signal obtained after spatial rendering processing of the two are also respectively composed of a plurality of audio signals.

[0104] If it is necessary to output an audio signal through the left channel and the right channel respectively, the forward target audio signal and the backward target audio signal can be mixed to obtain audio signals that can be output through the left channel and the right channel.

[0105] In one embodiment, Figure 8 , which shows a schematic diagram of the process of forward azimuth space rendering provided by an embodiment of the present application. The forward candidate audio signal is filtered according to the forward azimuth space parameter to obtain the forward target audio signal, including:

[0106] Step 801 : filter a candidate sub-audio signal in a forward candidate audio signal according to a first left channel front azimuth spatial parameter and a first right channel front azimuth spatial parameter to obtain a first left front audio signal and a first right front audio signal.

[0107] The front spatial parameters may include the first left channel front spatial parameter H RL (fθ R ) and the first right channel front position space parameter H RR (fθ R ), which are used to filter one of the candidate sub-audio signals in the forward candidate audio signal. Taking the filtering of the first candidate sub-audio signal in the forward candidate audio signal as an example, illustratively, in the embodiment of the present application, the first candidate sub-audio signal can be multiplied by the first left channel front position spatial parameter to obtain the first left front audio signal; the first candidate sub-audio signal can be multiplied by the first right channel front position spatial parameter to obtain the first right front audio signal, so as to achieve sound field expansion and spatial enhancement.

[0108] Step 802 : Filter another candidate sub-audio signal in the forward candidate audio signal according to the second left channel front azimuth spatial parameter and the second right channel front azimuth spatial parameter to obtain a second left front audio signal and a second right front audio signal.

[0109] The front position space parameter may also include the second left channel front position space parameter H LL (fθ L ) and the second right channel front position space parameter H LR (fθ L ), which are used to filter another candidate sub-audio signal in the forward candidate audio signal. Taking the filtering of the second candidate sub-audio signal in the forward candidate audio signal as an example, illustratively, in the embodiment of the present application, the second candidate sub-audio signal can be multiplied by the second left channel front position spatial parameter to obtain a second left front audio signal; the second candidate sub-audio signal can be multiplied by the second right channel front position spatial parameter to obtain a second right front audio signal, so as to achieve sound field expansion and spatial enhancement.

[0110] Step 803: Use the first left front audio signal, the first right front audio signal, the second left front audio signal, and the second right front audio signal as forward target audio signals.

[0111] Thus, different front audio signals are obtained by filtering the front spatial parameters of different channels to form a forward target audio signal.

[0112] In one embodiment, Fig. 9As shown, it shows a schematic diagram of the process of rearward spatial rendering provided by an embodiment of the present application. The backward candidate audio signal is filtered according to the rearward spatial parameter to obtain the backward target audio signal, including:

[0113] Step 901 : filter a candidate sub-audio signal in a rear candidate audio signal according to a first left channel rear azimuth spatial parameter and a first right channel rear azimuth spatial parameter to obtain a first left rear audio signal and a first right rear audio signal.

[0114] The rear spatial parameters may include the first left channel rear spatial parameter H LL (bθ L ) and the first right channel rear azimuth spatial parameter H LR (bθ L ), which are used to filter one of the candidate sub-audio signals in the backward candidate audio signal. Taking the filtering of the third candidate sub-audio signal in the backward candidate audio signal as an example, illustratively, in the embodiment of the present application, the third candidate sub-audio signal may be multiplied by the first left channel rear azimuth spatial parameter to obtain the first left rear audio signal; and the third candidate sub-audio signal may be multiplied by the first right channel rear azimuth spatial parameter to obtain the first right rear audio signal.

[0115] Step 902: Filter another candidate sub-audio signal in the rear candidate audio signal according to the second left channel rear azimuth spatial parameter and the second right channel rear azimuth spatial parameter to obtain a second left rear audio signal and a second right rear audio signal.

[0116] The rear spatial parameters may also include the second left channel rear spatial parameter H RL (bθ R ) and the second right channel rear azimuth spatial parameter H RR (bθ R ), which are used to filter another candidate sub-audio signal in the backward candidate audio signal. Taking the filtering of the fourth candidate sub-audio signal in the backward candidate audio signal as an example, illustratively, in the embodiment of the present application, the fourth candidate sub-audio signal may be multiplied by the second left channel rear azimuth spatial parameter to obtain the second left rear audio signal; and the fourth candidate sub-audio signal may be multiplied by the second right channel rear azimuth spatial parameter to obtain the second right rear audio signal.

[0117] Step 903: Use the first left rear audio signal, the first right rear audio signal, the second left rear audio signal, and the second right rear audio signal as rear target audio signals.

[0118] Thus, different rear audio signals are obtained by filtering the rear spatial parameters of different channels to form a rear target audio signal.

[0119] In conjunction with the above, please refer to Fig.10 A schematic diagram of the process of spatial rendering of an audio signal is shown.

[0120] In one embodiment, reference Fig.10 , mixing the forward target audio signal and the rearward target audio signal to obtain the target audio signal, including: mixing the first left front audio signal, the second left front audio signal, the first left rear audio signal and the second left rear audio signal to obtain the first target sub-audio signal corresponding to the left channel; mixing the first right front audio signal, the second right front audio signal, the first right rear audio signal and the second right rear audio signal to obtain the second target sub-audio signal corresponding to the right channel; using the first target sub-audio signal and the second sub-target audio signal as the target audio signal.

[0121] Among them, the first left front audio signal, the second left front audio signal, the first left rear audio signal and the second left rear audio signal are related to the left channel, so they can be mixed and processed to generate a first target sub audio signal corresponding to the left channel. The first right front audio signal, the second right front audio signal, the first right rear audio signal and the second right rear audio signal are related to the right channel, so they can be mixed and processed to generate a second target sub audio signal corresponding to the right channel. Subsequent audio output can be performed according to the first target sub audio signal and the second sub target audio signal.

[0122] In an embodiment of the present application, on the basis of decorrelation processing of the initial audio signal, spatial rendering processing is continued on the candidate sub-audio signals of different directions included in the obtained candidate audio signal to achieve spatial rendering of the front surround signal and the rear surround signal, thereby widening the sound field, enhancing the stereoscopic effect, and effectively improving the sense of space.

[0123] The target audio signal obtained is an audio signal after spatial rendering processing, so audio output can be performed according to the target audio signal. Optionally, the electronic device can directly output the first target sub-audio signal and the second target sub-audio signal in the target audio signal. Alternatively, the electronic device can also continue to perform reverberation processing on the first target sub-audio signal and the second target sub-audio signal to continue to reduce the correlation between the audio signals played back by the left and right channels and enhance the sense of space. The following describes two methods for audio output respectively.

[0124] In one embodiment, performing audio output according to the target audio signal includes: outputting a first target sub-audio signal corresponding to a left channel included in the target audio signal through a speaker of the left channel, and outputting a second target sub-audio signal corresponding to a right channel included in the target audio signal through a speaker of the right channel. That is, taking the electronic device as a headset as an example, the first target sub-audio signal can be directly output through a speaker in the left headset, and the second target sub-audio signal can be output through a speaker in the right headset.

[0125] It should be noted that the electronic device can convert the first target sub-audio signal and the second target sub-audio signal into time domain signals and then output the audio. Optionally, the conversion of the time domain signal into the frequency domain signal mentioned above can be achieved by Fourier transform, and the conversion of the frequency domain signal into the time domain signal can be achieved by inverse Fourier transform.

[0126] In one embodiment, Fig.11 FIG. 1 is a flow chart of an audio output process according to an embodiment of the present application. The audio output is performed according to a target audio signal, including:

[0127] Step 1101: Filter a first target sub-audio signal corresponding to a left channel and a second target sub-audio signal corresponding to a right channel included in a target audio signal using a reverberation model to obtain a first enhanced target audio signal corresponding to the left channel and a second enhanced target audio signal corresponding to the right channel.

[0128] Step 1102: output the first enhanced target audio signal through the speaker of the left channel, and output the second enhanced target audio signal through the speaker of the right channel.

[0129] Among them, the reverberation model is used to simulate the room impulse response to make the spatial sense more realistic. The reverberation model can include direct sound, early reflection sound and late reverberation sound. The reverberation model can be used to adjust the ratio between reverberation sound and direct sound, reflect the sense of distance and depth of sound, and deeply enhance the spatial sense of the target audio signal.

[0130] Optionally, the electronic device may use the reverberation model to perform reverberation filtering processing of different degrees on the first target sub-audio signal and the second target sub-audio signal, and then output the obtained first enhanced target audio signal through the left earphone and the obtained second enhanced target audio signal through the right earphone.

[0131] In the embodiment of the present application, the target audio signal is filtered using the reverberation model before being output, so that the spatial sense of the output audio signal is more realistic.

[0132] In one embodiment, the reverberation model is constructed from a plurality of all-pass filters.

[0133] That is, the embodiment of the present application can construct a reverberation model through all-pass filter modeling to simulate the room impulse response. Optionally, the function of the all-pass filter can be implemented through a software algorithm, or through a combination of software and hardware.

[0134] In one embodiment, the all-pass filter includes a delay module, and the delay module is used to perform different delay processing on the first target sub-audio signal and the second target sub-audio signal.

[0135] Optional, such as Fig.12 The structural schematic diagram of an all-pass filter provided by the embodiment of the present application shown. The all-pass filter may include a first gain module 1201, a first mixing module 1202, a delay module 1203, a second mixing module 1204, and a second gain module 1205. The first gain module is used to adjust the gain of the signal input to the all-pass filter, and output the adjusted signal to the second mixing module. The first mixing module is used to perform superposition processing on the signal input to the all-pass filter and the signal output by the second gain module, and output the superimposed signal to the delay module. The delay module performs delay processing on the input signal and outputs it to the second mixing module. Among them, when the all-pass filter performs reverberation processing on the first target sub-audio signal and the second target sub-audio signal respectively, the delay line of the delay processing performed by the delay module on the first target sub-audio signal and the second target sub-audio signal can be different, that is, the delay module modulation can dynamically change the delay line size, increase the discrete degree of reverberation, reduce the correlation between the left and right channel playback sound signals, and enhance the sense of space.

[0136] The second mixing module performs superposition processing on the signals output by the delay module and the first gain module to obtain a signal output by the all-pass filter.

[0137] like Fig.13 As shown, it shows a schematic diagram of a room impulse response of a reverberation model provided in an embodiment of the present application.

[0138] In an optional embodiment, the delay duration of the delay module is determined according to the time sequence of the target audio signal.

[0139] Among them, based on the all-pass filter, the delay time d can be adaptively adjusted to obtain a filter with constant amplitude-frequency response and dynamically changing phase for filtering. Optionally, d can be determined by the following formula:

[0140] d=sin(2*π*f / fs*n)*M+Q (2)

[0141] Among them, f is the frequency of the low-frequency sine wave oscillator, fs is the sampling rate, and f can be selected to be less than 5Hz; M is the maximum modulation delay sample value; Q is the fixed delay sample value. Optionally, the oscillator is not limited to sine waves, but can also be a triangle wave, etc. n is a time series.

[0142] In one embodiment, the all-pass filter includes a gain module, and the gain module is used to perform gain adjustment processing with different intensities on the first target sub audio signal and the second target sub audio signal.

[0143] The first gain module and the second gain module can be used to adjust the strength of the room impulse response, and the delay module can be used to adjust the density of the room impulse response. When the first target sub-audio signal and the second target sub-audio signal are respectively subjected to reverberation processing using an all-pass filter, the gain values ​​adjusted by the first gain module and the second gain module can be different to enhance the sense of space.

[0144] In one embodiment, the reverberation model is specifically composed of multiple all-pass filters connected in series and / or in parallel. That is, in the embodiment of the present application, the reverberation model can be composed of multiple all-pass filters connected in series, or multiple all-pass filters connected in series and in parallel, which is not specifically limited here, as long as the reverberation model can be realized. For example, Fig.14 , which shows a schematic diagram of the structure of a reverberation model provided by an embodiment of the present application. The reverberation model is obtained by connecting two all-pass filters in parallel and then connecting an all-pass filter in series.

[0145] In one embodiment, before decorrelation processing is performed on the initial audio signal, the method further comprises: delaying one of the sub-audio signals of different channels in the initial audio signal.

[0146] After the electronic device obtains the initial audio signal, it performs time domain dynamic delay processing on the sub-audio signal of one channel in the initial audio signal, and then continues to perform decorrelation processing on the sub-audio signal obtained by the time domain dynamic delay processing and the sub-audio signal that has not been delayed, thereby further reducing the correlation of the sub-audio signals of each channel in the initial audio signal and enhancing the stereo strength.

[0147] like Fig.15Schematic diagram of the spatial rendering process of the earphones shown in the embodiment of the present application. The hardware solution of the earphones in the embodiment of the present application can be composed of devices such as a gyroscope sensor, a speaker, a digital audio signal processing chip, a storage circuit, and a power supply. After receiving the stereo signals of the left and right channels, the earphone end converts them into frequency domain signals. The frequency domain signal of one of the channels is subjected to random phase processing to reduce the correlation of the audio signals of the left and right channels. The audio signal after random phase processing and the audio signal of the other channel that has not been subjected to random phase processing are respectively subjected to the preset front position HRTF filtering process. At the same time, the frequency domain signals of the left and right channels are mixed and processed and low-pass filtered, and then the audio signal of one of the channels in the audio signal after low-pass filtering is subjected to random phase processing as one of the rear surround signals, while the audio signal of the other channel in the audio signal after low-pass filtering is not subjected to random phase processing to continue to reduce the correlation of the left and right channels, and the two audio signals are respectively subjected to the preset rear position HRTF filtering process, and finally mixed and processed to obtain the frequency domain signals of the left and right channels and converted into time domain signals to output as left audio signals and right audio signals. Fig.15 Middle, H LL (fθ L ), H RL (fθ R ), H RR (fθ R ), H LR (fθ L ), H LL (bθ L ), H RL (bθ R ), H RR (bθ R ), H LR (bθ L ) is the HRTF of different angles / orientations measured in the free field. The calculation and implementation process of the spatial audio solution is as follows:

[0148]

[0149] LF out =H LL (fθ L )L in +H RL (fθ R )R′ in

[0150] RF out =H LR (fθ L )L in +H RR (fθ R )R′in

[0151]

[0152]

[0153] LS out =C′ in H LL (bθ L )+C in H RL (bθ R )

[0154] RS out =C in H RR (bθ R )+C′ in H LR (bθ L )

[0155] L out =LF out +LS out

[0156] R out =RF out +RS out

[0157] Among them, LF out ,RF out , C in , C′ in LS out , RS out is the audio signal at each stage of the processing. out is the left audio signal, R out For the right audio signal.

[0158] The left audio signal and the right audio signal are respectively input into the reverberation model for filtering processing to make the spatial sense more realistic, and the left enhanced audio signal and the right enhanced audio signal are obtained. The left enhanced audio signal is output through the speaker of the left earphone, and the right enhanced audio signal is output through the speaker of the right earphone. Among them, the reverberation model is constructed by modeling with an all-pass filter to simulate the room impulse response, making the spatial sense more realistic. The reverberation model includes direct sound, early reflection, and late reverberation. The reverberation model is constructed by connecting multiple all-pass filters in series and parallel to generate a higher density impulse response to simulate the room impulse response. The strength of the room impulse response is adjusted by adjusting the gain of the all-pass filter, and the density of the room impulse response is adjusted by adjusting the delay parameter of the all-pass filter.

[0159] In the embodiment of the present application, the correlation between the left and right channels is reduced by converting the stereo signal into a frequency domain signal and performing random phase processing on one of the channels, thereby increasing the width of the stereo sound field and enhancing the spatial effect. The left and right channels are mixed and then subjected to random phase decorrelation processing in the frequency domain to simulate the rear surround signal, and the rear surround space rendering is performed. The delay line size is dynamically changed by modulating the all-pass filter delay module of the reverberation model, increasing the discreteness of the reverberation, reducing the correlation between the left and right ear playback sound signals, and enhancing the sense of space.

[0160] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0161] Based on the same inventive concept, the embodiment of the present application also provides an audio processing device for implementing the audio processing method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more audio processing device embodiments provided below can refer to the limitations on the audio processing method above, and will not be repeated here.

[0162] In one embodiment, Fig.16 As shown, an audio processing device is provided, the audio processing device 1600 includes: a processing module 1601, a rendering module 1602 and an output module 1603, wherein:

[0163] The processing module 1601 is used to perform decorrelation processing on the initial audio signal to reduce the correlation between sub-audio signals of different channels in the initial audio signal to obtain a candidate audio signal;

[0164] A rendering module 1602 is used to perform spatial rendering processing on the candidate audio signal to obtain a target audio signal;

[0165] The output module 1603 is used to perform audio output according to the target audio signal.

[0166] In one embodiment, the rendering module 1602 is specifically configured to:

[0167] The candidate audio signal is spatially rendered using spatial parameters to obtain a target audio signal; wherein the spatial parameters are used to characterize a virtual sound transmission process in which the sound of a virtual sound source is transmitted to the ears of a listener through a virtual sound transmission environment.

[0168] In one embodiment, the candidate audio signal includes a forward candidate audio signal and a backward candidate audio signal, and the rendering module 1602 is specifically used to: perform forward spatial rendering processing on the forward candidate audio signal using spatial parameters, and perform backward spatial rendering processing on the backward candidate audio signal to obtain a forward target audio signal and a backward target audio signal; and perform mixing processing on the forward target audio signal and the backward target audio signal to obtain a target audio signal.

[0169] In one embodiment, the processing module 1601 is specifically configured to: perform a first decorrelation process on the initial audio signal to obtain a forward candidate audio signal; and perform a second decorrelation process on the initial audio signal to obtain a backward candidate audio signal.

[0170] In one embodiment, the initial audio signal includes a first initial sub-audio signal for a left channel and a second initial sub-audio signal for a right channel, and the processing module 1601 is specifically used to: perform random phase processing on one of the first initial sub-audio signal and the second initial sub-audio signal to obtain a first candidate sub-audio signal, and use the other of the first initial sub-audio signal and the second initial sub-audio signal as the second candidate sub-audio signal; and obtain a forward candidate audio signal based on the first candidate sub-audio signal and the second candidate sub-audio signal.

[0171] In one embodiment, the processing module 1601 is specifically used to: perform target signal processing on the initial audio signal to obtain an intermediate audio signal, the intermediate audio signal including a first intermediate sub audio signal of a left channel and a second intermediate sub audio signal of a right channel; perform random phase processing on one of the first intermediate sub audio signal and the second intermediate sub audio signal to obtain a third candidate sub audio signal, and use the other of the first intermediate sub audio signal and the second intermediate sub audio signal as a fourth candidate sub audio signal; and obtain a backward candidate audio signal based on the third candidate sub audio signal and the fourth candidate sub audio signal.

[0172] In one embodiment, the initial audio signal includes a first initial sub-audio signal of a left channel and a second initial sub-audio signal of a right channel, and the processing module 1601 is specifically used to: mix the first initial sub-audio signal and the second initial sub-audio signal to obtain a mixed audio signal; and perform low-pass filtering on the mixed audio signal to obtain an intermediate audio signal.

[0173] In one embodiment, the spatial parameters include forward spatial parameters and rearward spatial parameters; the rendering module 1602 is specifically used to: filter the forward candidate audio signal according to the forward spatial parameters to obtain a forward target audio signal; filter the backward candidate audio signal according to the rearward spatial parameters to obtain a backward target audio signal.

[0174] In one embodiment, the front azimuth spatial parameters include a first left channel front azimuth spatial parameter, a first right channel front azimuth spatial parameter, a second left channel front azimuth spatial parameter and a second right channel front azimuth spatial parameter; the rendering module 1602 is specifically used to: filter one candidate sub-audio signal in the forward candidate audio signal according to the first left channel front azimuth spatial parameter and the first right channel front azimuth spatial parameter, to obtain a first left front audio signal and a first right front audio signal; filter another candidate sub-audio signal in the forward candidate audio signal according to the second left channel front azimuth spatial parameter and the second right channel front azimuth spatial parameter, to obtain a second left front audio signal and a second right front audio signal; and use the first left front audio signal, the first right front audio signal, the second left front audio signal and the second right front audio signal as forward target audio signals.

[0175] In one embodiment, the rear azimuth spatial parameters include a first left channel rear azimuth spatial parameter, a first right channel rear azimuth spatial parameter, a second left channel rear azimuth spatial parameter, and a second right channel rear azimuth spatial parameter; the rendering module 1602 is specifically used to: filter one candidate sub-audio signal in the rear candidate audio signal according to the first left channel rear azimuth spatial parameter and the first right channel rear azimuth spatial parameter, to obtain a first left rear audio signal and a first right rear audio signal; filter another candidate sub-audio signal in the rear candidate audio signal according to the second left channel rear azimuth spatial parameter and the second right channel rear azimuth spatial parameter, to obtain a second left rear audio signal and a second right rear audio signal; use the first left rear audio signal, the first right rear audio signal, the second left rear audio signal, and the second right rear audio signal as rear target audio signals.

[0176] In one embodiment, the rendering module 1602 is specifically used to: mix the first left front audio signal, the second left front audio signal, the first left rear audio signal, and the second left rear audio signal to obtain a first target sub audio signal corresponding to the left channel; mix the first right front audio signal, the second right front audio signal, the first right rear audio signal, and the second right rear audio signal to obtain a second target sub audio signal corresponding to the right channel; and use the first target sub audio signal and the second sub target audio signal as target audio signals.

[0177] In one embodiment, the device also includes: a parameter determination module, used to obtain target angle data, the target angle data is used to characterize the angle between the virtual sound source and the preset direction; according to the target angle data, determine the spatial parameters corresponding to the target angle data from the spatial parameter set.

[0178] In one embodiment, the output module 1603 is specifically configured to:

[0179] A first target sub audio signal corresponding to a left channel included in the target audio signal is output through a speaker of the left channel, and a second target sub audio signal corresponding to a right channel included in the target audio signal is output through a speaker of the right channel.

[0180] In one embodiment, the output module 1603 is specifically configured to:

[0181] A reverberation model is used to filter the first target sub-audio signal corresponding to the left channel and the second target sub-audio signal corresponding to the right channel included in the target audio signal, respectively, to obtain a first enhanced target audio signal corresponding to the left channel and a second enhanced target audio signal corresponding to the right channel; the first enhanced target audio signal is output through the speaker of the left channel, and the second enhanced target audio signal is output through the speaker of the right channel.

[0182] In one embodiment, the reverberation model is constructed from a plurality of all-pass filters.

[0183] In one embodiment, the reverberation model is specifically composed of a plurality of all-pass filters connected in series and / or in parallel.

[0184] In one embodiment, the all-pass filter includes a delay module, and the delay module is used to perform different delay processing on the first target sub-audio signal and the second target sub-audio signal.

[0185] In one embodiment, the delay duration of the delay module is determined according to the time sequence of the target audio signal.

[0186] In one embodiment, the all-pass filter further includes a gain module, and the gain module is used to perform gain adjustment processing with different intensities on the first target sub-audio signal and the second target sub-audio signal.

[0187] In one embodiment, the device further comprises: a delay module, configured to perform delay processing on one of the sub-audio signals of different channels in the initial audio signal before performing decorrelation processing on the initial audio signal.

[0188] Each module in the above audio processing device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in an electronic device in the form of hardware, or can be stored in a memory in an electronic device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0189] In one embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as follows: Fig.17 As shown. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the electronic device is used to exchange information between the processor and an external device. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an audio processing method is implemented. The display unit of the electronic device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the electronic device casing, or an external keyboard, touchpad or mouse.

[0190] Those skilled in the art will understand that Fig.17 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0191] In one embodiment, an electronic device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0192] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0193] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0194] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0195] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0196] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An audio processing method, characterized in that: The method comprises: Performing decorrelation processing on the initial audio signal to reduce the correlation between sub-audio signals of different channels in the initial audio signal to obtain a candidate audio signal; Performing spatial rendering processing on the candidate audio signal to obtain a target audio signal; Audio output is performed according to the target audio signal.

2. The method according to claim 1, characterized in that The performing spatial rendering processing on the candidate audio signal to obtain a target audio signal includes: The candidate audio signal is spatially rendered using spatial parameters to obtain the target audio signal; wherein the spatial parameters are used to characterize a virtual sound transmission process in which the sound of a virtual sound source is transmitted to the ears of a listener through a virtual sound transmission environment.

3. The method according to claim 2, characterized in that The candidate audio signals include a forward candidate audio signal and a backward candidate audio signal, and the spatial rendering process is performed on the candidate audio signals by using the spatial parameters to obtain the target audio signal, including: Performing a forward spatial rendering process on the forward candidate audio signal using the spatial parameters, and performing a backward spatial rendering process on the backward candidate audio signal to obtain a forward target audio signal and a backward target audio signal; The forward target audio signal and the backward target audio signal are mixed to obtain the target audio signal.

4. The method according to claim 3, characterized in that The decorrelation processing of the initial audio signal comprises: Performing a first decorrelation process on the initial audio signal to obtain the forward candidate audio signal; A second decorrelation process is performed on the initial audio signal to obtain the backward candidate audio signal.

5. The method according to claim 4, characterized in that The initial audio signal includes a first initial sub-audio signal of a left channel and a second initial sub-audio signal of a right channel, and performing a first decorrelation process on the initial audio signal to obtain the forward candidate audio signal includes: performing random phase processing on one of the first initial sub audio signal and the second initial sub audio signal to obtain a first candidate sub audio signal, and using the other of the first initial sub audio signal and the second initial sub audio signal as a second candidate sub audio signal; The forward candidate audio signal is obtained based on the first candidate sub audio signal and the second candidate sub audio signal.

6. The method according to claim 4, characterized in that The performing a second decorrelation process on the initial audio signal to obtain the backward candidate audio signal includes: Performing target signal processing on the initial audio signal to obtain an intermediate audio signal, wherein the intermediate audio signal includes a first intermediate sub audio signal of a left channel and a second intermediate sub audio signal of a right channel; Performing random phase processing on one of the first middle sub audio signal and the second middle sub audio signal to obtain a third candidate sub audio signal, and using the other of the first middle sub audio signal and the second middle sub audio signal as a fourth candidate sub audio signal; The backward candidate audio signal is obtained based on the third candidate sub audio signal and the fourth candidate sub audio signal.

7. The method according to claim 6, characterized in that The initial audio signal includes a first initial sub-audio signal of a left channel and a second initial sub-audio signal of a right channel, and performing target signal processing on the initial audio signal to obtain an intermediate audio signal includes: Mixing the first initial sub audio signal and the second initial sub audio signal to obtain a mixed audio signal; The mixed audio signal is subjected to low-pass filtering to obtain the intermediate audio signal.

8. The method according to claim 3, characterized in that The spatial parameters include a front spatial parameter and a rear spatial parameter; performing a front spatial rendering process on the forward candidate audio signal using the spatial parameters, and performing a rear spatial rendering process on the rear candidate audio signal using the rear spatial parameter, comprises: Filtering the forward candidate audio signal according to the forward spatial parameter to obtain the forward target audio signal; The backward candidate audio signal is filtered according to the backward spatial parameter to obtain the backward target audio signal.

9. The method according to claim 8, characterized in that The forward azimuth space parameters include a first left channel forward azimuth space parameter, a first right channel forward azimuth space parameter, a second left channel forward azimuth space parameter, and a second right channel forward azimuth space parameter; the forward candidate audio signal is filtered according to the forward azimuth space parameters to obtain the forward target audio signal, including: Performing filtering processing on a candidate sub-audio signal in the forward candidate audio signal according to the first left channel front azimuth spatial parameter and the first right channel front azimuth spatial parameter, respectively, to obtain a first left front audio signal and a first right front audio signal; According to the second left channel front azimuth spatial parameter and the second right channel front azimuth spatial parameter, another candidate sub-audio signal in the forward candidate audio signal is filtered to obtain a second left front audio signal and a second right front audio signal; The first left front audio signal, the first right front audio signal, the second left front audio signal, and the second right front audio signal are used as the front target audio signals.

10. The method according to claim 9, characterized in that The rear azimuth spatial parameters include a first left channel rear azimuth spatial parameter, a first right channel rear azimuth spatial parameter, a second left channel rear azimuth spatial parameter, and a second right channel rear azimuth spatial parameter; The filtering process of the backward candidate audio signal according to the backward spatial parameter to obtain the backward target audio signal includes: According to the first left channel rear azimuth spatial parameter and the first right channel rear azimuth spatial parameter, respectively, a candidate sub-audio signal in the rear candidate audio signal is filtered to obtain a first left rear audio signal and a first right rear audio signal; According to the second left channel rear azimuth spatial parameter and the second right channel rear azimuth spatial parameter, another candidate sub-audio signal in the rear candidate audio signal is filtered to obtain a second left rear audio signal and a second right rear audio signal; The first left rear audio signal, the first right rear audio signal, the second left rear audio signal, and the second right rear audio signal are used as the rear target audio signals.

11. The method according to claim 10, characterized in that The mixing process of the forward target audio signal and the backward target audio signal to obtain the target audio signal includes: Mixing the first left front audio signal, the second left front audio signal, the first left rear audio signal, and the second left rear audio signal to obtain a first target sub audio signal corresponding to a left channel; Mixing the first right front audio signal, the second right front audio signal, the first right rear audio signal, and the second right rear audio signal to obtain a second target sub audio signal corresponding to the right channel; The first target sub audio signal and the second sub target audio signal are used as the target audio signals.

12. The method according to claim 2, characterized in that: The method further comprises: Acquiring target angle data, where the target angle data is used to represent the angle between the virtual sound source and a preset direction; According to the target angle data, the spatial parameter corresponding to the target angle data is determined from a spatial parameter set.

13. The method according to any one of claims 1 to 12, characterized in that: The step of performing audio output according to the target audio signal comprises: A first target sub audio signal corresponding to a left channel included in the target audio signal is output through a speaker of the left channel, and a second target sub audio signal corresponding to a right channel included in the target audio signal is output through a speaker of the right channel.

14. The method according to any one of claims 1 to 12, characterized in that: The step of performing audio output according to the target audio signal comprises: Using a reverberation model, filtering a first target sub-audio signal corresponding to a left channel and a second target sub-audio signal corresponding to a right channel included in the target audio signal respectively to obtain a first enhanced target audio signal corresponding to the left channel and a second enhanced target audio signal corresponding to the right channel; The first enhanced target audio signal is output through a speaker of a left channel, and the second enhanced target audio signal is output through a speaker of a right channel.

15. The method according to claim 14, characterized in that The reverberation model is composed of a plurality of all-pass filters.

16. The method according to claim 15, characterized in that The reverberation model is specifically composed of a plurality of the all-pass filters connected in series and / or in parallel.

17. The method according to claim 15, characterized in that The all-pass filter includes a delay module, and the delay module is used to perform different delay processing on the first target sub-audio signal and the second target sub-audio signal.

18. The method according to claim 17, characterized in that The delay duration of the delay module is determined according to the time sequence of the target audio signal.

19. The method according to claim 17, characterized in that The all-pass filter further includes a gain module, and the gain module is used to perform gain adjustment processing with different intensities on the first target sub-audio signal and the second target sub-audio signal.

20. The method according to any one of claims 1 to 12, characterized in that: Before the decorrelation processing is performed on the initial audio signal, the method further comprises: Delay processing is performed on one of the sub-audio signals of different channels in the initial audio signal.

21. An audio processing device, characterized in that: The device comprises: a processing module, configured to perform decorrelation processing on the initial audio signal to reduce the correlation between sub-audio signals of different channels in the initial audio signal, and obtain a candidate audio signal; A rendering module, used to perform spatial rendering processing on the candidate audio signal to obtain a target audio signal; An output module is used to perform audio output according to the target audio signal.

22. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 20 are implemented.

23. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 20 are implemented.

24. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 20 are implemented.