Scene audio decoding method and electronic device

By acquiring and encoding the high-order energy gain and target virtual speaker attribute information of the scene audio signal and generating code streams, the problem of low accuracy in HOA signal reconstruction is solved, and the audio quality and encoding efficiency of three-dimensional audio playback are improved.

WO2024245018A1PCT designated stage expired Publication Date: 2024-12-05HUAWEI TECH CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2024/094110
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-27
Filing Date
2024-05-20
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

The prior art has the problem of low accuracy in reconstruction of HOA signals, resulting in low audio quality in three-dimensional audio playback, high encoding complexity and high code rate.

Method used

By obtaining the high-order energy gain of the scene audio signal, encode it and encode it together with the attribute information of the target virtual speaker to generate a code stream. The decoding end reconstructs the scene audio signal based on the code stream, and adjusts the high-order channel energy using the attenuation factor to improve the Audio quality.

Benefits of technology

It improves the reconstruction audio quality of scene audio signals, reduces encoding complexity and code rate, avoids the introduction of errors, is highly stable, and can provide higher audio quality at the same code rate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024094110_05122024_PF_FP_ABST
    Figure CN2024094110_05122024_PF_FP_ABST
Patent Text Reader

Abstract

Provided in the present application are a scene audio decoding method and an electronic device. The decoding method comprises: receiving a first code stream; decoding the first code stream to obtain a first reconstructed signal, attribute information of a target virtual loudspeaker and a high-order energy gain coding result, the first reconstructed signal being a reconstructed signal of a first audio signal of a scene audio signal; on the basis of the attribute information of the target virtual loudspeaker and the first audio signal, generating a virtual loudspeaker signal corresponding to the target virtual loudspeaker; on the basis of the attribute information of the target virtual loudspeaker and the virtual loudspeaker signal, reconstructing a first reconstructed scene audio signal; according to a frequency band sequence number of the reconstructed signal of the first reconstructed scene audio signal and / or the order of the first reconstructed scene audio signal, determining an attenuation factor; and, according to the high-order energy gain coding result and the attenuation factor, adjusting the first reconstructed scene audio signal to obtain a reconstructed scene audio signal. Compared with the prior art, the present application has a lower decoding rate while achieving the same quality. In addition, the present application can improve the quality of decoding audio signals.
Need to check novelty before this filing date? Find Prior Art

Description

Scene audio decoding method and electronic device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on May 27, 2023, with application number 202310613965.7 and invention name “Scene audio decoding method and electronic device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of audio decoding, and in particular to a scene audio decoding method and electronic device. Background Art

[0003] 3D audio technology uses computers and signal processing to acquire, process, transmit, and render real-world sound events and three-dimensional sound field information. 3D audio imparts a strong sense of space, envelopment, and immersion, creating an extraordinary auditory experience of "being there." High-order ambisonics (HOA) technology, in particular, is gaining widespread attention and research due to its independence from speaker layout during recording, encoding, and playback, as well as the rotatable playback capabilities of HOA format data. This allows for greater flexibility in 3D audio playback.

[0004] For an N-order HOA signal, the corresponding number of channels is (N+1) 2 As the HOA order increases, the information in the HOA signal used to record more detailed sound scenes also increases; however, the amount of data in the HOA signal also increases, which makes transmission and storage difficult. Therefore, HOA signal encoding and decoding are necessary. However, existing technologies for reconstructing HOA signals have low accuracy.

[0005] Summary of the Invention

[0006] The present application provides a scene audio encoding and decoding method and an electronic device.

[0007] In a first aspect, an embodiment of the present application provides a scene audio encoding method, the method comprising: obtaining a scene audio signal to be encoded, the scene audio signal comprising audio signals of C1 channels, where C1 is a positive integer; obtaining attribute information of a target virtual speaker corresponding to the scene audio signal; obtaining a high-order energy gain of the scene audio signal; encoding the high-order energy gain to obtain a high-order energy gain encoding result; encoding a first audio signal in the scene audio signal, the attribute information of the target virtual speaker, and the high-order energy gain encoding result to obtain a first code stream; wherein the first audio signal is an audio signal of K channels in the scene audio signal, where K is a positive integer less than or equal to C1.

[0008] In one possible embodiment, the scene audio signal is an N1-order high-order ambisonic reverberation HOA signal, the N1-order HOA signal includes a second audio signal, the second audio signal is an audio signal in the N1-order HOA signal other than the first audio signal, and C1 is equal to the square of (N1+1); obtaining the high-order energy gain of the scene audio signal includes: obtaining the high-order energy gain according to feature information of the second audio signal and feature information of the first audio signal.

[0009] Exemplarily, the N1-order HOA signal includes the second audio signal, which can be understood as the N1-order HOA signal only including the second audio signal.

[0010] Exemplarily, the N1-order HOA signal includes the second audio signal, which can be understood as the N1-order HOA signal including the second audio signal and other audio signals.

[0011] Exemplarily, the first audio signal may be referred to as a low-order portion of the scene audio signal, and the second audio signal may be referred to as a high-order portion of the scene audio signal. That is, a portion of the low-order portion of the scene audio signal and the high-order portion of the scene audio signal may be encoded.

[0012] It should be understood that, compared with an N1-order HOA signal including the second audio signal, when the N1-order HOA signal includes only the first audio signal, the number of channels of the encoded N1-order HOA signal is smaller and the corresponding bit rate is lower.

[0013] In one possible manner, obtaining the high-order energy gain based on the feature information of the second audio signal and the feature information of the first audio signal includes: obtaining the energy gain of the first audio signal and the energy gain of the second audio signal; and obtaining the high-order energy gain based on the energy gain of the first audio signal and the energy gain of the second audio signal.

[0014] In one possible manner, obtaining the high-order energy gain according to the energy gain of the first audio signal and the energy gain of the second audio signal includes obtaining the high-order energy gain Gain'(i, b) by:

[0015] Gain'(i,b)=10*log10(E(i,b) / E(1,b));

[0016] Wherein, log10 represents the logarithmic function log, * represents a multiplication operation, E(1, b) is the channel energy of the b-th frequency band of the first audio signal, E(i, b) is the i-th channel energy of the b-th frequency band of the second audio signal, i is the number of the i-th channel of the second audio signal, and b is the frequency band sequence number of the second audio signal.

[0017] In one possible manner, encoding the high-order energy gain to obtain a high-order energy gain encoding result includes: quantizing the high-order energy gain to obtain a quantized high-order energy gain; and entropy encoding the quantized high-order energy gain to obtain the high-order energy gain encoding result.

[0018] It should be noted that the position of the target virtual speaker matches the position of the sound source in the scene audio signal; based on the attribute information of the target virtual speaker and the first audio signal in the scene audio signal, a virtual speaker signal corresponding to the target virtual speaker can be generated; and based on the virtual speaker signal and the high-order energy gain encoding result, the scene audio signal can be reconstructed. Therefore, the encoder encodes the first audio signal in the scene audio signal, the attribute information of the target virtual speaker, and the high-order energy gain encoding result together and sends them to the decoder. The decoder can reconstruct the scene audio signal based on the first reconstructed signal (i.e., the reconstructed signal of the first audio signal in the scene audio signal), the attribute information of the target virtual speaker, and the high-order energy gain encoding result obtained by decoding.

[0019] Compared with other methods of reconstructing scene audio signals in the prior art, the audio quality of the scene audio signal reconstructed based on the virtual speaker signal is higher; therefore, when K is equal to C1, at the same bit rate, the audio quality of the scene audio signal reconstructed by the present application is higher.

[0020] When K is less than C1, compared with the prior art, the number of channels of the audio signal encoded by the present application is smaller, and the amount of data of the attribute information of the target virtual speaker is much smaller than the amount of data of the audio signal of one channel; therefore, while achieving the same quality, the encoding bit rate of the present application is lower.

[0021] In addition, the existing technology converts the scene audio signal into a virtual speaker signal and a residual signal and then encodes it, while the encoding end of the present application directly encodes the first audio signal in the scene audio signal without calculating the virtual speaker signal and the residual signal, and the encoding complexity of the encoding end is lower.

[0022] Exemplarily, the scene audio signal involved in the embodiments of the present application may refer to a signal used to describe the sound field; wherein the scene audio signal may include: an HOA signal (wherein the HOA signal may include a three-dimensional HOA signal and a two-dimensional HOA signal (also referred to as a planar HOA signal)) and a three-dimensional audio signal; the three-dimensional audio signal may refer to other audio signals in the scene audio signal except the HOA signal.

[0023] In one possible embodiment, when N1 is equal to 1, K may be equal to C1; when N1 is greater than 1, K may be less than C1. It should be understood that when N1 is equal to 1, K may also be less than C1.

[0024] Exemplarily, the process of encoding the attribute information of the first audio signal and the target virtual speaker in the scene audio signal may include operations such as downmixing, transformation, quantization, and entropy coding, which are not limited in this application.

[0025] Exemplarily, the first code stream may include encoded data of a first audio signal in a scene audio signal and encoded data of attribute information of a target virtual speaker.

[0026] In one possible approach, a target virtual speaker may be selected from multiple candidate virtual speakers based on the scene audio signal, and then attribute information of the target virtual speaker may be determined. For example, the virtual speakers (including the candidate virtual speakers and the target virtual speakers) are virtual speakers, not real speakers.

[0027] For example, multiple candidate virtual speakers may be evenly distributed on a sphere, and the number of target virtual speakers may be one or more.

[0028] In one possible approach, a preset target virtual speaker may be obtained, and then the attribute information of the target virtual speaker may be determined.

[0029] It should be understood that the present application does not limit the manner of determining the target virtual speaker.

[0030] In a second aspect, an embodiment of the present application provides a scene audio decoding method, which includes: receiving a first bit stream; decoding the first bit stream to obtain a first reconstructed signal, attribute information of a target virtual speaker, and a high-order energy gain coding result, wherein the first reconstructed signal is a reconstructed signal of a first audio signal in a scene audio signal, the scene audio signal includes C1 channels of audio signals, the first audio signal is K channels of audio signals in the scene audio signal, C1 is a positive integer, and K is a positive integer less than or equal to C1; based on the attribute information of the target virtual speaker and the first audio signal, generating a virtual speaker signal corresponding to the target virtual speaker; reconstructing based on the attribute information of the target virtual speaker and the virtual speaker signal to obtain a first reconstructed scene audio signal; the first reconstructed scene audio signal includes C2 channels of audio signals, where C2 is a positive integer; determining an attenuation factor according to a frequency band number of the reconstructed signal in the first reconstructed scene audio signal and / or an order of the first reconstructed scene audio signal; and adjusting the first reconstructed scene audio signal according to the high-order energy gain coding result and the attenuation factor to obtain a reconstructed scene audio signal.

[0031] In one possible embodiment, the scene audio signal is an N1-order high-order ambisonic reverberation HOA signal, the N1-order HOA signal includes a second audio signal, the second audio signal is an audio signal in the N1-order HOA signal other than the first audio signal, and C1 is equal to the square of (N1+1); and / or, the first reconstructed scene audio signal is an N2-order HOA signal, the N2-order HOA signal includes a third audio signal, the third audio signal is a reconstructed signal in the N2-order HOA signal corresponding to each channel of the second audio signal, and C2 is equal to the square of (N2+1).

[0032] In one possible embodiment, adjusting the first reconstructed scene audio signal according to the high-order energy gain encoding result and the attenuation factor to obtain a reconstructed scene audio signal includes: entropy decoding the high-order energy gain encoding result to obtain a high-order energy gain after entropy decoding; inverse quantizing the high-order energy gain after entropy decoding to obtain a high-order energy gain; adjusting the high-order energy gain according to the feature information of the second audio signal and the feature information of the first audio signal to obtain an adjusted decoded high-order energy gain; adjusting the third audio signal in the N2-order HOA signal according to the adjusted decoded high-order energy gain and the attenuation factor to obtain an adjusted third audio signal, wherein the adjusted third audio signal belongs to the reconstructed scene audio signal. In the above scheme, after obtaining the adjusted decoded high-order energy gain, the gain of the third audio signal of the current frame can be weighted, and the gain is attenuated with the frequency band number of the third audio signal and / or the order of the N2-order HOA signal. The attenuation factor can be first obtained according to the frequency band number of the third audio signal and / or the order of the N2-order HOA signal. For example, the attenuation factor can be attenuated by two factors: the frequency band and the Ambisonic order. Then, the adjusted decoded high-order energy gain and the obtained attenuation factor are applied to the high-order channel of the third audio signal reconstructed in the current frame, so that the high-order channel energy is more uniform and smooth, thereby improving the quality of the reconstructed audio signal.

[0033] In one possible manner, the adjusting of the high-order energy gain according to the characteristic information of the second audio signal and the characteristic information of the first audio signal includes: obtaining the high-order energy of the second audio signal according to the channel energy of the first audio signal and the high-order energy gain; obtaining a decoding energy proportional factor according to the channel energy of the third audio signal and the high-order energy of the second audio signal; obtaining the decoded high-order energy gain of the third audio signal according to the channel energy of the third audio signal and the channel energy of the first audio signal; and adjusting the decoded high-order energy gain of the third audio signal according to the decoding energy proportional factor to obtain the adjusted decoded high-order energy gain. In the above scheme, in order to make the energy of the high-order channel more uniform and smooth, the decoded high-order energy gain of the third audio signal is adjusted using the decoding energy proportional factor, and the adjusted decoded high-order energy gain is determined. After adjustment using the decoding energy proportional factor, the energy of the high-order channel is more uniform and smooth, and the quality of the reconstructed audio signal is better.

[0034] In one possible approach, determining the attenuation factor based on the frequency band number of the reconstructed signal in the first reconstructed scene audio signal and / or the order of the first reconstructed scene audio signal includes obtaining the attenuation factor based on the frequency band number of the third audio signal and / or the order of the N2-order HOA signal. In the above scheme, the decoding end can obtain the attenuation factor based on the frequency band number of the third audio signal, or the decoding end can obtain the attenuation factor based on the order of the N2-order HOA signal, where the order of the N2-order HOA signal can specifically be an Ambisonic order, or the decoding end can obtain the attenuation factor based on the frequency band number and the order of the N2-order HOA signal. This attenuation factor can be called a dual attenuation factor.

[0035] In one possible embodiment, after adjusting the third audio signal in the N2-order HOA signal according to the high-order energy gain encoding result and the attenuation factor, the method further includes: obtaining the channel energy of the fourth audio signal corresponding to the adjusted third audio signal, the third audio signal including the audio signal of the current frame, and the fourth audio signal including the audio signal of the previous frame of the current frame; and adjusting the adjusted third audio signal again according to the channel energy of the fourth audio signal. In the above scheme, the decoding end can also use the previous frame of the third audio signal to adjust the adjusted third audio signal of the current frame again to improve the quality of the reconstructed audio signal.

[0036] In one possible embodiment, the adjusted third audio signal is further adjusted based on the channel energy of the fourth audio signal, including: obtaining the channel energy average value of the fourth audio signal and the channel energy of the adjusted third audio signal; obtaining an energy averaging threshold based on the channel energy average value of the fourth audio signal and the channel energy of the adjusted third audio signal; performing a weighted average calculation on the channel energy average value of the fourth audio signal and the channel energy of the adjusted third audio signal based on the energy averaging threshold to obtain a target energy; obtaining an energy smoothing factor based on the target energy and the channel energy of the adjusted third audio signal; and adjusting the third audio signal based on the energy smoothing factor. In the above scheme, the adjusted third audio signal is further adjusted by using the energy smoothing factor q(i, b) to further improve the decoding quality of the third audio signal.

[0037] In one possible manner, obtaining the attenuation factor according to the frequency band number of the third audio signal and / or the order of the N2-order HOA signal includes:

[0038] The attenuation factor g'(i, b) is obtained as follows:

[0039] Wherein, i is the number of the i-th channel of the third audio signal, b is the frequency band number of the third audio signal, α represents the number of the target virtual speakers, β represents the number of mapping channels of the N2-order HOA signal, and β is equal to (M+2) 2 , M is the order of the N2-order HOA signal, γ represents the order corresponding to the channel number i of the third audio signal, and × represents a multiplication operation.

[0040] In the above scheme, by the calculation method of the above attenuation factor g'(i, b), b is the frequency band number of the third audio signal, α represents the number of target virtual speakers, β represents the number of mapping channels of the N2-order HOA signal, and β is equal to (M+2) 2 , M is the order of the N2-order HOA signal, and γ represents the order corresponding to the channel number i of the third audio signal. The above parameters can be used to accurately calculate the attenuation factor. By adjusting the parameters, the attenuation factor changes with three factors: the number of speakers, the number of HOA signal mapping channels, and the HOA order. When the attenuation factor and the adjusted decoded high-order energy gain are used to adjust the third audio signal, the quality of the reconstructed audio signal is improved.

[0041] In one possible embodiment, the method further includes: when b≤d, updating α to α2, α2=0.375×α; d is a preset first threshold; when b>d, updating α to α3, α3=0.5×α.

[0042] In a possible manner, the method further includes: updating the α to α4, where α4=α+α×(1-1.25×b) / (bands+2); wherein bands represents the number of frequency bands of the third audio signal.

[0043] In the above scheme, when the frequency band b is less than the first threshold, that is, the bth frequency band can represent the low frequency band, then the attenuation coefficient is set to a smaller value, for example, the attenuation coefficient is equal to 0.375. When the frequency band b is greater than the threshold, that is, the bth frequency band can represent the high frequency band, then the attenuation coefficient is set to a larger value, for example, the attenuation coefficient is equal to 0.5. The above attenuation coefficient of 0.375 or 0.5 is only a possible example implementation method. For example, the above 0.375 can also be replaced by 0.38 or 0.37, and the above 0.5 can also be replaced by 0.55 or 0.6. The specific value of the attenuation coefficient needs to be determined in combination with the application scenario, and is not limited here. In the above scheme, the value of α can be flexibly adjusted according to the value of the frequency band b, so that the attenuation factor has a more significant attenuation effect as the frequency band increases, thereby making the reconstructed audio signal more in line with the auditory characteristics of the human ear.

[0044] In one possible manner, the method further includes: when b≤d, updating β to β2, β2=(1+w)×β; d is a preset first threshold, and w is a preset adjustment ratio threshold.

[0045] In the above scheme, when b≤the first threshold, the value of α can be flexibly adjusted according to the value of the frequency band b in the above scheme, so that the attenuation effect of the attenuation factor becomes more significant as the frequency band becomes higher, thereby making the reconstructed audio signal more consistent with the auditory characteristics of the human ear.

[0046] In one possible approach, the method further includes updating w to w2, where w2 = w + α × 0.05. In the above scheme, the value of w can be updated based on α, so that the weight w in the attenuation factor is related to the parameter α. As α increases, w also increases, achieving a more significant attenuation effect of the attenuation factor as the frequency band increases, thereby making the reconstructed audio signal more consistent with the auditory characteristics of the human ear.

[0047] In one possible approach, when the value of i is 0, 1, 2, or 3, the value of γ is 1; when the value of i is 4, 5, 6, 7, or 8, the value of γ is 2; and when the value of i is 9, 10, 11, 12, 13, 14, or 15, the value of γ is 3. The above values ​​of γ indicate that the attenuation factor is determined by the value of γ. γ is a piecewise function related to the HOA order of the channel. As the HOA order of i increases, the value of γ increases, but does not exceed the maximum HOA order. This attenuation factor is used to improve the quality of the reconstructed audio signal when adjusting the third audio signal.

[0048] In one possible embodiment, adjusting the first reconstructed scene audio signal based on the high-order energy gain encoding result and the attenuation factor includes: entropy decoding the high-order energy gain encoding result to obtain a high-order energy gain after entropy decoding; dequantizing the high-order energy gain after entropy decoding to obtain a high-order energy gain; obtaining the high-order energy of the second audio signal based on the channel energy of the first audio signal and the high-order energy gain; obtaining a decoding energy scale factor based on the channel energy of the third audio signal and the high-order energy of the second audio signal; determining the dispersion factor of the first reconstructed scene audio signal based on the first bitstream; linearly weighting the attenuation factor based on the dispersion factor to obtain a weighted attenuation factor; and adjusting the third audio signal in the N2-order HOA signal based on the weighted attenuation factor and the decoding energy scale factor to obtain an adjusted third audio signal. In the above scheme, the dispersion factor can also be used to linearly weight the attenuation factor, and the weighted attenuation factor is used to adjust the third audio signal. A weighting method is used to balance the proportions of the diffuse component and the directional component in the attenuation factor. The diffuseness factor can be used to measure the energy ratio of the non-directional component in the HOA signal to be encoded and to adjust the energy of each channel of the reconstructed HOA signal, so that the reconstructed HOA signal after energy adjustment is closer to the energy of the HOA signal to be encoded.

[0049] In one possible manner, linearly weighting the attenuation factor according to the diffusivity factor to obtain the weighted attenuation factor includes obtaining the weighted attenuation factor gd(i,b) by at least one of the following methods:

[0050] gd(i,b)=w×diffusion(b)+(1-w)×g'(i,b); where w is a preset adjustment ratio threshold, × represents a multiplication operation, diffusion(b) represents the diffusion factor of the bth frequency band of the third audio signal, and g'(i,b) represents the attenuation factor of the bth frequency band of the third audio signal;

[0051] or,

[0052] When the attenuation factor is a full-band signal, gd(i,b)=w×mean(diffusion)+(1-w)×g'(i,b), where mean(diffusion) is the average value of the diffusion factors of multiple frequency bands of the third audio signal, g'(i,b) represents the attenuation factor of the b-th frequency band of the third audio signal, and w is a preset adjustment ratio threshold;

[0053] or,

[0054] gd(i,b)=w×diffusion(b)+(1-w)×g'(i,b)+offset(i,b), where offset(i,b) represents the offset constant of the bth frequency band on the i-th channel of the third audio signal, g'(i,b) represents the attenuation factor of the bth frequency band of the third audio signal, w is a preset adjustment ratio threshold, and diffusion(b) represents the diffusion factor of the bth frequency band of the third audio signal;

[0055] or,

[0056] gd(i,b)=w×diffusion(b)+(1-w)×g'(i,b)+direction(i,b), where direction(i,b) represents the direction parameter of the bth frequency band on the i-th channel of the third audio signal, g'(i,b) represents the attenuation factor of the bth frequency band of the third audio signal, w is a preset adjustment ratio threshold, and diffusion(b) represents the diffusion factor of the bth frequency band of the third audio signal.

[0057] In one possible manner, the third audio signal in the N2-order HOA signal is adjusted according to the weighted attenuation factor and the decoding energy scale factor to obtain an adjusted third audio signal:

[0058] The adjusted third audio signal X'(i,b) is obtained by:

[0059] X'(i,b)=X(i)×g(i,b)×gd(i,b);

[0060] Wherein, gd(i,b) represents the weighted attenuation factor, g(i,b) represents the decoding energy proportional factor, and X(i) represents the third audio signal.

[0061] In the above scheme, since the dispersion factor can be used to measure the energy ratio of the non-directional component in the HOA signal to be encoded, and the decoding energy scaling factor can be used to measure the energy ratio of the reconstructed HOA signal, the dispersion factor and the decoding energy scaling factor are used to adjust the energy of each channel of the reconstructed HOA signal, so that the reconstructed HOA signal after energy adjustment is closer to the energy of the HOA signal to be encoded.

[0062] In one possible manner, obtaining the energy average threshold according to the channel energy average of the fourth audio signal and the adjusted channel energy of the third audio signal includes:

[0063] The energy average threshold k is obtained as follows:

[0064] E_mean(i, b) represents the channel energy average of the fourth audio signal, and E'_dec(i, b) represents the energy of the adjusted third audio signal.

[0065] In one possible manner, obtaining an energy smoothing factor according to the target energy and the adjusted channel energy of the third audio signal includes:

[0066] The energy smoothing factor q(i, b) is obtained as follows:

[0067] q(i, b)=sqrt(E_target(i, b)) / sqrt(E'_dec(i, b));

[0068] Wherein, E_target(i, b) represents the target energy, and E'_dec(i, b) represents the energy of the adjusted third audio signal.

[0069] In one possible manner, adjusting the decoded high-order energy gain of the third audio signal according to the decoded energy scale factor to obtain the adjusted decoded high-order energy gain includes:

[0070] The adjusted decoding high-order energy gain Gain_dec'(i, b) is obtained by:

[0071] Gain_dec'(i, b)=w*min(g(i, b), Gain_dec(i, b))+(1-w)*g(i, b);

[0072] Wherein, g(i, b) represents the decoding energy scaling factor, Gain_dec(i, b) represents the decoding high-order energy gain of the third audio signal, w is a preset adjustment ratio threshold, min represents a minimum value operation, and * represents a multiplication operation.

[0073] Compared with other methods of reconstructing scene audio signals in the prior art, the audio quality of the scene audio signal reconstructed based on the virtual speaker signal is higher; therefore, when K is equal to C1, at the same bit rate, the audio quality of the scene audio signal reconstructed by the present application is higher.

[0074] When K is less than C1, in the process of encoding the scene audio signal, the number of channels of the audio signal encoded by the present application is less than the number of channels of the audio signal encoded by the prior art, and the data volume of the attribute information of the target virtual speaker is much smaller than the data volume of the audio signal of one channel; therefore, under the premise of the same bit rate, the audio quality of the reconstructed scene audio signal obtained by decoding by the present application is higher.

[0075] Secondly, since the virtual speaker signal and residual information encoded and transmitted by the prior art are converted from the original audio signal (i.e., the scene audio signal to be encoded), and are not the original audio signal, errors will be introduced; while the present application encodes part of the original audio signal (i.e., the audio signals of K channels in the scene audio signal to be encoded), thereby avoiding the introduction of errors, and thus being able to improve the audio quality of the reconstructed scene audio signal obtained by decoding; and it can also avoid fluctuations in the reconstruction quality of the reconstructed scene audio signal obtained by decoding, and has high stability.

[0076] In addition, since the prior art encodes and transmits virtual speaker signals, and the data volume of virtual speaker signals is relatively large, the number of target virtual speakers selected by the prior art is subject to significant bandwidth limitations. The present application encodes and transmits the attribute information of the virtual speaker, and the data volume of the attribute information is much smaller than the data volume of the virtual speaker signal; therefore, the number of target virtual speakers selected by the present application is subject to less bandwidth limitations. The more target virtual speakers selected, the higher the quality of the reconstructed scene audio signal based on the virtual speaker signals of the target virtual speakers. Therefore, compared to the prior art, under the same bit rate, the present application can select a larger number of target virtual speakers, so that the quality of the reconstructed scene audio signal decoded by the present application is also higher.

[0077] In addition, compared with the encoding end and decoding end of the prior art, the encoding end and decoding end of the present application do not need to perform residual and superposition operations. Therefore, the comprehensive complexity of the encoding end and decoding end of the present application is lower than the comprehensive complexity of the encoding end and decoding end of the prior art.

[0078] It should be understood that when the encoder performs lossy compression on the first audio signal in the scene audio signal, the first reconstructed signal decoded by the decoder differs from the first audio signal encoded by the encoder. When the encoder performs lossless compression on the first audio signal, the first reconstructed signal decoded by the decoder is the same as the first audio signal encoded by the encoder.

[0079] It should be understood that when the encoder performs lossy compression on the target virtual speaker attribute information, the attribute information decoded by the decoder differs from the attribute information encoded by the encoder. When the encoder performs lossless compression on the virtual speaker attribute information, the attribute information decoded by the decoder is the same as the attribute information encoded by the encoder. However, this application does not distinguish between the attribute information encoded by the encoder and the attribute information decoded by the decoder in terms of name.

[0080] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0081] In a third aspect, an embodiment of the present application provides a method for generating a bitstream, which can generate a bitstream according to the first aspect and any one of the implementation methods of the first aspect.

[0082] The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0083] In a fourth aspect, an embodiment of the present application provides a scene audio encoding device, the device comprising:

[0084] An acquisition module, configured to acquire a scene audio signal to be encoded, wherein the scene audio signal includes audio signals of C1 channels, where C1 is a positive integer;

[0085] The acquisition module is further configured to acquire attribute information of a target virtual speaker corresponding to the scene audio signal;

[0086] The acquisition module is further configured to acquire a high-order energy gain of the scene audio signal;

[0087] an encoding module, configured to encode the high-order energy gain to obtain a high-order energy gain encoding result;

[0088] The encoding module is further used to encode the first audio signal in the scene audio signal, the attribute information of the target virtual speaker, and the high-order energy gain encoding result to obtain a first code stream; wherein the first audio signal is the audio signal of K channels in the scene audio signal, and K is a positive integer less than or equal to C1.

[0089] The scene audio encoding device of the fourth aspect can execute the steps of the first aspect and any one of the implementation methods of the first aspect, which will not be repeated here.

[0090] The fourth aspect and any implementation of the fourth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourth aspect and any implementation of the fourth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0091] In a fifth aspect, an embodiment of the present application provides a scene audio decoding device, the device comprising:

[0092] A code stream receiving module, configured to receive a first code stream;

[0093] a decoding module, configured to decode the first bitstream to obtain a first reconstructed signal, attribute information of a target virtual speaker, and a high-order energy gain encoding result, wherein the first reconstructed signal is a reconstructed signal of a first audio signal in a scene audio signal, the scene audio signal includes audio signals of C1 channels, the first audio signal is audio signals of K channels in the scene audio signal, where C1 is a positive integer and K is a positive integer less than or equal to C1;

[0094] a virtual speaker signal generating module, configured to generate a virtual speaker signal corresponding to the target virtual speaker based on the attribute information of the target virtual speaker and the first audio signal;

[0095] a scene audio signal reconstruction module, configured to reconstruct based on the attribute information of the target virtual speaker and the virtual speaker signal to obtain a first reconstructed scene audio signal; the first reconstructed scene audio signal includes audio signals of C2 channels, where C2 is a positive integer;

[0096] an attenuation factor determining module, configured to determine an attenuation factor according to a frequency band number of a reconstructed signal in the first reconstructed scene audio signal and / or an order of the first reconstructed scene audio signal;

[0097] The scene audio signal adjustment module is configured to adjust the first reconstructed scene audio signal according to the high-order energy gain encoding result and the attenuation factor to obtain a reconstructed scene audio signal.

[0098] The scene audio decoding device of the fifth aspect can execute the steps of the second aspect and any one of the implementation methods of the second aspect, which will not be repeated here.

[0099] The fifth aspect and any implementation of the fifth aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can be referred to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, and will not be repeated here.

[0100] In a sixth aspect, an embodiment of the present application provides an electronic device comprising: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, which, when executed by the processor, enables the electronic device to execute the scene audio encoding method of the first aspect or any possible implementation of the first aspect.

[0101] The sixth aspect and any implementation of the sixth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the sixth aspect and any implementation of the sixth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0102] In the seventh aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, which, when executed by the processor, enables the electronic device to execute the scene audio decoding method in the second aspect or any possible implementation of the second aspect.

[0103] The seventh aspect and any implementation of the seventh aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the seventh aspect and any implementation of the seventh aspect can be referred to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, and will not be repeated here.

[0104] In an eighth aspect, an embodiment of the present application provides a chip comprising one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of an electronic device and send a signal to the processor, the signal comprising a computer instruction stored in the memory; when the processor executes the computer instruction, the electronic device executes the scene audio encoding method of the first aspect or any possible implementation of the first aspect.

[0105] The eighth aspect and any implementation of the eighth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the eighth aspect and any implementation of the eighth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0106] In the ninth aspect, an embodiment of the present application provides a chip comprising one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of an electronic device and send a signal to the processor, the signal including a computer instruction stored in the memory; when the processor executes the computer instruction, the electronic device executes the scene audio decoding method in the second aspect or any possible implementation of the second aspect.

[0107] The ninth aspect and any implementation of the ninth aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the ninth aspect and any implementation of the ninth aspect can be referred to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, and will not be repeated here.

[0108] In the tenth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computer or processor, it enables the computer or processor to execute the scene audio encoding method in the first aspect or any possible implementation of the first aspect.

[0109] The tenth aspect and any implementation of the tenth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the tenth aspect and any implementation of the tenth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0110] In the eleventh aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computer or processor, it enables the computer or processor to execute the scene audio decoding method in the second aspect or any possible implementation of the second aspect.

[0111] The eleventh aspect and any implementation of the eleventh aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the eleventh aspect and any implementation of the eleventh aspect can be referred to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, and will not be repeated here.

[0112] In the twelfth aspect, an embodiment of the present application provides a computer program product, which includes a software program. When the software program is executed by a computer or a processor, the computer or the processor executes the scene audio encoding method in the first aspect or any possible implementation of the first aspect.

[0113] The twelfth aspect and any implementation of the twelfth aspect respectively correspond to the first aspect and any implementation of the first aspect. The technical effects corresponding to the twelfth aspect and any implementation of the twelfth aspect can be referred to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, and will not be repeated here.

[0114] In a thirteenth aspect, an embodiment of the present application provides a computer program product, which includes a software program. When the software program is executed by a computer or a processor, the computer or the processor executes the scene audio decoding method in the second aspect or any possible implementation of the second aspect.

[0115] The thirteenth aspect and any implementation of the thirteenth aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the thirteenth aspect and any implementation of the thirteenth aspect can be referred to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, and will not be repeated here.

[0116] In a fourteenth aspect, an embodiment of the present application provides a device for storing a code stream, the device comprising: a receiver and at least one storage medium, the receiver being used to receive the code stream; the at least one storage medium being used to store the code stream; the code stream being generated according to the first aspect and any one of the implementation methods of the first aspect.

[0117] The fourteenth aspect and any implementation of the fourteenth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourteenth aspect and any implementation of the fourteenth aspect can be referred to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, and will not be repeated here.

[0118] In a fifteenth aspect, an embodiment of the present application provides a device for transmitting a code stream, the device comprising: a transmitter and at least one storage medium, the at least one storage medium being used to store the code stream, the code stream being generated according to the first aspect and any one of the implementation methods of the first aspect; the transmitter being used to obtain the code stream from the storage medium and send the code stream to the end-side device through the transmission medium.

[0119] The fifteenth aspect and any implementation of the fifteenth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fifteenth aspect and any implementation of the fifteenth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0120] In a sixteenth aspect, an embodiment of the present application provides a system for distributing code streams, the system comprising: at least one storage medium for storing at least one code stream, the at least one code stream being generated according to the first aspect and any one of the implementation methods of the first aspect; a streaming media device for obtaining a target code stream from the at least one storage medium and sending the target code stream to an end-side device, wherein the streaming media device comprises a content server or a content distribution server.

[0121] The sixteenth aspect and any implementation of the sixteenth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the sixteenth aspect and any implementation of the sixteenth aspect can be referred to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0122] FIG1a is a schematic diagram of an exemplary application scenario;

[0123] FIG1b is a schematic diagram of an exemplary application scenario;

[0124] FIG2a is a schematic diagram illustrating an exemplary encoding process;

[0125] FIG2 b is a schematic diagram showing an exemplary distribution of candidate virtual speakers;

[0126] FIG3 is a schematic diagram illustrating an exemplary decoding process;

[0127] FIG4 is a schematic diagram illustrating an exemplary encoding process;

[0128] FIG5a is a schematic diagram illustrating an exemplary decoding process;

[0129] FIG5b is a schematic diagram illustrating another decoding process;

[0130] FIG6a is a schematic structural diagram of an exemplary encoding end;

[0131] FIG6 b is a schematic structural diagram of an exemplary decoding end;

[0132] FIG7 is a schematic structural diagram illustrating an exemplary scene audio encoding device;

[0133] FIG8 is a schematic structural diagram illustrating an exemplary scene audio decoding device;

[0134] FIG9 is a schematic structural diagram of an exemplary device. DETAILED DESCRIPTION

[0135] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0136] The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0137] In the description and claims of the embodiments of this application, the terms "first" and "second" are used to distinguish different objects, rather than to describe a specific order of objects. For example, the terms "first target object" and "second target object" are used to distinguish different objects, rather than to describe a specific order of objects.

[0138] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0139] In the description of the embodiments of this application, unless otherwise specified, "multiple" means two or more. For example, "multiple processing units" means two or more processing units; "multiple systems" means two or more systems.

[0140] In order to make the description of the following embodiments clear and concise, a brief introduction to the relevant technology is first given.

[0141] Sound is a continuous wave produced by the vibration of an object. The object that vibrates and emits sound waves is called the sound source. As sound waves propagate through a medium (such as air, solid, or liquid), they are detected by the human or animal auditory system.

[0142] The characteristics of sound waves include pitch, intensity, and timbre. Pitch indicates the highness of a sound. Intensity indicates how loud a sound is. Intensity can also be called loudness or volume. The unit of intensity is decibel (dB). Timbre is also called timbre quality.

[0143] The frequency of a sound wave determines its pitch. The higher the frequency, the higher the pitch. The frequency is the number of times an object vibrates in one second, measured in hertz (Hz). The human ear can detect sounds with frequencies between 20Hz and 20,000Hz.

[0144] The amplitude of a sound wave determines its intensity. The greater the amplitude, the greater the intensity. The closer to the sound source, the greater the intensity.

[0145] The waveform of the sound wave determines the timbre, which includes square wave, sawtooth wave, sine wave and pulse wave.

[0146] Based on the characteristics of sound waves, sounds can be divided into regular sounds and irregular sounds. Irregular sounds are sounds produced by the irregular vibration of a sound source. For example, irregular sounds can disrupt people's work, study, and rest. Regular sounds are sounds produced by the regular vibration of a sound source. Regular sounds include speech and music. When represented electrically, regular sounds are analog signals that continuously vary in the time-frequency domain. These analog signals are called audio signals. Audio signals are information carriers that carry speech, music, and sound effects.

[0147] Since human hearing has the ability to distinguish the positional distribution of sound sources in space, when listeners hear sounds in space, in addition to being able to feel the pitch, intensity and timbre of the sounds, they can also feel the direction of the sounds.

[0148] As people's attention to and demand for higher quality in their auditory systems grows, three-dimensional audio technology has emerged to enhance the sense of depth, presence, and spatiality. This allows listeners to perceive not only sounds from sources in front, behind, left, and right, but also a sense of being enveloped by the spatial sound field (or "sound field") generated by these sources, with the sound spreading outward in all directions. This creates an immersive sound experience that transports listeners to places like a movie theater or concert hall.

[0149] The scene audio signal involved in the embodiments of the present application may refer to a signal used to describe a sound field. The scene audio signal may include: an HOA signal (wherein an HOA signal may include a three-dimensional HOA signal and a two-dimensional HOA signal (also referred to as a planar HOA signal)) and a three-dimensional audio signal. The three-dimensional audio signal may refer to other audio signals in the scene audio signal in addition to the HOA signal. The following description uses the HOA signal as an example.

[0150] As we all know, when sound waves propagate in an ideal medium, the wave number is k = w / c and the angular frequency is w = 2πf, where f is the sound wave frequency and c is the speed of sound. The sound pressure p satisfies the formula (1, b), is the Laplace operator.

[0151] Assume that the spatial system outside the human ear is a sphere, with the listener at the center. Sounds from outside the sphere are projected onto the sphere, filtering out sounds outside the sphere. Assuming that the sound sources are distributed on this sphere, the sound field generated by the sound sources on the sphere is used to fit the sound field generated by the original sound sources. Three-dimensional audio technology is a method for fitting sound fields. Specifically, equation (1, b) is solved in a spherical coordinate system. Within the passive spherical region, equation (1, b) is solved as follows: equation (2).

[0152] Among them, r represents the radius of the sphere, θ represents the horizontal angle information (or azimuth information), represents the pitch angle information (or called elevation angle information), k represents the wave number, s represents the amplitude of the ideal plane wave, and m represents the order number of the HOA signal (or called the order number of the HOA signal). represents the spherical Bessel function, which is also called the radial basis function, where the first j represents the imaginary unit, Does not change with angle. represents θ, Spherical harmonics of the direction, The spherical harmonic function representing the direction of the sound source. The HOA signal satisfies formula (3).

[0153] Substituting formula (3) into formula (2), formula (2) can be transformed into formula (4).

[0154] Among them, m is truncated to the Nth item, that is, m=N, with As an approximate description of the sound field; at this time, It can be called the HOA coefficient (which can be used to represent the Nth-order HOA signal). The sound field refers to the area in the medium where sound waves exist. N is an integer greater than or equal to 1.

[0155] The scene audio signal is an information carrier that carries the spatial position information of the sound source in the sound field and describes the sound field of the listener in space. Formula (4) shows that the sound field can be expanded on a sphere using spherical harmonics, that is, the sound field can be decomposed into the superposition of multiple plane waves. Therefore, the sound field described by the HOA signal can be expressed as the superposition of multiple plane waves, and the sound field can be reconstructed using the HOA coefficients.

[0156] The HOA signal to be encoded involved in the embodiments of the present application may refer to an N1-order HOA signal, which may be represented by an HOA coefficient or an Ambisonic (stereo reverberation) coefficient, where N1 is an integer greater than or equal to 1 (wherein, when N1 is equal to, the first-order HOA signal may be referred to as a FOA (First Order Ambisonic, first-order stereo reverberation) signal). The N1-order HOA signal includes (N1+1) 2 channels of audio signals.

[0157] Figure 1a is a schematic diagram of an exemplary application scenario, which shows the encoding and decoding scenario of a scene audio signal.

[0158] Referring to Figure 1a, illustratively, the first electronic device may include a first audio acquisition module, a first scene audio encoding module, a first channel encoding module, a first channel decoding module, a first scene audio decoding module, and a first audio playback module. It should be understood that the first electronic device may include more or fewer modules than those shown in Figure 1a, and this application does not limit this.

[0159] 1a, illustratively, the second electronic device may include a second audio acquisition module, a second scene audio encoding module, a second channel encoding module, a second channel decoding module, a second scene audio decoding module, and a second audio playback module. It should be understood that the second electronic device may include more or fewer modules than those shown in FIG1a, and this application does not limit this.

[0160] Exemplarily, the process of encoding and transmitting a scene audio signal by a first electronic device to a second electronic device, and decoding and audio playback by the second electronic device can be as follows: the first audio acquisition module can perform audio acquisition and output the scene audio signal to the first scene audio encoding module. Then, the first scene audio encoding module can encode the scene audio signal and output a code stream to the first channel encoding module. Thereafter, the first channel encoding module can channel encode the code stream and transmit the channel-encoded code stream to the second electronic device via a wireless or wired network communication device. Then, the second channel decoding module of the second electronic device can channel decode the received data to obtain a code stream and output the code stream to the second scene audio decoding module. Then, the second scene audio decoding module can decode the code stream to obtain a reconstructed scene audio signal; then, the reconstructed scene audio signal is output to the second audio playback module, and the second audio playback module performs audio playback.

[0161] It should be noted that the second audio playback module can perform post-processing (such as audio rendering (for example, the audio signal containing (N1+1) 2 The invention relates to a method for converting a reconstructed scene audio signal of a plurality of channel audio signals into an audio signal with the same number of channels as the number of speakers in the second electronic device), loudness normalization, user interaction, audio format conversion or noise removal, etc., to convert the reconstructed scene audio signal into an audio signal suitable for playback by the speakers in the second electronic device.

[0162] It should be understood that the process of the second electronic device encoding and transmitting the scene audio signal to the first electronic device, and the first electronic device decoding and audio playback is similar to the above-mentioned process of the first electronic device transmitting the scene audio signal to the second electronic device, and the second electronic device playing back the audio, and will not be repeated here.

[0163] Exemplarily, the first electronic device and the second electronic device may include, but are not limited to: personal computers, computer workstations, smart phones, tablet computers, servers, smart cameras, smart cars or other types of cellular phones, media consumption devices, wearable devices, set-top boxes, game consoles, etc.

[0164] For example, this application can be specifically applied to VR (Virtual Reality) / AR (Augmented Reality) scenarios. In one possible embodiment, the first electronic device is a server and the second electronic device is a VR / AR device. In another possible embodiment, the second electronic device is a server and the first electronic device is a VR / AR device.

[0165] Exemplarily, the first scene audio encoding module and the second scene audio encoding module may be scene audio encoders, and the first scene audio decoding module and the second scene audio decoding module may be scene audio decoders.

[0166] For example, when a first electronic device encodes a scene audio signal and a second electronic device reconstructs the scene audio signal, the first electronic device can be referred to as an encoding end and the second electronic device can be referred to as a decoding end. When a second electronic device encodes a scene audio signal and a first electronic device reconstructs the scene audio signal, the second electronic device can be referred to as an encoding end and the first electronic device can be referred to as a decoding end.

[0167] Figure 1b is a schematic diagram of an exemplary application scenario, which shows a transcoding scenario of a scene audio signal.

[0168] 1b(1), illustratively, a wireless or core network device may include: a channel decoding module, other audio decoding modules, a scene audio encoding module, and a channel encoding module. The wireless or core network device may be used for audio transcoding.

[0169] Exemplarily, the specific application scenario of Figure 1b (1) may be: when the first electronic device is not provided with a scene audio encoding module but only with other audio encoding modules; and the second electronic device is only provided with a scene audio decoding module but not with other audio decoding modules, in order to enable the second electronic device to decode and play back the scene audio signal encoded by the first electronic device using other audio encoding modules, wireless or core network equipment can be used for transcoding.

[0170] Specifically, the first electronic device uses other audio encoding modules to encode the scene audio signal to obtain a first code stream; and then performs channel encoding on the first code stream and sends it to the wireless or core network device. Then, the channel decoding module of the wireless or core network device can perform channel decoding and output the first code stream obtained by channel decoding to the other audio decoding modules. After that, the other audio decoding modules decode the first code stream to obtain a scene audio signal and output the scene audio signal to the scene audio encoding module. Then, the scene audio encoding module can encode the scene audio signal to obtain a second code stream and output the second code stream to the channel encoding module. The channel encoding module performs channel encoding on the second code stream and sends it to the second electronic device. In this way, the second electronic device can call the scene audio decoding module to decode the second code stream obtained by channel decoding to obtain a reconstructed scene audio signal; and then the reconstructed scene audio signal can be played back.

[0171] 1b(2), illustratively, a wireless or core network device may include: a channel decoding module, a scene audio decoding module, other audio encoding modules, and a channel encoding module. The wireless or core network device may be used for audio transcoding.

[0172] Exemplarily, the specific application scenario of Figure 1b (2) may be: when the first electronic device is only provided with a scene audio encoding module and no other audio encoding modules; and the second electronic device is not provided with a scene audio decoding module and only provided with other audio decoding modules, in order to enable the second electronic device to decode and play back the scene audio signal encoded by the scene audio encoding module of the first electronic device, wireless or core network equipment may be used for transcoding.

[0173] Specifically, the first electronic device uses a scene audio encoding module to encode the scene audio signal to obtain a first code stream; and then performs channel encoding on the first code stream and sends it to a wireless or core network device. Then, the channel decoding module of the wireless or core network device can perform channel decoding and output the first code stream obtained by channel decoding to the scene audio decoding module. After that, the scene audio decoding module decodes the first code stream to obtain a scene audio signal and outputs the scene audio signal to other audio encoding modules. Then, other audio encoding modules can encode the scene audio signal to obtain a second code stream and output the second code stream to the channel encoding module, which performs channel encoding on the second code stream and sends it to the second electronic device. In this way, the second electronic device can call other audio decoding modules to decode the second code stream obtained by channel decoding to obtain a reconstructed scene audio signal; the reconstructed scene audio signal can then be played back.

[0174] The following describes the encoding and decoding process of the scene audio signal.

[0175] FIG2 a is a schematic diagram illustrating an exemplary encoding process.

[0176] S201 : Acquire a scene audio signal to be encoded, where the scene audio signal includes audio signals of C1 channels, where C1 is a positive integer.

[0177] For example, when the scene audio signal is an HOA signal, the HOA signal may be an N1-order HOA signal, that is, the N1-term in the above formula (3)

[0178] For example, the N1-order HOA signal may include C1-channel audio signals, where C1=(N1+1) 2 For example, when N1=3, the N1-order HOA signal includes 16 channels of audio signals; when N1=4, the N1-order HOA signal includes 25 channels of audio signals.

[0179] S202: Acquire attribute information of a target virtual speaker corresponding to a scene audio signal.

[0180] Based on the scene audio signal, a target virtual speaker is selected from a plurality of candidate virtual speakers, and attribute information of the target virtual speaker is obtained.

[0181] S203: Obtain high-order energy gain of the scene audio signal.

[0182] Exemplarily, characteristic information of the HOA signal is obtained from the HOA signal to be encoded, and a high-order energy gain is obtained through the characteristic information of the HOA signal. The high-order energy gain can be used to indicate the energy gain of the high-order channel signal of the scene audio signal.

[0183] Exemplarily, the scene audio signal includes C1 channels of audio signals, and the first audio signal is K channels of audio signals in the scene audio signal, where K is a positive integer less than or equal to C1, and the value of K is not limited.

[0184] The scene audio signal is an N1-order HOA signal, which includes a first audio signal and a second audio signal. The second audio signal is an audio signal in the N1-order HOA signal except the first audio signal, and C1 is equal to the square of (N1+1).

[0185] In one possible approach, assuming that N1 = 3 and C1 = 10, the N1-order HOA signal includes audio signals of channels 1 to 16, the first audio signal is the audio signals of channels 1 to 10 in the N1-order HOA signal, and the second audio signal is the audio signals of channels 11 to 16 in the N1-order HOA signal.

[0186] For example, N1 = 3, C1 = 9. The N1-order HOA signal includes audio signals of channels 1 to 16, the first audio signal is the audio signals of channels 1 to 9 in the N1-order HOA signal, and the second audio signal is the audio signals of channels 10 to 16 in the N1-order HOA signal.

[0187] Exemplarily, N1 = 3, C1 = 8. The N1-order HOA signal includes audio signals of channels 1 to 16, the first audio signal is the audio signals of channels 1 to 6 and 8 and 9 in the N1-order HOA signal, and the second audio signal is the audio signals of channels 7 and 10 to 16 in the N1-order HOA signal.

[0188] In one possible implementation, obtaining a high-order energy gain of a scene audio signal includes:

[0189] A high-order energy gain is obtained according to the feature information of the second audio signal and the feature information of the first audio signal.

[0190] The scene audio signal includes a first audio signal and a second audio signal, and feature information of the second audio signal and feature information of the first audio signal are obtained respectively. The feature information corresponding to the scene audio signal includes, but is not limited to, gain information and diffusion information. A high-order energy gain of the scene audio signal can be obtained based on the feature information of the second audio signal and the feature information of the first audio signal.

[0191] Exemplarily, the gain information Gain(i, b) of the second audio signal in the scene audio signal may be calculated with reference to the following formula:

[0192] Gain(i,b)=E(i,b) / E(1,b)

[0193] Wherein, i is the number of the i-th channel contained in the second audio signal in the scene audio signal, which can also be called the channel number, b is the frequency band number of the second audio signal, E(i, b) is the i-th channel energy of the b-th frequency band of the second audio signal, and E(1, b) is the channel energy of the b-th frequency band of the first audio signal. For example, the channel of the first audio signal can specifically be the 1st channel of the N1-order HOA signal.

[0194] The following steps can be performed within a frame signal or on a subframe signal. The following steps can be performed on the full frequency band or on a subband signal.

[0195] For example, after Gain(i, b) is calculated, Gain'(i, b) is calculated as follows:

[0196] Gain'(i, b)=10*log10(Gain(i, b)).

[0197] S204: Encode the high-order energy gain to obtain a high-order energy gain encoding result.

[0198] After the encoder obtains the high-order energy gain of the scene audio signal, it can encode it to generate a high-order energy gain encoding result. The role of the high-order energy gain is to adjust the high-order channel energy at the decoder, improving the encoding and decoding quality of the HOA signal.

[0199] S205, encode attribute information and high-order energy gain encoding results of a first audio signal and a target virtual speaker in the scene audio signal to obtain a first bitstream; wherein the first audio signal is an audio signal of K channels in the scene audio signal, and K is a positive integer less than or equal to C1.

[0200] Exemplarily, the virtual speaker is a virtual speaker, not a real speaker.

[0201] Exemplarily, based on the above, the scene audio signal can be expressed by the superposition of multiple plane waves, and then the target virtual speaker used to simulate the sound source in the scene audio signal can be determined; in this way, in the subsequent decoding process, the virtual speaker signal corresponding to the target virtual speaker is used to reconstruct the scene audio signal.

[0202] In one possible approach, multiple candidate virtual speakers at different positions may be set on a sphere; then, a target virtual speaker whose position matches the sound source position in the scene audio signal may be selected from the multiple candidate virtual speakers.

[0203] Figure 2b is a schematic diagram showing an exemplary distribution of candidate virtual speakers. In Figure 2b, multiple candidate virtual speakers can be evenly distributed on a sphere, and a point on the sphere represents a candidate virtual speaker.

[0204] It should be noted that this application does not limit the number and distribution of candidate virtual speakers, which can be set according to needs, and will be explained in detail later.

[0205] Exemplarily, based on the scene audio signal, a target virtual speaker whose position corresponds to the sound source position in the scene audio signal can be selected from these multiple candidate virtual speakers; wherein, the number of target virtual speakers can be one or more, and this application does not impose any restrictions on this.

[0206] In one possible approach, a target virtual speaker may be preset.

[0207] It should be understood that the present application does not limit the manner of determining the target virtual speaker.

[0208] Exemplarily, in one possible method, during the decoding process, the scene audio signal can be reconstructed based on the virtual speaker signal; however, directly transmitting the virtual speaker signal of the target virtual speaker will increase the bit rate; and the virtual speaker signal of the target virtual speaker can be generated based on the attribute information of the target virtual speaker and the scene audio signal of some or all channels; therefore, the attribute information of the target virtual speaker and the audio signals of K channels in the scene audio signal can be obtained as the first audio signal; then the first audio signal, the attribute information of the target virtual speaker and the high-order energy gain encoding result are encoded to obtain a first code stream.

[0209] For example, downmixing, transformation, quantization, and entropy coding operations may be performed on the first audio signal and the target virtual speaker attribute information to obtain a first bitstream. Furthermore, the high-order energy gain coding result may be written into the first bitstream. In other words, the first bitstream may include coded data of the first audio signal in the scene audio signal, coded data of the target virtual speaker attribute information, and the high-order energy gain coding result.

[0210] Compared with other methods of reconstructing scene audio signals in the prior art, the scene audio signal reconstructed based on the virtual speaker signal has higher audio quality; therefore, when K is equal to C1, at the same bit rate, the scene audio signal reconstructed by the present application has higher audio quality.

[0211] When K is less than C1, in the process of encoding the scene audio signal, the number of channels of the audio signal encoded by the present application is less than the number of channels of the audio signal encoded by the prior art, and the amount of data of the attribute information of the target virtual speaker is also much smaller than the amount of data of the audio signal of one channel; therefore, under the premise of achieving the same quality, the encoding bit rate of the present application is lower.

[0212] In addition, the existing technology converts the scene audio signal into a virtual speaker signal and a residual signal and then encodes it, while the encoding end of the present application directly encodes the audio signals of some channels in the scene audio signal without calculating the virtual speaker signal and the residual signal, and the encoding complexity of the encoding end is lower.

[0213] Figure 3 is a schematic diagram illustrating an exemplary decoding process. Figure 3 is a decoding process corresponding to the encoding process of Figure 2.

[0214] S301: Receive a first code stream.

[0215] S302: Decode the first code stream to obtain a first reconstructed signal and attribute information of a target virtual speaker.

[0216] For example, the encoded data of the first audio signal in the scene audio signal included in the first bitstream can be decoded to obtain a first reconstructed signal; that is, the first reconstructed signal is a reconstructed signal of the first audio signal. Furthermore, the encoded data of the attribute information of the target virtual speaker included in the first bitstream can be decoded to obtain the attribute information of the target virtual speaker.

[0217] It should be understood that when the encoder performs lossy compression on the first audio signal in the scene audio signal, the first reconstructed signal decoded by the decoder differs from the first audio signal encoded by the encoder. When the encoder performs lossless compression on the first audio signal, the first reconstructed signal decoded by the decoder is the same as the first audio signal encoded by the encoder.

[0218] It should be understood that when the encoder performs lossy compression on the target virtual speaker attribute information, the attribute information decoded by the decoder differs from the attribute information encoded by the encoder. When the encoder performs lossless compression on the virtual speaker attribute information, the attribute information decoded by the decoder is the same as the attribute information encoded by the encoder. However, this application does not distinguish between the attribute information encoded by the encoder and the attribute information decoded by the decoder in terms of name.

[0219] S303 : Generate a virtual speaker signal corresponding to the target virtual speaker based on the attribute information of the target virtual speaker and the first reconstructed signal.

[0220] S304: Reconstruct based on the attribute information of the target virtual speaker and the virtual speaker signal to obtain a first reconstructed scene audio signal. The first reconstructed scene audio signal includes audio signals of C2 channels, where C2 is a positive integer.

[0221] For example, a scene audio signal can be reconstructed based on a virtual speaker signal. Furthermore, a virtual speaker signal corresponding to the target virtual speaker can be generated based on the target virtual speaker's attribute information and the first reconstructed signal. Each target virtual speaker corresponds to one virtual speaker signal, and the virtual speaker signal is a plane wave. Next, reconstruction is performed based on the target virtual speaker's attribute information and the virtual speaker signal to generate a first reconstructed scene audio signal.

[0222] Exemplarily, when the scene audio signal is an HOA signal, the reconstructed first reconstructed scene audio signal may also be an HOA signal, and the HOA signal may be an N2-order HOA signal, where N2 is a positive integer. Exemplarily, the N2-order HOA signal may include audio signals of C2 channels, where C2=(N2+1). 2 .

[0223] Exemplarily, the order N2 of the first reconstructed scene audio signal may be greater than or equal to the order N1 of the scene audio signal in the embodiment of Figure 2a; correspondingly, the number of channels C2 of the audio signal included in the first reconstructed scene audio signal may be greater than or equal to the number of channels C1 of the audio signal included in the scene audio signal in the embodiment of Figure 2a.

[0224] Exemplarily, the scene audio signal is an N1-order HOA signal, the N1-order HOA signal includes a second audio signal, the second audio signal is an audio signal in the N1-order HOA signal excluding the first audio signal, and C1 is equal to the square of (N1+1); and / or,

[0225] The first reconstructed scene audio signal is an N2-order HOA signal, the N2-order HOA signal includes a third audio signal, the third audio signal is a reconstructed signal corresponding to each channel of the second audio signal in the N2-order HOA signal, and C2 is equal to the square of (N2+1).

[0226] In one possible approach, the first reconstructed scene audio signal may be directly used as the final decoding result.

[0227] S305 : Determine an attenuation factor according to a frequency band number of a reconstructed signal in the first reconstructed scene audio signal and / or an order of the first reconstructed scene audio signal.

[0228] For example, the decoding end can obtain the attenuation factor based on the frequency band number of the reconstructed signal in the first reconstructed scene audio signal, or the decoding end can obtain the attenuation factor based on the order of the first reconstructed scene audio signal. The order of the first reconstructed scene audio signal can specifically be the order of an N2-order HOA signal, such as an Ambisonic order. Alternatively, the decoding end can obtain the attenuation factor based on the frequency band number and the order of the first reconstructed scene audio signal. This attenuation factor can be called a dual attenuation factor. The attenuation factor can be attenuated based on at least one of the two factors: the frequency band number of the reconstructed signal and / or the order of the first reconstructed scene audio signal. The attenuation factor can be used to adjust the first reconstructed scene audio signal to improve the quality of the reconstructed scene audio signal.

[0229] S306 : Adjust the first reconstructed scene audio signal according to the high-order energy gain coding result and the attenuation factor to obtain a reconstructed scene audio signal.

[0230] The decoding end obtains a high-order energy gain coding result from the first bitstream and uses the high-order energy gain coding result and an attenuation factor to perform energy adjustment on the first reconstructed scene audio signal. The decoding end uses the high-order energy gain coding result to adjust the high-order channel energy of the first reconstructed scene audio signal, thereby improving the decoding quality of the scene audio signal.

[0231] Compared with other methods of reconstructing scene audio signals in the prior art, the audio quality of the scene audio signal reconstructed based on the virtual speaker signal is higher; therefore, when K is equal to C1, at the same bit rate, the audio quality of the scene audio signal reconstructed by the present application is higher.

[0232] When K is less than C1, in the process of encoding the scene audio signal, the number of channels of the audio signal encoded by the present application is less than the number of channels of the audio signal encoded by the prior art, and the data volume of the attribute information of the target virtual speaker is much smaller than the data volume of the audio signal of one channel; therefore, under the premise of the same bit rate, the audio quality of the reconstructed scene audio signal obtained by decoding by the present application is higher.

[0233] Secondly, since the virtual speaker signal and residual information encoded and transmitted by the prior art are converted from the original audio signal (i.e., the scene audio signal to be encoded), and are not the original audio signal, errors will be introduced; while the present application encodes part of the original audio signal (i.e., the audio signals of K channels in the scene audio signal to be encoded), thereby avoiding the introduction of errors, and thus being able to improve the audio quality of the reconstructed scene audio signal obtained by decoding; and it can also avoid fluctuations in the reconstruction quality of the reconstructed scene audio signal obtained by decoding, and has high stability.

[0234] In addition, since the prior art encodes and transmits virtual speaker signals, and the data volume of virtual speaker signals is relatively large, the number of target virtual speakers selected by the prior art is subject to significant bandwidth limitations. The present application encodes and transmits the attribute information of the virtual speaker, and the data volume of the attribute information is much smaller than the data volume of the virtual speaker signal; therefore, the number of target virtual speakers selected by the present application is subject to less bandwidth limitations. The more target virtual speakers selected, the higher the quality of the reconstructed scene audio signal based on the virtual speaker signals of the target virtual speakers. Therefore, compared to the prior art, under the same bit rate, the present application can select a larger number of target virtual speakers, so that the quality of the reconstructed scene audio signal decoded by the present application is also higher.

[0235] Furthermore, compared to the encoding and decoding ends of the prior art, the encoding and decoding ends of the present application do not require residual and superposition operations, thus reducing the combined complexity of the encoding and decoding ends of the present application. Because the first bitstream sent by the encoding end includes the high-order energy gain encoding result, the high-order energy gain can be used to adjust the high-order channel energy at the decoding end, thereby improving the encoding and decoding quality of the scene audio signal.

[0236] The following describes the encoding process of the high-order energy gain during encoding and the adjustment process of the audio signal by the high-order energy gain during decoding.

[0237] FIG4 is a schematic diagram illustrating an exemplary encoding process.

[0238] S401 : Acquire a scene audio signal to be encoded, where the scene audio signal includes audio signals of C1 channels, where C1 is a positive integer.

[0239] For example, S401 may refer to the description of S201 above, which will not be repeated here.

[0240] S402: Acquire attribute information of a target virtual speaker corresponding to the scene audio signal.

[0241] In one possible approach, attribute information of the target virtual speaker is generated based on the position information of the target virtual speaker. In one possible approach, the position information of the target virtual speaker (including pitch angle information and horizontal angle information) can be used as the attribute information of the target virtual speaker. In another possible approach, a position index corresponding to the position information of the target virtual speaker (including a pitch angle index (which can be used to uniquely identify the pitch angle information) and a horizontal angle index (which can be used to uniquely identify the horizontal angle information)) is used as the attribute information of the target virtual speaker.

[0242] In one possible approach, the virtual speaker index (eg, virtual speaker identifier) ​​of the target virtual speaker may be used as the attribute information of the target virtual speaker, wherein the virtual speaker index corresponds one-to-one to the position information.

[0243] In one possible approach, the virtual speaker coefficients of the target virtual speaker may be used as the attribute information of the target virtual speaker. For example, C2 virtual speaker coefficients of the target virtual speaker may be determined and used as the attribute information of the target virtual speaker; wherein the C2 virtual speaker coefficients of the target virtual speaker correspond one-to-one to the C2 channels of audio signals included in the first reconstructed scene audio signal.

[0244] It should be noted that the data volume of the virtual speaker coefficient is much larger than the data volume of the position information, the index of the position information, and the virtual speaker index; according to the bandwidth, it can be decided which information among the position information, the index of the position information, the virtual speaker index, and the virtual speaker coefficient is used as the attribute information of the target virtual speaker. For example, when the bandwidth is large, the virtual speaker coefficient can be used as the attribute information of the target virtual speaker; in this way, the decoding end does not need to calculate the virtual speaker coefficient of the target virtual speaker, which can save the computing power of the decoding end. When the bandwidth is small, any one of the position information, the index of the position information, and the virtual speaker index can be used as the attribute information of the target virtual speaker; in this way, the bit rate can be saved. It should be understood that it can also be pre-set which information among the position information, the index of the position information, the virtual speaker index, and the virtual speaker coefficient is used as the attribute information of the target virtual speaker; this is not limited in this application.

[0245] S403: Obtain an energy gain of the first audio signal and an energy gain of the second audio signal.

[0246] The feature information corresponding to the scene audio signal includes gain information. The scene audio signal includes a first audio signal and a second audio signal. The energy gain E(1, b) of the first audio signal and the energy gain E(i, b) of the second audio signal are calculated respectively.

[0247] S404: Obtain a high-order energy gain according to the energy gain of the first audio signal and the energy gain of the second audio signal.

[0248] After the encoder obtains the high-order energy gain of the scene audio signal, it can encode it to generate a high-order energy gain encoding result. The role of the high-order energy gain is to adjust the high-order channel energy at the decoder, improving the encoding and decoding quality of the HOA signal.

[0249] Exemplarily, obtaining a high-order energy gain according to the energy gain of the first audio signal and the energy gain of the second audio signal includes:

[0250] The high-order energy gain Gain'(i, b) is obtained as follows:

[0251] Gain'(i,b)=10*log10(E(i,b) / E(1,b));

[0252] Wherein, log10 represents the logarithmic function log, * represents a multiplication operation, E(1, b) is the channel energy of the first audio signal, E(i, b) is the energy of each channel of the second audio signal, i is the number of the i-th channel of the second audio signal, and b is the frequency band number of the second audio signal.

[0253] Exemplarily, the characteristic information of the second audio signal may be the high-order energy gain of the N1-order HOA signal, specifically the energy ratio of each channel of the second audio signal to the W channel (the first channel of the N1-order HOA signal), where the W channel may be the channel of the first audio signal.

[0254] Exemplarily, the characteristic information of the second audio signal may be obtained by referring to the following steps:

[0255] The N1-order HOA signal is subjected to time-frequency transformation, and the time-domain N1-order HOA signal is transformed into the frequency-domain N1-order HOA signal.

[0256] Calculate W channel energies E(1, b) and each channel energy E(i, b) of the second audio signal, where i is the channel number of the second audio signal.

[0257] The following formula can be used to calculate the high-order energy gain Gain'(i, b):

[0258] Gain(i,b)=E(i,b) / E(1,b);

[0259] Gain'(i, b)=10*log10(Gain(i, b)).

[0260] S405 , quantizing the high-order energy gain to obtain a quantized high-order energy gain.

[0261] S406 : Perform entropy coding on the quantized high-order energy gain to obtain the high-order energy gain coding result.

[0262] Herein, characteristic information of the second audio signal in the scene audio signal is obtained, a high-order energy gain of the scene audio signal is obtained according to the characteristic information of the second audio signal, and the high-order energy gain is sequentially quantized and entropy coded.

[0263] For example, scalar quantization may be used to quantize the high-order energy gain.

[0264] Perform entropy coding on the quantized high-order energy gain. The entropy coding method is not limited.

[0265] Exemplarily, the high-order energy gain is differentially coded, and then the number of bits of entropy coding is estimated. If the estimated number of bits is less than the fixed-length coding, the high-order energy gain is variable-length coded, such as Huffman coding; otherwise, the high-order energy gain is fixed-length coded.

[0266] After obtaining the high-order energy gain coding result, the coding result is written into the bitstream.

[0267] S407 : Encode the attribute information of the first audio signal and the target virtual speaker in the scene audio signal and the high-order energy gain encoding result to obtain a first bitstream.

[0268] It should be understood that the number of channels of the audio signal included in the first audio signal can be determined according to requirements and bandwidth, and this application does not impose any limitation on this.

[0269] For example, S401 may refer to the description of S201 above, which will not be repeated here.

[0270] In this embodiment of the present application, the encoder can calculate the energy ratio of the high-order channel to the W channel to obtain a high-order energy gain encoding result. Then, based on the estimated number of bits in the inter-subframe difference result, it selects Huffman coding or direct encoding. As a result, the first bitstream sent by the encoder includes the high-order energy gain encoding result. The high-order energy gain can then be used to adjust the high-order channel energy at the decoder, improving the encoding and decoding quality of the scene audio signal.

[0271] Figure 5a is a schematic diagram of an exemplary decoding process. Figure 5a is a decoding process corresponding to the encoding process in Figure 4.

[0272] S501: Receive a first code stream.

[0273] S502 : Decode the first code stream to obtain a first reconstructed signal, property information of a target virtual speaker, and a high-order energy gain encoding result.

[0274] S503, generating a virtual speaker signal corresponding to the target virtual speaker based on the attribute information of the target virtual speaker and the first audio signal;

[0275] S504, reconstructing based on the attribute information of the target virtual speaker and the virtual speaker signal to obtain a first reconstructed scene audio signal; the first reconstructed scene audio signal includes C2 channels of audio signals, where C2 is a positive integer

[0276] For example, S501 to S504 can refer to the description of S301 to S304 and will not be repeated here.

[0277] For example, the above S306 may refer to the description of S505 to S508.

[0278] S505 , performing entropy decoding on the high-order energy gain coding result to obtain the high-order energy gain after entropy decoding.

[0279] S506 , dequantizing the high-order energy gain after entropy decoding to obtain the high-order energy gain.

[0280] Exemplarily, the high-order energy gain coding result is read from the first bitstream and entropy decoding is performed on the high-order energy gain coding result. The entropy decoding method is the inverse process of entropy coding at the encoding end.

[0281] For example, if the encoding end adopts fixed-length coding, the decoding end uses the corresponding fixed-length decoding; if the encoding end adopts coding encoding, the decoding end uses the corresponding edge-length decoding, such as Huffman decoding.

[0282] The entropy decoding result is dequantized. The dequantization method is the inverse process of the quantization method at the encoding end.

[0283] S507 : Adjust the high-order energy gain according to the characteristic information of the second audio signal and the characteristic information of the first audio signal to obtain an adjusted decoded high-order energy gain.

[0284] Among them, after the decoding end performs signal reconstruction to obtain the first reconstructed scene audio signal, the first audio signal and the third audio signal are determined from the first reconstructed scene audio signal, where the third audio signal is a reconstructed signal corresponding to each channel of the second audio signal in the N2-order HOA signal, and the characteristic information of the second audio signal is determined based on the characteristic information of the third audio signal. Finally, the high-order energy gain is adjusted based on the characteristic information of the second audio signal and the characteristic information of the first audio signal to obtain an adjusted decoded high-order energy gain. The high-order energy gain is adjusted so that the high-order channel energy is more uniform and smooth, and the quality of the reconstructed audio signal is better.

[0285] Exemplarily, S507 adjusts the high-order energy gain according to the characteristic information of the second audio signal and the characteristic information of the first audio signal, including:

[0286] S5071: Obtain high-order energy of the second audio signal according to the channel energy and the high-order energy gain of the first audio signal;

[0287] The decoding end obtains the high-order energy gain encoding result from the first bitstream, performs entropy decoding on the high-order energy gain encoding result, and dequantizes it to obtain the high-order energy gain. The energy of the second audio signal is then estimated based on the channel energy of the first audio signal and the high-order energy gain to determine the high-order energy of the second audio signal.

[0288] Exemplarily, the first reconstructed scene audio signal is an N2-order HOA signal, and a time-frequency transform is performed on the N2-order HOA signal, and the time-domain N2-order HOA signal is transformed to obtain a frequency-domain N2-order HOA signal.

[0289] The high-order energy E_Ref(i, b) of the second audio signal may be calculated using the following formula:

[0290] E_Ref(i, b)=E_dec(1, b)*10^(Gain'(i, b) / 10)

[0291] Wherein, E_dec(1, b) is the channel energy of the b-th frequency band of the first audio signal in the N2-order HOA signal, i is the channel number corresponding to the second audio signal, Gain'(i, b) is the high-order energy gain, and b is the frequency band number of the first audio signal.

[0292] S5072: Obtain a decoding energy scaling factor according to the channel energy of the third audio signal and the high-order energy of the second audio signal.

[0293] Specifically, the third audio signal is a reconstructed signal corresponding to each channel of the second audio signal in the N2-order HOA signal, and an energy ratio calculation is performed on the third audio signal and the second audio signal to obtain a decoding energy ratio factor.

[0294] Exemplarily, the decoding energy proportional factor g(i, b) may be calculated using the following formula:

[0295] g(i,b)=sqrt(E_Ref(i,b)) / sqrt(E_dec(i,b))

[0296] Wherein, sqrt() is a square root operation, E_dec(i, b) is the channel energy of the b-th frequency band of the third audio signal, i is the channel number corresponding to the third audio signal, and E_Ref(i, b) is the high-order energy of the b-th frequency band of the second audio signal.

[0297] S5073: Obtain a decoded high-order energy gain of the third audio signal according to the channel energy of the third audio signal and the channel energy of the first audio signal.

[0298] The channel energy of the first audio signal is used as a reference to perform gain calculation on the channel energy of the third audio signal to obtain a decoded high-order energy gain of the third audio signal.

[0299] Exemplarily, the decoding high-order energy gain Gain_dec(i, b) may be calculated using the following formula:

[0300] Gain_dec(i,b)=E_dec(i,b) / E_dec(1,b)

[0301] Wherein, E_dec(1, b) is the channel energy of the b-th frequency band of the first audio signal in the N2-order HOA signal, and E_dec(i, b) is the channel energy of the b-th channel of the third audio signal.

[0302] S5074: Adjust the decoded high-order energy gain of the third audio signal according to the decoded energy proportional factor to obtain an adjusted decoded high-order energy gain.

[0303] Specifically, to make the energy of the high-order channels more uniform and smooth, the decoded high-order energy gain of the third audio signal is adjusted using the decoded energy scaling factor, and an adjusted decoded high-order energy gain is determined. After adjustment using the decoded energy scaling factor, the energy of the high-order channels is more uniform and smooth, and the quality of the reconstructed audio signal is improved.

[0304] Exemplarily, adjusting the decoded high-order energy gain of the third audio signal according to the decoded energy scale factor to obtain an adjusted decoded high-order energy gain includes:

[0305] The adjusted decoding high-order energy gain Gain_dec'(i, b) is obtained as follows:

[0306] Gain_dec'(i, b)=w*min(g(i, b), Gain_dec(i, b))+(1-w)*g(i, b);

[0307] Wherein, g(i, b) represents the decoding energy scaling factor, Gain_dec(i, b) represents the decoding high-order energy gain of the b-th frequency band of the third audio signal, w is a preset adjustment ratio threshold, min represents a minimum value operation, and * represents a multiplication operation.

[0308] For example, min(a, b) is the minimum value of a and b, and w is the adjustment ratio threshold. There are many ways to take the value of w, for example, the value of w is 0.25.

[0309] S508 : Adjust the third audio signal in the N 2 -order HOA signal according to the adjusted decoded high-order energy gain to obtain an adjusted third audio signal.

[0310] The decoder obtains a high-order energy gain coding result from the first bitstream and uses the high-order energy gain coding result to perform energy adjustment on the third audio signal in the N2-order HOA signal. The decoder uses the high-order energy gain coding result to adjust the high-order channel energy of the third audio signal, thereby improving the decoding quality of the third audio signal.

[0311] The third audio signal is a channel audio signal corresponding to each channel of the second audio signal in the N2-order HOA signal.

[0312] Exemplarily, the third audio signal may be adjusted based on the feature information corresponding to the second audio signal in the N1-order HOA signal to improve the quality of the N2-order HOA signal.

[0313] Exemplarily, S508 adjusts the third audio signal in the N2-order HOA signal according to the adjusted decoded high-order energy gain, including:

[0314] S5081: Obtain an attenuation factor according to the frequency band number of the third audio signal and / or the order of the N2-order HOA signal.

[0315] For example, the decoding end can obtain the attenuation factor according to the frequency band number of the third audio signal, or the decoding end can obtain the attenuation factor according to the order of the N2-order HOA signal, where the order of the N2-order HOA signal can specifically be an Ambisonic order, or the decoding end can obtain the attenuation factor according to the above-mentioned frequency band number and the order of the N2-order HOA signal. This attenuation factor can be called a dual attenuation factor.

[0316] S5082: Adjust the third audio signal according to the adjusted decoded high-order energy gain and attenuation factor to obtain an adjusted third audio signal, where the adjusted third audio signal belongs to the reconstructed scene audio signal.

[0317] After obtaining the adjusted decoded high-order energy gain, the gain of the third audio signal of the current frame can be weighted, and the gain can be attenuated with the frequency band number of the third audio signal and / or the order of the N2-order HOA signal. The attenuation factor can be first obtained based on the frequency band number of the third audio signal and / or the order of the N2-order HOA signal. For example, the attenuation factor can be attenuated with two factors: the frequency band and the ambisonic order. The adjusted decoded high-order energy gain and the obtained attenuation factor are then applied to the high-order channel of the third audio signal reconstructed in the current frame, making the high-order channel energy more uniform and smooth, thereby improving the quality of the reconstructed audio signal.

[0318] Exemplarily, the third audio signal is adjusted using the adjusted decoded high-order energy gain Gain_dec'(i, b) and the attenuation factor g'(i, b).

[0319] For example, the following formula can be used for adjustment:

[0320] X'(i, b)=X(i, b)*Gain_dec'(i, b)*g'(i, b);

[0321] Wherein, X(i, b) is the third audio signal before adjustment, and X'(i, b) is the third audio signal after adjustment.

[0322] In one possible manner, S5081 obtains the attenuation factor according to the frequency band number of the third audio signal and the order of the N2-order HOA signal, including:

[0323] The attenuation factor g'(i, b) is obtained as follows:

[0324] Wherein, i is the number of the i-th channel of the third audio signal, b is the frequency band number of the third audio signal, α represents the number of the target virtual speakers, β represents the number of mapping channels of the N2-order HOA signal, and β is equal to (M+2) 2 , M is the order of the N2-order HOA signal, γ represents the order corresponding to the channel number i of the third audio signal, and × represents a multiplication operation.

[0325] Exemplarily, b is the frequency band number of the third audio signal. The frequency band number may also be called a sub-band number, b=0, 1, 2, ..., 11.

[0326] By calculating the attenuation factor g'(i, b) as described above, b is the frequency band number of the third audio signal, α represents the number of target virtual speakers, and β represents the number of mapping channels of the N2-order HOA signal, where β is equal to (M+2). 2 , M is the order of the N2-order HOA signal, and γ represents the order corresponding to the channel number i of the third audio signal. The above parameters can be used to accurately calculate the attenuation factor. By adjusting the parameters, the attenuation factor changes with three factors: the number of speakers, the number of HOA signal mapping channels, and the HOA order. When the attenuation factor and the adjusted decoded high-order energy gain are used to adjust the third audio signal, the quality of the reconstructed audio signal is improved.

[0327] In one possible embodiment, the scene audio signal decoding method provided in the embodiment of the present application further includes:

[0328] When b≤d, update α to α2, α2=0.375×α; d is the preset first threshold;

[0329] When b>d, α is updated to α3, α3=0.5×α.

[0330] Exemplarily, d is a preset first threshold value, and the value of the first threshold value is not limited. When b ≤ d, the value of α can be reduced to α2, and the relationship between α2 and α satisfies the above-mentioned relationship α2 = 0.375 × α. When b > d, the value of α can be reduced to α3, and the relationship between α3 and α satisfies the above-mentioned relationship α3 = 0.5 × α. When the frequency band b is less than the first threshold value, that is, the bth frequency band can represent the low frequency band, then the attenuation coefficient is set to a smaller value, for example, the attenuation coefficient is equal to 0.375. When the frequency band b is greater than the threshold value, that is, the bth frequency band can represent the high frequency band, then the attenuation coefficient is set to a larger value, for example, the attenuation coefficient is equal to 0.5. The above-mentioned attenuation coefficient of 0.375 or 0.5 is only a possible example implementation method. For example, the above-mentioned 0.375 can also be replaced by 0.38 or 0.37, and the above-mentioned 0.5 can also be replaced by 0.55 or 0.6. The specific value of the attenuation coefficient needs to be determined in combination with the application scenario and is not limited here. In the above scheme, the value of α can be flexibly adjusted according to the value of the frequency band b, so that the attenuation effect of the attenuation factor becomes more significant as the frequency band increases, thereby making the reconstructed audio signal more in line with the auditory characteristics of the human ear.

[0331] Exemplarily, α represents the number of target virtual loudspeakers and has values ​​of 0, 1, 2, 3...N, for example, α has values ​​of 2, 3, and 4. α can be further adjusted: when subband b ≤ the first threshold, α2 = 0.375×α, otherwise α3 = 0.5×α.

[0332] In one possible embodiment, the scene audio signal decoding method provided in the embodiment of the present application further includes:

[0333] Update the α to α4, α4 = α + α × (1-1.25 × b) / (bands + 2);

[0334] Bands represents the number of frequency bands of the third audio signal, that is, bands represents the number of sub-bands, and for example, the value of bands can be configured to be 12. This is merely an example and does not limit the embodiments of the present application.

[0335] In the embodiment of the present application, the value of α is updated to α4, where α4 = α + α × (1 - 1.25 × b) / (bands + 2). That is, by expanding the value of α, the value of the attenuation factor is related to the i-th frequency band, thereby achieving different attenuation efficiencies for the low-frequency band and the high-frequency band. This improves the quality of the reconstructed audio signal when the attenuation factor is used to adjust the third audio signal.

[0336] In one possible embodiment, the scene audio signal decoding method provided in the embodiment of the present application further includes:

[0337] When b≤d, β is updated to β2, β2=(1+w)×β; d is a preset first threshold, and w is a preset adjustment ratio threshold.

[0338] For example, β represents the number of mapping channels of the HOA signal, and its value is (M+2) 2 , M is the HOA signal order, and β ranges from 0, 1, 2, 3, ..., N. When subband b ≤ the first threshold, β2 = (1 + w) × β. When b ≤ the first threshold, the above scheme allows for flexible adjustment of the value of α based on the value of the frequency band b, achieving a more significant attenuation effect with higher frequency bands, thus making the reconstructed audio signal more consistent with the human hearing characteristics.

[0339] In a possible manner, the scene audio signal decoding method provided in the embodiment of the present application further includes: updating w to w2, where w2=w+α×0.05.

[0340] For example, the method for determining the value of w is as follows: w is initially set to 0, and the values ​​of α are iterated sequentially to calculate the final adjusted value of w. For example, w is updated to w2, where w2 = w + α × 0.05. The value of w can be updated based on α, so that the weight w in the attenuation factor is related to the parameter α. As α increases, w also increases, achieving a more significant attenuation effect with higher frequency bands, thereby making the reconstructed audio signal more consistent with the human ear's auditory characteristics.

[0341] In one possible approach, when the value of i is 0, 1, 2, or 3, the value of γ is 1;

[0342] When the value of i is 4, 5, 6, 7, or 8, the value of γ is 2;

[0343] When the value of i is 9, 10, 11, 12, 13, 14, or 15, the value of γ is 3;

[0344] Here, i is the number of the i-th channel of the third audio signal.

[0345] Exemplarily, γ represents the order of the HOA signal in the i-th channel, and γ and i satisfy the following Table 1:

[0346] Table 1 is the corresponding relationship table between γ and i:

[0347] It can be seen from the above value of γ that the attenuation factor is determined by the value of γ. The value of γ is a piecewise function related to the HOA order of the channel. As the HOA order of i increases, the value of γ increases, but will not exceed the maximum HOA order. This attenuation factor is used to improve the quality of the reconstructed audio signal when adjusting the third audio signal.

[0348] Exemplarily, after S5082 adjusts the third audio signal in the N2-order HOA signal according to the adjusted decoded high-order energy gain and attenuation factor, the method further includes:

[0349] S5083: Obtain channel energy of a fourth audio signal corresponding to the adjusted third audio signal, where the third audio signal includes an audio signal of a current frame, and the fourth audio signal includes an audio signal of a frame preceding the current frame.

[0350] S5084: Adjust the adjusted third audio signal again according to the channel energy of the fourth audio signal.

[0351] Among them, the decoding end can also use the previous frame of the third audio signal to adjust the adjusted third audio signal of the current frame again to improve the quality of the reconstructed audio signal. The third audio signal includes the audio signal of the current frame, and the fourth audio signal includes the audio signal of the previous frame of the current frame. For example, the previous frame can be the audio signal of the previous frame adjacent to the current frame, or the previous frame can also be the audio signal of the previous frame not adjacent to the current frame. The channel energy of the fourth audio signal can be used to adjust the third audio signal. For example, the decoding end linearly weights the high-order channel of the current frame of the third audio signal and the corresponding sub-bands of the high-order channels of the previous two frames to obtain the high-order channel of the current frame after energy smoothing.

[0352] Exemplarily, S5084 adjusts the third audio signal according to the channel energy of the fourth audio signal, including:

[0353] S50841: Obtain an average channel energy of the fourth audio signal and an adjusted channel energy of the third audio signal;

[0354] The average channel energy value of the fourth audio signal may be an average value of energies of all channels of the fourth audio signal.

[0355] S50842: Obtain an energy average threshold according to the channel energy average of the fourth audio signal and the channel energy of the third audio signal.

[0356] The energy average threshold is a threshold obtained by calculating the channel energies of the third audio signal and the fourth audio signal.

[0357] Exemplarily, obtaining the energy average threshold according to the channel energy average of the fourth audio signal and the channel energy of the third audio signal includes:

[0358] The energy average threshold k is obtained as follows:

[0359] The following formula can be used:

[0360] E_mean(i) represents the channel energy average of the fourth audio signal, and E'_dec(i) represents the energy of the adjusted third audio signal.

[0361] S50843: Perform a weighted average calculation on the channel energy average of the fourth audio signal and the adjusted channel energy of the third audio signal according to the energy average threshold to obtain a target energy.

[0362] To calculate the target energy E_target(i, b), the following formula can be used:

[0363] E_target(i, b)=k*E_mean(i, b)+(1-k)*E'_dec(i, b);

[0364] Wherein, E_mean(i, b) is the average value of the energy of the previous frame, and E'_dec(i, b) is the energy of the third audio signal after adjustment.

[0365] S50844: Obtain an energy smoothing factor according to the target energy and the adjusted channel energy of the third audio signal;

[0366] The energy smoothing factor can be used to adjust the third audio signal, so that the decoding quality of the third audio signal is higher.

[0367] Exemplarily, obtaining the energy smoothing factor according to the target energy and the adjusted channel energy of the third audio signal includes:

[0368] The energy smoothing factor q(i, b) is obtained as follows:

[0369] q(i, b)=sqrt(E_target(i, b)) / sqrt(E'_dec(i, b));

[0370] Wherein, E_target(i, b) represents the target energy, and E'_dec(i, b) represents the energy of the third audio signal.

[0371] S50845: Adjust the third audio signal according to the energy smoothing factor.

[0372] The adjusted third audio signal is further adjusted by using the energy smoothing factor q(i, b) to further improve the decoding quality of the third audio signal.

[0373] Exemplarily, the third audio signal may be adjusted according to the following formula:

[0374] X”(i,b)=X’(i,b)*q(i,b);

[0375] Illustratively, after obtaining the adjusted third audio signal, the average value of the energy of the previous frame may be updated using the energy of the adjusted third audio signal.

[0376] Figure 5b is a schematic diagram of an exemplary decoding process. Figure 5b is a decoding process corresponding to the encoding process in Figure 4.

[0377] S501: Receive a first code stream.

[0378] S502 : Decode the first code stream to obtain a first reconstructed signal, property information of a target virtual speaker, and a high-order energy gain encoding result.

[0379] S503, generating a virtual speaker signal corresponding to the target virtual speaker based on the attribute information of the target virtual speaker and the first audio signal;

[0380] S504, reconstructing based on the attribute information of the target virtual speaker and the virtual speaker signal to obtain a first reconstructed scene audio signal; the first reconstructed scene audio signal includes C2 channels of audio signals, where C2 is a positive integer

[0381] For example, S501 to S504 can refer to the description of S301 to S304 and will not be repeated here.

[0382] For example, the above S306 may refer to the description of S505 to S508.

[0383] S505 , performing entropy decoding on the high-order energy gain coding result to obtain the high-order energy gain after entropy decoding.

[0384] S506 , dequantizing the high-order energy gain after entropy decoding to obtain the high-order energy gain.

[0385] Exemplarily, the high-order energy gain coding result is read from the first bitstream and entropy decoding is performed on the high-order energy gain coding result. The entropy decoding method is the inverse process of entropy coding at the encoding end.

[0386] For example, if the encoding end adopts fixed-length coding, the decoding end uses the corresponding fixed-length decoding; if the encoding end adopts coding encoding, the decoding end uses the corresponding edge-length decoding, such as Huffman decoding.

[0387] The entropy decoding result is dequantized. The dequantization method is the inverse process of the quantization method at the encoding end.

[0388] S509: Determine a diffusion factor of the first reconstructed scene audio signal according to the first bitstream.

[0389] Because the reconstructed HOA signal is calculated from the virtual speaker signal, it only contains the clearly oriented sound source components of the HOA signal to be encoded, also known as directional components, and lacks the environmental components of the HOA signal to be encoded, also known as non-directional components. Therefore, adjusting the energy of each channel of the reconstructed HOA signal through audio gain can only bring the energy of the directional components closer to the energy of the HOA signal to be encoded, while the adjustment of the non-directional components is unstable.

[0390] Dispersion is a parameter that can be used to describe the degree of diffusion of the HOA signal, and is applied to the sound field reconstruction and sound field playback of the HOA signal. In the embodiment of the present application, the dispersion of the HOA signal is used to measure the energy ratio of the non-directional component in the HOA signal to be encoded, and is used to adjust the energy of each channel of the reconstructed HOA signal, so that the reconstructed HOA signal after energy adjustment is closer to the energy of the HOA signal to be encoded. Using dispersion as a parameter for adjusting the energy of the reconstructed HOA signal, the dispersion can also use its own ability to describe the directionality of the sound source to adaptively compensate for the non-directional component when adjusting the energy of the reconstructed HOA signal, and not compensate for the directional component. Therefore, the dispersion can make up for the shortcomings of the gain adjustment method based on the signal energy ratio without introducing additional errors.

[0391] In the embodiment of the present application, the dispersion is used to adjust the energy gain so as to make the reconstructed HOA signal energy more accurate.

[0392] In a possible implementation, S509 determines the dispersivity factor of the first reconstructed scene audio signal according to the first bitstream, including:

[0393] The dispersivity factor is obtained by decoding the first code stream, wherein the first code stream includes the dispersivity factor.

[0394] Specifically, the dispersion is calculated based on the HOA signal to be encoded at the encoding end, and the dispersion is quantized and encoded into the first bit stream. The decoding end decodes the first bit stream and then obtains the dispersion through inverse quantization, which is then used to adjust the gain of the uncoded channel in the reconstructed HOA signal.

[0395] In a possible implementation, S509 determines the dispersivity factor of the first reconstructed scene audio signal according to the first bitstream, including:

[0396] The dispersion factor is obtained according to the first reconstructed signal obtained by decoding the first code stream.

[0397] There are many methods for calculating dispersion. At the decoding end, the dispersion is calculated based on the low-order HOA signal in the decoded transmission channel, and then used to adjust the gain of the uncoded channel in the reconstructed HOA signal.

[0398] In a possible implementation, S509, obtaining the dispersivity factor according to the first reconstructed signal obtained by decoding the first bitstream, includes:

[0399] S5091, obtaining the sound field intensity of each frequency band in the first reconstructed signal;

[0400] S5092, obtaining energy of each frequency band;

[0401] S5093: Determine the diffusion factor according to the sound field intensity of each frequency band and the energy of each frequency band.

[0402] Exemplarily, S5091 acquires the sound field intensity of each frequency band in the first reconstructed signal, including:

[0403] The sound field intensity I(b) of the bth frequency band is obtained as follows:

[0404] Wherein, Re represents a real part operation, and W(b), X(b), Y(b), and Z(b) represent four channel signals of the bth frequency band in the first reconstructed signal.

[0405] Exemplarily, the energy of each frequency band in S5092 can be obtained by calculating the sum of the square of the real part and the square of the imaginary part of each frequency band.

[0406] S5093 determines the diffusivity factor according to the sound field intensity of each frequency band and the energy of each frequency band, including:

[0407] The diffusion factor diffusion (b) is obtained as follows:

[0408] Wherein, E represents the expectation operation, ||E{I(b)}|| represents the calculation of the bi-norm of E{I(b)}, ||M(b)|| represents the calculation of the bi-norm of M(b)I, and M(b) represents the energy of the b-th frequency band.

[0409] It is understandable that the calculation of the diffusion degree in the above S5091 to S5093 is only an example implementation method and is not used here to limit the embodiments of the present application.

[0410] S510: Determine an attenuation factor according to a frequency band number of a reconstructed signal in a first reconstructed scene audio signal and / or an order of the first reconstructed scene audio signal, and linearly weight the attenuation factor according to a diffusion factor to obtain a weighted attenuation factor.

[0411] Specifically, the dispersion factor can also be used to linearly weight the attenuation factor, and the weighted attenuation factor is used to adjust the third audio signal. A weighting method is used to balance the proportions of the diffuse component and the directional component in the attenuation factor. Because the dispersion factor can be used to measure the energy proportion of the non-directional component in the HOA signal to be encoded and is used to adjust the energy of each channel of the reconstructed HOA signal, the energy of the reconstructed HOA signal after energy adjustment is closer to the energy of the HOA signal to be encoded.

[0412] There is no limitation on the specific implementation method of linearly weighting the attenuation factor by the diffusivity factor. An example is given below:

[0413] In one possible implementation, the attenuation factor is linearly weighted according to the diffusivity factor to obtain a weighted attenuation factor, including:

[0414] Obtain the weighted attenuation factor gd(i,b) by at least one of the following methods:

[0415] gd(i,b)=w×diffusion(b)+(1-w)×g'(i,b); where w is a preset adjustment ratio threshold, × represents a multiplication operation, diffusion(b) represents the diffusion factor of the b-th frequency band of the third audio signal, and g'(i,b) represents the attenuation factor of the b-th frequency band of the third audio signal;

[0416] or,

[0417] When the attenuation factor is for a full-band signal, gd(i,b) = w × mean(diffusion) + (1-w) × g'(i,b), where mean(diffusion) is the average of the diffusion factors of multiple frequency bands of the third audio signal, g'(i,b) represents the attenuation factor of the b-th frequency band of the third audio signal, and w is a preset adjustment ratio threshold.

[0418] or,

[0419] gd(i,b)=w×diffusion(b)+(1-w)×g'(i,b)+offset(i,b), where offset(i,b) represents the offset constant of the b-th frequency band on the i-th channel of the third audio signal, g'(i,b) represents the attenuation factor of the b-th frequency band of the third audio signal, w is the preset adjustment ratio threshold, and diffusion(b) represents the diffusion factor of the b-th frequency band of the third audio signal;

[0420] or,

[0421] gd(i,b)=w×diffusion(b)+(1-w)×g'(i,b)+direction(i,b), where direction(i,b) represents the direction parameter of the b-th frequency band on the i-th channel of the third audio signal, g'(i,b) represents the attenuation factor of the b-th frequency band of the third audio signal, w is the preset adjustment ratio threshold, and diffusion(b) represents the diffusion factor of the b-th frequency band of the third audio signal.

[0422] It is understandable that the weighted calculation of the attenuation factor in the above example is only an example implementation method and is not intended to limit the embodiments of the present application.

[0423] S511 , obtaining high-order energy of the second audio signal according to the channel energy and the high-order energy gain of the first audio signal; and obtaining a decoding energy scaling factor according to the channel energy of the third audio signal and the high-order energy of the second audio signal.

[0424] Specifically, the third audio signal is a reconstructed signal corresponding to each channel of the second audio signal in the N2-order HOA signal, and an energy ratio calculation is performed on the third audio signal and the second audio signal to obtain a decoding energy ratio factor.

[0425] Exemplarily, the decoding energy proportional factor g(i, b) may be calculated using the following formula:

[0426] g(i, b)=sqrt(E_Ref(i, b)) / sqrt(E_dec(i, b));

[0427] Wherein, sqrt() is a square root operation, E_dec(i, b) is the channel energy of the b-th frequency band of the third audio signal, i is the channel number corresponding to the third audio signal, and E_Ref(i, b) is the high-order energy of the b-th frequency band of the second audio signal.

[0428] S512: Adjust the third audio signal in the N2-order HOA signal according to the weighted attenuation factor and the decoding energy scale factor to obtain an adjusted third audio signal.

[0429] The decoder adjusts the third audio signal in the N2-order HOA signal based on the weighted attenuation factor and the decoding energy scaling factor. The decoder uses the weighted attenuation factor and the decoding energy scaling factor to adjust the high-order channel energy of the third audio signal, thereby improving the decoding quality of the third audio signal.

[0430] In one possible implementation, the third audio signal in the N2-order HOA signal is adjusted according to the weighted attenuation factor and the decoding energy scale factor to obtain an adjusted third audio signal:

[0431] The adjusted third audio signal X'(i,b) is obtained as follows:

[0432] X'(i,b)=X(i)×g(i,b)×gd(i,b);

[0433] Wherein, gd(i,b) represents the weighted attenuation factor, g(i,b) represents the decoding energy proportional factor, and X(i) represents the third audio signal.

[0434] In the above scheme, since the dispersion factor can be used to measure the energy ratio of the non-directional component in the HOA signal to be encoded, and the decoding energy scaling factor can be used to measure the energy ratio of the reconstructed HOA signal, the dispersion factor and the decoding energy scaling factor are used to adjust the energy of each channel of the reconstructed HOA signal, so that the reconstructed HOA signal after energy adjustment is closer to the energy of the HOA signal to be encoded.

[0435] It will be understood that the adjustment calculation of the third audio signal in the above example is only an example implementation manner and is not intended to limit the embodiments of the present application.

[0436] Take the following example to illustrate that the input signal in the encoding end is a 3rd-order HOA signal, which includes a total of 16 channels of audio signals, the first audio signal is the audio signal of channels 1 to 5, channel 7, and channels 9 to 10, and the second audio signal is the audio signal of channels 6, channel 8, and channels 11 to 16. There are three ways to implement the bit stream encoded by the encoding end: 1. The bit stream obtained by encoding does not include the high-order energy gain encoding result. 2. The bit stream obtained by encoding includes the high-order energy gain encoding result. For the scene audio decoding method performed by the decoding end, there are three ways to implement it: 1. When the received bit stream does not include the high-order energy gain encoding result, the decoding end reconstructs the scene audio signal in the bit stream. 2. When the received bit stream includes the high-order energy gain encoding result, the decoding end reconstructs the scene audio signal in the bit stream and adjusts the reconstructed scene audio signal according to the high-order energy gain encoding result to obtain the reconstructed scene audio signal. 3. In an embodiment of the present application, when the received bit stream includes a high-order energy gain coding result, the decoding end reconstructs the scene audio signal in the bit stream and adjusts the reconstructed scene audio signal according to the high-order energy gain coding result and the attenuation factor to obtain a reconstructed scene audio signal.

[0437] Analysis of the signal quality of the reconstructed scene audio signal shows that the decoded HOA signal without the high-order energy gain coding result has poor quality. The decoded HOA signal with the high-order energy gain coding result, but without the attenuation factor to reconstruct the scene audio signal, has medium quality. The decoded HOA signal with the high-order energy gain coding result and with the attenuation factor to reconstruct the scene audio signal has the best quality.

[0438] The above analysis demonstrates that, in this embodiment of the present application, the decoder can adjust the reconstructed scene audio signal based on the high-order energy gain encoding result and the attenuation factor, thereby achieving more uniform and smooth high-order channel energy in the reconstructed scene audio signal, resulting in improved quality. For example, the attenuation factor can be attenuated by factors such as the frequency band and ambisonic order of the reconstructed scene audio signal, effectively improving the encoding and decoding quality of the HOA signal.

[0439] FIG6 a is a schematic diagram showing the structure of an encoding end.

[0440] Parameter Figure 6a, illustratively, the encoding end may include a configuration unit, a virtual speaker generation unit, a target speaker generation unit, and a core encoder. It should be understood that Figure 6a is only an example of the present application, and the encoding end of the present application may include more or fewer modules than shown in Figure 6a, which will not be repeated here.

[0441] Exemplarily, the configuration unit may be configured to determine configuration information of the candidate virtual speakers.

[0442] Exemplarily, the virtual speaker generating unit may be configured to generate a plurality of candidate virtual speakers according to configuration information of the candidate virtual speakers and determine a virtual speaker coefficient corresponding to each candidate virtual speaker.

[0443] Exemplarily, the target speaker generation unit may be configured to select a target virtual speaker from a plurality of candidate virtual speakers and determine attribute information of the target virtual speaker based on a scene audio signal and a plurality of groups of virtual speaker coefficients.

[0444] Exemplarily, the core encoder can be used to obtain the high-order energy gain of the scene audio signal and obtain the high-order energy gain encoding result; encode the attribute information of the first audio signal, the target virtual speaker and the high-order energy gain encoding result in the scene audio signal.

[0445] Exemplarily, the scene audio encoding module in FIG. 1a and FIG. 1b may include the configuration unit, the virtual speaker generation unit, the target speaker generation unit, and the core encoder in FIG. 6a ; or, may include only the core encoder.

[0446] FIG6 b is a schematic diagram showing the structure of a decoding end.

[0447] 6b, illustratively, the decoding end may include a core decoder, a virtual speaker coefficient generation unit, a virtual speaker signal generation unit, a reconstruction unit, and a signal adjustment unit. It should be understood that FIG6b is merely an example of the present application, and the decoding end of the present application may include more or fewer modules than those shown in FIG6b, which will not be further described here.

[0448] Exemplarily, the core decoder may be used to decode the first code stream to obtain a first reconstructed signal, property information of a target virtual speaker, and a high-order energy gain encoding result.

[0449] Exemplarily, the virtual speaker coefficient generation unit may be configured to determine the virtual speaker coefficient based on attribute information of the target virtual speaker.

[0450] Exemplarily, the virtual speaker signal generating unit may be configured to generate a virtual speaker signal based on the first reconstructed signal and the virtual speaker coefficients.

[0451] Exemplarily, the reconstruction unit may be configured to perform reconstruction based on the virtual speaker signal and the attribute information to obtain a first reconstructed scene audio signal.

[0452] Exemplarily, the signal adjustment unit can be used to determine the attenuation factor based on the frequency band number of the reconstructed signal in the first reconstructed scene audio signal and / or the order of the first reconstructed scene audio signal; and adjust the first reconstructed scene audio signal according to the high-order energy gain encoding result and the attenuation factor to obtain the reconstructed scene audio signal.

[0453] Exemplarily, the scene audio decoding module in FIG. 1a and FIG. 1b may include the core decoder, virtual speaker coefficient generation unit, virtual speaker signal generation unit, reconstruction unit and signal adjustment unit of FIG. 6b ; or, may only include the core decoder.

[0454] FIG7 is a schematic diagram illustrating the structure of an exemplary scene audio encoding device. The scene audio encoding device in FIG7 can be used to implement the encoding method of the aforementioned embodiment. Therefore, the beneficial effects achieved by the scene audio encoding device can refer to the beneficial effects of the corresponding method provided above and will not be repeated here. The scene audio encoding device may include:

[0455] An acquisition module 701 is configured to acquire a scene audio signal to be encoded, wherein the scene audio signal includes audio signals of C1 channels, where C1 is a positive integer;

[0456] The acquisition module is further configured to acquire attribute information of a target virtual speaker corresponding to the scene audio signal;

[0457] The acquisition module is further configured to acquire a high-order energy gain of the scene audio signal;

[0458] An encoding module 702 is configured to encode the high-order energy gain to obtain a high-order energy gain encoding result;

[0459] The encoding module is further used to encode the first audio signal in the scene audio signal, the attribute information of the target virtual speaker, and the high-order energy gain encoding result to obtain a first code stream; wherein the first audio signal is the audio signal of K channels in the scene audio signal, and K is a positive integer less than or equal to C1.

[0460] FIG8 is a schematic diagram illustrating the structure of an exemplary scene audio decoding device. The scene audio decoding device in FIG8 can be used to perform the decoding method of the aforementioned embodiment. Therefore, the beneficial effects achieved by the scene audio decoding device can refer to the beneficial effects of the corresponding method provided above and will not be repeated here. The scene audio decoding device may include:

[0461] The code stream receiving module 801 is configured to receive a first code stream;

[0462] a decoding module 802 configured to decode the first bitstream to obtain a first reconstructed signal, attribute information of a target virtual speaker, and a high-order energy gain encoding result, wherein the first reconstructed signal is a reconstructed signal of a first audio signal in a scene audio signal, the scene audio signal includes audio signals of C1 channels, the first audio signal is audio signals of K channels in the scene audio signal, where C1 is a positive integer and K is a positive integer less than or equal to C1;

[0463] A virtual speaker signal generating module 803 is configured to generate a virtual speaker signal corresponding to the target virtual speaker based on the attribute information and the first audio signal;

[0464] a scene audio signal reconstruction module 804 configured to reconstruct the scene audio signal based on the attribute information of the target virtual speaker and the virtual speaker signal to obtain a first reconstructed scene audio signal; the first reconstructed scene audio signal includes audio signals of C2 channels, where C2 is a positive integer;

[0465] an attenuation factor determining module 805, configured to determine an attenuation factor according to a frequency band number of a reconstructed signal in the first reconstructed scene audio signal and / or an order of the first reconstructed scene audio signal;

[0466] The scene audio signal adjustment module 806 is configured to adjust the first reconstructed scene audio signal according to the high-order energy gain coding result and the attenuation factor to obtain a reconstructed scene audio signal.

[0467] In an example, FIG9 shows a schematic block diagram of a device 900 according to an embodiment of the present application. The device 900 may include: a processor 901 and a transceiver / transceiver pin 902 , and optionally, a memory 903 .

[0468] The various components of the device 900 are coupled together via a bus 904, wherein the bus 904 includes, in addition to a data bus, a power bus, a control bus, and a status signal bus. However, for the sake of clarity, all buses are referred to as bus 904 in the figure.

[0469] Optionally, the memory 903 may be used to store instructions in the aforementioned method embodiment. The processor 901 may be used to execute the instructions in the memory 903 and control the receiving pin to receive a signal and control the transmitting pin to send a signal.

[0470] The apparatus 900 may be the electronic device or a chip of the electronic device in the above method embodiment.

[0471] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0472] This embodiment further provides a chip comprising one or more interface circuits and one or more processors. The interface circuits are configured to receive signals from a memory of an electronic device and send signals to the processors, the signals comprising computer instructions stored in the memory. When the processors execute the computer instructions, the electronic device executes the method of the above embodiment. The interface circuits may refer to transceiver 902 in FIG. 9 .

[0473] This embodiment further provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the scene audio encoding and decoding method in the above-mentioned embodiment.

[0474] This embodiment further provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the above-mentioned related steps to implement the scene audio encoding and decoding method in the above-mentioned embodiment.

[0475] This embodiment also provides a device for storing a code stream, which includes: a receiver and at least one storage medium, the receiver is used to receive the code stream; the at least one storage medium is used to store the code stream; the code stream is generated according to the scene audio encoding method in the above embodiment.

[0476] An embodiment of the present application provides a device for transmitting a code stream, comprising: a transmitter and at least one storage medium, wherein the at least one storage medium is used to store the code stream, and the code stream is generated according to the scene audio encoding method in the above embodiment; the transmitter is used to obtain the code stream from the storage medium and send the code stream to the end-side device via the transmission medium.

[0477] An embodiment of the present application provides a system for distributing bitstreams, the system comprising: at least one storage medium for storing at least one bitstream, the at least one bitstream being generated according to the scene audio encoding method of the above embodiment; a streaming media device for obtaining a target bitstream from the at least one storage medium and sending the target bitstream to a terminal device, wherein the streaming media device comprises a content server or a content distribution server.

[0478] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory to enable the chip to execute the scene audio encoding and decoding method in the above-mentioned method embodiments.

[0479] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0480] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0481] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0482] Units described as separate components may or may not be physically separate, and components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0483] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0484] Any content of each embodiment of this application, as well as any content of the same embodiment, can be freely combined. Any combination of the above content is within the scope of this application.

[0485] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0486] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

[0487] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in a hardware manner, or can be implemented by a processor executing a software instruction. The software instruction can be composed of corresponding software modules, and the software module can be stored in a random access memory (Random Access Memory, RAM), a flash memory, a read-only memory (Read Only Memory, ROM), an erasable programmable read-only memory (Erasable Programmable ROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), a register, a hard disk, a mobile hard disk, a read-only compact disc (CD-ROM) or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0488] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer-readable storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0489] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A scene audio decoding method, characterized in that: The method comprises: Receive a first code stream; Decoding the first bit stream to obtain a first reconstructed signal, attribute information of a target virtual speaker, and a high-order energy gain encoding result, wherein the first reconstructed signal is a reconstructed signal of a first audio signal in a scene audio signal, the scene audio signal includes audio signals of C1 channels, the first audio signal is audio signals of K channels in the scene audio signal, C1 is a positive integer, and K is a positive integer less than or equal to C1; Based on the attribute information of the target virtual speaker and the first audio signal, generating a virtual speaker signal corresponding to the target virtual speaker; Reconstructing based on the attribute information of the target virtual speaker and the virtual speaker signal to obtain a first reconstructed scene audio signal; the first reconstructed scene audio signal includes audio signals of C2 channels, where C2 is a positive integer; determining an attenuation factor according to a frequency band number of a reconstructed signal in the first reconstructed scene audio signal and / or an order of the first reconstructed scene audio signal; The first reconstructed scene audio signal is adjusted according to the high-order energy gain encoding result and the attenuation factor to obtain a reconstructed scene audio signal.

2. The method according to claim 1, characterized in that: The scene audio signal is an N1-order high-order stereo reverberation HOA signal, the N1-order HOA signal includes a second audio signal, the second audio signal is an audio signal in the N1-order HOA signal except the first audio signal, C1 is equal to the square of (N1+1); and / or, The first reconstructed scene audio signal is an N2-order HOA signal, the N2-order HOA signal includes a third audio signal, the third audio signal is a reconstructed signal corresponding to each channel of the second audio signal in the N2-order HOA signal, and C2 is equal to the square of (N2+1).

3. The method according to claim 2, characterized in that The adjusting the first reconstructed scene audio signal according to the high-order energy gain encoding result and the attenuation factor to obtain a reconstructed scene audio signal includes: Performing entropy decoding on the high-order energy gain encoding result to obtain the high-order energy gain after entropy decoding; Dequantizing the high-order energy gain after entropy decoding to obtain a high-order energy gain; adjusting the high-order energy gain according to the characteristic information of the second audio signal and the characteristic information of the first audio signal to obtain an adjusted decoded high-order energy gain; The third audio signal in the N2-order HOA signal is adjusted according to the adjusted decoded high-order energy gain and the attenuation factor to obtain an adjusted third audio signal, where the adjusted third audio signal belongs to the reconstructed scene audio signal.

4. The method according to claim 3, characterized in that: The adjusting the high-order energy gain according to the characteristic information of the second audio signal and the characteristic information of the first audio signal includes: Acquire high-order energy of the second audio signal according to the channel energy of the first audio signal and the high-order energy gain; Acquire a decoding energy proportional factor according to the channel energy of the third audio signal and the high-order energy of the second audio signal; acquiring a decoded high-order energy gain of the third audio signal according to the channel energy of the third audio signal and the channel energy of the first audio signal; The decoded high-order energy gain of the third audio signal is adjusted according to the decoded energy proportional factor to obtain the adjusted decoded high-order energy gain.

5. The method according to any one of claims 3 to 4, characterized in that The determining the attenuation factor according to the frequency band number of the reconstructed signal in the first reconstructed scene audio signal and / or the order of the first reconstructed scene audio signal comprises: The attenuation factor is acquired according to the frequency band number of the third audio signal and / or the order of the N2-order HOA signal.

6. The method according to any one of claims 3 to 5, characterized in that After adjusting the third audio signal in the N2-order HOA signal according to the high-order energy gain encoding result and the attenuation factor, the method further includes: Acquire channel energy of a fourth audio signal corresponding to the adjusted third audio signal, where the third audio signal includes an audio signal of a current frame, and the fourth audio signal includes an audio signal of a previous frame of the current frame; The adjusted third audio signal is adjusted again according to the channel energy of the fourth audio signal.

7. The method according to claim 6, characterized in that The adjusting the adjusted third audio signal again according to the channel energy of the fourth audio signal comprises: Acquire a channel energy average value of the fourth audio signal and the adjusted channel energy of the third audio signal; acquiring an energy average threshold according to the channel energy average of the fourth audio signal and the adjusted channel energy of the third audio signal; performing weighted average calculation on the channel energy average value of the fourth audio signal and the adjusted channel energy of the third audio signal according to the energy average threshold to obtain a target energy; Acquire an energy smoothing factor according to the target energy and the adjusted channel energy of the third audio signal; The third audio signal is adjusted according to the energy smoothing factor.

8. The method according to claim 7, characterized in that The acquiring the attenuation factor according to the frequency band number of the third audio signal and / or the order of the N2-order HOA signal includes: The attenuation factor g'(i, b) is obtained as follows: Wherein, i is the number of the i-th channel of the third audio signal, b is the frequency band number of the third audio signal, α represents the number of the target virtual speakers, β represents the number of mapping channels of the N2-order HOA signal, and β is equal to (M+2) 2 , M is the order of the N2-order HOA signal, γ represents the order corresponding to the channel number i of the third audio signal, and × represents a multiplication operation.

9. The method according to claim 8, characterized in that The method further comprises: When b≤d, the α is updated to α2, α2=0.375×α; d is a preset first threshold; When b>d, the α is updated to α3, α3=0.5×α.

10. The method according to claim 8, characterized in that The method further comprises: Update the α to α4, α4=α+α×(1-1.25×b) / (bands+2); Here, bands represents the number of frequency bands of the third audio signal.

11. The method according to any one of claims 8 to 10, characterized in that The method further comprises: When b≤d, the β is updated to β2, β2=(1+w)×β; d is a preset first threshold, and w is a preset adjustment ratio threshold.

12. The method according to any one of claims 8 to 11, characterized in that The method further comprises: Update w to w2, w2=w+α×0.

05.

13. The method according to any one of claims 8 to 12, characterized in that When the value of i is 0, 1, 2, or 3, the value of γ is 1; When the value of i is 4, 5, 6, 7, or 8, the value of γ is 2; When the value of i is 9, 10, 11, 12, 13, 14, or 15, the value of γ is 3.

14. The method according to claim 2, characterized in that The adjusting the first reconstructed scene audio signal according to the high-order energy gain encoding result and the attenuation factor includes: Performing entropy decoding on the high-order energy gain encoding result to obtain the high-order energy gain after entropy decoding; Dequantizing the high-order energy gain after entropy decoding to obtain a high-order energy gain; Acquire high-order energy of the second audio signal according to the channel energy of the first audio signal and the high-order energy gain; Acquire a decoding energy proportional factor according to the channel energy of the third audio signal and the high-order energy of the second audio signal; Determine a dispersion factor of the first reconstructed scene audio signal according to the first bitstream; The attenuation factor is linearly weighted according to the diffusivity factor to obtain a weighted attenuation factor: The third audio signal in the N2-order HOA signal is adjusted according to the weighted attenuation factor and the decoding energy proportional factor to obtain an adjusted third audio signal.

15. The method according to claim 14, characterized in that: The step of linearly weighting the attenuation factor according to the diffusivity factor to obtain a weighted attenuation factor includes: The weighted attenuation factor gd(i,b) is obtained by at least one of the following methods: gd(i,b)=w×diffusion(b)+(1-w)×g'(i,b); wherein w is a preset adjustment ratio threshold, × represents a multiplication operation, diffusion(b) represents a diffusion factor of the b-th frequency band of the third audio signal, and g'(i,b) represents an attenuation factor of the b-th frequency band of the third audio signal; or, When the attenuation factor is a full-band signal, gd(i,b)=w×mean(diffusion)+(1-w)×g'(i,b), wherein mean(diffusion) is an average value of the diffusion factors of multiple frequency bands of the third audio signal, g'(i,b) represents the attenuation factor of the b-th frequency band of the third audio signal, and w is a preset adjustment ratio threshold; or, gd(i,b)=w×diffusion(b)+(1-w)×g'(i,b)+offset(i,b), wherein offset(i,b) represents the bias constant of the bth frequency band on the i-th channel of the third audio signal, g'(i,b) represents the attenuation factor of the bth frequency band of the third audio signal, w is a preset adjustment ratio threshold, and diffusion(b) represents the diffusion factor of the bth frequency band of the third audio signal; or, gd(i,b)=w×diffusion(b)+(1-w)×g'(i,b)+direction(i,b), wherein direction(i,b) represents the direction parameter of the bth frequency band on the i-th channel of the third audio signal, g'(i,b) represents the attenuation factor of the bth frequency band of the third audio signal, w is a preset adjustment ratio threshold, and diffusion(b) represents the diffusion factor of the bth frequency band of the third audio signal.

16. The method according to any one of claims 14 to 15, characterized in that The third audio signal in the N2-order HOA signal is adjusted according to the weighted attenuation factor and the decoding energy proportional factor to obtain an adjusted third audio signal: The adjusted third audio signal X'(i,b) is obtained in the following manner: X'(i,b)=X(i)×g(i,b)×gd(i,b); Wherein, gd(i,b) represents the weighted attenuation factor, g(i,b) represents the decoding energy proportional factor, and X(i) represents the third audio signal.

17. A scene audio decoding device, characterized in that: The device comprises: A code stream receiving module, used for receiving a first code stream; a decoding module, configured to decode the first bit stream to obtain a first reconstructed signal, attribute information of a target virtual speaker, and a high-order energy gain encoding result, wherein the first reconstructed signal is a reconstructed signal of a first audio signal in a scene audio signal, the scene audio signal includes audio signals of C1 channels, the first audio signal is audio signals of K channels in the scene audio signal, C1 is a positive integer, and K is a positive integer less than or equal to C1; A virtual speaker signal generating module, configured to generate a virtual speaker signal corresponding to the target virtual speaker based on the attribute information of the target virtual speaker and the first audio signal; a scene audio signal reconstruction module, configured to reconstruct the scene audio signal based on the attribute information of the target virtual speaker and the virtual speaker signal, To obtain a first reconstructed scene audio signal; the first reconstructed scene audio signal includes audio signals of C2 channels, where C2 is a positive integer; an attenuation factor determination module, configured to determine an attenuation factor according to a frequency band number of a reconstructed signal in the first reconstructed scene audio signal and / or an order of the first reconstructed scene audio signal; The scene audio signal adjustment module is used to adjust the first reconstructed scene audio signal according to the high-order energy gain encoding result and the attenuation factor to obtain a reconstructed scene audio signal.

18. An electronic device, characterized in that: include: a memory and a processor, the memory being coupled to the processor; The memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the scene audio decoding method according to any one of claims 1 to 16.

19. A chip, characterized in that: It comprises one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from a memory of an electronic device and send the signal to the processor, the signal comprising a computer instruction stored in the memory; when the processor executes the computer instruction, the electronic device executes the scene audio decoding method according to any one of claims 1 to claim 16.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program runs on a computer or a processor, the computer or the processor executes the scene audio decoding method as described in any one of claims 1 to 16.

21. A computer program product, characterized in that The computer program product comprises a software program, and when the software program is executed by a computer or a processor, the steps of the method according to any one of claims 1 to 16 are performed.

Citation Information

Patent Citations

  • Decoding method and decoding device

    CN104299614A

  • Energy lossless coding method and device, signal coding method and device, energy lossless decoding method and device, and signal decoding method and device

    CN105723454A

  • Scalable unified audio renderer

    CN111670583A

  • Audio encoding and decoding method and device

    CN114582356A

  • Three-dimensional audio signal encoding method and device and encoder

    CN115376527A