An audio signal processing method, apparatus, device and storage medium

Abstract

The method comprises the following steps: a first terminal acquires a three-dimensional audio signal and performs signal layering on the three-dimensional audio signal to obtain a base layer audio signal and an enhancement layer audio signal; the base layer audio signal is encoded to obtain a base audio code stream, and the enhancement layer audio signal is encoded to obtain an enhancement audio code stream; a compatible code stream format pre-constructed is used to perform stream combining processing on the base audio code stream and the enhancement audio code stream to obtain a target audio code stream; and the target audio code stream is transmitted to a second terminal, so that the second terminal decodes the base audio code stream and the enhancement audio code stream and plays or only decodes the base audio code stream and plays. By using the method, only one audio code stream can be formed, effective encoding of the three-dimensional audio signal is realized, the storage cost and the bandwidth cost of an audio encoding end are reduced, and the encoding stability of an encoding end device is improved.

Description

Technical Field

[0001] This disclosure relates to the field of audio processing technology, and in particular to an audio signal processing method, apparatus, device, and storage medium. Background Technology

[0002] With the continuous evolution of audio technology, 3D audio technology (also known as spatial audio technology) has become a representative cutting-edge technology in the audio field. 3D audio technology breaks the planar limitations of audio, enabling sound to be presented in a three-dimensional spatial form, enhancing the listener's immersion and realism. Its applications span numerous fields. Specifically, 3D audio technology primarily relies on 3D audio signals to present sound.

[0003] In practical applications, traditional audio decoders widely deployed in terminal devices do not support the decoding of 3D audio signals. Therefore, in order to ensure that these devices, which do not support the decoding of 3D audio signals, can still play the audio content involved in 3D audio signals normally, the encoding device, in addition to generating the 3D audio bitstream normally, also needs to separately transcode and store a common stereo bitstream.

[0004] While the above methods ensure the compatibility of terminal devices with 3D audio signal decoding, the additional stereo encoding process increases the performance overhead of the encoding device, potentially leading to issues such as slow encoding speed and overheating, affecting the overall stability of the encoding device and increasing its storage overhead. Furthermore, the encoding device also needs to transmit both the 3D audio stream and the stereo stream simultaneously, thus increasing the additional bandwidth required for transmitting the stereo stream and significantly increasing bandwidth costs. Summary of the Invention

[0005] This disclosure provides an audio signal processing method, apparatus, device, and storage medium, which realizes effective encoding of three-dimensional audio signals, reduces storage overhead and bandwidth costs at the audio encoding end, and improves the encoding stability of the encoding end device.

[0006] In a first aspect, embodiments of this disclosure provide an audio signal processing method applied to a first terminal, the method comprising:

[0007] A three-dimensional audio signal is acquired, and the three-dimensional audio signal is layered to obtain a base layer audio signal and an enhancement layer audio signal. The base layer audio signal is an audio signal in the three-dimensional audio signal that supports the reference codec, and the enhancement layer audio signal is an audio signal in the three-dimensional audio signal that does not support the reference codec.

[0008] The base layer audio signal is encoded to obtain a base audio bitstream, and the enhancement layer audio signal is encoded to obtain an enhancement audio bitstream;

[0009] According to a pre-built compatible bitstream format, the basic audio bitstream and the enhanced audio bitstream are merged to obtain the target audio bitstream. The compatible bitstream format is formed by reusing and expanding the fields in the original bitstream format. The original bitstream format is the bitstream format used by the reference encoding and decoding.

[0010] The target audio stream is transmitted to a second terminal, so that the second terminal can decode the basic audio stream and the enhanced audio stream from the target audio stream according to the decoding capability of the deployed audio decoder for playback, or decode only the basic audio stream from the target audio stream for playback.

[0011] Secondly, embodiments of this disclosure provide an audio signal processing method applied to a second terminal, the method comprising:

[0012] The system receives a target audio bitstream transmitted by a first terminal, wherein the target audio bitstream is formed by the first terminal merging a basic audio bitstream and an enhanced audio bitstream according to a pre-constructed compatible bitstream format. The basic audio bitstream is obtained by encoding a basic layer audio signal, and the enhanced audio bitstream is obtained by encoding an enhanced layer audio signal. The basic layer audio signal and the enhanced layer audio signal are obtained by the first terminal through signal layer processing of the acquired three-dimensional audio signal.

[0013] When the deployed audio decoder only supports basic audio decoding, the target audio bitstream is parsed to obtain the basic audio bitstream, and the basic layer audio signal is formed by restoring the basic audio bitstream for playback;

[0014] When the deployed audio decoder supports 3D audio decoding, the target audio bitstream is parsed to obtain the basic audio bitstream and the enhanced audio bitstream. The target audio signal is then formed by restoring the basic audio bitstream and the enhanced audio bitstream respectively and played. The target audio signal is the 3D audio signal involved in the target audio bitstream.

[0015] Thirdly, embodiments of this disclosure provide an audio signal processing apparatus configured with a first terminal, the apparatus comprising:

[0016] A layering module is used to acquire a three-dimensional audio signal and perform signal layering on the three-dimensional audio signal to obtain a base layer audio signal and an enhancement layer audio signal. The base layer audio signal is an audio signal in the three-dimensional audio signal that supports the reference codec, and the enhancement layer audio signal is an audio signal in the three-dimensional audio signal that does not support the reference codec.

[0017] The encoding module is used to encode the base layer audio signal to obtain a base audio bitstream, and to encode the enhancement layer audio signal to obtain an enhancement audio bitstream;

[0018] The merging module is used to merge the basic audio bitstream and the enhanced audio bitstream according to a pre-built compatible bitstream format to obtain the target audio bitstream. The compatible bitstream format is formed by reusing and expanding the fields in the original bitstream format. The original bitstream format is the bitstream format used by the reference codec.

[0019] The transmission module is used to transmit the target audio bitstream to a second terminal, so that the second terminal can decode the basic audio bitstream and the enhanced audio bitstream from the target audio bitstream according to the decoding capability of the deployed audio decoder for playback, or decode only the basic audio bitstream from the target audio bitstream for playback.

[0020] Fourthly, embodiments of this disclosure also provide an audio signal processing device configured in a second terminal, the device comprising:

[0021] The receiving module is used to receive the target audio bitstream transmitted by the first terminal. The target audio bitstream is formed by the first terminal merging the basic audio bitstream and the enhanced audio bitstream according to a pre-constructed compatible bitstream format. The basic audio bitstream is obtained by encoding the basic layer audio signal, and the enhanced audio bitstream is obtained by encoding the enhanced layer audio signal. The basic layer audio signal and the enhanced layer audio signal are obtained by the first terminal through signal layer processing of the acquired three-dimensional audio signal.

[0022] The first decoding module is used to parse the target audio bitstream to obtain the basic audio bitstream when the deployed audio decoder only supports basic audio decoding, and to form the basic layer audio signal for playback by restoring the basic audio bitstream.

[0023] The second decoding module is used to parse the target audio bitstream to obtain the basic audio bitstream and the enhanced audio bitstream when the deployed audio decoder supports three-dimensional audio decoding, and to play the target audio signal by restoring the basic audio bitstream and the enhanced audio bitstream respectively. The target audio signal is the three-dimensional audio signal involved in the target audio bitstream.

[0024] Fifthly, embodiments of this disclosure also provide a computer device, the computer device comprising:

[0025] One or more processors;

[0026] Storage device for storing one or more programs.

[0027] When the one or more programs are executed by the one or more processors, the one or more processors implement the audio signal processing method provided in any embodiment of this disclosure.

[0028] Sixthly, embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the audio signal processing method provided in any embodiment of this disclosure.

[0029] In a seventh aspect, embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements the audio signal processing method provided in any embodiment of this disclosure.

[0030] The technical solution of this disclosure specifically discloses an audio signal processing method, apparatus, device, and storage medium. The method first acquires a three-dimensional audio signal and then performs signal layering on the three-dimensional audio signal to obtain a base layer audio signal and an enhancement layer audio signal. The base layer audio signal is an audio signal in the three-dimensional audio signal that supports a reference codec, and the enhancement layer audio signal is an audio signal in the three-dimensional audio signal that does not support the reference codec. The method then encodes the base layer audio signal to obtain a base audio bitstream, and encodes the enhancement layer audio signal to obtain enhanced audio. The system merges the base audio stream and the enhanced audio stream according to a pre-constructed compatible bitstream format to obtain a target audio stream. The compatible bitstream format is formed by reusing and expanding fields in the original bitstream format, which is the bitstream format used for the baseline encoding and decoding. The target audio stream is then transmitted to a second terminal, which decodes the base audio stream and the enhanced audio stream from the target audio stream based on the decoding capabilities of its deployed audio decoder, or decodes only the base audio stream from the target audio stream for playback. This embodiment considers layering the audio signals in a three-dimensional audio signal and encoding the layered audio signals independently to obtain the base audio stream of the base layer audio signal and the enhanced audio stream of the enhanced layer audio signal. The base audio stream and the enhanced audio stream can then be merged according to the constructed compatible bitstream format to form a single target audio stream. The resulting target audio stream is adaptable to different decoding scenarios, enabling flexible decoding of the target audio stream by the audio decoder. Compared to existing technologies, the technical solution described in this embodiment only generates one audio bitstream after encoding, which is then sent to the audio decoding end. This enables compatible decoding of the encoding end device, allowing for either traditional basic audio decoding or complete three-dimensional audio decoding. This processing method significantly reduces the storage and bandwidth costs of the audio bitstream for the audio encoding end device, and improves the encoding stability of the audio encoding end. Simultaneously, the compatible decoding of the target audio bitstream by the audio decoding end is better adapted to the decoding characteristics of different audio decoding ends, ensuring the effective advancement of the audio decoding logic. Attached Figure Description

[0031] To more clearly illustrate the technical solutions of the exemplary embodiments of this disclosure, the accompanying drawings used in describing the embodiments are briefly introduced below. Obviously, the accompanying drawings described are only a portion of the embodiments to be described in this disclosure, and not all of them. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0032] Figure 1aA schematic flowchart of an audio signal processing method provided in an embodiment of this disclosure;

[0033] Figure 1b A format illustration diagram of the original bitstream format involved in the construction of a compatible bitstream format in the audio signal processing method provided in the embodiments of this disclosure;

[0034] Figure 1c A format illustration of the compatible bitstream format constructed in the audio signal processing method provided in the embodiments of this disclosure;

[0035] Figure 2 A schematic flowchart of an audio signal processing method provided in an embodiment of this disclosure;

[0036] Figure 3 This is a schematic diagram of the structure of an audio signal processing device provided in an embodiment of the present disclosure;

[0037] Figure 4 This is a schematic diagram of the structure of an audio signal processing device provided in an embodiment of the present disclosure;

[0038] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0039] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0040] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0041] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0042] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules, or units, and are not used to limit the order of functions performed by these devices, modules, or units or their interdependencies. It should also be noted that the modifications of "a" and "a plurality of" mentioned in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0043] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0044] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0045] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0046] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0047] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0048] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0049] Figure 1a This is a flowchart illustrating an audio signal processing method provided in an embodiment of the present disclosure. This embodiment is applicable to the processing of three-dimensional audio signals. The method can be executed by an audio signal processing device, which can be implemented by software and / or hardware and can be configured in a terminal and / or server to implement the audio signal processing method in this embodiment of the present disclosure.

[0050] like Figure 1a As shown, the audio signal processing method provided in this embodiment may include:

[0051] It should be noted that the execution terminal of the audio signal processing method provided in this embodiment can be considered as a first terminal involving audio encoding operations, and the first terminal can preferably be a server or terminal related to audio services.

[0052] S101. Acquire a three-dimensional audio signal and perform signal layering on the three-dimensional audio signal to obtain a base layer audio signal and an enhancement layer audio signal. The base layer audio signal is an audio signal in the three-dimensional audio signal that supports the reference codec, and the enhancement layer audio signal is an audio signal in the three-dimensional audio signal that does not support the reference codec.

[0053] In this embodiment, the three-dimensional audio signal can be considered as an audio signal formed using three-dimensional audio technology. This execution terminal can acquire the generated three-dimensional audio signal in real time, or it can acquire the three-dimensional audio signal to be encoded from the audio signal resource pool.

[0054] In this embodiment, the three-dimensional audio signal formed by three-dimensional audio technology can have different signal format types, and it can be considered that the representation of the three-dimensional audio signal is different under different signal format types.

[0055] In this embodiment, the base layer audio signal can be considered as an audio signal that can be encoded and decoded using traditional audio codec methods, i.e., audio information that supports reference audio codecs, such as common stereo audio signals and surround sound audio signals, which can all serve as base layer audio signals. The enhancement layer audio signal can be considered as an audio signal that requires encoding and decoding using specific audio codec methods, such as planar sound field signals in three-dimensional audio signals that do not support reference audio codecs, and height sound field signals in three-dimensional audio signals, which can all serve as enhancement layer audio signals. Reference audio codecs can be understood as the codec methods used to encode and decode conventional two-dimensional audio signals, commonly including advanced audio codecs or enhanced advanced audio codecs.

[0056] In this embodiment, the three-dimensional audio signal can be considered to be represented in the form of signal channels, and each signal channel contains the channel signals that constitute the three-dimensional audio signal. Therefore, the signal layering processing of the three-dimensional audio signal in this embodiment is specifically achieved by layering the channel signals in different signal channels. Specifically, the basic enhanced audio signal and the enhanced layer audio signal can be considered to originate from different signal channels corresponding to the three-dimensional audio signal. Therefore, this embodiment can locate the channel signals in different signal channels and classify them as either basic layer audio signals or enhanced layer audio signals.

[0057] It is also known that different signal formats of 3D audio signals have different channel representations. Therefore, when performing signal layering based on signal channels, the specific implementation of signal layering for different signal formats of 3D audio signals can be considered different. Thus, this step can set different signal layering processing methods according to different signal format types. Then, based on the different signal layering processing methods, the 3D audio signal is divided into a base layer audio signal and an enhancement layer audio signal.

[0058] In this embodiment, the signal format of the three-dimensional audio signal can be either omnidirectional pickup or multi-channel pickup, and the signal channel representation of the three-dimensional audio signal can be considered different under the two different signal format types. For omnidirectional pickup, the three-dimensional audio signal can have different numbers of signal channels depending on the signal order, which is equivalent to having (N+1) squared signal channels. For example, when the signal order is 3, the three-dimensional audio signal has 16 signal channels. In addition, for the channel signal in each signal channel, the signal subscript can be represented by the signal order and the degree value under the signal order.

[0059] In this omnidirectional pickup type, one implementation of signal layering processing can be described as follows: the signal indices of the channel signals contained in the signal channel can be used to divide the channel signals as the base layer audio signal and the channel signals as the enhancement layer audio signal. Specifically, the signal indices can be composed of the signal level and the frequency value of the signal level. In this embodiment, the base layer audio signal can be determined based on the channel signal with a signal level of 0 and the channel signal with a signal level of 1 and a frequency value at the first level of that level. At the same time, the enhancement layer audio signal can be determined starting from the channel signal with a frequency value at the second level of that level. For example, the enhancement layer audio signal can be further subdivided into planar enhancement signals and height layer audio signals. Based on the sum of the signal level and the frequency value, the planar enhancement signal and the height layer audio signal can be determined from the channel signals after the second level of that level. Finally, the determined planar enhancement signal and height layer audio signal are combined as the enhancement layer audio signal.

[0060] For multi-channel audio pickup, the number of signal channels in a three-dimensional audio signal is basically fixed, and the distribution of channel signals in different sound fields is also basically fixed. A common example is a channel signal sequence in the form of 7.1.4, where 7 can represent 7 planar sound field signal channels, 1 bass sound field signal channel, and 4 height sound field signal channels. Correspondingly, the channel signals in different signal channels also have the sound field characteristics of various signal channels.

[0061] In this multi-channel audio pickup type, one implementation of signal layering processing can be described as follows: After knowing the number of signal channels in the three-dimensional audio signal, the number of signal channels corresponding to different sound fields can be further determined. Then, based on the encoding method to be used for the base layer audio signal, the audio mixing matrix to be selected to participate in the determination of the base layer audio signal can be determined. The number of signal channels occupied by the base layer audio signal is also determined by the encoding method to be used. After determining the base layer audio signal, the channel signals in the remaining signal channels can be divided into enhancement layer audio signals. Similarly, in this multi-channel audio pickup type, the enhancement layer audio signal can also be subdivided. In specific subdivision, based on the sound field characteristics of each channel, the number of planar enhancement signals and the number of height layer audio signals can be determined. Finally, the determined planar enhancement signals and height layer audio signals can be combined into the enhancement layer audio signal.

[0062] It should be noted that the enhanced audio signal can also be spatial sound field parameter metadata and metadata under three degrees of freedom. Specifically, in low bitrate encoding environments, the relevant spatial generation parameter information can be determined through analysis of the three-dimensional audio signal to serve as the enhanced audio signal for the three-dimensional audio signal.

[0063] S102. Encode the base layer audio signal to obtain a base audio bitstream, and encode the enhancement layer audio signal to obtain an enhancement audio bitstream.

[0064] In this embodiment, the base layer audio signal can be considered to be either stereo encoding or surround sound encoding. The specific choice between stereo and surround sound encoding can be determined in advance based on the decoding capabilities of the client involved in the audio service. When the client has surround sound decoding capabilities, this embodiment can perform surround sound encoding on the base layer audio signal. Similarly, when the client does not have surround sound decoding capabilities, it is assumed to have stereo decoding capabilities, and this embodiment can perform stereo encoding on the base layer audio signal.

[0065] In this embodiment, the enhancement layer audio signal can be considered to be specifically subdivided into planar enhancement signal and height layer audio signal. In this embodiment, the planar enhancement signal in the enhancement layer audio signal is actually coded in a planar layer and the height layer audio signal is coded in a height layer.

[0066] It should be noted that, regardless of the encoding method used in this embodiment, the encoded result will be a bitstream based on '0' and '1'. Specifically, after stereo encoding or surround sound encoding of the base layer audio signal, the bitstream corresponding to that base layer audio signal can be obtained. In this embodiment, this bitstream is preferably referred to as the base audio bitstream. Similarly, after performing planar layer encoding and / or height layer audio encoding on the enhancement layer audio signal, the bitstream corresponding to that enhancement layer audio signal can be obtained. In this embodiment, this bitstream is preferably referred to as the enhancement audio bitstream.

[0067] S103. According to the pre-constructed compatible bitstream format, the basic audio bitstream and the enhanced audio bitstream are merged to obtain the target audio bitstream. The compatible bitstream format is formed by reusing and expanding the fields in the original bitstream format. The original bitstream format is the bitstream format used by the reference encoding and decoding.

[0068] It should be noted that, to ensure that the audio decoding end can decode 3D audio signals using only one audio stream, or only decode conventional stereo audio signals and surround sound audio signals, it is necessary to merge the basic audio stream and the enhanced audio stream formed by audio encoding. Compared to the existing audio processing method that first merges the 3D audio stream according to the 3D stream format (directly encoding the 3D audio signal to form one 3D audio stream), and then encodes the audio signal into another conventional audio stream according to the traditional encoding format, and finally transmits the two streams, the method provided in this embodiment only needs to perform one merging process on the 3D audio signal to use one audio stream for the transmission of the 3D audio signal. This allows the audio decoding end to either decode only the basic audio stream from the target audio stream for playback of the basic audio signal, or decode both the basic audio stream and the enhanced audio stream from the target audio stream to play the 3D audio signal.

[0069] As described above, in this embodiment, it is equivalent to compatiblely merging the basic audio bitstream and the enhanced audio bitstream, so that the merged audio bitstream can be decoded into the basic layer audio signal through traditional decoding methods, and can also be decoded into the basic layer audio signal and the enhanced layer audio signal constituting the three-dimensional audio signal through three-dimensional audio decoding methods.

[0070] Specifically, this embodiment can merge the basic audio stream and the enhanced audio stream according to a pre-built compatible bitstream format, ultimately forming the target audio stream. The compatible bitstream format can be obtained by multiplexing and expanding fields of a general bitstream format. The general bitstream format can be considered the bitstream format of traditional audio encoding, which in this embodiment can be referred to as the original bitstream format. The fields involved in the original bitstream format can be header information fields and data payload fields. The header information field contains audio encoding / decoding control information, such as sampling rate and channel layout, while the data payload field is used to write the encoded audio bitstream, thus forming an audio bitstream that can be sent to the audio decoder.

[0071] Based on reusing the header information field and data payload field included in the original bitstream format, this embodiment additionally adds an extended header information field and an extended data payload field. Specifically, the extended header information field can be inserted between the header information field and the data payload field, and the extended data payload field can be added after the data payload field. The extended header information field can contain audio layout information related to the three-dimensional audio layout, supplementing the information in the header information field; the extended data payload field can be used to write the enhanced audio bitstream after encoding the enhanced layer audio signal. In this embodiment, after completing the writing of the basic audio bitstream and the enhanced audio bitstream, the merging of the audio encoding information can be considered complete, and the final audio bitstream can be denoted as the target audio bitstream.

[0072] In this embodiment, the compatible bitstream format formed in the above manner is equivalent to including an audio header information field, an extended header information field, a data payload field, and an extended data payload field. Specifically, the header information field can be considered to carry the basic attribute information of the base layer audio signal in the encoded 3D audio signal, which may include the sampling rate and the channel layout of the base audio signal. The extended header information field can be considered to supplement the basic attribute information in the header information field, and may carry the channel layout parameters of the complete 3D audio. The data payload field can be used to write the base audio bitstream corresponding to the base layer audio signal, and the extended data payload field can be used to write the enhanced audio bitstream corresponding to the enhancement layer audio signal.

[0073] It should be noted that during the merging process using a compatible bitstream format, apart from the header information field, the remaining extended header information field, data payload field, and extended data payload field all use control fields and encoded payloads to write information. Specifically, the control field can be considered to carry information such as the payload type and payload length related to the encoded payload. The encoded payload can be considered to carry the corresponding bitstream.

[0074] S104. The target audio bitstream is transmitted to the second terminal, so that the second terminal can decode the basic audio bitstream and the enhanced audio bitstream from the target audio bitstream according to the decoding capability of the deployed audio decoder for playback, or decode only the basic audio bitstream from the target audio bitstream for playback.

[0075] As is known, in the implementation of audio services, the encoded audio stream needs to be transmitted to a decoding terminal for decoding in order to play the audio signal. In this embodiment, the second terminal can be regarded as the decoding terminal of the audio stream, and the target audio stream can be transmitted to the second terminal based on this step so that the second terminal can decode the target audio stream.

[0076] It should be noted that, as described above in this embodiment, the decoding terminal involved in audio decoding may be limited by its decoding capabilities and can only perform decoding of general audio bitstreams, or it may have better hardware performance and decoding capabilities, thereby enabling the decoding of three-dimensional audio signals and thus obtaining three-dimensional audio signals.

[0077] Based on this, this embodiment considers layering the three-dimensional audio signal through signal layering. The resulting bitstream includes a basic audio bitstream and an enhanced audio bitstream. The resulting basic and enhanced audio bitstreams can be merged and packaged into a single bitstream as the target audio bitstream. Although a single target audio bitstream is formed, it actually contains both the basic and enhanced audio bitstreams. Therefore, during the decoding stage, the basic and enhanced audio bitstreams contained in the target audio bitstream can be decoded separately.

[0078] Based on the above, after receiving the target audio bitstream that can be decoded separately, the second terminal can determine, according to its own decoding capabilities, whether to perform only basic audio bitstream decoding or also enhance audio bitstream decoding. Thus, after determining the specific decoding method through the above approach, the second terminal can either decode only the basic layer audio signal or decode both the basic layer and enhance layer audio signals. Finally, by restoring the basic layer and enhance layer audio signals, the three-dimensional audio signal from the audio encoding end can be obtained.

[0079] This embodiment provides an audio processing method that considers layering the audio signals in a three-dimensional audio signal and independently encoding each layered audio signal to obtain a base audio bitstream of the base layer and an enhanced audio bitstream of the enhancement layer. The base and enhanced audio bitstreams are then merged according to a constructed compatible bitstream format to form a single target audio bitstream. This target audio bitstream is adaptable to different decoding scenarios, enabling flexible decoding by the audio decoder. Compared to existing technologies, this embodiment's solution only generates one audio bitstream for the audio decoder, achieving compatible decoding by the encoding device. It can perform either traditional basic audio decoding or complete three-dimensional audio decoding. This processing method significantly reduces the storage and bandwidth costs of the audio encoding device and improves the encoding stability. Simultaneously, the compatible decoding of the target audio bitstream by the audio decoder better adapts to the decoding characteristics of different audio decoders, ensuring the effective advancement of the audio decoding logic.

[0080] As a first optional embodiment of this disclosure, based on the above embodiments, the signal layering of the three-dimensional audio signal to obtain the base layer audio signal and the enhancement layer audio signal can be specified as follows:

[0081] a1) When the signal format type of the three-dimensional audio signal is omnidirectional pickup type, the first signal channel included in the three-dimensional audio signal is determined according to the signal order of the three-dimensional audio signal, and the first channel signal and the signal label of the first channel signal in each first signal channel are obtained.

[0082] This optional embodiment provides a layered processing method for three-dimensional audio signals when the signal format type is omnidirectional pickup. As described above, the signal channel format used to represent three-dimensional audio signals under omnidirectional pickup is determined by the signal order.

[0083] This embodiment uses a Higher Order Ambisonics (HOA) audio signal from the omnidirectional pickup type as an example to specifically describe the layering of a three-dimensional audio signal. For an Nth-order HOA signal, it can have N+1 squared signal channels. In this embodiment, the determined signal channels are denoted as the first signal channels included in the three-dimensional audio signal. The channel signal in each first signal channel of the HOA audio signal can be denoted as the first channel signal, and each first channel signal can be expressed as a spherical harmonic basis function, which can be represented as ACN. k =a n,m (.), where k can be considered as the channel number of the first channel signal, satisfying 0≤k<(N+1).2 , n and m can be regarded as the signal labels of the first channel signal.

[0084] b1) According to each of the signal labels, perform hierarchical conversion processing on each of the first channel signals to obtain the first base layer audio signal and the first enhancement layer audio signal of the three-dimensional audio signal.

[0085] In this embodiment, signal stratification can be performed on the three-dimensional audio signal of the omnidirectional pickup type through signal labels. Specifically, each first channel signal constituting the three-dimensional audio signal corresponds to a signal label. Through this signal label, it can be determined which signal layer of the three-dimensional audio signal the first channel signal belongs to, and which order value it belongs to in this signal layer. Thus, the first channel signal can be divided into a base layer audio signal, a planar enhancement signal, and a height layer audio signal according to the specific signal layer and order value. The planar enhancement signal and the height layer audio signal can be collectively referred to as the enhancement layer audio signal.

[0086] Based on this first alternative embodiment, the signal label can be preferably a two-digit number composed of the signal layer under the signal order and the order value under the corresponding signal layer. Taking the HOA signal as an example, for any first channel signal represented by ACN k = a nm (.), in nm representing the signal label, n represents the signal layer under the signal order, satisfying 0 ≤ n < N, and m is the order value under the signal layer of n, satisfying -n ≤ m ≤ n.

[0087] For a HOA signal with a signal order of 4, its first channel signals can be sequentially represented as:

[0088]

[0089] Based on the above optimization, the process of performing hierarchical conversion processing on each of the first channel signals according to each of the signal labels to obtain the first base layer audio signal and the first enhancement layer audio signal of the three-dimensional audio signal can be further specified as:

[0090] b11) Obtain the first channel signal with a signal layer of 0 order and the first first channel signal with a signal layer of 1 order in the signal label and form a channel signal matrix.

[0091] Continuing with the above exemplary description related to the HOA signal with a signal order of 4, the first channel signal (ACN0, a 0,0 ) with a signal layer of 0 order and the first first channel signal (ACN1, a 1,-1 ) with a signal layer of 1 order can be obtained, and these two first channel signals can be used to form a channel signal matrix

[0092] b12) Based on the audio conversion matrix corresponding to the omnidirectional pickup type, process the first channel signal matrix to obtain the first basic signal matrix, and determine the element signals in the first basic signal matrix as the first basic layer audio signal.

[0093] In this embodiment, W can be hoa The audio conversion matrix, which is considered to be the corresponding omnidirectional pickup type, can be multiplied by the channel signal matrix. The resulting matrix can be used as the first basic signal matrix in this embodiment.

[0094] For example, The element signals L and R in this first basic signal matrix can be used as the first basic layer audio signals in this embodiment. It is known that the first basic signal matrix... L and R in the stereo signal can represent the left channel signal and the right channel signal, respectively.

[0095] b13) Starting from the next first channel signal with a signal level of 1, determine the first enhancement layer audio signal according to the signal level and number value in the signal label of the corresponding first channel signal.

[0096] Following the above exemplary description of a HOA signal with a signal order of 4, this step can start from the next first channel signal (ACN2, a) with a signal order of 1. 1,0 Starting with (ACN2, a) 1,0 ) to (ACN) 15 a 3,3 All first channel signals can be used to determine the enhancement layer audio signal based on their respective signal labels, and the determined enhancement layer audio signal can be recorded as the first enhancement layer audio signal.

[0097] The technical solution described in this embodiment provides a signal layering processing implementation for omnidirectional sound pickup type three-dimensional audio signals. This layering method can divide the three-dimensional audio signal into a base layer audio signal and an enhancement layer audio signal, providing basic data support for subsequent layered encoding.

[0098] As a second optional embodiment of this disclosure, based on the above embodiments, the signal layering of the three-dimensional audio signal to obtain the base layer audio signal and the enhancement layer audio signal can be specified as follows:

[0099] a2) When the signal format type of the three-dimensional audio signal is a multi-channel pickup type, obtain a multi-channel signal sequence representing the three-dimensional audio signal, wherein the multi-channel signal sequence includes multiple second channel signals.

[0100] This optional embodiment provides a signal layering process for three-dimensional audio signals when the signal format type is multi-channel pickup. As can be seen from the above description, the channels involved in representing three-dimensional audio signals under multi-channel pickup are pre-designed, and the distribution and number of channels are specifically set during the design phase.

[0101] For example, in the multi-channel audio signal 7.1.4, 7, 1 and 4 are the channels involved in the multi-channel audio signal. 7 can be considered as having 7 planar sound field channels, 1 can be considered as a bass sound field channel and 4 can be considered as a height sound field channel.

[0102] This embodiment can obtain a multi-channel signal sequence formed according to a pre-designed multi-channel structure. The multi-channel signal sequence includes multiple channel signals, which are referred to as the second channel signal in this embodiment.

[0103] b2) Based on each of the second channel signals, determine the second base layer audio signal and the second enhancement layer audio signal of the three-dimensional audio signal.

[0104] In this embodiment, analyzing the attribute characteristics of the base layer audio signal reveals that it corresponds to either a stereo audio signal or a surround sound audio signal. Analyzing the attribute characteristics of the enhancement layer audio signal reveals that it corresponds to both planar sound field audio signals and height sound field audio signals. Therefore, the signal layering of the three-dimensional audio signal in this embodiment is equivalent to dividing the three-dimensional audio signal into stereo or surround sound audio signals that can be decoded by the audio decoder, and filtering out planar and height layer audio signals that the audio decoder does not support decoding.

[0105] In this embodiment, the base layer audio signal and the enhancement layer audio signal can be obtained directly from the second channel signals in the multi-channel signal sequence involved in the three-dimensional audio signal.

[0106] Specifically, based on this second optional embodiment, as one implementation method, the determination of the second base layer audio signal and the second enhancement layer audio signal of the three-dimensional audio signal according to each of the second channel signals can be further optimized by the following steps:

[0107] b21) Based on the set audio signal downmixing matrix, perform signal mixing processing on all second channel signals to generate a second basic signal matrix, and determine the element signals in the second basic signal matrix as the second basic layer audio signals.

[0108] It should be noted that when the signal format is multi-channel audio pickup, stereo audio signals or surround sound audio signals can be separated from the three-dimensional audio signal to serve as the base layer audio signal. The difference between stereo audio signals and surround sound audio signals is that stereo audio signals have two channels, left and right, while surround sound audio signals have more than two channels.

[0109] Therefore, the number of channels required for the base layer audio signal differs depending on whether a stereo audio signal or a surround sound audio signal is used as the base layer audio signal. This embodiment considers using different audio signal downmixing matrices to participate in the division of the base layer audio signal, in order to obtain a stereo audio signal with two channels, or a surround sound audio signal with more than two channels.

[0110] The specific audio signal downmixing matrix used can be preset, depending on the decoding format supported by the client of the audio service involved. For example, if the client supports surround sound decoding, an audio signal downmixing matrix supporting surround sound layering can be set on the execution terminal side. Similarly, if the client supports stereo decoding, an audio signal downmixing matrix supporting stereo layering can be set on the execution terminal side.

[0111] This step involves dividing the base layer audio signal based on the pre-defined downmixing matrix of the audio signal. Specifically, this step obtains all the second-channel signals in the multi-channel signal sequence. For example, the multi-channel signal sequence can be represented as: Q L (t)=[q1(t)q2(t)......q L (t)] T When the set audio signal downmixing matrix is ​​W 2L In this case, the basic layer audio signal to be divided can be considered as a stereo audio signal.

[0112] At this point, W can be calculated. 2L With Q L The product matrix of (t), which is represented as and This embodiment can use the product matrix It is determined as the second fundamental signal matrix, and can be used as... In The signal is identified as an element in the second fundamental signal matrix. It can be seen that... It contains two element signals, which can be used as the base layer audio signal represented by stereo audio signals.

[0113] As another implementation, Dangdang sets the audio signal downmixing matrix to W. ML In this case, the basic layer audio signal to be divided can be considered as a surround sound audio signal. At this point, W can be calculated. ML With Q L The product matrix of (t), which is represented as and This embodiment can use the product matrix It is determined as the second fundamental signal matrix, and can be used as... Determined as the second fundamental signal matrix It includes multiple element signals. It can be seen that... It contains M element signals, where M is greater than 2, representing the number of surround sound channels. These M element signals can be used as the base layer audio signal characterized by surround sound audio signals. In this embodiment, the base layer audio signal determined in this step can be denoted as the second base layer audio signal.

[0114] It should be noted that this embodiment uses all the second-channel signals from the multi-channel signal sequence to participate in the determination of the base layer audio signal. This is equivalent to using the entire three-dimensional audio signal to participate in the determination of the base layer audio signal. This method ensures that even if the decoding terminal does not support the decoding of the three-dimensional audio signal, it can still obtain all the audio content represented by the three-dimensional audio signal when decoding only the base layer audio signal. This better guarantees the integrity of the audio content involved in the three-dimensional audio signal.

[0115] b22) Based on the number of signals in the second base layer audio signal, determine the second signal to be divided from the multi-channel signal sequence, and determine the second enhancement layer audio signal based on the signal attributes of the second signal to be divided, wherein the number of signals is the same as the number of rows in the audio signal downmixing matrix.

[0116] Analysis shows that the total number of channel signals in the multi-channel pickup type is a fixed value set in advance. The number of channel signals in the enhanced layer audio signal is equivalent to the total number of channel signals minus the number of signals in the second base layer audio signal.

[0117] In this embodiment, when determining the enhanced audio signal, the signal to be divided can be selected from the next channel signal with the specific value related to the number of signals in the multi-channel signal sequence, and this signal to be divided can be recorded as the second signal to be divided. For example, assuming the number of signals in the second base layer audio signal is 2, the second signal to be divided can be selected starting from the third channel signal in the multi-channel signal sequence. In this embodiment, the number of signals in the second base layer audio signal can be the same as the number of rows in the set audio signal downmixing matrix. If the number of rows in the audio signal downmixing matrix is ​​M, then the number of signals in the second base layer audio signal is also M, where M can be a value greater than 2 and less than the total number of channels.

[0118] It is also known that the signal attributes of each channel signal in the multi-channel signal sequence are predetermined in the design stage. The signal attributes may include planar signals and height signals. In this embodiment, the signal attributes of each second signal to be divided can be used to divide the signal into a planar enhancement signal of the planar layer or a height enhancement signal of the height layer.

[0119] The technical solution described in this embodiment provides a signal layering processing implementation for multi-channel pickup-type three-dimensional audio signals. This layering method can also divide the three-dimensional audio signal into a base layer audio signal and an enhancement layer audio signal, providing basic data support for subsequent layered encoding.

[0120] As a third optional embodiment of this example, based on the above embodiments, the construction steps for the compatible bitstream format can be specified as follows:

[0121] a3) Obtain the original bitstream format used by the reference codec, wherein the original bitstream format includes: audio transport header field and original audio payload field.

[0122] In this embodiment, the original bitstream format can be considered as the bitstream format used by a general audio codec (such as Advanced Audio Codec and Enhanced Advanced Audio Codec) for baseline encoding and decoding. This step can obtain the original bitstream format of this general encoding, which includes an audio transport header field and a raw audio payload field. The audio transport header field can be considered as carrying header information related to audio transmission, while the raw audio payload field can be used to carry the encoded audio bitstream. The compatible bitstream format used in this embodiment can be considered as an extension of the fields in the original bitstream format. This embodiment does not limit the specific extension length.

[0123] For example, Figure 1b This diagram illustrates the original bitstream format involved in constructing a compatible bitstream format in the audio signal processing method provided in this embodiment. For example... Figure 1bAs shown, the format of the raw bitstream corresponding to general-purpose audio encoders such as advanced audio encoders is used as an example. The raw bitstream format specifically includes two fields: audio header field 10 and raw audio payload field 11. The raw audio payload field 11 contains the payload content based on the payload flag bit and audio payload information.

[0124] b3) Insert a set audio extension header field between the audio transmission header field and the original audio payload field. The audio extension header field is used to write the audio layout information of the three-dimensional audio signal.

[0125] In this embodiment, the extension of the original bitstream format can be achieved by first inserting a defined audio extension header field between the audio transport header field and the original audio payload field. Attribute information involved in the three-dimensional audio signal encoding and decoding, such as channel layout parameters, can be filled into this audio extension header field to serve as the audio layout information for the three-dimensional audio signal.

[0126] c3) Add an extended audio load field after the original audio load field, the extended audio load field being used to fill the enhanced audio bitstream.

[0127] It is understandable that, in addition to adding an audio extension header field to carry the attribute information involved in the encoding and decoding of the 3D audio signal, a payload field also needs to be added to fill in the encoded data of the enhancement layer audio signal divided from the 3D audio signal. The encoded data can be the enhanced audio bitstream. In this embodiment, an extended audio payload field is added after the original audio payload field.

[0128] It should be noted that, in this embodiment, as an audio extension header field and an extended audio payload field are added to the original bitstream format, the overall field length will increase accordingly. This embodiment does not specify the exact length of the added fields. It can be dynamically adjusted according to the size of the information carried.

[0129] d3) Based on the audio transmission header field, audio extension header field, original audio payload field, and extended audio payload field, the compatible bitstream format is constructed.

[0130] In this embodiment, the bitstream format, including the audio transport header field, audio extension header field, original audio payload field, and extended audio payload field, can be directly determined as a compatible bitstream format. Specifically, the audio transport header field carries basic attribute information of the 3D audio signal, such as attribute information of the base layer audio signal; the audio extension header field carries channel layout parameters of the complete 3D audio signal; the original audio payload field carries the base audio bitstream corresponding to the base layer audio signal; and the extended audio payload field carries the enhanced audio bitstream corresponding to the enhancement layer audio.

[0131] Furthermore, to better describe the field information carried in each field, this embodiment can refine the field structure of the audio extension header field, the original audio payload field, and the extended audio payload field. Specifically, it can optimize the inclusion of control fields and encoding payloads in the audio extension header field, the original audio payload field, and the extended audio payload field. Each control field carries attribute information of the encoding payload, such as descriptions of payload type and payload length.

[0132] As described above, for the extended audio payload field, it can also be divided into subfields according to the specific attribute information to be written to the enhanced audio bitstream. For example, the extended audio payload field can preferably include multiple extended audio payload subfields. Thus, for each extended audio payload subfield, a control field and encoding payload can be set separately. The set control field is used to carry the payload attributes of the corresponding audio payload, which can also be payload type and payload length, etc.

[0133] For example, Figure 1c This diagram illustrates an example of a compatible bitstream format constructed in an audio signal processing method provided in this embodiment of the disclosure. Figure 1c As shown, based on reusing the original bitstream format, the compatible bitstream format can be divided into four fields: audio header field 10, audio extension header field 12, original audio payload field 11, and extended audio payload field 13. The audio extension header field 12 includes an extension header control field 121 and an extension header payload field 122. The extension header control field 121 includes payload flags, payload type, and payload length related to the extension header payload. Furthermore, the extended audio payload field 13 can include multiple subfields, each of which can include a corresponding sub-control field and a sub-audio payload field. Specifically, each sub-control field can include payload flags, payload type, and payload length related to the sub-audio payload.

[0134] The technical solution described in this embodiment presents a construction and implementation of a compatible bitstream format for merging multiple audio bitstreams into a single audio bitstream. The constructed compatible bitstream format enables the merging of the base layer audio signal and the enhancement layer audio signal after independent encoding of the three-dimensional audio signal layering, ultimately resulting in only one audio bitstream sent to the audio decoder. This provides fundamental information support for reducing the storage and bandwidth costs at the audio encoding end.

[0135] As a fourth optional embodiment of this example, based on the above embodiments, the process of merging the basic audio stream and the enhanced audio stream according to a pre-built compatible stream format to obtain the target audio stream can be specified as follows:

[0136] a4) Determine the audio transmission header field information based on the encoding attribute information of the deployed audio encoder, and write the audio transmission header field information into the audio transmission header field of the compatible bitstream format.

[0137] In this embodiment, an audio encoder for audio encoding is deployed on the first terminal. The audio encoder may have corresponding encoding attribute information. The encoding attribute information can be understood as information describing the encoding capabilities and encoding parameters of the audio encoder. Through the encoding attribute information, it is possible to determine the encoding format, encoding length division, and sampling rate of the audio encoder. In this embodiment, the determined encoding format, encoding length, encoding length division, sampling rate, and basic channel layout involved in the encoding format can be determined as audio transmission header field information. Correspondingly, the determined audio transmission header field information can be written into the audio transmission header field of the constructed compatible bitstream format.

[0138] This step is equivalent to determining the information carried in the audio transmission header field, which is also equivalent to completing the writing operation of the carried information to the audio transmission header field.

[0139] b4) Determine the audio layout information corresponding to the three-dimensional audio signal and write the audio layout information into the audio extension header field of the compatible bitstream format.

[0140] In this embodiment, to ensure complete filling of the encoded content involved in the three-dimensional audio signal, it is necessary to determine which channel layouts the three-dimensional audio signal corresponds to. For example, in a multi-channel pickup type three-dimensional audio signal, taking 7.1.4 as an example, the channel layout information involved in 7.1 can be carried in the audio transmission header field. The 4 in 7.1.4 involves the channels of the three-dimensional audio signal, thereby determining the channel layout parameters of these four channels as audio layout information. Similarly...

[0141] This step can use the determined audio layout information as a supplement to the information carried in the audio transmission header field and write it into the audio extension header field under the compatible bitstream format.

[0142] It should be noted that this audio extension header field can be further divided into a control field and a coding payload field. The control field describes the attributes of the coding payload, and audio layout information can be written into the coding payload field as the coding payload. Considering that the audio layout information is related to the channel layout of the three-dimensional audio signal, the payload type of the audio layout information can be determined as a three-dimensional audio layout type. At the same time, the field length occupied by the audio layout information can also be determined, which is used as the payload length. This step fills the control field with the determined payload type, payload length, and the flag bits involved in the bitstream format of this audio extension header field.

[0143] As one implementation method, determining the audio layout information corresponding to the three-dimensional audio signal and writing the audio layout information into the audio extension header field of the compatible bitstream format can be specified as the following steps:

[0144] b41) Analyze the three-dimensional audio signal to obtain channel layout information, and determine the channel layout information as the audio layout information.

[0145] In this embodiment, the channel layout varies depending on the signal format type of the 3D audio signal. This embodiment uses pre-defined program configuration elements to parse the 3D audio signal, determine its signal format type, and thus identify the specific number and layout of channels involved in that signal format type. This information can then be used as the channel layout information for the 3D audio signal. This step obtains the channel layout information and uses it as the audio layout information required for the encoding and decoding of the 3D audio signal.

[0146] b42) The set bitstream flag is determined as the first load flag of the audio layout information, and the first load type and first load length corresponding to the audio layout information are determined.

[0147] It is known that for the original bitstream format involved in the construction of the compatible bitstream format, multiple bitstream flags are predefined. Different bitstream flags can respectively characterize the payload flags carried by the fields in the original bitstream format. In this embodiment, considering that the compatible bitstream format is a reuse and extension of the original bitstream format, the compatible bitstream format is equivalent to inheriting the determination of the payload flags in the original bitstream format. Specifically, it can be considered that the compatible bitstream format reuses a bitstream flag inherent in the original bitstream format itself, specifically reusing the bitstream flag representing the padding element in the original bitstream format. Further, in this embodiment, the padding element flag defined in the original bitstream format is equivalent to using it as the payload flag for the audio extension header field and the extended audio payload field in the compatible bitstream format. And relative to the audio extension header field, the payload flag representing the padding element can be denoted as the first payload flag.

[0148] As described above, compared to the basic layout information carried in the audio transmission header field, the audio extension header field mainly carries channel layout information relative to the three-dimensional audio signal. This channel layout information (audio layout information) can be considered as extension information of the basic layout information. If this audio layout information is to be carried in the audio extension header field, it can be determined as a padding element to be written into the field, and the bitstream flag bit representing the padding element in the original bitstream format can be used as the load flag bit of the audio layout information. Preferably, this load flag bit can be designated as the first load flag bit.

[0149] Similarly, once the audio layout information is known, its length can be determined, and this length can be used as the first payload length of the audio layout information. Furthermore, the first payload type corresponding to the audio layout information can be determined from predefined payload types.

[0150] b43) Write the first load flag, the first load type, and the first load length into the extension header control field of the audio extension header field.

[0151] It can be seen that in this embodiment, the audio extension header field is refined into an extension header control field and an extension header payload field. The first payload flag, the first payload type, and the first payload length determined in the above steps can all be used as control information for the audio layout information and written into the extension header control field.

[0152] b44) Write the audio layout information as load data into the extension header load field of the audio extension header field.

[0153] Similarly, in this embodiment, audio layout information can also be directly written as payload data into the payload field of the extension header.

[0154] The above operations ultimately enabled the audio layout information to be filled into the audio extension header field.

[0155] c4) Write the basic audio bitstream into the original audio payload field of the compatible bitstream format according to the audio transmission header field information.

[0156] It is known that the compatible bitstream format extends the original bitstream format with an audio extension header field and an audio extension payload field. Considering that the basic audio bitstream and the enhanced audio bitstream are obtained through the above steps, and to ensure that the audio encoding end can decode the basic audio bitstream independently, or to achieve the decoding of three-dimensional audio signals, spatial isolation is required when filling these two audio bitstreams. However, to ensure that only one audio bitstream is ultimately formed, a compatible bitstream format is needed to fill the basic audio bitstream and the enhanced audio bitstream in their respective independent forms, ultimately forming a single bitstream. Therefore, this embodiment considers reusing the existing original bitstream format and reusing the original audio payload field carried in the original bitstream format to fill the basic audio bitstream. Simultaneously, an extended audio payload field is added to the original bitstream format to fill the enhanced audio bitstream.

[0157] In this embodiment, the filling of the basic audio bitstream into the original audio payload field can be achieved through the audio transmission header field information determined above (specifically, the channel layout information carried by the audio transmission header field information). For example, taking the three-dimensional audio signal of 7.1.4 as an example, M channels (M greater than or equal to 1 and less than 7) are mainly selected from the channels corresponding to 7 to constitute the basic layer audio signal. The audio transmission header field information includes the basic channel quantity M occupied by the basic layer audio signal and the specific basic channel layout involved.

[0158] This step determines how many subfields the basic audio bitstream can be divided into based on the basic channel quantity and basic channel layout information in the audio transmission header field. The basic audio bitstream is then divided according to the number of subfields. Each basic audio sub-bitstream can be written into a subfield, and each subfield contains a control field. The payload flags, payload type, and payload length involved in the basic audio sub-bitstream can be written into the control field. The basic audio sub-bitstream can then be written into the encoded payload field contained in the subfield.

[0159] By performing the above operations on each of the divided basic audio sub-streams, the entire basic audio sub-stream can ultimately be written to the original audio payload field.

[0160] As one implementation, this optional embodiment can further specify the writing of the basic audio bitstream into the original audio payload field of the compatible bitstream format according to the audio transport header field information as follows:

[0161] c41) Based on the audio transmission header field information, determine the second payload flag bit and the number of subfields corresponding to the basic audio bitstream.

[0162] In this embodiment, by using the sampling rate and basic channel layout information included in the audio transport header field, it is possible to determine how many sub-streams the basic audio bitstream should be transmitted in. This step determines the required number of sub-streams as the number of sub-fields that the original audio payload field can form.

[0163] In this embodiment, the type of bitstream flag (such as stereo element) and data type can be determined based on the signal attributes of the basic audio bitstream itself. The determined bitstream flag is recorded as the second load flag. Similarly, the determined data type can be determined as the load type and recorded as the second load type.

[0164] c42) Divide the basic audio bitstream to form the basic sub-bitstream of the number of sub-fields.

[0165] c43) For each base sub-bitstream, merge the second payload flag bit with the base sub-bitstream into a base sub-field.

[0166] In this embodiment, the second load flag bit determined for each basic sub-bitstream can be merged with the basic sub-bitstream itself to form a sub-field, which can be denoted as the basic sub-field.

[0167] c44) Write the merged basic sub-streams into the original audio payload field in the order of division.

[0168] Each of the resulting basic subfields can be sorted according to the division order when the basic audio bitstream is divided, and written sequentially into the original audio payload field to form the basic encoded payload content of the basic audio bitstream.

[0169] d4) Write the enhanced audio stream into the extended audio payload field of the compatible stream format according to the audio layout information.

[0170] In this embodiment, the extended audio load field can be considered as being used to fill the enhanced audio bitstream. The specific filling of the extended audio load field with the enhanced audio bitstream can be achieved using the audio layout information determined above. For example, taking the three-dimensional audio signal in 7.1.4 as an example, it includes the basic layer audio signal formed after signal layering processing. The number of basic channels is M (M is greater than or equal to 1 and less than 7). The remaining channels can be considered as enhancement channels. The number of enhancement channels can be the difference between 7 and M, and the number of channels involved in 1 and 4 in 7.1.4. Knowing the basic channels is equivalent to determining which channels will be enhancement channels. The audio layout information can then include the number of enhancement channels occupied by the enhanced layer audio signal and the specific enhancement channel layout.

[0171] This step determines how many subfields the enhanced audio stream can be divided into based on the enhanced channel quantity and enhanced channel layout information in the audio layout information. The enhanced audio stream is then divided according to the number of subfields. Each enhanced audio substream can be written into a subfield, and each subfield contains a control field. The payload flags, payload type, and payload length involved in the enhanced audio substream can be written into the control field. The enhanced audio substream can then be written into the encoded payload field contained in the subfield.

[0172] By performing the above operations on each of the divided enhanced audio sub-streams, the entire enhanced audio stream can be written to the extended audio payload field.

[0173] As one implementation, this optional embodiment can specify writing the enhanced audio bitstream into the extended audio payload field of the compatible bitstream format according to the audio layout information as follows:

[0174] d41) Based on the audio layout information, determine the number of stream channels involved in the enhanced audio stream, and based on the predefined information, determine the third load flag and the second load type corresponding to the enhanced audio stream.

[0175] In this embodiment, by using the enhancement channel layout and other information included in the audio layout information, it is possible to determine how many sub-streams the enhanced audio bitstream should be transmitted as. This step determines the required number of sub-streams as the number of subfields that can be formed by the extended audio payload field. This number of subfields can also be used as the actual number of bitstream channels involved in the enhanced audio bitstream.

[0176] In this embodiment, the enhanced audio bitstream itself is not filled into the original bitstream format. Instead, it is used as payload content to fill the extended audio payload field of the compatible bitstream format provided in this embodiment. This extended audio payload field reuses the payload flag bits from the original bitstream format, specifically the payload flag bits representing the padding elements. In this embodiment, the padding element can be identified as the third payload flag bit corresponding to the enhanced audio bitstream. Similarly, this embodiment can determine which extended data type the enhanced audio bitstream belongs to based on the extended data type predefined for the compatible bitstream format, and can identify this extended data type as the second payload type.

[0177] d42) Divide the enhanced audio bitstream to form the number of enhanced sub-bitstreams of the bitstream channels, and determine the second payload length of each enhanced sub-bitstream.

[0178] In this embodiment, the enhanced audio bitstream can be divided according to the number of bitstream channels determined above, thereby forming the number of enhanced sub-bitstreams. This step can also determine the corresponding bitstream length for each enhanced sub-bitstream, and this bitstream length can be determined as the second payload length of the enhanced sub-bitstream.

[0179] It should be noted that each of the divided enhancement sub-streams corresponds to the original signal channel in the three-dimensional audio signal. In this embodiment, each enhancement sub-stream can be mapped to the corresponding original signal channel to obtain the original channel mapping information. This original channel mapping information also needs to be filled in the extended audio load field so that the enhancement sub-stream can be effectively restored to the enhancement layer audio signal during the decoding process.

[0180] d43) For each enhanced sub-stream, the third payload flag, the second payload type, and the corresponding second payload length are used as the enhancement control field, and the enhancement control field is merged with the enhanced sub-stream into an enhancement sub-field.

[0181] In this embodiment, the third payload flag, second payload type, and corresponding second payload length and original channel mapping information determined for each enhanced sub-stream can be used as an enhancement control field. This enhancement control field, combined with the enhanced sub-stream itself, can form a sub-field, which can be denoted as the enhancement sub-field.

[0182] d44) Write the merged enhanced sub-streams into the extended audio payload field in the order of division.

[0183] In this embodiment, each enhanced subfield can be sorted according to the division order when the enhanced audio bitstream is divided, and written sequentially into the extended audio payload field to form the enhanced coding payload content of the enhanced audio bitstream.

[0184] e4) After detecting that the writing of the enhanced audio stream is complete, a target audio stream with the compatible stream format is obtained.

[0185] In this embodiment, the filling of the basic audio bitstream and the enhanced audio bitstream according to the compatible bitstream format is performed sequentially. Therefore, the filling of the audio bitstream is considered complete after the detection that the filling of the enhanced audio bitstream is complete. The compatible bitstream formed after the filling is completed can be recorded as the target audio bitstream.

[0186] The above-described technical solution in this embodiment provides a specific implementation of filling the basic audio bitstream and enhancing the audio bitstream to form a single target audio bitstream. After encoding, only one audio bitstream is formed and sent to the audio decoding end, enabling compatible decoding of the encoding end device. This allows for either traditional basic audio decoding or complete three-dimensional audio decoding. This processing method significantly reduces the storage and bandwidth costs of the audio bitstream for the audio encoding end device and improves the encoding stability of the audio encoding end.

[0187] Figure 2 This is a flowchart illustrating an audio signal processing method provided in an embodiment of the present disclosure. This embodiment is applicable to the processing of three-dimensional audio signals, specifically to the decoding of the encoded bitstream of a three-dimensional audio signal. The method can be executed by an audio signal processing device, which can be implemented by software and / or hardware, and can be configured in a terminal and / or server to implement the audio signal processing method in this embodiment of the present disclosure.

[0188] like Figure 2 As shown, the audio signal processing method provided in this embodiment may specifically include:

[0189] It should be noted that the execution terminal of the audio signal processing method provided in this embodiment can be considered as a second terminal involved in audio decoding operations, and the second terminal can be considered as a decoding terminal with an audio service-related client installed.

[0190] S201. Receive the target audio bitstream transmitted by the first terminal, wherein the target audio bitstream is formed by the first terminal merging the basic audio bitstream and the enhanced audio bitstream according to a pre-constructed compatible bitstream format, the basic audio bitstream is obtained by encoding the basic layer audio signal, the enhanced audio bitstream is obtained by encoding the enhanced layer audio signal, and the basic layer audio signal and the enhanced layer audio signal are obtained by the first terminal through signal layer processing of the acquired three-dimensional audio signal.

[0191] In this embodiment, the target audio bitstream can be received from the first terminal through a pre-established communication connection. The first terminal can be considered as the execution terminal that performs the method provided in the above embodiment. It can perform signal layering processing on the acquired three-dimensional audio signal, first dividing the three-dimensional audio signal into a base layer audio signal and an enhancement layer audio signal, and then performing audio encoding on the base layer audio signal to form a base audio bitstream and audio encoding on the enhancement layer audio signal to form an enhancement audio bitstream. The target audio bitstream received in this embodiment is equivalent to the audio bitstream formed by merging and packaging the base audio bitstream and the enhancement audio bitstream according to a compatible bitstream format.

[0192] The compatible bitstream format, compared to the existing original bitstream format, can be considered as a reuse and expansion of the original bitstream format's fields. Specifically, it adds an audio extension header field and an extended audio payload field to the fields included in the original bitstream format, which can fill the enhanced audio bitstream. This achieves the merging of the basic audio bitstream and the enhanced audio bitstream in a single stream.

[0193] S202. When the deployed audio decoder only supports basic audio decoding, the target audio bitstream is parsed to obtain the basic audio bitstream, and the basic layer audio signal is formed by restoring the basic audio bitstream for playback.

[0194] In this embodiment, the received target audio bitstream is equivalent to the audio bitstream formed by the first terminal through audio encoding of the three-dimensional audio signal. It can also be considered to possess decoding characteristics capable of independently decoding both the basic audio bitstream and the enhanced audio bitstream. Based on these decoding characteristics of the received target audio bitstream, the execution terminal can further determine whether it possesses the decoding capability to decode the three-dimensional audio signal by considering the decoder attribute information of its deployed audio decoder.

[0195] In this embodiment, if the analysis of the decoder attribute information determines that the execution terminal does not have the ability to decode three-dimensional audio signals, it can be assumed that it only supports basic audio decoding. In this case, only the basic audio bitstream in the target audio bitstream corresponding to the three-dimensional audio signal can be decoded.

[0196] Decoding the target audio bitstream can be achieved by parsing each field in the target audio bitstream bit by bit. When the audio decoder only supports basic audio decoding, it essentially only defines the decoding information related to the basic layer audio signal, without defining the decoding information related to the enhancement layer audio signal. Therefore, without defining the enhancement layer audio information, parsing each field in the target audio bitstream bit by bit is equivalent to being unable to properly parse the bitstream fields related to the enhancement layer audio signal. These bitstream fields are often discarded during the parsing process. Ultimately, after parsing the entire target audio bitstream, it is equivalent to only obtaining the basic audio bitstream related to the basic layer audio signal.

[0197] This step can restore the basic audio stream according to the signal format type of the three-dimensional audio signal involved in the target audio stream. In the restoration process, other parameter information, such as channel layout, can also be obtained in combination with the parsing stage to restore the audio signal of the basic audio stream. Finally, the basic layer audio signal in the three-dimensional audio signal can be obtained and played through the speaker of this execution terminal.

[0198] As described in the above embodiments, the base layer audio signal can be either a stereo audio signal or a surround sound audio signal. This execution terminal decodes the base audio stream to obtain either a stereo audio signal or a surround sound audio signal. It should be noted that when the signal format type of the three-dimensional audio signal is omnidirectional pickup, its base layer audio signal is primarily a stereo audio signal. Therefore, even if this execution terminal has surround sound decoding capabilities, the base layer audio signal obtained through decoding will still be a stereo audio signal.

[0199] S203. When the deployed audio decoder supports three-dimensional audio decoding, the target audio bitstream is parsed to obtain the basic audio bitstream and the enhanced audio bitstream. The target audio signal is then formed by restoring the basic audio bitstream and the enhanced audio bitstream respectively and played. The target audio signal is the three-dimensional audio signal involved in the target audio bitstream.

[0200] In this embodiment, this step can be considered as supporting the decoding of the entire three-dimensional audio signal if the analysis of the decoder attribute information determines that the execution terminal has the capability to decode three-dimensional audio signals. At this time, it is possible to consider decoding the enhanced audio stream while simultaneously decoding the base audio stream from the target audio stream.

[0201] It is understood that decoding the target audio bitstream can be specifically achieved by parsing each field in the target audio bitstream bitstream bit by bit. Therefore, when the audio decoder supports decoding the entire three-dimensional audio signal, it is equivalent to the audio decoder not only having the definition of the decoding information involved in the base layer audio signal, but also the definition of the decoding information involved in the enhancement layer audio signal.

[0202] Given the aforementioned information definition, parsing each field in the target audio bitstream bitwise allows for the parsing of each bitstream field. This yields not only the base audio bitstream related to the base layer audio signal but also the enhanced audio bitstream related to the enhancement layer audio signal. Next, this step reconstructs the base and enhanced audio bitstreams according to the signal format type of the three-dimensional audio signals involved in the target audio bitstream. Similarly, the reconstruction process can incorporate more parameter information obtained during the parsing phase, such as channel layout information, to reconstruct the enhanced and base audio bitstreams separately. Ultimately, a complete three-dimensional audio signal composed of the base and enhanced layer audio signals is obtained, which can then be played through the speaker of this execution terminal.

[0203] It should be noted that after the basic layer audio signal and the enhanced layer audio signal are restored as described above in this embodiment, further recovery operations are required on the basic layer audio signal and the enhanced layer audio signal to form the three-dimensional audio signal initially acquired by the first terminal.

[0204] The recovery of the base layer audio signal and the enhancement layer audio signal into a 3D audio signal can be described as follows: First, consider the signal format type of the 3D audio signal involved in the target audio bitstream. If the signal format type is omnidirectional pickup, the decoded base layer audio signal needs to be recovered to the first two signal channels in the channel signal sequence involved in this omnidirectional pickup type, and the enhancement layer audio signal, combined with the corresponding signal label, needs to be recovered to the subsequent signal channels in the channel signal sequence other than the first two signal channels. For example, when the base layer audio signal is a stereo audio signal, it can recover two audio signals, thus occupying the first two signal channels of the channel signal sequence. When the base layer audio signal is a surround sound audio signal, it can recover more than two M audio signals, thus occupying the first M signal channels of the channel signal sequence. M is the number of channels occupied by surround sound.

[0205] This embodiment provides an audio processing method, which is equivalent to the execution logic implementation of a decoding terminal. The target audio bitstream formed on the encoding terminal using the method described above can be flexibly decoded by combining its own decoding capabilities. When 3D audio signal decoding capabilities are lacking, only the basic layer audio signal can be decoded to form a suitable stereo or surround sound audio signal. When 3D audio signal decoding capabilities are available, a 3D audio signal in a 3D audio signal format can be formed. The method provided in this embodiment significantly improves the encoding and decoding compatibility of 3D audio signals, expands the application scenarios of 3D audio signals, and is more adaptable to the decoding characteristics of different audio decoding terminals, ensuring the effective advancement of audio decoding logic.

[0206] As a fifth optional embodiment of this embodiment, based on the above embodiments, the parsing of the target audio bitstream to obtain the basic audio bitstream can be specified as follows: the steps in this embodiment are specifically designed for the case where the deployed audio decoder only supports basic audio decoding.

[0207] a5) Traverse the target audio bitstream, parse the audio transmission header field in the target audio bitstream, and obtain the audio transmission header field information.

[0208] In this embodiment, the audio decoder parses the fields of the target audio bitstream byte by byte, which is equivalent to traversing and parsing the target audio bitstream. First, this step can be used to parse the audio transmission header field in the target audio bitstream, and obtain the sampling rate related to audio encoding and decoding, the channel layout information involved in the basic audio bitstream, and the parsing interface information, etc. In this embodiment, the parsed information can be determined as the audio transmission header field information.

[0209] It should be noted that the target audio stream is essentially a merging of the basic and enhanced audio streams using a compatible bitstream format. It consists of four fields: an audio transport header field, an audio extension header field, a raw audio payload field, and an extended audio payload field. This embodiment can begin parsing the audio transport header field, with the subsequent parsing order being the audio extension header field, the raw audio payload field, and the extended audio payload field, respectively. Only after the parsing of a preceding field is completed or discarded can the traversal and parsing of subsequent fields continue.

[0210] b5) After parsing the audio transmission header field, if the payload type parsed from the audio extension header field of the target audio bitstream represents a three-dimensional audio layout, then discard the audio extension header field.

[0211] In this embodiment, after traversing the audio header field and confirming that the parsing of the audio header field is complete, the subsequent fields in the target audio bitstream can be traversed. Based on the specific format structure of the compatible bitstream format, the subsequent field can be determined to be the audio extension header field of the target audio bitstream. Similarly, this step can also consider that the audio extension header field of the target audio bitstream has been traversed when it is determined that the obtained subsequent field represents the payload flag bit, and the payload flag bit is a pre-set bitstream flag bit.

[0212] This step can continue to traverse the audio extension header field. If the payload type in the audio extension header field is reached and the description text representing the three-dimensional audio layout is obtained, considering that the payload type of the three-dimensional audio layout is not defined in the audio decoder, this step cannot parse the subsequent content in the audio extension field. At this time, the payload content carried in the audio extension field can be considered as invalid payload, and the audio extension header field can be discarded directly according to the parsed field length.

[0213] c5) Based on the audio transmission header field information, parse the original audio payload field in the target audio bitstream to obtain the basic audio payload.

[0214] In this embodiment, following the traversal order of the target audio bitstream, after discarding the audio extension header field, the subsequent fields in the target audio bitstream can be traversed. Similarly, after obtaining the payload type and payload length from the traversed subsequent fields, it can be determined that the payload type represents the basic audio bitstream. Then, the field with a length equal to the payload length in the subsequent field can be parsed to obtain the payload content, which can be recorded as the basic audio payload.

[0215] It should be noted that, assuming the original audio payload field includes multiple basic subfields, this step can be used to parse each basic subfield in turn to obtain the audio payload in the corresponding basic subfield, and finally the audio payload parsed from all basic subfields is recorded as the basic audio payload.

[0216] d5) After parsing the original audio payload field, if the payload type parsed from the extended audio payload field of the target audio bitstream indicates enhanced audio coding, then the extended audio payload field is discarded.

[0217] In this embodiment, after obtaining the basic audio load through the above steps, it is determined that the parsing of the original audio load field is complete. Similarly, the load flag bit represented by the obtained subsequent field is determined. When the load flag bit is a pre-set bitstream flag bit, it is considered that the extended audio load field of the target audio bitstream has been traversed. Assuming that the load type representing the enhanced audio encoding is parsed from the extended audio load field, the load content carried in the extended audio load field can be considered as invalid load, and the extended audio load field can also be discarded.

[0218] e5) After completing and ending the traversal of the target audio bitstream, the bitstream containing the basic audio payload is determined as the basic audio bitstream.

[0219] In this embodiment, the parsing of the entire target audio bitstream is considered complete when the end flag of the target audio bitstream is encountered, at which point the traversal of the target audio bitstream can be ended. This step can then construct the basic audio bitstream based on the parsed basic audio payload bitstream after the traversal of the entire target audio bitstream is complete.

[0220] Similarly, if other payload streams (not enhanced audio streams) are parsed from the target audio stream, these payload streams also need to be restored in subsequent logic execution to obtain their corresponding signals. It can be understood that regardless of the presence of other payload streams, the basic audio stream corresponding to the base layer audio signal can be obtained through the steps and logic of the above embodiment. Therefore, the payload fields related to the three-dimensional audio layout and enhanced audio encoding are discarded; thus, this embodiment does not implement the parsing of the enhanced audio stream.

[0221] The above-described technical solution in this embodiment provides a decoding implementation when the deployed audio decoder only supports basic audio decoding. By parsing the fields in the compatible bitstream format, the independent parsing of the basic audio bitstream can be accurately achieved. This ensures that the audio decoder can play audio signals in three-dimensional audio formats, improving audio playback quality without increasing encoding / decoding storage and maintenance costs.

[0222] As a sixth optional embodiment of this example, based on the above embodiments, the process of parsing the target audio bitstream to obtain the basic audio bitstream and the enhanced audio bitstream can be further determined. Similarly, the steps in this embodiment are specifically geared towards the case where the deployed audio decoder supports three-dimensional audio decoding.

[0223] a6) Traverse the target audio bitstream, parse the audio transmission header field in the target audio bitstream, and obtain the audio transmission header field information.

[0224] Based on the above description, it can be understood that the audio decoder parses the fields of the target audio bitstream byte by byte, which is equivalent to traversing and parsing the target audio bitstream. Similarly, it can be understood that the target audio bitstream is obtained by merging the basic audio bitstream and the enhanced audio bitstream using a compatible bitstream format. The audio content of the target audio bitstream is distributed across four fields: the audio transport header field, the audio extension header field, the original audio payload field, and the extended audio payload field. This embodiment can start parsing from the audio transport header field, with the subsequent parsing order being the audio extension header field, the original audio payload field, and the extended audio payload field, respectively. Only after the parsing of a preceding field is completed or discarded can the traversal and parsing of subsequent fields continue.

[0225] This embodiment first uses this step to parse the audio transmission header field in the target audio bitstream, and can obtain the sampling rate related to audio encoding and decoding, the channel layout information involved in the basic audio bitstream, and the parsing interface information, etc. This embodiment can determine the parsed information as the audio transmission header field information.

[0226] b6) After completing the parsing of the audio transmission header field, parse the audio extension header field in the target audio bitstream to obtain the audio layout information for the three-dimensional audio layout.

[0227] In this embodiment, after traversing the audio header field and confirming that the parsing of the audio header field is complete, the subsequent fields in the target audio bitstream can be traversed. Based on the specific format structure of the compatible bitstream format, the subsequent field can be determined to be the audio extension header field of the target audio bitstream. Similarly, this step can also consider that the audio extension header field of the target audio bitstream has been traversed when it is determined that the obtained subsequent field represents the payload flag bit, and the payload flag bit is a pre-set bitstream flag bit (such as a padding element).

[0228] This step can continue traversing the audio extension header field. If the payload type in the audio extension header field is reached, and the 3D audio layout information representing the 3D audio layout is obtained, considering that the audio decoder has the capability to decode 3D audio signals, it is equivalent to the audio decoder defining the payload type of the 3D audio layout. Therefore, this step can be used to continue parsing the subsequent content in the audio extension field, thereby obtaining the complete 3D audio layout information. In this embodiment, this 3D audio layout information can be recorded as audio layout information.

[0229] This embodiment can update the channel layout parameters in the audio decoder based on the three-dimensional audio layout information, so that the audio decoder has the ability to decode three-dimensional audio signals normally.

[0230] c6) After completing the parsing of the audio extension header field, parse the original audio payload field in the target audio bitstream according to the audio transmission header field information to obtain the basic audio payload.

[0231] In this embodiment, after traversing the audio header fields and confirming that the parsing of the audio header fields is complete, the subsequent fields in the target audio bitstream can be traversed. Similarly, after obtaining the payload type and payload length from the traversed subsequent fields, and determining that the payload type represents the basic audio bitstream, the field with a length equal to the payload length in the subsequent field can be parsed to obtain the continued payload content, which can be recorded as the basic audio payload.

[0232] It should also be noted that if the original audio payload field includes multiple basic subfields, this step can be used to parse each basic subfield in turn to obtain the audio payload in the corresponding basic subfield. Finally, the audio payload parsed from all basic subfields is recorded as the basic audio payload.

[0233] d6) After parsing the original audio payload field, parse the extended audio payload field of the target audio bitstream according to the audio layout information to obtain the enhanced audio payload.

[0234] In this embodiment, after obtaining the basic audio payload through the above steps, it is determined that the parsing of the original audio payload field is complete. Similarly, the payload flag bit representing the acquired subsequent fields is determined. When the payload flag bit is a pre-set bitstream flag bit, it is considered that the extended audio payload field of the target audio bitstream has been traversed. Assuming that the payload type representing enhanced audio encoding is subsequently parsed from the extended audio payload field, since the audio decoder has the ability to parse the enhanced audio encoding type, it can continue to parse the audio payload in the extended audio payload field.

[0235] In this embodiment, the extended audio load field may also have multiple enhanced subfields. In this embodiment, each enhanced subfield can be parsed sequentially using this step to obtain the audio load in the corresponding enhanced subfield. Finally, the audio load derived from all enhanced subfields is recorded as the enhanced audio load.

[0236] Similarly, it can be known that when the audio decoder has the ability to resolve three-dimensional audio signals, regardless of the audio extension header field or the extended audio load field of the target audio bitstream, the load type parsed from the corresponding fields may also include other load types that do not represent the three-dimensional audio layout or enhanced audio coding. When it is determined that the parsed other load types can be recognized by the audio decoder, in addition to parsing the relevant information of the three-dimensional audio layout from the audio extension header field and parsing the enhanced audio load from the extended audio load field, this embodiment can obtain other audio loads included in the target audio bitstream.

[0237] e6) After completing and ending the traversal of the target audio bitstream, the bitstream containing the basic audio load is determined as the basic audio bitstream, and the bitstream containing the enhanced audio load is determined as the enhanced audio bitstream.

[0238] In this embodiment, the parsing of the entire target audio bitstream is considered complete when the end flag of the target audio bitstream is encountered, at which point the traversal of the target audio bitstream can be ended. This step can be used to construct a basic audio bitstream based on the parsed basic audio payload bitstream after completing the traversal of the entire target audio bitstream, and to construct an enhanced audio bitstream based on the parsed enhanced audio payload bitstream.

[0239] The above-described technical solution in this embodiment provides a decoding implementation when the deployed audio decoder supports 3D audio decoding. By parsing the fields in the compatible bitstream format, the independent parsing of the basic audio bitstream can be accurately achieved. This also ensures that the audio decoder can play audio signals in 3D audio formats, improving audio playback quality without increasing encoding / decoding storage and maintenance costs.

[0240] Figure 3 This is a schematic diagram of an audio signal processing device provided in an embodiment of the present disclosure. This embodiment is applicable to the processing of three-dimensional audio signals, specifically for encoding three-dimensional audio signals. The device can be implemented through software and / or hardware, and can be configured in a terminal and / or server to implement the audio signal processing method in this embodiment. Specifically, the device may include: a layering module 31, an encoding module 32, a merging module 33, and a transmission module 34.

[0241] The layering module 31 is used to acquire a three-dimensional audio signal and perform signal layering on the three-dimensional audio signal to obtain a base layer audio signal and an enhancement layer audio signal. The base layer audio signal is an audio signal in the three-dimensional audio signal that supports the reference codec, and the enhancement layer audio signal is an audio signal in the three-dimensional audio signal that does not support the reference codec.

[0242] The encoding module 32 is used to encode the base layer audio signal to obtain a base audio bitstream, and to encode the enhancement layer audio signal to obtain an enhancement audio bitstream;

[0243] The merging module 33 is used to merge the basic audio bitstream and the enhanced audio bitstream according to a pre-built compatible bitstream format to obtain a target audio bitstream. The compatible bitstream format is formed by reusing and expanding the fields in the original bitstream format. The original bitstream format is the bitstream format used by the reference encoding and decoding.

[0244] The transmission module 34 is used to transmit the target audio bitstream to the second terminal, so that the second terminal can decode the basic audio bitstream and the enhanced audio bitstream from the target audio bitstream according to the decoding capability of the deployed audio decoder for playback, or decode only the basic audio bitstream from the target audio bitstream for playback.

[0245] This embodiment provides an audio signal processing device that considers layering the audio signals in a three-dimensional audio signal and independently encoding each layered audio signal to obtain a base audio bitstream of the base layer and an enhanced audio bitstream of the enhancement layer. The base and enhanced audio bitstreams are then merged according to a constructed compatible bitstream format to form a single target audio bitstream. This target audio bitstream is adaptable to different decoding scenarios, enabling flexible decoding by the audio decoder. Compared to existing technologies, this embodiment's solution only generates one audio bitstream for the audio decoder, achieving compatible decoding by the encoding device. It can perform either traditional basic audio decoding or complete three-dimensional audio decoding. This processing method significantly reduces the storage and bandwidth costs of the audio encoding device and improves the encoding stability. Simultaneously, the compatible decoding of the target audio bitstream by the audio decoder better adapts to the decoding characteristics of different audio decoders, ensuring the effective advancement of the audio decoding logic.

[0246] Furthermore, the layered module 31 can be specifically used for:

[0247] When the signal format type of the three-dimensional audio signal is omnidirectional pickup type, the first channel signal and the signal label of the first channel signal are determined according to the signal order of the three-dimensional audio signal.

[0248] Based on the respective signal labels, each of the first channel signals undergoes layered conversion processing to obtain the first base layer audio signal and the first enhancement layer audio signal of the three-dimensional audio signal.

[0249] Furthermore, the layered module 31 can also be used specifically for:

[0250] When the signal format type of the three-dimensional audio signal is a multi-channel pickup type, a multi-channel signal sequence representing the three-dimensional audio signal is obtained, and the multi-channel signal sequence includes multiple second channel signals;

[0251] Based on each of the second channel signals, the second base layer audio signal and the second enhancement layer audio signal of the three-dimensional audio signal are determined.

[0252] Furthermore, the device may also include a compatible format building module, which may specifically be used for:

[0253] Obtain the original bitstream format used by the reference codec, wherein the original bitstream format includes: an audio transport header field and an original audio payload field;

[0254] An audio extension header field is inserted between the audio transmission header field and the original audio payload field. The audio extension header field is used to write the audio layout information of the three-dimensional audio signal.

[0255] An extended audio payload field is added after the original audio payload field, which is used to fill the enhanced audio bitstream;

[0256] The compatible bitstream format is formed based on the audio transmission header field, audio extension header field, original audio payload field, and extended audio payload field.

[0257] Furthermore, the merging module may specifically include:

[0258] The first field filling module is used to determine the audio transmission header field information based on the encoding attribute information of the deployed audio encoder, and write the audio transmission header field information into the audio transmission header field of the compatible bitstream format;

[0259] The second field filling module is used to determine the audio layout information corresponding to the three-dimensional audio signal and write the audio layout information into the audio extension header field in the compatible bitstream format;

[0260] The third field filling module is used to write the basic audio bitstream into the original audio payload field of the compatible bitstream format according to the audio transmission header field information;

[0261] The fourth field filling module is used to write the enhanced audio bitstream into the extended audio payload field of the compatible bitstream format according to the audio layout information;

[0262] The bitstream acquisition module is used to obtain a target audio bitstream with the compatible bitstream format after detecting that the writing of the enhanced audio bitstream is completed.

[0263] Furthermore, the second field filling module can be specifically used for:

[0264] The channel layout information is obtained by analyzing the three-dimensional audio signal, and the channel layout information is determined as the audio layout information.

[0265] The set bitstream flag is determined as the first load flag of the audio layout information, and the first load type and first load length corresponding to the audio layout information are determined.

[0266] Write the first load flag, the first load type, and the first load length into the extension header control field of the audio extension header field;

[0267] The audio layout information is written as load data into the extension header load field of the audio extension header field.

[0268] Furthermore, the third field filling module can specifically be used for:

[0269] Based on the audio transmission header field information, determine the second payload flag bit and the number of subfields corresponding to the basic audio bitstream;

[0270] Divide the basic audio bitstream to form the basic sub-bitstream of the specified number of sub-fields;

[0271] For each base sub-bitstream, the second payload flag is merged with the base sub-bitstream into a base sub-field;

[0272] The merged basic sub-streams are written into the original audio payload field in the order of division.

[0273] Furthermore, the fourth character filling module can be specifically used for:

[0274] Based on the audio layout information, determine the number of stream channels involved in the enhanced audio stream, and determine the third load flag and the second load type corresponding to the enhanced audio stream based on the predefined information;

[0275] The enhanced audio bitstream is divided into enhanced sub-bitstreams of the specified number of bitstream channels, and the second payload length of each enhanced sub-bitstream is determined.

[0276] For each enhanced sub-stream, the third payload flag, the second payload type, and the corresponding second payload length are used as an enhancement control field, and the enhancement control field is merged with the enhanced sub-stream into an enhancement sub-field;

[0277] The merged enhanced sub-streams are written into the extended audio payload field in the order of division.

[0278] The above-described apparatus can execute the methods provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the methods.

[0279] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0280] Figure 4 This is a schematic diagram of an audio signal processing device provided in an embodiment of the present disclosure. This embodiment is applicable to the processing of three-dimensional audio signals, specifically for decoding the encoded bitstream of a three-dimensional audio signal. The device can be implemented by software and / or hardware, and can be configured in a terminal and / or server to implement the audio signal processing method in this embodiment. Specifically, the device may include: a receiving module 41, a first decoding module 42, and a second decoding module 43.

[0281] The receiving module 41 is used to receive the target audio bitstream transmitted by the first terminal. The target audio bitstream is formed by the first terminal merging the basic audio bitstream and the enhanced audio bitstream according to a pre-constructed compatible bitstream format. The basic audio bitstream is obtained by encoding the basic layer audio signal, and the enhanced audio bitstream is obtained by encoding the enhanced layer audio signal. The basic layer audio signal and the enhanced layer audio signal are obtained by the first terminal through signal layer processing of the acquired three-dimensional audio signal.

[0282] The first decoding module 42 is used to parse the target audio bitstream to obtain the basic audio bitstream when the deployed audio decoder only supports basic audio decoding, and to form the basic layer audio signal for playback by restoring the basic audio bitstream.

[0283] The second decoding module 43 is used to parse the target audio bitstream to obtain the basic audio bitstream and the enhanced audio bitstream when the deployed audio decoder supports three-dimensional audio decoding, and to play the target audio signal by restoring the basic audio bitstream and the enhanced audio bitstream respectively. The target audio signal is the three-dimensional audio signal involved in the target audio bitstream.

[0284] This embodiment provides an audio signal processing device, which is equivalent to the execution logic implementation of a decoding terminal. The target audio bitstream formed at the encoding terminal using the method described above can be flexibly decoded by combining its own decoding capabilities. When lacking 3D audio signal decoding capabilities, it can decode only the basic layer audio signal to form a suitable stereo or surround sound audio signal. When possessing 3D audio signal decoding capabilities, it can form a 3D audio signal in a 3D audio signal format. The method provided in this embodiment better improves the encoding and decoding compatibility of 3D audio signals, expands the application scenarios of 3D audio signals, and is more adaptable to the decoding characteristics of different audio decoding terminals, ensuring the effective advancement of audio decoding logic.

[0285] Furthermore, the first decoding module 42 can specifically be used for:

[0286] Traverse the target audio bitstream, parse the audio transmission header field in the target audio bitstream, and obtain the audio transmission header field information;

[0287] After parsing the audio transmission header field, if the payload type parsed from the audio extension header field of the target audio bitstream represents a three-dimensional audio layout, then the audio extension header field is discarded.

[0288] Based on the audio transmission header field information, the original audio payload field in the target audio bitstream is parsed to obtain the basic audio payload;

[0289] After parsing the original audio payload field, if the payload type parsed from the extended audio payload field of the target audio bitstream indicates enhanced audio coding, then the extended audio payload field is discarded.

[0290] After completing and ending the traversal of the target audio bitstream, the bitstream containing the basic audio payload is determined as the basic audio bitstream.

[0291] Furthermore, the second decoding module 43 may include:

[0292] Traverse the target audio bitstream, parse the audio transmission header field in the target audio bitstream, and obtain the audio transmission header field information;

[0293] After parsing the audio transmission header field, the audio extension header field in the target audio bitstream is parsed to obtain audio layout information for the three-dimensional audio layout.

[0294] After parsing the audio extension header field, the original audio payload field in the target audio bitstream is parsed according to the audio transmission header field information to obtain the basic audio payload;

[0295] After parsing the original audio payload field, the extended audio payload field of the target audio bitstream is parsed according to the audio layout information to obtain the enhanced audio payload;

[0296] After completing and ending the traversal of the target audio bitstream, the bitstream containing the basic audio payload is determined as the basic audio bitstream, and the bitstream containing the enhanced audio payload is determined as the enhanced audio bitstream.

[0297] The above-described apparatus can execute the methods provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the methods.

[0298] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0299] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure. Reference is made below. Figure 5 It illustrates a computer device suitable for implementing embodiments of the present disclosure (e.g., Figure 5 The diagram below shows the structure of the terminal device or server 50. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The computer device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0300] like Figure 5 As shown, the computer device 50 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 51, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 52 or a program loaded from a storage device 58 into a random access memory (RAM) 55. The RAM 55 also stores various programs and data required for the operation of the computer device 50. The processing unit 51, ROM 52, and RAM 55 are interconnected via a bus 55. An edit / output (I / O) interface 54 is also connected to the bus 55.

[0301] Typically, the following devices can be connected to I / O interface 54: input devices 56 including, for example, information displays, information boards, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 57 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 58 including, for example, magnetic tapes, hard disks, etc.; and communication devices 59. Communication device 59 allows computer device 50 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 A computer device 50 with various devices is shown; however, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0302] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 59, or installed from a storage device 58, or installed from a ROM 52. When the computer program is executed by the processing device 51, it performs the functions defined in the methods of embodiments of this disclosure.

[0303] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0304] The computer device provided in this embodiment and the audio signal processing method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0305] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the audio signal processing method provided in the above embodiments.

[0306] It should be noted that the computer-readable medium described above in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0307] In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0308] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0309] The aforementioned computer-readable medium may be included in the aforementioned computer device; or it may exist independently and not assembled into the computer device.

[0310] The aforementioned computer-readable medium carries one or more programs that, when executed by the computer device, cause the computer device to:

[0311] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as general-purpose procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0312] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0313] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0314] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0315] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0316] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0317] Furthermore, although the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while some specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0318] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An audio signal processing method, characterized in that, Applied to the first terminal, including: A three-dimensional audio signal is acquired, and the three-dimensional audio signal is layered to obtain a base layer audio signal and an enhancement layer audio signal. The base layer audio signal is an audio signal in the three-dimensional audio signal that supports the reference codec, and the enhancement layer audio signal is an audio signal in the three-dimensional audio signal that does not support the reference codec. The base layer audio signal is encoded to obtain a base audio bitstream, and the enhancement layer audio signal is encoded to obtain an enhancement audio bitstream; According to a pre-built compatible bitstream format, the basic audio bitstream and the enhanced audio bitstream are merged to obtain the target audio bitstream. The compatible bitstream format is formed by reusing and expanding the fields in the original bitstream format. The original bitstream format is the bitstream format used by the reference encoding and decoding. The target audio stream is transmitted to a second terminal, so that the second terminal can decode the basic audio stream and the enhanced audio stream from the target audio stream according to the decoding capability of the deployed audio decoder for playback, or decode only the basic audio stream from the target audio stream for playback.

2. The method according to claim 1, characterized in that, The step of performing signal layering on the three-dimensional audio signal to obtain a base layer audio signal and an enhancement layer audio signal includes: When the signal format type of the three-dimensional audio signal is omnidirectional pickup type, the first channel signal and the signal label of the first channel signal are determined according to the signal order of the three-dimensional audio signal. Based on the respective signal labels, each of the first channel signals undergoes layered conversion processing to obtain the first base layer audio signal and the first enhancement layer audio signal of the three-dimensional audio signal.

3. The method according to claim 1, characterized in that, The step of performing signal layering on the three-dimensional audio signal to obtain a base layer audio signal and an enhancement layer audio signal includes: When the signal format type of the three-dimensional audio signal is a multi-channel pickup type, a multi-channel signal sequence representing the three-dimensional audio signal is obtained, and the multi-channel signal sequence includes multiple second channel signals; Based on each of the second channel signals, the second base layer audio signal and the second enhancement layer audio signal of the three-dimensional audio signal are determined.

4. The method according to claim 1, characterized in that, The steps for constructing the compatible bitstream format include: Obtain the original bitstream format used by the reference codec, wherein the original bitstream format includes: an audio transport header field and an original audio payload field; An audio extension header field is inserted between the audio transmission header field and the original audio payload field. The audio extension header field is used to write the audio layout information of the three-dimensional audio signal. An extended audio payload field is added after the original audio payload field, which is used to fill the enhanced audio bitstream; The compatible bitstream format is formed based on the audio transmission header field, audio extension header field, original audio payload field, and extended audio payload field.

5. The method according to claim 1, characterized in that, The step of merging the basic audio stream and the enhanced audio stream according to a pre-built compatible bitstream format to obtain the target audio stream includes: Based on the encoding attribute information of the deployed audio encoder, determine the audio transmission header field information and write the audio transmission header field information into the audio transmission header field of the compatible bitstream format; Determine the audio layout information corresponding to the three-dimensional audio signal, and write the audio layout information into the audio extension header field of the compatible bitstream format; According to the audio transmission header field information, the basic audio bitstream is written into the raw audio payload field of the compatible bitstream format; According to the audio layout information, the enhanced audio bitstream is written into the extended audio payload field of the compatible bitstream format; After detecting the completion of writing the enhanced audio stream, a target audio stream with the compatible stream format is obtained.

6. The method according to claim 5, characterized in that, The step of determining the audio layout information corresponding to the three-dimensional audio signal and writing the audio layout information into the audio extension header field of the compatible bitstream format includes: The channel layout information is obtained by analyzing the three-dimensional audio signal, and the channel layout information is determined as the audio layout information. The set bitstream flag is determined as the first load flag of the audio layout information, and the first load type and first load length corresponding to the audio layout information are determined. Write the first load flag, the first load type, and the first load length into the extension header control field of the audio extension header field; The audio layout information is written as load data into the extension header load field of the audio extension header field.

7. The method according to claim 5, characterized in that, The step of writing the basic audio bitstream into the original audio payload field of the compatible bitstream format according to the audio transmission header field information includes: Based on the audio transmission header field information, determine the second payload flag bit and the number of subfields corresponding to the basic audio bitstream; Divide the basic audio bitstream to form the basic sub-bitstream of the specified number of sub-fields; For each base sub-bitstream, the second payload flag is merged with the base sub-bitstream into a base sub-field; The merged basic sub-streams are written into the original audio payload field in the order of division.

8. The method according to claim 5, characterized in that, The step of writing the enhanced audio bitstream into the extended audio payload field of the compatible bitstream format according to the audio layout information includes: Based on the audio layout information, determine the number of stream channels involved in the enhanced audio stream, the third load flag, and the third load flag and second load type corresponding to the enhanced audio stream based on the predefined information; The enhanced audio bitstream is divided into enhanced sub-bitstreams of the specified number of bitstream channels, and the second payload length of each enhanced sub-bitstream is determined. For each enhanced sub-stream, the third payload flag, the second payload type, and the corresponding second payload length are used as an enhancement control field, and the enhancement control field is merged with the enhanced sub-stream into an enhancement sub-field; The merged enhanced sub-streams are written into the extended audio payload field in the order of their division.

9. An audio signal processing method, characterized in that, Applied to the second terminal, including: The system receives a target audio bitstream transmitted by a first terminal, wherein the target audio bitstream is formed by the first terminal merging a basic audio bitstream and an enhanced audio bitstream according to a pre-constructed compatible bitstream format. The basic audio bitstream is obtained by encoding a basic layer audio signal, and the enhanced audio bitstream is obtained by encoding an enhanced layer audio signal. The basic layer audio signal and the enhanced layer audio signal are obtained by the first terminal through signal layer processing of the acquired three-dimensional audio signal. When the deployed audio decoder only supports basic audio decoding, the target audio bitstream is parsed to obtain the basic audio bitstream, and the basic layer audio signal is formed by restoring the basic audio bitstream for playback; When the deployed audio decoder supports 3D audio decoding, the target audio bitstream is parsed to obtain the basic audio bitstream and the enhanced audio bitstream. The target audio signal is then formed by restoring the basic audio bitstream and the enhanced audio bitstream respectively and played. The target audio signal is the 3D audio signal involved in the target audio bitstream.

10. The method according to claim 9, characterized in that, The process of parsing the target audio bitstream to obtain the basic audio bitstream includes: Traverse the target audio bitstream, parse the audio transmission header field in the target audio bitstream, and obtain the audio transmission header field information; After parsing the audio transmission header field, if the payload type parsed from the audio extension header field of the target audio bitstream represents a three-dimensional audio layout, then the audio extension header field is discarded. Based on the audio transmission header field information, the original audio payload field in the target audio bitstream is parsed to obtain the basic audio payload; After parsing the original audio payload field, if the payload type parsed from the extended audio payload field of the target audio bitstream indicates enhanced audio coding, then the extended audio payload field is discarded. After completing and ending the traversal of the target audio bitstream, the bitstream containing the basic audio payload is determined as the basic audio bitstream.

11. The method according to claim 9, characterized in that, The process of parsing the target audio bitstream to obtain the basic audio bitstream and the enhanced audio bitstream includes: Traverse the target audio bitstream, parse the audio transmission header field in the target audio bitstream, and obtain the audio transmission header field information; After parsing the audio transmission header field, the audio extension header field in the target audio bitstream is parsed to obtain audio layout information for the three-dimensional audio layout. After parsing the audio extension header field, the original audio payload field in the target audio bitstream is parsed according to the audio transmission header field information to obtain the basic audio payload; After parsing the original audio payload field, the extended audio payload field of the target audio bitstream is parsed according to the audio layout information to obtain the enhanced audio payload; After completing and ending the traversal of the target audio bitstream, the bitstream containing the basic audio payload is determined as the basic audio bitstream, and the bitstream containing the enhanced audio payload is determined as the enhanced audio bitstream.

12. An audio signal processing device, configured in a first terminal, comprising: A layering module is used to acquire a three-dimensional audio signal and perform signal layering on the three-dimensional audio signal to obtain a base layer audio signal and an enhancement layer audio signal. The base layer audio signal is an audio signal in the three-dimensional audio signal that supports the reference codec, and the enhancement layer audio signal is an audio signal in the three-dimensional audio signal that does not support the reference codec. The encoding module is used to encode the base layer audio signal to obtain a base audio bitstream, and to encode the enhancement layer audio signal to obtain an enhancement audio bitstream; The merging module is used to merge the basic audio bitstream and the enhanced audio bitstream according to a pre-built compatible bitstream format to obtain the target audio bitstream. The compatible bitstream format is formed by reusing and expanding the fields in the original bitstream format. The original bitstream format is the bitstream format used by the reference codec. The transmission module is used to transmit the target audio bitstream to a second terminal, so that the second terminal can decode the basic audio bitstream and the enhanced audio bitstream from the target audio bitstream according to the decoding capability of the deployed audio decoder for playback, or decode only the basic audio bitstream from the target audio bitstream for playback.

13. An audio signal processing device, characterized in that, Configured on the second terminal, including: The receiving module is used to receive the target audio bitstream transmitted by the first terminal. The target audio bitstream is formed by the first terminal merging the basic audio bitstream and the enhanced audio bitstream according to a pre-constructed compatible bitstream format. The basic audio bitstream is obtained by encoding the basic layer audio signal, and the enhanced audio bitstream is obtained by encoding the enhanced layer audio signal. The basic layer audio signal and the enhanced layer audio signal are obtained by the first terminal through signal layer processing of the acquired three-dimensional audio signal. The first decoding module is used to parse the target audio bitstream to obtain the basic audio bitstream when the deployed audio decoder only supports basic audio decoding, and to form the basic layer audio signal for playback by restoring the basic audio bitstream. The second decoding module is used to parse the target audio bitstream to obtain the basic audio bitstream and the enhanced audio bitstream when the deployed audio decoder supports three-dimensional audio decoding, and to play the target audio signal by restoring the basic audio bitstream and the enhanced audio bitstream respectively. The target audio signal is the three-dimensional audio signal involved in the target audio bitstream.

14. A computer device, characterized in that, The computer device includes: One or more processors; a storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the audio signal processing method as described in any one of claims 1-11.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the audio signal processing method as described in any one of claims 1-11.

16. A computer program product comprising a computer program that, when executed by a processor, implements the audio signal processing method according to any one of claims 1-11.

Images