Coding method, apparatus, device, storage medium and computer program

Abstract

The embodiment of the application discloses a kind of coding and decoding method, device, equipment, storage medium and computer program, belong to three-dimensional audio coding technical field.In the method, the signal of the M channels included in the time domain three-dimensional audio signal of current frame is detected respectively to obtain M transient detection results, determine global transient detection result based on M transient detection results, based on global transient detection result, time domain three-dimensional audio signal is converted into frequency domain three-dimensional audio signal, and spatial encoding is carried out to frequency domain three-dimensional audio signal to obtain spatial encoding parameter and the frequency domain signal of N transmission channels, based on global transient detection result, the frequency domain signal of N transmission channels is encoded to obtain frequency domain signal encoding result, spatial encoding parameter is encoded to obtain spatial encoding parameter encoding result, spatial encoding parameter encoding result and frequency domain signal encoding result are written into bitstream.In this way, it can reduce the encoding complexity, improve coding efficiency.

Description

Technical Field

[0001] This application relates to the field of three-dimensional audio encoding and decoding technology, and in particular to an encoding and decoding method, apparatus, device, storage medium and computer program. Background Technology

[0002] 3D audio technology is an audio technology that uses computers, signal processing, and other methods to acquire, process, transmit, render, and play back sound events and 3D sound field information from the real world. To achieve better audio listening effects, 3D audio signals typically require a large amount of data to record the spatial information of the sound scene in more detail. However, transmitting and storing large amounts of data presents challenges, thus necessitating the encoding and decoding of 3D audio signals.

[0003] Higher order ambisonics (HOA) audio technology, as a three-dimensional audio technology, has gained wider attention because it is independent of speaker layout during recording, encoding, and playback, and because HOA format data has the characteristic of being rotatable for playback. Therefore, HOA signals have greater flexibility during playback and have thus attracted more attention.

[0004] Related technologies propose a method for encoding HOA signals. This method first performs a time-frequency transform on the time-domain HOA signal to obtain a frequency-domain HOA signal. Then, spatial encoding is performed on the frequency-domain HOA signal to obtain frequency-domain signals for multiple channels. Next, an inverse time-frequency transform is performed on the frequency-domain signals of each channel to obtain the time-domain signals for each channel. Transient detection is then performed on the time-domain signals of each channel to obtain the transient detection results for each channel. Finally, a time-frequency transform is performed again on the time-domain signals of each channel to obtain the frequency-domain signals for each channel. Finally, the transient detection results for each channel are used to encode the frequency-domain signals for each channel.

[0005] However, the above method requires multiple transformations of the audio signal between the time and frequency domains, which increases the coding complexity and reduces coding efficiency. Summary of the Invention

[0006] This application provides an encoding / decoding method, apparatus, device, storage medium, and computer program, which can reduce encoding complexity and improve encoding efficiency. The technical solution is as follows:

[0007] Firstly, an encoding method is provided, which involves transiently detecting the signals of M channels in the temporal-domain three-dimensional audio signal of the current frame to obtain M transient detection results corresponding to the M channels, where M is an integer greater than 1; determining a global transient detection result based on the M transient detection results; converting the temporal-domain three-dimensional audio signal into a frequency-domain three-dimensional audio signal based on the global transient detection results; spatially encoding the frequency-domain three-dimensional audio signal based on the global transient detection results to obtain spatial encoding parameters and frequency-domain signals of N transmission channels, where N is an integer greater than or equal to 1 and less than or equal to M; encoding the frequency-domain signals of the N transmission channels based on the global transient detection results to obtain frequency-domain signal encoding results; encoding the spatial encoding parameters to obtain spatial encoding parameter encoding results; and writing the spatial encoding parameter encoding results and frequency-domain signal encoding results into the bitstream.

[0008] The transient detection result includes a transient flag, or it includes both a transient flag and transient position information. The transient flag indicates whether the signal in the corresponding channel is a transient signal, and the transient position information indicates the location where the transient occurs within the signal of the corresponding channel. There are multiple ways to determine the M transient detection results corresponding to the M channels; one method will be described below. Since the method for determining the transient detection result is the same for each of the M channels, we will take one channel as an example to describe the method for determining the transient detection result for that channel. For ease of description, this channel will be referred to as the target channel, and the transient flag and transient position information of the target channel will be described below.

[0009] The method for determining the transient flag of the target channel includes: determining the transient detection parameters corresponding to the target channel based on the signal of the target channel; and determining the transient flag corresponding to the target channel based on the transient detection parameters corresponding to the target channel.

[0010] As an example, the transient detection parameter corresponding to the target channel is the absolute value of the inter-frame energy difference. That is, the energy of the target channel signal in the current frame and the energy of the target channel signal in the previous frame are determined. The absolute value of the difference between the energy of the target channel signal in the current frame and the energy of the target channel signal in the previous frame is determined to obtain the absolute value of the inter-frame energy difference. If the absolute value of this inter-frame energy difference exceeds a first energy difference threshold, the transient flag corresponding to the target channel in the current frame is determined to be a first value; otherwise, the transient flag corresponding to the target channel in the current frame is determined to be a second value.

[0011] As another example, the transient detection parameter corresponding to the target channel is the absolute value of the subframe energy difference. That is, the signal of the target channel in the current frame includes signals from multiple subframes. The absolute value of the subframe energy difference for each of these subframes is determined, and thus the transient flag for each subframe is determined. If there is a subframe among these subframes with a transient flag of the first value, then the transient flag corresponding to the target channel in the current frame is determined to be the first value. If there is no subframe among these subframes with a transient flag of the first value, then the transient flag corresponding to the target channel in the current frame is determined to be the second value.

[0012] The methods for determining the transient position information of the target channel include: determining the transient position information of the target channel based on the transient flag corresponding to the target channel.

[0013] As an example, if the transient flag corresponding to the target channel is the first value, then the transient position information corresponding to the target channel is determined. If the transient flag corresponding to the target channel is the second value, then it is determined that the target channel does not have corresponding transient position information, or the transient position information corresponding to the target channel is set to a preset value, such as -1.

[0014] In some embodiments, the transient detection result includes a transient flag, and the global transient detection result includes a global transient flag. The transient flag is used to indicate whether the signal of the corresponding channel is a transient signal. Determining the global transient detection result based on M transient detection results includes: if the number of transient flags with a first value among the M transient flags is greater than or equal to m, then the global transient flag is determined to be the first value, where m is a positive integer greater than 0 and less than M. Alternatively, if the number of channels among the M channels that satisfy a first preset condition and whose corresponding transient flags are the first value is greater than or equal to n, then the global transient flag is determined to be the first value, where n is a positive integer greater than 0 and less than M.

[0015] Here, m and n are preset values, and m and n can be adjusted according to different needs. When the 3D audio signal is a HOA signal, the first preset condition includes channels belonging to the first-order ambisonics (FOA) signal. For example, the channels of the FOA signal can include the first four channels of the HOA signal. In other words, when the 3D audio signal is a HOA signal, if the number of channels in the FOA signal of the current frame whose transient flags are of the first value is greater than or equal to n, then the global transient flag is determined to be the first value. Of course, the first preset condition can also be other conditions.

[0016] In other embodiments, the transient detection result further includes transient location information, and the global transient detection result further includes global transient location information, which is used to indicate the location where the transient occurs in the signal of the corresponding channel. Determining the global transient detection result based on M transient detection results includes: if only one of the M transient flags has a first value, then the transient location information corresponding to the channel with the first value is determined as the global transient location information. If at least two of the M transient flags have a first value, then the transient location information corresponding to the channel with the largest transient detection parameter among the at least two channels corresponding to the at least two transient flags is determined as the global transient location information.

[0017] Alternatively, if at least two of the M transient flags have the first value, and the difference between the transient position information corresponding to the two channels is less than the position difference threshold, then the average value of the transient position information corresponding to the two channels is determined as the global transient position information. The position difference threshold is preset and can be adjusted according to different needs.

[0018] Based on the above description, the transient detection parameter corresponding to a channel is the absolute value of the inter-frame energy difference or the absolute value of the sub-frame energy difference. When the transient detection parameter corresponding to a channel is the absolute value of the inter-frame energy difference, one channel corresponds to one absolute value of the inter-frame energy difference. In this case, the channel with the largest absolute value of the inter-frame energy difference can be selected from at least two channels, and the transient position information corresponding to the selected channel can be determined as the global transient position information. When the transient detection parameter corresponding to a channel is the absolute value of the sub-frame energy difference, one channel corresponds to multiple absolute values ​​of the sub-frame energy difference. In this case, the channel with the largest absolute value of the sub-frame energy difference can be selected from at least two channels, and the transient position information corresponding to the selected channel can be determined as the global transient position information.

[0019] Optionally, based on the global transient detection results, the temporal-domain three-dimensional audio signal is converted into a frequency-domain three-dimensional audio signal, including: determining target coding parameters based on the global transient detection results, wherein the target coding parameters include the window function type of the current frame and / or the frame type of the current frame; and converting the temporal-domain three-dimensional audio signal into a frequency-domain three-dimensional audio signal based on the target coding parameters.

[0020] As an example, the global transient detection result includes a global transient flag. The process of determining the window function type of the current frame based on the global transient detection result includes: if the global transient flag is a first value, then the type of the first preset window function is determined as the window function type of the current frame. If the global transient flag is a second value, then the type of the second preset window function is determined as the window function type of the current frame. The window length of the first preset window function is less than the window length of the second preset window function.

[0021] As another example, the global transient detection result includes a global transient flag and global transient location information. The process of determining the window function type of the current frame based on the global transient detection result includes: if the global transient flag is a first value, then the window function type of the current frame is determined based on the global transient location information. If the global transient flag is a second value, then the type of the third preset window function is determined as the window function type of the current frame, or the window function type of the current frame is determined based on the window function type of the previous frame.

[0022] Since the global transient detection result can include only the global transient flag, or it can include both the global transient flag and global transient location information, and the global transient location information may be the transient location information corresponding to the channel with the transient flag having the first value, or it may be a preset value, the method for determining the frame type of the current frame differs depending on the global transient detection result. Therefore, the following three cases will be explained separately:

[0023] In the first scenario, the global transient detection result includes a global transient flag. The process of determining the frame type of the current frame based on the global transient detection result includes: if the global transient flag has a first value, then the frame type of the current frame is determined to be type one, which indicates that the current frame includes multiple short frames. If the global transient flag has a second value, then the frame type of the current frame is determined to be type two, which indicates that the current frame includes one long frame.

[0024] In the second scenario, the global transient detection result includes a global transient flag and global transient location information. The process of determining the frame type of the current frame based on the global transient detection result includes: if the global transient flag is a first value and the global transient location information meets a second preset condition, then the frame type of the current frame is determined to be a third type, indicating that the current frame includes multiple ultra-short frames. If the global transient flag is a first value and the global transient location information does not meet the second preset condition, then the frame type of the current frame is determined to be a first type, indicating that the current frame includes multiple short frames. If the global transient flag is a second value, then the frame type of the current frame is determined to be a second type, indicating that the current frame includes one long frame. The length of an ultra-short frame is less than the length of a short frame, and the length of a short frame is less than the length of a long frame. The second preset condition can be that the distance from the transient occurrence position indicated by the global transient location information to the start position of the current frame is less than the length of an ultra-short frame, or the distance from the transient occurrence position indicated by the global transient location information to the end position of the current frame is less than the length of an ultra-short frame.

[0025] The third scenario involves global transient detection results including global transient location information. The process of determining the frame type of the current frame based on the global transient detection results includes: if the global transient location information is a preset value, such as -1, then the frame type of the current frame is determined to be the second type, indicating that the current frame includes a long frame. If the global transient location information is not a preset value but meets the second preset condition, then the frame type of the current frame is determined to be the third type, indicating that the current frame includes multiple ultra-short frames. If the global transient location information is not a preset value and does not meet the second preset condition, then the frame type of the current frame is determined to be the first type, indicating that the current frame includes multiple short frames. The length of an ultra-short frame is less than the length of a short frame, and the length of a short frame is less than the length of a long frame. The second preset condition can be that the distance from the transient occurrence position indicated by the global transient location information to the start position of the current frame is less than the length of an ultra-short frame, or the distance from the transient occurrence position indicated by the global transient location information to the end position of the current frame is less than the length of an ultra-short frame.

[0026] It should be noted that the window function type of the current frame indicates the shape and length of the window function corresponding to the current frame, and the window function of the current frame is used to perform windowing processing on the temporal three-dimensional audio signal of the current frame. The frame type of the current frame indicates whether the current frame is an ultra-short frame, a short frame, or a long frame. Ultra-short frames, short frames, and long frames can be distinguished based on the frame duration, and the specific duration can be set according to different needs; this application does not limit this.

[0027] Based on the above description, the target encoding parameters include the window function type and / or the frame type of the current frame. That is, the target encoding parameters include the window function type of the current frame, or the target encoding parameters include the frame type of the current frame, or the target encoding parameters include both the window function type and the frame type of the current frame. The process of converting the temporal-domain 3D audio signal of the current frame into a frequency-domain 3D audio signal based on these target encoding parameters differs depending on the parameters included in the target encoding parameters; therefore, they will be explained separately below.

[0028] In the first case, the target encoding parameters include the window function type of the current frame. In this case, the temporal 3D audio signal of the current frame is windowed based on the window function indicated by the window function type. Then, the windowed 3D audio signal is converted into a frequency domain 3D audio signal.

[0029] In the second case, the target encoding parameters include the frame type of the current frame. In this case, if the frame type is type one, indicating that the current frame comprises multiple short frames, the temporal 3D audio signals of each short frame are converted to frequency-domain 3D audio signals. If the frame type is type two, indicating that the current frame comprises one long frame, the temporal 3D audio signals of the long frame are directly converted to frequency-domain 3D audio signals. If the frame type is type three, indicating that the current frame comprises multiple ultra-short frames, the temporal 3D audio signals of each ultra-short frame are converted to frequency-domain 3D audio signals.

[0030] In the third case, the target encoding parameters include the window function type and frame type of the current frame. In this case, if the frame type is type one, indicating that the current frame includes multiple short frames, then windowing is applied to the temporal 3D audio signals of each short frame based on the window function indicated by the current frame's window function type, and the windowed temporal 3D audio signals of each short frame are then converted to frequency domain 3D audio signals. If the frame type is type two, indicating that the current frame includes one long-short frame, then windowing is applied to the temporal 3D audio signals of the long frame based on the window function indicated by the current frame's window function type, and the windowed temporal 3D audio signals of the long frame are then converted to frequency domain 3D audio signals. If the frame type is type three, indicating that the current frame includes multiple ultra-short frames, then windowing is applied to the temporal 3D audio signals of each ultra-short frame based on the window function indicated by the current frame's window function type, and the windowed temporal 3D audio signals of each ultra-short frame are then converted to frequency domain 3D audio signals.

[0031] In some embodiments, the target encoding parameters can also be encoded to obtain the target encoding parameter encoding result. The target encoding parameter encoding result is then written into the bitstream.

[0032] In some embodiments, spatial encoding of the frequency domain three-dimensional audio signal based on global transient detection results includes: spatial encoding of the frequency domain three-dimensional audio signal based on frame type.

[0033] When spatially encoding the frequency-domain 3D audio signal of the current frame based on its frame type, if the current frame type is type 1 (i.e., the current frame includes multiple short frames), the frequency-domain 3D audio signals of the multiple short frames are interleaved to obtain a long frame's frequency-domain 3D audio signal, and then spatially encoded. If the current frame type is type 2 (i.e., the current frame includes a single long frame), the frequency-domain 3D audio signal of this long frame is spatially encoded. If the current frame type is type 3 (i.e., the current frame includes multiple ultra-short frames), the frequency-domain 3D audio signals of the multiple ultra-short frames are interleaved to obtain a long frame's frequency-domain 3D audio signal, and then spatially encoded.

[0034] In some embodiments, encoding the frequency domain signals of N transmission channels based on the global transient detection results includes: encoding the frequency domain signals of N transmission channels based on the frame type of the current frame.

[0035] As an example, the implementation process of encoding the frequency domain signals of the N transmission channels includes: performing noise shaping processing on the frequency domain signals of the N transmission channels based on the frame type of the current frame; performing transmission channel downmixing processing on the noise-shaped frequency domain signals of the N transmission channels to obtain the downmixed signals; performing quantization and encoding processing on the low-frequency components of the downmixed signals and writing the encoding results into the bitstream; and performing bandwidth expansion and encoding processing on the high-frequency components of the downmixed signals and writing the encoding results into the bitstream.

[0036] Optionally, the method further includes: encoding the global transient detection result to obtain an encoded global transient detection result; and writing the encoded global transient detection result into the bitstream.

[0037] Secondly, a decoding method is provided, which parses the global transient detection results and spatial coding parameters from the bitstream; decodes the global transient detection results and bitstream to obtain the frequency domain signals of N transmission channels; performs spatial decoding on the frequency domain signals of the N transmission channels based on the global transient detection results and spatial coding parameters to obtain the reconstructed frequency domain three-dimensional audio signal; and determines the reconstructed time domain three-dimensional audio signal based on the global transient detection results and the reconstructed frequency domain three-dimensional audio signal.

[0038] Optionally, based on the global transient detection results and the reconstructed frequency domain three-dimensional audio signal, the reconstructed time domain three-dimensional audio signal is determined, including: determining target coding parameters based on the global transient detection results, wherein the target coding parameters include the window function type of the current frame and / or the frame type of the current frame; and converting the reconstructed frequency domain three-dimensional audio signal into a reconstructed time domain three-dimensional audio signal based on the target coding parameters.

[0039] Based on the above description, the target coding parameters include the window function type and / or the frame type of the current frame. That is, the target coding parameters include the window function type of the current frame, or the target coding parameters include the frame type of the current frame, or the target coding parameters include both the window function type and the frame type of the current frame. The process of converting the reconstructed frequency-domain 3D audio signal into a reconstructed temporal-domain 3D audio signal based on these target coding parameters differs depending on the parameters included in the target coding parameters; therefore, they will be explained separately below.

[0040] In the first case, the target encoding parameters include the window function type of the current frame. In this case, the reconstructed frequency-domain 3D audio signal is de-windowed based on the window function indicated by the window function type of the current frame. Then, the de-windowed frequency-domain 3D audio signal is converted into a reconstructed time-domain 3D audio signal.

[0041] Among them, the window removal process is also called the window addition and overlay addition process.

[0042] In the second scenario, the target encoding parameters include the frame type of the current frame. In this case, if the frame type is type one, indicating that the current frame comprises multiple short frames, the reconstructed frequency-domain 3D audio signals of each short frame are converted into time-domain 3D audio signals to obtain the reconstructed time-domain 3D audio signal. If the frame type is type two, indicating that the current frame comprises one long frame, the reconstructed frequency-domain 3D audio signals of the long frame included in the current frame are directly converted into time-domain 3D audio signals to obtain the reconstructed time-domain 3D audio signal. If the frame type is type three, indicating that the current frame comprises multiple ultra-short frames, the reconstructed frequency-domain 3D audio signals of each ultra-short frame are converted into time-domain 3D audio signals to obtain the reconstructed time-domain 3D audio signal.

[0043] In the third case, the target encoding parameters include the window function type and frame type of the current frame. In this case, if the frame type of the current frame is type one, indicating that the current frame includes multiple short frames, then based on the window function indicated by the window function type of the current frame, the frequency domain 3D audio signals of each short frame included in the current frame are de-windowed, and the reconstructed frequency domain 3D audio signals of each short frame after de-windowing are converted into time domain 3D audio signals to obtain the reconstructed time domain 3D audio signal. If the frame type of the current frame is type two, indicating that the current frame includes one long and one short frame, then based on the window function indicated by the window function type of the current frame, the reconstructed frequency domain 3D audio signal of the long frame included in the current frame is de-windowed, and the frequency domain 3D audio signal of the long frame after de-windowing is converted into time domain 3D audio signals to obtain the reconstructed time domain 3D audio signal. If the current frame type is type 3, it indicates that the current frame includes multiple ultra-short frames. In this case, based on the window function indicated by the window function type of the current frame, the frequency domain 3D audio signals of each ultra-short frame included in the current frame are dewindowed, and the reconstructed frequency domain 3D audio signals of each ultra-short frame after dewindowing are converted into time domain 3D audio signals to obtain the reconstructed time domain 3D audio signal.

[0044] Optionally, the global transient detection result includes a global transient flag, and the target encoding parameters include the window function type of the current frame. Determining the target encoding parameters based on the global transient detection result includes: if the global transient flag is a first value, then the type of the first preset window function is determined as the window function type of the current frame; if the global transient flag is a second value, then the type of the second preset window function is determined as the window function type of the current frame; wherein the window length of the first preset window function is less than the window length of the second preset window function.

[0045] Optionally, the global transient detection result includes a global transient flag and global transient location information, and the target coding parameters include the window function type of the current frame; determining the target coding parameters based on the global transient detection result includes: if the global transient flag is a first value, then determining the window function type of the current frame based on the global transient location information.

[0046] Thirdly, an encoding apparatus is provided, which has the function of implementing the encoding method behavior described in the first aspect. The encoding apparatus includes at least one module for implementing the encoding method provided in the first aspect.

[0047] Fourthly, a decoding apparatus is provided, which has the function of implementing the decoding method behavior described in the second aspect above. The decoding apparatus includes at least one module for implementing the decoding method provided in the second aspect above.

[0048] Fifthly, an encoding end device is provided, comprising a processor and a memory, the memory being used to store a program for executing the encoding method provided in the first aspect. The processor is configured to execute the program stored in the memory to implement the encoding method provided in the first aspect.

[0049] Optionally, the encoding end device may further include a communication bus for establishing a connection between the processor and the memory.

[0050] Sixthly, a decoding device is provided, comprising a processor and a memory, the memory being used to store a program for executing the decoding method provided in the second aspect. The processor is configured to execute the program stored in the memory to implement the decoding method provided in the second aspect.

[0051] Optionally, the decoding device may further include a communication bus for establishing a connection between the processor and the memory.

[0052] In a seventh aspect, a computer-readable storage medium is provided, wherein the storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the encoding method described in the first aspect, or to perform the steps of the decoding method described in the second aspect.

[0053] Eighthly, a computer program product comprising instructions is provided, which, when executed on a computer, cause the computer to perform the steps of the encoding method described in the first aspect, or the steps of the decoding method described in the second aspect. Alternatively, a computer program is provided that, when executed, implements the steps of the encoding method described in the first aspect, or the steps of the decoding method described in the second aspect.

[0054] A ninth aspect provides a computer-readable storage medium comprising a bitstream obtained by the encoding method described in the first aspect.

[0055] The technical effects obtained by the third, fourth, fifth, sixth, seventh, eighth and ninth aspects mentioned above are similar to the technical effects obtained by the corresponding technical means in the first or second aspects, and will not be repeated here.

[0056] The technical solutions provided in this application can bring at least the following beneficial effects:

[0057] By performing transient detection on the M channels of the temporal-domain 3D audio signal in the current frame, a global transient detection result is determined. Then, based on the global transient detection result, time-frequency transformation, spatial coding, and frequency domain coding of each transmission channel are performed sequentially. In particular, when coding the frequency domain signals of each transmission channel obtained after spatial coding, the global transient detection result guides the coding of the frequency domain signals of each transmission channel. This eliminates the need to convert the frequency domain signals of each transmission channel to the time domain to determine the corresponding transient detection result for each channel. Consequently, multiple transformations of the 3D audio signal between the time and frequency domains are avoided, reducing coding complexity and improving coding efficiency. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application;

[0059] Figure 2 This is a schematic diagram of an implementation environment for a terminal scenario provided in an embodiment of this application;

[0060] Figure 3 This is a schematic diagram illustrating the implementation environment of a transcoding scenario for a wireless or core network device provided in an embodiment of this application;

[0061] Figure 4 This is a schematic diagram of an implementation environment for a broadcast television scenario provided in an embodiment of this application;

[0062] Figure 5 This is a schematic diagram of an implementation environment for a virtual reality streaming scene provided in an embodiment of this application;

[0063] Figure 6 This is a flowchart of the first encoding method provided in the embodiments of this application;

[0064] Figure 7 This is the first type of embodiment provided in this application regarding... Figure 6 An exemplary block diagram of the encoding method shown;

[0065] Figure 8 This is the second type of embodiment provided in this application regarding... Figure 6 An exemplary block diagram of the encoding method shown;

[0066] Figure 9 This is a flowchart of the first decoding method provided in the embodiments of this application;

[0067] Figure 10 This application provides an embodiment regarding... Figure 9 An exemplary block diagram of the decoding method shown;

[0068] Figure 11This is a flowchart of the second encoding method provided in the embodiments of this application;

[0069] Figure 12 This is the first type of embodiment provided in this application regarding... Figure 11 An exemplary block diagram of the encoding method shown;

[0070] Figure 13 This is the second type of embodiment provided in this application regarding... Figure 11 An exemplary block diagram of the encoding method shown;

[0071] Figure 14 This is a flowchart of the second decoding method provided in the embodiments of this application;

[0072] Figure 15 This application provides an embodiment regarding... Figure 14 An exemplary block diagram of the decoding method shown;

[0073] Figure 16 This is a schematic diagram of the structure of an encoding device provided in an embodiment of this application;

[0074] Figure 17 This is a schematic diagram of the structure of a decoding device provided in an embodiment of this application;

[0075] Figure 18 This is a schematic block diagram of an encoding / decoding device provided in an embodiment of this application. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0077] Before providing a detailed explanation of the encoding and decoding methods provided in the embodiments of this application, the terminology and implementation environment involved in the embodiments of this application will be introduced first.

[0078] To facilitate understanding, the terms used in the embodiments of this application will be explained first.

[0079] Encoding refers to the process of compressing an audio signal into a bitstream. It should be noted that an audio signal compressed into a bitstream can be called an encoded audio signal or a compressed audio signal.

[0080] Decoding refers to the process of restoring an encoded bitstream into a reconstructed audio signal according to specific syntax rules and processing methods.

[0081] Three-dimensional audio signals: These are signals with multiple channels used to characterize the sound field in three-dimensional space. They can be one or more combinations of HOA signals, multi-channel signals, and object audio signals. For HOA signals, the number of channels is related to the order of the three-dimensional audio signal. For example, if the three-dimensional audio signal is an A-order signal, then the number of channels is (A+1). 2 .

[0082] The three-dimensional audio signal mentioned below can be any three-dimensional audio signal, such as one or more combinations of HOA signals, multi-channel signals, and object audio signals.

[0083] Transient signals: These are used to characterize the transient phenomena of signals in corresponding channels of a three-dimensional audio signal. If the signal in a certain channel is a transient signal, it indicates that the signal in that channel is a non-stationary signal. For example, signals with large energy changes over a short period of time, such as the sound of drums or percussion instruments.

[0084] The implementation environment involved in the embodiments of this application will be described next.

[0085] Please refer to Figure 1 , Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application. The implementation environment includes a source device 10, a destination device 20, a link 30, and a storage device 40. The source device 10 can generate an encoded three-dimensional audio signal. Therefore, the source device 10 can also be referred to as a three-dimensional audio signal encoding device. The destination device 20 can decode the encoded three-dimensional audio signal generated by the source device 10. Therefore, the destination device 20 can also be referred to as a three-dimensional audio signal decoding device. The link 30 can receive the encoded three-dimensional audio signal generated by the source device 10 and transmit the encoded three-dimensional audio signal to the destination device 20. The storage device 40 can receive the encoded three-dimensional audio signal generated by the source device 10 and store the encoded three-dimensional audio signal. Under these conditions, the destination device 20 can directly obtain the encoded three-dimensional audio signal from the storage device 40. Alternatively, the storage device 40 can correspond to a file server or another intermediate storage device that can store the encoded three-dimensional audio signal generated by the source device 10. Under these conditions, the destination device 20 can stream or download the encoded three-dimensional audio signal stored in the storage device 40.

[0086] Both source device 10 and destination device 20 may include one or more processors and memory coupled to the one or more processors. This memory may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or any other media that can be used to store desired program code in the form of computer-accessible instructions or data structures. For example, both source device 10 and destination device 20 may include desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, handsets such as so-called "smartphones," televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or the like.

[0087] Link 30 may include one or more media or devices capable of transmitting encoded three-dimensional audio signals from source device 10 to destination device 20. In one possible implementation, link 30 may include one or more communication media enabling source device 10 to directly transmit encoded three-dimensional audio signals to destination device 20 in real time. In this embodiment, source device 10 may modulate the encoded three-dimensional audio signals based on a communication standard, such as a wireless communication protocol, and transmit the modulated three-dimensional audio signals to destination device 20. The one or more communication media may include wireless and / or wired communication media, such as radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, wide area network, or global network (e.g., the Internet). The one or more communication media may include routers, switches, base stations, or other devices facilitating communication from source device 10 to destination device 20, etc., which are not specifically limited in this embodiment.

[0088] In one possible implementation, storage device 40 can store the received encoded 3D audio signal sent by source device 10, and destination device 20 can directly retrieve the encoded 3D audio signal from storage device 40. Under such conditions, storage device 40 can include any of a variety of distributed or locally accessed data storage media. For example, any of these distributed or locally accessed data storage media can be a hard disk drive, Blu-ray disc, digital versatile disc (DVD), compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded 3D audio signals.

[0089] In one possible implementation, storage device 40 may correspond to a file server or another intermediate storage device capable of storing the encoded 3D audio signal generated by source device 10, and destination device 20 may stream or download the 3D audio signal stored in storage device 40. The file server may be any type of server capable of storing and transmitting the encoded 3D audio signal to destination device 20. In one possible implementation, the file server may include a web server, a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive, etc. Destination device 20 may acquire the encoded 3D audio signal via any standard data connection (including an Internet connection). Any standard data connection may include a wireless channel (e.g., Wi-Fi connection), a wired connection (e.g., digital subscriber line (DSL), cable modem, etc.), or a combination of both suitable for acquiring the encoded 3D audio data stored on a file server. The transmission of the encoded 3D audio signal from storage device 40 may be streaming, downloading, or a combination of both.

[0090] The technology described in this application embodiment can be applied to... Figure 1 The source device 10 shown, which encodes a three-dimensional audio signal, can also be applied to a destination device 20, which decodes the encoded three-dimensional audio signal.

[0091] exist Figure 1In the illustrated implementation environment, source device 10 includes a data source 120, an encoder 100, and an output interface 140. In some embodiments, the output interface 140 may include a modem / demodulator and / or a transmitter, wherein the transmitter may also be referred to as a transmitter. Data source 120 may include an image capture device (e.g., a camera, etc.), an archive containing previously captured 3D audio signals, a feed interface for receiving 3D audio signals from a 3D audio signal content provider, and / or a computer graphics system for generating 3D audio signals, or a combination of these sources of 3D audio signals.

[0092] Data source 120 can send three-dimensional audio signals to encoder 100, which can encode the received three-dimensional audio signals from data source 120 to obtain encoded three-dimensional audio signals. The encoder can then send the encoded three-dimensional audio signals to an output interface. In some embodiments, source device 10 directly sends the encoded three-dimensional audio signals to destination device 20 via output interface 140. In other embodiments, the encoded three-dimensional audio signals can also be stored on storage device 40 for later retrieval by destination device 20 for decoding and / or display.

[0093] exist Figure 1 In the illustrated implementation environment, destination device 20 includes an input interface 240, a decoder 200, and a display device 220. In some embodiments, the input interface 240 includes a receiver and / or a modem. The input interface 240 may receive encoded 3D audio signals via link 30 and / or from storage device 40, and then send them to the decoder 200, which may decode the received encoded 3D audio signals to obtain decoded 3D audio signals. The decoder may send the decoded 3D audio signals to the display device 220. The display device 220 may be integrated with destination device 20 or may be external to destination device 20. Generally, the display device 220 displays the decoded 3D audio signals. The display device 220 may be any type of display device, for example, a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices.

[0094] although Figure 1Not shown, but in some respects, encoder 100 and decoder 200 may be integrated with each other and may include appropriate multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software for encoding both audio and video in a common data stream or separate data streams. In some embodiments, the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol, or other protocols such as User Datagram Protocol (UDP), if applicable.

[0095] Encoder 100 and decoder 200 may each be any of the following circuits: one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, or any combination thereof. If the techniques of the embodiments of this application are implemented in part in software, the apparatus may store instructions for software in a suitable non-volatile computer-readable storage medium, and the instructions may be executed in hardware using one or more processors to implement the techniques of the embodiments of this application. Any of the foregoing (including hardware, software, combinations of hardware and software, etc.) may be considered as one or more processors. Each of encoder 100 and decoder 200 may be included in one or more encoders or decoders, either of which may be integrated as part of a combined encoder / decoder (encoder-decoder) in the respective apparatus.

[0096] In this application embodiment, encoder 100 may be generally referred to as an apparatus that “signals” or “sends” certain information to, for example, decoder 200. The terms “signals” or “sends” may generally refer to the transmission of syntax elements and / or other data for decoding a compressed three-dimensional audio signal. This transmission may occur in real-time or nearly real-time. Alternatively, this communication may occur after a period of time, for example, during encoding when syntax elements are stored in a computer-readable storage medium in a encoded bitstream, and the decoding apparatus may then retrieve the syntax elements at any time after they have been stored in this medium.

[0097] The encoding and decoding methods provided in this application can be applied to a variety of scenarios, and several of these scenarios will be described below.

[0098] Please refer to Figure 2 , Figure 2 This is a schematic diagram illustrating an implementation environment of an encoding / decoding method provided in this application, applied to a terminal scenario. The implementation environment includes a first terminal 101 and a second terminal 201, which are communicatively connected. This communication connection can be wireless or wired; this application does not limit the type of connection.

[0099] In this configuration, the first terminal 101 can be either a transmitting device or a receiving device. Similarly, the second terminal 201 can be either a receiving device or a transmitting device. When the first terminal 101 is a transmitting device, the second terminal 201 is a receiving device; when the first terminal 101 is a receiving device, the second terminal 201 is a transmitting device.

[0100] The following section will use the first terminal 101 as the transmitting device and the second terminal 201 as the receiving device as an example for further explanation.

[0101] The first terminal 101 can be as described above. Figure 1 The source device 10 in the implementation environment shown. The second terminal 201 can be the aforementioned Figure 1 The destination device 20 in the implementation environment shown. Both the first terminal 101 and the second terminal 201 include an audio acquisition module, an audio playback module, an encoder, a decoder, a channel coding module, and a channel decoding module.

[0102] The audio acquisition module in the first terminal 101 acquires the three-dimensional audio signal and transmits it to the encoder. The encoder encodes the three-dimensional audio signal using the encoding method provided in this embodiment, which can be called source coding. Then, in order to realize the transmission of the three-dimensional audio signal in the channel, the channel coding module needs to perform channel coding again, and then transmit the encoded bitstream in the digital channel through a wireless or wired network communication device.

[0103] The second terminal 201 receives the bit stream transmitted in the digital channel through a wireless or wired network communication device. The channel decoding module performs channel decoding on the bit stream, and then the decoder uses the decoding method provided in this application embodiment to decode the three-dimensional audio signal, which is then played through the audio playback module.

[0104] The first terminal 101 and the second terminal 201 can be any electronic product that can interact with the user through one or more means such as a keyboard, touchpad, touch screen, remote control, voice interaction or handwriting device, such as personal computer (PC), mobile phone, smartphone, personal digital assistant (PDA), wearable device, handheld PC (PPC), tablet computer, smart car system, smart TV, smart speaker, etc.

[0105] Those skilled in the art should understand that the above-described terminals are merely examples, and other existing or future terminals that are applicable to the embodiments of this application should also be included within the scope of protection of the embodiments of this application, and are hereby incorporated by reference.

[0106] Please refer to Figure 3 , Figure 3 This is a schematic diagram illustrating the implementation environment of an encoding / decoding method provided in this application, applied to a transcoding scenario in a wireless or core network device. The implementation environment includes a channel decoding module, an audio decoder, an audio encoder, and a channel coding module.

[0107] The audio decoder can be a decoder using the decoding method provided in the embodiments of this application, or it can be a decoder using other decoding methods. The audio encoder can be an encoder using the encoding method provided in the embodiments of this application, or it can be an encoder using other encoding methods. When the audio decoder is a decoder using the decoding method provided in the embodiments of this application, the audio encoder is an encoder using other encoding methods; conversely, when the audio decoder is a decoder using other decoding methods, the audio encoder is an encoder using the encoding method provided in the embodiments of this application.

[0108] In the first case, the audio decoder is a decoder using the decoding method provided in the embodiments of this application, and the audio encoder is an encoder using other encoding methods.

[0109] At this point, the channel decoding module performs channel decoding on the received bitstream, and then the audio decoder performs source decoding using the decoding method provided in this embodiment. The audio encoder then encodes the data using other encoding methods, achieving a conversion from one format to another, i.e., transcoding. Finally, the data is channel-encoded and transmitted.

[0110] In the second case, the audio decoder is a decoder that uses other decoding methods, and the audio encoder is an encoder that uses the encoding method provided in the embodiments of this application.

[0111] At this point, the channel decoding module performs channel decoding on the received bitstream, then the audio decoder performs source decoding using other decoding methods, and then the audio encoder performs encoding using the encoding method provided in this application embodiment, achieving a conversion from one format to another, i.e., transcoding. Afterwards, it is transmitted after channel encoding.

[0112] Wireless devices can include wireless access points, wireless routers, wireless connectors, etc. Core network devices can include mobility management entities, gateways, etc.

[0113] Those skilled in the art should understand that the above-described wireless devices or core network devices are merely examples. Other existing or future wireless or core network devices that are applicable to the embodiments of this application should also be included within the scope of protection of the embodiments of this application, and are hereby incorporated by reference.

[0114] Please refer to Figure 4 , Figure 4 This is a schematic diagram illustrating the implementation environment of an encoding / decoding method provided in this application, applied to a broadcast television scenario. Broadcast television scenarios are divided into live broadcast scenarios and post-production scenarios. For live broadcast scenarios, the implementation environment includes a live program 3D audio production module, a 3D audio encoding module, a set-top box, and a speaker assembly; the set-top box includes a 3D audio decoding module. For post-production scenarios, the implementation environment includes a post-production program 3D audio production module, a 3D audio encoding module, a network receiver, a mobile terminal, and headphones, etc.

[0115] In a live broadcast scenario, the 3D sound production module generates a 3D sound signal, which includes a 3D audio signal. This 3D sound signal is encoded using the encoding method described in this application to obtain a bitstream. This bitstream is transmitted to the user side via a broadcast network, where a 3D sound decoder in a set-top box decodes it using the decoding method provided in this application to reconstruct the 3D sound signal, which is then played back by a speaker array. Alternatively, the bitstream can be transmitted to the user side via the internet, where a 3D sound decoder in a network receiver decodes it using the decoding method provided in this application to reconstruct the 3D sound signal, which is then played back by a speaker array. Or, the bitstream can be transmitted to the user side via the internet, where a 3D sound decoder in a mobile terminal decodes it using the decoding method provided in this application to reconstruct the 3D sound signal, which is then played back by headphones.

[0116] In post-production scenarios, the 3D sound production module generates a 3D sound signal. This 3D sound signal is encoded using the encoding method described in this application to obtain a bitstream. This bitstream is transmitted to the user side via a broadcast network, where a 3D sound decoder in a set-top box uses the decoding method provided in this application to decode and reconstruct the 3D sound signal, which is then played back by a speaker system. Alternatively, the bitstream can be transmitted to the user side via the internet, where a 3D sound decoder in a network receiver uses the decoding method provided in this application to decode and reconstruct the 3D sound signal, which is then played back by a speaker system. Or, the bitstream can be transmitted to the user side via the internet, where a 3D sound decoder in a mobile terminal uses the decoding method provided in this application to decode and reconstruct the 3D sound signal, which is then played back by headphones.

[0117] Please refer to Figure 5 , Figure 5 This is a schematic diagram illustrating an implementation environment for an encoding / decoding method provided in this application, applied to a virtual reality streaming scene. The implementation environment includes an encoding end and a decoding end. The encoding end includes an acquisition module, a preprocessing module, an encoding module, a packaging module, and a sending module. The decoding end includes an unpacking module, a decoding module, a rendering module, and headphones.

[0118] The acquisition module acquires the three-dimensional audio signal, and then the preprocessing module performs preprocessing operations, including filtering out low-frequency components of the signal, typically using 20Hz or 50Hz as the dividing point, and extracting directional information from the signal. Afterwards, the encoding module performs encoding processing using the encoding method provided in this application embodiment. The encoded signal is then packaged by the packaging module and finally sent to the decoding end by the sending module.

[0119] The unpacking module at the decoding end first unpacks the signal, then the decoding module decodes it using the decoding method provided in this application embodiment. Finally, the rendering module performs binaural rendering processing on the decoded signal, and the rendered signal is mapped onto the listener's headphones. These headphones can be standalone headphones or headphones integrated into virtual reality glasses.

[0120] It should be noted that the system architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0121] The encoding and decoding methods provided in the embodiments of this application will now be explained in detail. It should be noted that, in conjunction with... Figure 1In the implementation environment shown, any of the encoding methods described below can be performed by the encoder 100 in the source device 10. Any of the decoding methods described below can be performed by the decoder 200 in the destination device 20.

[0122] Please refer to Figure 6 , Figure 6 This is a flowchart of the first encoding method provided in the embodiments of this application. The encoding method is applied to an encoding end device and includes the following steps.

[0123] Step 601: Perform transient detection on the M channels of the temporal three-dimensional audio signal of the current frame to obtain M transient detection results corresponding to the M channels, where M is an integer greater than 1.

[0124] The M transient detection results correspond one-to-one with the M channels of the temporal-domain three-dimensional audio signal in the current frame. Each transient detection result includes a transient flag, or alternatively, a transient flag and transient position information. The transient flag indicates whether the signal in the corresponding channel is transient, and the transient position information indicates the location where the transient occurs within the signal of the corresponding channel.

[0125] There are several ways to determine the M transient detection results corresponding to the M channels. One method will be described below. Since the method for determining the transient detection results for each of the M channels is the same, we will take one channel as an example to describe the method for determining the transient detection results for that channel. For ease of description, this channel will be referred to as the target channel, and the transient flag and transient position information of the target channel will be described below.

[0126] Transient flags of the target channel

[0127] Based on the signal from the target channel, determine the transient detection parameters corresponding to the target channel. Based on the transient detection parameters corresponding to the target channel, determine the transient flag corresponding to the target channel.

[0128] As an example, the transient detection parameter corresponding to the target channel is the absolute value of the inter-frame energy difference. That is, the energy of the target channel signal in the current frame and the energy of the target channel signal in the previous frame are determined. The absolute value of the difference between the energy of the target channel signal in the current frame and the energy of the target channel signal in the previous frame is determined to obtain the absolute value of the inter-frame energy difference. If the absolute value of this inter-frame energy difference exceeds a first energy difference threshold, the transient flag corresponding to the target channel in the current frame is determined to be a first value; otherwise, the transient flag corresponding to the target channel in the current frame is determined to be a second value.

[0129] Based on the above description, the transient flag is used to indicate whether the signal of the corresponding channel is a transient signal. Therefore, if the absolute value of the inter-frame energy difference exceeds the first energy difference threshold, it indicates that the signal of the target channel in the current frame is a transient signal, and the transient flag corresponding to the target channel in the current frame is determined to be the first value. If the absolute value of the inter-frame energy difference does not exceed the first energy difference threshold, it indicates that the signal of the target channel in the current frame is not a transient signal, and the transient flag corresponding to the target channel in the current frame is determined to be the second value.

[0130] It should be noted that the first and second values ​​can be represented in various ways. For example, the first value can be true and the second value can be false. Or, the first value can be 1 and the second value can be 0. Of course, other methods are also possible. The first energy difference threshold is preset, and the first energy difference threshold can be adjusted according to different needs.

[0131] As another example, the transient detection parameter corresponding to the target channel is the absolute value of the subframe energy difference. That is, the signal of the target channel in the current frame includes signals from multiple subframes. The absolute value of the subframe energy difference for each of these subframes is determined, and thus the transient flag for each subframe is determined. If there is a subframe among these subframes with a transient flag of the first value, then the transient flag corresponding to the target channel in the current frame is determined to be the first value. If there is no subframe among these subframes with a transient flag of the first value, then the transient flag corresponding to the target channel in the current frame is determined to be the second value.

[0132] The implementation method for determining the transient flag of each subframe in the multiple subframes is the same. Therefore, the following explanation uses the i-th subframe as an example, where i is a positive integer. That is, the energy of the signal in the i-th subframe and the energy of the signal in the (i-1)-th subframe are determined. The absolute value of the difference between the energy of the signal in the i-th subframe and the energy of the signal in the (i-1)-th subframe is determined to obtain the absolute value of the subframe energy difference corresponding to the i-th subframe. If the absolute value of the subframe energy difference corresponding to the i-th subframe exceeds a second energy difference threshold, the transient flag of the i-th subframe is determined to be the first value; otherwise, the transient flag of the i-th subframe is determined to be the second value.

[0133] Based on the above description, the transient flag is used to indicate whether the signal of the corresponding channel is a transient signal. Therefore, if the absolute value of the subframe energy difference corresponding to the i-th subframe exceeds the second energy difference threshold, it indicates that the signal of the i-th subframe is a transient signal, and the transient flag of the i-th subframe is determined to be the first value. If the absolute value of the subframe energy difference corresponding to the i-th subframe does not exceed the second energy difference threshold, it indicates that the signal of the i-th subframe is not a transient signal, and the transient flag of the i-th subframe is determined to be the second value.

[0134] It should be noted that when i = 0, the energy of the signal in the (i-1)th subframe is the energy of the signal in the last subframe of the target channel in the previous frame. The second energy difference threshold is preset and can be adjusted according to different needs. Furthermore, the second energy difference threshold can be the same as or different from the first energy difference threshold.

[0135] Transient position information of the target channel

[0136] Based on the transient flags corresponding to the target channel, determine the transient location information corresponding to the target channel.

[0137] As an example, if the transient flag corresponding to the target channel is the first value, then the transient position information corresponding to the target channel is determined. If the transient flag corresponding to the target channel is the second value, then it is determined that the target channel does not have corresponding transient position information, or the transient position information corresponding to the target channel is set to a preset value, such as -1.

[0138] That is, when the transient flag corresponding to the target channel is the second value, it indicates that the signal of the target channel is not a transient signal. In this case, the transient detection result of the target channel does not include transient position information, or the transient position information corresponding to the target channel is directly set to a preset value, which is used to indicate that the signal of the target channel is not a transient signal. In other words, the transient detection result of a transient signal includes both transient flag and transient position information, while the transient detection result of a non-transient signal may include either a transient flag or both transient flag and transient position information.

[0139] It should be noted that when the transient flag corresponding to the target channel is the first value, there are multiple ways to determine the transient position information corresponding to the target channel. As an example, the signal of the target channel in the current frame includes signals from multiple subframes. The subframe with the first transient flag and the highest absolute value of the subframe energy difference is selected from these multiple subframes, and the sequence number of the selected subframe is determined as the transient position information corresponding to the target channel in the current frame.

[0140] For example, the transient flag corresponding to the target channel in the current frame is the first value. The signal of the target channel in the current frame includes signals from four subframes, i = 0, 1, 2, 3. The absolute value of the subframe energy difference of the 0th subframe is 18, the absolute value of the subframe energy difference of the 1st subframe is 21, the absolute value of the subframe energy difference of the 2nd subframe is 24, and the absolute value of the subframe energy difference of the 3rd subframe is 35. Assuming that the second energy difference threshold is preset to 20, the signal of the 1st subframe, the signal of the 2nd subframe, and the signal of the 3rd subframe are all transient signals. At this time, it is determined that the transient flags of the 1st, 2nd, and 3rd subframes are all the first value, and the subframe with the highest absolute value of the subframe energy difference among these three subframes is the 3rd subframe. Therefore, the sequence number 3 of the 3rd subframe is determined as the transient position information corresponding to the target channel in the current frame.

[0141] Step 602: Based on the M transient detection results, determine the global transient detection result.

[0142] In some embodiments, the global transient detection result includes a global transient flag. If the number of transient flags with a first value among the M transient flags is greater than or equal to m, then the global transient flag is determined to be the first value, where m is a positive integer greater than 0 and less than M. Alternatively, if the number of channels among the M channels that meet a first preset condition and whose corresponding transient flags are the first value is greater than or equal to n, then the global transient flag is determined to be the first value, where n is a positive integer greater than 0 and less than M.

[0143] For example, the 3D audio signal of the current frame is a 3rd-order HOA signal, and the number of channels of this HOA signal is (3+1). 2 That is, 16. Assuming m is 1, if the number of transient flags with the first value among these 16 transient flags is greater than or equal to 1, then the global transient flag is determined to be the first value. Alternatively, the first preset condition includes channels belonging to the FOA signal. For example, the channels of the FOA signal can include the first 4 channels of the HOA signal. Assuming that the channels among these M channels that satisfy the first preset condition are the channels where the FOA signal is located in the current frame, and n is 1. If the number of channels with the first value among the channels belonging to the FOA among these 16 channels is greater than or equal to 1, then the global transient flag is determined to be the first value.

[0144] Where m and n are preset values, and m and n can be adjusted according to different needs. When the three-dimensional audio signal is a HOA signal, the first preset condition includes channels belonging to the FOA signal. Among the M channels, the channel that satisfies the first preset condition is the channel where the FOA signal in the three-dimensional audio signal of the current frame is located. The FOA signal is the signal of the first 4 channels in the HOA signal. Of course, the first preset condition can also be other conditions.

[0145] In other embodiments, the global transient detection result also includes global transient position information. If only one of the M transient flags has a first value, the transient position information corresponding to the channel with the first value is determined as the global transient position information. If at least two of the M transient flags have a first value, the transient position information corresponding to the channel with the largest transient detection parameter among the at least two channels corresponding to these at least two flags is determined as the global transient position information. Alternatively, if at least two of the M transient flags have a first value, and the difference between the transient position information corresponding to the two channels is less than a position difference threshold, the average value of the transient position information corresponding to the two channels is determined as the global transient position information. The position difference threshold is preset and can be adjusted according to different needs.

[0146] Based on the above description, the transient detection parameter corresponding to a channel is the absolute value of the inter-frame energy difference or the absolute value of the sub-frame energy difference. When the transient detection parameter corresponding to a channel is the absolute value of the inter-frame energy difference, one channel corresponds to one absolute value of the inter-frame energy difference. In this case, the channel with the largest absolute value of the inter-frame energy difference can be selected from at least two channels, and the transient position information corresponding to the selected channel can be determined as the global transient position information. When the transient detection parameter corresponding to a channel is the absolute value of the sub-frame energy difference, one channel corresponds to multiple absolute values ​​of the sub-frame energy difference. In this case, the channel with the largest absolute value of the sub-frame energy difference can be selected from at least two channels, and the transient position information corresponding to the selected channel can be determined as the global transient position information.

[0147] For example, for a third-order HOA signal, if only the transient flag corresponding to the third channel among the 16 transient flags of the HOA signal has the first value, then the transient position information corresponding to the third channel can be directly determined as the global transient position information.

[0148] If three out of the 16 transient flags of the HOA signal have the first value, namely channel 1, channel 2, and channel 3, and the transient position information corresponding to channel 1 is 1, with an absolute value of 22 for the inter-frame energy difference, the transient position information corresponding to channel 2 is 2, with an absolute value of 23 for the inter-frame energy difference, and the transient position information corresponding to channel 3 is 3, with an absolute value of 28 for the inter-frame energy difference, then the transient position information 3 corresponding to channel 3 is determined as the global transient position information.

[0149] For example, if three out of the 16 transient flags of the HOA signal have the first value, namely channel 1, channel 2, and channel 3, and the transient position information corresponding to channel 1 is 1, the signal of channel 1 includes three subframes, and the absolute values ​​of the subframe energy differences of these three subframes are 20, 18, and 22 respectively, the transient position information corresponding to channel 2 is 2, the signal of channel 2 includes three subframes, and the absolute values ​​of the subframe energy differences of these three subframes are 20, 23, and 25 respectively, and the transient position information corresponding to channel 3 is 3, the signal of channel 3 includes three subframes, and the absolute values ​​of the subframe energy differences of these three subframes are 25, 28, and 30, then the channel with the largest absolute value of the subframe energy difference among these three channels is channel 3, and the transient position information 3 corresponding to channel 3 is determined as the global transient position information.

[0150] If three out of the 16 transient flags of the HOA signal have the first value, namely channel 1, channel 2, and channel 3, and the transient position information corresponding to channel 1 is 1, that of channel 2 is 3, and that of channel 3 is 6, and the difference 2 between the transient position information corresponding to channel 1 and channel 2 is less than a pre-set position difference threshold 3, then the average value 2 of the transient position information corresponding to channel 1 and channel 2 is determined as the global transient position information.

[0151] Step 603: Based on the global transient detection results, convert the time-domain three-dimensional audio signal of the current frame into a frequency-domain three-dimensional audio signal.

[0152] In some embodiments, target coding parameters are determined based on global transient detection results. These target coding parameters include the window function type and / or the frame type of the current frame. Based on these target coding parameters, the temporal-domain three-dimensional audio signal of the current frame is converted into a frequency-domain three-dimensional audio signal.

[0153] As an example, the global transient detection result includes a global transient flag. The process of determining the window function type of the current frame based on the global transient detection result includes: if the global transient flag is a first value, then the type of the first preset window function is determined as the window function type of the current frame. If the global transient flag is a second value, then the type of the second preset window function is determined as the window function type of the current frame. The window length of the first preset window function is less than the window length of the second preset window function.

[0154] As another example, the global transient detection result includes a global transient flag and global transient location information. The process of determining the window function type of the current frame based on the global transient detection result includes: if the global transient flag is a first value, then the window function type of the current frame is determined based on the global transient location information. If the global transient flag is a second value, then the type of the third preset window function is determined as the window function type of the current frame, or the window function type of the current frame is determined based on the window function type of the previous frame.

[0155] When the global transient flag is set to its first value, there are several ways to determine the window function type of the current frame based on the global transient location information. For example, the type of the fourth preset window function can be adjusted based on the global transient location information so that the center position of the fourth preset window function corresponds to the location where the global transient occurs, thereby maximizing the value of the window function corresponding to the location where the global transient occurs. Alternatively, a window function corresponding to the location where the global transient occurs can be selected from the window function set, and the type of the selected window function can be determined as the window function type of the current frame. That is, the window function set stores the window functions corresponding to each transient location, allowing selection of the window function corresponding to the location where the global transient occurs.

[0156] In addition, there are various methods for determining the window function type of the current frame based on the window function type of the previous frame. For details, please refer to the relevant technologies. This application will not elaborate on these methods in the embodiments.

[0157] Since the global transient detection result can include only the global transient flag, or it can include both the global transient flag and global transient location information, and the global transient location information may be the transient location information corresponding to the channel with the transient flag having the first value, or it may be a preset value, the method for determining the frame type of the current frame differs depending on the global transient detection result. Therefore, the following three cases will be explained separately:

[0158] In the first scenario, the global transient detection result includes a global transient flag. The process of determining the frame type of the current frame based on the global transient detection result includes: if the global transient flag has a first value, then the frame type of the current frame is determined to be type one, which indicates that the current frame includes multiple short frames. If the global transient flag has a second value, then the frame type of the current frame is determined to be type two, which indicates that the current frame includes one long frame.

[0159] In the second scenario, the global transient detection result includes a global transient flag and global transient location information. The process of determining the frame type of the current frame based on the global transient detection result includes: if the global transient flag is a first value and the global transient location information meets a second preset condition, then the frame type of the current frame is determined to be a third type, indicating that the current frame includes multiple ultra-short frames. If the global transient flag is a first value and the global transient location information does not meet the second preset condition, then the frame type of the current frame is determined to be a first type, indicating that the current frame includes multiple short frames. If the global transient flag is a second value, then the frame type of the current frame is determined to be a second type, indicating that the current frame includes one long frame. The length of an ultra-short frame is less than the length of a short frame, and the length of a short frame is less than the length of a long frame. The second preset condition can be that the distance from the transient occurrence position indicated by the global transient location information to the start position of the current frame is less than the length of an ultra-short frame, or the distance from the transient occurrence position indicated by the global transient location information to the end position of the current frame is less than the length of an ultra-short frame.

[0160] The third scenario involves global transient detection results including global transient location information. The process of determining the frame type of the current frame based on the global transient detection results includes: if the global transient location information is a preset value, such as -1, then the frame type of the current frame is determined to be the second type, indicating that the current frame includes a long frame. If the global transient location information is not a preset value but meets the second preset condition, then the frame type of the current frame is determined to be the third type, indicating that the current frame includes multiple ultra-short frames. If the global transient location information is not a preset value and does not meet the second preset condition, then the frame type of the current frame is determined to be the first type, indicating that the current frame includes multiple short frames. The length of an ultra-short frame is less than the length of a short frame, and the length of a short frame is less than the length of a long frame. The second preset condition can be that the distance from the transient occurrence position indicated by the global transient location information to the start position of the current frame is less than the length of an ultra-short frame, or the distance from the transient occurrence position indicated by the global transient location information to the end position of the current frame is less than the length of an ultra-short frame.

[0161] It should be noted that the window function type of the current frame indicates the shape and length of the window function corresponding to the current frame, and the window function of the current frame is used to perform windowing processing on the temporal three-dimensional audio signal of the current frame. The frame type of the current frame indicates whether the current frame is an ultra-short frame, a short frame, or a long frame. Among them, ultra-short frames, short frames, and long frames can be distinguished based on the frame duration, and the specific duration can be set according to different needs. This application embodiment does not limit this.

[0162] The method to convert the time-domain three-dimensional audio signal of the current frame into the frequency-domain three-dimensional audio signal can be the modified discrete cosine transform (MDCT), the modified discrete sine transform (MDST), or the fast fourier transform (FFT).

[0163] Based on the above description, the target encoding parameters include the window function type and / or the frame type of the current frame. That is, the target encoding parameters include the window function type of the current frame, or the target encoding parameters include the frame type of the current frame, or the target encoding parameters include both the window function type and the frame type of the current frame. The process of converting the temporal-domain 3D audio signal of the current frame into a frequency-domain 3D audio signal based on these target encoding parameters differs depending on the parameters included in the target encoding parameters; therefore, they will be explained separately below.

[0164] In the first case, the target encoding parameters include the window function type of the current frame. In this case, the temporal 3D audio signal of the current frame is windowed based on the window function indicated by the window function type. Then, the windowed 3D audio signal is converted into a frequency domain 3D audio signal.

[0165] In the second case, the target encoding parameters include the frame type of the current frame. In this case, if the frame type is type one, indicating that the current frame comprises multiple short frames, the temporal 3D audio signals of each short frame are converted to frequency-domain 3D audio signals. If the frame type is type two, indicating that the current frame comprises one long frame, the temporal 3D audio signals of the long frame are directly converted to frequency-domain 3D audio signals. If the frame type is type three, indicating that the current frame comprises multiple ultra-short frames, the temporal 3D audio signals of each ultra-short frame are converted to frequency-domain 3D audio signals.

[0166] In the third case, the target encoding parameters include the window function type and frame type of the current frame. In this case, if the frame type is type one, indicating that the current frame includes multiple short frames, then windowing is applied to the temporal 3D audio signals of each short frame based on the window function indicated by the current frame's window function type, and the windowed temporal 3D audio signals of each short frame are then converted to frequency domain 3D audio signals. If the frame type is type two, indicating that the current frame includes one long-short frame, then windowing is applied to the temporal 3D audio signals of the long frame based on the window function indicated by the current frame's window function type, and the windowed temporal 3D audio signals of the long frame are then converted to frequency domain 3D audio signals. If the frame type is type three, indicating that the current frame includes multiple ultra-short frames, then windowing is applied to the temporal 3D audio signals of each ultra-short frame based on the window function indicated by the current frame's window function type, and the windowed temporal 3D audio signals of each ultra-short frame are then converted to frequency domain 3D audio signals.

[0167] In other words, when the current frame includes multiple ultra-short frames and short frames, converting the time-domain 3D audio signal of the current frame into a frequency-domain 3D audio signal yields the frequency-domain 3D audio signals of each of the ultra-short frames and short frames included in the current frame. When the current frame includes a single long frame, converting the time-domain 3D audio signal of the current frame into a frequency-domain 3D audio signal yields the frequency-domain 3D audio signal of the single long frame included in the current frame.

[0168] Step 604: Based on the global transient detection results, spatially encode the frequency domain three-dimensional audio signal of the current frame to obtain spatial encoding parameters and frequency domain signals of N transmission channels, where N is an integer greater than or equal to 1 and less than or equal to M.

[0169] In some embodiments, based on the frame type of the current frame, the frequency domain three-dimensional audio signal of the current frame is spatially encoded to obtain spatial encoding parameters and frequency domain signals of N transmission channels.

[0170] When spatially encoding the frequency-domain 3D audio signal of the current frame based on its frame type, if the current frame type is type 1 (i.e., the current frame includes multiple short frames), the frequency-domain 3D audio signals of the multiple short frames are interleaved to obtain a long frame's frequency-domain 3D audio signal, and then spatially encoded. If the current frame type is type 2 (i.e., the current frame includes a single long frame), the frequency-domain 3D audio signal of this long frame is spatially encoded. If the current frame type is type 3 (i.e., the current frame includes multiple ultra-short frames), the frequency-domain 3D audio signals of the multiple ultra-short frames are interleaved to obtain a long frame's frequency-domain 3D audio signal, and then spatially encoded.

[0171] The spatial coding method can be any method that can obtain spatial coding parameters and frequency domain signals of N transmission channels based on the frequency domain three-dimensional audio signal of the current frame. For example, the spatial coding method of matching projection can be adopted. The embodiments of this application do not limit the spatial coding method.

[0172] Spatial coding parameters refer to the parameters determined during the spatial coding of the frequency domain three-dimensional audio signal of the current frame, including side information, bit pre-allocation side information, etc. The frequency domain signals of N transmission channels can include virtual speaker signals of one or more channels, as well as residual signals of one or more channels. In addition, when the number of coding bits is insufficient, the frequency domain signals of N transmission channels can also include only virtual speaker signals of one or more channels.

[0173] Step 605: Based on the global transient detection results, encode the frequency domain signals of the N transmission channels to obtain the frequency domain signal encoding results.

[0174] In some embodiments, the frequency domain signals of the N transmission channels are encoded based on the frame type of the current frame.

[0175] As an example, the implementation process of encoding the frequency domain signals of the N transmission channels includes: performing noise shaping processing on the frequency domain signals of the N transmission channels based on the frame type of the current frame; performing transmission channel downmixing processing on the noise-shaped frequency domain signals of the N transmission channels to obtain the downmixed signals; performing quantization and encoding processing on the low-frequency components of the downmixed signals and writing the encoding results into the bitstream; and performing bandwidth expansion and encoding processing on the high-frequency components of the downmixed signals and writing the encoding results into the bitstream.

[0176] It should be noted that the method of noise shaping based on the frame type of the current frame can refer to relevant technologies, and will not be described in detail in the embodiments of this application. The noise shaping process includes temporal noise shaping (TNS) and frequency domain noise shaping (FDNS).

[0177] When performing downmixing on the frequency domain signals of the N transmission channels after noise shaping, the N transmission channels can be paired according to pre-set criteria, or the frequency domain signals of the N transmission channels can be paired according to signal correlation. Then, mid-side (MS) downmixing is performed based on the paired two frequency domain signals.

[0178] For example, if the N transmission channels include 2 virtual speaker signals and 4 residual signals, the 2 virtual speaker signals can be paired according to a pre-set criterion and then downmixed. Alternatively, the correlation between any two residual signals in the 4 residual signals can be determined, and the two residual signals with high correlation can be paired together, while the remaining two residual signals can be paired separately and downmixed.

[0179] It should be noted that the downmixing of the two paired frequency domain signals may result in either one or two frequency domain signals, depending on the encoding process.

[0180] The low-frequency and high-frequency components of a signal can be divided in several ways. For example, using 2000Hz as the dividing point, the portion of the down-mixed signal with frequencies below 2000Hz is considered the low-frequency component, and the portion with frequencies above 2000Hz is considered the high-frequency component. Alternatively, using 5000Hz as the dividing point, the portion of the down-mixed signal with frequencies below 5000Hz is considered the low-frequency component, and the portion with frequencies above 5000Hz is considered the high-frequency component.

[0181] Step 606: Encode the spatial coding parameters to obtain the spatial coding parameter encoding result, and write the spatial coding parameter encoding result and the frequency domain signal encoding result into the bit stream.

[0182] Optionally, the global transient detection result can be encoded to obtain an encoded global transient detection result, and then written into the bitstream. Alternatively, the target encoding parameters can be encoded to obtain an encoded target encoding parameter result, and then written into the bitstream.

[0183] In this embodiment, transient detection can first be performed on the signals of the M channels of the time-domain 3D audio signal of the current frame to determine the global transient detection result. Then, based on the global transient detection result, time-frequency transformation, spatial coding, and frequency domain coding of the audio signal are performed sequentially. In particular, when coding the frequency domain signals of each transmission channel obtained after spatial coding, the transient detection result of each transmission channel reuses the global transient detection result. It is not necessary to convert the frequency domain signals of each transmission channel to the time domain to determine the corresponding transient detection result for each transmission channel. Therefore, it is not necessary to perform multiple transformations of the 3D audio signal between the time and frequency domains, thereby reducing coding complexity and improving coding efficiency. Furthermore, this embodiment does not require encoding the transient detection results of each transmission channel; only the global transient detection result needs to be encoded into the bitstream, thus reducing the number of coding bits.

[0184] Please refer to Figure 7 and Figure 8 , Figure 7 and Figure 8 These are all block diagrams of an exemplary encoding method provided in the embodiments of this application. Figure 7 and Figure 8 Mainly for Figure 6 The encoding method shown is explained exemplarily. Figure 7 In this process, transient detection is performed on the signals of M channels in the temporal domain three-dimensional audio signal of the current frame to obtain M transient detection results corresponding to the M channels. Based on these M transient detection results, a global transient detection result is determined and encoded to obtain a global transient detection result encoding result, which is then written into the bitstream. Based on the global transient detection result, the temporal domain three-dimensional audio signal of the current frame is converted into a frequency domain three-dimensional audio signal. Based on the global transient detection result, spatial encoding is performed on the frequency domain three-dimensional audio signal of the current frame to obtain spatial encoding parameters and frequency domain signals of N transmission channels. The spatial encoding parameters are then encoded to obtain a spatial encoding parameter encoding result, which, along with the frequency domain signal encoding result, is written into the bitstream. Based on the global transient detection result, the frequency domain signals of the N transmission channels are encoded. Further, in Figure 8 In the process, spatial encoding is performed on the frequency domain 3D audio signal of the current frame to obtain spatial encoding parameters and frequency domain signals of N transmission channels. The spatial encoding parameters are then encoded to obtain the spatial encoding parameter encoding result, and the spatial encoding parameter encoding result and the frequency domain signal encoding result are written into the bitstream. Then, based on the global transient detection results, noise shaping, transmission channel downmixing, quantization and encoding, and bandwidth expansion processing are performed on the frequency domain signals of the N transmission channels. The encoded result of the bandwidth-expanded signal is then written into the bitstream.

[0185] Based on the description in step 606 above, the encoding device may or may not encode the global transient detection result into the bitstream. Furthermore, the encoding device may or may not encode the target encoding parameters into the bitstream. If the encoding device encodes the global transient detection result into the bitstream, the decoding device can proceed as follows: Figure 9 Decoding is performed using the method shown. When the encoding device encodes the target encoding parameters into the bitstream, the decoding device can parse the target encoding parameters from the bitstream and then perform decoding based on the frame type of the current frame included in the target encoding parameters. The specific implementation process is as follows... Figure 9 The process is similar. Of course, the encoding device may not encode the global transient detection results into the bitstream, nor may it encode the target encoding parameters into the bitstream. In this case, the decoding process of the three-dimensional audio signal can refer to relevant technologies, which will not be described in the embodiments of this application.

[0186] Please refer to Figure 9 , Figure 9 This is a flowchart of the first decoding method provided in the embodiments of this application. The method is applied to the decoding end and includes the following steps.

[0187] Step 901: Parse the global transient detection results and spatial coding parameters from the bitstream.

[0188] Step 902: Decode the data based on the global transient detection results and the bitstream to obtain the frequency domain signals of N transmission channels.

[0189] In some embodiments, the frame type of the current frame is determined based on the global transient detection results. Decoding is performed based on the frame type of the current frame and the bitstream to obtain the frequency domain signals of the N transmission channels.

[0190] The implementation method for determining the frame type of the current frame based on the global transient detection result can refer to the relevant description in step 603 above, and will not be repeated here. The implementation method for decoding based on the frame type of the current frame and the bitstream can refer to relevant technologies, and will not be elaborated in detail in the embodiments of this application.

[0191] Step 903: Based on the global transient detection results and spatial coding parameters, perform spatial decoding on the frequency domain signals of the N transmission channels to obtain the reconstructed frequency domain three-dimensional audio signal.

[0192] In some embodiments, based on the frame type and spatial coding parameters of the current frame, spatial decoding is performed on the frequency domain signals of the N transmission channels to obtain the reconstructed frequency domain three-dimensional audio signal. The frame type of the current frame is determined based on the global transient detection results. That is, the frame type of the current frame is determined based on the global transient detection results, and then, based on the frame type and spatial coding parameters of the current frame, spatial decoding is performed on the frequency domain signals of the N transmission channels to obtain the reconstructed frequency domain three-dimensional audio signal.

[0193] The process of spatially decoding the frequency domain signals of the N transmission channels based on the frame type and spatial coding parameters of the current frame can be referred to relevant technologies, and will not be described in detail in the embodiments of this application.

[0194] Step 904: Based on the global transient detection results and the reconstructed frequency domain three-dimensional audio signal, determine the reconstructed time domain three-dimensional audio signal.

[0195] In some embodiments, target coding parameters are determined based on global transient detection results. These target coding parameters include the window function type and / or the frame type of the current frame. Based on these target coding parameters, the reconstructed frequency-domain 3D audio signal is converted into a reconstructed time-domain 3D audio signal.

[0196] The implementation method for determining the target encoding parameters based on the global transient detection results can be referred to the relevant description in step 603 above, and will not be repeated here.

[0197] Based on the above description, the target coding parameters include the window function type and / or the frame type of the current frame. That is, the target coding parameters include the window function type of the current frame, or the target coding parameters include the frame type of the current frame, or the target coding parameters include both the window function type and the frame type of the current frame. The process of converting the reconstructed frequency-domain 3D audio signal into a reconstructed temporal-domain 3D audio signal based on these target coding parameters differs depending on the parameters included in the target coding parameters; therefore, they will be explained separately below.

[0198] In the first case, the target encoding parameters include the window function type of the current frame. In this case, the reconstructed frequency-domain 3D audio signal is de-windowed based on the window function indicated by the window function type of the current frame. Then, the de-windowed frequency-domain 3D audio signal is converted into a reconstructed time-domain 3D audio signal.

[0199] Among them, the window removal process is also called the window addition and overlay addition process.

[0200] In the second scenario, the target encoding parameters include the frame type of the current frame. In this case, if the frame type is type one, indicating that the current frame comprises multiple short frames, the reconstructed frequency-domain 3D audio signals of each short frame are converted into time-domain 3D audio signals to obtain the reconstructed time-domain 3D audio signal. If the frame type is type two, indicating that the current frame comprises one long frame, the reconstructed frequency-domain 3D audio signals of the long frame included in the current frame are directly converted into time-domain 3D audio signals to obtain the reconstructed time-domain 3D audio signal. If the frame type is type three, indicating that the current frame comprises multiple ultra-short frames, the reconstructed frequency-domain 3D audio signals of each ultra-short frame are converted into time-domain 3D audio signals to obtain the reconstructed time-domain 3D audio signal.

[0201] In the third case, the target encoding parameters include the window function type and frame type of the current frame. In this case, if the frame type of the current frame is type one, indicating that the current frame includes multiple short frames, then based on the window function indicated by the window function type of the current frame, the frequency domain 3D audio signals of each short frame included in the current frame are de-windowed, and the reconstructed frequency domain 3D audio signals of each short frame after de-windowing are converted into time domain 3D audio signals to obtain the reconstructed time domain 3D audio signal. If the frame type of the current frame is type two, indicating that the current frame includes one long and one short frame, then based on the window function indicated by the window function type of the current frame, the reconstructed frequency domain 3D audio signal of the long frame included in the current frame is de-windowed, and the frequency domain 3D audio signal of the long frame after de-windowing is converted into time domain 3D audio signals to obtain the reconstructed time domain 3D audio signal. If the current frame type is type 3, it indicates that the current frame includes multiple ultra-short frames. In this case, based on the window function indicated by the window function type of the current frame, the frequency domain 3D audio signals of each ultra-short frame included in the current frame are dewindowed, and the reconstructed frequency domain 3D audio signals of each ultra-short frame after dewindowing are converted into time domain 3D audio signals to obtain the reconstructed time domain 3D audio signal.

[0202] In this embodiment, the decoding end parses the global transient detection results and spatial coding parameters from the bitstream. This allows for the reconstruction of the temporal three-dimensional audio signal based on the global transient detection results and spatial coding parameters, without needing to parse the transient detection results of each transmission channel from the bitstream. This reduces decoding complexity and improves decoding efficiency. Furthermore, without incorporating the target coding parameters into the bitstream, the target coding parameters can be directly determined based on the global transient detection results, thereby achieving the reconstruction of the temporal three-dimensional audio signal.

[0203] Please refer to Figure 10 , Figure 10 This is a block diagram of an exemplary decoding method provided in an embodiment of this application. Figure 10 Mainly for Figure 9 The decoding method shown is explained exemplarily. Figure 10 In this process, global transient detection results and spatial coding parameters are parsed from the bitstream. Decoding is then performed based on the global transient detection results and the bitstream to obtain the frequency domain signals of N transmission channels. Spatial decoding is then performed on the frequency domain signals of these N transmission channels based on the global transient detection results and spatial coding parameters to obtain the reconstructed frequency domain 3D audio signal. Finally, based on the global transient detection results and the reconstructed frequency domain 3D audio signal, dewindowing processing and inverse time-frequency transform are used to determine the reconstructed time domain 3D audio signal.

[0204] Please refer to Figure 11 , Figure 11 This is a flowchart of a second encoding method provided in an embodiment of this application. The encoding method is applied to an encoding device and includes the following steps.

[0205] Step 1101: Perform transient detection on the M channels of the temporal three-dimensional audio signal of the current frame to obtain M transient detection results corresponding to the M channels, where M is an integer greater than 1.

[0206] The method for determining the M transient detection results corresponding to the M channels can be found in the relevant description in step 601, and will not be repeated here.

[0207] Step 1102: Based on the M transient detection results, determine the global transient detection result.

[0208] The method for determining the global transient location information based on the M transient detection results can be found in the relevant description in step 602, and will not be repeated here.

[0209] Step 1103: Based on the global transient detection results, convert the time-domain three-dimensional audio signal of the current frame into a frequency-domain three-dimensional audio signal.

[0210] The method of converting the time-domain three-dimensional audio signal of the current frame into a frequency-domain three-dimensional audio signal based on the global transient detection results can be referred to the relevant description in step 603, and will not be repeated here.

[0211] Step 1104: Based on the global transient detection results, spatially encode the frequency domain three-dimensional audio signal of the current frame to obtain spatial coding parameters and frequency domain signals of N transmission channels, where N is an integer greater than or equal to 1 and less than or equal to M.

[0212] The implementation method of spatial encoding the frequency domain three-dimensional audio signal of the current frame based on the global transient detection results can be referred to the relevant description in step 604, which will not be repeated here.

[0213] Step 1105: Based on the M transient detection results, determine the N transient detection results corresponding to the N transmission channels.

[0214] In some embodiments, based on the M transient flags, transient flags of the virtual speaker signals comprising one or more channels in the N transmission channels are determined according to a first preset rule. Based on the M transient flags, transient flags of the residual signals comprising one or more channels in the N transmission channels are determined according to a second preset rule.

[0215] As an example, the first preset rule includes: if the number of transient flags with the first value among the M transient flags is greater than or equal to P, then the transient flags of the virtual speaker signals of the N transmission channels, including one or more channels, are all of the first value. The second preset rule includes: if the number of transient flags with the first value among the M transient flags is greater than or equal to Q, then the transient flags of the residual signals of the N transmission channels, including one or more channels, are all of the first value.

[0216] Where P and Q are both positive integers less than M. P and Q are preset values, and can also be adjusted according to different needs. Optionally, since the virtual speaker signal is used to record the real three-dimensional audio signal, it is more important than the residual signal, so P is less than Q.

[0217] As another example, the first preset rule includes: if the number of transient flags with the first value among the M transient flags is greater than or equal to P, then the transient flags corresponding to the virtual speaker signals of the N transmission channels, including one or more channels, are all of the first value. The second preset rule includes: if the number of channels among the M channels that meet the first preset condition and whose corresponding transient flags are of the first value is greater than or equal to R, then the transient flags corresponding to the residual signals of the N transmission channels, including one or more channels, are all of the first value.

[0218] Where P and R are both positive integers less than M. P and R are preset values, and can also be adjusted according to different needs. When the three-dimensional audio signal is a HOA signal, the first preset condition includes channels belonging to the FOA signal. Among the M channels, the channel that satisfies the first preset condition is the channel where the FOA signal in the three-dimensional audio signal of the current frame is located. The FOA signal is the signal of the first 4 channels in the HOA signal. Of course, the first preset condition can also be other conditions.

[0219] In other embodiments, the N transient flags can be determined based on the M transient flags and according to the mapping relationship between the M transient flags and the N transmission channels. The mapping relationship is predetermined.

[0220] For example, if a certain transmission channel among the N transmission channels is mapped to one or more channels among the M channels, and if at least one of the M transient flags has a first value, then the transient flag of that transmission channel among the N transmission channels has the first value.

[0221] It should be noted that step 1105 can be executed at any time after step 1101 and before step 1106. This application embodiment does not limit the timing of execution of step 1105.

[0222] Step 1106: Based on the N transient detection results, encode the frequency domain signals of the N transmission channels to obtain the frequency domain signal encoding results.

[0223] In some embodiments, based on the N transient detection results, the frame type corresponding to each of the N transmission channels is determined. Based on the frame type corresponding to each of the N transmission channels, the frequency domain signal of the corresponding transmission channel in the N transmission channels is encoded.

[0224] Since the implementation method for the frame type corresponding to each of the N transmission channels is the same, the following explanation will use one of the transmission channels as an example. For ease of description, this transmission channel will be referred to as the target transmission channel.

[0225] The process of determining the frame type corresponding to the target transmission channel based on the transient detection result includes: if the transient flag corresponding to the target transmission channel is a first value, then the frame type corresponding to the target transmission channel is determined to be a first type, which indicates that the signal of the target transmission channel includes multiple short frames. If the transient flag corresponding to the target transmission channel is a second value, then the frame type corresponding to the target transmission channel is determined to be a second type, which indicates that the signal of the target transmission channel includes a long frame.

[0226] It should be noted that the frame type of the current frame is used to indicate whether the current frame is a short frame or a long frame. Short frames and long frames can be distinguished based on the frame duration, and the specific duration can be set according to different needs; this embodiment does not limit this.

[0227] Once the frame type corresponding to each transmission channel is determined, noise shaping can be performed on the frequency domain signal of each transmission channel based on the corresponding frame type. Then, the frequency domain signals of the N noise-shaped transmission channels are down-mixed to obtain the down-mixed signal. The low-frequency component of the down-mixed signal is quantized and encoded, and the encoding result is written into the bitstream. The high-frequency component of the down-mixed signal is bandwidth-extended and encoded, and the encoding result is written into the bitstream.

[0228] For details regarding noise shaping, transmission channel downmixing, quantization and coding of low-frequency components, and bandwidth expansion and coding, please refer to the relevant descriptions in step 605; they will not be repeated here.

[0229] Step 1107: Encode the spatial coding parameters and the N transient detection results to obtain the spatial coding parameter encoding result and the N transient detection result encoding result, and write the spatial coding parameter encoding result and the N transient detection result encoding result into the bitstream.

[0230] Optionally, the global transient detection result can be encoded to obtain an encoded global transient detection result, and then written into the bitstream. Alternatively, the target encoding parameters can be encoded to obtain an encoded target encoding parameter result, and then written into the bitstream.

[0231] In this embodiment, based on the M transient detection results corresponding to the M channels of the three-dimensional audio signal, the transient detection results corresponding to the virtual speaker signal and residual signal of each transmission channel are determined. This ensures improved encoding accuracy when encoding the frequency domain signals of each transmission channel. Moreover, since the transient detection results for each transmission channel are determined based on the M transient detection results, it is not necessary to convert the frequency domain signals of each transmission channel to the time domain to determine the transient detection results for each transmission channel. Consequently, it is not necessary to perform multiple transformations between the time and frequency domains of the three-dimensional audio signal, thereby reducing encoding complexity and improving encoding efficiency.

[0232] Please refer to Figure 12 and Figure 13 , Figure 12 and Figure 13 This is a block diagram of another exemplary encoding method provided in the embodiments of this application. Figure 12 and Figure 13 Mainly for Figure 11 The encoding method shown is explained exemplarily. Figure 12In this process, transient detection is performed on the signals of M channels in the temporal domain 3D audio signal of the current frame to obtain M transient detection results corresponding to the M channels. Based on the M transient detection results, a global transient detection result is determined and encoded to obtain a global transient detection result encoding result, which is then written into the bitstream. Based on the global transient detection result, the temporal domain 3D audio signal of the current frame is converted into a frequency domain 3D audio signal. Based on the global transient detection result, spatial encoding is performed on the frequency domain 3D audio signal of the current frame to obtain spatial encoding parameters and the frequency domain signals of the N transmission channels. The spatial encoding parameters are then encoded to obtain a spatial encoding parameter encoding result, which is then written into the bitstream. Based on the M transient detection results, N transient detection results corresponding to the N transmission channels are determined and encoded to obtain N transient detection result encoding results, which are then written into the bitstream. Based on the N transient detection results, the frequency domain signals of the N transmission channels are encoded. Furthermore, in Figure 13 After determining the N transient detection results, noise shaping is performed on the frequency domain signals of the N transmission channels based on these N transient detection results. Then, the frequency domain signals of each transmission channel after noise shaping are subjected to transmission channel downmixing, quantization and encoding, and bandwidth expansion. The encoded result of the bandwidth-expanded signal is then written into the bitstream.

[0233] Based on the description in step 1107 above, the encoding device may or may not encode the global transient detection result into the bitstream. Furthermore, the encoding device may or may not encode the target encoding parameters into the bitstream. If the encoding device encodes the global transient detection result into the bitstream, the decoding device can proceed as follows: Figure 14 Decoding is performed using the method shown. When the encoding device encodes the target encoding parameters into the bitstream, the decoding device can parse the target encoding parameters from the bitstream and then perform decoding based on the frame type of the current frame included in the target encoding parameters. The specific implementation process is as follows... Figure 14 The process is similar. Of course, the encoding device may not encode the global transient detection results into the bitstream, nor may it encode the target encoding parameters into the bitstream. In this case, the decoding process of the three-dimensional audio signal can refer to relevant technologies, which will not be described in the embodiments of this application.

[0234] Please refer to Figure 14 , Figure 14 This is a flowchart of a second decoding method provided in an embodiment of this application. The method is applied to the decoding end and includes the following steps.

[0235] Step 1401: Parse the global transient detection result, the N transient detection results corresponding to the N transmission channels, and the spatial coding parameters from the bitstream.

[0236] Step 1402: Decode the N transient detection results and the bitstream to obtain the frequency domain signals of the N transmission channels.

[0237] In some embodiments, the frame type corresponding to each transmission channel is determined based on N transient detection results. Decoding is performed based on the frame type corresponding to each transmission channel and the bitstream to obtain the frequency domain signals of the N transmission channels.

[0238] The implementation method for determining the frame type corresponding to each transmission channel based on N transient detection results can refer to the relevant description in step 1106 above, and will not be repeated here. The implementation method for decoding based on the frame type corresponding to each transmission channel and the bitstream can refer to relevant technologies, and will not be elaborated in detail in the embodiments of this application.

[0239] Step 1403: Based on the frequency domain signals and spatial coding parameters of the N transmission channels, perform spatial decoding on the frequency domain signals of the N transmission channels to obtain the reconstructed frequency domain three-dimensional audio signal.

[0240] In some embodiments, the frame type corresponding to each transmission channel is determined based on N transient detection results. Based on the frame type and spatial coding parameters corresponding to each transmission channel, the frequency domain signals of the N transmission channels are spatially decoded to obtain the reconstructed frequency domain three-dimensional audio signal.

[0241] The process of spatial decoding of the frequency domain signals of the N transmission channels based on the frame type and spatial coding parameters corresponding to each transmission channel can be referred to relevant technologies, and will not be described in detail in the embodiments of this application.

[0242] Step 1404: Based on the global transient detection results and the reconstructed frequency domain three-dimensional audio signal, determine the reconstructed time domain three-dimensional audio signal.

[0243] The method for determining the reconstructed time-domain three-dimensional audio signal based on the global transient detection results and the reconstructed frequency-domain three-dimensional audio signal can be found in the relevant description in step 904, and will not be repeated here.

[0244] In this embodiment, the decoding end parses the global transient detection results, the transient detection results corresponding to each transmission channel, and the spatial coding parameters from the bitstream. Thus, when decoding based on the transient detection results corresponding to each transmission channel, the frequency domain signal of each transmission channel can be accurately obtained. Furthermore, without encoding the target coding parameters into the bitstream, the target coding parameters can be directly determined based on the global transient detection results, thereby achieving the reconstruction of the temporal three-dimensional audio signal.

[0245] Please refer to Figure 15 , Figure 15 This is a block diagram of another exemplary decoding method provided in the embodiments of this application. Figure 15 Mainly for Figure 14 The decoding method shown is explained exemplarily. Figure 15 In this process, global transient detection results, N transient detection results corresponding to N transmission channels, and spatial coding parameters are parsed from the bitstream. Based on these N transient detection results and the bitstream, decoding is performed to obtain the frequency domain signals of the N transmission channels. Based on the frequency domain signals of the N transmission channels and the spatial coding parameters, spatial decoding is performed on the frequency domain signals of the N transmission channels to obtain the reconstructed frequency domain three-dimensional audio signal. Based on the global transient detection results and the reconstructed frequency domain three-dimensional audio signal, the reconstructed time domain three-dimensional audio signal is determined.

[0246] Figure 16 This is a schematic diagram of an encoding device provided in an embodiment of this application. The encoding device can be implemented as part or all of an encoding end device by software, hardware, or a combination of both. The encoding end device can be... Figure 1 The source device is shown. See also Figure 16 The device includes: a transient detection module 1601, a determination module 1602, a conversion module 1603, a spatial encoding module 1604, a first encoding module 1605, a second encoding module 1606, and a first writing module 1607.

[0247] The transient detection module 1601 is used to perform transient detection on the M channels of the temporal three-dimensional audio signal of the current frame to obtain M transient detection results corresponding to the M channels, where M is an integer greater than 1. For detailed implementation process, please refer to the corresponding content in the above embodiments, which will not be repeated here.

[0248] The determination module 1602 is used to determine the global transient detection result based on M transient detection results. For detailed implementation details, please refer to the corresponding content in the above embodiments; they will not be repeated here.

[0249] The conversion module 1603 is used to convert the time-domain three-dimensional audio signal into a frequency-domain three-dimensional audio signal based on the global transient detection results. For detailed implementation details, please refer to the corresponding contents in the above embodiments; they will not be repeated here.

[0250] The spatial coding module 1604 is used to spatially code the frequency-domain three-dimensional audio signal based on the global transient detection results, so as to obtain spatial coding parameters and frequency-domain signals of N transmission channels, where N is an integer greater than or equal to 1 and less than or equal to M. For detailed implementation process, please refer to the corresponding content in the above embodiments, which will not be repeated here.

[0251] The first encoding module 1605 is used to encode the frequency domain signals of N transmission channels based on the global transient detection results to obtain the frequency domain signal encoding results. For detailed implementation processes, please refer to the corresponding contents in the above embodiments, which will not be repeated here.

[0252] The second encoding module 1606 is used to encode the spatial encoding parameters to obtain the spatial encoding parameter encoding result. For detailed implementation process, please refer to the corresponding content in the above embodiments, which will not be repeated here.

[0253] The first writing module 1607 is used to write the spatial coding parameter encoding result and the frequency domain signal encoding result into the bitstream. For detailed implementation process, please refer to the corresponding content in the above embodiments, which will not be repeated here.

[0254] Optionally, the conversion module 1603 includes:

[0255] The determination unit is used to determine the target coding parameters based on the global transient detection results. The target coding parameters include the window function type of the current frame and / or the frame type of the current frame.

[0256] The conversion unit is used to convert time-domain three-dimensional audio signals into frequency-domain three-dimensional audio signals based on target coding parameters.

[0257] Optionally, the global transient detection result includes a global transient flag, and the target encoding parameters include the window function type of the current frame;

[0258] The specific unit is used for:

[0259] If the global transient flag is the first value, then the type of the first preset window function is determined as the window function type of the current frame;

[0260] If the global transient flag is the second value, then the type of the second preset window function is determined as the window function type of the current frame;

[0261] The window length of the first preset window function is less than the window length of the second preset window function.

[0262] Optionally, the global transient detection result includes a global transient flag and global transient location information, and the target encoding parameters include the window function type of the current frame;

[0263] The specific unit is used for:

[0264] If the global transient flag is set to the first value, the window function type of the current frame is determined based on the global transient position information.

[0265] Optionally, the device further includes:

[0266] The third encoding module is used to encode the target encoding parameters to obtain the encoding result of the target encoding parameters;

[0267] The second writing module is used to write the encoding result of the target encoding parameters into the bitstream.

[0268] Optionally, the spatial coding module 1604 is specifically used for:

[0269] Spatial encoding of frequency domain three-dimensional audio signals is performed based on frame type.

[0270] Optionally, the first encoding module 1605 is specifically used for:

[0271] Encode the frequency domain signals of N transmission channels based on the frame type of the current frame.

[0272] Optionally, the transient detection result includes a transient flag, and the global transient detection result includes a global transient flag. The transient flag is used to indicate whether the signal of the corresponding channel is a transient signal.

[0273] Module 1602 is specifically used for:

[0274] If the number of transient flags with the first value among the M transient flags is greater than or equal to m, then the global transient flag is determined to be the first value, where m is a positive integer greater than 0 and less than M; or

[0275] If the number of channels among the M channels that meet the first preset condition and whose corresponding transient flag is the first value is greater than or equal to n, then the global transient flag is determined to be the first value, where n is a positive integer greater than 0 and less than M.

[0276] Optionally, the transient detection result also includes transient location information, and the global transient detection result also includes global transient location information, which is used to indicate the location where the transient occurs in the signal of the corresponding channel;

[0277] Module 1602 is specifically used for:

[0278] If only one of the M transient flags has the first value, then the transient position information corresponding to the channel with the first value is determined as the global transient position information.

[0279] If at least two of the M transient flags have the first value, then the transient position information corresponding to the channel with the largest transient detection parameter among the at least two channels corresponding to the at least two transient flags is determined as the global transient position information.

[0280] Optionally, the device further includes:

[0281] The fourth encoding module is used to encode the global transient detection results to obtain the encoded global transient detection results;

[0282] The third writing module is used to write the encoded result of the global transient detection result into the bitstream.

[0283] In this embodiment, transient detection can first be performed on the signals of the M channels of the time-domain 3D audio signal of the current frame to determine the global transient detection result. Then, based on the global transient detection result, time-frequency transformation, spatial coding, and frequency domain coding of the audio signal are performed sequentially. In particular, when coding the frequency domain signals of each transmission channel obtained after spatial coding, the transient detection result of each transmission channel reuses the global transient detection result. It is not necessary to convert the frequency domain signals of each transmission channel to the time domain to determine the corresponding transient detection result for each transmission channel. Therefore, it is not necessary to perform multiple transformations of the 3D audio signal between the time and frequency domains, thereby reducing coding complexity and improving coding efficiency. Furthermore, this embodiment does not require encoding the transient detection results of each transmission channel; only the global transient detection result needs to be encoded into the bitstream, thus reducing the number of coding bits.

[0284] It should be noted that the encoding device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the encoding device and encoding method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0285] Figure 17 This is a schematic diagram of a decoding device provided in an embodiment of this application. The decoding device can be implemented as part or all of a decoding end device by software, hardware, or a combination of both. The decoding end device can be... Figure 1 The destination device is shown. See also Figure 17 The device includes: a parsing module 1701, a decoding module 1702, a spatial decoding module 1703, and a determination module 1704.

[0286] The parsing module 1701 is used to parse the global transient detection results and spatial coding parameters from the bitstream. For detailed implementation details, please refer to the corresponding content in the above embodiments; they will not be repeated here.

[0287] Decoding module 1702 is used to decode based on the global transient detection results and the bitstream to obtain the frequency domain signals of N transmission channels. Detailed implementation processes are described in the corresponding contents of the above embodiments and will not be repeated here.

[0288] The spatial decoding module 1703 is used to spatially decode the frequency domain signals of N transmission channels based on the global transient detection results and spatial coding parameters to obtain the reconstructed frequency domain three-dimensional audio signal. Detailed implementation processes are described in the corresponding contents of the above embodiments and will not be repeated here.

[0289] The determination module 1704 is used to determine the reconstructed time-domain three-dimensional audio signal based on the global transient detection results and the reconstructed frequency-domain three-dimensional audio signal. Detailed implementation processes are described in the corresponding contents of the above embodiments and will not be repeated here.

[0290] Optionally, the determining module 1704 includes:

[0291] The determination unit is used to determine the target coding parameters based on the global transient detection results. The target coding parameters include the window function type of the current frame and / or the frame type of the current frame.

[0292] The conversion unit is used to convert the reconstructed frequency domain three-dimensional audio signal into a reconstructed time domain three-dimensional audio signal based on the target coding parameters.

[0293] Optionally, the global transient detection result includes a global transient flag, and the target encoding parameters include the window function type of the current frame;

[0294] The specific unit is used for:

[0295] If the global transient flag is the first value, then the type of the first preset window function is determined as the window function type of the current frame;

[0296] If the global transient flag is the second value, then the type of the second preset window function is determined as the window function type of the current frame;

[0297] The window length of the first preset window function is less than the window length of the second preset window function.

[0298] Optionally, the global transient detection result includes a global transient flag and global transient location information, and the target encoding parameters include the window function type of the current frame;

[0299] The specific unit is used for:

[0300] If the global transient flag is set to the first value, the window function type of the current frame is determined based on the global transient position information.

[0301] In this embodiment, the decoding end parses the global transient detection results and spatial coding parameters from the bitstream. This allows for the reconstruction of the temporal three-dimensional audio signal based on the global transient detection results and spatial coding parameters, without needing to parse the transient detection results of each transmission channel from the bitstream. This reduces decoding complexity and improves decoding efficiency. Furthermore, without incorporating the target coding parameters into the bitstream, the target coding parameters can be directly determined based on the global transient detection results, thereby achieving the reconstruction of the temporal three-dimensional audio signal.

[0302] It should be noted that the decoding device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the decoding device and decoding method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0303] Figure 18 This is a schematic block diagram of an encoding / decoding apparatus 1800 used in an embodiment of this application. The encoding / decoding apparatus 1800 may include a processor 1801, a memory 1802, and a bus system 1803. The processor 1801 and the memory 1802 are connected via the bus system 1803. The memory 1802 stores instructions, and the processor 1801 executes the instructions stored in the memory 1802 to perform various encoding or decoding methods described in the embodiments of this application. To avoid repetition, detailed descriptions are omitted here.

[0304] In this embodiment, the processor 1801 may be a central processing unit (CPU), or it may be other general-purpose processors, DSPs, ASICs, FPGAs, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0305] The memory 1802 may include a ROM device or a RAM device. Any other suitable type of storage device may also be used as memory 1802. Memory 1802 may include code and data 18021 accessed by processor 1801 using bus 1803. Memory 1802 may further include an operating system 18023 and an application program 18022, which includes at least one program that allows processor 1801 to execute the encoding or decoding methods described in the embodiments of this application. For example, application program 18022 may include applications 1 to N, which further include encoding or decoding applications (hereinafter referred to as encoding / decoding applications) that execute the encoding or decoding methods described in the embodiments of this application.

[0306] In addition to the data bus, the bus system 1803 may also include a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus system 1803 in the diagram.

[0307] Optionally, the codec device 1800 may also include one or more output devices, such as a display 1804. In one example, the display 1804 may be a haptic display that combines a display with a haptic unit capable of operatively sensing touch input. The display 1804 may be connected to the processor 1801 via a bus 1803.

[0308] It should be noted that the encoding / decoding device 1800 can execute the encoding method in the embodiments of this application, and can also execute the decoding method in the embodiments of this application.

[0309] Those skilled in the art will appreciate that the functionality described in conjunction with the various illustrative logic blocks, modules, and algorithmic steps disclosed herein can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality described by the various illustrative logic blocks, modules, and steps can be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may comprise a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium, or a communication medium that includes any medium facilitating the transfer of a computer program from one place to another (e.g., based on a communication protocol). In this way, the computer-readable medium may substantially correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium, such as a signal or carrier wave. The data storage medium may be any available medium accessible by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this application. A computer program product may comprise a computer-readable medium.

[0310] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other media that can be used to store desired program code in the form of instructions or data structures and is accessible by a computer. Furthermore, any connection is properly referred to as computer-readable media. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. However, it should be understood that the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other temporary media, but are specifically directed to non-temporary tangible storage media. As used herein, disks and optical discs include compact optical discs (CDs), laser optical discs, optical discs, DVDs, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. The combination of the above items should also be included in the scope of computer-readable media.

[0311] Instructions can be executed by one or more processors, such as digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein can refer to any of the foregoing structures or any other structures suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described in the various illustrative logic blocks, modules, and steps described herein can be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Moreover, the techniques can be fully implemented in one or more circuit or logic elements. In one example, the various illustrative logic blocks, units, and modules in encoder 100 and decoder 200 can be understood as corresponding circuit devices or logic elements.

[0312] The techniques described in this application can be implemented in a wide variety of devices or apparatuses, including wireless handheld devices, integrated circuits (ICs), or a set of ICs (e.g., chipsets). The various components, modules, or units described in this application are intended to emphasize functional aspects of the apparatus for performing the disclosed techniques, but do not necessarily need to be implemented by different hardware units. In fact, as described above, the various units can be combined with suitable software and / or firmware within a codec hardware unit, or provided via interoperable hardware units (including one or more processors as described above).

[0313] In other words, the above embodiments can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital versatile disc (DVD)), or a semiconductor medium (e.g., solid state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiments of this application can be a non-volatile storage medium; in other words, it can be a non-transient storage medium.

[0314] It should be understood that "multiple" as mentioned herein refers to two or more. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In addition, to facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and the terms "first," "second," etc., do not necessarily imply that they are different.

[0315] The above descriptions are embodiments provided in this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An encoding method, characterized in that, The method includes: Transient detection is performed on the signals of the M channels of the temporal three-dimensional audio signal of the current frame to obtain M transient detection results corresponding to the M channels, where M is an integer greater than 1; Based on the M transient detection results, a global transient detection result is determined. The transient detection result includes a transient flag, and the global transient detection result includes a global transient flag. The transient flag is used to indicate whether the signal of the corresponding channel is a transient signal. Based on the global transient detection results, the time-domain three-dimensional audio signal is converted into a frequency-domain three-dimensional audio signal; Based on the global transient detection results, the frequency domain three-dimensional audio signal is spatially encoded to obtain spatial encoding parameters and frequency domain signals of N transmission channels, where N is an integer greater than or equal to 1 and less than or equal to M; Based on the global transient detection results, the frequency domain signals of the N transmission channels are encoded to obtain the frequency domain signal encoding results; The spatial coding parameters are encoded to obtain the spatial coding parameter encoding result; The spatial coding parameter encoding result and the frequency domain signal encoding result are written into the code stream.

2. The method as described in claim 1, characterized in that, The step of converting the time-domain three-dimensional audio signal into a frequency-domain three-dimensional audio signal based on the global transient detection result includes: The target coding parameters are determined based on the global transient detection results. The target coding parameters include the window function type of the current frame and / or the frame type of the current frame. The time-domain three-dimensional audio signal is converted into the frequency-domain three-dimensional audio signal based on the target encoding parameters.

3. The method as described in claim 2, characterized in that, The target encoding parameters include the window function type of the current frame; The determination of target encoding parameters based on the global transient detection results includes: If the global transient flag is a first value, then the type of the first preset window function is determined as the window function type of the current frame; If the global transient flag is the second value, then the type of the second preset window function is determined as the window function type of the current frame; Wherein, the window length of the first preset window function is less than the window length of the second preset window function.

4. The method as described in claim 2, characterized in that, The global transient detection result also includes global transient location information, and the target encoding parameters include the window function type of the current frame; The determination of target encoding parameters based on the global transient detection results includes: If the global transient flag is a first value, then the window function type of the current frame is determined based on the global transient position information.

5. The method according to any one of claims 2-4, characterized in that, The method further includes: The target encoding parameters are encoded to obtain the target encoding parameter encoding result; The encoding result of the target encoding parameters is written into the bitstream.

6. The method according to any one of claims 2-4, characterized in that, The step of spatially encoding the frequency domain three-dimensional audio signal based on the global transient detection results includes: Based on the frame type, the frequency domain three-dimensional audio signal is spatially encoded.

7. The method according to any one of claims 2-4, characterized in that, The encoding of the frequency domain signals of the N transmission channels based on the global transient detection results includes: Based on the frame type of the current frame, the frequency domain signals of the N transmission channels are encoded.

8. The method according to any one of claims 1-4, characterized in that, The determination of the global transient detection result based on the M transient detection results includes: If the number of transient flags with the first value among the M transient flags is greater than or equal to m, then the global transient flag is determined to be the first value, where m is a positive integer greater than 0 and less than M; or If the number of channels among the M channels that meet the first preset condition and whose corresponding transient flag is the first value is greater than or equal to n, then the global transient flag is determined to be the first value, where n is a positive integer greater than 0 and less than M.

9. The method as described in claim 8, characterized in that, The transient detection result also includes transient location information, and the global transient detection result also includes global transient location information. The transient location information is used to indicate the location where the transient occurs in the signal of the corresponding channel. The determination of the global transient detection result based on the M transient detection results includes: If only one of the M transient flags has the first value, then the transient position information corresponding to the channel with the first value is determined as the global transient position information; If at least two of the M transient flags have a first value, then the transient position information corresponding to the channel with the largest transient detection parameter among the at least two channels corresponding to the at least two transient flags is determined as the global transient position information.

10. The method according to any one of claims 1-4, characterized in that, The method further includes: The global transient detection results are encoded to obtain the global transient detection result encoding result; The encoded result of the global transient detection is written into the bitstream.

11. A decoding method, characterized in that, The method includes: The global transient detection result and spatial coding parameters are parsed from the bitstream. The global transient detection result is determined based on the M transient detection results corresponding to the M channels of the time-domain three-dimensional audio signal. The transient detection result includes a transient flag, and the global transient detection result includes a global transient flag. The transient flag is used to indicate whether the signal of the corresponding channel is a transient signal. Decoding is performed based on the global transient detection results and the bitstream to obtain the frequency domain signals of N transmission channels; Based on the global transient detection results and the spatial coding parameters, the frequency domain signals of the N transmission channels are spatially decoded to obtain the reconstructed frequency domain three-dimensional audio signal; Based on the global transient detection results and the reconstructed frequency domain three-dimensional audio signal, the reconstructed time domain three-dimensional audio signal is determined.

12. The method as described in claim 11, characterized in that, The step of determining the reconstructed time-domain three-dimensional audio signal based on the global transient detection result and the reconstructed frequency-domain three-dimensional audio signal includes: The target coding parameters are determined based on the global transient detection results. The target coding parameters include the window function type of the current frame and / or the frame type of the current frame. Based on the target encoding parameters, the reconstructed frequency domain three-dimensional audio signal is converted into the reconstructed time domain three-dimensional audio signal.

13. The method as described in claim 12, characterized in that, The target encoding parameters include the window function type of the current frame; The determination of target encoding parameters based on the global transient detection results includes: If the global transient flag is a first value, then the type of the first preset window function is determined as the window function type of the current frame; If the global transient flag is the second value, then the type of the second preset window function is determined as the window function type of the current frame; Wherein, the window length of the first preset window function is less than the window length of the second preset window function.

14. The method as described in claim 12, characterized in that, The transient detection result also includes transient location information, and the global transient detection result also includes global transient location information. The transient location information is used to indicate the location where the transient occurs in the signal of the corresponding channel. The target encoding parameters include the window function type of the current frame. The determination of target encoding parameters based on the global transient detection results includes: If the global transient flag is a first value, then the window function type of the current frame is determined based on the global transient position information.

15. An encoding device, characterized in that, The device includes: The transient detection module is used to perform transient detection on the signals of the M channels of the temporal three-dimensional audio signal of the current frame, so as to obtain M transient detection results corresponding to the M channels, where M is an integer greater than 1; The determination module is used to determine a global transient detection result based on the M transient detection results. The transient detection result includes a transient flag, and the global transient detection result includes a global transient flag. The transient flag is used to indicate whether the signal of the corresponding channel is a transient signal. The conversion module is used to convert the time-domain three-dimensional audio signal into a frequency-domain three-dimensional audio signal based on the global transient detection result; The spatial coding module is used to spatially code the frequency domain three-dimensional audio signal based on the global transient detection result to obtain spatial coding parameters and frequency domain signals of N transmission channels, where N is an integer greater than or equal to 1 and less than or equal to M. The first encoding module is used to encode the frequency domain signals of the N transmission channels based on the global transient detection results to obtain the frequency domain signal encoding results; The second encoding module is used to encode the spatial encoding parameters to obtain the spatial encoding parameter encoding result; The first writing module is used to write the spatial coding parameter encoding result and the frequency domain signal encoding result into the code stream.

16. The apparatus as claimed in claim 15, characterized in that, The conversion module includes: A determining unit is configured to determine target encoding parameters based on the global transient detection results, wherein the target encoding parameters include the window function type of the current frame and / or the frame type of the current frame; A conversion unit is used to convert the time-domain three-dimensional audio signal into the frequency-domain three-dimensional audio signal based on the target encoding parameters.

17. The apparatus as claimed in claim 16, characterized in that, The target encoding parameters include the window function type of the current frame; The determining unit is specifically used for: If the global transient flag is a first value, then the type of the first preset window function is determined as the window function type of the current frame; If the global transient flag is the second value, then the type of the second preset window function is determined as the window function type of the current frame; Wherein, the window length of the first preset window function is less than the window length of the second preset window function.

18. The apparatus as claimed in claim 16, characterized in that, The global transient detection result also includes global transient location information, and the target encoding parameters include the window function type of the current frame; The determining unit is specifically used for: If the global transient flag is a first value, then the window function type of the current frame is determined based on the global transient position information.

19. The apparatus according to any one of claims 16-18, characterized in that, The device further includes: The third encoding module is used to encode the target encoding parameters to obtain the target encoding parameter encoding result; The second writing module is used to write the encoding result of the target encoding parameters into the bitstream.

20. The apparatus according to any one of claims 16-18, characterized in that, The spatial coding module is specifically used for: Based on the frame type of the current frame, the frequency domain three-dimensional audio signal is spatially encoded.

21. The apparatus according to any one of claims 16-18, characterized in that, The first encoding module is specifically used for: Based on the frame type of the current frame, the frequency domain signals of the N transmission channels are encoded.

22. The apparatus according to any one of claims 15-18, characterized in that, The determining module is specifically used for: If the number of transient flags with the first value among the M transient flags is greater than or equal to m, then the global transient flag is determined to be the first value, where m is a positive integer greater than 0 and less than M; or If the number of channels among the M channels that meet the first preset condition and whose corresponding transient flag is the first value is greater than or equal to n, then the global transient flag is determined to be the first value, where n is a positive integer greater than 0 and less than M.

23. The apparatus as claimed in claim 22, characterized in that, The transient detection result also includes transient location information, and the global transient detection result also includes global transient location information. The transient location information is used to indicate the location where the transient occurs in the signal of the corresponding channel. The determining module is specifically used for: If only one of the M transient flags has the first value, then the transient position information corresponding to the channel with the first value is determined as the global transient position information; If at least two of the M transient flags have a first value, then the transient position information corresponding to the channel with the largest transient detection parameter among the at least two channels corresponding to the at least two transient flags is determined as the global transient position information.

24. The apparatus as described in any one of claims 15-18, characterized in that, The device further includes: The fourth encoding module is used to encode the global transient detection result to obtain the global transient detection result encoding result; The third writing module is used to write the encoded result of the global transient detection result into the bitstream.

25. A decoding device, characterized in that, The device includes: The parsing module is used to parse the global transient detection result and spatial coding parameters from the bitstream. The global transient detection result is determined based on the M transient detection results corresponding to the M channels of the time-domain three-dimensional audio signal. The transient detection result includes a transient flag, and the global transient detection result includes a global transient flag. The transient flag is used to indicate whether the signal of the corresponding channel is a transient signal. A decoding module is used to decode based on the global transient detection results and the bitstream to obtain frequency domain signals of N transmission channels; The spatial decoding module is used to perform spatial decoding on the frequency domain signals of the N transmission channels based on the global transient detection results and the spatial coding parameters, so as to obtain the reconstructed frequency domain three-dimensional audio signal. The determination module is used to determine the reconstructed time-domain three-dimensional audio signal based on the global transient detection results and the reconstructed frequency-domain three-dimensional audio signal.

26. The apparatus as claimed in claim 25, characterized in that, The determining module includes: A determining unit is configured to determine target coding parameters based on the global transient detection results, wherein the target coding parameters include the window function type of the current frame and / or the frame type of the current frame; A conversion unit is used to convert the reconstructed frequency-domain three-dimensional audio signal into the reconstructed time-domain three-dimensional audio signal based on the target encoding parameters.

27. The apparatus as claimed in claim 26, characterized in that, The target encoding parameters include the window function type of the current frame; The determining unit is specifically used for: If the global transient flag is a first value, then the type of the first preset window function is determined as the window function type of the current frame; If the global transient flag is the second value, then the type of the second preset window function is determined as the window function type of the current frame; Wherein, the window length of the first preset window function is less than the window length of the second preset window function.

28. The apparatus as claimed in claim 26, characterized in that, The transient detection result also includes transient location information, and the global transient detection result also includes global transient location information. The transient location information is used to indicate the location where the transient occurs in the signal of the corresponding channel. The target encoding parameters include the window function type of the current frame. The determining unit is specifically used for: If the global transient flag is a first value, then the window function type of the current frame is determined based on the global transient position information.

29. An encoding terminal device, characterized in that, The encoding device includes a memory and a processor; The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to implement the encoding method according to any one of claims 1-10.

30. A decoding device, characterized in that, The decoding device includes a memory and a processor; The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the decoding method according to any one of claims 11-14.

31. A computer-readable storage medium, characterized in that, The storage medium stores instructions that, when executed on the computer, cause the computer to perform the steps of the method described in any one of claims 1-14.

32. A computer-readable storage medium, characterized in that, Includes the bitstream obtained by the encoding method as described in any one of claims 1-10.

33. A computer program, characterized in that, When the computer program is executed, it implements the method as described in any one of claims 1-14.

Images