Signal encoding and decoding method, apparatus, user equipment, network side device, and storage medium
By determining coding modes based on signal characteristics, the method optimizes encoding efficiency for mixed-format audio signals in 3D audio systems, addressing inefficiencies in existing technologies.
Patent Information
- Application Number
- JP2024525517
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2021-11-02
- Publication Date
- 2024-11-06
- Estimated Expiration
- 2041-11-02
AI Technical Summary
The existing methods for encoding mixed-format audio signals in 3D audio systems are inefficient due to the lack of consideration for signal characteristics, leading to low data compression rates and inadequate bandwidth utilization.
A method that determines a coding mode for each audio signal format based on its characteristics, encoding sound channel-based, object-based, and scene-based audio signals using specific encoding kernels to optimize compression and efficiency.
This approach enhances encoding efficiency by adapting to the unique characteristics of different audio formats, improving data compression rates and bandwidth utilization.
Smart Images

Figure 2024541036000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to the field of communications technology, and in particular to a signal encoding and decoding method, apparatus, encoding device, decoding device, and storage medium. [Background technology]
[0002] 3D audio has been widely applied because it can provide users with a better three-dimensional experience and a sense of spatial immersion. Here, when constructing an end-to-end 3D audio experience, the collection side usually collects mixed-format audio signals, which may include at least two formats, for example, sound channel-based audio signals, object-based audio signals, and scene-based audio signals, and then encodes and decodes the collected signals, and finally renders them into binaural signals or multi-speaker signals and plays them based on the capabilities of the playback device (such as the capabilities of the terminal).
[0003] In the related art, a method for encoding mixed-format audio signals is to process each type of format therein with a corresponding encoding kernel, i.e., the sound channel-based audio signals are processed with a sound channel signal encoding kernel, the object-based audio signals are processed with an object signal encoding kernel, and the scene-based audio signals are processed with a scene signal encoding kernel.
[0004] However, in the related technologies, the efficiency of encoding a mixed-format audio signal is low because parameter information such as control information on the encoding side, characteristics of the input mixed-format audio signal, advantages and disadvantages between audio signals of different formats, and actual playback needs on the playback side are not taken into consideration during encoding. Summary of the Invention [Problem to be solved by the invention]
[0005] The present disclosure provides a signal encoding and decoding method, apparatus, user equipment, network side device and storage medium to solve the technical problem that the encoding method in the related art has a low data compression rate and cannot save bandwidth. [Means for solving the problem]
[0006] The signal encoding and decoding method provided by an embodiment of the present disclosure is applied to an encoding side, obtaining a mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal; determining an encoding mode for each of the audio signals of different formats based on signal characteristics of the audio signals of different formats; The method includes a step of: encoding an audio signal of each format using an encoding mode of the audio signal of each format to obtain signal parameter information after the encoding of the audio signal of each format; and writing the signal parameter information after the encoding of the audio signal of each format into an encoded code stream and transmitting it to a decoding side.
[0007] A signal encoding and decoding method provided by an embodiment of another aspect of the present disclosure is applied to a decoding side, receiving an encoded codestream transmitted from an encoding side; and decoding the encoded code stream to obtain a mixed-format audio signal, the mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal.
[0008] According to another embodiment of the present disclosure, there is provided a signal encoding and decoding device, an acquisition module for acquiring a mixed-format audio signal including at least one of a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal; a decision module for determining an encoding mode of the audio signals of different formats based on signal characteristics of the audio signals of different formats; and an encoding module for encoding an audio signal of each format using an encoding mode for the audio signal of each format to obtain signal parameter information after the encoding of the audio signal of each format, and for writing the signal parameter information after the encoding of the audio signal of each format into an encoded code stream and transmitting it to a decoding side.
[0009] According to another embodiment of the present disclosure, there is provided a signal encoding and decoding device, a receiving module for receiving the encoded code stream transmitted from the encoding side; and a decoding module for decoding the encoded code stream to obtain a mixed-format audio signal, the mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal.
[0010] An embodiment of another aspect of the present disclosure provides a communication device, the device comprising a processor and a memory, the memory storing a computer program, and the processor causing the device to perform a method provided by the embodiment of the above one aspect by storing the computer program stored in the memory.
[0011] An embodiment of another aspect of the present disclosure provides a communication device, the device comprising a processor and a memory, the memory storing a computer program, and the processor causing the device to execute a method provided by the embodiment of the above another aspect by storing the computer program stored in the memory.
[0012] According to another embodiment of the present disclosure, there is provided a communication device comprising a processor and an interface circuit, the interface circuit receives and transmits code instructions to the processor; The processor executes the code instructions to perform the method provided by an embodiment of an aspect.
[0013] According to another embodiment of the present disclosure, there is provided a communication device comprising a processor and an interface circuit, the interface circuit receives and transmits code instructions to the processor; The processor executes the code instructions to perform the method provided by an embodiment of an aspect.
[0014] A computer-readable storage medium provided by an embodiment of another aspect of the present disclosure stores instructions that, when executed, result in the method provided by the embodiment of the one aspect.
[0015] A computer-readable storage medium provided by an embodiment of another aspect of the present disclosure stores instructions that, when executed, result in a method provided by the embodiment of the other aspect. Effect of the Invention
[0016] From the above, the signal encoding and decoding method, apparatus, encoding device, decoding device and storage medium provided by one embodiment of the present disclosure first obtain a mixed-format audio signal including at least one format of a sound channel-based audio signal, an object-based audio signal and a scene-based audio signal, and then determine an encoding mode of the audio signal of each format according to the signal characteristics of the audio signals of different formats, and then encode the audio signals of each format using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signals of each format, and write the encoded signal parameter information of the audio signals of each format into the encoded code stream and send it to the decoding side. It can be seen from this that, in the embodiment of the present disclosure, when encoding the mixed-format audio signal, the audio signals of different formats are reconstructed and analyzed according to the characteristics of the audio signals of different formats, and a self-adaptive encoding mode is determined for the audio signals of different formats, and then the corresponding encoding kernel is used for encoding, thereby achieving better encoding efficiency. [Brief description of the drawings]
[0017] The above and / or additional aspects and advantages of the present disclosure will become apparent and readily understood from the following detailed description of the embodiments taken in conjunction with the drawings. [Figure 1a] 1 is a schematic flowchart of an encoding and decoding method provided by an embodiment of the present disclosure. [Figure 1b] FIG. 2 is a schematic diagram of a microphone collection layout on the collection side provided by one embodiment of the present disclosure. [Figure 1c] FIG. 1C is a schematic diagram of a speaker playback layout corresponding to the playback side of FIG. 1b provided by one embodiment of the present disclosure. [Figure 2a] 4 is a schematic flowchart of another signal encoding and decoding method provided by an embodiment of the present disclosure. [Figure 2b] 4 is a flowchart of a signal encoding method provided by one embodiment of the present disclosure. [Diagram 3] 4 is a schematic flowchart of an encoding and decoding method provided by a further embodiment of the present disclosure; [Figure 4a] 4 is a schematic flowchart of an encoding and decoding method provided by another embodiment of the present disclosure. [Figure 4b] 1 is a flowchart of a signal encoding method for an object-based audio signal provided by one embodiment of the present disclosure. [Figure 5a] 4 is a schematic flowchart of an encoding and decoding method provided by another embodiment of the present disclosure. [Figure 5b] 4 is a flowchart of a signal encoding method for another object-based audio signal provided by an embodiment of the present disclosure. [Figure 6a] 4 is a schematic flowchart of an encoding and decoding method provided by another embodiment of the present disclosure. [Figure 6b] 4 is a flowchart of a signal encoding method for another object-based audio signal provided by an embodiment of the present disclosure. [Figure 7a] 4 is a schematic flowchart of an encoding and decoding method provided by another embodiment of the present disclosure. [Figure 7b] FIG. 1 is a block diagram of an ACELP coding principle provided by another embodiment of the present disclosure. [Figure 7c] FIG. 1 is a block diagram of a frequency domain coding principle provided by one embodiment of the present disclosure. [Figure 7d] 4 is a flowchart of a method for encoding a second type of object signal set provided by an embodiment of the present disclosure. [Figure 8a] 4 is a schematic flowchart of an encoding and decoding method provided by another embodiment of the present disclosure. [Figure 8b]11 is a flowchart of another method for encoding a second type of object signal set provided by an embodiment of the present disclosure. [Figure 9a] 4 is a schematic flowchart of an encoding and decoding method provided by another embodiment of the present disclosure. [Figure 9b] 11 is a flowchart of another method for encoding a second type of object signal set provided by an embodiment of the present disclosure. [Figure 10] 4 is a schematic flowchart of an encoding and decoding method provided by another embodiment of the present disclosure. [Figure 11a] 4 is a schematic flowchart of an encoding and decoding method provided by another embodiment of the present disclosure. [Figure 11b] 4 is a flowchart of a signal decoding method provided by one embodiment of the present disclosure. [Figure 12a] 4 is a schematic flowchart of an encoding and decoding method provided by another embodiment of the present disclosure. [Fig. 12b-12d] 5A and 5B are flowcharts of a method for decoding an object-based audio signal provided by one embodiment of the present disclosure, respectively. [Fig. 12e-12f] 6 is a flowchart of a method for decoding a second type of object signal set provided by an embodiment of the present disclosure, respectively. [Figure 13] 4 is a schematic flowchart of an encoding and decoding method provided by another embodiment of the present disclosure. [Figure 14] 4 is a schematic flowchart of an encoding and decoding method provided by another embodiment of the present disclosure. [Figure 15] 4 is a schematic flowchart of an encoding and decoding method provided by another embodiment of the present disclosure. [Figure 16] 4 is a schematic flowchart of an encoding and decoding method provided by another embodiment of the present disclosure. [Figure 17]4 is a schematic flowchart of an encoding and decoding method provided by another embodiment of the present disclosure. [Figure 18] FIG. 2 is a structural schematic diagram of an encoding and decoding device provided by an embodiment of the present disclosure; [Figure 19] FIG. 2 is a structural schematic diagram of an encoding and decoding device provided by another embodiment of the present disclosure. [Figure 20] FIG. 2 is a block diagram of user equipment provided by one embodiment of the present disclosure. [Figure 21] FIG. 2 is a block diagram of a network side device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0018] DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS Now, exemplary embodiments will be described in detail, examples of which are illustrated in the drawings. When the following description refers to the drawings, the same numerals in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with embodiments of the present invention. Rather, they are merely examples of apparatus and methods consistent with some aspects of embodiments of the present invention, as detailed in the appended claims.
[0019] The terms used in the embodiments of the present disclosure are for the purpose of describing particular embodiments, and are not intended to limit the embodiments of the present disclosure. Unless otherwise clearly indicated by the context, the singular forms "a" and "the" used in the embodiments of the present disclosure and the appended claims also include the plural forms. In addition, the term "and / or" as used herein refers to and includes any and all possible combinations of one or more associated and listed items.
[0020] It should be understood that, although various information may be described with terms such as first, second, and third in the embodiments of the present disclosure, the information should not be limited to these terms. These terms are used only to distinguish the same type of information from each other. For example, the first information may be referred to as the second information, and similarly, the second information may be referred to as the first information, without departing from the scope of the embodiments of the present disclosure. Depending on the context, the term "when" as used herein may be interpreted as "when" or "when" or "in response to determining."
[0021] Hereinafter, the encoding and decoding method, apparatus, user equipment, network side device and storage medium provided by an embodiment of the present disclosure will be described in detail with reference to the accompanying drawings.
[0022] FIG. 1a is a schematic flowchart of a signal encoding and decoding method provided by one embodiment of the present disclosure, the method being performed by an encoding side. As shown in FIG. 1a, the signal encoding and decoding method may include the following steps 101 to 103.
[0023] In step 101, a mixed-format audio signal is obtained, the mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal.
[0024] Here, in one embodiment of the present disclosure, the encoding side may be a UE (User Equipment, terminal device) or a base station, and the UE may be a device that provides voice and / or data connectivity to a user. The terminal device may communicate with one or more core networks via a RAN (Radio Access Network), and the UE may be, for example, a sensor device, an Internet of Things such as a mobile phone (also called a "cellular" phone), and a computer with an Internet of Things, and may be, for example, a fixed, portable, pocket, handheld, computer-embedded, or vehicle-mounted device. For example, a station (STA), subscriber unit, subscriber station, mobile station, mobile, remote station, access point, remote terminal, access terminal, user terminal, or user agent. Or, the UE may be an unmanned aerial vehicle device. Or, the UE may be an in-vehicle device, and may be, for example, a mobile computer with wireless communication capabilities, or a wireless communication device with an external mobile computer. Or, the UE may be a roadside device, such as a street lamp, a traffic light or other roadside device with wireless communication capabilities.
[0025] In one embodiment of the present disclosure, the above three types of audio signal formats are specifically divided based on the signal collection format, and the scenes to which the audio signals of different formats are mainly applied are also different.
[0026] Specifically, in one embodiment of the present disclosure, the main application scene of the above sound channel-based audio signal is a scene where the same microphone collection layout and speaker playback layout are pre-set on the collection side and the playback side, respectively, for example, Fig. 1b is a schematic diagram of a microphone collection layout on the collection side provided by one embodiment of the present disclosure, which can collect a 5.0 format sound channel-based audio signal. Fig. 1c is a schematic diagram of a speaker playback layout on the playback side corresponding to Fig. 1b provided by one embodiment of the present disclosure, which can play the 5.0 format sound channel-based audio signal collected by the collection side of Fig. 1b.
[0027] In another embodiment of the present disclosure, the above-mentioned object-based audio signal is usually recorded by using an independent microphone to record a vocalizing object, and its main application scenario is a scenario in which the playback side needs to perform independent control operations on the audio signal, such as audio on / off, volume adjustment, audio and video direction adjustment, and frequency band equalization processing.
[0028] In another embodiment of the present disclosure, the main application scene of the above scene-based audio signal is a scene where the complete sound field where the collecting side is located needs to be recorded, such as live recording of a concert, live recording of a soccer match, etc.
[0029] In step 102, an encoding mode for the audio signals of different formats is determined based on signal characteristics of the audio signals of each format.
[0030] Here, in one embodiment of the present disclosure, the above step of "determining an encoding mode for an audio signal of each format based on signal characteristics of the audio signal of the different formats" may include a step of determining an encoding mode for a sound channel-based audio signal based on signal characteristics of a sound channel-based audio signal, a step of determining an encoding mode for an object-based audio signal based on signal characteristics of an object-based audio signal, and a step of determining an encoding mode for a scene-based audio signal based on signal characteristics of a scene-based audio signal.
[0031] It should be noted that in an embodiment of the present disclosure, for audio signals of different formats, the methods of determining the corresponding encoding modes based on the signal characteristics are different, whereby the determination of the encoding modes of the audio signals of each format based on the signal characteristics of the audio signals of each format will be described in detail in the following embodiments.
[0032] In step 103, the audio signal of each format is encoded using the encoding mode of the audio signal of each format to obtain signal parameter information after encoding of the audio signal of each format, and the signal parameter information after encoding of the audio signal of each format is written into an encoded code stream and transmitted to the decoding side.
[0033] In one embodiment of the present disclosure, the step of encoding the audio signal of each format using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signal of each format includes: encoding the sound channel based audio signal using a coding mode of the sound channel based audio signal; encoding the object-based audio signal using a coding mode for the object-based audio signal; and encoding the scene-based audio signal using a scene-based audio signal encoding mode.
[0034] Furthermore, in one embodiment of the present disclosure, when writing the encoded signal parameter information of the audio signals of each of the above formats into the encoded code stream, side information parameters corresponding to the determined audio signals of each format are simultaneously written into the encoded code stream, where the side information parameters indicate the encoding mode corresponding to the audio signals of the corresponding formats.
[0035] In addition, in one embodiment of the present disclosure, side information parameters corresponding to audio signals of each format are written into an encoded code stream and transmitted to a decoding side, so that the decoding side can determine an encoding mode corresponding to the audio signals of each format based on the side information parameters corresponding to the audio signals of each format, and then decode the audio signals of each format using a corresponding decoding mode based on the encoding mode.
[0036] It should be noted that in one embodiment of the present disclosure, for an object-based audio signal, the corresponding encoded signal parameter information may retain some object signals, while for a scene-based audio signal and a sound channel-based audio signal, the corresponding encoded signal parameter information is converted into other format signals without needing to retain the original format signals.
[0037] From the above, in the signal encoding and decoding method provided by an embodiment of the present disclosure, firstly, obtain a mixed-format audio signal, the mixed-format audio signal includes at least one format of a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal, and then determine an encoding mode of the audio signal of each format according to the signal characteristics of the audio signals of different formats, and then encode the audio signals of each format using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signals of each format, and write the encoded signal parameter information of the audio signals of each format into the encoded code stream and send it to the decoding side. From this, it can be seen that in the embodiment of the present disclosure, when encoding the mixed-format audio signal, the audio signals of different formats are reconstructed and analyzed according to the characteristics of the audio signals of different formats, and an adaptive encoding mode is determined for the audio signals of different formats, and then encoded using the corresponding encoding kernel to achieve better encoding efficiency.
[0038] FIG. 2a is a schematic flowchart of another signal encoding and decoding method provided by an embodiment of the present disclosure, which is performed by an encoding side. As shown in FIG. 2a, the signal encoding and decoding method may include the following steps 201 to 205.
[0039] In step 201, a mixed-format audio signal is obtained, the mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal.
[0040] At step 202, in response to the sound channel-based audio signal being included in the mixed format audio signal, an encoding mode for the sound channel-based audio signal is determined based on signal characteristics of the sound channel-based audio signal.
[0041] Wherein, in one embodiment of the present disclosure, determining an encoding mode of the sound channel-based audio signal based on signal characteristics of the sound channel-based audio signal includes: This may include obtaining a number of object signals included in the sound channel-based audio signal, and determining whether the number of object signals included in the sound channel-based audio signal is less than a first threshold (which may be, for example, 5).
[0042] Here, in one embodiment of the present disclosure, if the number of object signals included in the sound channel-based audio signal is smaller than a first threshold, it is determined that the encoding mode of the sound channel-based audio signal is at least one of the following measures 1 to 2.
[0043] In strategy 1, an object signal coding kernel is used to code each object signal in a sound channel-based audio signal.
[0044] In a second method, the method obtains input first command line control information, and uses an object signal encoding kernel to encode at least some object signals in the sound channel-based audio signal according to the first command line control information, where the first command line control information indicates object signals that need to be encoded among the object signals included in the sound channel-based audio signal, and the number of object signals that need to be encoded is equal to or greater than 1 and is smaller than a total number of object signals included in the sound channel-based audio signal.
[0045] As can be seen from this, in one embodiment of the present disclosure, when it is determined that the number of object signals contained in the sound channel-based audio signal is less than a first threshold, all or some of the object signals in the sound channel-based audio signal are encoded, thereby greatly reducing the encoding difficulty and improving the encoding efficiency.
[0046] In another embodiment of the present disclosure, when the number of object signals included in the sound channel-based audio signal is equal to or greater than a first threshold, the encoding mode of the sound channel-based audio signal is determined to be at least one of the following measures 3 to 5.
[0047] In the third method, the sound channel-based audio signal is converted into a first other format audio signal (which may be, for example, a scene-based audio signal or an object-based audio signal), and the number of sound channels of the first other format audio signal is equal to or less than the number of sound channels of the sound channel-based audio signal, and the coding kernel corresponding to the first other format audio signal is used to code the first other format audio signal. Exemplarily, in one embodiment of the present disclosure, when the sound channel-based audio signal is a 7.1.4 format sound channel-based audio signal (total number of sound channels is 13), the first other format audio signal may be, for example, a FOA (First Order Ambisonics) signal (total number of sound channels is 4), and by converting the 7.1.4 format sound channel-based audio signal into a FOA signal, the total number of sound channels of the signal that needs to be coded can be converted from 13 to 4, which can greatly reduce the coding difficulty and improve the coding efficiency.
[0048] In method 4, an input first command line control information is obtained, and an object signal encoding kernel is used to encode at least some object signals in the sound channel-based audio signal based on the first command line control information, wherein the first command line control information indicates object signals that need to be encoded among the object signals included in the sound channel-based audio signal, and the number of object signals that need to be encoded is greater than or equal to 1 and less than the total number of object signals included in the sound channel-based audio signal.
[0049] In method 5, the input second command line control information is obtained, and at least some sound channel signals in the sound channel-based audio signal are encoded using the object signal encoding kernel according to the second command line control information, where the second command line control information indicates sound channel signals that need to be encoded among the sound channel signals included in the sound channel-based audio signal, and the number of the sound channel signals that need to be encoded is greater than or equal to 1 and less than or equal to a total number of sound channel signals included in the sound channel-based audio signal.
[0050] As can be seen from this, in an embodiment of the present disclosure, when it is determined that the number of object signals contained in a sound channel-based audio signal is large, if the sound channel-based audio signal is directly encoded, the encoding difficulty is high. At this time, only some of the object signals in the sound channel-based audio signal may be encoded, and / or some of the sound channel signals in the sound channel-based audio signal may be encoded, and / or the sound channel-based audio signal may be converted into a signal with a smaller number of sound channels before encoding, so that the encoding difficulty can be greatly reduced and the encoding efficiency can be optimized.
[0051] At step 203, in response to an object-based audio signal being included in the mixed format audio signal, an encoding mode for the object-based audio signal is determined based on signal characteristics of the object-based audio signal.
[0052] A detailed description of step 203 will be given in the following embodiment.
[0053] At step 204, in response to a scene-based audio signal being included in the mixed-format audio signal, an encoding mode for the scene-based audio signal is determined based on signal characteristics of the scene-based audio signal.
[0054] In one embodiment of the present disclosure, the step of determining an encoding mode of the scene-based audio signal based on signal features of the scene-based audio signal includes: The method may include obtaining the number of object signals included in the scene-based audio signal, and determining whether the number of object signals included in the scene-based audio signal is less than a second threshold value (which may be 5, for example).
[0055] Here, in one embodiment of the present disclosure, when the number of object signals contained in the scene-based audio signal is smaller than a second threshold, the encoding mode of the scene-based audio signal is determined to be at least one of the following measures a to b.
[0056] In strategy a, each object signal of a scene-based audio signal is coded using an object signal coding kernel.
[0057] In method b, input fourth command line control information is obtained, and at least some of the object signals in the scene-based audio signal are encoded using an object signal encoding kernel based on the fourth command line control information, where the fourth command line control information indicates the object signals that need to be encoded among the object signals included in the scene-based audio signal, and the number of object signals that need to be encoded is greater than or equal to 1 and less than or equal to the total number of object signals included in the scene-based audio signal.
[0058] As can be seen from this, in one embodiment of the present disclosure, when it is determined that the number of object signals contained in the scene-based audio signal is less than a second threshold, all or some of the object signals of the scene-based audio signal are encoded, thereby significantly reducing the difficulty of encoding and improving the encoding efficiency.
[0059] In another embodiment of the present disclosure, when the number of object signals contained in the scene-based audio signal is greater than or equal to a second threshold, the encoding mode of the scene-based audio signal is determined to be at least one of the following measures c to d.
[0060] In method c, the scene-based audio signal is converted into an audio signal of a second other format, the number of sound channels of the audio signal of the second other format being less than or equal to the number of sound channels of the scene-based audio signal, and the audio signal of the second other format is encoded using a scene signal encoding kernel.
[0061] In method d, a low-order transformation is performed on the scene-based audio signal to transform the scene-based audio signal into a low-order scene-based audio signal whose order is lower than the current order of the scene-based audio signal, and the low-order scene-based audio signal is encoded using a scene signal encoding kernel. Note that, in one embodiment of the present disclosure, when a low-order transformation is performed on the scene-based audio signal, the scene-based audio signal may be low-order transformed into a signal of another format. For example, a third-order scene-based audio signal may be transformed into a sound channel-based audio signal of a low-order 5.0 format, and the total sound channel number of the signal that needs to be encoded is changed from 16 ((3+1)*(3+1)) to 5, thereby greatly reducing the difficulty of encoding and improving the encoding efficiency.
[0062] As can be seen from this, in an embodiment of the present disclosure, when it is determined that the number of object signals contained in a scene-based audio signal is large, if the scene-based audio signal is directly encoded, the encoding difficulty is high. At this time, only the scene-based audio signal may be converted into a signal with a small number of sound channels and then encoded, and / or the scene-based audio signal may be converted into a low-order signal and then encoded, thereby greatly reducing the encoding difficulty and improving the encoding efficiency.
[0063] In step 205, the audio signal of each format is encoded using the encoding mode of the audio signal of each format to obtain signal parameter information after encoding of the audio signal of each format, and the signal parameter information after encoding of the audio signal of each format is written into an encoded code stream and transmitted to the decoding side.
[0064] Here, for an introduction to step 205, please refer to the description of the above embodiment, and a detailed description will be omitted in the embodiment of the present disclosure.
[0065] Finally, based on the above description, Figure 2b is a flowchart of a signal encoding method provided by one embodiment of the present disclosure. As can be seen in combination with the above description and Figure 2b, when the encoding side receives a mixed-format audio signal, it classifies the audio signal of each format through signal feature analysis, and then, based on the command line control information (i.e., the above first command line control information, and / or the second command line control information (described in the following content), and / or the fourth command line control information), encodes the audio signal of each format in a corresponding encoding mode using a corresponding encoding kernel, and writes the encoded signal parameter information of the audio signal of each format into an encoded code stream and sends it to the decoding side.
[0066] From the above, in the signal encoding and decoding method provided by an embodiment of the present disclosure, firstly, obtain a mixed-format audio signal, the mixed-format audio signal includes at least one format of a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal, and then determine an encoding mode of the audio signal of each format according to the signal characteristics of the audio signals of different formats, and then encode the audio signals of each format using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signals of each format, and write the encoded signal parameter information of the audio signals of each format into the encoded code stream and send it to the decoding side. From this, it can be seen that in the embodiment of the present disclosure, when encoding the mixed-format audio signal, the audio signals of different formats are reconstructed and analyzed according to the characteristics of the audio signals of different formats, and an adaptive encoding mode is determined for the audio signals of different formats, and then encoded using the corresponding encoding kernel to achieve better encoding efficiency.
[0067] FIG. 3 is a schematic flowchart of a signal encoding and decoding method provided by one embodiment of the present disclosure, the method being performed by an encoding side. As shown in FIG. 3, the signal encoding and decoding method may include the following steps 301 to 306.
[0068] In step 301, a mixed-format audio signal is obtained, the mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal.
[0069] At step 302, in response to an object-based audio signal being included in the mixed-format audio signal, a signal feature analysis is performed on the object-based audio signal to obtain an analysis result.
[0070] Here, in one embodiment of the present disclosure, the signal feature analysis may be a cross-correlation parameter value analysis of the signal. In another embodiment of the present disclosure, the feature analysis may be a frequency bandwidth range analysis of the signal. And the cross-correlation parameter value analysis and the frequency bandwidth range analysis will be described in detail in the following embodiments.
[0071] In step 303, the object-based audio signal is classified to obtain a first type of object signal set and a second type of object signal set, and the first type of object signal set and the second type of object signal set both include at least one object-based audio signal.
[0072] The object-based audio signal may include different types of object signals, and the subsequent encoding modes for different types of object signals are different. Therefore, in one embodiment of the present disclosure, the different types of object signals in the object-based audio signal can be classified to obtain a first type of object signal set and a second type of object signal set, and then corresponding encoding modes can be determined for the first type of object signal set and the second type of object signal set, respectively. Here, the classification manner of the first type of object signal set and the second type of object signal set will be described in detail in the following embodiment.
[0073] In step 304, a coding mode corresponding to a set of object signals of a first type is determined.
[0074] In an embodiment of the present disclosure, when the classification methods for the first type of object signal set in the above step 303 are different, the encoding modes for the first type of object signal set determined in this step are also different. Here, a specific method for "determining the encoding mode corresponding to the first type of object signal set" will be described in the following embodiment.
[0075] In step 305, classify the second type object signal set based on the analysis result to obtain at least one object signal subset, and determine an encoding mode corresponding to each object signal subset based on the classification result, wherein the object signal subset includes at least one object-based audio signal.
[0076] Here, if the signal feature analysis method adopted in step 302 is different, the method for classifying the object-based audio signals in this step and the method for determining the coding mode corresponding to each object signal subset are also different.
[0077] Specifically, in one embodiment of the present disclosure, when the signal feature analysis method adopted in step 302 is a signal cross-correlation parameter value analysis method, the classification method of the second type of object signal set in this step may be a classification method based on the signal cross-correlation parameter values, and the method of determining the encoding mode corresponding to each object signal subset may be to determine the encoding mode corresponding to each object signal subset based on the signal cross-correlation parameter values.
[0078] In another embodiment of the present disclosure, when the signal feature analysis method adopted in step 302 is a signal frequency bandwidth range analysis method, the classification method of the second type of object signal set in this step may be a classification method based on the signal frequency bandwidth range, and the method of determining the coding mode corresponding to each object signal subset may be to determine the coding mode corresponding to each object signal subset based on the signal frequency bandwidth range.
[0079] In addition, detailed explanations of the above "classification method based on cross-correlation parameter values of signals or frequency bandwidth ranges of signals" and "determining an encoding mode corresponding to each object signal subset based on cross-correlation parameter values of signals or frequency bandwidth ranges of signals" will also be provided in the examples below.
[0080] In step 306, the audio signal of each format is encoded using the encoding mode of the audio signal of each format to obtain signal parameter information after encoding of the audio signal of each format, and the signal parameter information after encoding of the audio signal of each format is written into an encoded code stream and transmitted to the decoding side.
[0081] Here, in an embodiment of the present disclosure, if the classification manner of the second type object signal set in step 307 is different, the coding situation for the second type object signal subset is also different.
[0082] Based on this, in one embodiment of the present disclosure, the signal parameter information after encoding of the audio signal in each of the above formats is written into the encoded code stream and transmitted to the decoding side, specifically, determining a classification side information parameter indicative of a classification scheme for a set of object signals of a second type; a step 2 of determining side information parameters corresponding to each format of the audio signal, the side information parameters indicating a coding mode corresponding to the audio signal of the corresponding format; and step 3 of performing code stream multiplexing on the classification side information parameters, the side information parameters corresponding to the audio signals of each format, and the signal parameter information after encoding of the audio signals of each format to obtain an encoded code stream, and transmitting the encoded code stream to a decoding side.
[0083] Here, in one embodiment of the present disclosure, by sending the classification side information parameters and the side information parameters corresponding to the audio signals of each format to the decoding side, the decoding side can determine the encoding status corresponding to the object signal subsets in the second type of object signal set based on the classification side information parameters, and determine the encoding mode corresponding to each object signal subset based on the side information parameters corresponding to each object signal subset, so that the object-based audio signals can then be decoded using the corresponding decoding mode and decoding mode based on the encoding status and encoding mode; and the decoding side can determine the encoding modes corresponding to the sound channel-based audio signals and the scene-based audio signals based on the side information parameters corresponding to the audio signals of each format, thereby realizing the decoding of the sound channel-based audio signals and the scene-based audio signals.
[0084] From the above, in the signal encoding and decoding method provided by an embodiment of the present disclosure, firstly, obtain a mixed-format audio signal, the mixed-format audio signal includes at least one format of a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal, and then determine an encoding mode of the audio signal of each format according to the signal characteristics of the audio signals of different formats, and then encode the audio signals of each format using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signals of each format, and write the encoded signal parameter information of the audio signals of each format into the encoded code stream and send it to the decoding side. From this, it can be seen that in the embodiment of the present disclosure, when encoding the mixed-format audio signal, the audio signals of different formats are reconstructed and analyzed according to the characteristics of the audio signals of different formats, and an adaptive encoding mode is determined for the audio signals of different formats, and then encoded using the corresponding encoding kernel to achieve better encoding efficiency.
[0085] FIG. 4a is a schematic flowchart of a signal encoding and decoding method provided by another embodiment of the present disclosure, the method is performed by an encoding side, and as shown in FIG. 4a, the signal encoding and decoding method may include the following steps 401 to 406.
[0086] In step 401, a mixed-format audio signal is obtained, the mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal.
[0087] At step 402, in response to an object-based audio signal being included in the mixed-format audio signal, a signal feature analysis is performed on the object-based audio signal to obtain an analysis result.
[0088] Here, for an explanation of steps 401 and 402, please refer to the explanation of the above-mentioned embodiment, and a detailed explanation will be omitted in the embodiment of the present disclosure.
[0089] In step 403, the object-based audio signals that do not require separate manipulation processing are classified into a first type of object signal set, and the remaining signals are classified into a second type of object signal set, and both the first type of object signal set and the second type of object signal set include at least one object-based audio signal.
[0090] In step 404, it is determined that an encoding mode corresponding to the first type of object signal set is to perform a first pre-rendering process on the object-based audio signals in the first type of object signal set, and encode the first pre-rendered processed signals using a multi-channel encoding kernel.
[0091] Here, in one embodiment of the present disclosure, the first pre-rendering process may include performing a signal format conversion process on the object-based audio signal to convert the object-based audio signal into a sound channel-based audio signal.
[0092] In step 405, classify the second type object signal set based on the analysis result to obtain at least one object signal subset, and determine an encoding mode corresponding to each object signal subset based on the classification result, where the object signal subset includes at least one object-based audio signal.
[0093] In step 406, the audio signal of each format is encoded using the encoding mode of the audio signal of each format to obtain signal parameter information after encoding of the audio signal of each format, and the signal parameter information after encoding of the audio signal of each format is written into an encoded code stream and transmitted to the decoding side.
[0094] Here, for an explanation of steps 405 to 406, please refer to the explanation of the above-mentioned embodiment, and a detailed explanation will be omitted in the embodiment of the present disclosure.
[0095] Finally, based on the above description, FIG. 4b is a flowchart of a signal encoding method for an object-based audio signal provided by an embodiment of the present disclosure. In combination with the above description and FIG. 4b, it can be seen that first, feature analysis is performed on the object-based audio signal, and then the object-based audio signal is classified into a first type of object signal set and a second type of object signal set, and then a first pre-rendering process is performed on the first type of object signal set and encoded using a multi-sound channel encoding kernel, and the second type of object signal set is classified based on the analysis result to obtain at least one object signal subset (e.g., object signal subset 1, object signal subset 2, object signal subset n), and then the at least one object signal subset is encoded respectively.
[0096] From the above, in the signal encoding and decoding method provided by an embodiment of the present disclosure, firstly, obtain a mixed-format audio signal, the mixed-format audio signal includes at least one format of a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal, and then determine an encoding mode of the audio signal of each format according to the signal characteristics of the audio signals of different formats, and then encode the audio signals of each format using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signals of each format, and write the encoded signal parameter information of the audio signals of each format into the encoded code stream and send it to the decoding side. From this, it can be seen that in the embodiment of the present disclosure, when encoding the mixed-format audio signal, the audio signals of different formats are reconstructed and analyzed according to the characteristics of the audio signals of different formats, and an adaptive encoding mode is determined for the audio signals of different formats, and then encoded using the corresponding encoding kernel to achieve better encoding efficiency.
[0097] FIG. 5a is a schematic flowchart of a signal encoding and decoding method provided by one embodiment of the present disclosure, the method is performed by an encoding side, and as shown in FIG. 5a, the signal encoding and decoding method may include the following steps 501 to 506.
[0098] In step 501, a mixed-format audio signal is obtained, the mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal.
[0099] At step 502, in response to an object-based audio signal being included in the mixed-format audio signal, a signal feature analysis is performed on the object-based audio signal to obtain an analysis result.
[0100] Here, for an explanation of steps 501 and 502, please refer to the explanation of the above-mentioned embodiment, and a detailed explanation will be omitted in the embodiment of the present disclosure.
[0101] In step 503, the object-based audio signals that belong to background sound are classified into a first type of object signal set, and the remaining signals are classified into a second type of object signal set, and both the first type of object signal set and the second type of object signal set include at least one object-based audio signal.
[0102] In step 504, it is determined that an encoding mode corresponding to the first type of object signal set is to perform a second pre-rendering process on the object-based audio signals in the first type of object signal set and encode the second pre-rendered signals using a High Order Ambisonics (HOA) encoding kernel.
[0103] Here, in one embodiment of the present disclosure, the second pre-rendering process may be to perform a signal format conversion process on the object-based audio signal to convert the object-based audio signal into a scene-based audio signal.
[0104] In step 505, classify the second type object signal set based on the analysis result to obtain at least one object signal subset, and determine an encoding mode corresponding to each object signal subset based on the classification result, wherein the object signal subset includes at least one object-based audio signal.
[0105] In step 506, the audio signal of each format is encoded using the encoding mode of the audio signal of each format to obtain signal parameter information after encoding of the audio signal of each format, and the signal parameter information after encoding of the audio signal of each format is written into an encoded code stream and transmitted to the decoding side.
[0106] Here, for an explanation of steps 505 to 506, please refer to the explanation of the above-mentioned embodiment, and a detailed explanation will be omitted in the embodiment of the present disclosure.
[0107] Finally, based on the above description, FIG. 5b is a flowchart of another signal encoding method for an object-based audio signal provided by an embodiment of the present disclosure. In combination with the above description and FIG. 5b, it can be seen that first, feature analysis is performed on the object-based audio signal, and then the object-based audio signal is classified into a first type of object signal set and a second type of object signal set, and then a second pre-rendering process is performed on the first type of object signal set and encoded using the HOA encoding kernel, and the second type of object signal set is classified based on the analysis result to obtain at least one object signal subset (e.g., object signal subset 1, object signal subset 2, object signal subset n), and then the at least one object signal subset is encoded respectively.
[0108] From the above, in the signal encoding and decoding method provided by an embodiment of the present disclosure, firstly, obtain a mixed-format audio signal, the mixed-format audio signal includes at least one format of a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal, and then determine an encoding mode of the audio signal of each format according to the signal characteristics of the audio signals of different formats, and then encode the audio signals of each format using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signals of each format, and write the encoded signal parameter information of the audio signals of each format into the encoded code stream and send it to the decoding side. From this, it can be seen that in the embodiment of the present disclosure, when encoding the mixed-format audio signal, the audio signals of different formats are reconstructed and analyzed according to the characteristics of the audio signals of different formats, and an adaptive encoding mode is determined for the audio signals of different formats, and then encoded using the corresponding encoding kernel to achieve better encoding efficiency.
[0109] FIG. 6a is a schematic flow chart of a signal encoding and decoding method provided by one embodiment of the present disclosure, the method including: encoding side The difference between the embodiment of Fig. 6a and the embodiments of Fig. 4a and Fig. 5a is that in this embodiment, the first type object signal set is further divided into a first object signal subset and a second object signal subset. As shown in Fig. 6a, the signal encoding and decoding method may include the following steps:
[0110] In step 601, a mixed-format audio signal is obtained, the mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal.
[0111] In step 602, a signal feature analysis is performed on the object-based audio signal to obtain an analysis result.
[0112] In step 603, the object-based audio signals that do not require individual operation processing are classified into a first object signal subset, the object-based audio signals that belong to background sounds are classified into a second object signal subset, and the remaining signals are classified into a second type of object signal set, where the first type of object signal subset, the second type of object signal subset and the second type of object signal set all include at least one object-based audio signal.
[0113] In step 604, coding modes for a first subset of object signals and a second subset of object signals in a first type of object signal set are determined.
[0114] Here, in one embodiment of the present disclosure, it is determined that the encoding mode corresponding to a first object signal subset in the first type of object signal set is to perform a first pre-rendering process on the object-based audio signals in the first object signal subset and encode the first pre-rendered processed signals using a multi-channel encoding kernel, and the first pre-rendering process includes performing a signal format conversion process on the object-based audio signals to convert the object-based audio signals into sound channel-based audio signals.
[0115] In one embodiment of the present disclosure, it is determined that the encoding mode corresponding to a second object signal subset in the first type of object signal set is to perform a second pre-rendering process on the object-based audio signals in the second object signal subset and encode the second pre-rendered processed signals using an HOA encoding kernel, where the second pre-rendering process includes performing a signal format conversion process on the object-based audio signals to convert the object-based audio signals into scene-based audio signals.
[0116] In step 605, classify the second type object signal set based on the analysis result to obtain at least one object signal subset, and determine an encoding mode corresponding to each object signal subset based on the classification result, wherein the object signal subset includes at least one object-based audio signal.
[0117] In step 606, the audio signal of each format is encoded using the encoding mode of the audio signal of each format to obtain signal parameter information after encoding of the audio signal of each format, and the signal parameter information after encoding of the audio signal of each format is written into an encoded code stream and transmitted to the decoding side.
[0118] For a detailed explanation of steps 601 to 606, please refer to the explanation of the above embodiment, and a detailed explanation will be omitted in the embodiment of the present disclosure.
[0119] Finally, based on the above description, FIG. 6b is a flowchart of another signal encoding method for object-based audio signals provided by an embodiment of the present disclosure. As can be seen in combination with the above description and FIG. 6b, firstly, a feature analysis is performed on the object-based audio signal, and then the object-based audio signal is classified into a first type of object signal set and a second type of object signal set, where the first type of object signal set includes a first object signal subset and a second object signal subset, a first pre-rendering process is performed on the first object signal subset and the first pre-rendering process is performed on the second ... pre-rendering process.
[0120] From the above, in the signal encoding and decoding method provided by an embodiment of the present disclosure, firstly, obtain a mixed-format audio signal, the mixed-format audio signal includes at least one format of a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal, and then determine an encoding mode of the audio signal of each format according to the signal characteristics of the audio signals of different formats, and then encode the audio signals of each format using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signals of each format, and write the encoded signal parameter information of the audio signals of each format into the encoded code stream and send it to the decoding side. From this, it can be seen that in the embodiment of the present disclosure, when encoding the mixed-format audio signal, the audio signals of different formats are reconstructed and analyzed according to the characteristics of the audio signals of different formats, and an adaptive encoding mode is determined for the audio signals of different formats, and then encoded using the corresponding encoding kernel to achieve better encoding efficiency.
[0121] FIG. 7a is a schematic flowchart of a signal encoding and decoding method provided by one embodiment of the present disclosure, the method is performed by an encoding side, and as shown in FIG. 7a, the signal encoding and decoding method may include the following steps 701 to 707.
[0122] In step 701, a mixed-format audio signal is obtained, the mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal.
[0123] At step 702, in response to the mixed format audio signal including an object-based audio signal, a high-pass filtering process is performed on the object-based audio signal.
[0124] In one embodiment of the present disclosure, a filter may be used to high-pass filter the object signal.
[0125] Here, the cutoff frequency of the filter is set to 20 Hz (Hertz). The filter equation used in the filter is as shown in the following equation (1).
number
[0126] In step 703, a correlation analysis is performed on the high-pass filtered signal to determine cross-correlation parameter values between each object-based audio signal.
[0127] Here, in one embodiment of the present disclosure, the correlation analysis can be specifically calculated using the following formula (2).
number
[0128] It should be understood that the above method of "calculating the cross-correlation parameter value using Equation (2)" is one optional method provided by one embodiment of the present disclosure, and other methods of calculating the cross-correlation parameter value between object signals in the art are also applicable to the present disclosure.
[0129] In step 704, the object-based audio signal is classified to obtain a first type of object signal set and a second type of object signal set, each of the first type of object signal set and the second type of object signal set including at least one object-based audio signal.
[0130] In step 705, a coding mode corresponding to a set of object signals of a first type is determined.
[0131] Here, for an introduction of steps 704 to 705, please refer to the description of the above-mentioned embodiment, and a detailed description will be omitted in the embodiment of the present disclosure.
[0132] In step 706, classify the second type object signal set based on the analysis result to obtain at least one object signal subset, and determine an encoding mode corresponding to each object signal subset based on the classification result, wherein the object signal subset includes at least one object-based audio signal.
[0133] In one embodiment of the present disclosure, the step of classifying the second type object signal set to obtain at least one object signal subset and determining an encoding mode corresponding to each object signal subset based on the classification result includes: The method may include setting a normalized correlation interval based on the correlation degree, classifying at least one second type object signal set to obtain at least one object signal subset based on the cross-correlation parameters of the signals and the normalized correlation degree interval, and then determining a corresponding encoding mode based on the correlation degree corresponding to the object signal set.
[0134] In addition, the number of normalized correlation intervals is determined by the correlation division method, and the present disclosure does not limit the correlation division method, nor does it limit the lengths of the different normalized correlation intervals. A corresponding number of normalized correlation intervals and different interval lengths may be set based on the different correlation division methods.
[0135] In one embodiment of the present disclosure, the correlation level is classified into four types of correlation levels: weak correlation, actual correlation, significant correlation, and high correlation. Table 1 is a classification table of normalized correlation level intervals provided by one embodiment of the present disclosure. [Table 1]
[0136] Based on the above, for example, divide the object signals whose cross-correlation parameter values are in a first interval into an object signal set 1, and determine that the object signal set 1 corresponds to an independent coding mode; Divide the object signals whose cross-correlation parameter values are in a second interval into an object signal set 2, and determine that the object signal set 2 corresponds to a joint coding mode 1; Divide the object signals whose cross-correlation parameter values are in a third interval into an object signal set 3, and determine that the object signal set 3 corresponds to a joint coding mode 2; The object signals whose cross-correlation parameter values are in a fourth interval are divided into object signal set 4, and it is determined that object signal set 4 corresponds to joint coding mode 3.
[0137] Here, in one embodiment of the present disclosure, the first interval may be [0.00 to ±0.30), the second interval may be [±0.30-±0.50), the third interval may be [±0.50-±0.80], and the fourth interval may be [±0.80-±1.00]. If the cross-correlation parameter value of the object signals is in the first interval, it indicates that the correlation between the object signals is weak, and in this case, in order to ensure the accuracy of the coding, the coding should be performed using the independent coding mode. If the cross-correlation parameter value between the object signals is in the second, third, or fourth interval, it indicates that the cross-correlation between the object signals is high, and in this case, in order to ensure the compression rate and save the bandwidth, the coding can be performed using the joint coding mode.
[0138] In one embodiment of the present disclosure, the coding modes corresponding to the object signal subsets include an independent coding mode or a joint coding mode.
[0139] In one embodiment of the present disclosure, the independent coding mode corresponds to a time domain processing method or a frequency domain processing method; Wherein, if the object signal in the object signal subset is a voice signal or a similar voice signal, the independent coding mode adopts a time domain processing manner; If the object signals in the object signal subset are audio signals of other formats than speech or similar speech signals, the independent coding mode employs a frequency domain processing scheme.
[0140] In one embodiment of the present disclosure, the above-mentioned time domain processing method can be realized by an ACELP coding model, and Fig. 7b is a block diagram of the principle of ACELP coding provided by one embodiment of the present disclosure. And, the principle of ACELP encoder is specifically referred to the description in the prior art, and the detailed description is omitted in the embodiment of the present disclosure.
[0141] In an embodiment of the present disclosure, the above frequency domain processing manner may include a transform domain processing manner, and Fig. 7c is a block diagram of the principle of frequency domain coding provided by an embodiment of the present disclosure. Referring to Fig. 7c, first, a transform module performs MDCT transform on the input object signal to transform it into frequency domain, where the transform formula and inverse transform formula of MDCT transform are respectively shown in the following formula (3) and formula (4).
number
[0142] Then, a psychoacoustic model is used to adjust each frequency band of the object signal transformed into the frequency domain, and a quantization module is used to quantize each frequency band envelope coefficient through bit allocation to obtain a quantization parameter, and finally, an entropy coding module is used to entropy code the quantization parameter to output an encoded object signal.
[0143] In step 707, the audio signal of each format is encoded using the encoding mode of the audio signal of each format to obtain signal parameter information after encoding of the audio signal of each format, and the signal parameter information after encoding of the audio signal of each format is written into an encoded code stream and transmitted to the decoding side.
[0144] Here, in one embodiment of the present disclosure, encoding the audio signal of each format using the encoding mode of the audio signal of each format to obtain the signal parameter information after the encoding of the audio signal of each format includes: encoding the sound channel based audio signal using a coding mode of the sound channel based audio signal; encoding the object-based audio signal using a coding mode for the object-based audio signal; and encoding the scene-based audio signal using a scene-based audio signal coding mode.
[0145] And, in one embodiment of the present disclosure, encoding the object-based audio signal using the encoding mode of the object-based audio signal includes: Encoding signals in a first type of object signal set using a coding mode corresponding to the first type of object signal set.
[0146] Pre-process the object signal subsets in the second type object signal set, and use the same object signal coding kernel to code all the pre-processed object signal subsets in the second type object signal set in the corresponding coding mode. Then, based on the above description, Figure 7d is a flowchart of a method for coding the second type object signal set provided by an embodiment of the present disclosure.
[0147] From the above, in the signal encoding and decoding method provided by an embodiment of the present disclosure, firstly, obtain a mixed-format audio signal, the mixed-format audio signal includes at least one format of a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal, and then determine an encoding mode of the audio signal of each format according to the signal characteristics of the audio signals of different formats, and then encode the audio signals of each format using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signals of each format, and write the encoded signal parameter information of the audio signals of each format into the encoded code stream and send it to the decoding side. From this, it can be seen that in the embodiment of the present disclosure, when encoding the mixed-format audio signal, the audio signals of different formats are reconstructed and analyzed according to the characteristics of the audio signals of different formats, and an adaptive encoding mode is determined for the audio signals of different formats, and then encoded using the corresponding encoding kernel to achieve better encoding efficiency.
[0148] FIG. 8a is a schematic flowchart of a signal encoding and decoding method provided by one embodiment of the present disclosure, the method is performed by an encoding side, and as shown in FIG. 8a, the signal encoding and decoding method may include the following steps 801 to 806.
[0149] In step 801, a mixed-format audio signal is obtained, the mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal.
[0150] At step 802, in response to an object-based audio signal being included in the mixed-format audio signal, a frequency bandwidth range of the object signal is analyzed.
[0151] In step 803, the object-based audio signal is classified to obtain a first type of object signal set and a second type of object signal set, and both the first type of object signal set and the second type of object signal set include at least one object-based audio signal.
[0152] In step 804, a coding mode corresponding to a set of object signals of a first type is determined.
[0153] In step 805, classify the second type object signal set based on the analysis result to obtain at least one object signal subset, and determine an encoding mode corresponding to each object signal subset based on the classification result, wherein the object signal subset includes at least one object-based audio signal.
[0154] In one embodiment of the present disclosure, classifying the second type object signal set based on the analysis result to obtain at least one object signal subset, and determining an encoding mode corresponding to each object signal subset based on the classification result, includes: determining bandwidth intervals corresponding to different frequency bandwidths; classifying a second type of object signal set to obtain at least one object signal subset based on a frequency bandwidth range of the object signal and a bandwidth interval corresponding to a different frequency bandwidth, and determining a corresponding encoding mode based on a frequency bandwidth corresponding to the at least one object signal subset.
[0155] Here, the frequency bandwidth of the signal usually includes narrowband, wideband, ultra-wideband and fullband. The bandwidth interval corresponding to the narrowband may be a first interval, the bandwidth interval corresponding to the wideband may be a second interval, the bandwidth interval corresponding to the ultra-wideband may be a third interval, and the bandwidth interval corresponding to the fullband may be a fourth interval. Thus, the second type of object signal set may be classified to obtain at least one object signal subset by determining the bandwidth interval to which the frequency bandwidth range of the object signal belongs. Then, a corresponding coding mode is determined based on the frequency bandwidth corresponding to the at least one object signal subset, where the narrowband, wideband, ultra-wideband and fullband correspond to a narrowband coding mode, a wideband coding mode, an ultra-wideband coding mode and a fullband coding mode, respectively.
[0156] It should be noted that the embodiments of the present disclosure do not limit the length of different bandwidth intervals, and the bandwidth intervals between different frequency bandwidths may overlap.
[0157] Also, as an example, an object signal having a frequency bandwidth range in a first interval is divided into an object signal subset 1, and the object signal subset 1 is determined to correspond to a narrowband coding mode; Dividing the object signal in a second frequency bandwidth range into an object signal subset 2, and determining that the object signal subset 2 corresponds to a wideband coding mode; Dividing the object signal in a third frequency bandwidth range into an object signal subset 3, and determining that the object signal subset 3 corresponds to an ultra-wideband coding mode; It may be possible to divide the object signals in the fourth frequency bandwidth range into object signal subset 4, and determine that object signal subset 4 corresponds to the fullband coding mode.
[0158] Here, in one embodiment of the present disclosure, the first section may be 0 to 4 kHz, the second section may be 0 to 8 kHz, the third section may be 0 to 16 kHz, and the fourth section may be 0 to 20 kHz. When the frequency bandwidth of the object signal is in a first interval, it indicates that the object signal is a narrowband signal, and it can be determined that the coding mode corresponding to the object signal is to code with fewer bits (i.e., use the narrowband coding mode); when the frequency bandwidth of the object signal is in a second interval, it indicates that the object signal is a wideband signal, and it can be determined that the coding mode corresponding to the object signal is to code with a relatively larger number of bits (i.e., use the wideband coding mode); when the frequency bandwidth of the object signal is in a third interval, it indicates that the object signal is an ultra-wideband signal, and it can be determined that the coding mode corresponding to the object signal is to code with a larger number of bits (i.e., use the ultra-wideband coding mode); and when the frequency bandwidth of the object signal is in a fourth interval, it indicates that the object signal is a full-band signal, and it can be determined that the coding mode corresponding to the object signal is to code with a larger number of bits (i.e., use the full-band coding mode).
[0159] This allows signals of different frequency bandwidths to be encoded with different bits, thus ensuring compression ratios for the signals and saving bandwidth.
[0160] In step 806, the audio signal of each format is encoded using the encoding mode of the audio signal of each format to obtain signal parameter information after encoding of the audio signal of each format, and the signal parameter information after encoding of the audio signal of each format is written into an encoded code stream and transmitted to the decoding side.
[0161] Here, in one embodiment of the present disclosure, encoding the audio signal of each format using the encoding mode of the audio signal of each format to obtain the signal parameter information after the encoding of the audio signal of each format includes: encoding the sound channel based audio signal using a coding mode of the sound channel based audio signal; encoding the object-based audio signal using a coding mode for the object-based audio signal; and encoding the scene-based audio signal using a scene-based audio signal coding mode.
[0162] In addition, in one embodiment of the present disclosure, encoding an object-based audio signal using the encoding mode of the object-based audio signal includes: encoding signals in a first type of object signal set using an encoding mode corresponding to the first type of object signal set; and pre-processing object signal subsets in the second type object signal set, and encoding the different pre-processed object signal subsets in corresponding encoding modes using different object signal encoding kernels. Based on the above description, Fig. 8b is a flowchart of another method for encoding the second type object signal set provided by an embodiment of the present disclosure.
[0163] From the above, in the signal encoding and decoding method provided by an embodiment of the present disclosure, firstly, obtain a mixed-format audio signal, the mixed-format audio signal includes at least one format of a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal, and then determine an encoding mode of the audio signal of each format according to the signal characteristics of the audio signals of different formats, and then encode the audio signals of each format using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signals of each format, and write the encoded signal parameter information of the audio signals of each format into the encoded code stream and send it to the decoding side. From this, it can be seen that in the embodiment of the present disclosure, when encoding the mixed-format audio signal, the audio signals of different formats are reconstructed and analyzed according to the characteristics of the audio signals of different formats, and an adaptive encoding mode is determined for the audio signals of different formats, and then encoded using the corresponding encoding kernel to achieve better encoding efficiency.
[0164] FIG. 9a is a schematic flowchart of a signal encoding and decoding method provided by one embodiment of the present disclosure, the method is performed by an encoding side, and as shown in FIG. 9a, the signal encoding and decoding method may include the following steps 901 to 907.
[0165] In step 901, a mixed-format audio signal is obtained, the mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal.
[0166] At step 902, in response to an object-based audio signal being included in the mixed-format audio signal, a frequency bandwidth range of the object signal is analyzed.
[0167] In step 903, the object-based audio signal is classified to obtain a first type of object signal set and a second type of object signal set, each of the first type of object signal set and the second type of object signal set including at least one object-based audio signal.
[0168] In step 904, a coding mode corresponding to a set of object signals of a first type is determined.
[0169] At step 905, an input third command line control information is obtained indicating an encoded frequency bandwidth range corresponding to the object-based audio signal.
[0170] In step 906, the third command line control information and the analysis result are integrated to classify the second type object signal set to obtain at least one object signal subset, and an encoding mode corresponding to each object signal subset is determined based on the classification result.
[0171] Here, in one embodiment of the present disclosure, classifying the second type object signal set by integrating the third command line control information and the analysis result to obtain at least one object signal subset, and determining an encoding mode corresponding to each object signal subset based on the classification result. If the frequency bandwidth range indicated by the third command line control information is different from the frequency bandwidth range obtained from the analysis result, classifying the second type of object signal set preferentially in the frequency bandwidth range indicated by the third command line control information, and determining the coding mode corresponding to each object signal set according to the classification result; If the frequency bandwidth range indicated by the third command line control information is the same as the frequency bandwidth range obtained from the analysis result, classifying the second type of object signal set in the frequency bandwidth range indicated by the third command line control information or the frequency bandwidth range obtained from the analysis result, and determining an encoding mode corresponding to each object signal set based on the classification result.
[0172] For example, in one embodiment of the present disclosure, assuming that the analysis result of the object signal is an ultra-wideband signal and the frequency bandwidth range indicated by the third command line control information of the object signal is a full-band signal, the object signal can be divided into object signal subsets 4 based on the third command line control information, and the coding mode corresponding to the object signal subset 4 can be determined to be a full-band coding mode.
[0173] In step 907, the audio signal of each format is encoded using the encoding mode of the audio signal of each format to obtain signal parameter information after encoding of the audio signal of each format, and the signal parameter information after encoding of the audio signal of each format is written into an encoded code stream and transmitted to the decoding side.
[0174] Here, in one embodiment of the present disclosure, encoding an audio signal of each format using an encoding mode of the audio signal of each format to obtain signal parameter information after encoding of the audio signal of each format includes: encoding the sound channel based audio signal using a coding mode of the sound channel based audio signal; encoding the object-based audio signal using a coding mode for the object-based audio signal; and encoding the scene-based audio signal using a scene-based audio signal coding mode.
[0175] And, in one embodiment of the present disclosure, encoding the object-based audio signal using the encoding mode of the object-based audio signal includes: encoding signals in a first type of object signal set using an encoding mode corresponding to the first type of object signal set; and pre-processing object signal subsets in the second type object signal set, and encoding the different pre-processed object signal subsets in corresponding encoding modes using different object signal encoding kernels. Based on the above description, Fig. 9b is a flowchart of another method for encoding the second type object signal set provided by an embodiment of the present disclosure.
[0176] From the above, in the signal encoding and decoding method provided by an embodiment of the present disclosure, firstly, obtain a mixed-format audio signal, the mixed-format audio signal includes at least one format of a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal, and then determine an encoding mode of the audio signal of each format according to the signal characteristics of the audio signals of different formats, and then encode the audio signals of each format using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signals of each format, and write the encoded signal parameter information of the audio signals of each format into the encoded code stream and send it to the decoding side. From this, it can be seen that in the embodiment of the present disclosure, when encoding the mixed-format audio signal, the audio signals of different formats are reconstructed and analyzed according to the characteristics of the audio signals of different formats, and an adaptive encoding mode is determined for the audio signals of different formats, and then encoded using the corresponding encoding kernel to achieve better encoding efficiency.
[0177] FIG. 10 is a schematic flowchart of a signal encoding and decoding method provided by one embodiment of the present disclosure, the method being performed by a decoding side. As shown in FIG. 10, the signal encoding and decoding method may include the following steps 1001 to 1002.
[0178] In step 1001, an encoded codestream transmitted from an encoding side is received.
[0179] Here, in one embodiment of the present disclosure, the decoding side may be a UE or a base station.
[0180] In step 1002, the encoded codestream is decoded to obtain a mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal.
[0181] From the above, in the signal encoding and decoding method provided by an embodiment of the present disclosure, firstly, obtain a mixed-format audio signal, the mixed-format audio signal includes at least one format of a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal, and then determine an encoding mode of the audio signal of each format according to the signal characteristics of the audio signals of different formats, and then encode the audio signals of each format using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signals of each format, and write the encoded signal parameter information of the audio signals of each format into the encoded code stream and send it to the decoding side. From this, it can be seen that in the embodiment of the present disclosure, when encoding the mixed-format audio signal, the audio signals of different formats are reconstructed and analyzed according to the characteristics of the audio signals of different formats, and an adaptive encoding mode is determined for the audio signals of different formats, and then encoded using the corresponding encoding kernel to achieve better encoding efficiency.
[0182] FIG. 11a is a schematic flowchart of a signal encoding and decoding method provided by one embodiment of the present disclosure, the method is performed by a decoding side, and as shown in FIG. 11a, the signal encoding and decoding method may include the following steps 1101 to 1105.
[0183] In step 1101, an encoded codestream transmitted from an encoding side is received.
[0184] In step 1102, codestream analysis is performed on the encoded codestream to obtain classification side information parameters, side information parameters corresponding to the audio signals of each format, and post-encoded signal parameter information of the audio signals of each format.
[0185] Here, the classification side information parameter indicates a classification scheme for a set of object signals of a second type of the object-based audio signal, and the side information parameter indicates a corresponding coding mode for the audio signal of a corresponding format.
[0186] In step 1103, the encoded signal parameter information of the sound channel-based audio signal is decoded based on the side information parameters corresponding to the sound channel-based audio signal.
[0187] Here, in one embodiment of the present disclosure, decoding the encoded signal parameter information of the sound channel-based audio signal based on the side information parameters corresponding to the sound channel-based audio signal may include: determining an encoding mode corresponding to the sound channel-based audio signal based on the side information parameters corresponding to the sound channel-based audio signal; and decoding the encoded signal parameter information of the sound channel-based audio signal using a corresponding decoding mode based on the encoding mode corresponding to the sound channel-based audio signal.
[0188] At step 1104, the encoded signal parameter information of the scene-based audio signal is decoded based on the side information parameters corresponding to the scene-based audio signal.
[0189] In one embodiment of the present disclosure, decoding the encoded signal parameter information of the scene-based audio signal based on side information parameters corresponding to the scene-based audio signal may include determining an encoding mode corresponding to the scene-based audio signal based on the side information parameters corresponding to the scene-based audio signal, and decoding the encoded signal parameter information of the scene-based audio signal using a corresponding decoding mode based on the encoding mode corresponding to the scene-based audio signal.
[0190] In step 1105, the encoded signal parameter information of the object-based audio signal is decoded based on the classification side information parameters and the side information parameters corresponding to the object-based audio signal.
[0191] A specific method for implementing step 1105 will be described in the following embodiment.
[0192] Finally, based on the above description, FIG. 11b is a flowchart of a signal decoding method provided by one embodiment of the present disclosure.
[0193] From the above, in the signal encoding and decoding method provided by an embodiment of the present disclosure, firstly, obtain a mixed-format audio signal, the mixed-format audio signal includes at least one format of a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal, and then determine an encoding mode of the audio signal of each format according to the signal characteristics of the audio signals of different formats, and then encode the audio signals of each format using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signals of each format, and write the encoded signal parameter information of the audio signals of each format into the encoded code stream and send it to the decoding side. From this, it can be seen that in the embodiment of the present disclosure, when encoding the mixed-format audio signal, the audio signals of different formats are reconstructed and analyzed according to the characteristics of the audio signals of different formats, and an adaptive encoding mode is determined for the audio signals of different formats, and then encoded using the corresponding encoding kernel to achieve better encoding efficiency.
[0194] FIG. 12a is a schematic flowchart of a signal encoding and decoding method provided by one embodiment of the present disclosure, the method being performed by a decoding side, and as shown in FIG. 12a, the signal encoding and decoding method may include the following steps 1201 to 1205.
[0195] In step 1201, an encoded codestream transmitted from an encoding side is received.
[0196] In step 1202, codestream analysis is performed on the encoded codestream to obtain classification side information parameters, side information parameters corresponding to the audio signals of each format, and post-encoded signal parameter information of the audio signals of each format.
[0197] In step 1203, from the encoded signal parameter information of the object-based audio signal, encoded signal parameter information corresponding to a first type of object signal set and encoded signal parameter information corresponding to a second type of object signal set are determined.
[0198] Here, in one embodiment of the present disclosure, coded signal parameter information corresponding to a first type of object signal set and coded signal parameter information corresponding to a second type of object signal set can be determined from coded signal parameter information of the object-based audio signal based on side information parameters corresponding to the object-based audio signal.
[0199] At step 1204, the encoded signal parameter information corresponding to the first type of object signal set is decoded based on the side information parameters corresponding to the first type of object signal set.
[0200] Specifically, in one embodiment of the present disclosure, decoding the coded signal parameter information corresponding to the first type of object signal set based on the side information parameters corresponding to the first type of object signal set may include determining an encoding mode corresponding to the first type of object signal set based on the side information parameters corresponding to the first type of object signal set, and decoding the coded signal parameter information of the first type of object signal set using the corresponding decoding mode based on the encoding mode corresponding to the first type of object signal set.
[0201] In step 1205, the encoded signal parameter information corresponding to the set of object signals of the second type is decoded based on the classification side information parameters and the side information parameters corresponding to the set of object signals of the second type.
[0202] In one embodiment of the present disclosure, a method for decoding encoded signal parameter information corresponding to a second type of object signal set based on classification side information parameters and side information parameters corresponding to the second type of object signal set includes the following steps a and b.
[0203] In step a, a classification scheme for a set of second type object signals is determined based on the classification side information parameters.
[0204] Here, as can be seen from the description of the above embodiment, when the classification manner of the second type of object signal set is different, the corresponding encoding situation is also different. Specifically, in one embodiment of the present disclosure, when the classification manner of the second type of object signal set is a classification method based on the cross-correlation parameter value of signals, the encoding situation corresponding to the encoding side is to use the same encoding kernel to encode all the object signal sets in the corresponding encoding mode.
[0205] In another embodiment of the present disclosure, when the classification method of the second type of object signal set is a classification method based on frequency bandwidth range, the encoding situation corresponding to the encoding side is to use different encoding kernels to encode different object signal sets in corresponding encoding modes.
[0206] Therefore, in this step, it is first necessary to determine the classification manner of the second type of object signal set being encoded based on the classification side information parameters, so as to determine the encoding situation during encoding, and then decoding can be performed based on the encoding situation.
[0207] In step b, the encoded signal parameter information corresponding to each object signal subset in the second type of object signal set is decoded based on the classification scheme of the second type of object signal set and the side information parameters corresponding to the second type of object signal set.
[0208] Wherein, in one embodiment of the present disclosure, decoding the coded signal parameter information corresponding to each object signal subset in the second type object signal set based on the classification manner of the second type object signal set and the side information parameters corresponding to the second type object signal set includes: The method may include first determining an encoding status during encoding based on the classification scheme, then determining a corresponding decoding status based on the encoding status, and then decoding the coded signal parameter information corresponding to each object signal subset using a corresponding decoding mode based on the corresponding decoding status and based on an encoding mode corresponding to the coded signal parameter information corresponding to each object signal subset.
[0209] Specifically, in one embodiment of the present disclosure, if it is determined based on the classification side information parameters that the encoding situation during encoding is to encode all object signal subsets in corresponding encoding modes using the same encoding kernel, it is determined that the decoding situation of the decoding process is to decode the encoded signal parameter information corresponding to all object signal subsets using the same decoding kernel, where during decoding, specifically, based on the encoding modes corresponding to the encoded signal parameter information corresponding to each object signal subset, the encoded signal parameter information corresponding to the object signal subset is decoded using the corresponding decoding mode.
[0210] Also, in another embodiment of the present disclosure, if it is determined based on the classification side information parameters that the encoding situation during encoding is to encode different object signal subsets with different encoding kernels in corresponding encoding modes, it is determined that the decoding mode of the decoding process is to respectively decode the encoded signal parameter information corresponding to each object signal subset with different decoding kernels, where during decoding, specifically, based on the encoding modes corresponding to the encoded signal parameter information corresponding to each object signal subset, the encoded signal parameter information corresponding to each object signal subset is decoded using the corresponding decoding mode.
[0211] Finally, based on the above description, Figures 12b, 12c and 12d are respectively flowcharts of an object-based audio signal decoding method provided by an embodiment of the present disclosure, and Figures 12e and 12f are respectively flowcharts of a second type object signal set decoding method provided by an embodiment of the present disclosure.
[0212] From the above, in the signal encoding and decoding method provided by an embodiment of the present disclosure, firstly, obtain a mixed-format audio signal, the mixed-format audio signal includes at least one format of a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal, and then determine an encoding mode of the audio signal of each format according to the signal characteristics of the audio signals of different formats, and then encode the audio signals of each format using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signals of each format, and write the encoded signal parameter information of the audio signals of each format into the encoded code stream and send it to the decoding side. From this, it can be seen that in the embodiment of the present disclosure, when encoding the mixed-format audio signal, the audio signals of different formats are reconstructed and analyzed according to the characteristics of the audio signals of different formats, and an adaptive encoding mode is determined for the audio signals of different formats, and then encoded using the corresponding encoding kernel to achieve better encoding efficiency.
[0213] FIG. 13 is a schematic flowchart of a signal encoding and decoding method provided by one embodiment of the present disclosure, the method being performed by a decoding side. As shown in FIG. 13, the signal encoding and decoding method may include the following steps 1301 to 1303.
[0214] In step 1301, an encoded codestream transmitted from an encoding side is received.
[0215] In step 1302, the encoded codestream is decoded to obtain a mixed-format audio signal, the mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal.
[0216] In step 1303, the decoded object-based audio signal is post-processed.
[0217] From the above, in the signal encoding and decoding method provided by an embodiment of the present disclosure, firstly, obtain a mixed-format audio signal, the mixed-format audio signal includes at least one format of a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal, and then determine an encoding mode of the audio signal of each format according to the signal characteristics of the audio signals of different formats, and then encode the audio signals of each format using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signals of each format, and write the encoded signal parameter information of the audio signals of each format into the encoded code stream and send it to the decoding side. From this, it can be seen that in the embodiment of the present disclosure, when encoding the mixed-format audio signal, the audio signals of different formats are reconstructed and analyzed according to the characteristics of the audio signals of different formats, and an adaptive encoding mode is determined for the audio signals of different formats, and then encoded using the corresponding encoding kernel to achieve better encoding efficiency.
[0218] FIG. 14 is a schematic flowchart of another signal encoding and decoding method provided by one embodiment of the present disclosure, which is performed by the encoding side. As shown in FIG. 14, the signal encoding and decoding method may include the following steps 1401 to 1403.
[0219] In step 1401, a mixed-format audio signal is obtained, the mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal.
[0220] At step 1402, in response to a sound channel-based audio signal being included in the mixed format audio signal, an encoding mode for the sound channel-based audio signal is determined based on signal characteristics of the sound channel-based audio signal.
[0221] Wherein, in one embodiment of the present disclosure, determining an encoding mode of the sound channel-based audio signal based on signal characteristics of the sound channel-based audio signal includes: This may include obtaining a number of object signals included in the sound channel-based audio signal, and determining whether the number of object signals included in the sound channel-based audio signal is less than a first threshold (which may be, for example, 5).
[0222] Here, in one embodiment of the present disclosure, if the number of object signals included in the sound channel-based audio signal is smaller than a first threshold, it is determined that the encoding mode of the sound channel-based audio signal is at least one of the following measures 1 to 2.
[0223] In strategy 1, an object signal coding kernel is used to code each object signal in a sound channel-based audio signal.
[0224] In a second method, the method obtains input first command line control information, and uses an object signal encoding kernel to encode at least some object signals in the sound channel-based audio signal according to the first command line control information, where the first command line control information indicates object signals that need to be encoded among the object signals included in the sound channel-based audio signal, and the number of object signals that need to be encoded is equal to or greater than 1 and is smaller than a total number of object signals included in the sound channel-based audio signal.
[0225] As can be seen from this, in one embodiment of the present disclosure, when it is determined that the number of object signals contained in the sound channel-based audio signal is less than a first threshold, all or some of the object signals in the sound channel-based audio signal are encoded, thereby greatly reducing the encoding difficulty and improving the encoding efficiency.
[0226] In another embodiment of the present disclosure, when the number of object signals included in the sound channel-based audio signal is equal to or greater than a first threshold, the encoding mode of the sound channel-based audio signal is determined to be at least one of the following measures 3 to 5.
[0227] In the third method, the sound channel-based audio signal is converted into a first other format audio signal (which may be, for example, a scene-based audio signal or an object-based audio signal), and the number of sound channels of the first other format audio signal is equal to or less than the number of sound channels of the sound channel-based audio signal, and the coding kernel corresponding to the first other format audio signal is used to code the first other format audio signal. Exemplarily, in one embodiment of the present disclosure, when the sound channel-based audio signal is a 7.1.4 format sound channel-based audio signal (total number of sound channels is 13), the first other format audio signal may be, for example, a FOA (First Order Ambisonics) signal (total number of sound channels is 4), and by converting the 7.1.4 format sound channel-based audio signal into a FOA signal, the total number of sound channels of the signal that needs to be coded can be converted from 13 to 4, which can greatly reduce the coding difficulty and improve the coding efficiency.
[0228] In method 4, an input first command line control information is obtained, and an object signal encoding kernel is used to encode at least some object signals in the sound channel-based audio signal based on the first command line control information, wherein the first command line control information indicates object signals that need to be encoded among the object signals included in the sound channel-based audio signal, and the number of object signals that need to be encoded is greater than or equal to 1 and less than the total number of object signals included in the sound channel-based audio signal.
[0229] In method 5, the input second command line control information is obtained, and at least some sound channel signals in the sound channel-based audio signal are encoded using the object signal encoding kernel according to the second command line control information, where the second command line control information indicates sound channel signals that need to be encoded among the sound channel signals included in the sound channel-based audio signal, and the number of the sound channel signals that need to be encoded is greater than or equal to 1 and less than or equal to a total number of sound channel signals included in the sound channel-based audio signal.
[0230] As can be seen from this, in an embodiment of the present disclosure, when it is determined that the number of object signals contained in a sound channel-based audio signal is large, if the sound channel-based audio signal is directly encoded, the encoding difficulty is high. At this time, only some of the object signals in the sound channel-based audio signal may be encoded, and / or some of the sound channel signals in the sound channel-based audio signal may be encoded, and / or the sound channel-based audio signal may be converted into a signal with a smaller number of sound channels before encoding, so that the encoding difficulty can be greatly reduced and the encoding efficiency can be optimized.
[0231] In step 1403, the encoding mode of the sound channel-based audio signal is used to encode the sound channel-based audio signal to obtain encoded signal parameter information of the sound channel-based audio signal, and the encoded signal parameter information of the sound channel-based audio signal is written into an encoded code stream and sent to a decoding side.
[0232] Here, for an introduction to step 1403, please refer to the description in the above embodiment, and a detailed description will be omitted in the embodiment of the present disclosure.
[0233] From the above, in the signal encoding and decoding method provided by an embodiment of the present disclosure, firstly, obtain a mixed-format audio signal, the mixed-format audio signal includes at least one format of a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal, and then determine an encoding mode of the audio signal of each format according to the signal characteristics of the audio signals of different formats, and then encode the audio signals of each format using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signals of each format, and write the encoded signal parameter information of the audio signals of each format into the encoded code stream and send it to the decoding side. From this, it can be seen that in the embodiment of the present disclosure, when encoding the mixed-format audio signal, the audio signals of different formats are reconstructed and analyzed according to the characteristics of the audio signals of different formats, and an adaptive encoding mode is determined for the audio signals of different formats, and then encoded using the corresponding encoding kernel to achieve better encoding efficiency.
[0234] FIG. 15 is a schematic flowchart of another signal encoding and decoding method provided by one embodiment of the present disclosure, which is performed by the encoding side. As shown in FIG. 15, the signal encoding and decoding method may include the following steps 1501 to 1503.
[0235] In step 1501, Sound Channels A mixed-format audio signal is obtained, the mixed-format audio signal including at least one of a format of a scene-based audio signal, an object-based audio signal, and a scene-based audio signal.
[0236] At step 1502, in response to a scene-based audio signal being included in the mixed-format audio signal, an encoding mode for the scene-based audio signal is determined based on signal characteristics of the scene-based audio signal.
[0237] In one embodiment of the present disclosure, the step of determining an encoding mode of the scene-based audio signal based on signal features of the scene-based audio signal includes: The method may include obtaining the number of object signals included in the scene-based audio signal, and determining whether the number of object signals included in the scene-based audio signal is less than a second threshold (which may be, for example, 5).
[0238] Here, in one embodiment of the present disclosure, when the number of object signals contained in the scene-based audio signal is smaller than a second threshold, the encoding mode of the scene-based audio signal is determined to be at least one of the following measures a to b.
[0239] In strategy a, each object signal of a scene-based audio signal is coded using an object signal coding kernel.
[0240] In method b, input fourth command line control information is obtained, and at least some of the object signals in the scene-based audio signal are encoded using an object signal encoding kernel based on the fourth command line control information, where the fourth command line control information indicates the object signals that need to be encoded among the object signals included in the scene-based audio signal, and the number of object signals that need to be encoded is greater than or equal to 1 and less than or equal to the total number of object signals included in the scene-based audio signal.
[0241] As can be seen from this, in one embodiment of the present disclosure, when it is determined that the number of object signals contained in the scene-based audio signal is less than a second threshold, all or some of the object signals of the scene-based audio signal are encoded, thereby significantly reducing the difficulty of encoding and improving the encoding efficiency.
[0242] In another embodiment of the present disclosure, when the number of object signals contained in the scene-based audio signal is greater than or equal to a second threshold, the encoding mode of the scene-based audio signal is determined to be at least one of the following measures c to d.
[0243] In method c, the scene-based audio signal is converted into an audio signal of a second other format, the number of sound channels of the audio signal of the second other format being less than or equal to the number of sound channels of the scene-based audio signal, and the audio signal of the second other format is encoded using a scene signal encoding kernel.
[0244] In method d, a low-order transformation is performed on the scene-based audio signal to transform the scene-based audio signal into a low-order scene-based audio signal whose order is lower than the current order of the scene-based audio signal, and the low-order scene-based audio signal is encoded using a scene signal encoding kernel. Note that, in one embodiment of the present disclosure, when a low-order transformation is performed on the scene-based audio signal, the scene-based audio signal may be low-order transformed into a signal of another format. For example, a 3-order scene-based audio signal can be transformed into a low-order 5.0 format sound channel-based audio signal, and the total number of sound channels of the signal that needs to be encoded is changed from 16 ((3+1)*(3+1)) to 5, which greatly reduces the difficulty of encoding and improves the encoding efficiency.
[0245] As can be seen from this, in an embodiment of the present disclosure, when it is determined that the number of object signals contained in a scene-based audio signal is large, if the scene-based audio signal is directly encoded, the encoding difficulty is high. At this time, only the scene-based audio signal may be converted into a signal with a small number of sound channels and then encoded, and / or the scene-based audio signal may be converted into a low-order signal and then encoded, thereby greatly reducing the encoding difficulty and improving the encoding efficiency.
[0246] In step 1503, the scene-based audio signal is encoded using an encoding mode of the scene-based audio signal to obtain encoded signal parameter information of the scene-based audio signal, and the encoded signal parameter information of the scene-based audio signal is written into an encoded code stream and transmitted to a decoding side.
[0247] Here, for an explanation of step 1503, please refer to the explanation of the above embodiment, and a detailed explanation will be omitted in the embodiment of the present disclosure.
[0248] As described above, a signal encoding and decoding method provided by an embodiment of the present disclosure includes the steps of: Sound Channels In the embodiment of the present disclosure, the audio signal of the mixed format is obtained, which includes at least one format of a scene-based audio signal, an object-based audio signal, and a scene-based audio signal, and the encoding mode of the audio signal of each format is determined according to the signal characteristics of the audio signals of different formats, and then the audio signal of each format is encoded using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signal of each format, and the encoded signal parameter information of the audio signal of each format is written into the encoded code stream and sent to the decoding side. It can be seen from this that, in the embodiment of the present disclosure, when encoding the mixed format audio signal, the audio signals of different formats are reconstructed and analyzed according to the characteristics of the audio signals of different formats, and an adaptive encoding mode is determined for the audio signals of different formats, and then the corresponding encoding kernel is used to encode them, so as to achieve better encoding efficiency.
[0249] FIG. 16 is a schematic flowchart of a signal encoding and decoding method provided by one embodiment of the present disclosure, the method being performed by a decoding side. As shown in FIG. 16, the signal encoding and decoding method may include the following steps 1601 to 1603.
[0250] In step 1601, an encoded codestream transmitted from an encoding side is received.
[0251] In step 1602, codestream analysis is performed on the encoded codestream to obtain classification side information parameters, side information parameters corresponding to the audio signals of each format, and post-encoded signal parameter information of the audio signals of each format.
[0252] At step 1603, the encoded signal parameter information of the sound channel-based audio signal is decoded based on the side information parameters corresponding to the sound channel-based audio signal.
[0253] As described above, a signal encoding and decoding method provided by an embodiment of the present disclosure includes the steps of: Sound Channels In the embodiment of the present disclosure, the audio signal of the mixed format is obtained, which includes at least one format of a scene-based audio signal, an object-based audio signal, and a scene-based audio signal, and the encoding mode of the audio signal of each format is determined according to the signal characteristics of the audio signals of different formats, and then the audio signal of each format is encoded using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signal of each format, and the encoded signal parameter information of the audio signal of each format is written into the encoded code stream and sent to the decoding side. It can be seen from this that, in the embodiment of the present disclosure, when encoding the mixed format audio signal, the audio signals of different formats are reconstructed and analyzed according to the characteristics of the audio signals of different formats, and an adaptive encoding mode is determined for the audio signals of different formats, and then the corresponding encoding kernel is used to encode them, so as to achieve better encoding efficiency.
[0254] FIG. 17 is a schematic flowchart of a signal encoding and decoding method provided by one embodiment of the present disclosure, the method being performed by a decoding side. As shown in FIG. 17, the signal encoding and decoding method may include the following steps 1701 to 1703.
[0255] In step 1701, an encoded codestream transmitted from an encoding side is received.
[0256] In step 1702, codestream analysis is performed on the encoded codestream to obtain classification side information parameters, side information parameters corresponding to the audio signals of each format, and post-encoded signal parameter information of the audio signals of each format.
[0257] In step 1703, the encoded signal parameter information of the scene-based audio signal is decoded based on the side information parameters corresponding to the scene-based audio signal.
[0258] As described above, a signal encoding and decoding method provided by an embodiment of the present disclosure includes the steps of: Sound Channels In the embodiment of the present disclosure, the audio signal of the mixed format is obtained, which includes at least one format of a scene-based audio signal, an object-based audio signal, and a scene-based audio signal, and the encoding mode of the audio signal of each format is determined according to the signal characteristics of the audio signals of different formats, and then the audio signal of each format is encoded using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signal of each format, and the encoded signal parameter information of the audio signal of each format is written into the encoded code stream and sent to the decoding side. It can be seen from this that, in the embodiment of the present disclosure, when encoding the mixed format audio signal, the audio signals of different formats are reconstructed and analyzed according to the characteristics of the audio signals of different formats, and an adaptive encoding mode is determined for the audio signals of different formats, and then the corresponding encoding kernel is used to encode them, so as to achieve better encoding efficiency.
[0259] FIG. 18 is a structural schematic diagram of an apparatus for a signal encoding and decoding method provided by an embodiment of the present disclosure, which is applied to the encoding side. As shown in FIG. 18, the apparatus 1800 includes: An acquisition module 1801 for acquiring a mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal; A decision module 1802 for deciding an encoding mode of the audio signal of each format based on signal characteristics of the audio signal of different formats; and an encoding module 1803 for encoding an audio signal of each format using an encoding mode of the audio signal of each format to obtain encoded signal parameter information of the audio signal of each format, and writing the encoded signal parameter information of the audio signal of each format into an encoded code stream to transmit to a decoding side.
[0260] From the above, in the signal encoding and decoding device provided by an embodiment of the present disclosure, firstly, a mixed-format audio signal including at least one format of a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal is obtained, and then, according to the signal characteristics of the audio signals of different formats, an encoding mode of the audio signal of each format is determined, and then, the audio signal of each format is encoded using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signal of each format, and the encoded signal parameter information of the audio signal of each format is written into the encoded code stream and transmitted to the decoding side. From this, it can be seen that, in the embodiment of the present disclosure, when encoding the mixed-format audio signal, according to the characteristics of the audio signals of different formats, the audio signals of different formats are reconstructed and analyzed, and an adaptive encoding mode is determined for the audio signals of different formats, and then the corresponding encoding kernel is used for encoding, so as to achieve better encoding efficiency.
[0261] Optionally, in one embodiment of the present disclosure, the decision module further comprises: determining an encoding mode for the sound channel-based audio signal based on signal characteristics of the sound channel-based audio signal; determining an encoding mode for the object-based audio signal based on signal characteristics of the object-based audio signal; An encoding mode for the scene-based audio signal is determined based on signal characteristics of the scene-based audio signal.
[0262] Optionally, in one embodiment of the present disclosure, the decision module further comprises: obtaining a number of object signals included in the sound channel-based audio signal; determining whether a number of object signals included in the sound channel-based audio signal is less than a first threshold; if the number of object signals contained in the sound channel-based audio signal is less than a first threshold, the coding mode of the sound channel-based audio signal is encoding each object signal in the sound channel-based audio signal using an object signal coding kernel; Obtain input first command line control information, and use an object signal encoding kernel to encode at least some object signals in the sound channel-based audio signal based on the first command line control information, wherein it is determined that at least one of the following is true: the first command line control information indicates object signals that need to be encoded among the object signals included in the sound channel-based audio signal, and the number of object signals that need to be encoded is greater than or equal to one and is less than the total number of object signals included in the sound channel-based audio signal.
[0263] Optionally, in one embodiment of the present disclosure, the decision module further comprises: obtaining a number of object signals included in the sound channel-based audio signal; determining whether a number of object signals included in the sound channel-based audio signal is less than a first threshold; if the number of object signals included in the sound channel based audio signal is equal to or greater than a first threshold, determining an encoding mode of the sound channel based audio signal: converting the sound channel-based audio signal into an audio signal of a first other format, the audio signal having a number of sound channels being less than the number of sound channels of the sound channel-based audio signal, and encoding the audio signal of the first other format using an encoding kernel corresponding to the audio signal of the first other format; obtaining input first command line control information, and encoding at least some object signals in the sound channel-based audio signal based on the first command line control information using an object signal encoding kernel, wherein the first command line control information indicates object signals that need to be encoded among the object signals included in the sound channel-based audio signal, and the number of object signals that need to be encoded is one or more and is smaller than a total number of object signals included in the sound channel-based audio signal; Obtain input second command line control information, and use an object signal encoding kernel to encode at least some of the sound channel signals in the sound channel-based audio signal based on the second command line control information, wherein it is determined that the second command line control information indicates sound channel signals that need to be encoded among the sound channel signals included in the sound channel-based audio signal, and the number of sound channel signals that need to be encoded is one or more and is less than the total number of sound channel signals included in the sound channel-based audio signal.
[0264] Optionally, in one embodiment of the present disclosure, the encoding module further comprises: The sound channel-based audio signal is encoded using an encoding mode of the sound channel-based audio signal.
[0265] Optionally, in one embodiment of the present disclosure, the decision module further comprises: performing a signal feature analysis on the object-based audio signal to obtain an analysis result; classifying the object-based audio signal to obtain a first type of object signal set and a second type of object signal set, each of the first type of object signal set and the second type of object signal set including at least one object-based audio signal; determining a coding mode corresponding to the first type of object signal set; classifying the second type object signal set based on the analysis result to obtain at least one object signal subset, and determining an encoding mode corresponding to each object signal subset based on the classification result, wherein the object signal subset includes at least one object-based audio signal.
[0266] Optionally, in one embodiment of the present disclosure, the decision module further comprises: Among the object-based audio signals, signals that do not require individual manipulation processing are classified into a first type of object signal set, and the remaining signals are classified into a second type of object signal set.
[0267] Optionally, in one embodiment of the present disclosure, the decision module further comprises: determining that an encoding mode corresponding to the first type of object signal set is performing a first pre-rendering process on object-based audio signals in the first type of object signal set, and encoding the first pre-rendered signals using a multi-channel encoding kernel; Here, the first pre-rendering process includes performing a signal format conversion process on the object-based audio signal to convert the object-based audio signal into a sound channel-based audio signal.
[0268] Optionally, in one embodiment of the present disclosure, the decision module further comprises: Among the object-based audio signals, signals belonging to background sounds are classified into a first type of object signal set, and the remaining signals are classified into a second type of object signal set.
[0269] Optionally, in one embodiment of the present disclosure, the decision module further comprises: determining that an encoding mode corresponding to the first type of object signal set is performing a second pre-rendering process on object-based audio signals in the first type of object signal set, and encoding the second pre-rendered signals using a Higher Order Ambisonics (HOA) encoding kernel; Here, the second pre-rendering process includes performing a signal format conversion process on the object-based audio signal to convert the object-based audio signal into a scene-based audio signal.
[0270] Optionally, in one embodiment of the present disclosure, the decision module further comprises: Among the object-based audio signals, signals that do not require individual manipulation processing are classified into a first object signal subset, among the object-based audio signals, signals that belong to background sounds are classified into a second object signal subset, and the remaining signals are classified into a second type of object signal set.
[0271] Optionally, in one embodiment of the present disclosure, the decision module further comprises: determining that an encoding mode corresponding to a first object signal subset in the first type object signal set is performing a first pre-rendering operation on object-based audio signals in the first object signal subset and encoding the first pre-rendered signals using a multi-channel encoding kernel, the first pre-rendering operation including performing a signal format conversion operation on the object-based audio signals to convert the object-based audio signals into sound channel-based audio signals; Determine that the encoding mode corresponding to a second object signal subset in the first type object signal set is to perform a second pre-rendering process on the object-based audio signals in the second object signal subset and encode the second pre-rendered signals using an HOA encoding kernel, wherein the second pre-rendering process includes performing a signal format conversion process on the object-based audio signals to convert the object-based audio signals into scene-based audio signals.
[0272] Optionally, in one embodiment of the present disclosure, the decision module further comprises: performing a high-pass filtering process on the object-based audio signal; A correlation analysis is performed on the high-pass filtered signal to determine cross-correlation parameter values between each of the object-based audio signals.
[0273] Optionally, in one embodiment of the present disclosure, the decision module further comprises: Setting a normalized correlation interval based on the correlation; Based on the cross-correlation parameter values of the object-based audio signals and the normalized correlation degree interval, classify the second type object signal set to obtain at least one object signal subset, and determine a corresponding encoding mode based on the correlation degree corresponding to the at least one object signal subset.
[0274] Optionally, in one embodiment of the present disclosure, 、 before The encoding modes corresponding to the object signal subsets include independent encoding modes or joint encoding modes. nothing.
[0275] Selectably, in one embodiment of the present disclosure, the independent coding mode corresponds to a time domain processing manner or a frequency domain processing manner; Wherein, if the object signals in the object signal subset are speech signals or similar speech signals, the independent coding mode adopts a time domain processing manner; If the object signals in the object signal subset are audio signals of other formats than speech or similar speech signals, the independent coding mode employs a frequency domain processing scheme.
[0276] Optionally, in one embodiment of the present disclosure, the encoding module further comprises: encoding the object-based audio signal using a coding mode of the object-based audio signal; encoding the object-based audio signal using a coding mode of the object-based audio signal, encoding signals in the first type of object signal set using a coding mode corresponding to the first type of object signal set; pre-processing subsets of object signals in the set of object signals of the second type, and encoding all pre-processed subsets of object signals in the set of object signals of the second type in a corresponding encoding mode using a same object signal encoding kernel.
[0277] Optionally, in one embodiment of the present disclosure, the decision module further comprises: The frequency bandwidth range of the object signal is analyzed.
[0278] Optionally, in one embodiment of the present disclosure, the decision module further comprises: determining bandwidth intervals corresponding to different frequency bandwidths; Based on a frequency bandwidth range of the object-based audio signal and bandwidth intervals corresponding to different frequency bandwidths, classify the second type object signal set to obtain at least one object signal subset, and determine a corresponding encoding mode based on a frequency bandwidth corresponding to the at least one object signal subset.
[0279] Optionally, in one embodiment of the present disclosure, the decision module further comprises: obtaining input third command line control information indicating an encoded frequency bandwidth range corresponding to the object-based audio signal; The third command line control information and the analysis result are integrated to classify the second type object signal set to obtain at least one object signal subset, and an encoding mode corresponding to each object signal subset is determined based on the classification result.
[0280] Optionally, in one embodiment of the present disclosure, the encoding module further comprises: encoding the object-based audio signal using a coding mode of the object-based audio signal; encoding the object-based audio signal using a coding mode of the object-based audio signal, encoding signals in the first type of object signal set using a coding mode corresponding to the first type of object signal set; pre-processing subsets of object signals in the set of object signals of the second type, and encoding the different pre-processed subsets of object signals in corresponding encoding modes using different object signal encoding kernels.
[0281] Optionally, in one embodiment of the present disclosure, the decision module further comprises: obtaining a number of object signals included in the scene-based audio signal; determining whether a number of object signals included in the scene-based audio signal is less than a second threshold; If the number of object signals included in the scene-based audio signal is smaller than a second threshold, the coding mode of the scene-based audio signal is encoding each object signal of the scene-based audio signal using an object signal coding kernel; Obtain input fourth command line control information, and use an object signal encoding kernel to encode at least some of the object signals in the scene-based audio signal based on the fourth command line control information, wherein it is determined that the fourth command line control information indicates object signals that need to be encoded among the object signals included in the scene-based audio signal, and the number of object signals that need to be encoded is one or more and is less than the total number of object signals included in the scene-based audio signal.
[0282] Optionally, in one embodiment of the present disclosure, the decision module further comprises: obtaining a number of object signals included in the scene-based audio signal; determining whether a number of object signals included in the scene-based audio signal is less than a second threshold; If the number of object signals included in the scene-based audio signal is equal to or greater than a second threshold, the coding mode of the scene-based audio signal is converting the scene-based audio signal into an audio signal of a second other format, the number of sound channels of which is less than the number of sound channels of the scene-based audio signal, and encoding the audio signal of the second other format using a scene signal encoding kernel; performing a low-order transform on the scene-based audio signal to convert the scene-based audio signal into a low-order scene-based audio signal whose order is lower than a current order of the scene-based audio signal, and encoding the low-order scene-based audio signal using a scene signal encoding kernel.
[0283] Optionally, in one embodiment of the present disclosure, the encoding module further comprises: The scene-based audio signal is encoded using an encoding mode of the scene-based audio signal.
[0284] Optionally, in one embodiment of the present disclosure, the encoding module further comprises: determining a classification side information parameter indicative of a classification scheme for the set of second type object signals; determining side information parameters corresponding to each format of the audio signal, the side information parameters indicating a coding mode corresponding to the audio signal of the corresponding format; The classification side information parameters, side information parameters corresponding to the audio signals of each format, and signal parameter information after the audio signals of each format are coded to obtain a coded code stream, and the coded code stream is transmitted to the decoding side.
[0285] FIG. 19 is a structural schematic diagram of an apparatus for the signal encoding and decoding method provided by one embodiment of the present disclosure, which is applied to the decoding side. As shown in FIG. 19, the apparatus 1900 includes: a receiving module 1901 for receiving an encoded code stream sent from an encoding side; and a decoding module 1902 for decoding the encoded code stream to obtain a mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal.
[0286] From the above, in the signal encoding and decoding device provided by an embodiment of the present disclosure, firstly, a mixed-format audio signal including at least one format of a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal is obtained, and then, according to the signal characteristics of the audio signals of different formats, an encoding mode of the audio signal of each format is determined, and then, the audio signal of each format is encoded using the encoding mode of the audio signal of each format to obtain the encoded signal parameter information of the audio signal of each format, and the encoded signal parameter information of the audio signal of each format is written into the encoded code stream and transmitted to the decoding side. From this, it can be seen that, in the embodiment of the present disclosure, when encoding the mixed-format audio signal, according to the characteristics of the audio signals of different formats, the audio signals of different formats are reconstructed and analyzed, and an adaptive encoding mode is determined for the audio signals of different formats, and then the corresponding encoding kernel is used for encoding, so as to achieve better encoding efficiency.
[0287] Optionally, in one embodiment of the present disclosure, the device further comprises: Performing codestream analysis on the encoded codestream to obtain classification side information parameters, side information parameters corresponding to the audio signals of each format, and signal parameter information after encoding of the audio signals of each format; Here, the classification side information parameter indicates a classification scheme for a set of object signals of a second type of the object-based audio signal, and the side information parameter indicates a corresponding coding mode for an audio signal of a corresponding format.
[0288] Optionally, in one embodiment of the present disclosure, the decoding module further comprises: decoding the encoded signal parameter information of the sound channel based audio signal based on side information parameters corresponding to the sound channel based audio signal; decoding the encoded signal parameter information of the object-based audio signal based on the classification side information parameters and on side information parameters corresponding to the object-based audio signal; The encoded signal parameter information of the scene-based audio signal is decoded based on side information parameters corresponding to the scene-based audio signal.
[0289] Optionally, in one embodiment of the present disclosure, the decoding module further comprises: determining from the encoded signal parameter information of the object-based audio signal encoded signal parameter information corresponding to a first type of object signal set and encoded signal parameter information corresponding to a second type of object signal set; Decoding the encoded signal parameter information corresponding to the first type of object signal set based on side information parameters corresponding to the first type of object signal set; Decoding the encoded signal parameter information corresponding to a set of object signals of a second type based on the classification side information parameters and side information parameters corresponding to the set of object signals of the second type.
[0290] Optionally, in one embodiment of the present disclosure, the decoding module further comprises: determining a classification scheme for the set of second type object signals based on the classification side information parameters; The encoded signal parameter information corresponding to the second type of object signal set is decoded based on the classification manner of the second type of object signal set and the side information parameters corresponding to the second type of object signal set.
[0291] Optionally, in one embodiment of the present disclosure, the classification side information parameter indicates that the classification manner of the set of second type object signals is to classify based on a cross-correlation parameter value, and the decoding module further comprises: Using the same object signal decoding kernel, the encoded signal parameter information of all signals in the second type object signal set is decoded based on the classification scheme of the second type object signal set and the side information parameters corresponding to the second type object signal set.
[0292] Optionally, in one embodiment of the present disclosure, the classification side information parameter indicates that the classification manner of the second type object signal set is to classify based on a frequency bandwidth range, and the decoding module further comprises: Using a different object signal decoding kernel, the encoded signal parameter information of the different signals in the second type of object signal set is decoded based on the classification scheme of the second type of object signal set and the side information parameters corresponding to the second type of object signal set.
[0293] Optionally, in one embodiment of the present disclosure, the device further comprises: Post-processing the decoded object-based audio signal.
[0294] Optionally, in one embodiment of the present disclosure, the decoding module further comprises: determining an encoding mode corresponding to the sound channel based audio signal based on side information parameters corresponding to the sound channel based audio signal; Based on an encoding mode corresponding to the sound channel-based audio signal, the encoded signal parameter information of the sound channel-based audio signal is decoded using a corresponding decoding mode.
[0295] Optionally, in one embodiment of the present disclosure, the decoding module further comprises: determining an encoding mode corresponding to the scene-based audio signal based on side information parameters corresponding to the scene-based audio signal; Based on an encoding mode corresponding to the scene-based audio signal, the encoded signal parameter information of the scene-based audio signal is decoded using a corresponding decoding mode.
[0296] 20 is a block diagram of a user equipment UE 2000 provided by an embodiment of the present disclosure. For example, the UE 2000 may be a mobile phone, a computer, a digital broadcast terminal device, a message sending and receiving device, a game console, a tablet terminal, a medical device, a fitness device, a personal digital assistant, etc.
[0297] Referring to FIG. 20 , the UE 2000 may include one or more of a processing component 2002, a memory 2004, a power component 2006, a multimedia component 2008, an audio component 2010, an input / output (I / O) interface 2012, a sensor component 2013, and a communication component 2016.
[0298] The processing component 2002 typically controls the overall operation of the UE 2000, such as operations related to display, phone calls, data communication, camera operation, and recording operation. The processing component 2002 may include one or more processors 2020 for executing instructions to complete all or some steps of the above method. The processing component 2002 may also include one or more modules to facilitate interaction with other components. For example, the processing component 2002 may include a multimedia module to facilitate interaction between the processing component 2002 and the multimedia component 2008.
[0299] The memory 2004 is configured to store various types of data, such as instructions for any application programs or methods operating on the UE 2000, contact data, phone book data, messages, photos, videos, etc., to support operation on the UE 2000. The memory 2004 may be implemented by any type of volatile or non-volatile storage device, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, optical disk, or any combination thereof.
[0300] The power component 2006 provides power for the various components of the UE 2000. The power component 2006 can include a power management system, at least one power source, and other components related to generating, managing, and allocating power for the UE 2000.
[0301] The multimedia component 2008 includes a screen that provides an output interface between the UE 2000 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). When the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensors detect the boundaries of a touch or slide action as well as the duration and pressure associated with the touch or slide action. In some embodiments, the multimedia component 2008 includes one front camera and / or a back camera. When the UE 2000 is in an operational mode, such as a photo mode or a video mode, the front camera and / or the back camera can receive external multimedia data. Each front camera and back camera may be a fixed optical lens system or may have a focal length and optical zoom capability.
[0302] The audio component 2010 is configured to output and / or input audio signals. For example, the audio component 2010 includes one microphone (MIC) configured to receive external audio signals when the UE 2000 is in an operation mode such as a call mode, a record mode, and a voice recognition mode. The received audio signals can be further stored in the memory 2004 or transmitted via the communication component 2016. In some embodiments, the audio component 2010 further includes one speaker for outputting audio signals.
[0303] The I / O interface 2012 provides an interface between the processing component 2002 and a peripheral interface module, which may be a keyboard, a click wheel, buttons, etc. These buttons may include, but are not limited to, a home button, volume buttons, a start button, and a lock button.
[0304] The sensor component 2013 includes at least one or more sensors to provide various aspects of status assessment for the UE 2000. For example, the sensor component 2013 can detect the on / off state of the UE 2000, the relative positioning of components, e.g., the display and keypad of the UE 2000, and the sensor component 2013 can also detect position changes of the UE 2000 or components of the UE 2000, the presence or absence of user contact with the UE 2000, the orientation or acceleration / deceleration of the UE 2000, and temperature changes of the UE 2000. The sensor component 2013 can also include a proximity sensor configured to detect the presence of a nearby object in the absence of any physical contact. The sensor component 2013 can further include an optical sensor, such as a CMOS or CCD image sensor for use in imaging applications. In some embodiments, the sensor component 2013 may also include an acceleration sensor, a gyro sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0305] The communication component 2016 is configured to facilitate wired or wireless communication between the UE 2000 and other devices. The UE 2000 can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 2016 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 2016 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0306] In an exemplary embodiment, the UE 2000 may be implemented by an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components, one or more applications to perform the above methods.
[0307] FIG. 21 is a block diagram of a network side device 2100 provided by an embodiment of the present disclosure. For example, the network side device 2100 can be provided as a base station. Referring to FIG. 21, the network side device 2100 further includes a processing component 2111 including at least one processor, and memory resources including a memory 2132, which are used to store instructions executable by the processing component 2122, such as application programs. The application programs stored in the memory 2132 may include one or more modules, each corresponding to a set of instructions. The processing component 2115 is also configured to execute instructions, thereby performing any method applied to the base station of the above methods, such as the method shown in FIG. 1.
[0308] The network side device 2100 may further include a power component 2126 configured to perform power management of the network side device 2100, a wired or wireless network interface 2150 configured to connect the network side device 2100 to a network, and an input / output (I / O) interface 2158. The network side device 2100 may operate an operating system stored in a memory 2132, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.
[0309] In the above embodiments provided by the present disclosure, the method provided by the embodiments of the present disclosure is introduced from the perspective of a network side device and a UE. To realize each function of the method provided by the embodiments of the present disclosure, the network side device and the UE may include a hardware structure and a software module, and each of the above functions is realized by a hardware structure, a software module, or a hardware structure plus a software module. A specific function of each of the above functions can be implemented by a hardware structure, a software module, or a hardware structure plus a software module.
[0310] In the above embodiments provided by the present disclosure, the method provided by the embodiments of the present disclosure is introduced from the perspective of a network side device and a UE. To realize each function of the method provided by the embodiments of the present disclosure, the network side device and the UE may include a hardware structure and a software module, and each of the above functions is realized by a hardware structure, a software module, or a hardware structure plus a software module. A specific function of each of the above functions can be implemented by a hardware structure, a software module, or a hardware structure plus a software module.
[0311] An embodiment of the present disclosure provides a communication device. The communication device may include a transceiver module and a processing module. The transceiver module may include a transmitting module and / or a receiving module, where the transmitting module is used to realize a transmitting function, the receiving module is used to realize a receiving function, and the transceiver module can realize the transmitting function and / or the receiving function.
[0312] The communication device may be a terminal device (e.g., the terminal device in the method embodiments described above), a device within a terminal device, or a device usable in combination with a terminal device, or the communication device may be a network device, a device within a network device, or a device usable in combination with a network device.
[0313] The embodiment of the present disclosure provides another communication device. The communication device may be a network device, may be a terminal device (terminal device in the above-mentioned method embodiment), may be a chip, chip system, or processor, etc. that supports the network device to realize the above-mentioned method, may be a chip, chip system, or processor, etc. that supports the terminal device to realize the above-mentioned method. The device may be used to realize the method described in the above-mentioned method embodiment, specifically, please refer to the description in the above-mentioned method embodiment.
[0314] The communication device may include one or more processors. The processor may be a general-purpose processor or a dedicated processor, etc. For example, it may be a baseband processor or a central processor. The baseband processor may be used to process communication protocols and communication data, and the central processor may be used to control the communication device (e.g., baseband, baseband chip, terminal device, terminal device chip, DU or CU, etc.), execute computer programs, and process data of computer programs.
[0315] Optionally, the communication device may further include one or more memories capable of storing computer programs, and the processor may execute the computer programs to cause the communication device to perform the methods described in the above method embodiments. Optionally, the memory may store data. The communication device and the memory may be provided separately or may be integrated together.
[0316] Optionally, the communication device may further include a transceiver and an antenna. The transceiver may be called a transceiver unit, transceiver, or transceiver circuit, etc., and is used to realize a transmission and reception function. The transceiver may include a receiver and a transmitter, and the receiver may be called a receiving device or receiving circuit, etc., and is used to realize a reception function, and the transmitter may be called a transmitting device or transmitting circuit, etc., and is used to realize a transmission function.
[0317] Optionally, the communication device may include one or more interface circuits used to receive and transmit code instructions to the processor, which executes the code instructions to cause the communication device to perform the methods described in the method embodiments above.
[0318] When the communication device is a terminal device (eg, the terminal device in the method embodiments described above), the processor is used to perform the method according to any of the methods of FIGS.
[0319] When the communication device is a network device, the transceiver is used to perform the method described in any of FIGS.
[0320] In one implementation, the processor may include a transceiver for implementing the receiving and transmitting functions. For example, the transceiver may be a transceiver circuit, or an interface, or an interface circuit. The transceiver circuit, interface, or interface circuit for implementing the receiving and transmitting functions may be separate or integrated together. The transceiver circuit, interface, or interface circuit may be used to read and write code / data, or the transceiver circuit, interface, or interface circuit may be used to transmit or convey signals.
[0321] In one implementation, the processor may have a computer program stored therein, the computer program being executed by the processor to enable the communication device to perform the method described in any of the method embodiments above. The computer program may be embedded in the processor, in which case the processor may be implemented by hardware.
[0322] In one implementation, the communication device may include a circuit, which may implement the functions of transmitting or receiving or communicating in the method embodiments described above. The processor and transceiver described in this disclosure may be integrated into an integrated circuit (IC), an analog IC, a radio frequency integrated circuit (RFIC), a mixed signal IC, an application specific integrated circuit (ASIC), a printed circuit board (PCB), an electronic device, etc. The processor and transceiver may be fabricated using a variety of IC process technologies, such as complementary metal oxide semiconductor (CMOS), n-type metal oxide semiconductor (nMOS), p-type metal oxide semiconductor (PMOS), bipolar junction transistor (BJT), bipolar CMOS (BiCMOS), silicon germanium (SiGe), gallium arsenide (Gas), etc.
[0323] The communication device in the above embodiment description may be a network device or a terminal device (terminal device in the above method embodiment), but the scope of the communication device described in this disclosure is not limited thereto, and the structure of the communication device may not be limited. The communication device may be an independent device or a part of a larger device. For example, the communication device may be as follows: (1) An independent integrated circuit IC or chip, or a chip system or subsystem; (2) A set having one or more ICs, which may optionally include a memory component for storing data, computer programs; (3) ASICs, such as modems, (4) A module that can be embedded into other devices; (5) Receivers, terminal devices, intelligent terminal devices, cellular phones, wireless devices, handhelds, mobile units, vehicle-mounted devices, network devices, cloud devices, artificial intelligence devices, etc. (6)Others.
[0324] When the communication device is a chip or a chip system, the chip includes a processor and an interface, where the number of processors may be one or more, and the number of interfaces may be more than one.
[0325] Optionally, the chip further includes a memory, which is used to store necessary computer programs and data.
[0326] As can be understood by those skilled in the art, the various illustrative logical blocks and steps enumerated in the embodiments of the present disclosure can be realized by electronic hardware, computer software, or a combination of both. Whether such functions are realized by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art can realize the functions using various methods for each specific application, but such realization should not be understood as going beyond the scope of protection of the embodiments of the present disclosure.
[0327] An embodiment of the present disclosure further provides a system for determining a sidelink time length, the system including a communication device as a terminal device in the above-mentioned embodiment (a first terminal device in the above-mentioned method embodiment) and a communication device as a network device, or the system including a communication device as a terminal device in the above-mentioned embodiment (a first terminal device in the above-mentioned method embodiment) and a communication device as a network device.
[0328] The present disclosure further provides a readable storage medium having instructions stored thereon which, when executed by a computer, implement the functions of any one of the method embodiments described above.
[0329] The present disclosure further provides a computer program product, which, when executed by a computer, implements the functions of any one of the above method embodiments.
[0330] In the above embodiments, all or part of the above may be implemented in software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the above may be implemented in the form of a computer program product. The computer program product includes one or more computer programs. When the computer programs are loaded and executed in a computer, the flow or function according to the description of the embodiments of the present disclosure is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program may be stored in a computer-readable storage medium or may be transmitted from one computer-readable storage medium to another, for example, the computer program may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, radio, microwave, etc.) methods. The computer-readable storage medium may be any available medium accessible to a computer, or a data storage device such as a server, data center, etc., including one or more available media integrations. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a high-density digital video disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)).
[0331] As will be appreciated by those skilled in the art, the various numerals, such as first, second, etc., used in the present disclosure are divisions made for ease of explanation and do not limit the scope of the embodiments of the present disclosure, but also represent a priority order.
[0332] In the present disclosure, "at least one" may be described as "one or more", and more may be two, three, four or more, and is not limited in the present disclosure. In the embodiment of the present disclosure, for one technical feature, the technical features in the technical feature type are distinguished by "first", "second", "third", "A", "B", "C" and "D", etc., and there is no priority or size order between the technical features described by "first", "second", "third", "A", "B", "C" and "D".
[0333] Those skilled in the art can easily envision other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any modifications, applications or adaptations of the present invention, including the general principles of the present invention and including common general knowledge or commonly used technical means in the art not disclosed in this disclosure. It is intended that the specification and examples be considered as merely exemplary, with the true scope and spirit of the present disclosure being indicated by the following claims.
[0334] It should be understood that the present disclosure is limited to the exact construction described above and illustrated in the drawings, and various modifications and variations can be made without departing from the scope of the present disclosure, which is limited only by the appended claims.
Claims
1. A signal encoding and decoding method, applied on the encoding side, comprising: obtaining a mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal; determining an encoding mode for each of the audio signals of different formats based on signal characteristics of the audio signals of different formats; encoding the audio signals of each format using an encoding mode of the audio signals of each format to obtain encoded signal parameter information of the audio signals of each format, and writing the encoded signal parameter information of the audio signals of each format into an encoded code stream and transmitting it to a decoding side; 13. A method for encoding and decoding a signal, comprising:
2. determining an encoding mode for each of the audio signals of different formats based on signal characteristics of the audio signals of different formats, determining an encoding mode for the sound channel-based audio signal based on signal characteristics of the sound channel-based audio signal; determining an encoding mode for the object-based audio signal based on signal characteristics of the object-based audio signal; determining an encoding mode for the scene-based audio signal based on signal characteristics of the scene-based audio signal; 2. A method for encoding and decoding a signal according to claim 1.
3. determining an encoding mode for the sound channel-based audio signal based on signal characteristics of the sound channel-based audio signal, obtaining a number of object signals included in the sound channel-based audio signal; determining whether a number of object signals included in the sound channel-based audio signal is less than a first threshold; if the number of object signals contained in the sound channel based audio signal is less than a first threshold, the coding mode of the sound channel based audio signal is encoding each object signal in the sound channel-based audio signal using an object signal coding kernel; obtaining input first command line control information, and encoding at least some object signals in the sound channel-based audio signal based on the first command line control information using an object signal encoding kernel, wherein the first command line control information indicates object signals that need to be encoded among the object signals included in the sound channel-based audio signal, and determining that the number of object signals that need to be encoded is one or more and is smaller than a total number of object signals included in the sound channel-based audio signal.
3. A method for encoding and decoding a signal according to claim 2.
4. determining an encoding mode for the sound channel-based audio signal based on signal characteristics of the sound channel-based audio signal, obtaining a number of object signals included in the sound channel-based audio signal; determining whether a number of object signals included in the sound channel-based audio signal is less than a first threshold; if the number of object signals contained in the sound channel based audio signal is equal to or greater than a first threshold, the coding mode of the sound channel based audio signal is converting the sound channel-based audio signal into an audio signal of a first other format, the audio signal having a number of sound channels being less than the number of sound channels of the sound channel-based audio signal, and encoding the audio signal of the first other format using an encoding kernel corresponding to the audio signal of the first other format; obtaining input first command line control information, and encoding at least some object signals in the sound channel-based audio signal based on the first command line control information using an object signal encoding kernel, wherein the first command line control information indicates object signals that need to be encoded among the object signals included in the sound channel-based audio signal, and the number of object signals that need to be encoded is one or more and is smaller than a total number of object signals included in the sound channel-based audio signal; obtaining input second command line control information, and encoding at least some sound channel signals in the sound channel-based audio signal based on the second command line control information using an object signal encoding kernel, wherein the second command line control information indicates sound channel signals that need to be encoded among the sound channel signals included in the sound channel-based audio signal, and determining that the number of sound channel signals that need to be encoded is one or more and is less than a total number of sound channel signals included in the sound channel-based audio signal.
3. A method for encoding and decoding a signal according to claim 2.
5. Encoding the audio signal of each format using the encoding mode of the audio signal of each format to obtain signal parameter information after encoding of the audio signal of each format, encoding the sound channel based audio signal using an encoding mode of the sound channel based audio signal.
5. A method for encoding and decoding a signal according to claim 3 or 4.
6. determining an encoding mode for the object-based audio signal based on signal characteristics of the object-based audio signal, performing a signal feature analysis on the object-based audio signal to obtain an analysis result; classifying the object-based audio signal to obtain a first type of set of object signals and a second type of set of object signals, each of the first type of set of object signals and the second type of set of object signals including at least one object-based audio signal; determining a coding mode corresponding to said set of first type object signals; classifying the second type object signal set based on the analysis result to obtain at least one object signal subset, and determining an encoding mode corresponding to each object signal subset based on a classification result, wherein the object signal subset includes at least one object-based audio signal.
3. A method for encoding and decoding a signal according to claim 2.
7. The step of classifying the object-based audio signal to obtain a set of object signals of a first type and a set of object signals of a second type comprises: classifying the object-based audio signals that do not require separate manipulation processing into a first type of object signal set and classifying the remaining signals into a second type of object signal set.
7. A method for encoding and decoding a signal according to claim 6.
8. The step of determining an encoding mode corresponding to a set of object signals of a first type comprises: determining that an encoding mode corresponding to the first type of object signal set is performing a first pre-rendering process on object-based audio signals in the first type of object signal set and encoding the first pre-rendered signals using a multi-channel encoding kernel; the first pre-rendering process includes performing a signal format conversion process on the object-based audio signal to convert the object-based audio signal into a sound channel-based audio signal; 8. A method for encoding and decoding a signal according to claim 7.
9. The step of classifying the object-based audio signal to obtain a set of object signals of a first type and a set of object signals of a second type comprises: classifying, among the object-based audio signals, signals belonging to background sounds into a first type of object signal set and classifying the remaining signals into a second type of object signal set; 7. A method for encoding and decoding a signal according to claim 6.
10. The step of determining an encoding mode corresponding to a set of object signals of a first type comprises: determining that an encoding mode corresponding to the first type of object signal set is to perform a second pre-rendering process on object-based audio signals in the first type of object signal set, and encode the second pre-rendered signals using a Higher Order Ambisonics (HOA) encoding kernel; the second pre-rendering process includes performing a signal format conversion process on the object-based audio signal to convert the object-based audio signal into a scene-based audio signal.
10. A method for encoding and decoding a signal according to claim 9.
11. the first type of object signal set includes a first subset of object signals and a second subset of object signals; The step of classifying the object-based audio signal to obtain a set of object signals of a first type and a set of object signals of a second type comprises: classifying, among the object-based audio signals, signals not requiring individual manipulation processing into a first object signal subset, classifying, among the object-based audio signals, signals belonging to background sounds into a second object signal subset, and classifying the remaining signals into a second type of object signal set; 7. A method for encoding and decoding a signal according to claim 6.
12. The step of determining an encoding mode corresponding to a set of object signals of a first type comprises: determining that an encoding mode corresponding to a first subset of object signals in the first type of object signal set is to perform a first pre-rendering operation on object-based audio signals in the first subset of object signals and encode the first pre-rendered signals using a multi-channel encoding kernel, the first pre-rendering operation including performing a signal format conversion operation on the object-based audio signals to convert the object-based audio signals into sound channel-based audio signals; determining that an encoding mode corresponding to a second subset of object signals in the first type of object signal set is to perform a second pre-rendering operation on object-based audio signals in the second subset of object signals and encode the second pre-rendered signals using an HOA encoding kernel, the second pre-rendering operation including performing a signal format conversion operation on the object-based audio signals to convert the object-based audio signals into scene-based audio signals.
12. A method for encoding and decoding a signal according to claim 11.
13. The step of performing a signal feature analysis on the object-based audio signal to obtain an analysis result includes: performing a high-pass filtering process on the object-based audio signal; performing a correlation analysis on the high-pass filtered signal to determine cross-correlation parameter values between each of the object-based audio signals; 13. A method for encoding and decoding a signal according to claim 8, 10 or 12.
14. The step of classifying the set of second type object signals based on the analysis result to obtain at least one object signal subset, and determining an encoding mode corresponding to each object signal subset based on the classification result, comprises: setting a normalized correlation interval based on the correlation; classifying the second type object signal set to obtain at least one object signal subset based on the cross-correlation parameter values of the object-based audio signals and a normalized correlation degree interval, and determining a corresponding coding mode based on the correlation degree corresponding to the at least one object signal subset.
14. A method for encoding and decoding a signal according to claim 13.
15. the encoding mode corresponding to the object signal subset includes an independent encoding mode or a joint encoding mode; 15. A method for encoding and decoding a signal according to claim 14.
16. The independent coding mode corresponds to a time domain processing method or a frequency domain processing method; If the object signals in the object signal subset are speech signals or similar speech signals, the independent coding mode adopts a time domain processing manner; if the object signals in the object signal subset are audio signals of other formats than speech or similar speech signals, the independent coding mode adopts a frequency domain processing manner; 16. A method for encoding and decoding a signal according to claim 15.
17. The step of encoding the audio signal of each format using the encoding mode of the audio signal of each format to obtain encoded signal parameter information of the audio signal of each format includes: encoding the object-based audio signal using a coding mode of the object-based audio signal; The step of encoding the object-based audio signal using a coding mode of the object-based audio signal comprises: encoding signals in said first type of object signal set using an encoding mode corresponding to said first type of object signal set; pre-processing subsets of object signals in the set of object signals of the second type and encoding all pre-processed subsets of object signals in the set of object signals of the second type in a corresponding encoding mode using a same object signal encoding kernel.
15. A method for encoding and decoding a signal according to claim 14.
18. The step of performing a signal feature analysis on the object-based audio signal to obtain an analysis result includes: analyzing a frequency bandwidth range of the object signal; 13. A method for encoding and decoding a signal according to claim 8, 10 or 12.
19. The step of classifying the set of second type object signals based on the analysis result to obtain at least one object signal subset, and determining an encoding mode corresponding to each object signal subset based on the classification result, comprises: determining bandwidth intervals corresponding to different frequency bandwidths; classifying the second type object signal set according to a frequency bandwidth range of the object-based audio signal and a bandwidth interval corresponding to a different frequency bandwidth to obtain at least one object signal subset, and determining a corresponding coding mode according to a frequency bandwidth corresponding to the at least one object signal subset.
20. A method for encoding and decoding a signal according to claim 18.
20. The step of classifying the set of second type object signals based on the analysis result to obtain at least one object signal subset, and determining an encoding mode corresponding to each object signal subset based on the classification result, comprises: obtaining input third command line control information indicating an encoded frequency bandwidth range corresponding to the object-based audio signal; classifying the second type of object signal set to obtain at least one object signal subset by integrating the third command line control information and the analysis result, and determining an encoding mode corresponding to each object signal subset based on a classification result.
20. A method for encoding and decoding a signal according to claim 18.
21. The step of encoding the audio signal of each format using the encoding mode of the audio signal of each format to obtain encoded signal parameter information of the audio signal of each format includes: encoding the object-based audio signal using a coding mode of the object-based audio signal; The step of encoding the object-based audio signal using a coding mode of the object-based audio signal comprises: encoding signals in said first type of object signal set using an encoding mode corresponding to said first type of object signal set; pre-processing subsets of object signals in the set of object signals of the second type and encoding the different pre-processed subsets of object signals in corresponding encoding modes using different object signal encoding kernels.
20. A method for encoding and decoding a signal according to claim 18.
22. determining an encoding mode for the scene-based audio signal based on signal features of the scene-based audio signal, obtaining a number of object signals included in the scene-based audio signal; determining whether a number of object signals included in the scene-based audio signal is less than a second threshold; If the number of object signals included in the scene-based audio signal is smaller than a second threshold, the coding mode of the scene-based audio signal is encoding each object signal of the scene-based audio signal using an object signal coding kernel; obtaining input fourth command line control information, and encoding at least some object signals in the scene-based audio signal based on the fourth command line control information using an object signal encoding kernel, wherein the fourth command line control information indicates object signals that need to be encoded among the object signals included in the scene-based audio signal, and determining that the number of object signals that need to be encoded is at least one of 1 and is smaller than a total number of object signals included in the scene-based audio signal.
3. A method for encoding and decoding a signal according to claim 2.
23. determining an encoding mode for the scene-based audio signal based on signal features of the scene-based audio signal, obtaining a number of object signals included in the scene-based audio signal; determining whether a number of object signals included in the scene-based audio signal is less than a second threshold; If the number of object signals included in the scene-based audio signal is equal to or greater than a second threshold, the coding mode of the scene-based audio signal is converting the scene-based audio signal into an audio signal of a second other format, the number of sound channels of which is less than the number of sound channels of the scene-based audio signal, and encoding the audio signal of the second other format using a scene signal encoding kernel; performing a low-order transformation on the scene-based audio signal to transform the scene-based audio signal into a low-order scene-based audio signal whose order is lower than a current order of the scene-based audio signal, and encoding the low-order scene-based audio signal using a scene signal encoding kernel.
23. A method for encoding and decoding a signal according to claim 22.
24. The step of encoding the audio signal of each format using the encoding mode of the audio signal of each format to obtain encoded signal parameter information of the audio signal of each format includes: encoding the scene-based audio signal using an encoding mode of the scene-based audio signal; 24. Method for encoding and decoding a signal according to claim 22 or 23.
25. The step of writing the signal parameter information of the encoded audio signal of each format into an encoded code stream and transmitting the encoded code stream to a decoding side includes: determining a classification side information parameter indicative of a classification scheme for the set of second type object signals; - determining side information parameters corresponding to each format of the audio signal, said side information parameters indicating a coding mode corresponding to the audio signal of the corresponding format; performing code stream multiplexing on the classification side information parameters, side information parameters corresponding to the audio signals of each format, and signal parameter information after encoding of the audio signals of each format to obtain an encoded code stream, and sending the encoded code stream to a decoding side. Method for encoding and decoding a signal according to claim 4, 6 or 22.
26. A signal encoding and decoding method, applied at the decoding side, comprising: receiving an encoded codestream transmitted from an encoding side; and decoding the encoded codestream to obtain a mixed-format audio signal, the mixed-format audio signal comprising at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal.
13. A method for encoding and decoding a signal, comprising:
27. The method comprises: performing codestream analysis on the encoded codestream to obtain classification side information parameters, side information parameters corresponding to the audio signals of each format, and signal parameter information after encoding of the audio signals of each format; the classification side information parameters indicate a classification scheme for a set of object signals of a second type of the object-based audio signal, and the side information parameters indicate a coding mode corresponding to an audio signal of a corresponding format.
27. A method for encoding and decoding a signal according to claim 26.
28. The step of decoding the encoded codestream to obtain a mixed format audio signal comprises: decoding the encoded signal parameter information of the sound channel based audio signal based on side information parameters corresponding to the sound channel based audio signal; - decoding the encoded signal parameter information of the object-based audio signal based on the classification side information parameters and on side information parameters corresponding to the object-based audio signal; and decoding the encoded signal parameter information of the scene-based audio signal based on side information parameters corresponding to the scene-based audio signal.
28. A method for encoding and decoding a signal according to claim 27.
29. Decoding the encoded signal parameter information of the object-based audio signal based on the classification side information parameters and on side information parameters corresponding to the object-based audio signal comprises the steps of: - determining from the encoded signal parameter information of the object-based audio signal encoded signal parameter information corresponding to a first type of object signal set and encoded signal parameter information corresponding to a second type of object signal set; decoding the encoded signal parameter information corresponding to said first type of object signal set based on side information parameters corresponding to said first type of object signal set; and decoding the encoded signal parameter information corresponding to a set of object signals of a second type based on the classification side information parameters and on side information parameters corresponding to a set of object signals of a second type.
29. A method for encoding and decoding a signal according to claim 28.
30. The step of decoding the coded signal parameter information corresponding to a set of object signals of a second type based on the classification side information parameters and on side information parameters corresponding to a set of object signals of a second type comprises: determining a classification scheme for the set of second type object signals based on the classification side information parameters; and decoding the encoded signal parameter information corresponding to the second type of object signal set based on a classification scheme of the second type of object signal set and side information parameters corresponding to the second type of object signal set.
30. A method for encoding and decoding a signal according to claim 29.
31. the classification side information parameter indicates that the classification scheme for the set of second type object signals is to classify based on cross-correlation parameter values; The step of decoding the coded signal parameter information corresponding to the second type of object signal set based on the classification manner of the second type of object signal set and the side information parameters corresponding to the second type of object signal set includes: using a same object signal decoding kernel to decode the coded signal parameter information of all signals in the second type object signal set based on the classification scheme of the second type object signal set and the side information parameters corresponding to the second type object signal set.
31. A method for encoding and decoding a signal according to claim 30.
32. The classification side information parameter indicates that the classification scheme of the second type of object signal set is to classify based on a frequency bandwidth range; The step of decoding the coded signal parameter information corresponding to the second type of object signal set based on the classification manner of the second type of object signal set and the side information parameters corresponding to the second type of object signal set includes: decoding, using a different object signal decoding kernel, the encoded signal parameter information of the different signals in the second type of object signal set based on the classification scheme of the second type of object signal set and the side information parameters corresponding to the second type of object signal set; 31. A method for encoding and decoding a signal according to claim 30.
33. The method comprises: and post-processing the decoded object-based audio signal. A method for encoding and decoding a signal according to any one of claims 29 to 32.
34. Decoding the encoded signal parameter information of the sound channel based audio signal based on side information parameters corresponding to the sound channel based audio signal comprises: determining an encoding mode corresponding to the sound channel based audio signal based on side information parameters corresponding to the sound channel based audio signal; and decoding the encoded signal parameter information of the sound channel based audio signal using a corresponding decoding mode based on an encoding mode corresponding to the sound channel based audio signal.
29. A method for encoding and decoding a signal according to claim 28.
35. Decoding the encoded signal parameter information of the scene based audio signal based on side information parameters corresponding to the scene based audio signal comprises: determining an encoding mode corresponding to the scene-based audio signal based on side information parameters corresponding to the scene-based audio signal; and decoding the encoded signal parameter information of the scene-based audio signal using a corresponding decoding mode based on an encoding mode corresponding to the scene-based audio signal.
29. A method for encoding and decoding a signal according to claim 28.
36. A device based on signal encoding and decoding, comprising: an acquisition module for acquiring a mixed-format audio signal including at least one of a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal; a decision module for determining an encoding mode of the audio signals of different formats based on signal characteristics of the audio signals of different formats; an encoding module for encoding the audio signal of each format using an encoding mode of the audio signal of each format to obtain signal parameter information after the encoding of the audio signal of each format, and writing the signal parameter information after the encoding of the audio signal of each format into an encoded code stream to transmit to a decoding side; 13. An apparatus based on signal encoding and decoding, characterized in that:
37. A device based on signal encoding and decoding, comprising: a receiving module for receiving the encoded code stream transmitted from the encoding side; a decoding module for decoding the encoded codestream to obtain a mixed-format audio signal, the mixed-format audio signal including at least one of the following formats: a sound channel-based audio signal, an object-based audio signal, and a scene-based audio signal; 13. An apparatus based on signal encoding and decoding, characterized in that:
38. 1. A communication device, comprising: The apparatus includes a processor and a memory, the memory stores a computer program, and the processor executes the computer program stored in the memory to cause the apparatus to perform the method according to any one of claims 1 to 25. A communication device comprising:
39. 1. A communication device, comprising: The apparatus includes a processor and a memory, the memory stores a computer program, and the processor executes the computer program stored in the memory to cause the apparatus to perform the method according to any one of claims 26 to 35. A communication device comprising:
40. 1. A communication device, comprising: A processor and an interface circuit, the interface circuit receives and transmits code instructions to the processor; The processor, by executing the code instructions, performs the method according to any one of claims 1 to 25. A communication device comprising:
41. 1. A communication device, comprising: A processor and an interface circuit, the interface circuit receives and transmits code instructions to the processor; The processor, by executing the code instructions, performs the method according to any one of claims 26 to 35. A communication device comprising:
42. A computer readable storage medium having instructions stored thereon which, when executed, effect the method according to any one of claims 1 to 25. A computer-readable storage medium comprising:
43. A computer readable storage medium having instructions stored thereon, which, when executed, effect the method according to any one of claims 26 to 35. A computer-readable storage medium comprising:
Citation Information
Patent Citations
Audio encoding device and audio decoding device
WO2015056383A1
Video sound processing device, video sound processing method, and program
WO2017208820A1
Coding apparatus and coding method
WO2018203471A1
Apparatus and method for encoding a spatial audio representation or apparatus and method for decoding an encoded audio signal using transport metadata and related computer programs
WO2020152154A1
