Signal processing method and apparatus thereof

Abstract

A signal processing method and an apparatus thereof, the method comprising: acquiring a mixed audio signal (S3101), the mixed audio signal comprising multiple sound channels; determining a first parameter of the mixed audio signal and / or energy parameters of one or more sound channels among the multiple sound channels (S3103); on the basis of the first parameter of the mixed audio signal and / or the energy parameters of the multiple sound channels, performing bit allocation on the multiple sound channels, to obtain a bit allocation parameter (S3104); on the basis of the bit allocation parameter, encoding the multiple sound channels to obtain a sound channel signal encoding parameter, and writing the bit allocation parameter and the sound channel signal encoding parameter into a code stream (S3105); sending the code stream (S3106). The present invention solves the problem in the related art of an encoding and decoding method not being able to reconstruct a desired and most suitable sound field.

Description

Signal processing method and device Technical Field

[0001] The present disclosure relates to the field of communication technology, and in particular to a signal processing method and device thereof. Background Art

[0002] With the increase of transmission bandwidth, the upgrade of terminal device signal acquisition equipment, the improvement of signal processor performance, and the upgrade of terminal playback equipment, three signal formats, namely channel-based audio signals, object-based audio signals, and scene-based audio signals, can provide three-dimensional audio services.

[0003] In 3D audio applications, 3D audio typically includes signals in multiple audio formats, i.e., mixed-format audio signals. In related technologies, the encoding and decoding methods for mixed-format audio signals typically involve performing a unified encoding and decoding process on the mixed-format audio signals. For example, the encoder allocates available bits based on the energy parameters of the mixed-format audio signals. Each channel then uses the corresponding encoding core to encode the allocated bits, outputting encoding parameters that are then written into the bitstream.

[0004] However, the encoder in the related art adopts bit allocation based on energy parameters for the input mixed format audio signal. This bit allocation method is relatively simple and one-sided, resulting in the inability to reconstruct the most suitable and / or desired sound field for a specific application scenario, thereby resulting in a poor reconstruction effect of the mixed format audio signal.

[0005] Summary of the Invention

[0006] The embodiments of the present disclosure provide a signal processing method and a device thereof.

[0007] According to a first aspect of an embodiment of the present disclosure, a signal processing method is proposed, the method comprising:

[0008] Acquire an audio signal in a mixed format, where the audio signal in the mixed format is an audio signal including multiple channels;

[0009] determining a first parameter of the audio signal in the mixed format and / or an energy parameter of one or more of the plurality of channels;

[0010] performing bit allocation on the multiple channels to obtain a bit allocation parameter according to the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels;

[0011] Encoding the multiple channels according to the bit allocation parameter to obtain channel signal encoding parameters, and writing the bit allocation parameter and the channel signal encoding parameters into a bitstream;

[0012] Send the code stream.

[0013] According to a second aspect of an embodiment of the present disclosure, a signal processing method is proposed, including:

[0014] Receive code stream;

[0015] parsing the code stream to obtain a bit allocation parameter and a channel signal coding parameter for each of the multiple channels of the mixed-format audio signal;

[0016] The audio signal in the mixed format is reconstructed based on the bit allocation parameters of the respective channels and the channel signal encoding parameters.

[0017] According to a third aspect of an embodiment of the present disclosure, a first communication device is provided, including:

[0018] a processing module, configured to obtain an audio signal in a mixed format, where the audio signal in the mixed format is an audio signal including multiple channels;

[0019] The processing module is further configured to determine a first parameter of the audio signal in the mixed format and / or an energy parameter of one or more of the multiple channels;

[0020] The processing module is further configured to perform bit allocation on the multiple channels according to the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels to obtain a bit allocation parameter;

[0021] The processing module is further configured to encode the plurality of channels according to the bit allocation parameter to obtain channel signal encoding parameters, and write the bit allocation parameter and the channel signal encoding parameters into a bitstream;

[0022] The transceiver module is used to send the code stream.

[0023] According to a fourth aspect of an embodiment of the present disclosure, a second communication device is provided, including:

[0024] Transceiver module, used for receiving code stream;

[0025] a processing module, configured to parse the code stream to obtain a bit allocation parameter and a channel signal coding parameter for each of the multiple channels of the audio signal in a mixed format;

[0026] The processing module is further configured to reconstruct the audio signal in the mixed format based on the bit allocation parameters of each channel and the channel signal encoding parameters.

[0027] According to a fifth aspect of an embodiment of the present disclosure, a communication system is provided, including:

[0028] An encoding terminal device, configured to perform an optional implementation of the aforementioned first aspect;

[0029] The decoding end device is configured to execute the optional implementation of the aforementioned second aspect.

[0030] According to a sixth aspect of an embodiment of the present disclosure, a communication device is provided, including: one or more processors;

[0031] The processor is used to call instructions to enable the communication device to execute the optional implementation of the first and second aspects mentioned above.

[0032] According to a seventh aspect of an embodiment of the present disclosure, a storage medium is proposed, which stores instructions. When the instructions are executed on a communication device, the communication device executes optional implementation methods of the aforementioned first and second aspects.

[0033] According to the technical solution disclosed in the present invention, the problem that the encoding and decoding methods in the related art cannot reconstruct the desired and most suitable sound field can be solved, thereby improving the reconstruction effect of the audio signal in the mixed format. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following drawings required for describing the embodiments are introduced. The following drawings are merely some embodiments of the present disclosure and do not impose specific limitations on the protection scope of the present disclosure.

[0035] FIG1 is a schematic diagram of the architecture of a communication system provided by an embodiment of the present disclosure;

[0036] FIG2A is an interactive schematic diagram illustrating a signal processing method according to an embodiment of the present disclosure;

[0037] FIG2B is an interactive schematic diagram illustrating a signal processing method according to an embodiment of the present disclosure;

[0038] FIG2C is an interactive schematic diagram illustrating a signal processing method according to an embodiment of the present disclosure;

[0039] FIG2D is an interactive schematic diagram illustrating a signal processing method according to an embodiment of the present disclosure;

[0040] FIG3 is a flow chart showing a signal processing method according to an embodiment of the present disclosure;

[0041] FIG4 is a flow chart of another signal processing method according to an embodiment of the present disclosure;

[0042] FIG5 is an interactive schematic diagram illustrating a signal processing method according to an embodiment of the present disclosure;

[0043] FIG6A is an example diagram 1 of a signal processing method proposed in an embodiment of the present disclosure;

[0044] FIG6B is a second example diagram of the signal processing method proposed in an embodiment of the present disclosure;

[0045] FIG7A is a schematic structural diagram of an encoding terminal device proposed in an embodiment of the present disclosure;

[0046] FIG7B is a schematic structural diagram of a decoding terminal device proposed in an embodiment of the present disclosure;

[0047] FIG8A is a schematic structural diagram of a communication device 8100 proposed in an embodiment of the present disclosure;

[0048] FIG8B is a schematic structural diagram of a chip 8200 according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0049] The embodiments of the present disclosure provide a signal processing method and a device thereof.

[0050] In a first aspect, an embodiment of the present disclosure provides a signal processing method, the method comprising:

[0051] Acquire an audio signal in a mixed format, where the audio signal in the mixed format is an audio signal including multiple channels;

[0052] determining a first parameter of the audio signal in the mixed format and / or an energy parameter of one or more of the plurality of channels;

[0053] performing bit allocation on the multiple channels to obtain a bit allocation parameter according to the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels;

[0054] Encoding the multiple channels according to the bit allocation parameter to obtain channel signal encoding parameters, and writing the bit allocation parameter and the channel signal encoding parameters into a bitstream;

[0055] Send the code stream.

[0056] In the above embodiment, by adopting the first parameter of the audio signal based on energy and / or mixed format, bits are allocated to multiple channels in the mixed format audio signal. The bit allocation parameters can be made closer to the actual sound field conditions of the specific application scenario, so that the desired and most suitable sound field can be reconstructed, which solves the problem in the related art that the encoding and decoding method cannot reconstruct the desired and most suitable sound field, thereby improving the reconstruction effect of the mixed format audio signal.

[0057] In conjunction with some embodiments of the first aspect, in some embodiments, the first parameter includes at least one of the following:

[0058] Sound field analysis parameters;

[0059] Control parameters, the control parameters including a second parameter and / or a third parameter, the second parameter being used to describe the priority ranking of the roles played by each of the multiple channels in the audio signal in the mixed format, and the third parameter being used to describe whether each of the channels is a diegetic sound or a non-diegetic sound.

[0060] In the above embodiment, when allocating bits to multiple channels in a mixed format audio signal, considering the sound field analysis parameters and / or control parameters can make the bit allocation parameters closer to the actual sound field conditions of the specific application scenario, so that the desired and most suitable sound field can be reconstructed, thereby improving the reconstruction effect of the mixed format audio signal.

[0061] In conjunction with some embodiments of the first aspect, in some embodiments, the mixed-format audio signal includes an audio signal in at least one of the following formats:

[0062] Channel-based audio signals;

[0063] object-based audio signals;

[0064] scene-based audio signals;

[0065] Metadata-based three-dimensional audio signals.

[0066] In the above embodiment, mixed audio signals in various formats can be coded and decoded, thereby enabling a three-dimensional audio communication service to be implemented between an audio signal acquisition end and an audio signal playback end.

[0067] In conjunction with some embodiments of the first aspect, in some embodiments, determining the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels includes:

[0068] determining energy parameters of one or more of the plurality of channels;

[0069] The sound field analysis parameters of the audio signal in the mixed format are determined.

[0070] In conjunction with some embodiments of the first aspect, in some embodiments, performing bit allocation on the multiple channels to obtain a bit allocation parameter based on the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels includes:

[0071] The bit allocation parameter is obtained by performing bit allocation on the multiple channels according to the energy parameter of one or more channels and the sound field analysis parameter of the audio signal in the mixed format.

[0072] In the above embodiment, when allocating bits to multiple channels in a mixed-format audio signal, in addition to considering the energy parameters, the sound field analysis parameters are also considered, so that the bit allocation parameters can be closer to the actual sound field conditions of the specific application scenario, thereby reconstructing the desired and most suitable sound field and improving the reconstruction effect of the mixed-format audio signal.

[0073] In conjunction with some embodiments of the first aspect, in some embodiments, determining the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels includes:

[0074] determining energy parameters of one or more of the plurality of channels;

[0075] The control parameter of the audio signal in the mixed format is determined.

[0076] In conjunction with some embodiments of the first aspect, in some embodiments, performing bit allocation on the multiple channels to obtain a bit allocation parameter based on the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels includes:

[0077] The bit allocation parameter is obtained by performing bit allocation on the multiple channels according to the energy parameter of one or more channels and the control parameter of the audio signal in the mixed format.

[0078] In the above embodiment, when allocating bits to multiple channels in a mixed format audio signal, external control parameters are also considered on the basis of considering energy parameters. The bit allocation can be adjusted according to the external control parameters, so that the bit allocation parameters can be closer to the actual sound field conditions of the specific application scenario, thereby reconstructing the desired and most suitable sound field and improving the reconstruction effect of the mixed format audio signal.

[0079] In conjunction with some embodiments of the first aspect, in some embodiments, determining the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels includes:

[0080] The sound field analysis parameter and / or the control parameter of the audio signal in the mixed format are determined.

[0081] In conjunction with some embodiments of the first aspect, in some embodiments, performing bit allocation on the multiple channels to obtain a bit allocation parameter based on the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels includes:

[0082] The bit allocation parameter is obtained by performing bit allocation on the multiple channels according to the sound field analysis parameter and / or the control parameter of the audio signal in the mixed format.

[0083] In the above embodiment, when allocating bits to multiple channels in a mixed format audio signal, considering the sound field analysis parameters and / or external control parameters can make the bit allocation parameters closer to the actual sound field conditions of the specific application scenario, so that the desired and most suitable sound field can be reconstructed, thereby improving the reconstruction effect of the mixed format audio signal.

[0084] In conjunction with some embodiments of the first aspect, in some embodiments, determining the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels includes:

[0085] determining energy parameters of one or more of the plurality of channels;

[0086] The sound field analysis parameter and the control parameter of the audio signal in the mixed format are determined.

[0087] In conjunction with some embodiments of the first aspect, in some embodiments, performing bit allocation on the multiple channels to obtain a bit allocation parameter based on the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels includes:

[0088] The bit allocation parameter is obtained by performing bit allocation on the multiple channels according to the energy parameter of one or more channels, the sound field analysis parameter of the audio signal in the mixed format, and the control parameter.

[0089] In the above embodiment, when allocating bits to multiple channels in a mixed-format audio signal, in addition to considering energy parameters, external control parameters and sound field analysis parameters are also considered, so that the bit allocation parameters can be further aligned with the actual sound field conditions of the specific application scenario, thereby reconstructing the desired and most suitable sound field, further improving the reconstruction effect of the mixed-format audio signal.

[0090] In conjunction with some embodiments of the first aspect, in some embodiments, determining the energy parameters of one or more of the multiple channels includes:

[0091] After filtering the plurality of channels, calculating the correlation coefficients between the plurality of channels;

[0092] Performing a channel combination operation according to the mutual correlation coefficient, and calculating inter-channel parameters for channels in the same channel combination;

[0093] The channels are aligned using the inter-channel parameters and then downmixed, and energy calculation is performed on the downmixed channels to obtain energy parameters of the channels.

[0094] In the above embodiment, filtering is performed on the sound channel, and energy calculation is performed on the sound channel after filtering, which can improve the accuracy of the calculation result and avoid noise interference.

[0095] In conjunction with some embodiments of the first aspect, in some embodiments, the method for calculating the energy parameter includes performing any one of the following processing on the sample points of the channel frame:

[0096] Calculating the amplitude value sum of the sample points;

[0097] Calculating the sum of squares of the amplitudes of the sample points;

[0098] Calculate the root mean square of the sample points.

[0099] In the above embodiment, by using any one of the methods of sample point amplitude sum, sample point amplitude square sum, sample point amplitude root mean square, etc. to perform energy calculation, the accuracy of the calculation result can be guaranteed.

[0100] In conjunction with some embodiments of the first aspect, in some embodiments, determining the sound field analysis parameter of the audio signal in the mixed format includes:

[0101] Based on direction of arrival (DOA) estimation and / or a multi-channel cross-correlation coefficient (MCCC) method, sound field analysis is performed on the audio signal in the mixed format to obtain the sound field analysis parameters of the audio signal in the mixed format.

[0102] In conjunction with some embodiments of the first aspect, in some embodiments, determining the control parameter of the audio signal in the mixed format includes at least one of the following:

[0103] determining control parameters of the mixed-format audio signal according to the functional division of each channel during acquisition by an audio signal acquisition device, wherein the audio signal acquisition device is used to acquire the mixed-format audio signal;

[0104] The control parameters of the audio signal in the mixed format are determined according to the setting parameters required by the decoding end device to reconstruct the desired sound field.

[0105] In the above embodiment, the above control parameters are determined based on the functional division of each channel during acquisition by the audio signal acquisition device and / or the setting parameters required for the desired sound field reconstructed by the decoding end device. This allows the above control parameters to be closer to the actual situation of the application scenario. In this way, by using the control parameters to adjust the bit allocation, the adjusted bit allocation parameters can be further closer to the actual sound field situation of the specific application scenario, thereby achieving the reconstruction of the desired mixed format audio signal at the decoding end.

[0106] In conjunction with some embodiments of the first aspect, in some embodiments, the sound field analysis parameter includes at least one of the following:

[0107] Number of audio and video;

[0108] the priority of sound and image;

[0109] Background ambient sound.

[0110] In conjunction with some embodiments of the first aspect, in some embodiments, allocating bits to the multiple channels based on the first parameter of the audio signal in the mixed format and / or energy parameters of one or more of the multiple channels includes:

[0111] performing bit allocation on a mixed-format audio signal consisting of the object-based audio signal and the scene-based audio signal according to the sound field analysis parameter of the mixed-format audio signal;

[0112] The number of bits allocated to any channel in the object-based audio signal is greater than the number of bits allocated to any channel in the scene-based audio signal.

[0113] In conjunction with some embodiments of the first aspect, in some embodiments, allocating bits to the multiple channels based on the first parameter of the audio signal in the mixed format and / or the energy parameters of one or more of the multiple channels includes:

[0114] performing bit allocation on a mixed-format audio signal composed of the object-based audio signal and the channel-based audio signal according to the sound field analysis parameter of the mixed-format audio signal;

[0115] The number of bits allocated to any channel in the object-based audio signal is greater than the number of bits allocated to any channel in the channel-based audio signal.

[0116] In conjunction with some embodiments of the first aspect, in some embodiments, allocating bits to the multiple channels based on the first parameter of the audio signal in the mixed format and / or the energy parameters of one or more of the multiple channels includes:

[0117] performing bit allocation on a mixed-format audio signal consisting of the channel-based audio signal and the scene-based audio signal according to the sound field analysis parameter of the mixed-format audio signal;

[0118] The number of bits allocated to any channel in the channel-based audio signal is greater than the number of bits allocated to any channel in the scene-based audio signal.

[0119] In conjunction with some embodiments of the first aspect, in some embodiments, allocating bits to the multiple channels based on the first parameter of the audio signal in the mixed format and / or the energy parameters of one or more of the multiple channels includes:

[0120] performing bit allocation on a hybrid audio signal composed of the object-based audio signal and the metadata-based three-dimensional audio signal according to the sound field analysis parameter of the hybrid audio signal;

[0121] The number of bits allocated to any channel in the object-based audio signal is greater than the number of bits allocated to any channel in the metadata-based three-dimensional audio signal.

[0122] In conjunction with some embodiments of the first aspect, in some embodiments, allocating bits to the multiple channels based on the first parameter of the audio signal in the mixed format and / or the energy parameters of one or more of the multiple channels includes:

[0123] performing bit allocation on an audio signal in a mixed format consisting of the channel-based audio signal and the metadata-based three-dimensional audio signal according to the sound field analysis parameter of the audio signal in the mixed format;

[0124] The number of bits allocated to any channel in the channel-based audio signal is greater than the number of bits allocated to any channel in the metadata-based three-dimensional audio signal.

[0125] In combination with some embodiments of the first aspect, in some embodiments, the number of bits allocated to the channel corresponding to the primary sound image in the object-based audio signal is greater than the number of bits allocated to the channel corresponding to the secondary sound image in the object-based audio signal.

[0126] In a second aspect, the present disclosure provides a signal processing method, comprising:

[0127] Receive code stream;

[0128] parsing the code stream to obtain a bit allocation parameter and a channel signal coding parameter for each of the multiple channels of the mixed-format audio signal;

[0129] The audio signal in the mixed format is reconstructed based on the bit allocation parameters of the respective channels and the channel signal encoding parameters.

[0130] In conjunction with some embodiments of the second aspect, in some embodiments, reconstructing the audio signal in the mixed format according to the bit allocation parameters and channel signal encoding parameters of each channel and the bit stream includes:

[0131] decoding the channel signal coding parameters based on the bit allocation parameters of the respective channels;

[0132] The audio signal in the mixed format is reconstructed using the decoded channels.

[0133] In a third aspect, an embodiment of the present disclosure proposes an encoding end device, comprising at least one of a transceiver module and a processing module; wherein the above-mentioned encoding end device is used to execute the optional implementation method of the first aspect.

[0134] In a fourth aspect, an embodiment of the present disclosure proposes a decoding end device, comprising at least one of a transceiver module and a processing module; wherein the above-mentioned decoding end device is used to execute the optional implementation method of the second aspect.

[0135] In a fifth aspect, an embodiment of the present disclosure provides a communication system, including:

[0136] An encoding end device, configured as an optional implementation of the aforementioned first aspect;

[0137] The decoding end device is configured to execute the optional implementation of the aforementioned second aspect.

[0138] In a sixth aspect, an embodiment of the present disclosure proposes a communication device, comprising: one or more processors; wherein the processor is used to call instructions to enable the communication device to execute the optional implementation method of the aforementioned first aspect.

[0139] In a seventh aspect, an embodiment of the present disclosure proposes a communication device, comprising: one or more processors; wherein the processor is used to call instructions to enable the communication device to execute the optional implementation method of the aforementioned second aspect.

[0140] In an eighth aspect, an embodiment of the present disclosure proposes a storage medium storing instructions. When the instructions are executed on a communication device, the communication device executes optional implementation methods of the aforementioned first and second aspects.

[0141] In a ninth aspect, an embodiment of the present disclosure proposes a program product. When the program product is executed by a communication device, the communication device executes the method described in the optional implementation of the first and second aspects.

[0142] In a tenth aspect, an embodiment of the present disclosure proposes a computer program, which, when executed on a computer, enables the computer to execute the method described in the optional implementation of the first and second aspects.

[0143] In an eleventh aspect, an embodiment of the present disclosure provides a chip or a chip system, wherein the chip or chip system includes a processing circuit configured to execute the method described in the optional implementation of the first and second aspects above.

[0144] It is understandable that the above-mentioned encoding end device, decoding end device, communication system, storage medium, program product, computer program, chip or chip system are all used to perform the method proposed in the embodiment of the present disclosure. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding method and will not be repeated here.

[0145] The present disclosure provides a signal processing method and apparatus thereof. In some embodiments, the terms signal processing method, communication method, signal encoding and decoding method, signal encoding method, and signal decoding method are interchangeable; the terms signal processing apparatus and communication apparatus are interchangeable; and the terms signal processing system, information processing system, and communication system are interchangeable.

[0146] The embodiments of the present disclosure are not exhaustive and are merely illustrative of some embodiments, and are not intended to be a specific limitation on the scope of protection of the present disclosure. In the absence of contradiction, each step in a certain embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a certain embodiment can also be implemented as an independent embodiment, and the order of the steps in a certain embodiment can be arbitrarily exchanged. In addition, the optional implementation methods in a certain embodiment can be arbitrarily combined; in addition, the embodiments can be arbitrarily combined. For example, some or all steps of different embodiments can be arbitrarily combined, and a certain embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0147] In each embodiment of the present disclosure, unless otherwise specified or provided for by logic, the terms and / or descriptions between the embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form a new embodiment based on their inherent logical relationships.

[0148] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure.

[0149] In the embodiments of the present disclosure, unless otherwise specified, elements expressed in the singular, such as "a", "an", "the", "above", "said", "the", "the", etc., may mean "one and only one", or "one or more", "at least one", etc. For example, when using articles such as "a", "an", "the" in English in translation, the noun following the article may be understood as a singular expression or a plural expression.

[0150] In the embodiments of the present disclosure, “plurality” refers to two or more.

[0151] In some embodiments, the terms "at least one," "one or more," "a plurality of," "multiple," etc. may be used interchangeably.

[0152] In some embodiments, descriptions such as "at least one of A and B," "A and / or B," "A in one case, B in another case," or "in response to one case A, in response to another case B" may include the following technical solutions depending on the situation: in some embodiments, A (A is executed independently of B); in some embodiments, B (B is executed independently of A); in some embodiments, execution is selected from A and B (A and B are selectively executed); and in some embodiments, A and B (both A and B are executed). The above is also applicable when there are more branches such as A, B, and C.

[0153] In some embodiments, "A or B" and other descriptions may include the following technical solutions depending on the situation: in some embodiments, A (A is executed independently of B); in some embodiments, B (B is executed independently of A); in some embodiments, execution is selected from A and B (A and B are selectively executed). The above is also applicable when there are more branches such as A, B, C, etc.

[0154] The prefixes such as "first" and "second" in the embodiments of the present disclosure are only used to distinguish different description objects and do not constitute any restriction on the position, order, priority, quantity or content of the description objects. For the statement of the description object, please refer to the description in the context of the claims or embodiments, and no unnecessary restriction should be constituted due to the use of prefixes. For example, if the description object is a "field", the ordinal number before the "field" in the "first field" and the "second field" does not limit the position or order between the "fields". "First" and "second" do not limit whether the "fields" they modify are in the same message, nor do they limit the order of the "first field" and the "second field". For another example, if the description object is a "level", the ordinal number before the "level" in the "first level" and the "second level" does not limit the priority between the "levels". For another example, the number of description objects is not limited by the ordinal number and can be one or more. Taking "first device" as an example, the number of "devices" can be one or more. In addition, the objects modified by different prefixes can be the same or different. For example, if the description object is "device", then the "first device" and the "second device" can be the same device or different devices, and their types can be the same or different; for another example, if the description object is "information", then the "first information" and the "second information" can be the same information or different information, and their contents can be the same or different.

[0155] In some embodiments, “including A,” “comprising A,” “used to indicate A,” and “carrying A” can be interpreted as directly carrying A or indirectly indicating A.

[0156] In some embodiments, terms such as "in response to...", "in response to determining...", "in the case of...", "at the time of...", "when...", "if...", "if...", etc. can be used interchangeably.

[0157] In some embodiments, terms such as "greater than", "greater than or equal to", "not less than", "more than", "more than or equal to", "not less than", "higher than", "higher than or equal to", "not less than", and "above" can be replaced with each other, and terms such as "less than", "less than or equal to", "not greater than", "less than", "less than or equal to", "not more than", "lower than", "lower than or equal to", "not higher than", and "below" can be replaced with each other.

[0158] In some embodiments, devices and equipment can be interpreted as physical or virtual, and their names are not limited to the names recorded in the embodiments. In some cases, they can also be understood as "equipment", "device", "circuit", "network element", "node", "function", "unit", "section", "system", "network", "chip", "chip system", "entity", "subject", etc.

[0159] In some embodiments, "network" can be interpreted as devices included in the network, such as access network equipment, core network equipment, etc.

[0160] In some embodiments, "access network device (AN device)" may also be referred to as "radio access network device (RAN device)", "base station (BS)", "radio base station", "fixed station", and in some embodiments may also be understood as "node", "access point", "transmission point (TP)", "reception point (RP)", "transmission and / or reception point (TRP)" "panel", "antenna panel", "antenna array", "cell", "macro cell", "small cell", "femto cell", "pico cell", "sector", "cell group", "serving cell", "carrier", "component carrier", "bandwidth part (BWP)", etc.

[0161] In some embodiments, "terminal" or "terminal device" may be referred to as "user equipment (UE)", "user terminal" "mobile station (MS)", "mobile terminal (MT)", subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, etc.

[0162] In some embodiments, obtaining data, information, etc. may comply with the laws and regulations of the country where the data is obtained.

[0163] In some embodiments, data, information, etc. may be obtained with the user's consent.

[0164] Figure 1 is a schematic diagram of the architecture of a communication system according to an embodiment of the present disclosure. The communication system may include, but is not limited to, one encoding device and one decoding device. The number and configuration of devices shown in Figure 1 are for example purposes only and do not constitute a limitation on the embodiments of the present disclosure. In actual applications, two or more encoding devices and two or more decoding devices may be included. The communication system 100 shown in Figure 1 includes, for example, one encoding device 101 and one decoding device 102.

[0165] In some embodiments, the encoding end device 101 may be a terminal with an encoding function, for example, the terminal includes an encoder. The terminal may be understood as an acquisition end for mixed-format audio signals. In some embodiments, the terminal may be an entity on the user side for receiving or transmitting signals, such as a mobile phone. It may also be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The terminal may be at least one of a car with communication functions, a smart car, a mobile phone, a wearable device, a tablet computer (Pad), a computer with wireless transceiver functions, a virtual reality (VR) terminal, an augmented reality (AR) terminal, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, and the like. The embodiments of the present disclosure do not limit the specific technology and specific device form adopted by the terminal.

[0166] In some embodiments, encoding device 101 may be an encoder on a network device. For example, an encoder may be deployed on the network device, and the encoder on the network device encodes an input audio signal in a mixed format. For example, a capture device may send a mixed-format audio signal to the network device. The encoder on the network device encodes the mixed-format audio signal and sends the resulting bitstream to the decoding device 102.

[0167] In some embodiments, the decoding end device 102 may be a terminal with a decoding function, for example, the terminal includes an encoder. The terminal may be understood as a playback end for mixed-format audio signals. In some embodiments, the terminal may be an entity on the user side for receiving or transmitting signals, such as a mobile phone. It may also be referred to as a terminal, user equipment (UE), mobile station (MS), mobile terminal (MT), etc. The terminal may be at least one of a car with communication functions, a smart car, a mobile phone, a wearable device, a tablet computer (Pad), a computer with wireless transceiver functions, a virtual reality (VR) terminal, an augmented reality (AR) terminal, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, and the like. The embodiments of the present disclosure do not limit the specific technology and specific device form adopted by the terminal.

[0168] In some embodiments, the decoding end device 102 may be a decoder on a network device. For example, a decoder is deployed on the network device, which decodes the received bitstream and reconstructs a mixed-format audio signal, and sends the reconstructed mixed-format audio signal to the playback end.

[0169] In some embodiments, the network device may be an access network device. In some embodiments, the access network device is, for example, a node or device that connects a terminal device to a wireless network. The access network device may include, but is not limited to, at least one of an evolved NodeB (eNB), a next generation evolved NodeB (ng-eNB), a next generation NodeB (gNB), a NodeB (NB), a home nodeB (HNB), a home evolved nodeB (HeNB), a wireless backhaul device, a radio network controller (RNC), a base station controller (BSC), a base transceiver station (BTS), a base band unit (BBU), a mobile switching center, a base station in a 6G communication system, an open RAN, a cloud RAN, a base station in other communication systems, and an access node in a Wi-Fi system.

[0170] In some embodiments, the technical solution of the present disclosure can be applied to the Open RAN architecture. In this case, the interfaces between or within the access network devices involved in the embodiments of the present disclosure can be transformed into internal interfaces of the Open RAN, and the processes and information interactions between these internal interfaces can be implemented through software or programs.

[0171] In some embodiments, the access network device can be composed of a centralized unit (CU) and a distributed unit (DU), where the CU can also be called a control unit. The CU-DU structure can be used to split the protocol layer of the access network device, with the functions of some protocol layers centrally controlled by the CU, and the functions of the remaining part or all of the protocol layers distributed in the DU, which is centrally controlled by the CU, but is not limited to this.

[0172] It can be understood that the communication system described in the embodiment of the present disclosure is for the purpose of more clearly illustrating the technical solution of the embodiment of the present disclosure, and does not constitute a limitation on the technical solution proposed in the embodiment of the present disclosure. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution proposed in the embodiment of the present disclosure is also applicable to similar technical problems.

[0173] The following embodiments of the present disclosure may be applied to the communication system 100 shown in FIG1 , or a portion thereof, but are not limited thereto. The entities shown in FIG1 are illustrative only. The communication system may include all or part of the entities shown in FIG1 , or may include other entities outside of FIG1 . The number and form of the entities are arbitrary, and the entities may be physical or virtual. The connection relationships between the entities are illustrative only. The entities may be connected or disconnected, and the connection may be in any manner, including direct or indirect, wired or wireless.

[0174] The embodiments of the present disclosure can be applied to Long Term Evolution (LTE), LTE-Advanced (LTE-A), LTE-Beyond (LTE-B), SUPER 3G, IMT-Advanced, 4th generation mobile communication system (4G), 5th generation mobile communication system (5G), 5G new radio (NR), future radio access (FRA), new radio access technology (RAT), new radio (NR), new radio access (NX), future generation radio access (FX), Global System for Mobile communications (GSM (registered trademark)), CDMA2000, Ultra Mobile Broadband (UMB), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, Ultra-WideBand (UWB), Bluetooth (registered trademark), Public Land Mobile Network (PLMN) networks, Device-to-Device (D2D) systems, Machine-to-Machine (M2M) systems, Internet of Things (IoT) systems, Vehicle-to-Everything (V2X), systems utilizing other communication methods, and next-generation systems based on and extending these methods. Furthermore, multiple systems may be combined (for example, a combination of LTE or LTE-A with 5G).

[0175] It should be noted that the first generation of mobile communication technology (1G) is the first generation of wireless cellular technology and belongs to the analog mobile communication network. The upgrade from 1G to 2G transitioned mobile phones from analog to digital communication, primarily using the GSM (Global System for Mobile Communications) network standard. Voice codecs used AMR (Adaptive Multi-Rate), EFR (Enhanced Full Rate), FR (Full Rate), and HR (Half Rate) to provide single-channel narrowband voice services. The 3G mobile communication system was proposed by the ITU (International Telecommunication Union) for international mobile communications in 2000. China Mobile uses TD-SCDMA, China Telecom uses CDMA2000, and China Unicom uses WCDMA. Their voice codecs use AMR-WB (Adaptive Multi-Rate Wideband) to provide single-channel wideband voice services. 4G is a further improvement on 3G technology. Both data and voice use an all-IP (Internet protocol) approach, providing real-time HD / HD+Voice services for voice and audio. The EVS codec used can achieve high-quality compression and reconstruction of both voice and audio.

[0176] The voice and audio communication services provided above have expanded from narrowband signals to ultra-wideband and even full-band services, but they are still mono services. People's demand for high-quality audio is increasing. Compared with mono audio, stereo audio has a sense of direction and distribution for each sound source and can improve clarity and clarity. With the increase of transmission bandwidth and the upgrade of terminal equipment signal acquisition equipment, the improvement of signal processor performance, and the upgrade of terminal playback equipment.

[0177] In some embodiments, three signal formats—channel-based, object-based, and scene-based—can provide 3D audio services. The IVAS (Immersive Voice and Audio Services) codec, currently being standardized by 3GPP SA4, supports the encoding and decoding requirements of these three signal formats.

[0178] In some embodiments, the channel-based audio signals include but are not limited to: mono signals, stereo signals (Stereo), binaural signals (Binaural), 5.1, 7.1 surround sound signals (Surround), 5.1.4, 7.1.4 surround sound signals (Surround), where .4 represents the height channel signal (Height).

[0179] In some embodiments, the scene-based audio signal includes but is not limited to: first order ambisonics (FOA), second order high ambisonics (HOA2), and third order high ambisonics (HOA3).

[0180] In some embodiments, the object-based audio signal includes audio data and metadata. In addition, IVAS also supports metadata-assisted spatial audio (MASA).

[0181] In some embodiments, terminals that can support three-dimensional audio services may include but are not limited to mobile phones, computers, tablets, conference system equipment, AR / VR equipment, cars, etc.

[0182] In some embodiments, in the application scenario of three-dimensional audio, the three-dimensional audio usually contains signals in multiple audio signal formats, that is, audio signals in mixed formats. The encoding end device receives the audio signal in the mixed format, and the audio code stream signal generated after encoding is sent from the transmitting end to the receiving end. The decoder at the receiving end decodes the received audio code stream and reconstructs the audio signal in the mixed format.

[0183] However, the encoding and decoding methods for mixed-format audio signals in related technologies are generally as follows: a corresponding unified encoding and decoding process is performed on the mixed-format audio signal, the encoding end processing process is to allocate the available bits according to the energy parameters of the mixed-format audio signal, and each channel is encoded using the encoding core based on the allocated bits to output encoding parameters, and the encoding parameters are placed in the bit stream. In other words, the encoder in the related technology uses energy parameter-based bit allocation for the input mixed-format audio signal, and does not perform sound field analysis on the mixed-format audio signal, resulting in the inability to reconstruct the most suitable sound field for a specific application scenario, or the bit allocation is not adjusted according to external control parameters, resulting in the failure to reconstruct the desired sound field.

[0184] To this end, the embodiments of the present disclosure provide a signal processing method and apparatus thereof, which can solve the problem in related arts that the coding and decoding methods cannot reconstruct the desired and most suitable sound field, thereby improving the reconstruction effect of mixed-format audio signals.

[0185] FIG2A is an interactive diagram illustrating a signal processing method according to an embodiment of the present disclosure. As shown in FIG2A , the signal processing method according to an embodiment of the present disclosure can be applied to a communication system 100 , and the method includes but is not limited to the following steps.

[0186] In step S2101 , the encoding end device 101 obtains a mixed format audio signal (Mixed Format Audio, MFA).

[0187] In some embodiments, the encoding end device 101 may be a terminal or a base station. A terminal may be a device that provides voice and / or data connectivity to a user. The terminal may communicate with one or more core networks via a RAN (Radio Access Network). The terminal may be an IoT terminal, such as a sensor device, a mobile phone (or "cellular" phone), and a computer with an IoT terminal. For example, the terminal may be a fixed, portable, pocket-sized, handheld, computer-built-in, or vehicle-mounted device. For example, a station (STA), subscriber unit, subscriber station, mobile station, mobile, remote station, access point, remote terminal, access terminal, user terminal, or user agent. Alternatively, the terminal may be a device of an unmanned aerial vehicle. Alternatively, the terminal may be a vehicle-mounted device, such as a driving computer with wireless communication capabilities, or a wireless terminal connected to an external driving computer. Alternatively, the UE may also be a roadside device, for example, a street lamp, a signal lamp, or other roadside device with a wireless communication function.

[0188] In some embodiments, the audio signal in the mixed format includes an audio signal in at least one of the following formats: a channel-based audio signal; an object-based audio signal; a scene-based audio signal; or a metadata-based three-dimensional audio signal.

[0189] In this embodiment, the audio signal in the mixed format may include any one of a channel-based audio signal, an object-based audio signal, a scene-based audio signal, and a metadata-based three-dimensional audio signal.

[0190] In this embodiment, the audio signal in the mixed format may include an object-based audio signal and a scene-based audio signal.

[0191] In this embodiment, the audio signal in the mixed format may include an object-based audio signal and a channel-based audio signal.

[0192] In this embodiment, the audio signal in the mixed format may include a channel-based audio signal and a scene-based audio signal.

[0193] In this embodiment, the audio signal in the hybrid format may include an object-based audio signal and a metadata-based three-dimensional audio signal.

[0194] In this embodiment, the audio signal in the mixed format may include a channel-based audio signal and a metadata-based three-dimensional audio signal.

[0195] It should be noted that the above embodiments are not exhaustive and are only illustrations of some embodiments. The above embodiments can be implemented individually or in combination. The above embodiments are only for illustration and are not intended to be a specific limitation on the scope of protection of the embodiments of the present disclosure.

[0196] In some embodiments, the four audio signal formats are specifically divided based on the signal acquisition format, and the application scenarios focused on by audio signals in different formats may also be different.

[0197] Specifically, in one embodiment of the present disclosure, the main application scenario of the above-mentioned channel-based audio signal can be: the acquisition end and the playback end are pre-set with the same microphone acquisition layout and speaker playback layout, respectively, wherein the microphone of the acquisition end can be used to collect 5.0 format channel-based audio signals; the speaker of the playback end can play back the 5.0 format channel-based audio signals collected by the acquisition end.

[0198] In another embodiment of the present disclosure, the above-mentioned object-based audio signal is usually recorded by using an independent microphone to record the sound of the sound-emitting object. Its main application scenario is: independent control operations need to be performed on this audio signal at the playback end, such as sound switch, volume adjustment, sound and image orientation adjustment, frequency band equalization processing and other control operations.

[0199] In another embodiment of the present disclosure, the main application scenario of the above-mentioned scene-based audio signal may be: the need to record the complete sound field where the acquisition end is located, such as live recording of a concert, live recording of a football game, etc.

[0200] In some embodiments, the "collection end" and the "encoding end device 101" can be located on the same device or on different devices. For example, if the "collection end" has encoding capabilities, then the "collection end" and the "encoding end device 101" can be located on the same device, or can be interchangeable. For another example, if the "collection end" does not have encoding capabilities, then the "collection end" and the "encoding end device 101" can be located on different devices, such as the "encoding end device 101" being located on a network device.

[0201] In some embodiments, the "playback end" and the "decoding end device 102" can be located on the same device or on different devices. For example, if the "playback end" has decoding functionality, then the "playback end" and the "decoding end device 102" can be located on the same device, or can be interchangeable. For another example, if the "playback end" does not have decoding functionality, then the "playback end" and the "decoding end device 101" can be located on different devices, such as the "decoding end device 102" being located on a network device.

[0202] In some embodiments, the mixed-format audio signal may include multiple channel audio signals. In one implementation, the channel-based audio signal may include one or more channels; the object-based audio signal may include one or more channels; the scene-based audio signal may include one or more channels; and the metadata-based 3D audio signal may include one or more channels.

[0203] In step S2102 , the encoding end device 101 performs filtering processing on the audio signal in the mixed format.

[0204] In some embodiments, the encoding end device 101 may perform high-pass filtering on the mixed-format audio signal. Optionally, the filter cutoff frequency may be set to 20 Hz. As an example, the filter formula used may be as shown in the following formula (1):

[0205] Among them, H 20 is a filter with a cutoff frequency of 20 Hz, Z is an audio signal in a mixed format, a1, a2, b0, b1, and b2 are all preset constants. For example, b0 = 0.9981492, b1 = -1.9963008, b2 = 0.9981498, a1 = 1.9962990, and a2 = -0.9963056.

[0206] In step S2103 , the encoding device 101 determines a sound field analysis parameter of the audio signal in the mixed format.

[0207] In some embodiments, the above-mentioned sound field analysis parameters include at least one of the following: the number of sound images; the priority of sound images; and background ambient sound. In one implementation, the above-mentioned sound field analysis parameters include any one of the number of sound images, the priority of sound images, and background ambient sound. In another implementation, the above-mentioned sound field analysis parameters include any two of the number of sound images, the priority of sound images, and background ambient sound. In yet another implementation, the above-mentioned sound field analysis parameters include the number of sound images, the priority of sound images, and background ambient sound.

[0208] In some embodiments, the encoding end device 101 can directly perform sound field analysis on the input mixed format audio signal to determine the sound field analysis parameters of the mixed format audio signal. Alternatively, in some embodiments, the encoding end device 101 can perform sound field analysis on the mixed format audio signal after filtering (such as high-pass filtering) to determine the above-mentioned sound field analysis parameters. Among them, sound field analysis refers to analyzing the number of sound images (images) in the mixed format audio signal, the priority of the sound images, the background ambient sound, etc.

[0209] In some embodiments, a sound field analysis can be performed on a mixed-format audio signal based on DOA (Direction-Of-Arrival) estimation and / or MCCC (Multichannel Cross-Correlation Coefficients) method to obtain sound field analysis parameters of the mixed-format audio signal. For example, DOA estimation can be used to estimate sound source localization, including azimuth and elevation angles, and the MCCC method can be used to solve the problem. The method can be used to identify sound field analysis parameters of the mixed-format audio signal, such as the number of sound images, the primary and secondary sound images, and background ambient sound.

[0210] In step S2104 , the encoding end device 101 determines energy parameters of one or more channels among the multiple channels.

[0211] In some embodiments, the encoding end device 101 can directly perform energy calculation on the input mixed format audio signal to determine the energy parameters of one or more channels among the above-mentioned multiple channels. Exemplarily, the encoding end device 101 calculates the cross-correlation coefficients between each channel among the multiple channels; performs a channel combination operation based on the cross-correlation coefficients, and calculates inter-channel parameters for the channels within the same channel combination; uses the inter-channel parameters to align the channels and then perform a downmix process, and performs energy calculation on the downmixed channels to obtain the energy parameters of each channel.

[0212] In some embodiments, the encoding end device 101 can perform energy calculation on the mixed format audio signal after filtering (such as high-pass filtering) to determine the energy parameters of one or more channels among the above-mentioned multiple channels. Exemplarily, after filtering the multiple channels, the encoding end device 101 calculates the cross-correlation coefficients between the multiple channels; performs channel combination operations based on the cross-correlation coefficients, and calculates inter-channel parameters for the channels in the same channel combination; uses the inter-channel parameters to align the channels and then perform downmixing, and performs energy calculation on the downmixed channels to obtain energy parameters for each channel.

[0213] In some embodiments, the inter-channel parameter may be one or more of ILD (Interaural Level Difference), ITD (Interaural Time Difference), IPD (Interaural Phase Difference), and the like.

[0214] In some embodiments, the energy parameter calculation method may include performing any of the following processing on the sample points of the channel frame: calculating the amplitude sum of the sample points; calculating the amplitude sum of the sample points; and calculating the root mean square of the sample points.

[0215] Exemplarily, the amplitude sum of the samples of a frame of channels after downmixing can be calculated to achieve energy calculation. For example, the amplitude sum of the samples of each channel frame is calculated, and the obtained amplitude sum of the samples of each channel frame is determined as the energy parameter of the corresponding channel. For example, assuming that the audio signal in the mixed format includes 3 channels, such as channel a, channel b and channel c, the amplitude sum of the samples of the 3 channels frame is calculated respectively, and the amplitude sum of the samples of channel a frame is determined as the energy parameter of channel a, the amplitude sum of the samples of channel b frame is determined as the energy parameter of channel b, and the amplitude sum of the samples of channel c frame is determined as the energy parameter of channel c. As an example, the calculation formula for the amplitude sum of the squares of the above-mentioned samples is expressed as follows:

[0216] Among them, S m is the amplitude value and the sample point of one frame of the mth channel. One frame of the mth channel includes N sample points. X m is the mth channel.

[0217] Exemplarily, the amplitude sum of the sample points of a frame of channels after downmixing can be calculated to achieve energy calculation. For example, the amplitude sum of the sample points of a frame of each channel is calculated, and the obtained amplitude sum of the sample points of a frame of each channel is determined as the energy parameter of the corresponding channel. For example, assuming that the audio signal in the mixed format includes 3 channels, such as channel a, channel b and channel c, the amplitude sum of the sample points of a frame of the 3 channels is calculated respectively, and the amplitude sum of the sample points of a frame of channel a is determined as the energy parameter of channel a, the amplitude sum of the sample points of b frame of channel b is determined as the energy parameter of channel b, and the amplitude sum of the sample points of c frame of channel c is determined as the energy parameter of channel c. As an example, the calculation formula for the amplitude sum of the sample points is expressed as follows:

[0218] Exemplarily, the root mean square of the samples of a frame of channels after downmixing can be calculated to achieve energy calculation. For example, the root mean square of the samples of each channel frame is calculated, and the root mean square of the samples of each channel frame obtained is determined as the energy parameter of the corresponding channel. For example, assuming that the audio signal in the mixed format includes 3 channels, such as channel a, channel b and channel c, the root mean square of the samples of the 3 channels frame is calculated respectively, and the root mean square of the samples of channel a frame obtained is determined as the energy parameter of channel a, the root mean square of the samples of channel b frame obtained is determined as the energy parameter of channel b, and the root mean square of the samples of channel c frame obtained is determined as the energy parameter of channel c. As an example, the calculation formula of the root mean square of the above-mentioned samples is expressed as follows:

[0219] Optionally, the above energy parameters may also be calculated and obtained using other methods, which are not limited in this disclosure and will not be described in detail.

[0220] In step S2105 , the encoding end device 101 performs bit allocation on the multiple channels according to the sound field analysis parameters of the mixed-format audio signal and the energy parameters of one or more of the multiple channels to obtain bit allocation parameters.

[0221] In some embodiments, the bit allocation parameter is used to describe the number of bits allocated to each of the multiple channels.

[0222] In some embodiments, the encoding end device 101 can allocate bits to the above multiple channels using the available number of bits based on the sound field analysis parameters of the mixed format audio signal and the energy parameters of one or more channels, thereby obtaining the bit allocation parameters.

[0223] In some embodiments, taking the above-mentioned mixed format audio signal including an object-based audio signal and a scene-based audio signal as an example, the encoding end device 101 can allocate bits to multiple channels in the mixed format audio signal composed of the object-based audio signal and the scene-based audio signal based on the sound field analysis parameters of the mixed format audio signal and the energy parameters of one or more channels in the multiple channels to obtain bit allocation parameters. For example, the object-based audio signal is the main sound element in the sound field, and the scene-based audio signal is the environmental background sound element in the sound field. The object-based audio signal is required to have lower distortion in the expected reconstructed sound field, while the scene-based audio signal is the environmental background sound and can allow a certain degree of distortion. Therefore, when allocating bits, more bits can be allocated to the object-based audio signal than to the scene-based audio signal. For example, the number of bits allocated to any channel in the object-based audio signal is greater than the number of bits allocated to any channel in the scene-based audio signal. The encoding end device 101 can also adjust the number of bits allocated to the corresponding channel based on the energy parameters of one or more channels in the mixed format audio signal. For example, the channel with a larger energy parameter is allocated a relatively larger number of bits.

[0224] In some embodiments, taking the above-mentioned mixed format audio signal including an object-based audio signal and a channel-based audio signal as an example, the encoding end device 101 can allocate bits to multiple channels in the mixed format audio signal composed of the object-based audio signal and the channel-based audio signal based on the sound field analysis parameters of the mixed format audio signal and the energy parameters of one or more channels in the multiple channels to obtain bit allocation parameters. For example, the object-based audio signal is the main sound element in the sound field, and the channel-based audio signal is the environmental background sound element in the sound field. The object-based audio signal is required to have a smaller distortion in the expected reconstructed sound field, while the channel-based audio signal is the environmental background sound and can allow a certain degree of distortion. Therefore, when allocating bits, more bits can be allocated to the object-based audio signal than to the channel-based audio signal. For example, the number of bits allocated to any channel in the object-based audio signal is more than the number of bits allocated to any channel in the channel-based audio signal. The encoding end device 101 may also adjust the number of bits allocated to the corresponding channel based on the energy parameters of each channel in the mixed format audio signal. For example, a channel with a larger energy parameter is allocated relatively more bits.

[0225] In some embodiments, taking the above-mentioned mixed format audio signal including a scene-based audio signal and a channel-based audio signal as an example, the encoding end device 101 can allocate bits to multiple channels in the mixed format audio signal composed of the scene-based audio signal and the channel-based audio signal according to the sound field analysis parameters of the mixed format audio signal and the energy parameters of one or more channels in the multiple channels to obtain bit allocation parameters. For example, the channel-based audio signal is the main sound element in the sound field, and the scene-based audio signal is the environmental background sound element in the sound field. The expected reconstructed sound field requires that the channel-based audio signal has lower distortion, while the scene-based audio signal is the environmental background sound and can allow a certain degree of distortion. Therefore, when allocating bits, more bits can be allocated to the channel-based audio signal than to the scene-based audio signal. Exemplarily, the number of bits allocated to any channel in the channel-based audio signal is more than the number of bits allocated to any channel in the scene-based audio signal. The encoding end device 101 may also adjust the number of bits allocated to the corresponding channels based on the energy parameters of one or more channels in the mixed format audio signal. For example, channels with larger energy parameters are allocated relatively more bits.

[0226] In some embodiments, taking the above-mentioned mixed-format audio signal including an object-based audio signal and a metadata-based three-dimensional audio signal as an example, the encoding end device 101 can allocate bits to multiple channels in the mixed-format audio signal composed of the object-based audio signal and the metadata-based three-dimensional audio signal based on the sound field analysis parameters of the mixed-format audio signal and the energy parameters of one or more of the multiple channels to obtain bit allocation parameters. For example, the object-based audio signal is the main sound element in the sound field, and the metadata-based three-dimensional audio signal is the ambient background sound element in the sound field. The object-based audio signal is required to have lower distortion in the expected reconstructed sound field, while the metadata-based three-dimensional audio signal, as the ambient background sound, can tolerate a certain degree of distortion. Therefore, when allocating bits, more bits can be allocated to the object-based audio signal than to the metadata-based three-dimensional audio signal. For example, the number of bits allocated to any channel in the object-based audio signal is greater than the number of bits allocated to any channel in the metadata-based three-dimensional audio signal. The encoding end device 101 may also adjust the number of bits allocated to the corresponding channels based on the energy parameters of one or more channels in the mixed format audio signal. For example, channels with larger energy parameters are allocated relatively more bits.

[0227] In some embodiments, taking the above-mentioned mixed-format audio signal including a metadata-based 3D audio signal and a channel-based audio signal as an example, the encoding end device 101 can allocate bits to multiple channels in the mixed-format audio signal composed of the metadata-based 3D audio signal and the channel-based audio signal based on the sound field analysis parameters of the mixed-format audio signal and the energy parameters of one or more of the multiple channels to obtain bit allocation parameters. For example, the channel-based audio signal is the main sound element in the sound field, and the metadata-based 3D audio signal is the ambient background sound element in the sound field. The channel-based audio signal is required to have lower distortion in the expected reconstructed sound field, while the metadata-based 3D audio signal, as the ambient background sound, can tolerate a certain degree of distortion. Therefore, when allocating bits, more bits can be allocated to the channel-based audio signal than to the metadata-based 3D audio signal. For example, the number of bits allocated to any channel in the channel-based audio signal is greater than the number of bits allocated to any channel in the metadata-based 3D audio signal. The encoding end device 101 may also adjust the number of bits allocated to the corresponding channels based on the energy parameters of one or more channels in the mixed format audio signal. For example, channels with larger energy parameters are allocated relatively more bits.

[0228] In some embodiments, the number of bits allocated to the channel corresponding to the primary sound image in the object-based audio signal is greater than the number of bits allocated to the channel corresponding to the secondary sound image in the object-based audio signal. In other words, in the object-based audio signal, the priority of each object signal in the mixed audio signal is different. For example, at a concert, the priority of the lead singer's voice signal is higher than that of the backing singer's voice signal, or in other words, the lead singer's voice signal is more important than that of the backing singer's voice signal. Therefore, the lead singer's object audio signal will be allocated more bits than the backing singer's voice signal.

[0229] It should be noted that the mixed format audio signal composition method shown in the above embodiment is to facilitate technical personnel in this field to understand how to allocate bits to each channel to be encoded according to the sound field analysis parameters and energy parameters. That is to say, the bit allocation of the mixed format audio signal composed of signals in other formats can also be carried out according to the above sound field analysis parameters and energy parameters, which will not be repeated here.

[0230] In some embodiments, the number of bits allocated to each channel in the audio signals of different formats in the mixed format audio signal may be evenly distributed, or the number of bits may be allocated based on other factors (such as primary and secondary factors). This disclosure does not limit this and will not elaborate on this.

[0231] In step S2106, the encoding end device 101 encodes the multiple channels according to the bit allocation parameters to obtain channel signal encoding parameters, and writes the bit allocation parameters and the channel signal encoding parameters into the bitstream.

[0232] In some embodiments, the encoding end device 101 may encode multiple channels using a corresponding encoding core based on the bit allocation parameter to obtain channel signal encoding parameters, and write the bit allocation parameter and the channel signal encoding parameters into a bitstream.

[0233] In some embodiments, the encoding cores corresponding to audio signals of different formats may be the same or different. Exemplarily, all channels in a mixed-format audio signal may be encoded using one encoding core based on a bit allocation parameter. Or, exemplarily, the mixed-format audio signal includes two audio signals of different formats, such as an object-based audio signal and a scene-based audio signal. Assume that the object-based audio signal includes channel a and channel b, and the scene-based audio signal includes channel c. The bit allocation parameters of channel a, channel b, and channel c are 3, 2, and 1, that is, the number of bits allocated to channel a is 3, the number of bits allocated to channel b is 2, and the number of bits allocated to channel c is 1. The encoding end device 101 may encode channel a based on 3 bits using the encoding core corresponding to the object-based audio signal, encode channel b based on 2 bits using the encoding core corresponding to the object-based audio signal, and encode channel c based on 1 bit using the encoding core corresponding to the scene-based audio signal.

[0234] In some embodiments, each channel in the mixed-format audio signal corresponds to a coding core, and the coding cores corresponding to each channel can be the same or different. In this way, each channel in the mixed-format audio signal can be encoded using the corresponding coding core based on the allocated number of bits.

[0235] Step S2107: the encoding end device 101 sends a code stream.

[0236] In some embodiments, the encoding end device 101 may send a code stream to the decoding end device 102 , or in other words, the encoding end device 101 sends the code stream to the decoding end device 102 .

[0237] In some embodiments, encoding device 101 may transmit a code stream to decoding device 102 based on a multiplex (MUX) method. In some embodiments, decoding device 102 receives the code stream. Alternatively, decoding device 102 receives the code stream transmitted by encoding device 101. The code stream is used by decoding device 102 to reconstruct the audio signal in the mixed format.

[0238] In step S2108 , the decoding end device 102 parses the received code stream to obtain a bit allocation parameter and a channel signal coding parameter for each of the multiple channels of the mixed-format audio signal.

[0239] In step S2109 , the decoding end device 102 reconstructs the audio signal in a mixed format based on the bit allocation parameters of each channel and the channel signal encoding parameters.

[0240] In some embodiments, the decoding end device 102 decodes the channel signal encoding parameters based on the bit allocation parameters of each channel; the decoding end device 102 uses the decoded channels to reconstruct the mixed format audio signal. In some embodiments, the decoding end device 102 decodes the channel signal encoding parameters based on the bit allocation parameters of each channel; the decoding end device 102 uses the decoded channels and bitstream to reconstruct the mixed format audio signal.

[0241] In some embodiments, the names of information, etc. are not limited to the names described in the embodiments, and terms such as "information", "message", "signal", "signaling", "report", "configuration", "indication", "instruction", "command", "channel", "parameter", "domain", "field", "symbol", "symbol", "codeword", "codebook", "codeword", "codepoint", "bit", "data", "program", and "chip" can be used interchangeably.

[0242] In some embodiments, "obtain", "get", "get", "receive", "transmit", "bidirectional transmission", "send and / or receive" can be interchangeable, and can be interpreted as receiving from other entities, obtaining from protocols, obtaining from higher layers, obtaining by self-processing, autonomous implementation, etc.

[0243] In some embodiments, terms such as "send", "transmit", "report", "download", "transmit", "bidirectional transmission", "send and / or receive" can be used interchangeably.

[0244] In some embodiments, "bit" and "bit number" can be used interchangeably.

[0245] In some embodiments, "encoding end device", "encoder", and "encoding end" can be used interchangeably.

[0246] In some embodiments, "decoding end device", "decoder" and "decoding end" can be used interchangeably.

[0247] In some embodiments, the terms "mixed-format audio signal" and "mixed-format audio signal" may be used interchangeably.

[0248] In some embodiments, “channel audio signal,” “channel signal,” and “channel” may be used interchangeably.

[0249] In some embodiments, “spatial audio signal based on auxiliary metadata” and “three-dimensional audio signal based on metadata” may be used interchangeably.

[0250] In some embodiments, "code stream" and "encoded code stream" may be used interchangeably, wherein the encoded code stream may refer to a code stream including the above-mentioned bit allocation parameters and the above-mentioned channel signal encoding parameters.

[0251] The method involved in the embodiment of the present disclosure may include at least one of steps S2101 to S2109. For example, step S2101+step S2102+step S2103+step S2104+step S2105 can be implemented as an independent embodiment, step S2101+step S2103+step S2104+step S2105 can be implemented as an independent embodiment, step S2101+step S2102+step S2103+step S2104+step S2105+step S2106+step S2107 can be implemented as an independent embodiment, step S2101+step S2103+step S2104+step S2105+step S2106+step S2107 can be implemented as an independent embodiment, and step S2101+step S2103+step S2104+step S2105+step S2106+step S2107 can be implemented as an independent embodiment. 2107 can be implemented as an independent embodiment, step S2108 + step S2109 can be implemented as an independent embodiment, step S2101 + step S2102 + step S2103 + step S2104 + step S2105 + step S2106 + step S2107 + step S2108 + step S2109 can be implemented as an independent embodiment, step S2101 + step S2103 + step S2104 + step S2105 + step S2106 + step S2107 + step S2108 + step S2109 can be implemented as an independent embodiment, but are not limited to this.

[0252] In some embodiments, step S2103 and step S2104 may be executed in an interchanged order or simultaneously.

[0253] In some embodiments, step S2106, step S2107, step S2108, and step S2109 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0254] In some embodiments, step S2102, step S2106, step S2107, step S2108, and step S2109 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0255] In some embodiments, step S2108 and step S2109 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0256] In some embodiments, step S2102, step S2108, and step S2109 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0257] In some embodiments, step S2101, step S2102, step S2103, step S2104, step S2105, step S2106, and step S2107 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0258] In some embodiments, step S2102 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0259] In some embodiments, reference may be made to other optional implementations described before or after the description corresponding to FIG. 2A .

[0260] FIG2B is an interactive diagram illustrating a signal processing method according to an embodiment of the present disclosure. As shown in FIG2B , the signal processing method according to an embodiment of the present disclosure can be applied to a communication system 100, and the method includes but is not limited to the following steps.

[0261] In step S2201 , the encoding device 101 obtains an audio signal in a mixed format.

[0262] The optional implementation of step S2201 can refer to the optional implementation of step S2101 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0263] In step S2202 , the encoding end device 101 performs filtering processing on the audio signal in the mixed format.

[0264] The optional implementation of step S2202 can refer to the optional implementation of step S2102 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0265] In step S2203 , the encoding device 101 determines control parameters of the audio signal in the mixed format.

[0266] In some embodiments, the control parameters may include a second parameter and / or a third parameter. In some embodiments, the second parameter is used to describe the priority ranking of the roles played by each of the multiple channels in the mixed audio signal. Optionally, the priorities may be manually set or obtained through processing other than encoding.

[0267] In some embodiments, the third parameter is used to describe whether each of the above sound channels is diegetic sound or non-diegetic sound.

[0268] In some embodiments, the encoding end device 101 can determine the control parameters of the mixed-format audio signal based on the functional division of each channel during acquisition by the audio signal acquisition device, wherein the audio signal acquisition device is used to acquire the mixed-format audio signal. In some embodiments, the "audio signal acquisition device" and "acquisition end" can be interchangeable.

[0269] In some embodiments, the encoding end device 101 may determine the control parameters of the audio signal in the mixed format according to the setting parameters required by the desired sound field reconstructed by the decoding end device 102 .

[0270] In some embodiments, the encoding end device 101 can determine the control parameters of the audio signal in the mixed format according to the functional division of each channel during acquisition by the audio signal acquisition device and the setting parameters required by the decoding end device 102 to reconstruct the desired sound field.

[0271] Exemplarily, the above-mentioned control parameters refer to the priority ranking of the roles played by each channel in the mixed format audio signal in advance according to external information. The basis for this ranking can be determined according to the functional division of each channel during acquisition, or it can be set according to the desired sound field reconstructed at the playback end. For example, the voice of the lead singer in a concert can be set as the most important channel. For another example, in order to focus on the sound of a certain instrument (such as the sound of a violin) in the reconstructed sound field, the sound of the violin can be set as the most important channel signal.

[0272] In step S2204 , the encoding end device 101 determines energy parameters of one or more channels among the multiple channels.

[0273] The optional implementation of step S2204 can refer to the optional implementation of step S2104 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0274] In step S2205 , the encoding end device 101 performs bit allocation on the multiple channels according to the energy parameters of the one or more channels and the control parameters of the mixed-format audio signal to obtain a bit allocation parameter.

[0275] In some embodiments, the bit allocation parameter is used to describe the number of bits allocated to each of the multiple channels.

[0276] In some embodiments, the encoding end device 101 may allocate bits to the multiple channels using available bits according to the control parameters of the mixed-format audio signal and the energy parameters of each of the multiple channels, thereby obtaining a bit allocation parameter.

[0277] In some embodiments, taking the above-mentioned mixed format audio signal including an object-based audio signal and a scene-based audio signal as an example, the encoding end device 101 can allocate bits to multiple channels in the mixed format audio signal composed of the object-based audio signal and the scene-based audio signal based on the control parameters of the mixed format audio signal and the energy parameters of each of the multiple channels to obtain bit allocation parameters. For example, the voice of the lead singer in a live concert can be set as the most important channel, and the bit allocation parameters can be obtained by allocating bits to multiple channels in the mixed format audio signal composed of the object-based audio signal and the scene-based audio signal based on the priority ranking of the roles played by each channel in the mixed format audio signal and the energy parameters of each of the multiple channels. For example, the number of bits allocated to any channel in the object-based audio signal is greater than the number of bits allocated to any channel in the scene-based audio signal. The encoding end device 101 can also adjust the number of bits allocated to the corresponding channel based on the energy parameters of each channel in the mixed format audio signal, for example, the channel with a larger energy parameter is allocated a relatively larger number of bits.

[0278] In some embodiments, taking the above-mentioned mixed format audio signal including an object-based audio signal and a channel-based audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the object-based audio signal and the channel-based audio signal according to the control parameters of the mixed format audio signal and the energy parameters of each channel in the multiple channels to obtain bit allocation parameters.

[0279] In some embodiments, taking the above-mentioned mixed format audio signal including a scene-based audio signal and a channel-based audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the scene-based audio signal and the channel-based audio signal according to the control parameters of the mixed format audio signal and the energy parameters of each channel in the multiple channels to obtain bit allocation parameters.

[0280] In some embodiments, taking the above-mentioned mixed format audio signal including the object-based audio signal and the metadata-based three-dimensional audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the object-based audio signal and the metadata-based three-dimensional audio signal according to the control parameters of the mixed format audio signal and the energy parameters of each channel in the multiple channels to obtain bit allocation parameters.

[0281] In some embodiments, taking the above-mentioned mixed format audio signal including the metadata-based three-dimensional audio signal and the channel-based audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the metadata-based three-dimensional audio signal and the channel-based audio signal according to the control parameters of the mixed format audio signal and the energy parameters of each of the multiple channels to obtain bit allocation parameters.

[0282] In some embodiments, the number of bits allocated to the channel corresponding to the primary sound image in the object-based audio signal is greater than the number of bits allocated to the channel corresponding to the secondary sound image in the object-based audio signal. In other words, in the object-based audio signal, the priority of each object signal in the mixed audio signal is different. For example, at a concert, the priority of the lead singer's voice signal is greater than that of the backing singer's voice signal, or in other words, the lead singer's voice signal is more important than that of the backing singer's voice signal. Therefore, the lead singer's object audio signal will be allocated more bits than the backing singer's voice signal.

[0283] It should be noted that the method of composing the mixed-format audio signal shown in the above embodiment is intended to facilitate those skilled in the art to understand how to allocate bits to each channel to be encoded based on the control parameters and energy parameters. In other words, bit allocation can also be performed on a mixed-format audio signal composed of signals in other formats based on the above-mentioned control parameters and energy parameters, which will not be repeated here.

[0284] In some embodiments, the number of bits allocated to each channel in the audio signals of different formats in the mixed format audio signal may be evenly distributed, or the number of bits may be allocated based on other factors (such as primary and secondary factors). This disclosure does not limit this and will not elaborate on this.

[0285] In step S2206, the encoding end device 101 encodes the multiple channels according to the bit allocation parameters to obtain channel signal encoding parameters, and writes the bit allocation parameters and the channel signal encoding parameters into the bitstream.

[0286] The optional implementation of step S2206 can refer to the optional implementation of step S2106 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0287] Step S2207: the encoding end device 101 sends a code stream.

[0288] The optional implementation of step S2207 can refer to the optional implementation of step S2107 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0289] In step S2208 , the decoding end device 102 parses the received code stream to obtain a bit allocation parameter and a channel signal coding parameter for each of the multiple channels of the mixed-format audio signal.

[0290] The optional implementation of step S2208 can refer to the optional implementation of step S2108 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0291] In step S2209 , the decoding end device 102 reconstructs the audio signal in a mixed format based on the bit allocation parameters of each channel and the channel signal encoding parameters.

[0292] The optional implementation of step S2209 can refer to the optional implementation of step S2109 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0293] The method involved in the embodiment of the present disclosure may include at least one of steps S2201 to S2209. For example, step S2201+step S2202+step S2203+step S2204+step S2205 can be implemented as an independent embodiment, step S2201+step S2203+step S2204+step S2205 can be implemented as an independent embodiment, step S2201+step S2202+step S2203+step S2204+step S2205+step S2206+step S2207 can be implemented as an independent embodiment, step S2201+step S2203+step S2204+step S2205+step S2206+step S2207 can be implemented as an independent embodiment, and step S2201+step S2203+step S2204+step S2205+step S2206+step S2207 can be implemented as an independent embodiment. 2207 can be implemented as an independent embodiment, step S2208 + step S2209 can be implemented as an independent embodiment, step S2201 + step S2202 + step S2203 + step S2204 + step S2205 + step S2206 + step S2207 + step S2208 + step S2209 can be implemented as an independent embodiment, step S2201 + step S2203 + step S2204 + step S2205 + step S2206 + step S2207 + step S2208 + step S2209 can be implemented as an independent embodiment, but is not limited to this.

[0294] In some embodiments, step S2203 and step S2204 may be executed in an interchangeable order or simultaneously.

[0295] In some embodiments, step S2206, step S2207, step S2208, and step S2209 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0296] In some embodiments, step S2202, step S2206, step S2207, step S2208, and step S2209 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0297] In some embodiments, step S2208 and step S2209 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0298] In some embodiments, step S2202, step S2208, and step S2209 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0299] In some embodiments, step S2201, step S2202, step S2203, step S2204, step S2205, step S2206, and step S2207 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0300] In some embodiments, step S2202 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0301] In some embodiments, reference may be made to other optional implementations described before or after the description corresponding to FIG. 2B .

[0302] FIG2C is an interactive diagram illustrating a signal processing method according to an embodiment of the present disclosure. As shown in FIG2C , the signal processing method according to an embodiment of the present disclosure can be applied to a communication system 100 , and the method includes but is not limited to the following steps.

[0303] In step S2301 , the encoding device 101 obtains an audio signal in a mixed format.

[0304] The optional implementation of step S2301 can refer to the optional implementation of step S2101 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0305] In step S2302 , the encoding device 101 performs filtering processing on the audio signal in the mixed format.

[0306] The optional implementation of step S2302 can refer to the optional implementation of step S2102 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0307] Step S2303: The encoding end device 101 determines a sound field analysis parameter and / or a control parameter of the audio signal in the mixed format.

[0308] In some embodiments, the encoding end device 101 determines the sound field analysis parameters of the mixed format audio signal. Optional implementations thereof can be found in the optional implementations of step S2103 of FIG. 2A and other related parts of the embodiment involved in FIG. 2A , which will not be described in detail here.

[0309] In some embodiments, the encoding end device 101 determines the control parameters of the mixed format audio signal. Optional implementations thereof can be found in the optional implementations of step S2203 of FIG. 2B and other related parts of the embodiment involved in FIG. 2B , which will not be described in detail here.

[0310] In some embodiments, the encoding end device 101 determines the sound field analysis parameters and control parameters of the mixed-format audio signal. Optional implementations thereof can be found in the optional implementation of step S2103 of FIG. 2A , the optional implementation of step S2203 of FIG. 2B , and other related portions of the embodiment involved in FIG. 2A , and other related portions of the embodiment involved in FIG. 2B , which are not further described here.

[0311] In step S2304, the encoding end device 101 performs bit allocation on multiple channels according to the sound field analysis parameters and / or control parameters of the audio signal in the mixed format to obtain bit allocation parameters.

[0312] In some embodiments, the encoding end device 101 allocates bits to the multiple channels using available bits according to the sound field analysis parameters and / or control parameters of the mixed-format audio signal, thereby obtaining a bit allocation parameter.

[0313] In some embodiments, the encoding end device 101 performs bit allocation on multiple channels according to the sound field analysis parameters of the audio signal in the mixed format to obtain bit allocation parameters.

[0314] In one implementation, taking the above-mentioned mixed format audio signal including an object-based audio signal and a scene-based audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels of the mixed format audio signal composed of the object-based audio signal and the scene-based audio signal according to the sound field analysis parameters of the mixed format audio signal to obtain bit allocation parameters. For example, the object-based audio signal is the main sound element in the sound field, and the scene-based audio signal is the environmental background sound element in the sound field. The object-based audio signal is required to have a smaller degree of distortion in the expected reconstructed sound field, while the scene-based audio signal is the environmental background sound and can allow a certain degree of distortion. Therefore, when allocating bits, more bits can be allocated to the object-based audio signal than to the scene-based audio signal. Exemplarily, the number of bits allocated to any channel in the object-based audio signal is greater than the number of bits allocated to any channel in the scene-based audio signal.

[0315] In another implementation, taking the above-mentioned mixed format audio signal including an object-based audio signal and a channel-based audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the object-based audio signal and the channel-based audio signal according to the sound field analysis parameters of the mixed format audio signal to obtain bit allocation parameters. For example, the object-based audio signal is the main sound element in the sound field, and the channel-based audio signal is the environmental background sound element in the sound field. The object-based audio signal is required to have a smaller distortion in the expected reconstructed sound field, while the channel-based audio signal is the environmental background sound and can allow a certain degree of distortion. Therefore, when allocating bits, more bits can be allocated to the object-based audio signal than to the channel-based audio signal. For example, the number of bits allocated to any channel in the object-based audio signal is more than the number of bits allocated to any channel in the channel-based audio signal.

[0316] In another implementation, taking the above-mentioned mixed format audio signal including a scene-based audio signal and a channel-based audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the scene-based audio signal and the channel-based audio signal according to the sound field analysis parameters of the mixed format audio signal to obtain bit allocation parameters. For example, the channel-based audio signal is the main sound element in the sound field, and the scene-based audio signal is the environmental background sound element in the sound field. The expected reconstructed sound field requires that the channel-based audio signal has a lower distortion, while the scene-based audio signal is the environmental background sound and can allow a certain degree of distortion. Therefore, when allocating bits, more bits can be allocated to the channel-based audio signal than to the scene-based audio signal. For example, the number of bits allocated to any channel in the channel-based audio signal is more than the number of bits allocated to any channel in the scene-based audio signal.

[0317] In another implementation, taking the above-mentioned mixed-format audio signal including an object-based audio signal and a metadata-based three-dimensional audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels of the mixed-format audio signal composed of the object-based audio signal and the metadata-based three-dimensional audio signal based on the sound field analysis parameters of the mixed-format audio signal to obtain bit allocation parameters. For example, the object-based audio signal is the primary sound element in the sound field, and the metadata-based three-dimensional audio signal is the ambient background sound element in the sound field. The object-based audio signal is required to have lower distortion in the expected reconstructed sound field, while the metadata-based three-dimensional audio signal, as the ambient background sound, can tolerate a certain degree of distortion. Therefore, when allocating bits, more bits can be allocated to the object-based audio signal than to the metadata-based three-dimensional audio signal. For example, the number of bits allocated to any channel in the object-based audio signal is greater than the number of bits allocated to any channel in the metadata-based three-dimensional audio signal.

[0318] In another implementation, taking the mixed-format audio signal comprising a metadata-based 3D audio signal and a channel-based audio signal as an example, the encoding end device 101 may allocate bits to multiple channels in the mixed-format audio signal composed of the metadata-based 3D audio signal and the channel-based audio signal based on the sound field analysis parameters of the mixed-format audio signal to obtain bit allocation parameters. For example, the channel-based audio signal is the primary sound element in the sound field, and the metadata-based 3D audio signal is the ambient background sound element in the sound field. The channel-based audio signal is required to have lower distortion in the expected reconstructed sound field, while the metadata-based 3D audio signal, as the ambient background sound, can tolerate a certain degree of distortion. Therefore, when allocating bits, more bits may be allocated to the channel-based audio signal than to the metadata-based 3D audio signal. For example, the number of bits allocated to any channel in the channel-based audio signal is greater than the number of bits allocated to any channel in the metadata-based 3D audio signal.

[0319] In some embodiments, the number of bits allocated to the channel corresponding to the primary sound image in the object-based audio signal is greater than the number of bits allocated to the channel corresponding to the secondary sound image in the object-based audio signal. In other words, in the object-based audio signal, the priority of each object signal in the mixed audio signal is different. For example, at a concert, the priority of the lead singer's voice signal is greater than that of the backing singer's voice signal, or in other words, the lead singer's voice signal is more important than that of the backing singer's voice signal. Therefore, the lead singer's object audio signal will be allocated more bits than the backing singer's voice signal.

[0320] It should be noted that the mixed format audio signal composition method shown in the above embodiment is to facilitate technical personnel in this field to understand how to allocate bits to each channel to be encoded according to the sound field analysis parameters. In other words, the bit allocation of the mixed format audio signal composed of signals in other formats can also be carried out according to the above sound field analysis parameters, which will not be repeated here.

[0321] In some embodiments, the encoding end device 101 may perform bit allocation on multiple channels according to the control parameters of the mixed format audio signal to obtain bit allocation parameters.

[0322] In one implementation, taking the above-mentioned mixed format audio signal including the object-based audio signal and the scene-based audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the object-based audio signal and the scene-based audio signal according to the control parameters of the mixed format audio signal to obtain bit allocation parameters.

[0323] In another implementation, taking the above-mentioned mixed format audio signal including an object-based audio signal and a channel-based audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the object-based audio signal and the channel-based audio signal according to the control parameters of the mixed format audio signal to obtain bit allocation parameters.

[0324] In another implementation, taking the above-mentioned mixed format audio signal including a scene-based audio signal and a channel-based audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the scene-based audio signal and the channel-based audio signal according to the control parameters of the mixed format audio signal to obtain bit allocation parameters.

[0325] In another implementation, taking the above-mentioned mixed format audio signal including the object-based audio signal and the metadata-based three-dimensional audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the object-based audio signal and the metadata-based three-dimensional audio signal according to the control parameters of the mixed format audio signal to obtain bit allocation parameters.

[0326] In another implementation, taking the above-mentioned mixed format audio signal including the metadata-based three-dimensional audio signal and the channel-based audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the metadata-based three-dimensional audio signal and the channel-based audio signal according to the control parameters of the mixed format audio signal to obtain bit allocation parameters.

[0327] It should be noted that the method of composing the mixed-format audio signal shown in the above embodiment is intended to facilitate those skilled in the art to understand how to allocate bits to each channel to be encoded according to the control parameters. In other words, the bit allocation of a mixed-format audio signal composed of signals in other formats can also be performed according to the above-mentioned control parameters, which will not be repeated here.

[0328] In some embodiments, the encoding end device 101 performs bit allocation on multiple channels according to the sound field analysis parameters and control parameters of the audio signal in the mixed format to obtain bit allocation parameters.

[0329] In one implementation, taking the above-mentioned mixed format audio signal including an object-based audio signal and a scene-based audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the object-based audio signal and the scene-based audio signal according to the sound field analysis parameters and control parameters of the mixed format audio signal to obtain bit allocation parameters.

[0330] In another implementation, taking the above-mentioned mixed format audio signal including an object-based audio signal and a channel-based audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the object-based audio signal and the channel-based audio signal according to the sound field analysis parameters and control parameters of the mixed format audio signal to obtain bit allocation parameters.

[0331] In another implementation, taking the above-mentioned mixed format audio signal including a scene-based audio signal and a channel-based audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the scene-based audio signal and the channel-based audio signal according to the sound field analysis parameters and control parameters of the mixed format audio signal to obtain bit allocation parameters.

[0332] In another implementation, taking the above-mentioned mixed format audio signal including the object-based audio signal and the metadata-based three-dimensional audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the object-based audio signal and the metadata-based three-dimensional audio signal according to the sound field analysis parameters and control parameters of the mixed format audio signal to obtain bit allocation parameters.

[0333] In another implementation, taking the above-mentioned mixed format audio signal including the metadata-based three-dimensional audio signal and the channel-based audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the metadata-based three-dimensional audio signal and the channel-based audio signal according to the sound field analysis parameters and control parameters of the mixed format audio signal to obtain bit allocation parameters.

[0334] In some embodiments, the number of bits allocated to the channel corresponding to the primary sound image in the object-based audio signal is greater than the number of bits allocated to the channel corresponding to the secondary sound image in the object-based audio signal. In other words, in the object-based audio signal, the priority of each object signal in the mixed audio signal is different. For example, at a concert, the priority of the lead singer's voice signal is greater than that of the backing singer's voice signal, or in other words, the lead singer's voice signal is more important than that of the backing singer's voice signal. Therefore, the lead singer's object audio signal will be allocated more bits than the backing singer's voice signal.

[0335] It should be noted that the mixed format audio signal composition method shown in the above embodiment is to facilitate technical personnel in this field to understand how to allocate bits to each channel to be encoded according to the sound field analysis parameters and control parameters. That is to say, the bit allocation of the mixed format audio signal composed of signals in other formats can also be carried out according to the above sound field analysis parameters and control parameters, which will not be repeated here.

[0336] In step S2305 , the encoding end device 101 encodes the multiple channels according to the bit allocation parameters to obtain channel signal encoding parameters, and writes the bit allocation parameters and the channel signal encoding parameters into the bitstream.

[0337] The optional implementation of step S2305 can refer to the optional implementation of step S2106 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0338] Step S2306: The encoding end device 101 sends a code stream.

[0339] The optional implementation of step S2306 can refer to the optional implementation of step S2107 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0340] In step S2307 , the decoding end device 102 parses the received code stream to obtain a bit allocation parameter and a channel signal coding parameter for each of the multiple channels of the mixed-format audio signal.

[0341] The optional implementation of step S2307 can refer to the optional implementation of step S2108 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0342] In step S2308 , the decoding end device 102 reconstructs the audio signal in a mixed format based on the bit allocation parameters of each channel and the channel signal encoding parameters.

[0343] The optional implementation of step S2308 can refer to the optional implementation of step S2109 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0344] The method involved in the embodiment of the present disclosure may include at least one of steps S2301 to S2308. For example, step S2301+step S2302+step S2303+step S2304 can be implemented as an independent embodiment, step S2301+step S2303+step S2304 can be implemented as an independent embodiment, step S2301+step S2302+step S2303+step S2304+step S2305+step S2306+step S2306 can be implemented as an independent embodiment, step S2301+step S2303+step S2304+step S2305+step S2306+step S2306 can be implemented as an independent embodiment, step S2301+step S2303+step S2304+step S2305+step S2306+step S2306 2306 can be implemented as an independent embodiment, step S2307 + step S2308 can be implemented as an independent embodiment, step S2301 + step S2302 + step S2303 + step S2304 + step S2305 + step S2306 + step S2307 + step S2308 can be implemented as an independent embodiment, step S2301 + step S2303 + step S2304 + step S2305 + step S2306 + step S2307 + step S2308 can be implemented as an independent embodiment, but is not limited to this.

[0345] In some embodiments, step S2305, step S2306, step S2307, and step S2308 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0346] In some embodiments, step S2302, step S2305, step S2306, step S2307, and step S2308 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0347] In some embodiments, step S2307 and step S2308 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0348] In some embodiments, step S2302, step S2307, and step S2308 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0349] In some embodiments, step S2301, step S2302, step S2303, step S2304, step S2305, and step S2306 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0350] In some embodiments, step S2302 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0351] In some embodiments, reference may be made to other optional implementations described before or after the description corresponding to FIG. 2C .

[0352] FIG2D is an interactive diagram illustrating a signal processing method according to an embodiment of the present disclosure. As shown in FIG2D , the signal processing method according to an embodiment of the present disclosure can be applied to a communication system 100, and the method includes but is not limited to the following steps.

[0353] In step S2401 , the encoding device 101 obtains an audio signal in a mixed format.

[0354] The optional implementation of step S2401 can refer to the optional implementation of step S2101 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0355] In step S2402 , the encoding device 101 performs filtering processing on the audio signal in the mixed format.

[0356] The optional implementation of step S2402 can refer to the optional implementation of step S2102 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0357] In step S2403 , the encoding device 101 determines a sound field analysis parameter of the audio signal in the mixed format.

[0358] The optional implementation of step S2403 can refer to the optional implementation of step S2103 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0359] In step S2404 , the encoding end device 101 determines energy parameters of one or more channels among the multiple channels.

[0360] The optional implementation of step S2404 can refer to the optional implementation of step S2104 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0361] In step S2405 , the encoding device 101 determines control parameters of the audio signal in the mixed format.

[0362] The optional implementation of step S2405 can refer to the optional implementation of step S2203 in Figure 2B and other related parts in the embodiment involved in Figure 2B, which will not be repeated here.

[0363] In step S2406 , the encoding end device 101 performs bit allocation on the multiple channels according to the energy parameters of the one or more channels, the control parameters of the mixed-format audio signal, and the sound field analysis parameters to obtain bit allocation parameters.

[0364] In some embodiments, the encoding end device 101 can use the available number of bits to allocate bits to the above-mentioned multiple channels according to the control parameters of the mixed format audio signal, the sound field analysis parameters and the energy parameters of each channel in the multiple channels, thereby obtaining the bit allocation parameters.

[0365] In some embodiments, taking the above-mentioned mixed format audio signal including an object-based audio signal and a scene-based audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the object-based audio signal and the scene-based audio signal according to the sound field analysis parameters, control parameters and energy parameters of each channel of the mixed format audio signal to obtain bit allocation parameters.

[0366] In some embodiments, taking the above-mentioned mixed format audio signal including an object-based audio signal and a channel-based audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the object-based audio signal and the channel-based audio signal according to the sound field analysis parameters, control parameters and energy parameters of each channel in the multiple channels of the mixed format audio signal to obtain bit allocation parameters.

[0367] In some embodiments, taking the above-mentioned mixed format audio signal including a scene-based audio signal and a channel-based audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the scene-based audio signal and the channel-based audio signal according to the sound field analysis parameters, control parameters and energy parameters of each channel of the mixed format audio signal to obtain bit allocation parameters.

[0368] In some embodiments, taking the above-mentioned mixed format audio signal including the object-based audio signal and the metadata-based three-dimensional audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the object-based audio signal and the metadata-based three-dimensional audio signal according to the sound field analysis parameters, control parameters and energy parameters of each channel of the mixed format audio signal to obtain bit allocation parameters.

[0369] In some embodiments, taking the above-mentioned mixed format audio signal including the metadata-based three-dimensional audio signal and the channel-based audio signal as an example, the encoding end device 101 can perform bit allocation on multiple channels in the mixed format audio signal composed of the metadata-based three-dimensional audio signal and the channel-based audio signal according to the sound field analysis parameters, control parameters and energy parameters of each channel of the mixed format audio signal to obtain bit allocation parameters.

[0370] In some embodiments, the number of bits allocated to the channel corresponding to the primary sound image in the object-based audio signal is greater than the number of bits allocated to the channel corresponding to the secondary sound image in the object-based audio signal. In other words, in the object-based audio signal, the priority of each object signal in the mixed audio signal is different. For example, at a concert, the priority of the lead singer's voice signal is greater than that of the backing singer's voice signal, or in other words, the lead singer's voice signal is more important than that of the backing singer's voice signal. Therefore, the lead singer's object audio signal will be allocated more bits than the backing singer's voice signal.

[0371] It should be noted that the mixed format audio signal composition method shown in the above embodiment is to facilitate technical personnel in this field to understand how to allocate bits to each channel to be encoded according to the sound field analysis parameters, control parameters and energy parameters. That is to say, the bit allocation of the mixed format audio signal composed of signals in other formats can also be carried out according to the above sound field analysis parameters, control parameters and energy parameters, which will not be repeated here.

[0372] In some embodiments, the number of bits allocated to each channel in the audio signals of different formats in the mixed format audio signal may be evenly distributed, or the number of bits may be allocated based on other factors (such as primary and secondary factors). This disclosure does not limit this and will not elaborate on this.

[0373] In step S2407 , the encoding end device 101 encodes the multiple channels according to the bit allocation parameters to obtain channel signal encoding parameters, and writes the bit allocation parameters and the channel signal encoding parameters into the bitstream.

[0374] The optional implementation of step S2407 can refer to the optional implementation of step S2106 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0375] Step S2408: The encoding end device 101 sends a code stream.

[0376] The optional implementation of step S2408 can refer to the optional implementation of step S2107 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0377] In step S2409 , the decoding end device 102 parses the received code stream to obtain a bit allocation parameter and a channel signal coding parameter for each of the multiple channels of the mixed-format audio signal.

[0378] The optional implementation of step S2409 can refer to the optional implementation of step S2108 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0379] In step S2410 , the decoding end device 102 reconstructs the audio signal in a mixed format based on the bit allocation parameters of each channel and the channel signal encoding parameters.

[0380] The optional implementation of step S2410 can refer to the optional implementation of step S2109 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0381] The method involved in the embodiment of the present disclosure may include at least one of steps S2401 to S2410. For example, step S2401+step S2402+step S2403+step S2404+step S2405+step S2406 can be implemented as an independent embodiment, step S2401+step S2403+step S2404+step S2405+step S2406 can be implemented as an independent embodiment, step S2401+step S2402+step S2403+step S2404+step S2405+step S2406+step S2407+step S2408 can be implemented as an independent embodiment, step S2401+step S2403+step S2404+step S2405+step S2406+step S2407+step S2408 can be implemented as an independent embodiment, and step S2401+step S2403+step S2404+step S2405+step S2406+step S2407 2407+step S2408 can be implemented as an independent embodiment, step S2409+step S2410 can be implemented as an independent embodiment, step S2401+step S2402+step S2403+step S2404+step S2405+step S2406+step S2407+step S2408+step S2409+step S2410 can be implemented as an independent embodiment, step S2401+step S2403+step S2404+step S2405+step S2406+step S2407+step S2408+step S2409+step S2410 can be implemented as an independent embodiment, but is not limited to this.

[0382] In some embodiments, step S2403, step S2404, and step S2405 may be executed in an interchanged order or simultaneously.

[0383] In some embodiments, step S2407, step S2408, step S2409, and step S2410 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0384] In some embodiments, step S2402, step S2407, step S2408, step S2409, and step S2410 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0385] In some embodiments, step S2409 and step S2410 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0386] In some embodiments, step S2402, step S2409, and step S2410 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0387] In some embodiments, step S2401, step S2402, step S2403, step S2404, step S2405, step S2406, step S2407, and step S2408 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0388] In some embodiments, step S2402 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0389] In some embodiments, reference may be made to other optional implementations described before or after the description corresponding to FIG. 2D .

[0390] FIG3 is a flow chart of a signal processing method according to an embodiment of the present disclosure. As shown in FIG3 , the embodiment of the present disclosure relates to a signal processing method, which can be executed by the encoding terminal device 101 and can include but is not limited to the following steps.

[0391] Step S3101: Acquire an audio signal in a mixed format.

[0392] In some embodiments, the audio signal in the mixed format is an audio signal including multiple channels.

[0393] The optional implementation of step S3101 can refer to the optional implementation of step S2101 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0394] Step S3102: filter the audio signal in the mixed format.

[0395] The optional implementation of step S3102 can refer to the optional implementation of step S2102 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0396] Step S3103: Determine a first parameter of the audio signal in the mixed format and / or an energy parameter of one or more channels among the multiple channels.

[0397] In some embodiments, the first parameter may include at least one of the following: a sound field analysis parameter; or a control parameter. In one implementation, the first parameter may include a sound field analysis parameter. In another implementation, the first parameter may include a control parameter. In yet another implementation, the first parameter may include both a sound field analysis parameter and a control parameter.

[0398] In some embodiments, the above-mentioned sound field analysis parameters include at least one of the following: the number of sound images; the priority of sound images; and background ambient sound. In one implementation, the above-mentioned sound field analysis parameters include any one of the number of sound images, the priority of sound images, and background ambient sound. In another implementation, the above-mentioned sound field analysis parameters include any two of the number of sound images, the priority of sound images, and background ambient sound. In yet another implementation, the above-mentioned sound field analysis parameters include the number of sound images, the priority of sound images, and background ambient sound.

[0399] In some embodiments, the control parameters may include a second parameter and / or a third parameter. In some embodiments, the second parameter is used to describe the priority ranking of the roles played by each of the multiple channels in the mixed audio signal. Optionally, the priorities may be manually set or obtained through processing other than encoding.

[0400] In some embodiments, the third parameter is used to describe whether each of the above sound channels is diegetic sound or non-diegetic sound.

[0401] In some embodiments, a first parameter of the audio signal in a mixed format may be determined.

[0402] For example, the sound field analysis parameters of the mixed format audio signal can be determined. Optional implementations thereof can be found in the optional implementations of step S2103 in FIG. 2A and other related parts of the embodiment involved in FIG. 2A , which will not be described in detail here.

[0403] For example, the control parameters of the mixed format audio signal may be determined. Optional implementations thereof may refer to the optional implementations of step S2203 in FIG. 2B and other related parts of the embodiment involved in FIG. 2B , which will not be described in detail here.

[0404] For example, the sound field analysis parameters and control parameters of the mixed-format audio signal can be determined. Optional implementations thereof can be found in the optional implementation of step S2103 of FIG. 2A , the optional implementation of step S2203 of FIG. 2B , and other related portions of the embodiment involved in FIG. 2A , and other related portions of the embodiment involved in FIG. 2B , which will not be described in detail here.

[0405] In some embodiments, energy parameters of one or more of the multiple channels may be determined. Optional implementations thereof may be referred to the optional implementations of step S2104 in FIG. 2A and other related parts of the embodiment involved in FIG. 2A , which will not be described in detail here.

[0406] In some embodiments, a first parameter of the audio signal in a mixed format and an energy parameter of one or more channels of the plurality of channels may be determined.

[0407] For example, the sound field analysis parameters of the mixed-format audio signal and the energy parameters of one or more of the multiple channels can be determined. Optional implementations thereof can be found in the optional implementations of step S2103 and step S2104 of FIG. 2A , as well as other related portions of the embodiment involved in FIG. 2A , and are not further described here.

[0408] For example, control parameters of the mixed-format audio signal and energy parameters of one or more of the multiple channels may be determined. Optional implementations thereof may be found in the optional implementation of step S2203 of FIG. 2B , the optional implementation of step S2104 of FIG. 2A , and other related portions of the embodiment involved in FIG. 2B , and other related portions of the embodiment involved in FIG. 2A , and are not further described herein.

[0409] For example, the sound field analysis parameters, control parameters, and energy parameters of one or more of the multiple channels of the mixed-format audio signal can be determined. Optional implementations thereof can be found in the optional implementations of step S2103 and step S2104 of FIG. 2A , the optional implementations of step S2203 of FIG. 2B , and other related portions of the embodiment involved in FIG. 2A , and other related portions of the embodiment involved in FIG. 2B , which are not further described here.

[0410] Step S3104: performing bit allocation on the multiple channels according to the first parameter of the mixed format audio signal and / or the energy parameter of one or more channels among the multiple channels to obtain a bit allocation parameter.

[0411] In some embodiments, bit allocation parameters may be obtained by performing bit allocation on multiple channels according to the first parameter of the audio signal in the mixed format.

[0412] For example, bit allocation parameters can be obtained by performing bit allocation on multiple channels based on the sound field analysis parameters of the mixed format audio signal. Optional implementations thereof can be found in the optional implementation of step S2304 of FIG. 2C and other related parts of the embodiment involved in FIG. 2C , which will not be described in detail here.

[0413] For example, the bit allocation parameters can be obtained by performing bit allocation on multiple channels according to the control parameters of the mixed format audio signal. Optional implementations thereof can be found in the optional implementation of step S2304 of FIG. 2C and other related parts of the embodiment involved in FIG. 2C , which will not be described in detail here.

[0414] For example, bit allocation parameters can be obtained by performing bit allocation on multiple channels based on the sound field analysis parameters and control parameters of the mixed-format audio signal. For optional implementations, see the optional implementation of step S2304 in FIG. 2C and other related parts of the embodiment involved in FIG. 2C , which will not be described in detail here.

[0415] In some embodiments, bit allocation may be performed on the multiple channels according to energy parameters of one or more channels of the multiple channels of the audio signal in the mixed format to obtain a bit allocation parameter.

[0416] In some embodiments, bit allocation may be performed on the multiple channels according to the first parameter of the mixed-format audio signal and the energy parameter of one or more channels among the multiple channels to obtain the bit allocation parameter.

[0417] For example, bit allocation parameters can be obtained by performing bit allocation on the multiple channels based on the sound field analysis parameters of the mixed-format audio signal and the energy parameters of one or more of the multiple channels. Optional implementations thereof can be found in the optional implementations of step S2105 of FIG. 2A and other related portions of the embodiment involved in FIG. 2A , and are not further described here.

[0418] For example, bit allocation parameters can be obtained by performing bit allocation on the multiple channels based on the control parameters of the mixed-format audio signal and the energy parameters of one or more of the multiple channels. Optional implementations thereof can be found in the optional implementations of step S2205 of FIG. 2B and other related portions of the embodiment involved in FIG. 2B , and are not further described here.

[0419] For example, bit allocation parameters can be obtained by performing bit allocation on the multiple channels based on the sound field analysis parameters, control parameters, and energy parameters of one or more channels of the mixed-format audio signal. Optional implementations thereof can be found in the optional implementations of step S2406 in FIG. 2D and other related portions of the embodiment involved in FIG. These will not be described in detail here.

[0420] Step S3105 , encoding multiple channels according to the bit allocation parameters to obtain channel signal encoding parameters, and writing the bit allocation parameters and the channel signal encoding parameters into a bitstream.

[0421] The optional implementation of step S3105 can refer to the optional implementation of step S2106 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0422] Step S3106: Send code stream.

[0423] The optional implementation of step S3106 can refer to the optional implementation of step S2107 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0424] The method involved in the embodiments of the present disclosure may include at least one of steps S3101 to S3106. For example, steps S3101, S3102, S3103, and S3104 can be implemented as independent embodiments, steps S3101, S3103, and S3104 can be implemented as independent embodiments, steps S3101, S3102, S3103, S3104, S3105, and S3106 can be implemented as independent embodiments, and steps S3101, S3103, S3104, S3105, and S3106 can be implemented as independent embodiments, but are not limited thereto.

[0425] In some embodiments, step S3105 and step S3106 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0426] In some embodiments, step S3102, step S3105, and step S3106 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0427] In some embodiments, step S3102 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0428] FIG4 is a flow chart of a signal processing method according to an embodiment of the present disclosure. As shown in FIG4 , the embodiment of the present disclosure relates to a signal processing method, which can be executed by a decoding end device 102 and may include but is not limited to the following steps.

[0429] Step S4101: Receive a code stream.

[0430] In some embodiments, decoding device 102 receives a code stream. Alternatively, decoding device 102 receives a code stream sent by encoding device 101. This code stream is used by decoding device 102 to reconstruct the mixed-format audio signal. Optional implementations of the code stream can be found in the optional implementations of any of the aforementioned embodiments, as well as other related sections of any of the aforementioned embodiments, and are not further detailed here.

[0431] Step S4102: parsing the code stream to obtain a bit allocation parameter and a channel signal coding parameter for each of the multiple channels of the mixed-format audio signal.

[0432] The optional implementation of step S4102 can refer to the optional implementation of step S2108 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0433] Step S4103: reconstructing a mixed format audio signal based on the bit allocation parameters of each channel and the channel signal coding parameters.

[0434] The optional implementation of step S4103 can refer to the optional implementation of step S2109 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0435] Figure 5 is an interactive diagram illustrating a signal processing method according to an embodiment of the present disclosure. As shown in Figure 5 , the method involved in the embodiment of the present disclosure can be applied to a communication system 100, and the method includes but is not limited to the following steps.

[0436] In step S5101 , the encoding end device 101 obtains an audio signal in a mixed format, where the audio signal in the mixed format is an audio signal including multiple channels.

[0437] The optional implementation of step S5101 can refer to the optional implementation of step S2101 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0438] In step S5102 , the encoding end device 101 determines a first parameter of the mixed-format audio signal and / or an energy parameter of one or more channels among the multiple channels.

[0439] The optional implementation of step S5102 can be found in step S2103 of Figure 2A, step S2203 of Figure 2B, step S2104 of Figure 2A, the optional implementation of step S3103 of Figure 3, and other related parts in the embodiments involved in Figures 2A, 2B, and 3, which will not be repeated here.

[0440] In step S5103, the encoding end device 101 performs bit allocation on the multiple channels according to the first parameter of the mixed format audio signal and / or the energy parameter of one or more channels in the multiple channels to obtain a bit allocation parameter.

[0441] The optional implementation methods of step S5103 can be found in step S2105 of Figure 2A, step S2205 of Figure 2B, step S2304 of Figure 2C, step S2406 of Figure 2D, the optional implementation methods of step S3104 of Figure 3, and other related parts in the embodiments involved in Figures 2A, 2B, 2C, 2D, and 3, which will not be repeated here.

[0442] In step S5104, the encoding end device 101 encodes the multiple channels according to the bit allocation parameters to obtain channel signal encoding parameters, and writes the bit allocation parameters and the channel signal encoding parameters into the bitstream.

[0443] The optional implementation of step S5104 can refer to the optional implementation of step S2106 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0444] Step S5105: the encoding end device 101 sends a code stream.

[0445] The optional implementation of step S5105 can refer to the optional implementation of step S2107 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0446] In step S5106 , the decoding end device 102 parses the received code stream to obtain a bit allocation parameter and a channel signal coding parameter for each of the multiple channels of the mixed-format audio signal.

[0447] The optional implementation of step S5106 can refer to the optional implementation of step S2108 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0448] In step S5107 , the decoding end device 102 reconstructs the audio signal in a mixed format based on the bit allocation parameters of each channel and the channel signal encoding parameters.

[0449] The optional implementation of step S5107 can refer to the optional implementation of step S2109 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0450] In some embodiments, the present disclosure discloses encoding and decoding processing for mixed-format audio signals as follows: when a mixed-format audio signal is input to an encoder, the encoder performs energy calculation and sound field analysis on the mixed-format audio signal, and allocates bits to each encoded channel signal based on the energy calculation and sound field analysis results. Each input audio channel signal uses the allocated bits to encode the audio signal to obtain a bit stream, and the decoding end decodes the bit stream to reconstruct the obtained mixed-format audio signal.

[0451] In some embodiments, the present disclosure performs encoding and decoding processing for mixed-format audio signals as follows: when a mixed-format audio signal is input to an encoder, the encoder performs energy calculation on the mixed-format audio signal, allocates bits to each encoded channel signal based on control parameters and energy calculation results, encodes the audio signal using the allocated bits to obtain a bit stream, and the decoding end decodes the bit stream to reconstruct the obtained mixed-format audio signal.

[0452] The purpose of the present disclosure is to design and complete the encoding and decoding processing of mixed-format audio signals, determine the encoding mode according to external control parameters and sound field analysis results, and thus reconstruct the desired mixed-format audio signal at the decoding end.

[0453] The mixed format input audio signal of the encoder in the present disclosure includes any combination of audio signals in the following four audio formats, namely: channel-based audio signal, object-based audio signal, scene-based audio signal, and metadata-based three-dimensional audio signal.

[0454] All mixed-format audio signals input are subjected to high-pass filtering. The filter cutoff frequency can be set to 20 Hz. As an example, the filter formula used can be shown in the following formula (1):

[0455] As shown in Figure 6A, the audio signal after high-pass filtering is subjected to energy calculation and sound field analysis:

[0456] Energy calculation is to calculate the energy value of each channel. The calculation formula for the sum of squares of a channel is as follows:

[0457] Among them, S m is the amplitude value and the sample point of one frame of the mth channel. One frame of the mth channel includes N sample points. X m is the mth channel.

[0458] Sound field analysis refers to analyzing the number of sound images in a mixed format audio signal, the priority of the sound images, background ambient sound, etc. The following is an example:

[0459] Case 1: The mixed format audio signal is composed of an object-based audio signal and a scene-based audio signal. The object-based audio signal is the main sound element in the sound field, and the scene-based audio signal is the ambient background sound element in the sound field. The object-based audio signal is required to have lower distortion in the expected reconstructed sound field, while the scene-based audio signal is the ambient background sound and can allow a certain degree of distortion. Therefore, more bits are allocated to the object-based audio signal than to the scene-based audio signal during bit allocation.

[0460] In the object-based audio signal, the priority of each object signal in the mixed format audio signal is different. For example, at a concert, the priority of the lead singer's voice signal is greater than that of the backing singer's voice signal, or in other words, the lead singer's voice signal is more important than the backing singer's voice signal. Therefore, the lead singer's object audio signal will be allocated more bits than the backing singer's voice signal.

[0461] Case 2: The mixed format audio signal is composed of an object-based audio signal and a channel-based audio signal. The object-based audio signal is the main sound element in the sound field, and the channel-based audio signal is the ambient background sound element in the sound field. The object-based audio signal is required to have smaller distortion in the expected reconstructed sound field, while the channel-based audio signal is used as the ambient background sound and can allow a certain degree of distortion. Therefore, more bits are allocated to the object-based audio signal than to the channel-based audio signal during bit allocation.

[0462] Case 3: The mixed-format audio signal is composed of a channel-based audio signal and a scene-based audio signal. The channel-based audio signal is the main sound element in the sound field, and the scene-based audio signal is the ambient background sound element in the sound field. The expected reconstructed sound field requires that the channel-based audio signal have lower distortion, while the scene-based audio signal is the ambient background sound and can allow a certain degree of distortion. Therefore, more bits are allocated to the channel-based audio signal than to the scene-based audio signal during bit allocation.

[0463] Case 4: In the above three cases, the bit allocation of each channel is performed based on the energy calculation results of each channel.

[0464] After obtaining the sound field analysis parameters of the mixed-format audio signal and the energy parameters of each channel, a corresponding coding core can be used to allocate bits for the multiple channels based on the sound field analysis parameters and the energy parameters of each channel to obtain bit allocation parameters. The multiple channels are encoded based on the bit allocation parameters to obtain channel signal encoding parameters, and the bit allocation parameters and channel signal encoding parameters are written into a bitstream. The encoder sends the bitstream to the decoder. The decoder decodes the received bitstream to reconstruct the mixed-format audio signal.

[0465] In some embodiments, after high-pass filtering all mixed-format input audio signals, as shown in FIG6B , energy calculation and sound field analysis can be performed on the high-pass filtered audio signals. External control parameters may also be considered when allocating bits to channels. Exemplary scenarios are as follows: Scenario 5: In the aforementioned three scenarios (e.g., scenarios 1, 2, and 3), bit allocation to each channel is performed in combination with external control parameters and the results of energy calculations for each channel.

[0466] For example, assuming that a mixed-format audio signal contains a 5.1-format multi-channel signal and four object audio signals, the mixed-format audio signal is high-pass filtered, the correlation coefficients between the channels are calculated, the channels are combined based on the correlation coefficients, and inter-channel parameters are calculated for the channels in the same channel combination. The inter-channel parameters can be ILD, ITD, IPD, etc. The channels are aligned using the inter-channel parameters and then down-mixed. Energy calculation is performed on the down-mixed channels. The energy calculation can be selected from the sample amplitude sum, the sample amplitude square sum, the sample amplitude root mean square, etc.

[0467] Perform sound field analysis on mixed-format audio signals. Sound field analysis calculates the number of sound images, their primary and secondary importance, and background ambient sound. Sound source localization can be used to estimate the direction of arrival (DOA), including azimuth and elevation angles. This can be solved using the Multi-Channel Cross Correlation Coefficient (MCCC) method. The identified channels identified by this algorithm are then treated as more important channels for processing.

[0468] Determine control parameters for the mixed-format audio signal. The control parameters refer to a priority ranking of the roles played by each channel in the mixed-format audio signal, pre-set based on external information. This ranking may be based on the functional division of each channel during acquisition, or may be based on the settings required for the desired sound field reconstructed by the playback end (e.g., in case 1, the lead singer's voice at a concert may be set as the most important channel; in case 2, in order to focus on the sound of a certain instrument (such as a violin) in the reconstructed sound field, the violin's voice may be set as the most important channel signal).

[0469] After obtaining the control parameters, sound field analysis parameters, and energy parameters of each channel of the mixed-format audio signal, a corresponding encoding core can be used to allocate bits for the multiple channels based on the sound field analysis parameters, control parameters, and energy parameters of each channel of the mixed-format audio signal to obtain bit allocation parameters. The multiple channels are encoded based on the bit allocation parameters to obtain channel signal encoding parameters, and the bit allocation parameters and channel signal encoding parameters are written into a bitstream. The encoder sends the bitstream to the decoder. The decoder decodes the received bitstream to reconstruct the mixed-format audio signal.

[0470] The present disclosure also provides an apparatus for implementing any of the above methods. For example, a device is provided that includes units or modules for implementing each step performed by an encoding device in any of the above methods. For another example, another device is provided that includes units or modules for implementing each step performed by a decoding device in any of the above methods.

[0471] It should be understood that the division of the various units or modules in the above device is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into a physical entity, or they may be physically separated. In addition, the units or modules in the device may be implemented in the form of a processor calling software: for example, the device includes a processor, the processor is connected to a memory, and the memory stores instructions. The processor calls the instructions stored in the memory to implement any of the above methods or implement the functions of the various units or modules of the above device, wherein the processor is, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory is a memory within the device or a memory outside the device. Alternatively, the units or modules in the device can be implemented in the form of hardware circuits, and the functions of some or all of the units or modules can be realized by designing the hardware circuits. The above-mentioned hardware circuits can be understood as one or more processors; for example, in one implementation, the above-mentioned hardware circuit is an application-specific integrated circuit (ASIC), which realizes the functions of some or all of the above units or modules by designing the logical relationship of the components in the circuit; for example, in another implementation, the above-mentioned hardware circuit can be realized by a programmable logic device (PLD). Taking a field programmable gate array (FPGA) as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by configuring the configuration file, thereby realizing the functions of some or all of the above units or modules. All units or modules of the above devices can be realized in the form of software called by the processor, or in the form of hardware circuits, or in part by the form of software called by the processor, and the rest by hardware circuits.

[0472] In the embodiments of the present disclosure, the processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationship of the hardware circuit. The logical relationship of the above-mentioned hardware circuit is fixed or reconfigurable. For example, the processor is a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and implementing the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units or modules. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc.

[0473] FIG7A is a schematic diagram of the structure of the decoding end device proposed in an embodiment of the present disclosure. As shown in FIG7A , the decoding end device 7100 may include: at least one of a transceiver module 7101 and a processing module 7102. In some embodiments, the processing module is used to obtain a mixed format audio signal, and the mixed format audio signal is an audio signal including multiple channels; the processing module is also used to determine the first parameter of the mixed format audio signal and / or the energy parameter of one or more channels in the multiple channels; the processing module is also used to allocate bits to the multiple channels according to the first parameter of the mixed format audio signal and / or the energy parameter of one or more channels in the multiple channels to obtain a bit allocation parameter; the processing module is also used to encode the multiple channels according to the bit allocation parameter to obtain a channel signal encoding parameter, and write the bit allocation parameter and the channel signal encoding parameter into the code stream; the transceiver module is used to send the code stream. Optionally, the above-mentioned transceiver module is used to execute at least one of the communication steps such as sending and / or receiving performed by the encoding end device 101 in any of the above methods (for example, step S2107, step S2207, step S2306, step S2408, but not limited to these), which will not be repeated here. Optionally, the above-mentioned processing module is used to execute at least one of the other steps performed by the terminal device 102 in any of the above methods (such as step S2101, step S2102, step S2103, step S2104, step S2105, step S2106, step S2201, step S2202, step S2203, step S2204, step S2205, step S2206, step S2301, step S2302, step S2303, step S2304, step S2305, step S2401, step S2402, step S2403, step S2404, step S2405, step S2406, step S2407, but not limited to these), which are not repeated here.

[0474] FIG7B is a schematic diagram of the structure of the decoding end device proposed in an embodiment of the present disclosure. As shown in FIG7B , the decoding end device 7200 may include: at least one of a transceiver module 7201 and a processing module 7202. In some embodiments, the transceiver module is used to receive a code stream; the processing module is used to parse the code stream to obtain the bit allocation parameters and channel signal encoding parameters of each channel in the multiple channels of the mixed format audio signal; the processing module is also used to reconstruct the mixed format audio signal based on the bit allocation parameters and channel signal encoding parameters of each channel. Optionally, the transceiver module is used to perform at least one of the communication steps such as sending and / or receiving performed by the decoding end device 7200 in any of the above methods, which will not be repeated here. Optionally, the above-mentioned processing module is used to execute at least one of the other steps (such as step S2108, step S2109, step S2208, step S2209, step S2307, step S2308, step S2409, step S2410, but not limited to these) performed by the decoding end device 102 in any of the above methods, which will not be repeated here.

[0475] In some embodiments, the transceiver module may include a transmitting module and / or a receiving module, and the transmitting module and the receiving module may be separate or integrated. Optionally, the transceiver module may be interchangeable with the transceiver.

[0476] In some embodiments, the processing module can be a single module or can include multiple submodules. Optionally, the multiple submodules respectively execute all or part of the steps required to be executed by the processing module. Optionally, the processing module can be interchangeable with the processor.

[0477] Figure 8A is a schematic diagram of the structure of a communication device 8100 proposed in an embodiment of the present disclosure. Communication device 8100 can be an encoding device or a decoding device, or a chip, chip system, or processor that supports an encoding device in implementing any of the above methods. It can also be a chip, chip system, or processor that supports a decoding device in implementing any of the above methods. Communication device 8100 can be used to implement the methods described in the above method embodiments. For details, please refer to the description of the above method embodiments.

[0478] As shown in Figure 8A, the communication device 8100 includes one or more processors 8101. The processor 8101 can be a general-purpose processor or a dedicated processor, for example, a baseband processor or a central processing unit. The baseband processor can be used to process communication protocols and communication data, and the central processing unit can be used to control the communication device (such as a base station, a baseband chip, a terminal device, a terminal device chip, a DU or a CU, etc.), execute programs, and process program data. The communication device 8100 is used to perform any of the above methods.

[0479] In some embodiments, the communication device 8100 further includes one or more memories 8102 for storing instructions. Optionally, all or part of the memories 8102 may be located outside the communication device 8100.

[0480] In some embodiments, the communication device 8100 further includes one or more transceivers 8103. When the communication device 8100 includes one or more transceivers 8103, the transceiver 8103 performs at least one of the communication steps such as sending and / or receiving in the above method (for example, step S2107, step S2207, step S2306, step S2408, but not limited thereto), and the processor 8101 performs other steps (for example, step S2101, step S2102, step S2103, step S2104, step S2105, step S2106, step S2201, step S2202, step S2203, step S2306, step S2307, step S2308, etc.). 204, step S2205, step S2206, step S2301, step S2302, step S2303, step S2304, step S2305, step S2401, step S2402, step S2403, step S2404, step S2405, step S2406, step S2407, step S2108, step S2109, step S2208, step S2209, step S2307, step S2308, step S2409, step S2410), but not limited to at least one of these.

[0481] In some embodiments, a transceiver may include a receiver and / or a transmitter. The receiver and transmitter may be separate or integrated. Optionally, the terms transceiver, transceiver unit, transceiver, and transceiver circuit may be used interchangeably; the terms transmitter, transmitting unit, transmitter, and transmitting circuit may be used interchangeably; and the terms receiver, receiving unit, receiver, and receiving circuit may be used interchangeably.

[0482] In some embodiments, the communication device 8100 may include one or more interface circuits 8104. Optionally, the interface circuit 8104 is connected to the memory 8102. The interface circuit 8104 may be configured to receive signals from the memory 8102 or other devices, and may be configured to send signals to the memory 8102 or other devices. For example, the interface circuit 8104 may read instructions stored in the memory 8102 and send the instructions to the processor 8101.

[0483] The communication device 8100 described in the above embodiments may be an encoding end device or a decoding end device, but the scope of the communication device 8100 described in the present disclosure is not limited thereto, and the structure of the communication device 8100 may not be limited by FIG. 8A. The communication device may be an independent device or may be part of a larger device. For example, the communication device may be: 1) an independent integrated circuit IC, or a chip, or a chip system or subsystem; (2) a collection of one or more ICs, optionally, the above IC collection may also include a storage component for storing data or programs; (3) an ASIC, such as a modem; (4) a module that can be embedded in other devices; (5) a receiver, a terminal device, an intelligent terminal device, a cellular phone, a wireless device, a handheld device, a mobile unit, an in-vehicle device, a network device, a cloud device, an artificial intelligence device, etc.; (6) others, etc.

[0484] FIG8B is a schematic diagram of the structure of a chip 8200 according to an embodiment of the present disclosure. If the communication device 8100 can be a chip or a chip system, please refer to the schematic diagram of the structure of the chip 8200 shown in FIG8B , but the present disclosure is not limited thereto.

[0485] The chip 8200 includes one or more processors 8201 , and the chip 8200 is configured to execute any of the above methods.

[0486] In some embodiments, the chip 8200 further includes one or more interface circuits 8202. Optionally, the interface circuit 8202 is connected to the memory 8203. The interface circuit 8202 can be used to receive signals from the memory 8203 or other devices, and can be used to send signals to the memory 8203 or other devices. For example, the interface circuit 8202 can read instructions stored in the memory 8203 and send the instructions to the processor 8201.

[0487] In some embodiments, the interface circuit 8202 performs at least one of the communication steps such as sending and / or receiving in the above method (for example, step S2107, step S2207, step S2306, step S2408, but not limited thereto), and the processor 8201 performs other steps (for example, step S2101, step S2102, step S2103, step S2104, step S2105, step S2106, step S2201, step S2202, step S2203, step S2204, step S2 205, step S2206, step S2301, step S2302, step S2303, step S2304, step S2305, step S2401, step S2402, step S2403, step S2404, step S2405, step S2406, step S2407, step S2108, step S2109, step S2208, step S2209, step S2307, step S2308, step S2409, step S2410, but not limited to at least one of these).

[0488] In some embodiments, terms such as interface circuit, interface, transceiver pin, and transceiver may be used interchangeably.

[0489] In some embodiments, the chip 8200 further includes one or more memories 8203 for storing instructions. Alternatively, all or part of the memories 8203 may be outside the chip 8200.

[0490] The present disclosure also proposes a storage medium having instructions stored thereon, which, when executed on the communication device 8100, causes the communication device 8100 to execute any of the above methods. Optionally, the storage medium is an electronic storage medium. Optionally, the storage medium is a computer-readable storage medium, but is not limited thereto, and may also be a storage medium readable by other devices. Optionally, the storage medium may be a non-transitory storage medium, but is not limited thereto, and may also be a temporary storage medium.

[0491] The present disclosure also provides a program product, which, when executed by the communication device 8100, enables the communication device 8100 to perform any of the above methods. Optionally, the program product is a computer program product.

[0492] The present disclosure also proposes a computer program, which, when executed on a computer, causes the computer to perform any one of the above methods.

[0493] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs. When the computer program is loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0494] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0495] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0496] The above description is merely a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A signal processing method, characterized in that: include: Acquire an audio signal in a mixed format, where the audio signal in the mixed format is an audio signal including multiple channels; Determining a first parameter of the audio signal in the mixed format and / or an energy parameter of one or more of the plurality of channels; According to the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels, performing bit allocation on the multiple channels to obtain a bit allocation parameter; Encode the multiple channels according to the bit allocation parameter to obtain channel signal encoding parameters, and write the bit allocation parameter and the channel signal encoding parameters into a bitstream; The code stream is sent.

2. The method according to claim 1, characterized in that The first parameter includes at least one of the following: Acoustic field analysis parameters; Control parameters, the control parameters include a second parameter and / or a third parameter, the second parameter is used to describe the priority ranking of the role played by each of the multiple channels in the audio signal of the mixed format, and the third parameter is used to describe whether each of the channels is a plot sound or a non-plot sound.

3. The method according to claim 1 or 2, characterized in that The mixed-format audio signal includes an audio signal in at least one of the following formats: Channel-based audio signals; Object-based audio signals; Scene-based audio signals; Metadata-based three-dimensional audio signals.

4. The method according to claim 2 or 3, characterized in that The determining of the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels comprises: determining an energy parameter of one or more of the plurality of channels; The sound field analysis parameters of the audio signal in the mixed format are determined.

5. The method according to claim 4, characterized in that The step of allocating bits to the multiple channels to obtain a bit allocation parameter according to the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels includes: The bit allocation parameter is obtained by performing bit allocation on the multiple channels according to the energy parameter of one or more channels and the sound field analysis parameter of the audio signal in the mixed format.

6. The method according to claim 2 or 3, characterized in that: The determining of the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels comprises: determining an energy parameter of one or more of the plurality of channels; The control parameter of the audio signal in the mixed format is determined.

7. The method according to claim 6, characterized in that The step of allocating bits to the multiple channels to obtain a bit allocation parameter according to the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels includes: The bit allocation parameter is obtained by performing bit allocation on the multiple channels according to the energy parameter of one or more channels and the control parameter of the audio signal in the mixed format.

8. The method according to claim 2 or 3, characterized in that: The determining of the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels comprises: The sound field analysis parameter and / or the control parameter of the audio signal in the mixed format is determined.

9. The method according to claim 8, characterized in that The step of allocating bits to the multiple channels to obtain a bit allocation parameter according to the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels includes: The bit allocation parameter is obtained by performing bit allocation on the multiple channels according to the sound field analysis parameter and / or the control parameter of the audio signal in the mixed format.

10. The method according to claim 2 or 3, characterized in that: The determining of the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels comprises: determining an energy parameter of one or more of the plurality of channels; The sound field analysis parameters and the control parameters of the audio signal in the mixed format are determined.

11. The method according to claim 10, characterized in that The step of allocating bits to the multiple channels to obtain a bit allocation parameter according to the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels includes: The bit allocation parameter is obtained by performing bit allocation on the multiple channels according to the energy parameter of one or more channels, the sound field analysis parameter of the audio signal in the mixed format, and the control parameter.

12. The method according to any one of claims 4 to 7, 10 to 11, characterized in that: The determining of energy parameters of one or more of the multiple channels comprises: After filtering the plurality of channels, calculating the correlation coefficients between the plurality of channels; Perform channel combination operation according to the mutual correlation coefficient, and calculate inter-channel parameters for channels in the same channel combination; The channels are aligned using the inter-channel parameters and then down-mixed, and energy is calculated for the down-mixed channels to obtain energy parameters of each of the channels.

13. The method according to claim 12, characterized in that The method for calculating the energy parameter comprises performing any of the following processing on the sample points of one frame of the channel: Calculate the amplitude value and the amplitude value of the sample points; Calculating the sum of squares of amplitudes of the sample points; Calculate the root mean square of the sample points.

14. The method according to any one of claims 4-5, 8-11, characterized in that: The determining of the sound field analysis parameters of the audio signal in the mixed format comprises: Based on the direction of arrival (DOA) estimation and / or the multi-channel cross-correlation coefficient (MCCC) method, the sound field analysis is performed on the audio signal in the mixed format to obtain the sound field analysis parameters of the audio signal in the mixed format.

15. The method according to any one of claims 6 to 11, characterized in that: The determining of the control parameter of the audio signal in the mixed format comprises at least one of the following: Determining control parameters of the mixed-format audio signal according to the functional division of each channel during collection by the audio signal collection device, wherein the audio signal collection device is used to collect the mixed-format audio signal; The control parameters of the audio signal in the mixed format are determined according to the setting parameters required by the decoding end device to reconstruct the desired sound field.

16. The method according to any one of claims 2, 4-5, 8-11, and 14, characterized in that: The sound field analysis parameters include at least one of the following: The number of sound images; The priority of sound and image; Background ambient sound.

17. The method according to claim 2 or 3, characterized in that: The bit allocation to the multiple channels according to the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels comprises: performing bit allocation on an audio signal in a mixed format consisting of the object-based audio signal and the scene-based audio signal according to the sound field analysis parameter of the audio signal in the mixed format; The number of bits allocated to any channel in the object-based audio signal is greater than the number of bits allocated to any channel in the scene-based audio signal.

18. The method according to claim 2 or 3, characterized in that The bit allocation to the multiple channels according to the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels comprises: performing bit allocation on a mixed-format audio signal composed of the object-based audio signal and the channel-based audio signal according to the sound field analysis parameter of the mixed-format audio signal; The number of bits allocated to any channel in the object-based audio signal is greater than the number of bits allocated to any channel in the channel-based audio signal.

19. The method according to claim 2 or 3, characterized in that: The bit allocation to the multiple channels according to the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels comprises: The sound field analysis parameters of the audio signal in the mixed format are used to analyze the channel-based audio signal and the scene-based audio signal. bit allocation for an audio signal in a mixed format composed of audio signals of The number of bits allocated to any channel in the channel-based audio signal is greater than the number of bits allocated to any channel in the scene-based audio signal.

20. The method according to claim 2 or 3, characterized in that The bit allocation to the multiple channels according to the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels comprises: performing bit allocation on an audio signal in a hybrid format consisting of the object-based audio signal and the metadata-based three-dimensional audio signal according to the sound field analysis parameter of the audio signal in the hybrid format; The number of bits allocated to any channel in the object-based audio signal is greater than the number of bits allocated to any channel in the metadata-based three-dimensional audio signal.

21. The method according to claim 2 or 3, characterized in that The bit allocation to the multiple channels according to the first parameter of the audio signal in the mixed format and / or the energy parameter of one or more of the multiple channels comprises: performing bit allocation on the audio signal in a mixed format consisting of the channel-based audio signal and the metadata-based three-dimensional audio signal according to the sound field analysis parameter of the audio signal in the mixed format; The number of bits allocated to any channel in the channel-based audio signal is greater than the number of bits allocated to any channel in the metadata-based three-dimensional audio signal.

22. The method according to any one of claims 17, 18 and 20, characterized in that The number of bits allocated to a channel corresponding to a primary sound image in the object-based audio signal is greater than the number of bits allocated to a channel corresponding to a secondary sound image in the object-based audio signal.

23. A signal processing method, characterized in that: include: Receive code stream; Parsing the bit stream to obtain a bit allocation parameter and a channel signal encoding parameter of each of the multiple channels of the audio signal in a mixed format; The audio signal in the mixed format is reconstructed based on the bit allocation parameters of each of the channels and the channel signal encoding parameter code stream.

24. The method of claim 23, wherein: The step of reconstructing the mixed format audio signal based on the bit allocation parameters of each channel and the channel signal encoding parameter bitstream comprises: Decoding the channel signal encoding parameters based on the bit allocation parameters of each of the channels; The decoded channel code stream is used to reconstruct the audio signal in the mixed format.

25. A first communication device, characterized in that: include: A processing module, used for acquiring an audio signal in a mixed format, where the audio signal in the mixed format is an audio signal including multiple channels; The processing module is further configured to determine a first parameter of the audio signal in the mixed format and / or an energy parameter of one or more of the multiple channels; The processing module is further configured to process the audio signal in the mixed format according to the first parameter of the audio signal and / or one or more of the multiple channels. energy parameters of the channels, and performing bit allocation on the multiple channels to obtain bit allocation parameters; The processing module is further configured to encode the plurality of channels according to the bit allocation parameter to obtain channel signal encoding parameters, and write the bit allocation parameter and the channel signal encoding parameters into a bit stream; The transceiver module is used to send the code stream.

26. A second communication device, characterized in that: include: Transceiver module, used for receiving code stream; A processing module, configured to parse the bit stream to obtain a bit allocation parameter and a channel signal encoding parameter of each of the multiple channels of the audio signal in a mixed format; The processing module is further used to reconstruct the audio signal in the mixed format based on the bit allocation parameters of each channel and the channel signal encoding parameter code stream.

27. A communication system, characterized in that: include: An encoding end device, configured to perform the signal processing method according to any one of claims 1 to 22; The decoding end device is configured to execute the signal processing method as claimed in claim 23 or 24.

28. A communication device, characterized in that: include: one or more processors; The processor is used to call instructions so that the communication device executes the signal processing method described in any one of claims 1-22 and 23-24.

29. A storage medium storing instructions, characterized in that: When the instruction is executed on a communication device, the communication device executes the signal processing method according to any one of claims 1 to 22 and 23 to 24.

Images