Audio data processing method and apparatus, and electronic device and storage medium

Abstract

Embodiments of the present disclosure provide an audio data processing method and apparatus, an electronic device and a storage medium, wherein the method comprises: acquiring audio data to be processed whose sampling frequency is greater than or equal to a preset frequency and whose sampling depth is greater than or equal to a preset depth; performing subband decomposition on the audio data to obtain a plurality of subband data items; determining a desired post-encoding bitrate of each subband data item; a sum of desired post-encoding bitrates of all subband data items being equal to a desired post-encoding bitrate of the audio data; a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data being greater than a preset value; encoding each subband data item according to the desired post-encoding bitrate of each subband data item; generating encoded audio data according to each encoded subband data item.

Description

CROSS REFERENCE TO RELATED APPLICATIONS

[0001] The present disclosure claims a priority right to the Chinese patent application No. 202310738824.8 filed on June 20, 2023 with China National Intellectual Property Administration, the entire disclosure of which is hereby incorporated by reference in its entirety.FIELD

[0002] The present disclosure relates to the field of audio data processing, and particularly to an audio data processing method and apparatus, an electronic device and a storage medium.BACKGROUND

[0003] Before being transmitted, audio data needs to be encoded, the encoded audio data is transmitted by an audio data sender to an audio data receiver, and the audio data receiver, after having received the encoded audio data, decodes the encoded audio data and plays the decoded audio data. In the prior art, when audio data with a larger sampling frequency and a larger sampling depth is encoded, a bitrate of the encoded audio data is still very large, which causes a low efficiency of encoding the audio data.SUMMARY

[0004] Embodiments of the present disclosure provide an audio data processing method and apparatus, an electronic device and a storage medium, which, for audio data with a larger sampling frequency and a larger sampling depth, can reduce a bitrate of the encoded audio data, and improve an efficiency of encoding the audio data.

[0005] In a first aspect, an embodiment of the present disclosure provides an audio data processing method, comprising: acquiring audio data to be processed whose sampling frequency is greater than or equal to a preset frequency and whose sampling depth is greater than or equal to a preset depth; performing subband decomposition on the audio data to obtain a plurality of subband data items; determining a desired post-encoding bitrate of each of the plurality of subband data items; a sum of desired post-encoding bitrates of all the subband data items is equal to a desired post-encoding bitrate of the audio data; a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than a preset value; encoding each of the plurality of subband data items according to the desired post-encoding bitrate of each of the plurality of subband data items; generating encoded audio data according to each of the encoded subband data items.

[0006] In a second aspect, an embodiment of the present disclosure provides an audio data processing apparatus, comprising: an acquisition unit configured to acquire audio data to be processed whose sampling frequency is greater than or equal to a preset frequency and whose sampling depth is greater than or equal to a preset depth; a decomposition unit configured to perform subband decomposition on the audio data to obtain a plurality of subband data items; a determination unit configured to determine a desired post-encoding bitrate of each of the plurality of subband data items; a sum of desired post-encoding bitrates of all the subband data items is equal to a desired post-encoding bitrate of the audio data; a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than a preset value; an encoding unit configured to encode each of the plurality of subband data items according to the desired post-encoding bitrate of each of the plurality of subband data items; a generation unit configured to generate encoded audio data according to each of the encoded subband data items.

[0007] In a third aspect, an embodiment of the present disclosure provides an electronic device comprising: a processor; and a memory configured to store computer-executable instructions which, when executed, cause the processor to implement the steps of the method in the first aspect.

[0008] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium for storing computer-executable instructions that, when executed by a processor, implement the steps of the method in the first aspect.BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to illustrate the technical solutions in the present disclosure or the prior art more clearly, a brief introduction will be given to the figures to be used in the description of the embodiments or the prior art. It is obvious that the figures described below are merely some embodiments of the present disclosure, and other figures can also be obtained by those skilled in the art according to these figures without involving any inventive effort: FIG. 1 illustrates a flow chart of an audio data processing method according to an embodiment of the present disclosure; FIG. 2a illustrates a schematic diagram of a preset bitstream structure according to an embodiment of the present disclosure; FIG. 2b illustrates a schematic diagram of encoded audio data according to an embodiment of the present disclosure; FIG. 3 illustrates a schematic diagram showing principles of audio data encoding according to an embodiment of the present disclosure; FIG. 4 illustrates a schematic diagram showing principles of audio data decoding according to an embodiment of the present disclosure; FIG. 5 illustrates a schematic diagram of an application scenario of audio data encoding and decoding according to an embodiment of the present disclosure; FIG. 6 illustrates a block diagram of an audio data processing apparatus according to an embodiment of the present disclosure; FIG. 7 illustrates a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF EMBODIMENTS

[0010] Technical solutions in one or more embodiments of the present disclosure will be described clearly and completely with reference to figures in the one or more embodiments of the present disclosure to enable those skilled in the art to better understand the technical solutions in the one or more embodiments. Obviously, the embodiments described are partial embodiments other than all embodiments of the present disclosure. All other embodiments obtained by those having ordinary skill in the art based on one or more embodiments in the present disclosure without making any inventive efforts all fall within the protection scope of the present disclosure.

[0011] Embodiments of the present disclosure provide an audio data processing method, which, for audio data with a larger sampling frequency and a larger sampling depth, can reduce a bitrate of the encoded audio data, and improve an efficiency of encoding the audio data. For ease of understanding embodiments of the present disclosure, technical terms involved are first defined as follows:

[0012] Sampling frequency: also referred to as a sampling rate, which means the amount of audio data collected per unit time, e.g., per second, i.e., the amount of audio data recorded per second, and a measure unit of the sampling frequency is kHz. For example, the sampling frequency of audio data is 192 kHz, which means that the amount of audio data collected per second is 192k. The higher the sampling frequency, the greater the amount of audio data.

[0013] Sampling depth: also referred to as a bit depth, which represents the number of bits required to acquire one audio data. For example, the sampling depth is 24 bits, which indicates that each audio data collected is represented by 24 bits. The larger the sampling depth, the greater the amount of data of the audio data.

[0014] PQMF: Pseudo-Quadrature Mirror Filter in a full form, which is a well-established filtering algorithm, through which sub-decomposition is performed on the audio data.

[0015] Opus: a well-established lossy encoder that supports a maximum desired encoding bitrate of 510 kbps for input.

[0016] Bitstream data: the encoded audio data is essentially a long string of binary data, which may be referred to as bitstream data. After receiving the bitstream data, a receiver end decodes the bitstream data and plays the decoded bitstream data. Since the audio data is encoded frame by frame during encoding, it may also be believed that each frame of the encoded audio data corresponds to one frame of bitstream data, and the receiver end decodes each frame of bitstream data and plays the decoded bitstream data.

[0017] Bitstream structure: the bitstream data is stored according to a certain data structure, and the data structure is a bitstream structure; generally, the bitstream structure at least comprises a frame header field and an encoding field, wherein the frame header field is used for storing parameters such as a length, an encoding bandwidth and a number of channels of the bitstream data, and the encoding field is used for storing encoded audio data. The receiver obtains the meaning and length of each field of the bitstream structure, parses the bitstream data and plays the parsed bitstream data.

[0018] In one or more embodiments of the present disclosure, audio data to be processed is acquired, a sampling frequency of the audio data is greater than or equal to a preset frequency and a sampling depth is greater than or equal to a preset depth, subband decomposition is performed on the audio data to obtain a plurality of subband data items, a desired bitrate of each encoded subband data item is determined, a sum of desired bitrates of all encoded subband data items is equal to a desired bitrate of the encoded audio data, a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than a preset value, each of the plurality of subband data items is encoded according to the desired bitrate of each encoded subband data item, and an encoded audio data is generated according to each encoded subband data item. Through the embodiment, as for the audio data whose sampling frequency is greater than or equal to the preset frequency and whose sampling depth is greater than or equal to the preset depth, the encoded audio data can be obtained by performing subband decomposition on the audio data and encoding the subband data items. Since the ratio of the pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than the preset value, the bitrate of the encoded audio data is greatly reduced and the efficiency of encoding the audio data is improved. Since the bitrate of the encoded audio data is reduced, an effect of saving the bandwidth for transmitting the encoded audio data and saving the data flow is also achieved.

[0019] FIG. 1 is a flow diagram of an audio data processing method according to an embodiment of the present disclosure. The method can be applied to and executed by a terminal device. The terminal device includes, but is not limited to, devices capable of collecting audio data and encoding and transmitting the audio data, such as a mobile phone, a computer, a tablet computer, a vehicle-mounted computer, a wearable device, etc. Certainly, the method can also be applied within a server and executed by the server. The method will be described below by taking a terminal device as an executing body. As shown in FIG. 1, the method comprises: Step S102: audio data to be processed is acquired whose sampling frequency is greater than or equal to a preset frequency and whose sampling depth is greater than or equal to a preset depth; Step S104: subband decomposition is performed on the audio data to obtain a plurality of subband data items; Step S106: a desired post-encoding bitrate of each encoded subband data item is determined; a sum of desired post-encoding bitrates of all subband data items being equal to a desired post-encoding bitrate of the audio data; a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data being greater than a preset value; Step S108: each of the plurality of subband data items is encoded according to the desired post-encoding bitrate of each subband data item; Step S110: an encoded audio data item is generated according to each encoded subband data item.

[0020] Through the embodiment, as for the audio data whose sampling frequency is greater than or equal to the preset frequency and whose sampling depth is greater than or equal to the preset depth, the encoded audio data can be obtained by performing subband decomposition on the audio data and encoding the subband data items. Since the ratio of the pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than the preset value, the bitrate of the encoded audio data is greatly reduced and the efficiency of encoding the audio data is improved. Since the bitrate of the encoded audio data is reduced, an effect of saving the bandwidth for transmitting the encoded audio data and saving the data flow is also achieved.

[0021] In the above step S102, audio data to be processed is acquired. The sampling frequency of the audio data to be processed is greater than or equal to the preset frequency and the sampling depth is greater than or equal to the preset depth. In one embodiment, the preset frequency may be 96 kHz and the preset depth may be 24 bits. Since the sampling frequency of the audio data to be processed is greater than or equal to the preset frequency and the sampling depth is greater than or equal to the preset depth, the audio data to be processed may also be referred to as high resolution Hi-Res audio data.

[0022] In the above step S104, subband decomposition is performed on the audio data to obtain a plurality of subband data items. In one embodiment, the subband decomposition may be performed on the audio data by a PQMF filter to obtain a plurality of subband data items. In one embodiment, the sampling frequency of each of the plurality of subband data items is less than or equal to the preset frequency so that the subband data items may be successfully encoded. For example, when each of the plurality of subband data items is encoded by an Opus encoder, since the Opus can encode audio data with a sampling frequency of at most 48 kHz, the sampling frequency of each of the plurality of subband data items is required to be less than or equal to 48 kHz. In this embodiment, with regard to the audio data with a sampling frequency of 192 kHz, the audio data may be decomposed into four subband data items with a sampling frequency of 48 kHz; based on the sampling theorem, an effective frequency band range of the signal is half of the sampling frequency, and therefore the effective frequency band range of the original 192 kHz audio data is 0-96 kHz; the decomposed subband data items respectively represent the effective frequency band ranges of 0-24 kHz, 24-48 kHz, 48-72 kHz and 72-96 kHz of the original audio data; and the subband data items are sequentially numbered to obtain serial numbers 1, 2, 3 and 4.

[0023] In the above step S106, a desired post-encoding bitrate of each of the plurality of subband data items is determined, a sum of desired post-encoding bitrates of all subband data items is equal to a desired post-encoding bitrate of the audio data, and a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than a preset value. First, the desired post-encoding bitrate of the audio data is obtained. The ratio of the pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than the preset value. The desired post-encoding bitrate of the audio data may be pre-stored in the terminal device. Upon pre-storing, the ratio of the pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than the preset value. For example, if the preset value is 10, the ratio of the pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than 10, i.e., an encoding compression rate of the audio data is allowed to reach more than 10 times.

[0024] In a specific example, for 192 kHz 24-bit dual-channel audio data, the pre-encoding bitrate is 192000*24*2bps, and the desired post-encoding bitrate may be configured to be 500000 bps, so that 192000*24*2 / 500000=18.432, so that the compression rate reaches more than 18 times. The magnitude of the preset value may be empirically determined to ensure a high quality of the encoded audio data while ensuring a large compression rate of the audio data.

[0025] In one embodiment, the preset value is determined empirically in advance; according to the preset value and the pre-encoding bitrate of commonly-used high-resolution audio data, the desired post-encoding bitrate of the commonly-used high-resolution audio data is determined and stored in the terminal device, and in this case, the desired post-encoding bitrate of the audio data is a fixed value.

[0026] In another embodiment, the preset value is configured in an encoder of the terminal device in advance; upon encoding, the terminal device may acquire the pre-encoding bitrate of the audio data, and automatically generate the desired post-encoding bitrate based on the preset value and the pre-encoding bitrate, and make the ratio of the pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data greater than the preset value. In this case, the desired post-encoding bitrate of the audio data is a value calculated in real time upon encoding.

[0027] In the above step S106, the desired post-encoding bitrate of each of the plurality of subband data items is also determined based on the desired post-encoding bitrate of the audio data, so that the sum of desired post-encoding bitrates of all subband data items is equal to the desired post-encoding bitrate of the audio data.

[0028] In one embodiment, the determining the desired post-encoding bitrate of each of the plurality of subband data items comprises: acquiring a proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data; determining the desired post-encoding bitrate of each of the plurality of subband data items according to the proportion of the desired post-encoding bitrate of the audio data to each of the plurality of subband data items.

[0029] Specifically, first, the proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data is obtained. It can be appreciated that a sum of proportions of each of the plurality of subband data items is 1 so that the sum of the desired post-encoding bitrates of the subband data items is equal to the desired post-encoding bitrate of the audio data. Then, the desired post-encoding bitrate of each of the plurality of subband data items is determined according to the proportion of the desired post-encoding bitrate of the audio data to each of the plurality of subband data items. For example, as for each of the plurality of subband data items, the corresponding proportion is multiplied by the desired post-encoding bitrate of the audio data, to obtain the desired post-encoding bitrate of each of the plurality of subband data items.

[0030] It can be seen that through the present embodiment, the desired post-encoding bitrate of each of the plurality of subband data items can be determined efficiently and quickly by determining the proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data, and the bitrate-determining process is simple and efficient.

[0031] In one embodiment, the acquiring a proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data comprises: determining a frequency band range of the audio data represented by each of the plurality of subband data items; determining the proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data, according to the frequency band range of the audio data represented by each of the plurality of subband data items.

[0032] First, the frequency band range of the audio data represented by each of the plurality of subband data items is determined. For example, in the preceding example, for subband data items 1, the frequency band range of the audio data represented by it is in a range of 0-24 kHz. Then, the proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data is determined according to the frequency band range of the audio data represented by each of the plurality of subband data items. For example, the desired post-encoding bitrate of a subband data items representing a lower frequency band range of the audio data corresponds to a larger proportion.

[0033] It can be seen that through the present embodiment, since a relationship between the frequency band range of the audio data represented by the subband data items and an importance degree of the subband data items is taken into consideration, the determining the proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data, according to the frequency band range of the audio data represented by each of the plurality of subband data items enables important subband data items to be allocated a larger bitrate, and improves the quality of the encoded audio data.

[0034] In one embodiment, the determining the proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data, according to the frequency band range of the audio data represented by each of the plurality of subband data items comprises: for a first subband data item representing a minimum frequency in the frequency band range of the audio data among all subband data items, determining the proportion of the first subband data item is greater than the proportion of a second subband data item among all subband data items; the second subband data item is a subband data items among all subband data items other than the first subband data item.

[0035] Specifically, the number of the subband data items is plural. The first subband data item representing a minimum frequency in the frequency band range of the audio data corresponds to a larger proportion than the proportions to which other subband data items correspond. Therefore, the first subband data item is allocated a maximum bitrate so that a low-frequency signal is allocated a larger bitrate such that the low-frequency signal exhibits a better quality. Since the low-frequency signal is more easily perceived by human ears than a highfrequency signal, the low-frequency signal is allocated a larger bitrate, and the quality of the encoded audio data may be better improved.

[0036] In a specific embodiment, for audio data with a sampling frequency of 192 kHz, the audio data may be decomposed into four subband data items with a sampling frequency of 48 kHz; based on the sampling theorem, the effective frequency band range of the signal is half of the sampling frequency, and thus the effective frequency band range of the original 192 kHz audio data is 0-96 kHz; the subband data items resulting from the decomposition represents the effective frequency band ranges of 0-24 kHz, 24-48 kHz, 48-72 kHz and 72-96 kHz of the original audio data, respectively; and the subband data items are sequentially numbered to obtain serial numbers 1, 2, 3 and 4.

[0037] Based on the principle that the higher the sampling frequency is, the lower the importance is, the subband data items 1 therein may be set as having a proportion 0.6, the subband data items 2 therein may be set as having a proportion of 0.2, and the subband data items 3 and 4 therein are respectively set as having a proportion 0.1, so that the low-frequency signal is allocated a larger bitrate, and the quality of low-frequency signal is improved.

[0038] It may be seen that through the present embodiment, the first subband data item representing the minimum frequency in the frequency band range of the audio data can be allocated the maximum bitrate, so that based on the principle that the low-frequency signal is more easily perceived by human ears than the highfrequency signal, the quality of the low-frequency signal is improved, and the quality of the encoded audio data is enhanced.

[0039] In the above step 108, after the desired post-encoding bitrate of each of the plurality of subband data items is determined, each of the plurality of subband data items is encoded according to the desired post-encoding bitrate of each of the plurality of subband data items. Each of the plurality of subband data items may be encoded by a well-established encoder such as an Opus encoder. The Opus encoder has advantages such as a high post-encoding compression rate and a high quality of the encoded audio data.

[0040] In the above step S110, the encoded audio data is generated according to all the encoded subband data items. For example, the encoded audio data is obtained by synthesizing all the encoded subband data items based on a preset bitstream structure.

[0041] In one embodiment, the generating encoded audio data according to all the encoded subband data items comprises: determining encoded first subband data item and encoded second subband data item from all encoded subband data items; the first subband data item is a subband data items representing a minimum frequency in the frequency band range of the audio data among all subband data items, and the second subband data item is a subband data items among all subband data items other than the first subband data item; filling the encoded first subband data item and the encoded second subband data item at corresponding fields in the preset bitstream structure, to generate the encoded audio data.

[0042] Specifically, all the encoded subband data items comprise a first subband data item and a plurality of second subband data item, the first subband data item is a subband data item representing a minimum frequency in the frequency band range of the audio data among all subband data items, and the second subband data item is a subband data items among all subband data items other than the first subband data item. First, the encoded first subband data item and each encoded second subband data item is determined from all the encoded subband data items.

[0043] Then, the encoded first subband data item and the encoded second subband data item are filled at corresponding fields in the preset bitstream structure, to generate the encoded audio data. The preset bitstream structure may be a bitstream structure corresponding to the encoder. For example, the respective subband data items are encoded by the Opus encoder, the preset bitstream structure is a bitstream structure possessed by the Opus encoder.

[0044] It can be seen that through the present embodiment, the first subband data item representing the minimum frequency in the frequency band range of the audio data and other subband data items can be filled at the corresponding fields in the preset bitstream structure, to thereby generate the encoded audio data. The generating the encoded audio data in a data-filling manner is advantageous in a high generation efficiency.

[0045] In one embodiment, the filling the encoded first subband data item and the encoded second subband data item at corresponding fields in the preset bitstream structure, to generate the encoded audio data comprises: filling the encoded first subband data item at a field in the bitstream structure for filling the encoded audio data; filling the encoded second subband data item at a field in the bitstream structure for filling additional data; generating the encoded audio data based on the filled bitstream structure.

[0046] Specifically, there are a field for filling the encoded audio data and a field for filling additional data in the preset bitstream structure. Regarding the audio data, it is data presented frame by frame in a chronological order. Therefore, each frame of audio data has a corresponding frame of bitstream data, and one frame of bitstream data is an encoded frame of audio data. When a length of each frame of bitstream data needs to be adjusted as the same length, additional data such as some meaningless character string such as 000 are filled at the field for filling the additional data, so that the length of each frame of bitstream data is adjusted as the same length.

[0047] In the present embodiment, the encoded first subband data item is filled at the field in the bitstream structure for filling the encoded audio data to enable the encoded first subband data item to be parsed and processed by the audio data receiver end as the encoded audio data; the encoded second subband data item is filled at the field in the bitstream structure for filling the additional data, so that when the audio data receiver end, upon having a capability of parsing and processing the additional data, extracts, from the field for filling the additional data, each encoded second subband data item to thereby decode the encoded audio data. Certainly, when the audio data receiver end does not have the capability of parsing and processing the additional data, the receiver end may directly discard the encoded second subband data item filled at the field for filling the additional data, and achieves the decoding work only based on the encoded first subband data item. Since the first subband data item is the most important signal representing the minimum frequency in the frequency band range of the audio data, achieving the decoding work based on the encoded first subband data item only may allow the receiver end to play the high-quality audio data.

[0048] In the present embodiment, the encoded audio data is also generated based on the filled bitstream structure, for example, the filled bitstream structure is taken as the encoded audio data.

[0049] It can be seen that through the present embodiment, the encoded first subband data item is filled at the field in the bitstream structure for filling the encoded audio data to enable the encoded first subband data item to be parsed and processed by the audio data receiver end as the encoded audio data; the encoded second subband data item is filled at the field in the bitstream structure for filling the additional data, so that when the audio data receiver end, upon having a capability of parsing and processing the additional data, extracts, from the field for filling the additional data, each encoded second subband data item to thereby decode the encoded audio data. In addition, when the audio data receiver end does not have the capability of parsing and processing the additional data, the receiver end may directly discard the encoded second subband data item filled at the field for filling the additional data, and achieves the decoding work only based on the encoded first subband data item, so that the encoded audio data may match the decoding capabilities of different receiver ends. Furthermore, since the first subband data item is the most important signal representing the minimum frequency in the frequency band range of the audio data, achieving the decoding work based on the encoded first subband data item only may allow the receiver end to play the high-quality audio data.

[0050] In one embodiment, the preset bitstream structure comprises a first field and a second field, a field value of the first field being used for indicating a length of each encoded second subband data item, and a field value of the second field being used for indicating a total length of all encoded second subband data item and the first field; accordingly, the generating the encoded audio data based on the filled bitstream structure comprises: in the filled bitstream structure, determining the field value of the first field according to the length of each encoded second subband data item; in the filled bitstream structure, determining the field value of the second field according to the total length of all encoded second subband data item and the first field; generating the encoded audio data based on the determined bitstream structure.

[0051] Specifically, the preset bitstream structure further comprises the first field and the second field, the field value of the first field being used for representing the length of each encoded second subband data item, and the field value of the second field being used for representing the total length of all encoded second subband data item and the first field. Based on this, in the present embodiment, after the encoded first subband data item and the encoded second subband data item are filled into the preset bitstream structure, the field value of the first field is also determined according to the length of each encoded second subband data item in the filled bitstream structure, the number of the first fields may be plural, each encoded second subband data item corresponds to one first field whose field value is equal to the length of the corresponding encoded second subband data item.

[0052] In addition, in the filled bitstream structure, the field value of the second field is determined according to the total length of all encoded second subband data items and the first field, the number of the second field is one, and the field value of the second field is equal to the total length of all encoded second subband data items and the first field.

[0053] Finally, the encoded audio data is generated based on the bitstream structure after the field value is determined. For example, the bitstream structure after the field value is determined is determined as the encoded audio data. Of course, then encoded audio data may further comprise some common fields such as a frame header field etc. The frame header field is used for storing parameters such as a length of the bitstream data, an encoding bandwidth, the number of channels, etc.

[0054] It may be seen that through the present embodiment, the encoded audio data can be enabled to carry the first field and the second field, the length of each encoded second subband data item is indicated by the field value of the first field, and the total length of all encoded second subband data items and the first field is indicated by the field value of the second field, thereby facilitating the decoding of the encoded audio data and improving the decoding efficiency.

[0055] In a specific embodiment, when each of the plurality of subband data items is encoded by an Opus encoder, the above-mentioned preset bitstream structure is a bitstream structure possessed by the Opus encoder. FIG. 2a is a schematic diagram of a preset bitstream structure according to an embodiment of the present disclosure. As shown in FIG. 2a, the bitstream structure of the Opus encoder comprises a frame header field, an additional data total length field, an encoded data field and an additional data field. The encoded data field is also referred to as a payload field, and the additional data field is also referred to as a filling field.

[0056] The length of the frame header field is a fixed value, typically 1-2 bytes. The frame header field is used to store parameters such as a length of the bitstream data, an encoding bandwidth, the number of channels, etc. The frame header field also includes a flag bit indicating whether the bitstream data contains filling data. When the flag bit is 1, it indicates that the bitstream data contains the filling data; if the flag bit is 0, this indicates that the bitstream data does not contain the filling data.

[0057] The additional data total length field is the aforementioned second field. The length of the additional data total length field is a fixed value, usually 1-2 bytes. The field value of the additional data total length field is used to indicate the total length of the additional data field.

[0058] The encoded data field is the above-mentioned field for filling the encoded audio data. The length of the encoded data field is not fixed, depending on the data length of the encoded first subband data item.

[0059] The additional data field is the above-mentioned field for filling the additional data. The length of the additional data field is not fixed, depending on the data length of the encoded second subband data item. The above-mentioned first field is located in the additional data field.

[0060] FIG.2b is a schematic diagram of encoded audio data according to an embodiment of the present disclosure. As shown in FIG.2b, encoded first subband data item is sub-bitstream-1 data in FIG.2b, and encoded second subband data item are sub-bitstream-2 data, sub-bitstream-3 data and sub-bitstream-4 data in FIG.2b. Continuing with the previous example, a frequency in the frequency band range of the above-mentioned audio data represented by the sub-bitstream -1 data is the lowest, and a bitrate accounts for 60% of the desired bitrate of the encoded audio data; a frequency in the frequency band range of the above-mentioned audio data represented by the sub-bitstream-2 data is higher than that of the sub-bitstream-1 data, and a bitrate accounts for 20% of the desired bitrate of the encoded audio data; a frequency in the frequency band range of the above-mentioned audio data represented by the sub-bitstream-3 data is higher than that of the sub-bitstream-2 data, and a bitrate accounts for 10% of the desired bitrate of the encoded audio data; a frequency in the frequency band range of the above-mentioned audio data represented by the sub-bitstream-4 data is higher than that of the sub-bitstream-3 data, and a bitrate accounts for 10% of the desired bitrate of the encoded audio data.

[0061] As shown in FIG. 2b, after the encoded first subband data item and each encoded second subband data item is obtained, the encoded first subband data item a filled in the encoded data field, and each encoded second subband data item is filled in the additional data field. Furthermore, the additional data field further comprises the first field corresponding to each encoded second subband data item, i.e., the field represented by "sub-bitstream-2 length", "sub-bitstream-3 length", and "sub-bitstream-4 length" in the figure. The first field is used to represent the length of the corresponding encoded second subband data item.

[0062] In FIG. 2b, the additional data total length field is the aforementioned second field, and the field value thereof is used to represent the length of the additional data. As can be seen from the figure, the length of the additional data is the total length of all encoded second subband data items and all first fields.

[0063] As can be seen from FIG. 2a and FIG. 2b, after each of the plurality of subband data items is encoded by the Opus encoder, the generated bitstream structure of the encoded audio data is the same as the bitstream structure carried by the Opus encoder itself, so that the encoded audio data is compatible with the Opus encoder. If the receiver end also uses the Opus encoder to decode the encoded audio data, and if the receiver end has the ability to recognize and parse the additional data field, the decoded data corresponding to each of the plurality of subband data items may be parsed and obtained; if the receiver end does not have the ability to recognize and parse the additional data field, the additional data field may be discarded, only the decoded data corresponding to the first subband data item may be parsed and obtained, and playing is performed based on the decoded data corresponding to the first subband data item.

[0064] When each of the plurality of subband data items is encoded by the Opus encoder, a limitation of the desired post-encoding bitrate native to the Opus encoder may be changed, and a desired post-encoding bitrate suitable for high-resolution audio data may be configured. Specifically, the Opus encoder natively supports a desired post-encoding bitrate of up to 510 kbps, and the value may be modified to 2400 kbps so that the Opus encoder may encode the audio data with high resolution, such as 192 kHz sampling frequency. The developer may also configure the desired bitrate of the encoded audio data in the Opus encoder, and the desired bitrate may be configured successfully as long as it is below 2400 kbps.

[0065] FIG. 3 illustrates a schematic diagram showing principles of audio data encoding according to an embodiment of the present disclosure. As shown in FIG.3, in the same way as the foregoing introduction, upon encoding, subband decomposition is first performed on the audio data to obtain a plurality of subband data items, and sequence numbers of the subband data items may be determined according to a level of the frequency in the above-mentioned frequency band range of the audio data represented by the subband data items. The lower the frequency in the represented frequency range, the smaller the sequence number. Then, each of the plurality of subband data items is encoded to obtain the encoded subband data items. The encoded subband data items may be referred to as a sub-bitstream. Then bitstream multiplexing is performed on each sub-bitstream in the manner shown in FIG. 2b. The bitstream multiplexing is filling each sub-bitstream into the preset bitstream structure to obtain the encoded audio data, and a bitstream is finally output. The output bitstream is the encoded audio data.

[0066] The process of generating the encoded audio data is introduced above, corresponding to a bitstream structure of the encoded audio data. In one embodiment, the encoded audio data is decoded in the following manner to obtain decoded data: extracting the encoded first subband data item and each encoded second subband data item from the encoded audio data according to a field value of the first field, a field value of the second field, a length of the second field, a length of a frame header field of the encoded audio data and a total length of the encoded audio data; obtaining decoded data based on the encoded first subband data item and each encoded second subband data item.

[0067] Firstly, the encoded first subband data item and each encoded second subband data item is extracted from the encoded audio data according to the field value of the first field, the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data and the total length of the encoded audio data. Then, decoded data is obtained by synthesizing the subbands based on the encoded first subband data item and each encoded second subband data item. Considering that the encoding process is lossy encoding, the data obtained by decoding the encoded audio data is referred to as decoded data.

[0068] It can be seen that through the present embodiment, the encoded first subband data item and each encoded second subband data item can be extracted from the encoded audio data, and then the decoded data corresponding to the encoded audio data can be obtained based on the encoded first subband data item and each encoded second subband data item, with a simple decoding process and a high decoding efficiency.

[0069] In one embodiment, the extracting the encoded first subband data item and each encoded second subband data item from the encoded audio data according to a field value of the first field, a field value of the second field, a length of the second field, a length of a frame header field of the encoded audio data and a total length of the encoded audio data comprises: extracting each encoded second subband data item from the encoded audio data according to the field value of the first field and the field value of the second field; extracting the encoded first subband data item from the encoded audio data according to the field value of the second field, the length of the second field, the length of a frame header field of the encoded audio data and the total length of the encoded audio data.

[0070] First, since the field value of the first field is used to represent the length of each encoded second subband data item and the field value of the second field is used to represent the sum of lengths of all encoded second subband data items and all first fields, each encoded second subband data item can be extracted from the encoded audio data according to the field value of the first field and the field value of the second field.

[0071] Then, the encoded first subband data item is extracted from the encoded audio data according to the field value of the second field, the length of the second field, the length of a frame header field of the encoded audio data and the total length of the encoded audio data.

[0072] It can be seen that through the present embodiment, the encoded first subband data item and each encoded second subband data item can be extracted efficiently and quickly from the encoded audio data.

[0073] In one embodiment, the extracting each encoded second subband data item from the encoded audio data according to the field value of the first field and the field value of the second field comprises: extracting additional data from the encoded audio data according to the field value of the second field; the additional data comprises each encoded second subband data item and each first field; extracting each encoded second subband data item from the additional data according to the field value of the first field.

[0074] Specifically, as known from the above depictions, the field value of each first field is used to represent the length of each encoded second subband data item, and the field value of the second field is used to represent the total length of all encoded second subband data items and all first fields. Based on this, with reference to FIG. 2b, all encoded second subband data items and all first fields may be obtained by looking up, i.e., the additional data field may be obtained by looking up starting from an end of the encoded audio data, i.e., the bitstream data, according to the field value of the second field.

[0075] Then, each encoded second subband data item is sequentially extracted starting from a starting portion of the additional data according to the field value of each first field. In FIG. 2b, a left side of the bitstream data is set as a data start position, a right side of the bitstream data is set as a data end position, and the bitstream data is arranged from left to right.

[0076] It can be seen that through the present embodiment, each encoded second subband data item can be efficiently and quickly extracted from the encoded audio data based on the field value of the first field and the field value of the second field.

[0077] In one embodiment, the extracting the encoded first subband data item from the encoded audio data according to the field value of the second field, the length of the second field, the length of a frame header field of the encoded audio data and the total length of the encoded audio data comprises: determining the length of the encoded first subband data item according to the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data and the total length of the encoded audio data; extracting the encoded first subband data item from the encoded audio data according to the length of the encoded first subband data item.

[0078] Specifically, as shown in FIG. 2b, the total length of the encoded audio data may be extracted from the frame header field, and the length of the encoded first subband data item, namely, the length of the encoded data field, can be calculated with the following equation: the length of the encoded data field=the total length-the length of the frame header field-the length of the additional data total length field-the length of the additional data field; wherein the length of the additional data field is the field value of the second field, the length of the additional data total length field is the length of the second field, and generally, the length of the additional data total length field is a fixed value and may also be recorded in the frame header field.

[0079] Therefore, the length of the encoded first subband data item is equal to the total length of the encoded audio data minus the length of the frame header field minus the field value of the second field minus the length of the second field.

[0080] Then, the encoded first subband data item may be obtained by looking up starting from a starting position of the encoded audio data by skipping the frame header field with a fixed length and the additional data total length field with a fixed length, according to the length of the encoded first subband data item.

[0081] It can be seen that through the present embodiment, the length of the encoded first subband data item can be determined according to the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data and the total length of the encoded audio data, and then the encoded first subband data item can be extracted accurately from the encoded audio data according to the length of the encoded first subband data item.

[0082] After extracting the encoded first subband data item and each encoded second subband data item, in one embodiment, the obtaining decoded data based on the encoded first subband data item and each encoded second subband data item comprises: decoding the encoded first subband data item and each encoded second subband data item to obtain first sub-decoded data item and each second sub-decoded data item, respectively; performing subband synthesis on the first sub-decoded data and each second sub-decoded data item to obtain decoded data.

[0083] Specifically, the encoded first subband data item is decoded to obtain first sub-decoded data; since the first subband data item is encoded in a lossy encoding manner, the data obtained by decoding the encoded first subband data item is referred to as first sub-decoded data; next, each encoded second subband data item is decoded to obtain the second sub-decoded data corresponding to each encoded second subband data item; since the second subband data item is encoded in a lossy encoding manner, the data obtained by decoding the encoded second subband data item is referred to as second sub-decoded data. One encoded second subband data item corresponds to one second sub-decoded data.

[0084] Then, subband synthesis is performed on the first sub-decoded data and each second sub-decoded data item to obtain the decoded data. The subband synthesis may be performed by a PQMF method.

[0085] In a specific embodiment, after receiving the bitstream data as shown in FIG. 2b, the receiver end firstly parses the frame header field, and acquires, from the frame header field, a flag bit indicating whether the current bitstream data contains the additional data field. If the current bitstream data contains the additional data field, the length of the additional data field is firstly parsed out, then the additional data is taken out from the tail of the bitstream data according to the additional data field, and the length of each encoded second subband data item is parsed in sequence from the additional data and each second subband data item is extracted. Then, the length of the encoded first subband data item is calculated, and the encoded first subband data item is extracted from the bitstream data according to the length.

[0086] Then, Opus decoding is performed on the encoded first subband data item and each encoded second subband data item to obtain the first sub-decoded data item and each second sub-decoded data item, respectively, and finally subband synthesis is performed on the first sub-decoded data item and each second sub-decoded data item by the PQMF method to obtain the decoded data.

[0087] It can be seen that through the present embodiment, decoding and synthesis can be performed on the encoded subband data items, to efficiently obtain the decoded data.

[0088] FIG. 4 illustrates a schematic diagram showing principles of audio data decoding according to an embodiment of the present disclosure. As shown in FIG.4, in the same way as the foregoing introduction, after acquiring an input bitstream, i.e., the encoded audio data, the receiver end performs bitstream parsing to obtain sub-bitstreams, i.e., each encoded subband data item, decodes each encoded subband data item to obtain each sub-decoded data item, the sub-decoded data comprising the aforementioned first sub-decoded data item and second sub-decoded data item, then performs subband synthesis on each sub-decoded data item to obtain output data which is decoded data.

[0089] In a specific embodiment, the encoding process and decoding process described above may be implemented based on an Opus encoder. A specific implementation process is as follows: 1. The developer modifies the desired post-encoding maximum bitrate 510 kbps natively supported by the Opus encoder to 2400 kbps, so that the Opus encoder may encode audio data with a high resolution, such as a sampling frequency of 192 kHz; 2. The developer configures a desired bitrate of the encoded audio data inside the Opus encoder, and the desired bitrate may be configured successfully as long as it is lower than 2400 kbps; 3. The high-resolution audio data is acquired, and the audio data is input into the Opus encoder; 4. The Opus encoder performs subband decomposition on audio data, and determines a desired post-encoding bitrate of each of the plurality of subband data items according to a pre-configured desired bitrate; 5. The Opus encoder encodes each of the plurality of subband data items according to the desired post-encoding bitrate of each of the plurality of subband data items; 6. The Opus encoder generates encoded audio data according to each encoded subband data items.

[0090] Certainly, the Opus encoder in the present embodiment may be replaced with an encoder such as an EVS encoder or an AAC encoder. It can be seen that the following technical effects are achieved through the present embodiment: 1. The sampling frequency which may be supported by the Opus encoder is expanded, and for high-resolution audio data, encoding and decoding may also be performed in the Opus encoder by means of subband decomposition; 2. The high-resolution audio data is encoded by subband decomposition so that a higher compression rate is achieved, and the encoded audio data may be applied to a real-time communication scenario such as a video conference scenario. Experiments prove that, for audio data with a sampling frequency of 192 kHz and a bit depth of 24 bits, the whole frequency band may be encoded at a bitrate of 500 kbps, the quality of the audio data after the encoding and decoding is substantially free from damages, and the compression rate reaches more than 10 times.

[0091] FIG. 5 illustrates a schematic diagram of an application scenario of audio data encoding and decoding according to an embodiment of the present disclosure. As shown in FIG.5, in a real-time communication scenario, such as a video conference scenario, a plurality of terminal devices (represented by terminal device 501 and terminal device 502 in the figure) may realize the transmission of real-time audio data through a server 503. Through the encoding and decoding method in the present embodiment, the bitrate of audio data may be substantially reduced, so as to achieve real-time transmission.

[0092] Furthermore, the server may also acquire an encoding and decoding capability supported by each terminal device; when any one of a plurality of terminal devices in real-time communication in the same scenario, such as the same room or the same conference, does not support the capability of encoding and decoding the high-resolution audio data in the above embodiments, the server notifies each terminal device to collect and generate low-resolution audio data (the sampling frequency is lower than the preset frequency or the sampling depth is lower than the preset depth); when each terminal device for real-time communication in the same scenario supports the capability of encoding and decoding the high-resolution audio data in the above embodiments, the server notifies each terminal device to collect and generate high-resolution audio data (the sampling frequency is greater than or equal to the preset frequency and the sampling depth is greater than or equal to the preset depth), and executes the above solution, thereby improving the real-time audio communication quality.

[0093] Corresponding to the above embodiments, an embodiment of the present disclosure provides an audio data processing apparatus. FIG. 6 illustrates a block diagram of an audio data processing apparatus according to an embodiment of the present disclosure. As shown in FIG. 6, the apparatus comprises: an acquisition unit 61 configured to acquire audio data to be processed whose sampling frequency is greater than or equal to a preset frequency and whose sampling depth is greater than or equal to a preset depth; a decomposition unit 62 configured to perform subband decomposition on the audio data to obtain a plurality of subband data items; a determination unit 63 configured to determine a desired post-encoding bitrate of each of the plurality of subband data items; a sum of desired post-encoding bitrates of all the subband data items is equal to a desired post-encoding bitrate of the audio data; a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than a preset value; an encoding unit 64 configured to encode each of the plurality of subband data items according to the desired post-encoding bitrate of each of the plurality of subband data items; a generation unit 65 configured to generate encoded audio data according to each of the encoded subband data items.

[0094] Optionally, the determination unit 63 is specifically configured to: acquire a proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data; determine the desired post-encoding bitrate of each of the plurality of subband data items according to the proportion of the desired post-encoding bitrate of the audio data to each of the plurality of subband data items.

[0095] Optionally, the determination unit 63 is specifically configured to: determine a frequency band range of the audio data represented by each of the plurality of subband data items; determine the proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data, according to the frequency band range of the audio data represented by each of the plurality of subband data items.

[0096] Optionally, the determination unit 63 is specifically configured to: for a first subband data item representing a minimum frequency in the frequency band range of the audio data among all the subband data items, determine the proportion of the first subband data items is greater than the proportion of a second subband data item among all the subband data items; the second subband data item is a subband data item among all the subband data items other than the first subband data item.

[0097] Optionally, the generation unit 65 is specifically configured to: determine encoded first subband data item and encoded second subband data item from all encoded subband data items; the first subband data item is a subband data items representing a minimum frequency in the frequency band range of the audio data among all the subband data items, and the second subband data item is a subband data items among all the subband data items other than the first subband data item; fill the encoded first subband data item and the encoded second subband data item at corresponding fields in a preset bitstream structure, to generate the encoded audio data.

[0098] Optionally, the generation unit 65 is further specifically configured to: fill the encoded first subband data item at a field in the bitstream structure for filling the encoded audio data; fill the encoded second subband data item at a field in the bitstream structure for filling additional data; generate the encoded audio data based on the filled bitstream structure.

[0099] Optionally, the bitstream structure comprises a first field and a second field, a field value of the first field being used for representing a length of each of the encoded second subband data item, and a field value of the second field being used for representing a total length of all the encoded second subband data items and the first field; the generation unit 65 is further specifically configured to: in the filled bitstream structure, determine the field value of the first field according to the length of each of the encoded second subband data item; in the filled bitstream structure, determine the field value of the second field according to the total length of all the encoded second subband data items and the first field; generate the encoded audio data based on the determined bitstream structure.

[0100] Optionally, the apparatus further comprises a decoding unit configured to: extract the encoded first subband data item and each of the encoded second subband data item from the encoded audio data according to a field value of the first field, a field value of the second field, a length of the second field, a length of a frame header field of the encoded audio data and a total length of the encoded audio data; obtain the decoded data based on the encoded first subband data item and each of the encoded second subband data item.

[0101] Optionally, the decoding unit is specifically configured to: extract each of the encoded second subband data item from the encoded audio data according to the field value of the first field and the field value of the second field; extract the encoded first subband data item from the encoded audio data according to the field value of the second field, the length of the second field, the length of a frame header field of the encoded audio data and the total length of the encoded audio data.

[0102] Optionally, the decoding unit is further specifically configured to: extract the additional data from the encoded audio data according to the field value of the second field; the additional data comprises each of the encoded second subband data items and the first field; extract each of the encoded second subband data items from the additional data according to the field value of the first field.

[0103] Optionally, the decoding unit is further specifically configured to: determine the length of the encoded first subband data item according to the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data and the total length of the encoded audio data; extract the encoded first subband data item from the encoded audio data according to the length of the encoded first subband data item.

[0104] Optionally, the decoding unit is specifically configured to: decode the encoded first subband data item and each of the encoded second subband data items to obtain first sub-decoded data item and each second sub-decoded data item, respectively; perform subband synthesis on the first sub-decoded data item and each second sub-decoded data item to obtain the decoded data.

[0105] The audio data processing apparatus in the embodiment of the present disclosure may implement the processes of the above audio data processing method embodiment, and achieve the same effects and functions, which will not be repeated any longer here.

[0106] An embodiment of the present disclosure further provides an electronic device. FIG.7 illustrates a structural diagram of the electronic device according to an embodiment of the present disclosure. As shown in FIG.7, the electronic device may vary greatly with different configuration or performance, and may include one or more processors 701 and one or more memories 702. The memory 702 may store one or more applications or data. The memory 702 may be transient storage or persistent storage. The application stored in the memory 702 may include one or more modules (not shown), each of which may include a series of computer-executable instructions in the electronic device. Furthermore, the processor 701 may be set in communication with the memory 702, and the series of computer-executable instructions in the memory 702 are executed on the electronic device. The electronic device may also include one or more power supplies 703, one or more wired or wireless network interfaces 704, one or more input or output interfaces 705, one or more keyboards 706, etc.

[0107] In a specific embodiment, the electronic device comprises a processor; and, a memory configured to store computer-executable instructions that, when executed, cause the processor to implement the following steps: acquiring audio data to be processed whose sampling frequency is greater than or equal to a preset frequency and whose sampling depth is greater than or equal to a preset depth; performing subband decomposition on the audio data to obtain a plurality of subband data items; determining a desired post-encoding bitrate of each of the plurality of subband data items; a sum of desired post-encoding bitrates of all the subband data items is equal to a desired post-encoding bitrate of the audio data; a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than a preset value; encoding each of the plurality of subband data items according to the desired post-encoding bitrate of each of the plurality of subband data items; generating encoded audio data according to each of the encoded subband data items.

[0108] The electronic device in the embodiment of the present disclosure may implement the steps of the above audio data processing method embodiment and achieve the same effects and functions, which will not be repeated any more here.

[0109] Another embodiment of the present disclosure further provides a computer-readable storage medium for storing computer-executable instructions that, when executed by a processor, implement the following steps: acquiring audio data to be processed whose sampling frequency is greater than or equal to a preset frequency and whose sampling depth is greater than or equal to a preset depth; performing subband decomposition on the audio data to obtain a plurality of subband data items; determining a desired post-encoding bitrate of each of the plurality of subband data items; a sum of desired post-encoding bitrates of all the subband data items is equal to a desired post-encoding bitrate of the audio data; a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than a preset value; encoding each of the plurality of subband data items according to the desired post-encoding bitrate of each of the plurality of subband data items; generating encoded audio data according to each of the encoded subband data items.

[0110] The storage medium in the present embodiment of the present disclosure may implement the steps of the above audio data processing method embodiment and achieve the same effects and functions, which will not be repeated any more here.

[0111] In various embodiments of the present disclosure, the computer-readable storage medium includes read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, an optical disk, or the like.

[0112] In the 1990s, whether a technical improvement is a hardware improvement (for example, an improvement to a circuit structure, such as a diode, a transistor, or a switch) or a software improvement (an improvement to a method procedure) can be clearly distinguished. However, as technologies develop, the current improvement for many method procedures can be considered as a direct improvement of a hardware circuit structure. A designer usually programs an improved method procedure into a hardware circuit, to obtain a corresponding hardware circuit structure. Therefore, a method procedure can be improved by using a hardware entity module. For example, a programmable logic device (PLD) (for example, a field programmable gate array (FPGA)) is such an integrated circuit, and a logical function of the programmable logic device is determined by a user through device programming. The designer performs programming to "integrate" a digital system to a PLD without requesting a chip manufacturer to design and produce an application-specific integrated circuit chip. In addition, at present, instead of manually manufacturing an integrated chip, this type of programming is mostly implemented by using "logic compiler" software. The programming is similar to a software compiler used to develop and write a program. Original code needs to be written in a particular programming language for compilation. The language is referred to as a hardware description language (HDL). There are many HDLs, such as the Advanced Boolean Expression Language (ABEL), the Altera Hardware Description Language (AHDL), Confluence, the Cornell University Programming Language (CUPL), HDCal, the Java Hardware Description Language (JHDL), Lava, Lola, MyHDL, PALASM, and the Ruby Hardware Description Language (RHDL). The very-high-speed integrated circuit hardware description language (VHDL) and Verilog are most commonly used. A person skilled in the art should also understand that a hardware circuit that implements a logical method procedure can be readily obtained once the method procedure is logically programmed by using the previously-mentioned several hardware description languages and is programmed into an integrated circuit.

[0113] A controller can be implemented by using any appropriate method. For example, the controller can be a microprocessor or a processor, or a computer-readable medium that stores computer readable program code (such as software or firmware) that can be executed by the microprocessor or the processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, or a built-in microprocessor. Examples of the controller include but are not limited to the following microprocessors: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as a part of the control logic of the memory. A person skilled in the art also knows that, in addition to implementing the controller by using the computer readable program code, logic programming can be performed on method steps to allow the controller to implement the same function in forms of the logic gate, the switch, the application-specific integrated circuit, the programmable logic controller, and the built-in microcontroller. Therefore, the controller can be considered as a hardware component, and an apparatus configured to implement various functions in the controller can also be considered as a structure in the hardware component. Or the apparatus configured to implement various functions can even be considered as both a software module implementing the method and a structure in the hardware component.

[0114] The system, apparatus, module, or unit illustrated in the previously-mentioned embodiments can be implemented by using a computer chip or an entity, or can be implemented by using a product having a certain function. A typical implementation device is a computer. A specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, an intelligent phone, a personal digital assistant, a media player, a navigation device, an email transceiver device, a game console, a tablet computer, a wearable device, or any combination thereof.

[0115] For convenience of description, the above apparatuses are described respectively in terms of their functions. Certainly, when the present disclosure is implemented, a function of each unit can be implemented in one or more pieces of software and / or hardware.

[0116] A person skilled in the art should understand that embodiments of the present disclosure can be provided as methods, systems, or computer program products. Therefore, the present disclosure can use a form of hardware only embodiments, software only embodiments, or embodiments with a combination of software and hardware. Further, the present disclosure can take a form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, and optical storage) containing computer-usable program code.

[0117] The present disclosure is described with reference to the flowcharts and / or block diagrams of the method, the device (system), and the computer program product based on the embodiments of the present disclosure. It is worthwhile to note that computer program instructions can be used to implement each process and / or each block in the flowcharts and / or the block diagrams and a combination of a process and / or a block in the flowcharts and / or the block diagrams. These computer program instructions can be provided for a general-purpose computer, a dedicated computer, an embedded processor, or a processor of another programmable data processing device to generate a machine, so the instructions executed by the computer or the processor of another programmable data processing device generate an apparatus for implementing a specific function in one or more processes in the flowcharts and / or in one or more blocks in the block diagrams.

[0118] These computer program instructions can be stored in a computer readable memory that can instruct the computer or another programmable data processing device to work in a specific way, so the instructions stored in the computer readable memory generate an artifact that includes an instruction apparatus. The instruction apparatus implements a specific function in one or more processes in the flowcharts and / or in one or more blocks in the block diagrams.

[0119] These computer program instructions can be loaded onto the computer or another programmable data processing device, so a series of operation steps are performed on the computer or another programmable device, thereby generating computer-implemented processing. Therefore, the instructions executed on the computer or another programmable device provide steps for implementing a specific function in one or more processes in the flowcharts and / or in one or more blocks in the block diagrams.

[0120] It is also worthwhile to note that terms "include", "contain" or any other variant is intended to cover non-exclusive inclusion, so processes, methods, products or devices that include a series of elements include not only those elements but also other elements that are not explicitly listed, or elements inherent in such processes, methods, products or devices. An element described by "includes a ..." further includes, without more constraints, another identical element in the process, method, product, or device that includes the element.

[0121] One or more embodiments of the present disclosure can be described in the general context of computer executable instructions executed by a computer, for example, a program module. Generally, the program module includes a routine, a program, an object, a component, a data structure, etc. executing a specific task or implementing a specific abstract data type. The one or more embodiments of the present disclosure can also be practiced in distributed computing environments. In the distributed computing environments, tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, the program module can be located in both local and remote computer storage media including storage devices.

[0122] The embodiments in the present disclosure are all described progressively, for same or similar parts in the embodiments, reference can be made to these embodiments, and each embodiment focuses on a difference from other embodiments. Particularly, a system embodiment is basically similar to a method embodiment, and therefore, is described briefly. For related parts, refer to partial descriptions in the method embodiment.

[0123] What are described above are embodiments of the present disclosure, and are not intended to limit the present disclosure. A person skilled in the art can make various modifications and changes to the present disclosure. Any modification, equivalent replacement or improvement made within spirit and principles of the present disclosure shall be included in claims of the present disclosure.

Examples

Embodiment Construction

[0010]Technical solutions in one or more embodiments of the present disclosure will be described clearly and completely with reference to figures in the one or more embodiments of the present disclosure to enable those skilled in the art to better understand the technical solutions in the one or more embodiments. Obviously, the embodiments described are partial embodiments other than all embodiments of the present disclosure. All other embodiments obtained by those having ordinary skill in the art based on one or more embodiments in the present disclosure without making any inventive efforts all fall within the protection scope of the present disclosure.

[0011]Embodiments of the present disclosure provide an audio data processing method, which, for audio data with a larger sampling frequency and a larger sampling depth, can reduce a bitrate of the encoded audio data, and improve an efficiency of encoding the audio data. For ease of understanding embodiments of the present disclosure,...

CROSS REFERENCE TO RELATED APPLICATIONS

[0001] The present disclosure claims a priority right to the Chinese patent application No. 202310738824.8 filed on June 20, 2023 with China National Intellectual Property Administration, the entire disclosure of which is hereby incorporated by reference in its entirety.FIELD

[0002] The present disclosure relates to the field of audio data processing, and particularly to an audio data processing method and apparatus, an electronic device and a storage medium.BACKGROUND

[0003] Before being transmitted, audio data needs to be encoded, the encoded audio data is transmitted by an audio data sender to an audio data receiver, and the audio data receiver, after having received the encoded audio data, decodes the encoded audio data and plays the decoded audio data. In the prior art, when audio data with a larger sampling frequency and a larger sampling depth is encoded, a bitrate of the encoded audio data is still very large, which causes a low efficiency of encoding the audio data.SUMMARY

[0004] Embodiments of the present disclosure provide an audio data processing method and apparatus, an electronic device and a storage medium, which, for audio data with a larger sampling frequency and a larger sampling depth, can reduce a bitrate of the encoded audio data, and improve an efficiency of encoding the audio data.

[0005] In a first aspect, an embodiment of the present disclosure provides an audio data processing method, comprising: acquiring audio data to be processed whose sampling frequency is greater than or equal to a preset frequency and whose sampling depth is greater than or equal to a preset depth; performing subband decomposition on the audio data to obtain a plurality of subband data items; determining a desired post-encoding bitrate of each of the plurality of subband data items; a sum of desired post-encoding bitrates of all the subband data items is equal to a desired post-encoding bitrate of the audio data; a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than a preset value; encoding each of the plurality of subband data items according to the desired post-encoding bitrate of each of the plurality of subband data items; generating encoded audio data according to each of the encoded subband data items.

[0006] In a second aspect, an embodiment of the present disclosure provides an audio data processing apparatus, comprising: an acquisition unit configured to acquire audio data to be processed whose sampling frequency is greater than or equal to a preset frequency and whose sampling depth is greater than or equal to a preset depth; a decomposition unit configured to perform subband decomposition on the audio data to obtain a plurality of subband data items; a determination unit configured to determine a desired post-encoding bitrate of each of the plurality of subband data items; a sum of desired post-encoding bitrates of all the subband data items is equal to a desired post-encoding bitrate of the audio data; a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than a preset value; an encoding unit configured to encode each of the plurality of subband data items according to the desired post-encoding bitrate of each of the plurality of subband data items; a generation unit configured to generate encoded audio data according to each of the encoded subband data items.

[0007] In a third aspect, an embodiment of the present disclosure provides an electronic device comprising: a processor; and a memory configured to store computer-executable instructions which, when executed, cause the processor to implement the steps of the method in the first aspect.

[0008] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium for storing computer-executable instructions that, when executed by a processor, implement the steps of the method in the first aspect.BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to illustrate the technical solutions in the present disclosure or the prior art more clearly, a brief introduction will be given to the figures to be used in the description of the embodiments or the prior art. It is obvious that the figures described below are merely some embodiments of the present disclosure, and other figures can also be obtained by those skilled in the art according to these figures without involving any inventive effort: FIG. 1 illustrates a flow chart of an audio data processing method according to an embodiment of the present disclosure; FIG. 2a illustrates a schematic diagram of a preset bitstream structure according to an embodiment of the present disclosure; FIG. 2b illustrates a schematic diagram of encoded audio data according to an embodiment of the present disclosure; FIG. 3 illustrates a schematic diagram showing principles of audio data encoding according to an embodiment of the present disclosure; FIG. 4 illustrates a schematic diagram showing principles of audio data decoding according to an embodiment of the present disclosure; FIG. 5 illustrates a schematic diagram of an application scenario of audio data encoding and decoding according to an embodiment of the present disclosure; FIG. 6 illustrates a block diagram of an audio data processing apparatus according to an embodiment of the present disclosure; FIG. 7 illustrates a block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF EMBODIMENTS

[0010] Technical solutions in one or more embodiments of the present disclosure will be described clearly and completely with reference to figures in the one or more embodiments of the present disclosure to enable those skilled in the art to better understand the technical solutions in the one or more embodiments. Obviously, the embodiments described are partial embodiments other than all embodiments of the present disclosure. All other embodiments obtained by those having ordinary skill in the art based on one or more embodiments in the present disclosure without making any inventive efforts all fall within the protection scope of the present disclosure.

[0011] Embodiments of the present disclosure provide an audio data processing method, which, for audio data with a larger sampling frequency and a larger sampling depth, can reduce a bitrate of the encoded audio data, and improve an efficiency of encoding the audio data. For ease of understanding embodiments of the present disclosure, technical terms involved are first defined as follows:

[0012] Sampling frequency: also referred to as a sampling rate, which means the amount of audio data collected per unit time, e.g., per second, i.e., the amount of audio data recorded per second, and a measure unit of the sampling frequency is kHz. For example, the sampling frequency of audio data is 192 kHz, which means that the amount of audio data collected per second is 192k. The higher the sampling frequency, the greater the amount of audio data.

[0013] Sampling depth: also referred to as a bit depth, which represents the number of bits required to acquire one audio data. For example, the sampling depth is 24 bits, which indicates that each audio data collected is represented by 24 bits. The larger the sampling depth, the greater the amount of data of the audio data.

[0014] PQMF: Pseudo-Quadrature Mirror Filter in a full form, which is a well-established filtering algorithm, through which sub-decomposition is performed on the audio data.

[0015] Opus: a well-established lossy encoder that supports a maximum desired encoding bitrate of 510 kbps for input.

[0016] Bitstream data: the encoded audio data is essentially a long string of binary data, which may be referred to as bitstream data. After receiving the bitstream data, a receiver end decodes the bitstream data and plays the decoded bitstream data. Since the audio data is encoded frame by frame during encoding, it may also be believed that each frame of the encoded audio data corresponds to one frame of bitstream data, and the receiver end decodes each frame of bitstream data and plays the decoded bitstream data.

[0017] Bitstream structure: the bitstream data is stored according to a certain data structure, and the data structure is a bitstream structure; generally, the bitstream structure at least comprises a frame header field and an encoding field, wherein the frame header field is used for storing parameters such as a length, an encoding bandwidth and a number of channels of the bitstream data, and the encoding field is used for storing encoded audio data. The receiver obtains the meaning and length of each field of the bitstream structure, parses the bitstream data and plays the parsed bitstream data.

[0018] In one or more embodiments of the present disclosure, audio data to be processed is acquired, a sampling frequency of the audio data is greater than or equal to a preset frequency and a sampling depth is greater than or equal to a preset depth, subband decomposition is performed on the audio data to obtain a plurality of subband data items, a desired bitrate of each encoded subband data item is determined, a sum of desired bitrates of all encoded subband data items is equal to a desired bitrate of the encoded audio data, a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than a preset value, each of the plurality of subband data items is encoded according to the desired bitrate of each encoded subband data item, and an encoded audio data is generated according to each encoded subband data item. Through the embodiment, as for the audio data whose sampling frequency is greater than or equal to the preset frequency and whose sampling depth is greater than or equal to the preset depth, the encoded audio data can be obtained by performing subband decomposition on the audio data and encoding the subband data items. Since the ratio of the pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than the preset value, the bitrate of the encoded audio data is greatly reduced and the efficiency of encoding the audio data is improved. Since the bitrate of the encoded audio data is reduced, an effect of saving the bandwidth for transmitting the encoded audio data and saving the data flow is also achieved.

[0019] FIG. 1 is a flow diagram of an audio data processing method according to an embodiment of the present disclosure. The method can be applied to and executed by a terminal device. The terminal device includes, but is not limited to, devices capable of collecting audio data and encoding and transmitting the audio data, such as a mobile phone, a computer, a tablet computer, a vehicle-mounted computer, a wearable device, etc. Certainly, the method can also be applied within a server and executed by the server. The method will be described below by taking a terminal device as an executing body. As shown in FIG. 1, the method comprises: Step S102: audio data to be processed is acquired whose sampling frequency is greater than or equal to a preset frequency and whose sampling depth is greater than or equal to a preset depth; Step S104: subband decomposition is performed on the audio data to obtain a plurality of subband data items; Step S106: a desired post-encoding bitrate of each encoded subband data item is determined; a sum of desired post-encoding bitrates of all subband data items being equal to a desired post-encoding bitrate of the audio data; a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data being greater than a preset value; Step S108: each of the plurality of subband data items is encoded according to the desired post-encoding bitrate of each subband data item; Step S110: an encoded audio data item is generated according to each encoded subband data item.

[0020] Through the embodiment, as for the audio data whose sampling frequency is greater than or equal to the preset frequency and whose sampling depth is greater than or equal to the preset depth, the encoded audio data can be obtained by performing subband decomposition on the audio data and encoding the subband data items. Since the ratio of the pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than the preset value, the bitrate of the encoded audio data is greatly reduced and the efficiency of encoding the audio data is improved. Since the bitrate of the encoded audio data is reduced, an effect of saving the bandwidth for transmitting the encoded audio data and saving the data flow is also achieved.

[0021] In the above step S102, audio data to be processed is acquired. The sampling frequency of the audio data to be processed is greater than or equal to the preset frequency and the sampling depth is greater than or equal to the preset depth. In one embodiment, the preset frequency may be 96 kHz and the preset depth may be 24 bits. Since the sampling frequency of the audio data to be processed is greater than or equal to the preset frequency and the sampling depth is greater than or equal to the preset depth, the audio data to be processed may also be referred to as high resolution Hi-Res audio data.

[0022] In the above step S104, subband decomposition is performed on the audio data to obtain a plurality of subband data items. In one embodiment, the subband decomposition may be performed on the audio data by a PQMF filter to obtain a plurality of subband data items. In one embodiment, the sampling frequency of each of the plurality of subband data items is less than or equal to the preset frequency so that the subband data items may be successfully encoded. For example, when each of the plurality of subband data items is encoded by an Opus encoder, since the Opus can encode audio data with a sampling frequency of at most 48 kHz, the sampling frequency of each of the plurality of subband data items is required to be less than or equal to 48 kHz. In this embodiment, with regard to the audio data with a sampling frequency of 192 kHz, the audio data may be decomposed into four subband data items with a sampling frequency of 48 kHz; based on the sampling theorem, an effective frequency band range of the signal is half of the sampling frequency, and therefore the effective frequency band range of the original 192 kHz audio data is 0-96 kHz; the decomposed subband data items respectively represent the effective frequency band ranges of 0-24 kHz, 24-48 kHz, 48-72 kHz and 72-96 kHz of the original audio data; and the subband data items are sequentially numbered to obtain serial numbers 1, 2, 3 and 4.

[0023] In the above step S106, a desired post-encoding bitrate of each of the plurality of subband data items is determined, a sum of desired post-encoding bitrates of all subband data items is equal to a desired post-encoding bitrate of the audio data, and a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than a preset value. First, the desired post-encoding bitrate of the audio data is obtained. The ratio of the pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than the preset value. The desired post-encoding bitrate of the audio data may be pre-stored in the terminal device. Upon pre-storing, the ratio of the pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than the preset value. For example, if the preset value is 10, the ratio of the pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than 10, i.e., an encoding compression rate of the audio data is allowed to reach more than 10 times.

[0024] In a specific example, for 192 kHz 24-bit dual-channel audio data, the pre-encoding bitrate is 192000*24*2bps, and the desired post-encoding bitrate may be configured to be 500000 bps, so that 192000*24*2 / 500000=18.432, so that the compression rate reaches more than 18 times. The magnitude of the preset value may be empirically determined to ensure a high quality of the encoded audio data while ensuring a large compression rate of the audio data.

[0025] In one embodiment, the preset value is determined empirically in advance; according to the preset value and the pre-encoding bitrate of commonly-used high-resolution audio data, the desired post-encoding bitrate of the commonly-used high-resolution audio data is determined and stored in the terminal device, and in this case, the desired post-encoding bitrate of the audio data is a fixed value.

[0026] In another embodiment, the preset value is configured in an encoder of the terminal device in advance; upon encoding, the terminal device may acquire the pre-encoding bitrate of the audio data, and automatically generate the desired post-encoding bitrate based on the preset value and the pre-encoding bitrate, and make the ratio of the pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data greater than the preset value. In this case, the desired post-encoding bitrate of the audio data is a value calculated in real time upon encoding.

[0027] In the above step S106, the desired post-encoding bitrate of each of the plurality of subband data items is also determined based on the desired post-encoding bitrate of the audio data, so that the sum of desired post-encoding bitrates of all subband data items is equal to the desired post-encoding bitrate of the audio data.

[0028] In one embodiment, the determining the desired post-encoding bitrate of each of the plurality of subband data items comprises: acquiring a proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data; determining the desired post-encoding bitrate of each of the plurality of subband data items according to the proportion of the desired post-encoding bitrate of the audio data to each of the plurality of subband data items.

[0029] Specifically, first, the proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data is obtained. It can be appreciated that a sum of proportions of each of the plurality of subband data items is 1 so that the sum of the desired post-encoding bitrates of the subband data items is equal to the desired post-encoding bitrate of the audio data. Then, the desired post-encoding bitrate of each of the plurality of subband data items is determined according to the proportion of the desired post-encoding bitrate of the audio data to each of the plurality of subband data items. For example, as for each of the plurality of subband data items, the corresponding proportion is multiplied by the desired post-encoding bitrate of the audio data, to obtain the desired post-encoding bitrate of each of the plurality of subband data items.

[0030] It can be seen that through the present embodiment, the desired post-encoding bitrate of each of the plurality of subband data items can be determined efficiently and quickly by determining the proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data, and the bitrate-determining process is simple and efficient.

[0031] In one embodiment, the acquiring a proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data comprises: determining a frequency band range of the audio data represented by each of the plurality of subband data items; determining the proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data, according to the frequency band range of the audio data represented by each of the plurality of subband data items.

[0032] First, the frequency band range of the audio data represented by each of the plurality of subband data items is determined. For example, in the preceding example, for subband data items 1, the frequency band range of the audio data represented by it is in a range of 0-24 kHz. Then, the proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data is determined according to the frequency band range of the audio data represented by each of the plurality of subband data items. For example, the desired post-encoding bitrate of a subband data items representing a lower frequency band range of the audio data corresponds to a larger proportion.

[0033] It can be seen that through the present embodiment, since a relationship between the frequency band range of the audio data represented by the subband data items and an importance degree of the subband data items is taken into consideration, the determining the proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data, according to the frequency band range of the audio data represented by each of the plurality of subband data items enables important subband data items to be allocated a larger bitrate, and improves the quality of the encoded audio data.

[0034] In one embodiment, the determining the proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data, according to the frequency band range of the audio data represented by each of the plurality of subband data items comprises: for a first subband data item representing a minimum frequency in the frequency band range of the audio data among all subband data items, determining the proportion of the first subband data item is greater than the proportion of a second subband data item among all subband data items; the second subband data item is a subband data items among all subband data items other than the first subband data item.

[0035] Specifically, the number of the subband data items is plural. The first subband data item representing a minimum frequency in the frequency band range of the audio data corresponds to a larger proportion than the proportions to which other subband data items correspond. Therefore, the first subband data item is allocated a maximum bitrate so that a low-frequency signal is allocated a larger bitrate such that the low-frequency signal exhibits a better quality. Since the low-frequency signal is more easily perceived by human ears than a highfrequency signal, the low-frequency signal is allocated a larger bitrate, and the quality of the encoded audio data may be better improved.

[0036] In a specific embodiment, for audio data with a sampling frequency of 192 kHz, the audio data may be decomposed into four subband data items with a sampling frequency of 48 kHz; based on the sampling theorem, the effective frequency band range of the signal is half of the sampling frequency, and thus the effective frequency band range of the original 192 kHz audio data is 0-96 kHz; the subband data items resulting from the decomposition represents the effective frequency band ranges of 0-24 kHz, 24-48 kHz, 48-72 kHz and 72-96 kHz of the original audio data, respectively; and the subband data items are sequentially numbered to obtain serial numbers 1, 2, 3 and 4.

[0037] Based on the principle that the higher the sampling frequency is, the lower the importance is, the subband data items 1 therein may be set as having a proportion 0.6, the subband data items 2 therein may be set as having a proportion of 0.2, and the subband data items 3 and 4 therein are respectively set as having a proportion 0.1, so that the low-frequency signal is allocated a larger bitrate, and the quality of low-frequency signal is improved.

[0038] It may be seen that through the present embodiment, the first subband data item representing the minimum frequency in the frequency band range of the audio data can be allocated the maximum bitrate, so that based on the principle that the low-frequency signal is more easily perceived by human ears than the highfrequency signal, the quality of the low-frequency signal is improved, and the quality of the encoded audio data is enhanced.

[0039] In the above step 108, after the desired post-encoding bitrate of each of the plurality of subband data items is determined, each of the plurality of subband data items is encoded according to the desired post-encoding bitrate of each of the plurality of subband data items. Each of the plurality of subband data items may be encoded by a well-established encoder such as an Opus encoder. The Opus encoder has advantages such as a high post-encoding compression rate and a high quality of the encoded audio data.

[0040] In the above step S110, the encoded audio data is generated according to all the encoded subband data items. For example, the encoded audio data is obtained by synthesizing all the encoded subband data items based on a preset bitstream structure.

[0041] In one embodiment, the generating encoded audio data according to all the encoded subband data items comprises: determining encoded first subband data item and encoded second subband data item from all encoded subband data items; the first subband data item is a subband data items representing a minimum frequency in the frequency band range of the audio data among all subband data items, and the second subband data item is a subband data items among all subband data items other than the first subband data item; filling the encoded first subband data item and the encoded second subband data item at corresponding fields in the preset bitstream structure, to generate the encoded audio data.

[0042] Specifically, all the encoded subband data items comprise a first subband data item and a plurality of second subband data item, the first subband data item is a subband data item representing a minimum frequency in the frequency band range of the audio data among all subband data items, and the second subband data item is a subband data items among all subband data items other than the first subband data item. First, the encoded first subband data item and each encoded second subband data item is determined from all the encoded subband data items.

[0043] Then, the encoded first subband data item and the encoded second subband data item are filled at corresponding fields in the preset bitstream structure, to generate the encoded audio data. The preset bitstream structure may be a bitstream structure corresponding to the encoder. For example, the respective subband data items are encoded by the Opus encoder, the preset bitstream structure is a bitstream structure possessed by the Opus encoder.

[0044] It can be seen that through the present embodiment, the first subband data item representing the minimum frequency in the frequency band range of the audio data and other subband data items can be filled at the corresponding fields in the preset bitstream structure, to thereby generate the encoded audio data. The generating the encoded audio data in a data-filling manner is advantageous in a high generation efficiency.

[0045] In one embodiment, the filling the encoded first subband data item and the encoded second subband data item at corresponding fields in the preset bitstream structure, to generate the encoded audio data comprises: filling the encoded first subband data item at a field in the bitstream structure for filling the encoded audio data; filling the encoded second subband data item at a field in the bitstream structure for filling additional data; generating the encoded audio data based on the filled bitstream structure.

[0046] Specifically, there are a field for filling the encoded audio data and a field for filling additional data in the preset bitstream structure. Regarding the audio data, it is data presented frame by frame in a chronological order. Therefore, each frame of audio data has a corresponding frame of bitstream data, and one frame of bitstream data is an encoded frame of audio data. When a length of each frame of bitstream data needs to be adjusted as the same length, additional data such as some meaningless character string such as 000 are filled at the field for filling the additional data, so that the length of each frame of bitstream data is adjusted as the same length.

[0047] In the present embodiment, the encoded first subband data item is filled at the field in the bitstream structure for filling the encoded audio data to enable the encoded first subband data item to be parsed and processed by the audio data receiver end as the encoded audio data; the encoded second subband data item is filled at the field in the bitstream structure for filling the additional data, so that when the audio data receiver end, upon having a capability of parsing and processing the additional data, extracts, from the field for filling the additional data, each encoded second subband data item to thereby decode the encoded audio data. Certainly, when the audio data receiver end does not have the capability of parsing and processing the additional data, the receiver end may directly discard the encoded second subband data item filled at the field for filling the additional data, and achieves the decoding work only based on the encoded first subband data item. Since the first subband data item is the most important signal representing the minimum frequency in the frequency band range of the audio data, achieving the decoding work based on the encoded first subband data item only may allow the receiver end to play the high-quality audio data.

[0048] In the present embodiment, the encoded audio data is also generated based on the filled bitstream structure, for example, the filled bitstream structure is taken as the encoded audio data.

[0049] It can be seen that through the present embodiment, the encoded first subband data item is filled at the field in the bitstream structure for filling the encoded audio data to enable the encoded first subband data item to be parsed and processed by the audio data receiver end as the encoded audio data; the encoded second subband data item is filled at the field in the bitstream structure for filling the additional data, so that when the audio data receiver end, upon having a capability of parsing and processing the additional data, extracts, from the field for filling the additional data, each encoded second subband data item to thereby decode the encoded audio data. In addition, when the audio data receiver end does not have the capability of parsing and processing the additional data, the receiver end may directly discard the encoded second subband data item filled at the field for filling the additional data, and achieves the decoding work only based on the encoded first subband data item, so that the encoded audio data may match the decoding capabilities of different receiver ends. Furthermore, since the first subband data item is the most important signal representing the minimum frequency in the frequency band range of the audio data, achieving the decoding work based on the encoded first subband data item only may allow the receiver end to play the high-quality audio data.

[0050] In one embodiment, the preset bitstream structure comprises a first field and a second field, a field value of the first field being used for indicating a length of each encoded second subband data item, and a field value of the second field being used for indicating a total length of all encoded second subband data item and the first field; accordingly, the generating the encoded audio data based on the filled bitstream structure comprises: in the filled bitstream structure, determining the field value of the first field according to the length of each encoded second subband data item; in the filled bitstream structure, determining the field value of the second field according to the total length of all encoded second subband data item and the first field; generating the encoded audio data based on the determined bitstream structure.

[0051] Specifically, the preset bitstream structure further comprises the first field and the second field, the field value of the first field being used for representing the length of each encoded second subband data item, and the field value of the second field being used for representing the total length of all encoded second subband data item and the first field. Based on this, in the present embodiment, after the encoded first subband data item and the encoded second subband data item are filled into the preset bitstream structure, the field value of the first field is also determined according to the length of each encoded second subband data item in the filled bitstream structure, the number of the first fields may be plural, each encoded second subband data item corresponds to one first field whose field value is equal to the length of the corresponding encoded second subband data item.

[0052] In addition, in the filled bitstream structure, the field value of the second field is determined according to the total length of all encoded second subband data items and the first field, the number of the second field is one, and the field value of the second field is equal to the total length of all encoded second subband data items and the first field.

[0053] Finally, the encoded audio data is generated based on the bitstream structure after the field value is determined. For example, the bitstream structure after the field value is determined is determined as the encoded audio data. Of course, then encoded audio data may further comprise some common fields such as a frame header field etc. The frame header field is used for storing parameters such as a length of the bitstream data, an encoding bandwidth, the number of channels, etc.

[0054] It may be seen that through the present embodiment, the encoded audio data can be enabled to carry the first field and the second field, the length of each encoded second subband data item is indicated by the field value of the first field, and the total length of all encoded second subband data items and the first field is indicated by the field value of the second field, thereby facilitating the decoding of the encoded audio data and improving the decoding efficiency.

[0055] In a specific embodiment, when each of the plurality of subband data items is encoded by an Opus encoder, the above-mentioned preset bitstream structure is a bitstream structure possessed by the Opus encoder. FIG. 2a is a schematic diagram of a preset bitstream structure according to an embodiment of the present disclosure. As shown in FIG. 2a, the bitstream structure of the Opus encoder comprises a frame header field, an additional data total length field, an encoded data field and an additional data field. The encoded data field is also referred to as a payload field, and the additional data field is also referred to as a filling field.

[0056] The length of the frame header field is a fixed value, typically 1-2 bytes. The frame header field is used to store parameters such as a length of the bitstream data, an encoding bandwidth, the number of channels, etc. The frame header field also includes a flag bit indicating whether the bitstream data contains filling data. When the flag bit is 1, it indicates that the bitstream data contains the filling data; if the flag bit is 0, this indicates that the bitstream data does not contain the filling data.

[0057] The additional data total length field is the aforementioned second field. The length of the additional data total length field is a fixed value, usually 1-2 bytes. The field value of the additional data total length field is used to indicate the total length of the additional data field.

[0058] The encoded data field is the above-mentioned field for filling the encoded audio data. The length of the encoded data field is not fixed, depending on the data length of the encoded first subband data item.

[0059] The additional data field is the above-mentioned field for filling the additional data. The length of the additional data field is not fixed, depending on the data length of the encoded second subband data item. The above-mentioned first field is located in the additional data field.

[0060] FIG.2b is a schematic diagram of encoded audio data according to an embodiment of the present disclosure. As shown in FIG.2b, encoded first subband data item is sub-bitstream-1 data in FIG.2b, and encoded second subband data item are sub-bitstream-2 data, sub-bitstream-3 data and sub-bitstream-4 data in FIG.2b. Continuing with the previous example, a frequency in the frequency band range of the above-mentioned audio data represented by the sub-bitstream -1 data is the lowest, and a bitrate accounts for 60% of the desired bitrate of the encoded audio data; a frequency in the frequency band range of the above-mentioned audio data represented by the sub-bitstream-2 data is higher than that of the sub-bitstream-1 data, and a bitrate accounts for 20% of the desired bitrate of the encoded audio data; a frequency in the frequency band range of the above-mentioned audio data represented by the sub-bitstream-3 data is higher than that of the sub-bitstream-2 data, and a bitrate accounts for 10% of the desired bitrate of the encoded audio data; a frequency in the frequency band range of the above-mentioned audio data represented by the sub-bitstream-4 data is higher than that of the sub-bitstream-3 data, and a bitrate accounts for 10% of the desired bitrate of the encoded audio data.

[0061] As shown in FIG. 2b, after the encoded first subband data item and each encoded second subband data item is obtained, the encoded first subband data item a filled in the encoded data field, and each encoded second subband data item is filled in the additional data field. Furthermore, the additional data field further comprises the first field corresponding to each encoded second subband data item, i.e., the field represented by "sub-bitstream-2 length", "sub-bitstream-3 length", and "sub-bitstream-4 length" in the figure. The first field is used to represent the length of the corresponding encoded second subband data item.

[0062] In FIG. 2b, the additional data total length field is the aforementioned second field, and the field value thereof is used to represent the length of the additional data. As can be seen from the figure, the length of the additional data is the total length of all encoded second subband data items and all first fields.

[0063] As can be seen from FIG. 2a and FIG. 2b, after each of the plurality of subband data items is encoded by the Opus encoder, the generated bitstream structure of the encoded audio data is the same as the bitstream structure carried by the Opus encoder itself, so that the encoded audio data is compatible with the Opus encoder. If the receiver end also uses the Opus encoder to decode the encoded audio data, and if the receiver end has the ability to recognize and parse the additional data field, the decoded data corresponding to each of the plurality of subband data items may be parsed and obtained; if the receiver end does not have the ability to recognize and parse the additional data field, the additional data field may be discarded, only the decoded data corresponding to the first subband data item may be parsed and obtained, and playing is performed based on the decoded data corresponding to the first subband data item.

[0064] When each of the plurality of subband data items is encoded by the Opus encoder, a limitation of the desired post-encoding bitrate native to the Opus encoder may be changed, and a desired post-encoding bitrate suitable for high-resolution audio data may be configured. Specifically, the Opus encoder natively supports a desired post-encoding bitrate of up to 510 kbps, and the value may be modified to 2400 kbps so that the Opus encoder may encode the audio data with high resolution, such as 192 kHz sampling frequency. The developer may also configure the desired bitrate of the encoded audio data in the Opus encoder, and the desired bitrate may be configured successfully as long as it is below 2400 kbps.

[0065] FIG. 3 illustrates a schematic diagram showing principles of audio data encoding according to an embodiment of the present disclosure. As shown in FIG.3, in the same way as the foregoing introduction, upon encoding, subband decomposition is first performed on the audio data to obtain a plurality of subband data items, and sequence numbers of the subband data items may be determined according to a level of the frequency in the above-mentioned frequency band range of the audio data represented by the subband data items. The lower the frequency in the represented frequency range, the smaller the sequence number. Then, each of the plurality of subband data items is encoded to obtain the encoded subband data items. The encoded subband data items may be referred to as a sub-bitstream. Then bitstream multiplexing is performed on each sub-bitstream in the manner shown in FIG. 2b. The bitstream multiplexing is filling each sub-bitstream into the preset bitstream structure to obtain the encoded audio data, and a bitstream is finally output. The output bitstream is the encoded audio data.

[0066] The process of generating the encoded audio data is introduced above, corresponding to a bitstream structure of the encoded audio data. In one embodiment, the encoded audio data is decoded in the following manner to obtain decoded data: extracting the encoded first subband data item and each encoded second subband data item from the encoded audio data according to a field value of the first field, a field value of the second field, a length of the second field, a length of a frame header field of the encoded audio data and a total length of the encoded audio data; obtaining decoded data based on the encoded first subband data item and each encoded second subband data item.

[0067] Firstly, the encoded first subband data item and each encoded second subband data item is extracted from the encoded audio data according to the field value of the first field, the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data and the total length of the encoded audio data. Then, decoded data is obtained by synthesizing the subbands based on the encoded first subband data item and each encoded second subband data item. Considering that the encoding process is lossy encoding, the data obtained by decoding the encoded audio data is referred to as decoded data.

[0068] It can be seen that through the present embodiment, the encoded first subband data item and each encoded second subband data item can be extracted from the encoded audio data, and then the decoded data corresponding to the encoded audio data can be obtained based on the encoded first subband data item and each encoded second subband data item, with a simple decoding process and a high decoding efficiency.

[0069] In one embodiment, the extracting the encoded first subband data item and each encoded second subband data item from the encoded audio data according to a field value of the first field, a field value of the second field, a length of the second field, a length of a frame header field of the encoded audio data and a total length of the encoded audio data comprises: extracting each encoded second subband data item from the encoded audio data according to the field value of the first field and the field value of the second field; extracting the encoded first subband data item from the encoded audio data according to the field value of the second field, the length of the second field, the length of a frame header field of the encoded audio data and the total length of the encoded audio data.

[0070] First, since the field value of the first field is used to represent the length of each encoded second subband data item and the field value of the second field is used to represent the sum of lengths of all encoded second subband data items and all first fields, each encoded second subband data item can be extracted from the encoded audio data according to the field value of the first field and the field value of the second field.

[0071] Then, the encoded first subband data item is extracted from the encoded audio data according to the field value of the second field, the length of the second field, the length of a frame header field of the encoded audio data and the total length of the encoded audio data.

[0072] It can be seen that through the present embodiment, the encoded first subband data item and each encoded second subband data item can be extracted efficiently and quickly from the encoded audio data.

[0073] In one embodiment, the extracting each encoded second subband data item from the encoded audio data according to the field value of the first field and the field value of the second field comprises: extracting additional data from the encoded audio data according to the field value of the second field; the additional data comprises each encoded second subband data item and each first field; extracting each encoded second subband data item from the additional data according to the field value of the first field.

[0074] Specifically, as known from the above depictions, the field value of each first field is used to represent the length of each encoded second subband data item, and the field value of the second field is used to represent the total length of all encoded second subband data items and all first fields. Based on this, with reference to FIG. 2b, all encoded second subband data items and all first fields may be obtained by looking up, i.e., the additional data field may be obtained by looking up starting from an end of the encoded audio data, i.e., the bitstream data, according to the field value of the second field.

[0075] Then, each encoded second subband data item is sequentially extracted starting from a starting portion of the additional data according to the field value of each first field. In FIG. 2b, a left side of the bitstream data is set as a data start position, a right side of the bitstream data is set as a data end position, and the bitstream data is arranged from left to right.

[0076] It can be seen that through the present embodiment, each encoded second subband data item can be efficiently and quickly extracted from the encoded audio data based on the field value of the first field and the field value of the second field.

[0077] In one embodiment, the extracting the encoded first subband data item from the encoded audio data according to the field value of the second field, the length of the second field, the length of a frame header field of the encoded audio data and the total length of the encoded audio data comprises: determining the length of the encoded first subband data item according to the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data and the total length of the encoded audio data; extracting the encoded first subband data item from the encoded audio data according to the length of the encoded first subband data item.

[0078] Specifically, as shown in FIG. 2b, the total length of the encoded audio data may be extracted from the frame header field, and the length of the encoded first subband data item, namely, the length of the encoded data field, can be calculated with the following equation: the length of the encoded data field=the total length-the length of the frame header field-the length of the additional data total length field-the length of the additional data field; wherein the length of the additional data field is the field value of the second field, the length of the additional data total length field is the length of the second field, and generally, the length of the additional data total length field is a fixed value and may also be recorded in the frame header field.

[0079] Therefore, the length of the encoded first subband data item is equal to the total length of the encoded audio data minus the length of the frame header field minus the field value of the second field minus the length of the second field.

[0080] Then, the encoded first subband data item may be obtained by looking up starting from a starting position of the encoded audio data by skipping the frame header field with a fixed length and the additional data total length field with a fixed length, according to the length of the encoded first subband data item.

[0081] It can be seen that through the present embodiment, the length of the encoded first subband data item can be determined according to the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data and the total length of the encoded audio data, and then the encoded first subband data item can be extracted accurately from the encoded audio data according to the length of the encoded first subband data item.

[0082] After extracting the encoded first subband data item and each encoded second subband data item, in one embodiment, the obtaining decoded data based on the encoded first subband data item and each encoded second subband data item comprises: decoding the encoded first subband data item and each encoded second subband data item to obtain first sub-decoded data item and each second sub-decoded data item, respectively; performing subband synthesis on the first sub-decoded data and each second sub-decoded data item to obtain decoded data.

[0083] Specifically, the encoded first subband data item is decoded to obtain first sub-decoded data; since the first subband data item is encoded in a lossy encoding manner, the data obtained by decoding the encoded first subband data item is referred to as first sub-decoded data; next, each encoded second subband data item is decoded to obtain the second sub-decoded data corresponding to each encoded second subband data item; since the second subband data item is encoded in a lossy encoding manner, the data obtained by decoding the encoded second subband data item is referred to as second sub-decoded data. One encoded second subband data item corresponds to one second sub-decoded data.

[0084] Then, subband synthesis is performed on the first sub-decoded data and each second sub-decoded data item to obtain the decoded data. The subband synthesis may be performed by a PQMF method.

[0085] In a specific embodiment, after receiving the bitstream data as shown in FIG. 2b, the receiver end firstly parses the frame header field, and acquires, from the frame header field, a flag bit indicating whether the current bitstream data contains the additional data field. If the current bitstream data contains the additional data field, the length of the additional data field is firstly parsed out, then the additional data is taken out from the tail of the bitstream data according to the additional data field, and the length of each encoded second subband data item is parsed in sequence from the additional data and each second subband data item is extracted. Then, the length of the encoded first subband data item is calculated, and the encoded first subband data item is extracted from the bitstream data according to the length.

[0086] Then, Opus decoding is performed on the encoded first subband data item and each encoded second subband data item to obtain the first sub-decoded data item and each second sub-decoded data item, respectively, and finally subband synthesis is performed on the first sub-decoded data item and each second sub-decoded data item by the PQMF method to obtain the decoded data.

[0087] It can be seen that through the present embodiment, decoding and synthesis can be performed on the encoded subband data items, to efficiently obtain the decoded data.

[0088] FIG. 4 illustrates a schematic diagram showing principles of audio data decoding according to an embodiment of the present disclosure. As shown in FIG.4, in the same way as the foregoing introduction, after acquiring an input bitstream, i.e., the encoded audio data, the receiver end performs bitstream parsing to obtain sub-bitstreams, i.e., each encoded subband data item, decodes each encoded subband data item to obtain each sub-decoded data item, the sub-decoded data comprising the aforementioned first sub-decoded data item and second sub-decoded data item, then performs subband synthesis on each sub-decoded data item to obtain output data which is decoded data.

[0089] In a specific embodiment, the encoding process and decoding process described above may be implemented based on an Opus encoder. A specific implementation process is as follows: 1. The developer modifies the desired post-encoding maximum bitrate 510 kbps natively supported by the Opus encoder to 2400 kbps, so that the Opus encoder may encode audio data with a high resolution, such as a sampling frequency of 192 kHz; 2. The developer configures a desired bitrate of the encoded audio data inside the Opus encoder, and the desired bitrate may be configured successfully as long as it is lower than 2400 kbps; 3. The high-resolution audio data is acquired, and the audio data is input into the Opus encoder; 4. The Opus encoder performs subband decomposition on audio data, and determines a desired post-encoding bitrate of each of the plurality of subband data items according to a pre-configured desired bitrate; 5. The Opus encoder encodes each of the plurality of subband data items according to the desired post-encoding bitrate of each of the plurality of subband data items; 6. The Opus encoder generates encoded audio data according to each encoded subband data items.

[0090] Certainly, the Opus encoder in the present embodiment may be replaced with an encoder such as an EVS encoder or an AAC encoder. It can be seen that the following technical effects are achieved through the present embodiment: 1. The sampling frequency which may be supported by the Opus encoder is expanded, and for high-resolution audio data, encoding and decoding may also be performed in the Opus encoder by means of subband decomposition; 2. The high-resolution audio data is encoded by subband decomposition so that a higher compression rate is achieved, and the encoded audio data may be applied to a real-time communication scenario such as a video conference scenario. Experiments prove that, for audio data with a sampling frequency of 192 kHz and a bit depth of 24 bits, the whole frequency band may be encoded at a bitrate of 500 kbps, the quality of the audio data after the encoding and decoding is substantially free from damages, and the compression rate reaches more than 10 times.

[0091] FIG. 5 illustrates a schematic diagram of an application scenario of audio data encoding and decoding according to an embodiment of the present disclosure. As shown in FIG.5, in a real-time communication scenario, such as a video conference scenario, a plurality of terminal devices (represented by terminal device 501 and terminal device 502 in the figure) may realize the transmission of real-time audio data through a server 503. Through the encoding and decoding method in the present embodiment, the bitrate of audio data may be substantially reduced, so as to achieve real-time transmission.

[0092] Furthermore, the server may also acquire an encoding and decoding capability supported by each terminal device; when any one of a plurality of terminal devices in real-time communication in the same scenario, such as the same room or the same conference, does not support the capability of encoding and decoding the high-resolution audio data in the above embodiments, the server notifies each terminal device to collect and generate low-resolution audio data (the sampling frequency is lower than the preset frequency or the sampling depth is lower than the preset depth); when each terminal device for real-time communication in the same scenario supports the capability of encoding and decoding the high-resolution audio data in the above embodiments, the server notifies each terminal device to collect and generate high-resolution audio data (the sampling frequency is greater than or equal to the preset frequency and the sampling depth is greater than or equal to the preset depth), and executes the above solution, thereby improving the real-time audio communication quality.

[0093] Corresponding to the above embodiments, an embodiment of the present disclosure provides an audio data processing apparatus. FIG. 6 illustrates a block diagram of an audio data processing apparatus according to an embodiment of the present disclosure. As shown in FIG. 6, the apparatus comprises: an acquisition unit 61 configured to acquire audio data to be processed whose sampling frequency is greater than or equal to a preset frequency and whose sampling depth is greater than or equal to a preset depth; a decomposition unit 62 configured to perform subband decomposition on the audio data to obtain a plurality of subband data items; a determination unit 63 configured to determine a desired post-encoding bitrate of each of the plurality of subband data items; a sum of desired post-encoding bitrates of all the subband data items is equal to a desired post-encoding bitrate of the audio data; a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than a preset value; an encoding unit 64 configured to encode each of the plurality of subband data items according to the desired post-encoding bitrate of each of the plurality of subband data items; a generation unit 65 configured to generate encoded audio data according to each of the encoded subband data items.

[0094] Optionally, the determination unit 63 is specifically configured to: acquire a proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data; determine the desired post-encoding bitrate of each of the plurality of subband data items according to the proportion of the desired post-encoding bitrate of the audio data to each of the plurality of subband data items.

[0095] Optionally, the determination unit 63 is specifically configured to: determine a frequency band range of the audio data represented by each of the plurality of subband data items; determine the proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data, according to the frequency band range of the audio data represented by each of the plurality of subband data items.

[0096] Optionally, the determination unit 63 is specifically configured to: for a first subband data item representing a minimum frequency in the frequency band range of the audio data among all the subband data items, determine the proportion of the first subband data items is greater than the proportion of a second subband data item among all the subband data items; the second subband data item is a subband data item among all the subband data items other than the first subband data item.

[0097] Optionally, the generation unit 65 is specifically configured to: determine encoded first subband data item and encoded second subband data item from all encoded subband data items; the first subband data item is a subband data items representing a minimum frequency in the frequency band range of the audio data among all the subband data items, and the second subband data item is a subband data items among all the subband data items other than the first subband data item; fill the encoded first subband data item and the encoded second subband data item at corresponding fields in a preset bitstream structure, to generate the encoded audio data.

[0098] Optionally, the generation unit 65 is further specifically configured to: fill the encoded first subband data item at a field in the bitstream structure for filling the encoded audio data; fill the encoded second subband data item at a field in the bitstream structure for filling additional data; generate the encoded audio data based on the filled bitstream structure.

[0099] Optionally, the bitstream structure comprises a first field and a second field, a field value of the first field being used for representing a length of each of the encoded second subband data item, and a field value of the second field being used for representing a total length of all the encoded second subband data items and the first field; the generation unit 65 is further specifically configured to: in the filled bitstream structure, determine the field value of the first field according to the length of each of the encoded second subband data item; in the filled bitstream structure, determine the field value of the second field according to the total length of all the encoded second subband data items and the first field; generate the encoded audio data based on the determined bitstream structure.

[0100] Optionally, the apparatus further comprises a decoding unit configured to: extract the encoded first subband data item and each of the encoded second subband data item from the encoded audio data according to a field value of the first field, a field value of the second field, a length of the second field, a length of a frame header field of the encoded audio data and a total length of the encoded audio data; obtain the decoded data based on the encoded first subband data item and each of the encoded second subband data item.

[0101] Optionally, the decoding unit is specifically configured to: extract each of the encoded second subband data item from the encoded audio data according to the field value of the first field and the field value of the second field; extract the encoded first subband data item from the encoded audio data according to the field value of the second field, the length of the second field, the length of a frame header field of the encoded audio data and the total length of the encoded audio data.

[0102] Optionally, the decoding unit is further specifically configured to: extract the additional data from the encoded audio data according to the field value of the second field; the additional data comprises each of the encoded second subband data items and the first field; extract each of the encoded second subband data items from the additional data according to the field value of the first field.

[0103] Optionally, the decoding unit is further specifically configured to: determine the length of the encoded first subband data item according to the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data and the total length of the encoded audio data; extract the encoded first subband data item from the encoded audio data according to the length of the encoded first subband data item.

[0104] Optionally, the decoding unit is specifically configured to: decode the encoded first subband data item and each of the encoded second subband data items to obtain first sub-decoded data item and each second sub-decoded data item, respectively; perform subband synthesis on the first sub-decoded data item and each second sub-decoded data item to obtain the decoded data.

[0105] The audio data processing apparatus in the embodiment of the present disclosure may implement the processes of the above audio data processing method embodiment, and achieve the same effects and functions, which will not be repeated any longer here.

[0106] An embodiment of the present disclosure further provides an electronic device. FIG.7 illustrates a structural diagram of the electronic device according to an embodiment of the present disclosure. As shown in FIG.7, the electronic device may vary greatly with different configuration or performance, and may include one or more processors 701 and one or more memories 702. The memory 702 may store one or more applications or data. The memory 702 may be transient storage or persistent storage. The application stored in the memory 702 may include one or more modules (not shown), each of which may include a series of computer-executable instructions in the electronic device. Furthermore, the processor 701 may be set in communication with the memory 702, and the series of computer-executable instructions in the memory 702 are executed on the electronic device. The electronic device may also include one or more power supplies 703, one or more wired or wireless network interfaces 704, one or more input or output interfaces 705, one or more keyboards 706, etc.

[0107] In a specific embodiment, the electronic device comprises a processor; and, a memory configured to store computer-executable instructions that, when executed, cause the processor to implement the following steps: acquiring audio data to be processed whose sampling frequency is greater than or equal to a preset frequency and whose sampling depth is greater than or equal to a preset depth; performing subband decomposition on the audio data to obtain a plurality of subband data items; determining a desired post-encoding bitrate of each of the plurality of subband data items; a sum of desired post-encoding bitrates of all the subband data items is equal to a desired post-encoding bitrate of the audio data; a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than a preset value; encoding each of the plurality of subband data items according to the desired post-encoding bitrate of each of the plurality of subband data items; generating encoded audio data according to each of the encoded subband data items.

[0108] The electronic device in the embodiment of the present disclosure may implement the steps of the above audio data processing method embodiment and achieve the same effects and functions, which will not be repeated any more here.

[0109] Another embodiment of the present disclosure further provides a computer-readable storage medium for storing computer-executable instructions that, when executed by a processor, implement the following steps: acquiring audio data to be processed whose sampling frequency is greater than or equal to a preset frequency and whose sampling depth is greater than or equal to a preset depth; performing subband decomposition on the audio data to obtain a plurality of subband data items; determining a desired post-encoding bitrate of each of the plurality of subband data items; a sum of desired post-encoding bitrates of all the subband data items is equal to a desired post-encoding bitrate of the audio data; a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data is greater than a preset value; encoding each of the plurality of subband data items according to the desired post-encoding bitrate of each of the plurality of subband data items; generating encoded audio data according to each of the encoded subband data items.

[0110] The storage medium in the present embodiment of the present disclosure may implement the steps of the above audio data processing method embodiment and achieve the same effects and functions, which will not be repeated any more here.

[0111] In various embodiments of the present disclosure, the computer-readable storage medium includes read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, an optical disk, or the like.

[0112] In the 1990s, whether a technical improvement is a hardware improvement (for example, an improvement to a circuit structure, such as a diode, a transistor, or a switch) or a software improvement (an improvement to a method procedure) can be clearly distinguished. However, as technologies develop, the current improvement for many method procedures can be considered as a direct improvement of a hardware circuit structure. A designer usually programs an improved method procedure into a hardware circuit, to obtain a corresponding hardware circuit structure. Therefore, a method procedure can be improved by using a hardware entity module. For example, a programmable logic device (PLD) (for example, a field programmable gate array (FPGA)) is such an integrated circuit, and a logical function of the programmable logic device is determined by a user through device programming. The designer performs programming to "integrate" a digital system to a PLD without requesting a chip manufacturer to design and produce an application-specific integrated circuit chip. In addition, at present, instead of manually manufacturing an integrated chip, this type of programming is mostly implemented by using "logic compiler" software. The programming is similar to a software compiler used to develop and write a program. Original code needs to be written in a particular programming language for compilation. The language is referred to as a hardware description language (HDL). There are many HDLs, such as the Advanced Boolean Expression Language (ABEL), the Altera Hardware Description Language (AHDL), Confluence, the Cornell University Programming Language (CUPL), HDCal, the Java Hardware Description Language (JHDL), Lava, Lola, MyHDL, PALASM, and the Ruby Hardware Description Language (RHDL). The very-high-speed integrated circuit hardware description language (VHDL) and Verilog are most commonly used. A person skilled in the art should also understand that a hardware circuit that implements a logical method procedure can be readily obtained once the method procedure is logically programmed by using the previously-mentioned several hardware description languages and is programmed into an integrated circuit.

[0113] A controller can be implemented by using any appropriate method. For example, the controller can be a microprocessor or a processor, or a computer-readable medium that stores computer readable program code (such as software or firmware) that can be executed by the microprocessor or the processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, or a built-in microprocessor. Examples of the controller include but are not limited to the following microprocessors: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as a part of the control logic of the memory. A person skilled in the art also knows that, in addition to implementing the controller by using the computer readable program code, logic programming can be performed on method steps to allow the controller to implement the same function in forms of the logic gate, the switch, the application-specific integrated circuit, the programmable logic controller, and the built-in microcontroller. Therefore, the controller can be considered as a hardware component, and an apparatus configured to implement various functions in the controller can also be considered as a structure in the hardware component. Or the apparatus configured to implement various functions can even be considered as both a software module implementing the method and a structure in the hardware component.

[0114] The system, apparatus, module, or unit illustrated in the previously-mentioned embodiments can be implemented by using a computer chip or an entity, or can be implemented by using a product having a certain function. A typical implementation device is a computer. A specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, an intelligent phone, a personal digital assistant, a media player, a navigation device, an email transceiver device, a game console, a tablet computer, a wearable device, or any combination thereof.

[0115] For convenience of description, the above apparatuses are described respectively in terms of their functions. Certainly, when the present disclosure is implemented, a function of each unit can be implemented in one or more pieces of software and / or hardware.

[0116] A person skilled in the art should understand that embodiments of the present disclosure can be provided as methods, systems, or computer program products. Therefore, the present disclosure can use a form of hardware only embodiments, software only embodiments, or embodiments with a combination of software and hardware. Further, the present disclosure can take a form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, and optical storage) containing computer-usable program code.

[0117] The present disclosure is described with reference to the flowcharts and / or block diagrams of the method, the device (system), and the computer program product based on the embodiments of the present disclosure. It is worthwhile to note that computer program instructions can be used to implement each process and / or each block in the flowcharts and / or the block diagrams and a combination of a process and / or a block in the flowcharts and / or the block diagrams. These computer program instructions can be provided for a general-purpose computer, a dedicated computer, an embedded processor, or a processor of another programmable data processing device to generate a machine, so the instructions executed by the computer or the processor of another programmable data processing device generate an apparatus for implementing a specific function in one or more processes in the flowcharts and / or in one or more blocks in the block diagrams.

[0118] These computer program instructions can be stored in a computer readable memory that can instruct the computer or another programmable data processing device to work in a specific way, so the instructions stored in the computer readable memory generate an artifact that includes an instruction apparatus. The instruction apparatus implements a specific function in one or more processes in the flowcharts and / or in one or more blocks in the block diagrams.

[0119] These computer program instructions can be loaded onto the computer or another programmable data processing device, so a series of operation steps are performed on the computer or another programmable device, thereby generating computer-implemented processing. Therefore, the instructions executed on the computer or another programmable device provide steps for implementing a specific function in one or more processes in the flowcharts and / or in one or more blocks in the block diagrams.

[0120] It is also worthwhile to note that terms "include", "contain" or any other variant is intended to cover non-exclusive inclusion, so processes, methods, products or devices that include a series of elements include not only those elements but also other elements that are not explicitly listed, or elements inherent in such processes, methods, products or devices. An element described by "includes a ..." further includes, without more constraints, another identical element in the process, method, product, or device that includes the element.

[0121] One or more embodiments of the present disclosure can be described in the general context of computer executable instructions executed by a computer, for example, a program module. Generally, the program module includes a routine, a program, an object, a component, a data structure, etc. executing a specific task or implementing a specific abstract data type. The one or more embodiments of the present disclosure can also be practiced in distributed computing environments. In the distributed computing environments, tasks are performed by remote processing devices connected through a communications network. In a distributed computing environment, the program module can be located in both local and remote computer storage media including storage devices.

[0122] The embodiments in the present disclosure are all described progressively, for same or similar parts in the embodiments, reference can be made to these embodiments, and each embodiment focuses on a difference from other embodiments. Particularly, a system embodiment is basically similar to a method embodiment, and therefore, is described briefly. For related parts, refer to partial descriptions in the method embodiment.

[0123] What are described above are embodiments of the present disclosure, and are not intended to limit the present disclosure. A person skilled in the art can make various modifications and changes to the present disclosure. Any modification, equivalent replacement or improvement made within spirit and principles of the present disclosure shall be included in claims of the present disclosure.

Examples

Embodiment Construction

[0010]Technical solutions in one or more embodiments of the present disclosure will be described clearly and completely with reference to figures in the one or more embodiments of the present disclosure to enable those skilled in the art to better understand the technical solutions in the one or more embodiments. Obviously, the embodiments described are partial embodiments other than all embodiments of the present disclosure. All other embodiments obtained by those having ordinary skill in the art based on one or more embodiments in the present disclosure without making any inventive efforts all fall within the protection scope of the present disclosure.

[0011]Embodiments of the present disclosure provide an audio data processing method, which, for audio data with a larger sampling frequency and a larger sampling depth, can reduce a bitrate of the encoded audio data, and improve an efficiency of encoding the audio data. For ease of understanding embodiments of the present disclosure,...

Claims

1. An audio data processing method, comprising: acquiring audio data to be processed whose sampling frequency is greater than or equal to a preset frequency and whose sampling depth is greater than or equal to a preset depth; performing subband decomposition on the audio data to obtain a plurality of subband data items; determining a desired post-encoding bitrate of each of the plurality of subband data items, a sum of desired post-encoding bitrates of the plurality of subband data items being equal to a desired post-encoding bitrate of the audio data, a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data being greater than a preset value; encoding each of the plurality of subband data items based on the desired post-encoding bitrate of each of the plurality of subband data items; and generating encoded audio data based on each of encoded subband data items.

2. The method according to claim 1, wherein determining the desired post-encoding bitrate of each of the plurality of subband data items comprises: acquiring a proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data; and determining the desired post-encoding bitrate of each of the plurality of subband data items based on the proportion of the desired post-encoding bitrate of the audio data to each of the plurality of subband data items.

3. The method according to claim 2, wherein acquiring the proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data comprises: determining a frequency band range of the audio data represented by each of the plurality of subband data items; and determining the proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data, based on the frequency band range of the audio data represented by each of the plurality of subband data items.

4. The method according to claim 3, wherein determining the proportion of the desired post-encoding bitrate of each of the plurality of subband data items to the desired post-encoding bitrate of the audio data, based on the frequency band range of the audio data represented by each of the plurality of subband data items comprises: for a first subband data item, among the plurality of subband data items, representing a minimum frequency in the frequency band range of the audio data, determining that a proportion of the first subband data item to the desired post-encoding bitrate of the audio data is greater than a proportion of a second subband data item, among the plurality of subband data items, to the desired post-encoding bitrate of the audio data, the second subband data item being a subband data item among the plurality of subband data items other than the first subband data item.

5. The method according to claim 1, wherein generating the encoded audio data based on each of the encoded subband data items comprises: determining an encoded first subband data item and encoded second subband data items among the encoded subband data items, a first subband data item being a subband data item, among the plurality of subband data items, representing a minimum frequency in the frequency band range of the audio data, and second subband data items being subband data items among the plurality of subband data items other than the first subband data item; and filling the encoded first subband data item and the encoded second subband data items into corresponding fields in a preset bitstream structure, to generate the encoded audio data.

6. The method according to claim 5, wherein filling the encoded first subband data item and the encoded second subband data items into the corresponding fields in the preset bitstream structure, to generate the encoded audio data comprises: filling the encoded first subband data item into a field in the bitstream structure for filling the encoded audio data; filling the encoded second subband data items into a field in the bitstream structure for filling additional data; and generating the encoded audio data based on the filled bitstream structure.

7. The method according to claim 6, wherein the bitstream structure comprises a first field and a second field, a field value of the first field is used for representing a length of each of the encoded second subband data items, and a field value of the second field is used for representing a total length of the encoded second subband data items and the first field; wherein generating the encoded audio data based on the filled bitstream structure comprises: in the filled bitstream structure, determining the field value of the first field based on the length of each of the encoded second subband data items; in the filled bitstream structure, determining the field value of the second field based on the total length of the encoded second subband data items and the first field; and generating the encoded audio data based on the determined bitstream structure.

8. The method according to claim 7, wherein the encoded audio data is decoded by the following operations to obtain decoded data: extracting the encoded first subband data item and each of the encoded second subband data items from the encoded audio data, based on the field value of the first field, the field value of the second field, a length of the second field, a length of a frame header field of the encoded audio data and a total length of the encoded audio data; and obtaining the decoded data based on the encoded first subband data item and each of the encoded second subband data items.

9. The method according to claim 8, wherein extracting the encoded first subband data item and each of the encoded second subband data items from the encoded audio data, based on the field value of the first field, the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data and the total length of the encoded audio data comprises: extracting each of the encoded second subband data items from the encoded audio data based on the field value of the first field and the field value of the second field; and extracting the encoded first subband data item from the encoded audio data based on the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data and the total length of the encoded audio data.

10. The method according to claim 9, wherein extracting each of the encoded second subband data items from the encoded audio data based on the field value of the first field and the field value of the second field comprises: extracting the additional data from the encoded audio data based on the field value of the second field, the additional data comprising each of the encoded second subband data items and the first field; and extracting each of the encoded second subband data items from the additional data based on the field value of the first field.

11. The method according to claim 9, wherein extracting the encoded first subband data item from the encoded audio data based on the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data and the total length of the encoded audio data comprises: determining a length of the encoded first subband data item based on the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data and the total length of the encoded audio data; and extracting the encoded first subband data item from the encoded audio data based on the length of the encoded first subband data item.

12. The method according to claim 8, wherein obtaining the decoded data based on the encoded first subband data item and each of the encoded second subband data items comprises: decoding the encoded first subband data item and each of the encoded second subband data items to obtain a first decoded sub-data item and each second decoded sub-data item, respectively; and performing subband synthesis on the first decoded sub-data item and each second decoded sub-data item to obtain the decoded data.

13. An audio data processing apparatus, comprising: an acquisition unit configured to acquire audio data to be processed whose sampling frequency is greater than or equal to a preset frequency and whose sampling depth is greater than or equal to a preset depth; a decomposition unit configured to perform subband decomposition on the audio data to obtain a plurality of subband data items; a determination unit configured to determine a desired post-encoding bitrate of each of the plurality of subband data items, a sum of desired post-encoding bitrates of the plurality of subband data items being equal to a desired post-encoding bitrate of the audio data, a ratio of a pre-encoding bitrate of the audio data to the desired post-encoding bitrate of the audio data being greater than a preset value; an encoding unit configured to encode each of the plurality of subband data items based on the desired post-encoding bitrate of each of the plurality of subband data items; and a generation unit configured to generate encoded audio data based on each of the encoded subband data items.

14. An electronic device, comprising: a processor; and a memory configured to store computer-executable instructions which, when executed, cause the processor to implement the steps of the method according to any of claims 1 to 12.

15. A computer-readable storage medium for storing computer-executable instructions that, when executed by a processor, implement the steps of the method according to any of claims 1 to 12.

Images