Audio data processing method and device, electronic equipment and storage medium

Abstract

Embodiments of the invention provide an audio data processing method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining to-be-processed audio data; the sampling frequency of the audio data is greater than or equal to a preset frequency and the sampling depth is greater than or equal to a preset depth; performing sub-band decomposition on the audio data to obtain a plurality of sub-band data; determining an expected code rate of each piece of sub-band data after coding; the sum of the encoded expected code rates of the sub-band data is equal to the encoded expected code rate of the audio data; the ratio of the code rate of the audio data before coding to the expected code rate of the audio data after coding is greater than a preset value; according to the expected code rate of each piece of sub-band data after coding, coding each piece of sub-band data; and generating coded audio data according to each piece of coded sub-band data. According to the embodiment of the invention, the code rate of the coded audio data can be reduced, and the coding efficiency of the audio data is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of audio data processing, and in particular to an audio data processing method, device, electronic device and storage medium. Background Art

[0002] Before transmitting the audio data, the audio data needs to be encoded, and the encoded audio data is transmitted from the sender of the audio data to the receiver of the audio data. After receiving the audio data, the receiver of the audio data decodes and plays the encoded audio data. In the prior art, when encoding audio data with a high sampling frequency and a deep sampling depth, the bit rate of the encoded audio data is still very large, resulting in low encoding efficiency of the audio data. Summary of the invention

[0003] The embodiments of the present disclosure provide an audio data processing method, device, electronic device and storage medium, which can reduce the bit rate of encoded audio data and improve the encoding efficiency of audio data for audio data with a high sampling frequency and a deep sampling depth.

[0004] In a first aspect, an embodiment of the present disclosure provides an audio data processing method, including:

[0005] Acquire audio data to be processed; the sampling frequency of the audio data is greater than or equal to a preset frequency and the sampling depth is greater than or equal to a preset depth;

[0006] Decomposing the audio data into sub-bands to obtain a plurality of sub-band data;

[0007] Determine an expected bit rate after encoding of each sub-band data; the sum of the expected bit rates after encoding of each sub-band data is equal to the expected bit rate after encoding of the audio data; the ratio of the bit rate before encoding of the audio data to the expected bit rate after encoding of the audio data is greater than a preset value;

[0008] Encoding each of the sub-band data according to an expected bit rate after encoding each of the sub-band data;

[0009] The encoded audio data is generated according to the encoded sub-band data.

[0010] In a second aspect, an embodiment of the present disclosure provides an audio data processing device, including:

[0011] An acquisition unit, configured to acquire audio data to be processed; the sampling frequency of the audio data is greater than or equal to a preset frequency and the sampling depth is greater than or equal to a preset depth;

[0012] A decomposition unit, configured to perform sub-band decomposition on the audio data to obtain a plurality of sub-band data;

[0013] a determining unit, configured to determine an expected bit rate after encoding of each of the sub-band data; the sum of the expected bit rates after encoding of the sub-band data is equal to the expected bit rate after encoding of the audio data; and a ratio of the bit rate before encoding of the audio data to the expected bit rate after encoding of the audio data is greater than a preset value;

[0014] an encoding unit, configured to encode each of the sub-band data according to an expected bit rate after encoding each of the sub-band data;

[0015] A generating unit is used to generate the encoded audio data according to the encoded sub-band data.

[0016] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory configured to store computer-executable instructions, wherein the computer-executable instructions, when executed, enable the processor to implement the steps of the method described in the first aspect above.

[0017] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and the computer-executable instructions implement the steps of the method described in the first aspect when executed by a processor.

[0018] In one or more embodiments of the present disclosure, audio data to be processed is obtained, the sampling frequency of the audio data is greater than or equal to the preset frequency and the sampling depth is greater than or equal to the preset depth, the audio data is sub-band decomposed to obtain multiple sub-band data, the expected bit rate of each sub-band data after encoding is determined, the sum of the expected bit rates of the sub-band data after encoding is equal to the expected bit rate of the audio data after encoding, the ratio of the bit rate of the audio data before encoding to the expected bit rate of the audio data after encoding is greater than the preset value, and each sub-band data is encoded according to the expected bit rate of each sub-band data after encoding, and the encoded audio data is generated according to each encoded sub-band data. Through this embodiment, for audio data with a sampling frequency greater than or equal to the preset frequency and a sampling depth greater than or equal to the preset depth, the encoded audio data can be obtained by sub-band decomposing and encoding the audio data. Since the ratio of the bit rate of the audio data before encoding to the expected bit rate of the audio data after encoding is greater than the preset value, the bit rate of the encoded audio data is greatly reduced, and the encoding efficiency of the audio data is improved. Since the bit rate of the encoded audio data is reduced, the bandwidth required for transmitting the encoded audio data and the data flow are saved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate one or more embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

[0020] Figure 1 A flowchart of an audio data processing method provided by an embodiment of the present disclosure;

[0021] Figure 2a A schematic diagram of a preset code stream structure provided in an embodiment of the present disclosure;

[0022] Figure 2b A schematic diagram of encoded audio data provided by an embodiment of the present disclosure;

[0023] Figure 3 A schematic diagram of the principle of audio data encoding provided by an embodiment of the present disclosure;

[0024] Figure 4 A schematic diagram of the principle of audio data decoding provided by an embodiment of the present disclosure;

[0025] Figure 5 A schematic diagram of an application scenario of audio data encoding and decoding provided by an embodiment of the present disclosure;

[0026] Figure 6 A schematic diagram of the structure of an audio data processing device provided by an embodiment of the present disclosure;

[0027] Figure 7 A schematic diagram of the structure of an electronic device provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of the present disclosure, the technical solutions in one or more embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in one or more embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on one or more embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present disclosure.

[0029] The embodiments of the present disclosure provide an audio data processing method, which can reduce the bit rate of the encoded audio data and improve the encoding efficiency of the audio data for audio data with a high sampling frequency and a deep sampling depth. To facilitate understanding of the various embodiments of the present disclosure, the technical terms involved are first explained here.

[0030] Sampling frequency: also called sampling rate, is the number of audio data collected per unit time, such as per second, that is, the number of audio data recorded per second. The unit of sampling frequency is kHz. For example, the sampling frequency of audio data is 192kHz, which means that the number of audio data collected per second is 192k. The higher the sampling frequency, the larger the amount of audio data.

[0031] Sampling depth: also called bit depth, is the number of bits required to represent a collected audio data. For example, a sampling depth of 24 bits means that each collected audio data is represented by 24 bits. The deeper the sampling depth, the larger the amount of audio data.

[0032] PQMF: Pseudo-Quadrature Mirror Filter is a mature filtering algorithm that can be used to decompose audio data into sub-bands.

[0033] Opus: A mature lossy encoder that supports a maximum input encoding bit rate of 510kbps.

[0034] Code stream data: The encoded audio data is essentially a long string of binary data, which can be called code stream data. After receiving the code stream data, the receiving end decodes and plays the code stream data. Since the audio data is encoded frame by frame during encoding, it can also be considered that each frame of encoded audio data corresponds to a frame of code stream data, and the receiving end decodes and plays each frame of code stream data.

[0035] Stream structure: Stream data is stored in a certain data structure, which is called stream structure. Generally speaking, the stream structure includes at least a header field and a coding field. The header field is used to store the length, coding bandwidth, number of channels and other parameters of the stream data, and the coding field is used to store the encoded audio data. The receiver obtains the meaning and length of each field of the stream structure, and then parses the stream data and plays it.

[0036] Figure 1 This is a flowchart of an audio data processing method provided by an embodiment of the present disclosure. The method can be applied to a terminal device and executed by the terminal device. The terminal device includes but is not limited to a mobile phone, a computer, a tablet computer, a car computer, a wearable device, and other devices that can collect audio data and encode and transmit audio data. Of course, the method can also be applied to a server and executed by the server. The following describes the method by taking the terminal device as the execution subject. Figure 1 As shown, the method includes:

[0037] Step S102, obtaining audio data to be processed; the sampling frequency of the audio data is greater than or equal to a preset frequency and the sampling depth is greater than or equal to a preset depth;

[0038] Step S104, performing sub-band decomposition on the audio data to obtain a plurality of sub-band data;

[0039] Step S106, determining the expected bit rate of each sub-band data after encoding; the sum of the expected bit rates of each sub-band data after encoding is equal to the expected bit rate of the audio data after encoding; the ratio of the bit rate of the audio data before encoding to the expected bit rate of the audio data after encoding is greater than a preset value;

[0040] Step S108, encoding each sub-band data according to the expected bit rate after encoding each sub-band data;

[0041] Step S110: Generate encoded audio data according to the encoded sub-band data.

[0042] Through this embodiment, for audio data having a sampling frequency greater than or equal to a preset frequency and a sampling depth greater than or equal to a preset depth, the encoded audio data can be obtained by decomposing the audio data into sub-bands and encoding the audio data. Since the ratio of the bit rate of the audio data before encoding to the expected bit rate of the audio data after encoding is greater than a preset value, the bit rate of the encoded audio data is greatly reduced, and the encoding efficiency of the audio data is improved. Since the bit rate of the encoded audio data is reduced, the bandwidth required for transmitting the encoded audio data and the data traffic are saved.

[0043] In the above step S102, the audio data to be processed is obtained. The sampling frequency of the audio data to be processed is greater than or equal to the preset frequency and the sampling depth is greater than or equal to the preset depth. In one embodiment, the preset frequency may be 96kHz and the preset depth may be 24 bits. Since the sampling frequency of the audio data to be processed is greater than or equal to the preset frequency and the sampling depth is greater than or equal to the preset depth, the audio data to be processed may also be referred to as high-resolution Hi-Res audio data.

[0044] In the above step S104, the audio data is sub-band decomposed to obtain a plurality of sub-band data. In one embodiment, the audio data can be sub-band decomposed by a PQMF filter to obtain a plurality of sub-band data. In one embodiment, the sampling frequency of each sub-band data is less than or equal to a preset frequency so that the sub-band data can be encoded successfully. For example, when encoding each sub-band data by an Opus encoder, since Opus can encode audio data with a sampling frequency of 48kHz at most, the sampling frequency of each sub-band data is required to be less than or equal to 48kHz. In this embodiment, for audio data with a sampling frequency of 192kHz, it can be decomposed into 4 sub-band data with a sampling frequency of 48kHz. Based on the sampling theorem, the effective frequency band range of the signal is half of the sampling frequency. Therefore, the effective frequency band range of the original 192kHz audio data is 0-96kHz. The decomposed sub-band data respectively represent the effective frequency band ranges of 0-24kHz, 24-48kHz, 48-72kHz, and 72-96kHz of the original audio data. The sub-band data are numbered in sequence to obtain serial numbers 1, 2, 3, and 4.

[0045] In the above step S106, the expected bit rate after encoding of each sub-band data is determined, the sum of the expected bit rates after encoding of each sub-band data is equal to the expected bit rate after encoding of the audio data, and the ratio of the bit rate before encoding of the audio data to the expected bit rate after encoding of the audio data is greater than a preset value. First, the expected bit rate after encoding of the audio data is obtained. The ratio of the bit rate before encoding of the audio data to the expected bit rate after encoding of the audio data is greater than a preset value. The expected bit rate after encoding of the audio data can be pre-stored in the terminal device. When pre-stored, it is required that the ratio of the bit rate before encoding of the audio data to the expected bit rate after encoding of the audio data is greater than a preset value. For example, if the preset value is 10, it is required that the ratio of the bit rate before encoding of the audio data to the expected bit rate after encoding of the audio data is greater than 10, that is, the encoding compression rate of the audio data is more than 10 times.

[0046] In a specific example, for 192kHz 24-bit dual-channel audio data, the bit rate before encoding is 192000*24*2bps, and the expected bit rate after encoding can be configured to be 500000bps, so that 192000*24*2 / 500000=18.432, so that the compression rate reaches more than 18 times. The size of the preset value can be determined based on experience to ensure that the quality of the encoded audio data is high while ensuring a high compression rate of the audio data.

[0047] In one embodiment, a preset value is determined in advance based on experience, and based on the preset value and the common bit rate of high-resolution audio data before encoding, the expected bit rate of the general high-resolution audio data after encoding is determined and stored in the terminal device. In this case, the expected bit rate of the audio data after encoding is a fixed value.

[0048] In another embodiment, a preset value is pre-configured inside the encoder of the terminal device. When encoding, the terminal device can obtain the bit rate of the audio data before encoding, and automatically generate the expected bit rate after encoding based on the preset value and the bit rate before encoding, and make the ratio of the bit rate of the audio data before encoding to the expected bit rate of the audio data after encoding greater than the preset value. In this case, the expected bit rate of the audio data after encoding is a value calculated in real time during encoding.

[0049] In the above step S106, the expected bit rate after encoding of each sub-band data is determined based on the expected bit rate after encoding of the audio data, so that the sum of the expected bit rates after encoding of each sub-band data is equal to the expected bit rate after encoding of the audio data.

[0050] In one embodiment, determining the expected bit rate after encoding of each sub-band data includes:

[0051] Obtaining a ratio of the expected bit rate after encoding of each sub-band data to the expected bit rate after encoding of the audio data;

[0052] The expected bit rate of each sub-band data after encoding is determined according to the expected bit rate of the audio data after encoding and the ratio of each sub-band data.

[0053] Specifically, first, the ratio of the expected bit rate after encoding of each sub-band data to the expected bit rate after encoding of the audio data is obtained. It can be understood that the sum of the ratios of the various sub-band data is 1, so that the sum of the expected bit rates after encoding of the various sub-band data is equal to the expected bit rate after encoding of the audio data. Then, according to the expected bit rate after encoding of the audio data and the ratio of each sub-band data, the expected bit rate after encoding of each sub-band data is determined. For example, for each sub-band data, the expected bit rate after encoding of each sub-band data can be obtained by multiplying the corresponding ratio by the expected bit rate after encoding of the audio data.

[0054] It can be seen that through this embodiment, the expected bit rate after encoding of each sub-band data can be determined efficiently and quickly by determining the ratio of the expected bit rate after encoding of each sub-band data to the expected bit rate after encoding of the audio data, and the bit rate determination process is simple and efficient.

[0055] In one embodiment, obtaining the ratio of the expected bit rate after encoding of each sub-band data to the expected bit rate after encoding of the audio data includes:

[0056] Determine the frequency band range of the audio data represented by each sub-band data;

[0057] According to the frequency band range of the audio data represented by each sub-band data, a ratio of the expected bit rate after encoding of each sub-band data to the expected bit rate after encoding of the audio data is determined.

[0058] First, the frequency band range of the audio data represented by each sub-band data is determined. For example, in the previous example, for sub-band data 1, the frequency band range of the audio data represented by it is 0 to 24 kHz. Then, according to the frequency band range of the audio data represented by each sub-band data, the proportion of the expected bit rate after encoding of each sub-band data to the expected bit rate after encoding of the audio data is determined. For example, the lower the frequency of the frequency band range of the audio data represented by the sub-band data, the greater the proportion of the expected bit rate after encoding of the sub-band data.

[0059] It can be seen that through this embodiment, the relationship between the frequency band range of the above-mentioned audio data represented by the sub-band data and the importance of the sub-band data is taken into consideration. Therefore, according to the frequency band range of the above-mentioned audio data represented by each sub-band data, the ratio of the expected bit rate after encoding of each sub-band data to the expected bit rate after encoding of the audio data is determined, so that more bit rates can be allocated to important sub-band data, thereby improving the quality of the encoded audio data.

[0060] In one embodiment, determining the ratio of the expected bit rate after encoding of each sub-band data to the expected bit rate after encoding of the audio data according to the frequency band range of the audio data represented by each sub-band data includes:

[0061] For the first sub-band data with the lowest frequency in the frequency band range of the above-mentioned audio data represented in each sub-band data, it is determined that the above-mentioned proportion of the first sub-band data is greater than the above-mentioned proportion of the second sub-band data in each sub-band data; the second sub-band data is the sub-band data in each sub-band data except the first sub-band data.

[0062] Specifically, there are multiple sub-band data. For the first sub-band data with the lowest frequency in the frequency band range of the above-mentioned audio data represented therein, the corresponding proportion is greater than the proportions corresponding to other sub-band data. Therefore, the first sub-band data is allocated the highest bit rate, so that the low-frequency signal is allocated more bit rate, and the quality of the low-frequency signal is better. Since the low-frequency signal is more easily perceived by the human ear than the high-frequency signal, allocating more bit rate to the low-frequency signal can better improve the quality of the encoded audio data.

[0063] In a specific embodiment, audio data with a sampling frequency of 192kHz can be decomposed into four sub-band data with a sampling frequency of 48kHz. Based on the sampling theorem, the effective frequency band range of the signal is half of the sampling frequency. Therefore, the effective frequency band range of the original 192kHz audio data is 0-96kHz. The decomposed sub-band data respectively represent the effective frequency band ranges of 0-24kHz, 24-48kHz, 48-72kHz, and 72-96kHz of the original audio data. The sub-band data are numbered in sequence to obtain serial numbers 1, 2, 3, and 4.

[0064] Based on the principle that the higher the sampling frequency, the lower the importance, for sub-band data 1, the ratio can be set to 0.6, for sub-band data 2, the ratio can be set to 0.2, and for sub-band data 3 and 4, the ratios are set to 0.1 respectively, so that the low-frequency signal is allocated more bit rate and the quality of the low-frequency signal is improved.

[0065] It can be seen that through this embodiment, the first sub-band data with the lowest frequency in the frequency band range of the above-mentioned audio data represented can be allocated the highest bit rate, thereby improving the quality of the low-frequency signal and the quality of the encoded voice data based on the principle that low-frequency signals are more easily perceived by the human ear than high-frequency signals.

[0066] In the above step S108, after determining the expected bit rate after encoding each sub-band data, each sub-band data is encoded according to the expected bit rate after encoding each sub-band data. Each sub-band data can be encoded by a mature encoder such as Opus encoder, which has the advantages of high compression rate after encoding and high quality of encoded audio data.

[0067] In the above step S110, the encoded audio data is generated according to each encoded sub-band data. For example, based on a preset bitstream structure, each encoded sub-band data is synthesized to obtain the encoded audio data.

[0068] In one embodiment, generating encoded audio data according to each encoded sub-band data includes:

[0069] Determine the encoded first sub-band data and the encoded second sub-band data in each encoded sub-band data; the first sub-band data is the sub-band data with the lowest frequency in the frequency band range of the audio data represented in each sub-band data, and the second sub-band data is the sub-band data other than the first sub-band data in each sub-band data;

[0070] The encoded first sub-band data and the encoded second sub-band data are filled in corresponding fields in a preset code stream structure to generate encoded audio data.

[0071] Specifically, each encoded sub-band data includes a first first sub-band data and multiple second sub-band data, the first sub-band data is the sub-band data with the lowest frequency in the frequency band range of the above-mentioned audio data represented in each sub-band data, and the second sub-band data is the other sub-band data in each sub-band data except the first sub-band data. First, the encoded first sub-band data and each encoded second sub-band data are determined in each encoded sub-band data.

[0072] Then, the encoded first sub-band data and the encoded second sub-band data are filled in corresponding fields in a preset code stream structure to generate encoded audio data. The preset code stream structure may be a code stream structure corresponding to an encoder. For example, when encoding each sub-band data through an Opus encoder, the preset code stream structure is a code stream structure possessed by the Opus encoder.

[0073] It can be seen that through this embodiment, the first sub-band data with the lowest frequency and other sub-band data in the frequency band range of the above-mentioned audio data represented can be filled in the corresponding fields in the preset code stream structure respectively, so as to generate the encoded audio data. The encoded audio data is generated by data filling, which has the advantage of high generation efficiency.

[0074] In one embodiment, filling the encoded first sub-band data and the encoded second sub-band data into corresponding fields in a preset bitstream structure to generate encoded audio data includes:

[0075] Filling the encoded first subband data into the field for filling the encoded audio data in the bitstream structure;

[0076] Filling the encoded second sub-band data into the field for filling additional data in the bitstream structure;

[0077] Generate encoded audio data based on the padded bitstream structure.

[0078] Specifically, the preset code stream structure has a field for filling the encoded audio data and a field for the user to fill in the additional data. For audio data, it is data presented frame by frame in time sequence, so each frame of audio data has a corresponding frame of code stream data, and a frame of code stream data is a frame of encoded audio data. When the length of each frame of code stream data needs to be adjusted to the same length, additional data such as some meaningless strings of 000 can be filled in the field for filling additional data, so that the length of each frame of code stream data is adjusted to the same length.

[0079] In this embodiment, the encoded first sub-band data is filled in the field for filling the encoded audio data in the code stream structure, so that the encoded first sub-band data can be parsed and processed by the audio data receiving end as the encoded audio data, and the encoded second sub-band data is filled in the field for filling the additional data in the code stream structure, so that when the audio data receiving end has the ability to parse and process the additional data, each encoded second sub-band data is extracted from the field for filling the additional data to decode the encoded audio data. Of course, if the audio data receiving end does not have the ability to parse and process the additional data, the receiving end can directly discard the encoded second sub-band data filled in the field for filling the additional data, and only implement the decoding work based on the encoded first sub-band data. Since the first sub-band data is the lowest frequency signal in the frequency band range of the above-mentioned audio data represented, that is, the most important signal, the decoding work is implemented only based on the encoded first sub-band data, which can also enable the receiving end to play audio data with higher quality.

[0080] In this embodiment, the encoded audio data is also generated based on the padded code stream structure, for example, the padded code stream structure is used as the encoded audio data.

[0081] It can be seen that, through this embodiment, the encoded first sub-band data is filled in the field for filling the encoded audio data in the code stream structure, so that the encoded first sub-band data can be parsed and processed by the audio data receiving end as the encoded audio data, and the encoded second sub-band data is filled in the field for filling the additional data in the code stream structure, so that when the audio data receiving end has the ability to parse and process the additional data, each encoded second sub-band data is extracted from the field for filling the additional data to decode the encoded audio data, and when the audio data receiving end does not have the ability to parse and process the additional data, the receiving end can directly discard the encoded second sub-band data filled in the field for filling the additional data, and only implement the decoding work based on the encoded first sub-band data, so that the encoded audio data can match the decoding capabilities of different receiving ends, and because the first sub-band data is the lowest frequency signal in the frequency band range of the above-mentioned audio data, that is, the most important signal, the decoding work is only based on the encoded first sub-band data, so that the receiving end can also play audio data with higher quality.

[0082] In one embodiment, the preset code stream structure includes a first field and a second field, the field value of the first field is used to indicate the length of each encoded second sub-band data, and the field value of the second field is used to indicate the total length of each encoded second sub-band data and the first field. Accordingly, based on the padded code stream structure, the encoded audio data is generated, including:

[0083] In the padded code stream structure, determining a field value of the first field according to the length of each encoded second sub-band data;

[0084] In the padded code stream structure, determining a field value of the second field according to the total length of each encoded second sub-band data and the first field;

[0085] Based on the determined code stream structure, the encoded audio data is generated.

[0086] Specifically, the preset code stream structure also includes a first field and a second field, the field value of the first field is used to indicate the length of each encoded second sub-band data, and the field value of the second field is used to indicate the total length of each encoded second sub-band data and the first field. Based on this, in this embodiment, after the encoded first sub-band data and the encoded second sub-band data are filled into the preset code stream structure, the field value of the first field is determined according to the length of each encoded second sub-band data in the filled code stream structure. The number of the first field can be multiple, each encoded second sub-band data corresponds to a first field, and the field value of the first field is equal to the length of the corresponding encoded second sub-band data.

[0087] Furthermore, in the padded code stream structure, the field value of the second field is determined according to the total length of each encoded second sub-band data and the first field, the number of the second field is one, and the field value of the second field is equal to the total length of each encoded second sub-band data and the first field.

[0088] Finally, based on the code stream structure determined by the field value, the encoded audio data is generated. For example, the code stream structure determined by the field value is determined as the encoded audio data. Of course, the encoded audio data may also include some common fields such as the frame header field, which is used to store parameters such as the length of the code stream data, the encoding bandwidth, and the number of channels.

[0089] It can be seen that through this embodiment, the encoded audio data can carry the first field and the second field, the length of each encoded second sub-band data is indicated by the field value of the first field, and the total length of each encoded second sub-band data and the first field is indicated by the field value of the second field, thereby facilitating the decoding of the encoded audio data and improving the decoding efficiency.

[0090] In a specific embodiment, when encoding each sub-band data through an Opus encoder, the preset bitstream structure is a bitstream structure possessed by the Opus encoder. Figure 2a A schematic diagram of a preset code stream structure provided in an embodiment of the present disclosure, such as Figure 2aAs shown in FIG. 1 , the code stream structure of the Opus encoder includes a frame header field, an additional data total length field, a coded data field, and an additional data field. The coded data field is also called a payload field, and the additional data field is also called a padding field.

[0091] The length of the frame header field is a fixed value, usually 1 to 2 bytes. The frame header field is used to store parameters such as the length of the bitstream data, the coding bandwidth, and the number of channels. The frame header field also carries a flag to indicate whether the bitstream data carries padding data. If the flag is 1, it means that the bitstream data carries padding data. If the flag is 0, it means that the bitstream data does not carry padding data.

[0092] The additional data total length field is the aforementioned second field, and the length of the additional data total length field is a fixed value, usually 1 to 2 bytes. The field value of the additional data total length field is used to indicate the total length of the additional data field.

[0093] The coded data field is the field mentioned above for filling the coded audio data. The length of the coded data field is not fixed, and depends on the data length of the coded first sub-band data.

[0094] The additional data field is the field mentioned above for filling additional data. The length of the additional data field is not fixed, and depends on the data length of the encoded second sub-band data. The first field mentioned above is located in the additional data field.

[0095] Figure 2b A schematic diagram of encoded audio data provided by an embodiment of the present disclosure, such as Figure 2b As shown, the encoded first subband data is Figure 2b The sub-stream-1 data in the coded second sub-band data is Figure 2b Substream-2 data, substream-3 data and substream-4 data in the above-mentioned audio data. Continuing with the previous example, the frequency of the frequency band range of the audio data represented by the substream-1 data is the lowest, and the code rate accounts for 60% of the expected code rate of the encoded audio data, the frequency of the frequency band range of the audio data represented by the substream-2 data is higher than the substream-1 data, and the code rate accounts for 20% of the expected code rate of the encoded audio data, the frequency of the frequency band range of the audio data represented by the substream-3 data is higher than the substream-2 data, and the code rate accounts for 10% of the expected code rate of the encoded audio data, and the frequency of the frequency band range of the audio data represented by the substream-4 data is higher than the substream-3 data, and the code rate accounts for 10% of the expected code rate of the encoded audio data.

[0096] like Figure 2bAs shown in the figure, after obtaining the encoded first sub-band data and each encoded second sub-band data, the encoded first sub-band data is filled in the encoded data field, and each encoded second sub-band data is filled in the additional data field. In addition, the additional data field also includes a first field corresponding to each encoded second sub-band data, that is, the field represented by "sub-stream-2 length", "sub-stream-3 length" and "sub-stream-4 length" in the figure, and the first field is used to represent the length of the corresponding encoded second sub-band data.

[0097] Figure 2b In the figure, the field of total length of additional data is the aforementioned second field, and its field value is used to indicate the length of the additional data. As can be seen from the figure, the length of the additional data is the total length of each encoded second sub-band data and each first field.

[0098] from Figure 2a and Figure 2b It can be seen that after encoding each sub-band data through the Opus encoder, the code stream structure of the generated encoded audio data is the same as the code stream structure of the Opus encoder itself, so that the encoded audio data is compatible with the Opus encoder. If the receiving end also uses the Opus encoder to decode the encoded audio data, then if the receiving end has the ability to identify and parse the additional data field, it can parse and obtain the decoded data corresponding to each sub-band data. If it does not have the ability, the additional data field can be discarded, and only the decoded data corresponding to the first sub-band data can be parsed to obtain the decoded data, and the decoded data corresponding to the first sub-band data can be played.

[0099] When encoding each sub-band data through the Opus encoder, you can change the native limit on the expected bit rate after encoding of the Opus encoder, and configure the expected bit rate after encoding suitable for high-resolution audio data. Specifically, the Opus encoder natively supports a maximum expected bit rate after encoding of 510kbps, and this value can be modified to 2400kbps, so that the Opus encoder can encode high-resolution audio data with a sampling frequency of 192kHz, for example. Developers can also configure the expected bit rate of the encoded audio data within the Opus encoder, and the expected bit rate can be successfully configured as long as it is lower than 2400kbps.

[0100] Figure 3 A schematic diagram of the principle of audio data encoding provided by an embodiment of the present disclosure is shown in FIG. Figure 3As shown in the figure, the same as described above, when encoding, the audio data is first decomposed into sub-bands to obtain multiple sub-band data. The sequence number of the sub-band data can be determined according to the frequency of the frequency band range of the audio data represented by the sub-band data. The lower the frequency range represented, the smaller the sequence number. Then, each sub-band data is encoded to obtain the encoded sub-band data. The encoded sub-band data can be called a sub-code stream. Then, according to Figure 2b The method shown performs stream multiplexing on each sub-stream, and stream multiplexing is to fill each sub-stream into a preset stream structure to obtain encoded audio data, and finally output a stream, which is the encoded audio data.

[0101] The above describes the process of generating the encoded audio data. Corresponding to the code stream structure of the encoded audio data, in one embodiment, the encoded audio data is decoded to obtain decoded data in the following manner:

[0102] extracting the encoded first sub-band data and each encoded second sub-band data from the encoded audio data according to the field value of the first field, the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data, and the total length of the encoded audio data;

[0103] Decoded data is obtained based on the encoded first sub-band data and each encoded second sub-band data.

[0104] First, according to the field value of the first field, the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data, and the total length of the encoded audio data, the encoded first subband data and each encoded second subband data are extracted from the encoded audio data, and then, based on the encoded first subband data and each encoded second subband data, decoded data is obtained by subband synthesis. Considering that the encoding process is lossy encoding, the data obtained by decoding the encoded audio data is called decoded data.

[0105] It can be seen that through this embodiment, the encoded first sub-band data and each encoded second sub-band data can be extracted from the encoded audio data, and then based on the encoded first sub-band data and each encoded second sub-band data, the decoded data corresponding to the encoded audio data can be obtained. The decoding process is simple and the decoding efficiency is high.

[0106] In one embodiment, extracting the encoded first sub-band data and each encoded second sub-band data from the encoded audio data according to the field value of the first field, the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data, and the total length of the encoded audio data includes:

[0107] Extracting each encoded second sub-band data from the encoded audio data according to the field value of the first field and the field value of the second field;

[0108] The encoded first subband data is extracted from the encoded audio data according to the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data, and the total length of the encoded audio data.

[0109] First, since the field value of the first field is used to represent the length of each encoded second subband data, and the field value of the second field is used to represent the sum of the lengths of each encoded second subband data and each first field, each encoded second subband data can be extracted from the encoded audio data based on the field value of the first field and the field value of the second field.

[0110] Next, the encoded first subband data is extracted from the encoded audio data according to the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data, and the total length of the encoded audio data.

[0111] It can be seen that, through this embodiment, the encoded first sub-band data and each encoded second sub-band data can be extracted from the encoded audio data efficiently and quickly.

[0112] In one embodiment, extracting each encoded second sub-band data from the encoded audio data according to the field value of the first field and the field value of the second field includes:

[0113] Extracting additional data from the encoded audio data according to the field value of the second field; the additional data includes each encoded second sub-band data and each first field;

[0114] Each encoded second sub-band data is extracted from the additional data according to the field value of the first field.

[0115] Specifically, according to the previous description, the field value of each first field is used to indicate the length of each encoded second sub-band data, and the field value of the second field is used to indicate the total length of each encoded second sub-band data and each first field. Based on this, combined with Figure 2b As shown, according to the field value of the second field, starting from the end of the encoded audio data, that is, the bitstream data, each encoded second sub-band data and each first field can be found, that is, the additional data field can be found.

[0116] Next, each encoded second sub-band data is extracted in sequence starting from the beginning of the additional data according to the field value of each first field. Figure 2bIn the code stream data, the left side is set as the data start position, the right side is set as the data end position, and the code stream data is arranged from left to right.

[0117] It can be seen that, through this embodiment, each encoded second sub-band data can be extracted efficiently and quickly from the encoded audio data based on the field value of the first field and the field value of the second field.

[0118] In one embodiment, extracting the encoded first subband data from the encoded audio data according to the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data, and the total length of the encoded audio data includes:

[0119] Determine the length of the encoded first sub-band data according to the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data, and the total length of the encoded audio data;

[0120] The encoded first sub-band data is extracted from the encoded audio data according to the length of the encoded first sub-band data.

[0121] Specifically, combined Figure 2b As shown, the total length of the encoded audio data can be extracted from the frame header field. The length of the encoded first sub-band data, that is, the length of the encoded data field, can be calculated by the following formula:

[0122] The length of the coded data field = the total length - the length of the frame header field - the length of the additional data total length field - the length of the additional data field;

[0123] Among them, the length of the additional data field is the field value of the second field, and the length of the additional data total length field is the length of the second field. Usually, the length of the additional data total length field is a fixed value and can also be recorded in the frame header field.

[0124] Therefore, the total length of the encoded audio data minus the length of the frame header field minus the field value of the second field minus the length of the second field can be used to obtain the length of the encoded first sub-band data.

[0125] Next, according to the length of the encoded first sub-band data, starting from the start position of the encoded audio data, skipping the fixed-length frame header field and the fixed-length additional data total length field, the encoded first sub-band data can be found.

[0126] It can be seen that through this embodiment, the length of the encoded first sub-band data can be determined according to the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data, and the total length of the encoded audio data, and the encoded first sub-band data can be accurately extracted from the encoded audio data according to the length of the encoded first sub-band data.

[0127] After extracting the encoded first sub-band data and each encoded second sub-band data, in one embodiment, obtaining decoded data based on the encoded first sub-band data and each encoded second sub-band data includes:

[0128] Decoding the encoded first sub-band data and each encoded second sub-band data to obtain first sub-decoded data and each second sub-decoded data respectively;

[0129] Sub-band synthesis is performed on the first sub-decoded data and each second sub-decoded data to obtain decoded data.

[0130] Specifically, the encoded first sub-band data is decoded to obtain first sub-decoded data. Since the first sub-band data is encoded in a lossy coding manner, the data obtained by decoding the encoded first sub-band data is called first sub-decoded data. Then, each encoded second sub-band data is decoded to obtain second sub-decoded data corresponding to each encoded second sub-band data. Since the second sub-band data is encoded in a lossy coding manner, the data obtained by decoding the encoded second sub-band data is called second sub-decoded data. One encoded second sub-band data corresponds to one second sub-decoded data.

[0131] Next, sub-band synthesis is performed on the first sub-decoded data and each second sub-decoded data to obtain decoded data. The sub-band synthesis may be performed using a PQMF method.

[0132] In a specific embodiment, the receiving end receives Figure 2b After the code stream data shown, the frame header field is first parsed, and the flag bit of whether the current code stream data carries the additional data field is obtained from the frame header field. If the additional data field is carried, the length of the additional data field is first parsed, and then the additional data is taken out from the tail of the code stream data according to the additional data field, and the length of each encoded second sub-band data is successively parsed in the additional data and each second sub-band data is extracted. Then, the length of the encoded first sub-band data is calculated, and the encoded first sub-band data is extracted from the code stream data according to the length.

[0133] Next, Opus decoding is performed on the encoded first sub-band data and each encoded second sub-band data to obtain first sub-decoded data and each second sub-decoded data respectively. Finally, sub-band synthesis of the first sub-decoded data and each second sub-decoded data is performed using the PQMF method to obtain decoded data.

[0134] It can be seen that, through this embodiment, the encoded sub-band data can be decoded and synthesized, thereby efficiently obtaining decoded data.

[0135] Figure 4 A schematic diagram of the principle of audio data decoding provided by an embodiment of the present disclosure is shown in FIG. Figure 4 As shown, as described above, after the receiving end obtains the input code stream, that is, the encoded audio data, it parses the code stream to obtain each sub-code stream, that is, each encoded sub-band data, decodes each encoded sub-band data to obtain each sub-decoded data, the sub-decoded data includes the aforementioned first sub-decoded data and the second sub-decoded data, and then performs sub-band synthesis on each sub-decoded data to obtain output data, which is the decoded data.

[0136] In a specific embodiment, the encoding process and decoding process described above can be implemented based on the Opus encoder, and the specific implementation process is:

[0137] 1. The developers changed the expected bitrate after encoding from the native maximum supported by the Opus encoder of 510kbps to 2400kbps, so that the Opus encoder can encode high-resolution audio data with a sampling frequency of 192kHz;

[0138] 2. The developer configures the expected bit rate of the encoded audio data in the Opus encoder. The configuration is successful as long as the expected bit rate is lower than 2400kbps.

[0139] 3. Obtain high-resolution audio data and input the audio data into the Opus encoder;

[0140] 4. The Opus encoder decomposes the audio data into sub-bands and determines the expected bit rate after encoding each sub-band data according to the pre-configured expected bit rate;

[0141] 5. The Opus encoder encodes each sub-band data according to the expected bit rate after encoding each sub-band data;

[0142] 6. The Opus encoder generates encoded audio data based on the encoded sub-band data.

[0143] Of course, the Opus encoder in this embodiment may also be replaced by an encoder such as EVS or AAC.

[0144] It can be seen that through this embodiment, the following technical effects are achieved:

[0145] 1. The sampling frequency supported by the Opus encoder has been expanded. For high-resolution audio data, it can also be encoded and decoded in the Opus encoder through sub-band decomposition;

[0146] 2. For high-resolution audio data, encoding is performed through sub-band decomposition to achieve a relatively high compression rate, so that the encoded audio data can be used in real-time communication scenarios such as video conferencing. Experiments have shown that for audio data with 192kHz sampling and 24-bit bit depth, the entire frequency band can be encoded at a bit rate of 500kbps, and the quality after encoding and decoding is basically undamaged, with a compression rate of more than 10 times.

[0147] Figure 5 A schematic diagram of an application scenario of audio data encoding and decoding provided by an embodiment of the present disclosure, such as Figure 5 As shown, in a real-time communication scenario such as a video conference scenario, multiple terminal devices (represented by terminal device 501 and terminal device 502 in the figure) can realize real-time audio data transmission through server 503. Through the encoding and decoding method in this embodiment, the bit rate of audio data can be greatly reduced, thereby realizing real-time transmission.

[0148] In addition, the server can also obtain the codec capabilities supported by each terminal device. When any of the multiple terminal devices for real-time communication in the same scenario, such as the same room or the same meeting, does not support the codec capabilities for high-resolution audio data in the above embodiments, the server notifies each terminal device to collect and generate low-resolution (sampling frequency is lower than the preset frequency or sampling depth is lower than the preset depth) audio data. When all terminal devices for real-time communication in the same scenario support the codec capabilities for high-resolution audio data in the above embodiments, the server notifies each terminal device to collect and generate high-resolution (sampling frequency is greater than or equal to the preset frequency and sampling depth is greater than or equal to the preset depth) audio data, and executes the above scheme, thereby improving the quality of real-time audio communication.

[0149] Corresponding to the above embodiments, an embodiment of the present disclosure provides an audio data processing device, Figure 6 A schematic diagram of the structure of an audio data processing device provided in an embodiment of the present disclosure is shown in FIG. Figure 6 As shown, the device comprises:

[0150] An acquisition unit 61 is used to acquire audio data to be processed; the sampling frequency of the audio data is greater than or equal to a preset frequency and the sampling depth is greater than or equal to a preset depth;

[0151] A decomposition unit 62, configured to perform sub-band decomposition on the audio data to obtain a plurality of sub-band data;

[0152] The determining unit 63 is used to determine the expected bit rate of each sub-band data after encoding; the sum of the expected bit rates of each sub-band data after encoding is equal to the expected bit rate of the audio data after encoding; the ratio of the bit rate of the audio data before encoding to the expected bit rate of the audio data after encoding is greater than a preset value;

[0153] The encoding unit 64 is used to encode each of the sub-band data according to the expected bit rate after encoding each of the sub-band data;

[0154] The generating unit 65 is configured to generate the encoded audio data according to the encoded sub-band data.

[0155] Optionally, the determining unit 63 is specifically configured to:

[0156] Obtaining a ratio of the expected bit rate after encoding of each of the sub-band data to the expected bit rate after encoding of the audio data;

[0157] The expected bit rate of each sub-band data after encoding is determined according to the expected bit rate of the audio data after encoding and the ratio of each sub-band data.

[0158] Optionally, the determining unit 63 is further specifically configured to:

[0159] Determine the frequency band range of the audio data represented by each sub-band data;

[0160] According to the frequency band range of the audio data represented by each sub-band data, a ratio of the expected bit rate after encoding of each sub-band data to the expected bit rate after encoding of the audio data is determined.

[0161] Optionally, the determining unit 63 is further specifically configured to:

[0162] For the first sub-band data with the lowest frequency in the frequency band range of the audio data represented in each of the sub-band data, it is determined that the proportion of the first sub-band data is greater than the proportion of the second sub-band data in each of the sub-band data; the second sub-band data is the sub-band data in each of the sub-band data except the first sub-band data.

[0163] Optionally, the generating unit 65 is specifically configured to:

[0164] Determine encoded first sub-band data and encoded second sub-band data in each of the encoded sub-band data; the first sub-band data is the sub-band data with the lowest frequency in the frequency band range of the audio data represented in each of the sub-band data, and the second sub-band data is the sub-band data other than the first sub-band data in each of the sub-band data;

[0165] The encoded first sub-band data and the encoded second sub-band data are filled in corresponding fields in a preset code stream structure to generate the encoded audio data.

[0166] Optionally, the generating unit 65 is further specifically configured to:

[0167] Filling the encoded first subband data into the field for filling the encoded audio data in the bitstream structure;

[0168] Filling the encoded second subband data into a field for filling additional data in the bitstream structure;

[0169] Based on the padded code stream structure, the encoded audio data is generated.

[0170] Optionally, the code stream structure includes a first field and a second field; the field value of the first field is used to indicate the length of each encoded second sub-band data; the field value of the second field is used to indicate the total length of each encoded second sub-band data and the first field; the generating unit 65 is further specifically used to:

[0171] In the padded code stream structure, determining a field value of the first field according to the length of each encoded second sub-band data;

[0172] In the padded code stream structure, determining a field value of the second field according to a total length of each encoded second sub-band data and the first field;

[0173] Based on the determined code stream structure, the encoded audio data is generated.

[0174] Optionally, a decoding unit is further included, configured to:

[0175] extracting the encoded first sub-band data and each encoded second sub-band data from the encoded audio data according to a field value of the first field, a field value of the second field, a length of the second field, a length of a frame header field of the encoded audio data, and a total length of the encoded audio data;

[0176] The decoded data is obtained based on the encoded first sub-band data and each encoded second sub-band data.

[0177] Optionally, the decoding unit is specifically used for:

[0178] Extracting each encoded second sub-band data from the encoded audio data according to the field value of the first field and the field value of the second field;

[0179] The encoded first subband data is extracted from the encoded audio data according to the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data, and the total length of the encoded audio data.

[0180] Optionally, the decoding unit is further specifically configured to:

[0181] extracting the additional data from the encoded audio data according to the field value of the second field; the additional data includes each encoded second sub-band data and the first field;

[0182] According to the field value of the first field, each encoded second sub-band data is extracted from the additional data.

[0183] Optionally, the decoding unit is further specifically configured to:

[0184] Determine the length of the encoded first sub-band data according to the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data, and the total length of the encoded audio data;

[0185] The encoded first sub-band data is extracted from the encoded audio data according to the length of the encoded first sub-band data.

[0186] Optionally, the decoding unit is specifically used for:

[0187] Decoding the encoded first sub-band data and each encoded second sub-band data to obtain first sub-decoded data and each second sub-decoded data respectively;

[0188] Sub-band synthesis is performed on the first sub-decoded data and each of the second sub-decoded data to obtain the decoded data.

[0189] The audio data processing device in the embodiment of the present disclosure can implement the various processes of the above-mentioned audio data processing method embodiment and achieve the same effects and functions, which will not be repeated here.

[0190] An embodiment of the present disclosure further provides an electronic device, Figure 7A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure, such as Figure 7 As shown, the electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors 701 and memory 702, and one or more applications or data may be stored in the memory 702. Among them, the memory 702 may be a temporary storage or a permanent storage. The application stored in the memory 702 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the electronic device. Furthermore, the processor 701 may be configured to communicate with the memory 702 to execute a series of computer executable instructions in the memory 702 on the electronic device. The electronic device may also include one or more power supplies 703, one or more wired or wireless network interfaces 704, one or more input or output interfaces 705, one or more keyboards 706, etc.

[0191] In a specific embodiment, the electronic device includes a processor; and a memory configured to store computer executable instructions, wherein when the computer executable instructions are executed, the processor implements the following process:

[0192] Acquire audio data to be processed; the sampling frequency of the audio data is greater than or equal to a preset frequency and the sampling depth is greater than or equal to a preset depth;

[0193] Decomposing the audio data into sub-bands to obtain a plurality of sub-band data;

[0194] Determine an expected bit rate after encoding of each sub-band data; the sum of the expected bit rates after encoding of each sub-band data is equal to the expected bit rate after encoding of the audio data; the ratio of the bit rate before encoding of the audio data to the expected bit rate after encoding of the audio data is greater than a preset value;

[0195] Encoding each of the sub-band data according to an expected bit rate after encoding each of the sub-band data;

[0196] The encoded audio data is generated according to the encoded sub-band data.

[0197] The electronic device in the embodiment of the present disclosure can implement each process of the above-mentioned audio data processing method embodiment and achieve the same effect and function, which will not be repeated here.

[0198] Another embodiment of the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and when the computer-executable instructions are executed by a processor, the following process is implemented:

[0199] Acquire audio data to be processed; the sampling frequency of the audio data is greater than or equal to a preset frequency and the sampling depth is greater than or equal to a preset depth;

[0200] Decomposing the audio data into sub-bands to obtain a plurality of sub-band data;

[0201] Determine an expected bit rate after encoding of each sub-band data; the sum of the expected bit rates after encoding of each sub-band data is equal to the expected bit rate after encoding of the audio data; the ratio of the bit rate before encoding of the audio data to the expected bit rate after encoding of the audio data is greater than a preset value;

[0202] Encoding each of the sub-band data according to an expected bit rate after encoding each of the sub-band data;

[0203] The encoded audio data is generated according to the encoded sub-band data.

[0204] The storage medium in the embodiment of the present disclosure can implement each process of the above-mentioned audio data processing method embodiment and achieve the same effect and function, which will not be repeated here.

[0205] In various embodiments of the present disclosure, the computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0206] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0207] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0208] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0209] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing the embodiments of the present disclosure, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0210] Those skilled in the art will appreciate that one or more embodiments of the present disclosure may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, one or more embodiments of the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0211] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0212] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0213] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0214] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0215] One or more embodiments of the present disclosure may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present disclosure may also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0216] Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0217] The above description is only an embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the scope of the claims of the present disclosure.

Claims

1. An audio data processing method, characterized in that: include: Get the audio data to be processed; The sampling frequency of the audio data is greater than or equal to a preset frequency and the sampling depth is greater than or equal to a preset depth; Decomposing the audio data into sub-bands to obtain a plurality of sub-band data; Determine an expected bit rate after encoding of each sub-band data; the sum of the expected bit rates after encoding of each sub-band data is equal to the expected bit rate after encoding of the audio data; the ratio of the bit rate before encoding of the audio data to the expected bit rate after encoding of the audio data is greater than a preset value; Encoding each of the sub-band data according to an expected bit rate after encoding each of the sub-band data; The encoded audio data is generated according to the encoded sub-band data.

2. The method according to claim 1, characterized in that The determining of the expected bit rate after encoding of each sub-band data comprises: Obtaining a ratio of the expected bit rate after encoding of each of the sub-band data to the expected bit rate after encoding of the audio data; The expected bit rate of each sub-band data after encoding is determined according to the expected bit rate of the audio data after encoding and the ratio of each sub-band data.

3. The method according to claim 2, characterized in that The obtaining a ratio of the expected bit rate after encoding of each sub-band data to the expected bit rate after encoding of the audio data includes: Determine the frequency band range of the audio data represented by each sub-band data; According to the frequency band range of the audio data represented by each sub-band data, a ratio of the expected bit rate after encoding of each sub-band data to the expected bit rate after encoding of the audio data is determined.

4. The method according to claim 3, characterized in that The determining, according to the frequency band range of the audio data represented by each sub-band data, a ratio of the expected bit rate after encoding of each sub-band data to the expected bit rate after encoding of the audio data, comprises: For the first sub-band data with the lowest frequency in the frequency band range of the audio data represented in each of the sub-band data, it is determined that the proportion of the first sub-band data is greater than the proportion of the second sub-band data in each of the sub-band data; the second sub-band data is the sub-band data in each of the sub-band data except the first sub-band data.

5. The method according to claim 1, characterized in that The step of generating the encoded audio data according to the encoded sub-band data comprises: Determine encoded first sub-band data and encoded second sub-band data in each of the encoded sub-band data; the first sub-band data is the sub-band data with the lowest frequency in the frequency band range of the audio data represented in each of the sub-band data, and the second sub-band data is the sub-band data other than the first sub-band data in each of the sub-band data; The encoded first sub-band data and the encoded second sub-band data are filled in corresponding fields in a preset code stream structure to generate the encoded audio data.

6. The method according to claim 5, characterized in that Filling the encoded first sub-band data and the encoded second sub-band data into corresponding fields in a preset code stream structure to generate the encoded audio data includes: Filling the encoded first subband data into the field for filling the encoded audio data in the bitstream structure; Filling the encoded second subband data into a field for filling additional data in the bitstream structure; Based on the padded code stream structure, the encoded audio data is generated.

7. The method according to claim 6, characterized in that The code stream structure includes a first field and a second field; the field value of the first field is used to indicate the length of each encoded second sub-band data; the field value of the second field is used to indicate the total length of each encoded second sub-band data and the first field; The step of generating the encoded audio data based on the padded code stream structure includes: In the padded code stream structure, determining a field value of the first field according to the length of each encoded second sub-band data; In the padded code stream structure, determining a field value of the second field according to a total length of each encoded second sub-band data and the first field; Based on the determined code stream structure, the encoded audio data is generated.

8. The method according to claim 7, characterized in that The encoded audio data is decoded to obtain decoded data in the following manner: extracting the encoded first sub-band data and each encoded second sub-band data from the encoded audio data according to a field value of the first field, a field value of the second field, a length of the second field, a length of a frame header field of the encoded audio data, and a total length of the encoded audio data; The decoded data is obtained based on the encoded first sub-band data and each encoded second sub-band data.

9. The method according to claim 8, characterized in that The extracting the encoded first sub-band data and each encoded second sub-band data from the encoded audio data according to the field value of the first field, the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data, and the total length of the encoded audio data, comprises: Extracting each encoded second sub-band data from the encoded audio data according to the field value of the first field and the field value of the second field; The encoded first subband data is extracted from the encoded audio data according to the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data, and the total length of the encoded audio data.

10. The method according to claim 9, characterized in that The step of extracting each encoded second sub-band data from the encoded audio data according to the field value of the first field and the field value of the second field comprises: extracting the additional data from the encoded audio data according to the field value of the second field; the additional data includes each encoded second sub-band data and the first field; According to the field value of the first field, each encoded second sub-band data is extracted from the additional data.

11. The method according to claim 9, characterized in that The extracting the encoded first sub-band data from the encoded audio data according to the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data, and the total length of the encoded audio data comprises: Determine the length of the encoded first sub-band data according to the field value of the second field, the length of the second field, the length of the frame header field of the encoded audio data, and the total length of the encoded audio data; The encoded first sub-band data is extracted from the encoded audio data according to the length of the encoded first sub-band data.

12. The method according to claim 8, characterized in that The obtaining the decoded data based on the encoded first sub-band data and each encoded second sub-band data comprises: Decoding the encoded first sub-band data and each encoded second sub-band data to obtain first sub-decoded data and each second sub-decoded data respectively; Sub-band synthesis is performed on the first sub-decoded data and each of the second sub-decoded data to obtain the decoded data.

13. An audio data processing device, characterized in that: include: An acquisition unit, used for acquiring audio data to be processed; The sampling frequency of the audio data is greater than or equal to a preset frequency and the sampling depth is greater than or equal to a preset depth; A decomposition unit, configured to perform sub-band decomposition on the audio data to obtain a plurality of sub-band data; a determining unit, configured to determine an expected bit rate after encoding of each of the sub-band data; the sum of the expected bit rates after encoding of the sub-band data is equal to the expected bit rate after encoding of the audio data; and a ratio of the bit rate before encoding of the audio data to the expected bit rate after encoding of the audio data is greater than a preset value; an encoding unit, configured to encode each of the sub-band data according to an expected bit rate after encoding each of the sub-band data; A generating unit is used to generate the encoded audio data according to the encoded sub-band data.

14. An electronic device, characterized in that: include: processor; as well as, A memory configured to store computer executable instructions, which, when executed, cause the processor to implement the steps of the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store computer-executable instructions, and the computer-executable instructions, when executed by a processor, implement the steps of the method described in any one of claims 1 to 12.

Images