Method, apparatus and storage medium for transmitting audio data

Abstract

The present disclosure relates to the field of information processing, and specifically discloses a kind of transmission method, device, electronic equipment and storage medium of audio data.The transmission method of audio data includes: the current sampling value is obtained by executing downsampling processing to original audio data, the difference between current sampling value and adjacent sampling value is calculated;From at least two kinds of candidate data packet formats, the target data packet format corresponding to the numerical value of difference is selected, and the difference is encoded into difference encoding data that matches the encoding bit number of target data packet format;Difference encoding data is encapsulated in the data packet matched with target data packet format, to be transmitted to audio receiving end.The combination of the above-mentioned various processing methods, such as downsampling, difference encoding and dynamic selection of data packet format, can greatly reduce the data volume of audio data, reduce transmission delay, and is especially suitable for service scenarios with high real-time requirements.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of information processing, and in particular, to a method and apparatus for transmitting audio data, an electronic device, and a storage medium. BACKGROUND

[0002] In the process of transmitting audio data, in order to reduce bandwidth consumption and transmission delay, it is usually necessary to compress the audio data by using a compression algorithm.

[0003] However, for some audio data with more low-frequency components, most compression algorithms need to rely on fast Fourier transform to process low-frequency components, and fast Fourier transform itself needs to consume a large time window, so it is difficult to achieve low-delay compression effect. Moreover, Fourier transform also introduces a large amount of calculation, thereby further increasing the transmission delay.

[0004] Therefore, how to provide an efficient compression method for audio data with more low-frequency components has become a technical problem to be solved. SUMMARY

[0005] In view of the above problems, the present disclosure is proposed to provide a method and apparatus for transmitting audio data, an electronic device, and a storage medium that overcome the above problems or at least partially solve the above problems.

[0006] According to an aspect of an embodiment of the present disclosure, a method for transmitting audio data is provided, which is applicable to an audio sending end, and the method comprises:

[0007] performing down-sampling processing on original audio data to obtain a current sampling value, and calculating a difference value between the current sampling value and a neighboring sampling value;

[0008] selecting a target data packet format corresponding to the numerical value of the difference value from at least two candidate data packet formats, and encoding the difference value into difference encoding data matching the number of encoding bits of the target data packet format;

[0009] encapsulating the difference encoding data in a data packet matching the target data packet format, so as to be transmitted to an audio receiving end.

[0010] In an optional implementation, the selecting a target data packet format corresponding to the numerical value of the difference value from at least two candidate data packet formats specifically comprises:

[0011] comparing the difference value with a preset interval threshold, and selecting the target data packet format from the at least two candidate data packet formats according to the comparison result;

[0012] Wherein, if the difference is greater than the interval threshold, the first data packet format among at least two candidate data packet formats is determined as the target data packet format; if the difference is not greater than the interval threshold, the second data packet format among at least two candidate data packet formats is determined as the target data packet format; wherein, the encoding bit length of the second data packet format is less than the encoding bit length of the first data packet format;

[0013] The target data packet format includes a bit width identifier, which indicates the number of bits in the target data packet format encoding.

[0014] In another optional implementation, the audio data includes: first audio data corresponding to the first channel and second audio data corresponding to the second channel; then calculating the difference between the current sample value and the adjacent sample value specifically includes: calculating the first difference corresponding to the first audio data and the second difference corresponding to the second audio data;

[0015] The step of selecting a target data packet format from at least two candidate data packet formats that corresponds to the numerical value of the difference specifically includes:

[0016] Based on the comparison result between the first difference and the second difference, the target data packet format is selected from at least two candidate data packet formats;

[0017] The target data packet format includes a channel identifier bit, which is used to indicate whether the first difference and the second difference are equal.

[0018] In another optional implementation, encoding the difference into difference-coded data that matches the number of bits in the target data packet format specifically includes:

[0019] Obtain the quantized value of the difference, and according to the encoding mapping table corresponding to the encoding bits of the target data packet format, look up the encoding value corresponding to the quantized value, and encode the quantized value into difference encoded data that matches the encoding value;

[0020] The encoding mapping table is used to store the mapping relationship between each encoding value corresponding to the encoding bit length and the quantization value of the difference.

[0021] In another alternative implementation, the number of quantized values ​​of the difference is greater than the number of encoded values ​​corresponding to the number of encoded bits, and the number of quantized values ​​corresponding to each encoded value is determined according to the frequency of occurrence of the quantized value in the historical audio.

[0022] If the frequency of the quantization value in the historical audio is greater than the preset frequency threshold, then the quantization value and the encoded value correspond one-to-one.

[0023] If the frequency of a quantization value in historical audio does not exceed a preset frequency threshold, then multiple quantization values ​​correspond to the same encoded value.

[0024] In another optional implementation, the adjacent sample value is the previous sample value of the current sample value; the step of querying the encoded value corresponding to the quantized value of the difference, and encoding the quantized value of the difference into difference-coded data that matches the encoded value, specifically includes:

[0025] If there is an error between the queried encoded value and the quantized value of the difference, the error value between the queried encoded value and the quantized value of the difference is stored; wherein, the error value is used for error compensation processing for the next sampled value of the current sampled value.

[0026] In another alternative implementation, the raw audio data is used to drive a target vibration device in the cloud gaming.

[0027] The downsampling process on the original audio data specifically includes: performing downsampling on the original audio data according to the downsampling frequency; wherein the downsampling frequency is at least twice the highest vibration frequency of the target vibration device.

[0028] According to another aspect of the present disclosure, a method for transmitting audio data is provided, applicable to an audio receiving end, the method comprising:

[0029] Acquire data packets from an audio transmitter, determine a target data packet format corresponding to the data packet from at least two candidate data packet formats, and the encoding bit length of the target data packet format;

[0030] The difference-coded data contained in the data packet is decoded according to the specified number of encoding bits to obtain the difference value corresponding to the difference-coded data;

[0031] The current decoded value is obtained by summing the difference with the adjacent decoded values.

[0032] Upsampling is performed on the current decoded value, and the original audio data is reconstructed based on the upsampling results.

[0033] In one optional implementation, determining the target data packet format corresponding to the data packet from at least two candidate data packet formats, and the encoding bit length of the target data packet format specifically includes:

[0034] Obtain the value of the bit identifier bit contained in the data packet, determine the target data packet format corresponding to the data packet and the encoding bit length of the target data packet format based on the value of the bit identifier bit; and / or,

[0035] Obtain the value of the channel identifier bit contained in the data packet, and determine the target data packet format corresponding to the data packet and the encoding bit length of the target data packet format based on the value of the channel identifier bit.

[0036] In another optional implementation, the step of decoding the difference-coded data contained in the data packet according to the number of encoding bits to obtain the difference value corresponding to the difference-coded data specifically includes:

[0037] According to the encoding mapping table corresponding to the encoding bits of the target data packet format, the quantization value corresponding to the encoding value that matches the difference encoding data is queried, and the difference value corresponding to the difference encoding data is determined according to the quantization value;

[0038] The encoding mapping table is used to store the mapping relationship between each encoding value corresponding to the encoding bit length and the quantization value of the difference.

[0039] In another alternative implementation, after reconstructing the original audio data based on the upsampling sampling results, the method further includes: driving a target vibration device in the cloud game to vibrate based on the reconstructed original audio data; wherein the cloud game is implemented based on streaming.

[0040] According to another aspect of the present disclosure, a method for transmitting audio data is provided, the method comprising:

[0041] The audio transmitter performs downsampling on the original audio data to obtain the current sample value, calculates the difference between the current sample value and the adjacent sample values, selects a target data packet format corresponding to the magnitude of the difference from at least two candidate data packet formats, encodes the difference into difference-coded data that matches the encoding bit length of the target data packet format, and encapsulates the difference-coded data in a data packet that matches the target data packet format.

[0042] The audio receiver acquires data packets from the audio transmitter, determines the target data packet format corresponding to the data packet from at least two candidate data packet formats, and the encoding bit length of the target data packet format; decodes the differential encoded data contained in the data packet according to the encoding bit length to obtain the difference value corresponding to the differential encoded data; calculates the sum between the difference value and adjacent decoded values ​​to obtain the current decoded value; performs upsampling processing on the current decoded value, and reconstructs the original audio data based on the upsampling sampling result.

[0043] According to another aspect of the present disclosure, an audio data transmission apparatus is provided, suitable for an audio transmitting end, the apparatus comprising:

[0044] The downsampling module is suitable for performing downsampling processing on the original audio data, and for the current sample value obtained after downsampling processing, it calculates the difference between the current sample value and the adjacent sample values;

[0045] The differential module is adapted to select a target data packet format from at least two candidate data packet formats that corresponds to the numerical value of the difference, and to encode the difference into differential encoded data that matches the encoding bit length of the target data packet format;

[0046] The encoding module is adapted to encapsulate the differential encoded data in a data packet that matches the target data packet format for transmission to the audio receiver.

[0047] In one alternative implementation, the differential module is specifically adapted to:

[0048] The difference is compared with a preset interval threshold, and the target data packet format is selected from at least two candidate data packet formats based on the comparison result.

[0049] Wherein, if the difference is greater than the interval threshold, the first data packet format among at least two candidate data packet formats is determined as the target data packet format; if the difference is not greater than the interval threshold, the second data packet format among at least two candidate data packet formats is determined as the target data packet format; wherein, the encoding bit length of the second data packet format is less than the encoding bit length of the first data packet format;

[0050] The target data packet format includes a bit width identifier, which indicates the number of bits in the target data packet format encoding.

[0051] In another alternative implementation, the audio data includes: first audio data corresponding to a first channel and second audio data corresponding to a second channel; then the downsampling module is specifically adapted to: calculate a first difference corresponding to the first audio data and a second difference corresponding to the second audio data;

[0052] The differential module is specifically suitable for:

[0053] Based on the comparison result between the first difference and the second difference, the target data packet format is selected from at least two candidate data packet formats;

[0054] The target data packet format includes a channel identifier bit, which is used to indicate whether the first difference and the second difference are equal.

[0055] In another alternative implementation, the differential module is specifically adapted to:

[0056] Obtain the quantized value of the difference, and according to the encoding mapping table corresponding to the encoding bits of the target data packet format, look up the encoding value corresponding to the quantized value, and encode the quantized value into difference encoded data that matches the encoding value;

[0057] The encoding mapping table is used to store the mapping relationship between each encoding value corresponding to the encoding bit length and the quantization value of the difference.

[0058] In another alternative implementation, the number of quantized values ​​of the difference is greater than the number of encoded values ​​corresponding to the number of encoded bits, and the number of quantized values ​​corresponding to each encoded value is determined according to the frequency of occurrence of the quantized value in the historical audio.

[0059] If the frequency of the quantization value in the historical audio is greater than the preset frequency threshold, then the quantization value and the encoded value correspond one-to-one.

[0060] If the frequency of a quantization value in historical audio does not exceed a preset frequency threshold, then multiple quantization values ​​correspond to the same encoded value.

[0061] In another alternative implementation, the adjacent sample value is the previous sample value of the current sample value; the difference module is specifically adapted to:

[0062] If there is an error between the queried encoded value and the quantized value of the difference, the error value between the queried encoded value and the quantized value of the difference is stored; wherein, the error value is used for error compensation processing for the next sampled value of the current sampled value.

[0063] In another alternative implementation, the raw audio data is used to drive a target vibration device in the cloud gaming.

[0064] The downsampling module is specifically adapted to: perform downsampling processing on the original audio data according to the downsampling frequency; wherein the downsampling frequency is at least twice the highest vibration frequency of the target vibration device.

[0065] According to another aspect of the present disclosure, an audio data transmission apparatus is provided, suitable for an audio receiving end, the apparatus comprising:

[0066] The acquisition module is adapted to acquire data packets from an audio transmitter, determine a target data packet format corresponding to the data packet from at least two candidate data packet formats, and the encoding bit length of the target data packet format;

[0067] The decoding module is adapted to decode the difference-coded data contained in the data packet according to the number of encoding bits, so as to obtain the difference value corresponding to the difference-coded data;

[0068] The calculation module is adapted to calculate the summation result between the difference and the adjacent decoded values ​​to obtain the current decoded value;

[0069] An upsampling module is adapted to perform upsampling processing on the current decoded value and reconstruct the original audio data based on the upsampling sampling result.

[0070] In one alternative implementation, the acquisition module is specifically adapted to:

[0071] Obtain the value of the bit identifier bit contained in the data packet, determine the target data packet format corresponding to the data packet and the encoding bit length of the target data packet format based on the value of the bit identifier bit; and / or,

[0072] Obtain the value of the channel identifier bit contained in the data packet, and determine the target data packet format corresponding to the data packet and the encoding bit length of the target data packet format based on the value of the channel identifier bit.

[0073] In yet another alternative implementation, the decoding module is specifically adapted to:

[0074] According to the encoding mapping table corresponding to the encoding bits of the target data packet format, the quantization value corresponding to the encoding value that matches the difference encoding data is queried, and the difference value corresponding to the difference encoding data is determined according to the quantization value;

[0075] The encoding mapping table is used to store the mapping relationship between each encoding value corresponding to the encoding bit length and the quantization value of the difference.

[0076] In another alternative implementation, the upsampling module is also adapted to: drive a target vibration device in a cloud game to vibrate based on the reconstructed original audio data; wherein the cloud game is implemented based on streaming.

[0077] According to another aspect of the present disclosure, an audio data transmission system is provided, including: an audio data transmission device suitable for an audio transmitting end, and an audio data transmission device suitable for an audio receiving end.

[0078] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;

[0079] The memory is used to store at least one executable instruction that causes the processor to perform the audio data transmission method described above.

[0080] According to another aspect of this disclosure, a computer storage medium is provided, the storage medium storing at least one executable instruction that causes a processor to perform the audio data transmission method described above.

[0081] In the audio data transmission method provided in this disclosure, firstly, downsampling processing is performed on the original audio data; then, the difference between the current sampled value obtained after downsampling and the adjacent sampled values ​​is calculated; finally, a target data packet format corresponding to the numerical value of the difference is selected from at least two candidate data packet formats, the difference is encoded into difference-coded data matching the encoding bit length of the target data packet format, and the difference-coded data is encapsulated in a data packet matching the target data packet format for transmission. Therefore, in this method, downsampling can significantly reduce the data volume, and the downsampling method is particularly suitable for processing signals with many low-frequency components. Moreover, in signals with many low-frequency components, the difference between signals at adjacent time points is often small (usually the difference is much smaller than the signal amplitude). Therefore, compared to directly encoding the original signal value, encoding the difference can further reduce the data volume and encoding bit length. Furthermore, this method pre-sets multiple data packet formats to dynamically select the appropriate format based on the difference between the current sample value and its adjacent samples. Since different data packet formats correspond to different bit widths, the bit width of the data packet can be dynamically adjusted according to the current difference, thereby further reducing the number of bytes in the data packet and lowering transmission latency. In summary, this method, through a combination of downsampling, differential encoding, and dynamic data packet format selection, can significantly reduce the amount of audio data and shorten transmission latency, making it particularly suitable for business scenarios with high real-time requirements.

[0082] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, specific embodiments of this disclosure are described below. Attached Figure Description

[0083] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this disclosure. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0084] Figure 1 A flowchart illustrating an audio data transmission method according to an embodiment of this disclosure is shown;

[0085] Figure 2 A flowchart of an audio data transmission method according to yet another embodiment of this disclosure is shown;

[0086] Figure 3 A flowchart of an audio data transmission apparatus according to yet another embodiment of the present disclosure is shown;

[0087] Figure 4 A schematic diagram of an audio data transmission apparatus according to another embodiment of the present disclosure is shown;

[0088] Figure 5 A schematic diagram of the structure of an electronic device provided in Embodiment 5 of this disclosure is shown. Detailed Implementation

[0089] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0090] Figure 1 A flowchart illustrating an audio data transmission method according to an embodiment of this disclosure is shown. This embodiment is primarily applicable to the audio transmitting end. Figure 1 As shown, the method includes:

[0091] Step S110: Perform downsampling processing on the original audio data to obtain the current sample value, and calculate the difference between the current sample value and the adjacent sample values.

[0092] The original audio data is the data that needs to be sent from the audio transmitter to the audio receiver. In this embodiment, the audio transmitter can be a cloud server or other network device, and the audio receiver can be a mobile terminal, tablet computer, or other network terminal. This invention does not limit the specific application scenario.

[0093] Downsampling is used to reduce the signal sampling rate. Through downsampling, the original audio data can be sampled at a lower sampling frequency, thereby significantly reducing the data volume. In one optional implementation, downsampling is performed on the original audio data according to the downsampling frequency; the downsampling frequency can be flexibly determined based on the signal characteristics of the original audio data and / or the actual application scenario. According to the Nyquist-Shannon sampling theorem, the sampling rate of a digital signal is greater than twice that of an analog signal. In other words, if the signal bandwidth is less than the Nyquist frequency (i.e., half the sampling frequency), the original signal can be reconstructed from discrete sampling points. Based on the sampling theorem, the downsampling frequency in this embodiment can be set according to the highest frequency of the original audio signal.

[0094] Furthermore, in audio-driven vibration applications, the raw audio data is used to drive the target vibration device. For example, it can be used to drive the target vibration device in cloud gaming. Accordingly, the downsampling frequency can be set to at least twice the highest vibration frequency of the target vibration device. Since vibration signals are low-frequency signals, the highest vibration frequency of the target vibration device is usually low. Taking advantage of this characteristic, the downsampling frequency can be significantly reduced without affecting the driving accuracy of the target vibration device.

[0095] By performing downsampling on the original audio data, multiple sample values ​​corresponding to the sampling order can be obtained. In implementing this invention, the inventors discovered that for low-frequency signals, the difference between adjacent sample values ​​is usually much smaller than the signal amplitude. Utilizing this characteristic, in this step, the difference between adjacent sample values ​​can be further calculated, and the amount of data can be reduced by transmitting the difference.

[0096] Accordingly, for the current sampled value obtained after downsampling, the difference between the current sampled value and its adjacent sampled values ​​is calculated. The adjacent sampled value is usually the previous sampled value. For example, assuming the current sampled value is the i-th sampled value, the adjacent sampled value could be the (i-1)-th sampled value, where i is a natural number greater than or equal to 1. Furthermore, if the current sampled value is the 1st sampled value, the value of the 0th sampled value (i.e., the initial sampling value) can be assumed to be a preset value (such as 0 or another value).

[0097] Step S120: Select a target data packet format from at least two candidate data packet formats that corresponds to the numerical value of the difference, and encode the difference into difference-coded data that matches the number of bits of the target data packet format.

[0098] In this embodiment, two or more candidate data packet formats are pre-defined. The candidate data packet format defines at least one of the following: the number of bytes contained in the data packet, the purpose and meaning of each bit in each byte, and the number of bits corresponding to the data packet's encoding. For example, some data packet formats may contain 2 bytes, while others may contain 4 bytes; the different byte counts result in different data volume and transmission delays. Furthermore, in some data packet formats, the first bit is defined as a metadata identifier bit characterizing the data packet format; in others, both the first and second bits are defined as metadata identifier bits characterizing the data packet format. In short, the metadata identifier bits included in the data packet format characterize format information such as the number of bits and encoding method of the data packet, and the specific number and type of metadata identifier bits can be flexibly set by those skilled in the art. For example, in some data packet formats, the first bit of the data packet's encoding is defined as the first bit (e.g., 7 bits); in others, the second bit is defined as the second bit (e.g., 15 bits). By using different encoding bit lengths, difference data with different value ranges can be encoded.

[0099] Therefore, by pre-setting multiple data packet formats, various data packets of different lengths can be flexibly configured to accommodate differences of different sizes. Accordingly, in this step, a target data packet format corresponding to the numerical value of the difference is selected from at least two candidate data packet formats, and the difference is encoded into difference-coded data that matches the encoding bit length of the target data packet format. Matching the encoding bit length of the target data packet format means that the number of bits in the difference-coded data obtained after encoding the difference matches the encoding bit length defined in the target data packet format. Matching means identical or equivalent. For example, the number of bits in the difference-coded data obtained after encoding the difference is the same as the encoding bit length defined in the target data packet format. For instance, if the encoding bit length defined in the target data packet format is 4 bits, then the difference is quantized and encoded into 4-bit difference-coded data.

[0100] Those skilled in the art can flexibly set the selection method for the target data packet format:

[0101] In the first optional implementation, the target data packet format is selected based on the matching relationship between the difference and a pre-set interval threshold. Specifically, the difference is compared with the preset interval threshold, and the target data packet format is selected from at least two candidate data packet formats based on the comparison result: if the difference is greater than the interval threshold, the first data packet format among the at least two candidate data packet formats is determined as the target data packet format; if the difference is not greater than the interval threshold, the second data packet format among the at least two candidate data packet formats is determined as the target data packet format; wherein, the encoding bit length of the second data packet format is less than that of the first data packet format. For example, the interval threshold can be set to 2 to the power of 7, the encoding bit length of the second data packet format can be 7 bits, and the encoding bit length of the first data packet format can be 15 bits.

[0102] Optionally, there can be multiple interval thresholds, and correspondingly, multiple candidate data packet formats. For example, with three interval thresholds, four numerical intervals can be divided using the three thresholds, and correspondingly, there can be four candidate data packet formats, each corresponding to one numerical interval.

[0103] In addition, to facilitate decoding, a bit depth flag can be set in the target data packet format. This bit depth flag indicates the number of bits encoded in the target data packet format. For example, if there are two candidate data packet formats, only one bit depth flag is needed; if there are four candidate data packet formats, two bit depth flags are required. This bit depth flag is the metadata flag, used to describe the number of bits encoded in the data packet for faster decoding.

[0104] In the second optional implementation, the audio data is stereo audio data. During the development of this invention, the inventors discovered that in many applications, such as games, the two channels often transmit identical data. Therefore, by identifying whether the data currently transmitted by the two channels is the same, identical data can be merged, reducing the data volume. Accordingly, if it is determined that the data currently transmitted by the two channels is the same, the audio data can be treated as mono audio data, and only the difference between the two channels needs to be encoded; only if it is determined that the data currently transmitted by the two channels is different, is the difference between the two channels encoded simultaneously. This method can significantly reduce the amount of transmitted data. Therefore, in this implementation, the audio data includes: first audio data corresponding to the first channel and second audio data corresponding to the second channel. Accordingly, when calculating the difference between the current sample value and adjacent sample values, it is necessary to calculate the first difference corresponding to the first audio data and the second difference corresponding to the second audio data separately. In this method, each sampling requires simultaneous sampling of both channels. Therefore, the first difference represents the difference between the current sampled value and its adjacent sampled value in the first audio data, and the second difference represents the difference between the current sampled value and its adjacent sampled value in the second audio data. When selecting a target data packet format corresponding to the magnitude of the difference from at least two candidate data packet formats, the target data packet format is selected based on the comparison result between the first and second differences. For example, the difference between the first and second differences can be calculated. If the difference is 0 or less than a preset difference threshold, the first and second differences are considered to be the same or approximately the same. In this case, the stereo audio data can be treated as mono audio data, and the difference for one channel is encoded only in the subsequent encoding process. That is, the data volume is reduced by merging the identical data of the two channels. Conversely, if the difference between the first and second differences is greater than the preset difference threshold, the first and second differences are considered to be different. In this case, the difference for each channel needs to be encoded separately to ensure that the stereo sound effect can be reproduced at the receiving end. Therefore, this method allows for flexible selection of different data packet formats based on whether the differences between the current sampled values ​​of the two channels are equal. For example, if the differences between the two channels are equal, only the difference of one channel needs to be encoded, in which case a shorter data packet format can be selected to reduce the amount of data transmitted. If the differences between the two channels are unequal, both channels need to be encoded simultaneously, in which case a longer data packet format can be selected to accurately represent the difference between each channel.

[0105] Furthermore, to facilitate decoding, in the second implementation, a channel identifier bit can be further set in the target data packet format. This channel identifier bit is used to indicate whether the first difference and the second difference are equal. That is, based on different values ​​of the channel identifier bit, it can be determined whether the data packet encapsulates encoded data of one difference or encoded data of two differences, thus facilitating the decoding of the data packet. The channel identifier bit is also a metadata identifier bit, used to describe the number of channels corresponding to the data packet, so as to facilitate fast decoding.

[0106] The two implementation methods described above can be used in one or both. This application does not limit the specific implementation details.

[0107] When both of the above implementation methods are used simultaneously, different priorities can be set for the channel identifier and the bit depth identifier. For example, in one implementation method, the channel identifier has a higher priority than the bit depth identifier, and whether or not to set the bit depth identifier depends on the value of the channel identifier.

[0108] For example, if the channel identifier value indicates that only the difference of a single channel is being transmitted, there is no need to set a bit width identifier (i.e., only one metadata identifier needs to be set, thus saving storage space). In this case, regardless of the specific value of the difference, it is transmitted using a data packet format with the first bit width as the encoding bit width. For example, if the first bit width is 15 bits, then the data packet only needs to occupy two bytes.

[0109] For example, when the channel identifier value represents the difference between two channels that need to be transmitted simultaneously, a bit width identifier needs to be further set (in this case, two metadata identifier bits need to be set). Correspondingly, the bit width identifier value represents the encoding bit width of the data packet. The value of the bit width identifier is determined based on the magnitude of the difference. For instance, if the difference is less than 2 to the power of 7, a data packet format with a 7-bit encoding width is selected. In this case, the difference encoded data of the two channels occupies a total of 14 bits, plus the two metadata identifier bits, the entire data packet occupies 2 bytes. If the difference is not less than 2 to the power of 7, a data packet format with a 15-bit encoding width is selected. In this case, the difference encoded data of the two channels occupies a total of 30 bits, plus the two metadata identifier bits, the entire data packet occupies 4 bytes. Therefore, this method allows for flexible adjustment of the data packet length based on the current channel configuration.

[0110] Step S130: Encapsulate the differential encoded data in a data packet that matches the target data packet format for transmission to the audio receiver.

[0111] Since the specific bit length of the differential encoded data is set according to the target data packet format, the differential encoded data can be encapsulated in a data packet that matches the target data packet format and transmitted to the audio receiver to achieve compressed transmission of the original audio data.

[0112] Matching the target data packet format means that the format of the encapsulated data packet is the same as that of the target data packet. The target data packet format defines the physical meaning of each byte and bit within the data packet. For example, suppose the target data packet format A defines a data packet containing two bytes (16 bits), with the first bit being a metadata identifier indicating the data packet format type, and the following 15 bits being data bits used to write differential encoded data. Since the number of bits in the differential encoded data is the same as the number of bits defined in the target data packet format, in this step, the differential encoded data can be directly written into the data portion of the data packet in format A; and the specific value of the metadata identifier in the data packet of format A can be set according to the metadata value corresponding to format A, thus obtaining the complete encapsulated data packet. Different formats correspond to different metadata values, allowing for rapid determination of the data packet format based on the metadata value. Therefore, this method significantly reduces the data volume through downsampling, and downsampling is particularly suitable for processing signals with many low-frequency components. Furthermore, in signals with a high proportion of low-frequency components, the difference between signals at adjacent time points is often small (typically much smaller than the signal amplitude). Therefore, compared to directly encoding the original signal value, encoding the difference can further reduce the data volume and the number of bits required for encoding. This method also pre-sets multiple data packet formats to dynamically select the appropriate format based on the difference between the current sample value and adjacent sample values. Since different data packet formats correspond to different bit widths, the bit width of the data packet can be dynamically adjusted according to the current difference, thereby further reducing the number of bytes in the data packet and lowering transmission latency. In summary, this method, through the combination of downsampling, difference encoding, and dynamic selection of data packet formats, can significantly reduce the amount of audio data and shorten transmission latency, making it particularly suitable for business scenarios with high real-time requirements.

[0113] In addition, those skilled in the art can make various modifications and variations to the above embodiments:

[0114] In one optional implementation, quantization encoding is used when encoding the difference into difference-coded data that matches the number of bits in the target data packet format. Specifically, the quantized value of the difference is obtained, converted into the corresponding encoded value, and thus encoded into difference-coded data that matches the encoded value. Furthermore, when there are many quantization levels and few bits, inconsistencies may arise between the encoded and quantized values; that is, there is an error between the quantized and encoded values. In this case, the error value corresponding to the current sample value can be stored, and when processing the next sample value, error compensation is performed on the next sample value based on the stored error value. This error compensation method avoids error accumulation and improves data accuracy.

[0115] In one alternative implementation, to enable a limited number of encoding bits to represent a larger range of difference quantization, an adaptive differential coding method is used to encode the difference. In this method, a pre-established encoding mapping table maps quantized values ​​to encoded values, allowing one encoded value to correspond to multiple quantized values. Accordingly, when encoding the difference into difference-coded data that matches the encoding bit length of the target data packet format, the quantized value of the difference is obtained. Based on the encoding mapping table corresponding to the encoding bit length of the target data packet format, the corresponding encoded value is looked up, and the quantized value is encoded into difference-coded data that matches the encoded value. This encoding mapping table stores the mapping relationship between each encoded value corresponding to a certain number of encoding bits and the quantized value of the difference.

[0116] Typically, the number of quantized values ​​for a difference is greater than the number of encoded values ​​corresponding to the number of encoded bits. Therefore, the purpose of the encoding map is to represent a larger quantization range using a limited number of encoded bits through mapping relationships. To achieve this, the correspondence between encoded values ​​and quantized values ​​in the encoding map is not always one-to-one; there will inevitably be cases where the same encoded value corresponds to multiple quantized values. To minimize errors and improve data accuracy, in one optional implementation, the number of quantized values ​​corresponding to each encoded value is determined by the frequency of the quantized value in historical audio. For example, if the frequency of a quantized value in historical audio is greater than a preset frequency threshold, then there is a one-to-one correspondence between quantized values ​​and encoded values; if the frequency of a quantized value in historical audio is not greater than the preset frequency threshold, then multiple quantized values ​​correspond to the same encoded value.

[0117] For example, in audio-driven vibration applications, since the vibration signal is a low-frequency signal, it is necessary to maximize the accuracy of the low-frequency audio. Therefore, when the quantization value is less than the preset frequency value, the quantization value and the encoded value are made to correspond one-to-one to achieve accurate restoration of the low-frequency difference; while when the quantization value is not less than the preset frequency value, multiple quantization values ​​are made to correspond to the same encoded value, thereby achieving the effect of reducing the number of bits of encoding.

[0118] In gaming applications, the types and quantities of audio are typically limited. Therefore, by statistically analyzing the frequency of occurrence of each frequency component in the historical audio data of the application, the correspondence between quantization values ​​and encoded values ​​can be rationally set based on the statistical results. For example, in gaming applications, low-frequency components occur more frequently; therefore, the quantization values ​​corresponding to each low-frequency component correspond one-to-one with the encoded values. Since high-frequency components occur less frequently, a range of quantization values ​​corresponding to a segment of high-frequency components can all correspond to the same encoded value. The endpoints of the quantization value range can be set based on the values ​​corresponding to the statistically obtained high-frequency components.

[0119] In another optional implementation, considering that adaptive differential coding is more prone to coding errors, especially when the quantized and encoded values ​​of high-frequency components are not in a one-to-one correspondence, to avoid error accumulation, when querying the encoded value corresponding to the quantized value of the difference and encoding the quantized value of the difference into difference-coded data that matches the encoded value, it is further determined whether there is an error between the encoded value and the quantized value of the difference. If the determination result is yes, error compensation processing is performed. Specifically, the adjacent sample value is usually the previous sample value of the current sample value. If there is an error between the queried encoded value and the quantized value of the difference, the error value between the queried encoded value and the quantized value of the difference is stored. This error value is used for error compensation processing for the next sample value of the current sample value. For example, suppose the actual difference between the current sample value and its previous sample value is 999, but the maximum value of the encoded value is 500. Therefore, an error of 499 will be generated in the difference encoding process of the current sample value. Therefore, after receiving the next sample value, the error is compensated for the next sample value. For example, assuming the actual difference between the next sampled value and the current sampled value is 0, the sum of the actual difference (0) between the next sampled value and the current sampled value and the error value (499) is calculated, and the sum is used as the encoding difference corresponding to the next sampled value, thereby achieving the effect of error compensation.

[0120] Figure 2 A flowchart illustrating a method for transmitting audio data according to another embodiment of this disclosure is shown. This embodiment is primarily applicable to an audio receiving end. Figure 2 As shown, the method includes:

[0121] Step S210: Obtain data packets from the audio transmitter, determine the target data packet format corresponding to the data packets from at least two candidate data packet formats, and the encoding bit length of the target data packet format.

[0122] The data packets from the audio transmitter are the same data packets transmitted in step S130 of the previous embodiment. Correspondingly, the specific meanings of the at least two candidate data packet formats can be found in the descriptions of the relevant parts of the previous embodiment, and will not be repeated here.

[0123] In one optional implementation, the target data packet format corresponding to the data packet can be determined based on the metadata identifier bits set in the data packet. These metadata identifier bits include the bit depth identifier bits and / or channel identifier bits mentioned above. Accordingly, the values ​​of the bit depth identifier bits contained in the data packet can be obtained, and the target data packet format corresponding to the data packet, as well as the encoding bit depth of the target data packet format, can be determined based on these values. Alternatively, the values ​​of the channel identifier bits contained in the data packet can also be obtained, and the target data packet format corresponding to the data packet, as well as the encoding bit depth of the target data packet format, can be determined based on these values.

[0124] Step S220: Decode the difference-coded data contained in the data packet according to the number of encoding bits to obtain the difference value corresponding to the difference-coded data.

[0125] Specifically, the decoding process is the reverse of the encoding process mentioned in the previous embodiment, and will not be repeated here.

[0126] In one optional implementation, when decoding the difference-coded data contained in the data packet according to the number of encoding bits to obtain the difference value corresponding to the difference-coded data, the quantization value corresponding to the encoding value matching the difference-coded data is looked up according to the encoding mapping table corresponding to the number of encoding bits of the target data packet format. The difference value corresponding to the difference-coded data is then determined based on the quantization value. The encoding mapping table stores the mapping relationship between each encoding value corresponding to the number of encoding bits and the quantization value of the difference. The specific setting of the encoding mapping table can be referred to the description in the previous embodiment, and will not be repeated here.

[0127] Step S230: Calculate the sum of the difference and the adjacent decoded values ​​to obtain the current decoded value.

[0128] Here, the adjacent decoded value refers to the decoded value corresponding to the previous time unit, and the current decoded value refers to the decoded value corresponding to the current time unit. Therefore, similar to the encoding process, the decoded value corresponding to the current time unit is determined by the sum of the difference obtained in the current time unit and the decoded value corresponding to the previous time unit.

[0129] Alternatively, a similar error compensation method to the encoding process can be used during decoding to compensate for errors between adjacent decoded values, thereby improving the accuracy of decoding.

[0130] Step S240: Perform upsampling on the current decoded value and reconstruct the original audio data based on the upsampling results.

[0131] Upsampling is the inverse of downsampling mentioned in the previous embodiment; the upsampling factor should be the same as the downsampling factor. Through upsampling, the original audio data can be reconstructed.

[0132] In practice, different methods can be used for upsampling. Since the sensitivity of the vibration motor is limited, nearest neighbor interpolation, linear interpolation, Lagrange interpolation or other interpolation algorithms can be flexibly selected. The overtones introduced by the interpolation operation have minimal impact on the actual vibration experience.

[0133] Therefore, it can be seen that by flexibly setting data packet format, combining multiple processing methods such as differential decoding and upsampling, the original audio signal can be restored at the receiving end while reducing the amount of audio data and reducing transmission delay.

[0134] Optionally, this audio transmission method can be applied to audio-driven vibration scenarios in gaming applications. Accordingly, after reconstructing the original audio data based on the upsampled results, the reconstructed original audio data is used to drive the target vibration device in the cloud game. The target vibration device can be various devices such as game controllers. Furthermore, cloud gaming can be implemented using streaming. In streaming technology, users do not need to download the complete audio; they can receive the audio stream in real time, and the game runs directly in the cloud, significantly reducing the device requirements and resource consumption of the local client. Therefore, in streaming cloud gaming scenarios, there is a greater sensitivity to transmission latency and a higher requirement for real-time performance. The compression transmission method in this embodiment can significantly reduce the data volume and improve transmission efficiency. Moreover, in audio-driven vibration scenarios, since the vibration signal is a low-frequency signal, high-frequency components can be significantly filtered out through downsampling without affecting the vibration signal.

[0135] Another embodiment of this application provides a method for transmitting audio data, specifically including the following steps:

[0136] Step 1: The audio transmitter performs downsampling on the original audio data to obtain the current sample value, calculates the difference between the current sample value and the adjacent sample values; selects a target data packet format from at least two candidate data packet formats that corresponds to the magnitude of the difference, encodes the difference into difference-encoded data that matches the encoding bit length of the target data packet format; and encapsulates the difference-encoded data in a data packet that matches the target data packet format.

[0137] Step 2: The audio receiver acquires data packets from the audio transmitter, determines the target data packet format corresponding to the data packet from at least two candidate data packet formats, and the encoding bit length of the target data packet format; decodes the difference-coded data contained in the data packet according to the encoding bit length to obtain the difference value corresponding to the difference-coded data; calculates the sum of the difference value and the adjacent decoded values ​​to obtain the current decoded value; performs upsampling processing on the current decoded value, and reconstructs the original audio data based on the upsampling sampling result.

[0138] To facilitate understanding, the audio data transmission method in this application is described in detail below using a specific example. In this example, the audio data is used to drive a game controller in cloud gaming. Compared to traditional controller vibration driving methods, audio-driven vibration has at least the following advantages: it does not require special modulation of the inherent frequency of the eccentric or linear motor inside the controller; the modulation is moved to the controller's firmware. Furthermore, it can modulate a more delicate and complex vibration feel, thus allowing for better integration with game audio.

[0139] However, the audio-driven vibration process typically introduces dual-channel audio data, usually at a frequency of 48kHz, requiring 16 bits. This results in significant bandwidth consumption, thus greatly impacting cloud gaming applications based on streaming technology. Traditional compression algorithms (such as MP3 or Opus) rely on Fast Fourier Transform (FFT) for low-frequency processing, requiring a larger time window and making it difficult to achieve low-latency compression. Furthermore, Fourier Transform introduces substantial computational complexity, making it unsuitable for the practical needs of low-latency streaming scenarios in modern computer CPU architectures.

[0140] To address the aforementioned issues, in this example, the inventors, by analyzing the characteristics of audio data in a controller vibration scenario, proposed a low-latency, low-loss, and fast lossy audio vibration compression algorithm. Experiments show that this algorithm can achieve a compression rate of approximately 1%-3% without considering the absence of silent data transmission, reducing the data volume by 30 to 100 times, thus effectively meeting the high real-time requirements of streaming cloud gaming scenarios.

[0141] In this example, the audio transmitter mainly includes a downsampling module, a differential module, and an encoding module. According to the Nyquist-Shannon sampling theorem, the sampling rate of a digital signal is more than twice that of an analog signal. Since vibration signals are low-frequency signals, the sampling frequency can be significantly reduced in this example to save space.

[0142] The downsampling module is used to perform downsampling processing. Specifically, based on experience, a target frequency of the downsampling module can be selected as m times (m>=2,~5) of the highest vibration frequency of the target vibration device (e.g., a game controller), thereby performing downsampling on the source audio (i.e., the original audio data). Typically, the value of m is greater than or equal to 2 and less than or equal to 5, where 5 can be replaced by other upper limit thresholds, which are not limited in this invention.

[0143] The differential module is used to perform differential processing on the current sampled value obtained from downsampling. Because the vibration frequency is low, the loudness variation is not as pronounced as in music. Utilizing this characteristic, the differential module calculates the difference between the sampled value at each downsampled time point and the sampled value at the previous time point, stores and transmits this difference. Since the difference fluctuates less than the original value, it can be transmitted with fewer bits. Typically, 7-15 bits can be used losslessly to encode the difference, significantly reducing the data volume compared to the high quantization bit depth of the audio itself (usually 16-24 bits). Furthermore, when the quantized value of the difference exceeds the range that 15 bits can represent, the error can be accumulated and compensated for in the next time slice to reduce audio quality loss.

[0144] In this example, the differential module can also be replaced with an adaptive differential module. In the adaptive differential module, a pre-set sequence is used to optimize the effect when loudness changes dramatically. This pre-set sequence is similar to the encoding mapping table mentioned above, and can pre-define the mapping relationship between the encoded values ​​and quantized values ​​in the sequence based on the statistical results of each frequency component, according to the specific transmitted audio.

[0145] The encoding module encodes the difference obtained by the differential module. During the encoding process, a suitable data packet format is dynamically selected based on factors such as the magnitude of the difference at the current time point. Accordingly, in this example, the following transmission protocol is predefined: a 2-bit metadata identifier is used at the beginning of the data packet to identify the data packet format type.

[0146] In this method, the first flag bit is the channel flag bit. When the first channel flag bit is not set (i.e., not set to 1), it indicates that the difference results of the two channels are the same, and only the difference of one channel needs to be encoded. In this method, the remaining 15 bits in the data packet are used to transmit the encoded difference data. Since only the difference of one channel needs to be encoded, the data packet length can be set to two bytes. Because a two-byte data packet length can satisfy various ranges of difference encoding, in mono mode, the bit depth flag bit does not need to be set, and a two-byte data packet can be used uniformly.

[0147] When the first channel identifier bit is set (i.e., set to 1), it indicates that the difference between the two channels is different, and the difference between the two channels needs to be encoded simultaneously. Considering that the amount of data for the difference between two channels is twice that of the difference between one channel, a second identifier bit, namely the bit depth identifier bit, is further set in the dual-channel mode to maximize storage space savings. The value of this bit depth identifier bit is determined according to the magnitude of the difference. For example, when the magnitude of the difference is less than 2 to the power of 7 (the bit depth identifier bit can be set to 0), the difference between each channel is defined to be encoded using 7 bits, thus using 2 7-bit data to encode the difference between the left and right channels, and then packaging the difference data between the two channels into 2 bytes (the data packet contains 14 encoded bits and two metadata identifier bits). For example, when the difference is not less than 2 to the power of 7 (the bit depth flag can be set to 1), the difference for each channel can be encoded using 15 bits, thus packing the two channel data into 4 bytes (without exceeding the size of the original quantized data). Therefore, the compressed data is always smaller than the original data, and by leveraging experience with vibrating audio, a higher compression ratio is achieved without introducing adaptive streaming compression.

[0148] Therefore, this example demonstrates that, without employing additional adaptive coding algorithms (such as Huffman coding) to improve encoding / decoding speed and reduce computational requirements, it can significantly improve compression ratio, reduce data volume, and decrease transmission latency. This example offers at least the following advantages:

[0149] First, based on the characteristics of audio-driven vibration, a one-bit metadata identifier (i.e., channel identifier) ​​can be used to merge identical data from two channels during compression. Second, based on the characteristics of audio-driven vibration, downsampling and differential coding processes are introduced to reduce the bit depth. Furthermore, various data packet formats with different lengths and bit depths are designed to flexibly adapt to differences in numerical values. This approach allows the data packet transmission delay to be controlled within one audioobuffer (better than Huffman coding), and enables very high-fidelity preservation of low-frequency vibration information (compared to Opus or MP3 algorithms), achieving a high compression rate of approximately 1%-3%.

[0150] Figure 3 A schematic diagram of an audio data transmission apparatus according to another embodiment of the present disclosure is shown. The apparatus is suitable for an audio transmitting end and includes:

[0151] The downsampling module 31 is adapted to perform downsampling processing on the original audio data to obtain the current sample value, and calculate the difference between the current sample value and the adjacent sample value;

[0152] The difference module 32 is adapted to select a target data packet format from at least two candidate data packet formats that corresponds to the numerical value of the difference, and encode the difference into difference-coded data that matches the encoding bit length of the target data packet format;

[0153] The encoding module 33 is adapted to encapsulate the differential encoded data in a data packet that matches the target data packet format for transmission to the audio receiving end.

[0154] In one alternative implementation, the differential module 32 is specifically adapted to:

[0155] The difference is compared with a preset interval threshold, and the target data packet format is selected from at least two candidate data packet formats based on the comparison result.

[0156] Wherein, if the difference is greater than the interval threshold, the first data packet format among at least two candidate data packet formats is determined as the target data packet format; if the difference is not greater than the interval threshold, the second data packet format among at least two candidate data packet formats is determined as the target data packet format; wherein, the encoding bit length of the second data packet format is less than the encoding bit length of the first data packet format;

[0157] The target data packet format includes a bit identifier, which indicates the number of bits in the target data packet format encoding.

[0158] In one optional implementation, the audio data includes: first audio data corresponding to a first channel and second audio data corresponding to a second channel; then the downsampling module is specifically adapted to: calculate a first difference corresponding to the first audio data and a second difference corresponding to the second audio data;

[0159] The differential module 32 is specifically adapted to:

[0160] Based on the comparison result between the first difference and the second difference, the target data packet format is selected from at least two candidate data packet formats;

[0161] The target data packet format includes a channel identifier bit, which is used to indicate whether the first difference and the second difference are equal.

[0162] In one alternative implementation, the differential module 32 is specifically adapted to:

[0163] Obtain the quantized value of the difference, and according to the encoding mapping table corresponding to the encoding bits of the target data packet format, look up the encoding value corresponding to the quantized value, and encode the quantized value into difference encoded data that matches the encoding value;

[0164] The encoding mapping table is used to store the mapping relationship between each encoding value corresponding to the encoding bit length and the quantization value of the difference.

[0165] In one optional implementation, the number of quantized values ​​of the difference is greater than the number of encoded values ​​corresponding to the number of encoded bits, and the number of quantized values ​​corresponding to each encoded value is determined according to the frequency of occurrence of the quantized value in historical audio.

[0166] If the frequency of the quantization value in the historical audio is greater than the preset frequency threshold, then the quantization value and the encoded value correspond one-to-one.

[0167] If the frequency of a quantization value in historical audio does not exceed a preset frequency threshold, then multiple quantization values ​​correspond to the same encoded value.

[0168] In one optional implementation, the adjacent sample value is the previous sample value of the current sample value; the difference module is specifically adapted to:

[0169] If there is an error between the queried encoded value and the quantized value of the difference, the error value between the queried encoded value and the quantized value of the difference is stored; wherein, the error value is used for error compensation processing for the next sampled value of the current sampled value.

[0170] In one alternative implementation, the raw audio data is used to drive a target vibration device in the cloud gaming.

[0171] The downsampling module is specifically adapted to: perform downsampling processing on the original audio data according to the downsampling frequency; wherein the downsampling frequency is at least twice the highest vibration frequency of the target vibration device.

[0172] Figure 4 A schematic diagram of an audio data transmission apparatus according to another embodiment of the present disclosure is shown. The apparatus is suitable for an audio receiving end and includes:

[0173] The acquisition module 41 is adapted to acquire data packets from the audio transmitter, determine a target data packet format corresponding to the data packet from at least two candidate data packet formats, and the encoding bit length of the target data packet format;

[0174] Decoding module 42 is adapted to decode the difference-coded data contained in the data packet according to the number of encoding bits, so as to obtain the difference value corresponding to the difference-coded data;

[0175] Calculation module 43 is adapted to calculate the summation result between the difference and the adjacent decoded values ​​to obtain the current decoded value;

[0176] Upsampling module 44 is adapted to perform upsampling processing on the current decoded value and reconstruct the original audio data based on the upsampling sampling result.

[0177] In one alternative implementation, the acquisition module 41 is specifically adapted to:

[0178] Obtain the value of the bit identifier bit contained in the data packet, determine the target data packet format corresponding to the data packet and the encoding bit length of the target data packet format based on the value of the bit identifier bit; and / or,

[0179] Obtain the value of the channel identifier bit contained in the data packet, and determine the target data packet format corresponding to the data packet and the encoding bit length of the target data packet format based on the value of the channel identifier bit.

[0180] In one alternative implementation, the decoding module 42 is specifically adapted to:

[0181] According to the encoding mapping table corresponding to the encoding bits of the target data packet format, the quantization value corresponding to the encoding value that matches the difference encoding data is queried, and the difference value corresponding to the difference encoding data is determined according to the quantization value;

[0182] The encoding mapping table is used to store the mapping relationship between each encoding value corresponding to the encoding bit length and the quantization value of the difference.

[0183] In one alternative implementation, the upsampling module 44 is further adapted to: drive a target vibration device in a cloud game to vibrate based on the reconstructed original audio data; wherein the cloud game is implemented based on streaming.

[0184] The specific structure and working principle of each of the above modules can be found in the description of the corresponding part of the method embodiment, and will not be repeated here.

[0185] In addition, another embodiment of this application provides an audio data transmission system, including: an audio data transmission device suitable for an audio transmitting end, and an audio data transmission device suitable for an audio receiving end. The specific structure of the audio transmitting end is as follows: Figure 3 As shown, the specific structure of the audio receiver is as follows: Figure 4 As shown. Furthermore, the audio transmitter and the audio receiver can belong to the same business system, or they can belong to different business systems; this application does not limit this.

[0186] Another embodiment of this application provides a non-volatile computer storage medium storing at least one executable instruction that can execute the object loading method in the virtual scene of any of the above method embodiments. Specifically, the executable instruction can be used to cause the processor to perform the corresponding operations in the above method embodiments.

[0187] Figure 5 The diagram shows a structural schematic of an electronic device according to yet another embodiment of the present disclosure. The specific embodiments of the present disclosure do not limit the specific implementation of the electronic device.

[0188] like Figure 5 As shown, the electronic device may include: a processor 502, a communications interface 506, a memory 504, and a communications bus 508.

[0189] in:

[0190] The processor 502, communication interface 506, and memory 504 communicate with each other via communication bus 508.

[0191] Communication interface 506 is used to communicate with other network elements such as clients or other servers.

[0192] The processor 502 is used to execute program 510, specifically to execute the relevant steps in the above-described video image detection method embodiment.

[0193] Specifically, program 510 may include program code that includes computer operation instructions.

[0194] Processor 502 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present disclosure. The electronic device includes one or more processors, which may be processors of the same type, such as one or more CPUs, or processors of different types, such as one or more CPUs and one or more ASICs.

[0195] Memory 504 is used to store program 510. Memory 504 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0196] Specifically, program 510 can be used to cause processor 502 to perform the corresponding operations in the above-described video image detection method embodiment.

[0197] The algorithms and displays provided herein are not inherently related to any particular computer, virtual device, or other equipment. Various general-purpose devices can also be used in conjunction with the teachings herein. The required structure for constructing such devices is apparent from the above description. Furthermore, this disclosure is not directed to any particular programming language. It should be understood that the contents of this disclosure described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of this disclosure.

[0198] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this disclosure may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0199] Similarly, it should be understood that, in order to simplify this disclosure and aid in understanding one or more of the various aspects of the disclosure, in the above description of exemplary embodiments of the disclosure, various features of the disclosure are sometimes grouped together in a single embodiment, figure, or description thereof. However, this approach to disclosure should not be construed as reflecting an intention that the claimed disclosure requires more features than are expressly recited in each claim.

[0200] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature serving the same, equivalent, or similar purpose.

[0201] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of this disclosure and form different embodiments. For example, any of the claimed embodiments can be used in any combination.

[0202] The various component embodiments of this disclosure can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the apparatus according to embodiments of this disclosure. This disclosure can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing some or all of the methods described herein. Such an implementation of this disclosure can be stored on a computer-readable medium or can take the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0203] It should be noted that the above embodiments are illustrative of this disclosure and not restrictive, and that alternative embodiments can be devised by those skilled in the art without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This disclosure can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

Claims

1. A method for transmitting audio data, applicable to an audio transmitting end, the method comprising: Perform downsampling on the original audio data to obtain the current sample value, and calculate the difference between the current sample value and the adjacent sample values; Select a target data packet format that corresponds to the numerical value of the difference from at least two candidate data packet formats, and encode the difference into difference-coded data that matches the number of bits of the encoding of the target data packet format; The differential encoded data is encapsulated in a data packet that matches the target data packet format for transmission to the audio receiver.

2. The method according to claim 1, wherein, The step of selecting a target data packet format from at least two candidate data packet formats that corresponds to the magnitude of the difference specifically includes: The difference is compared with a preset interval threshold, and the target data packet format is selected from at least two candidate data packet formats based on the comparison result. Wherein, if the difference is greater than the interval threshold, the first data packet format among at least two candidate data packet formats is determined as the target data packet format; if the difference is not greater than the interval threshold, the second data packet format among at least two candidate data packet formats is determined as the target data packet format; wherein, the encoding bit length of the second data packet format is less than the encoding bit length of the first data packet format; The target data packet format includes a bit identifier, which indicates the number of bits in the target data packet format encoding.

3. The method according to claim 1, wherein, The audio data includes: first audio data corresponding to the first channel and second audio data corresponding to the second channel; then the calculation of the difference between the current sample value and the adjacent sample value specifically includes: calculating the first difference corresponding to the first audio data and the second difference corresponding to the second audio data; The step of selecting a target data packet format from at least two candidate data packet formats that corresponds to the numerical value of the difference specifically includes: Based on the comparison result between the first difference and the second difference, the target data packet format is selected from at least two candidate data packet formats; The target data packet format includes a channel identifier bit, which is used to indicate whether the first difference and the second difference are equal.

4. The method according to any one of claims 1-3, wherein, Encoding the difference into difference-coded data that matches the number of bits in the target data packet format specifically includes: Obtain the quantized value of the difference, and according to the encoding mapping table corresponding to the encoding bits of the target data packet format, look up the encoding value corresponding to the quantized value, and encode the quantized value into difference encoded data that matches the encoding value; The encoding mapping table is used to store the mapping relationship between each encoding value corresponding to the encoding bit length and the quantization value of the difference.

5. The method according to claim 4, wherein, The number of quantized values ​​of the difference is greater than the number of encoded values ​​corresponding to the number of encoded bits, and the number of quantized values ​​corresponding to each encoded value is determined according to the frequency of occurrence of the quantized value in historical audio. If the frequency of the quantization value in the historical audio is greater than the preset frequency threshold, then the quantization value and the encoded value correspond one-to-one. If the frequency of a quantization value in historical audio does not exceed a preset frequency threshold, then multiple quantization values ​​correspond to the same encoded value.

6. The method according to claim 5, wherein, The adjacent sample value is the previous sample value of the current sample value; the step of querying the encoded value corresponding to the quantized value and encoding the quantized value into difference-coded data that matches the encoded value specifically includes: If there is an error between the queried encoded value and the quantized value of the difference, the error value between the queried encoded value and the quantized value of the difference is stored; wherein, the error value is used for error compensation processing for the next sampled value of the current sampled value.

7. The method according to any one of claims 1-3, wherein, The raw audio data is used to drive the target vibration device in the cloud game; The downsampling process on the original audio data specifically includes: performing downsampling on the original audio data according to the downsampling frequency; wherein the downsampling frequency is at least twice the highest vibration frequency of the target vibration device.

8. A method for transmitting audio data, applicable to an audio receiving end, the method comprising: Acquire data packets from an audio transmitter, determine a target data packet format corresponding to the data packet from at least two candidate data packet formats, and the encoding bit length of the target data packet format; The difference-coded data contained in the data packet is decoded according to the specified number of encoding bits to obtain the difference value corresponding to the difference-coded data; The current decoded value is obtained by summing the difference with the adjacent decoded values. Upsampling is performed on the current decoded value, and the original audio data is reconstructed based on the upsampling results.

9. The method according to claim 8, wherein, The step of determining the target data packet format corresponding to the data packet from at least two candidate data packet formats, and the encoding bit length of the target data packet format specifically includes: Obtain the value of the bit identifier bit contained in the data packet, determine the target data packet format corresponding to the data packet and the encoding bit length of the target data packet format based on the value of the bit identifier bit; and / or, Obtain the value of the channel identifier bit contained in the data packet, and determine the target data packet format corresponding to the data packet and the encoding bit length of the target data packet format based on the value of the channel identifier bit.

10. The method according to claim 8, wherein, Decoding the difference-coded data contained in the data packet according to the specified number of encoding bits to obtain the difference value corresponding to the difference-coded data specifically includes: According to the encoding mapping table corresponding to the encoding bits of the target data packet format, the quantization value corresponding to the encoding value that matches the difference encoding data is queried, and the difference value corresponding to the difference encoding data is determined according to the quantization value; The encoding mapping table is used to store the mapping relationship between each encoding value corresponding to the encoding bit length and the quantization value of the difference.

11. The method according to any one of claims 8-10, wherein, After reconstructing the original audio data based on the upsampling sampling results, the method further includes: driving the target vibration device in the cloud game to vibrate based on the reconstructed original audio data; wherein the cloud game is implemented based on streaming.

12. A method for transmitting audio data, the method comprising: The audio transmitter performs downsampling processing on the original audio data to obtain the current sample value, and calculates the difference between the current sample value and the adjacent sample values; Select a target data packet format that corresponds to the numerical value of the difference from at least two candidate data packet formats, and encode the difference into difference-coded data that matches the number of bits of the encoding of the target data packet format; The differential encoded data is then encapsulated in a data packet that matches the format of the target data packet. The audio receiver acquires data packets from the audio transmitter, determines a target data packet format corresponding to the data packet from at least two candidate data packet formats, and the encoding bit length of the target data packet format; The difference-coded data contained in the data packet is decoded according to the specified number of encoding bits to obtain the difference value corresponding to the difference-coded data; The current decoded value is obtained by summing the difference with the adjacent decoded values. Upsampling is performed on the current decoded value, and the original audio data is reconstructed based on the upsampling results.

13. An audio data transmission device, suitable for an audio transmitting end, the device comprising: The downsampling module is adapted to perform downsampling processing on the original audio data to obtain the current sample value, and calculate the difference between the current sample value and the adjacent sample values; The differential module is adapted to select a target data packet format from at least two candidate data packet formats that corresponds to the numerical value of the difference, and to encode the difference into differential encoded data that matches the encoding bit length of the target data packet format; The encoding module is adapted to encapsulate the differential encoded data in a data packet that matches the target data packet format for transmission to the audio receiver.

14. An audio data transmission device, suitable for an audio receiving end, the device comprising: The acquisition module is adapted to acquire data packets from an audio transmitter, determine a target data packet format corresponding to the data packet from at least two candidate data packet formats, and the encoding bit length of the target data packet format; The decoding module is adapted to decode the difference-coded data contained in the data packet according to the number of encoding bits, so as to obtain the difference value corresponding to the difference-coded data; The calculation module is adapted to calculate the summation result between the difference and the adjacent decoded values ​​to obtain the current decoded value; An upsampling module is adapted to perform upsampling processing on the current decoded value and reconstruct the original audio data based on the upsampling sampling result.

15. An audio data transmission system, comprising: The audio data transmission apparatus according to claim 13 and the audio data transmission apparatus according to claim 14.

16. An electronic device comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the audio data transmission method as described in any one of claims 1-12.

17. A computer storage medium storing at least one executable instruction that causes a processor to perform a method for transmitting audio data as claimed in any one of claims 1-12.

Images