A method for processing audio data, corresponding device, equipment and storage medium

Abstract

The embodiment of the present application discloses a method for processing audio data, a corresponding device, an equipment and a storage medium, the method comprising: for each data frame of the audio, processing the data part of the data frame to obtain basic data and extended data; determining the correction data corresponding to the basic data of the data frame according to the basic data of the data frame and the basic data of at least one other data frame; sending the basic data, extended data and correction data corresponding to each data frame of the audio to a data receiving end, so that the data receiving end recovers the damaged basic data according to the corresponding correction data, and then performs a playback operation in combination with the extended data. This is used to improve the probability of data recovery and the quality of the recovered audio data when the audio data is damaged.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method for processing audio data, a corresponding device, equipment and a storage medium. Background Art

[0002] Wireless broadcast terminals, such as one-to-many broadcasts with Bluetooth functions, will inevitably encounter interference and abnormal data packet transmission, which will cause sound interruption. However, in broadcast scenarios, due to the large number of connected devices, retransmission will further increase bandwidth pressure and may cause more receiving devices to detect transmission anomalies. Therefore, repeated retransmissions will cause the entire system to crash.

[0003] In the related art, in order to overcome the problem of interference, the commonly used solutions are as follows: first, discard abnormal frames and fill a frame of silent data. In this way, obvious discontinuity will be heard when an abnormality occurs; second, packet loss compensation simulates the discarded data. Although it can improve the listening experience to a certain extent, key sound information is often discarded; third, the broadcast end always transmits signals at a lower bit rate, which can easily reduce the probability of transmission abnormalities, but the listening quality will be significantly reduced. Summary of the invention

[0004] The embodiments of the present application provide a method for processing audio data, a corresponding device, an electronic device, and a storage medium, so as to improve the probability of data recovery and the quality of the recovered audio data when the audio data is damaged.

[0005] In a first aspect, an embodiment of the present application provides a method for transmitting audio data, including:

[0006] For each data frame of the audio, a data portion of the data frame is processed to obtain basic data and extended data;

[0007] Determining correction data corresponding to the basic data of the data frame according to the basic data of the data frame and basic data of at least one other data frame;

[0008] The basic data, the extended data and the correction data corresponding to each data frame of the audio are sent to a data receiving end, so that the data receiving end recovers the damaged basic data according to the corresponding correction data and then combines it with the extended data for playback operation.

[0009] In the embodiment of the present application, since data damage is usually caused by interference signals, and interference has local characteristics in data, the audio data is processed into basic data and extended data; thus, for the basic data part, the basic data of at least one other data frame is used to generate correction data, and the correction data characterizes the association relationship between the current basic data and the basic data of at least one other data frame, and then the basic data, extended data and correction data corresponding to each data frame of the audio are sent to the data receiving end, so that the receiving end restores the damaged basic data according to the correction data to ensure that the main signal is restored so that it can be combined with the extended data after restoration and played. Therefore, the probability of audio data recovery and the quality of the restored audio data are improved.

[0010] In some exemplary embodiments, determining, according to the basic data of the data frame and basic data of at least one other data frame, the correction data corresponding to the basic data of the data frame includes:

[0011] Correction data corresponding to the basic data of the data frame is determined according to a signal value of the basic data of the data frame and a signal value of basic data of an adjacent frame.

[0012] In the above embodiment, since the signal waveforms or frequency domain coefficients of adjacent frames are close, that is, similar to the data frame, when determining the correction data, the signal values ​​of the basic data of the adjacent frames of the signal value of the basic data are used for calculation. The correction data determined in this way can more accurately restore the damaged data, and the restored basic data has a higher degree of restoration.

[0013] In some exemplary embodiments, determining the correction data corresponding to the basic data of the data frame according to the signal value of the basic data of the data frame and the signal value of the basic data of an adjacent frame includes:

[0014] Determine, according to the signal value of the basic data of the data frame and the signal value of the basic data of the adjacent frames, a correction data set consisting of correction data corresponding to the basic data of each of the data frames; wherein the correction data set is one or more;

[0015] If there are multiple correction data sets, then for each correction data set, determine the amount of data in the correction data set; wherein different correction rules applied to different correction data sets are obtained;

[0016] A correction data set with the smallest data amount is determined as a target correction data set; wherein each correction data in the target data set is correction data corresponding to the basic data of each corresponding data frame.

[0017] In the above embodiment, each correction rule can determine a correction data set, and each correction data in the correction data set with the smallest data volume is determined as the correction data corresponding to the basic data of the current data frame, thereby reducing the data volume of the correction data and lowering the bandwidth burden.

[0018] In a second aspect, an embodiment of the present application provides a method for processing audio data, including:

[0019] Receive basic data, extended data and correction data corresponding to each data frame of the audio; wherein the basic data and the extended data are obtained by processing the data part of each data frame of the audio, and the correction data corresponding to the basic data of the data frame is determined based on the basic data of the data frame and the basic data of at least one other data frame;

[0020] If the received basic data is damaged, restoring the damaged basic data according to the corresponding correction data;

[0021] The restored basic data and the matched extended data are combined to perform a play operation; wherein the matched extended data is extended data obtained by processing the same data frame as the restored basic data.

[0022] In the embodiment of the present application, after receiving the basic data, extended data and correction data corresponding to each data frame, the data receiving end determines the correction data according to the basic data of the data frame and the basic data of at least one other data frame when the received basic data is damaged, so that when the damaged basic data is restored, the main signal is guaranteed to be restored so that it can be combined with the extended data after restoration and played. Therefore, the probability of audio data recovery and the quality of the restored audio data are improved.

[0023] In some exemplary embodiments, the recovering the damaged basic data according to the corresponding correction data includes:

[0024] Restoring the damaged basic data based on the correction data corresponding to the damaged basic data and the adjacent data frames of the data frame to which the damaged basic data belongs;

[0025] The adjacent data frame is at least one other data frame used when determining the correction data.

[0026] In the above embodiment, since the signal waveforms or frequency domain coefficients of adjacent frames are close, that is, similar to the data frame, when determining the correction data, the signal values ​​of the basic data of the adjacent frames of the signal value of the basic data are used for calculation. The correction data determined in this way can more accurately restore the damaged data, and the restored basic data has a higher degree of restoration.

[0027] In some exemplary embodiments, before combining the restored basic data and the matched extended data and playing them, the method further includes:

[0028] If the extended data is damaged, the damaged extended data is restored by applying smoothing or bandwidth expansion;

[0029] The restored extended data is used as the matched extended data.

[0030] In the above embodiment, when the extended data is damaged, the extended data is first restored, and then the restored extended data is combined with the restored basic data. In this way, since the combined data has a high degree of restoration after being restored, a good playback effect can be obtained.

[0031] In a third aspect, an embodiment of the present application provides an audio data processing device, the device comprising:

[0032] A data processing module, for processing the data part of each data frame of the audio to obtain basic data and extended data;

[0033] A correction data determination module, configured to determine correction data corresponding to the basic data of the data frame based on the basic data of the data frame and basic data of at least one other data frame;

[0034] The data sending module is used to send the basic data, the extended data and the correction data corresponding to each data frame of the audio to a data receiving end, so that the data receiving end can restore the damaged basic data according to the corresponding correction data and then combine it with the extended data for playback operation.

[0035] In some exemplary embodiments, the correction data determination module is specifically used to:

[0036] Correction data corresponding to the basic data of the data frame is determined according to a signal value of the basic data of the data frame and a signal value of basic data of an adjacent frame.

[0037] In some exemplary embodiments, the correction data determination module is further configured to:

[0038] Determine, according to the signal value of the basic data of the data frame and the signal value of the basic data of the adjacent frames, a correction data set consisting of correction data corresponding to the basic data of each of the data frames; wherein the correction data set is one or more;

[0039] If there are multiple correction data sets, then for each correction data set, determine the amount of data in the correction data set; wherein different correction rules applied to different correction data sets are obtained;

[0040] A correction data set with the smallest data amount is determined as a target correction data set; wherein each correction data in the target data set is correction data corresponding to the basic data of each corresponding data frame.

[0041] In a fourth aspect, an embodiment of the present application provides an audio data processing device, the device comprising:

[0042] A data receiving module, configured to receive basic data, extended data and correction data corresponding to each data frame of the audio; wherein the basic data and the extended data are obtained by processing the data part of each data frame of the audio, and the correction data corresponding to the basic data of the data frame is determined based on the basic data of the data frame and the basic data of at least one other data frame;

[0043] A data recovery module, used for recovering the damaged basic data according to the corresponding correction data if the received basic data is damaged;

[0044] The playing module is used to combine the restored basic data and the matching extended data to perform a playing operation; wherein the matching extended data is the extended data obtained by processing the same data frame as the restored basic data.

[0045] In some exemplary embodiments, the data recovery module is specifically used to:

[0046] Restoring the damaged basic data based on the correction data corresponding to the damaged basic data and the adjacent data frames of the data frame to which the damaged basic data belongs;

[0047] The adjacent data frame is at least one other data frame used when determining the correction data.

[0048] In some exemplary embodiments, an extended data recovery module is further included, which is used to recover the damaged extended data by applying smoothing processing or bandwidth expansion before the restored basic data and the matched extended data are combined and played, if the extended data is damaged; and use the restored extended data as the matched extended data.

[0049] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above methods when executing the computer program.

[0050] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon, which implement the steps of any of the above methods when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. Obviously, the drawings introduced below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 A schematic diagram of the structure of a wireless broadcasting system provided in one embodiment of the present application;

[0053] Figure 2 A flowchart of an audio data transmission method provided in one embodiment of the present application;

[0054] Figure 3 A flowchart of an audio data transmission method provided in one embodiment of the present application;

[0055] Figure 4 A signaling flow chart of an audio data transmission method provided in one embodiment of the present application;

[0056] Figure 5 A schematic diagram of the structure of an audio data processing device provided in one embodiment of the present application;

[0057] Figure 6 A schematic diagram of the structure of an audio data processing device provided in one embodiment of the present application;

[0058] Figure 7 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0060] For ease of understanding, the terms involved in the embodiments of the present application are explained below:

[0061] (1) Packet loss: refers to the inability of one or more data packets to reach the destination via the Internet; in the embodiment of the present application, it means that the audio data packet cannot reach the receiving terminal from the sending terminal; or part of the audio data sent to the receiving terminal is damaged.

[0062] Any number of elements in the drawings is for illustration and not limitation, and any naming is for distinction only and does not have any limiting meaning.

[0063] In actual practice, wireless broadcast terminals, such as one-to-many broadcasts with Bluetooth functions, will inevitably encounter interference and abnormal data packet transmission, which will cause sound interruption. However, in broadcast scenarios, due to the large number of access devices, retransmission will further increase bandwidth pressure and may cause more receiving devices to detect transmission abnormalities. Therefore, repeated retransmissions will cause the entire system to crash.

[0064] In the related art, in order to overcome the problem of interference, the commonly used solutions are as follows: first, discard abnormal frames and fill a frame of silent data. In this way, obvious discontinuity will be heard when an abnormality occurs; second, packet loss compensation simulates the discarded data. Although it can improve the listening experience to a certain extent, key sound information is often discarded; third, the broadcast end always transmits signals at a lower bit rate, which can reduce the probability of transmission abnormalities, but the listening quality will be significantly reduced.

[0065] To this end, the present application provides an audio data processing method, wherein, for each data frame of the audio, the data part of the data frame is processed to obtain basic data and extended data; based on the basic data of the data frame and the basic data of at least one other data frame, the correction data corresponding to the basic data of the data frame is determined; the basic data, extended data and correction data corresponding to each data frame of the audio are sent to the data receiving end, so that the data receiving end recovers the damaged basic data according to the corresponding correction data, and then performs a playback operation in combination with the extended data. This overcomes the problem of low recovery probability and reduced listening quality caused by damaged audio data in the prior art; and improves the probability of audio data recovery and the quality of the restored audio data.

[0066] After introducing the design ideas of the embodiments of the present application, the following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0067] refer to Figure 1 , which is a structural diagram of a wireless broadcasting system provided in an embodiment of the present application, the system includes a transmitting terminal 11 and at least one receiving terminal 12.

[0068] The data transmitting end 11 and the data receiving end 12 may be connected via Bluetooth. Figure 1Three data receiving ends 12 are used as examples. Specifically, the data sending end 11 processes the data part of each data frame of the audio to obtain basic data and extended data; determines the correction data corresponding to the basic data of the data frame according to the basic data of the data frame and the basic data of at least one other data frame; and sends the basic data, extended data and correction data corresponding to each data frame of the audio to the data receiving end. The data receiving end 12 receives the basic data, extended data and correction data corresponding to each data frame of the audio; if the received basic data is damaged, the damaged basic data is restored according to the corresponding correction data; the restored basic data and the matching extended data are combined for playback; wherein the matching extended data is the extended data obtained by processing the same data frame as the restored basic data.

[0069] In this way, compared with the prior art, taking interference as an example of the cause of data corruption, since interference has local characteristics in time and the amount of correction data is smaller than that of data frames, the data frames and correction data are transmitted separately. As long as there is no abnormality in the correction data and any corresponding data frame, the main signal of the abnormal data frame can be restored with a high probability and no need for retransmission. This not only ensures a higher signal quality in the absence of interference, but also significantly improves the signal restoration quality in the presence of interference. The smaller amount of correction data will not cause pressure on the transmission bandwidth.

[0070] Of course, the method provided in the embodiment of the present application is not limited to Figure 1 The application scenarios shown can also be used in other possible application scenarios, and the embodiments of the present application are not limited thereto. Figure 1 The functions that can be implemented by each device in the application scenario shown will be described in the subsequent method embodiments, and will not be described in detail here.

[0071] To further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation steps shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided in the embodiment of the present application.

[0072] Combine the following Figure 1 The application scenario shown illustrates the technical solution provided by the embodiment of the present application.

[0073] refer to Figure 2 The embodiment of the present application provides a method for processing audio data, which is applied to a data sending end and at least includes the following steps:

[0074] S201. For each data frame of the audio, process the data portion of the data frame to obtain basic data and extended data.

[0075] S202: Determine correction data corresponding to the basic data of the data frame according to the basic data of the data frame and basic data of at least one other data frame.

[0076] S203, sending the basic data, extended data and correction data corresponding to each data frame of the audio to the data receiving end, so that the data receiving end recovers the damaged basic data according to the corresponding correction data and then combines it with the extended data for playing operation.

[0077] In the embodiment of the present application, since data damage is usually caused by interference signals, and interference has local characteristics in data, the audio data is processed to obtain basic data and extended data; thus, for the basic data part, correction data is generated using the basic data of at least one other data frame, and the correction data characterizes the association between the current basic data and the basic data of at least one other data frame, and then the basic data, extended data and correction data corresponding to each data frame of the audio are sent to the data receiving end, so that the receiving end restores the damaged basic data according to the correction data to ensure that the main signal is restored so that it can be combined with the extended data after restoration and played. Therefore, the probability of audio data recovery and the quality of the restored audio data are improved.

[0078] In S201, under the premise of not losing useful information, in order to improve the transmission, storage and processing efficiency, the data is usually compressed before being sent, and the compressed data is transmitted. In order to take into account the broadcast quality, improve the listening experience when the transmission is abnormal, and better resist interference and recover when the data is damaged, before compression, the data part of each data frame is processed into basic data and extended data. The processing operation can be a block operation. In this way, by adding an appropriate amount of auxiliary information, the receiving end can recover the compressed data after the block.

[0079] In actual application, the audio data packets sent by the data transmitter are often compressed with high quality. Therefore, the higher the bit rate and the higher the precision, the smaller the difference between the compressed signal and the original signal. In a specific example, high bit rate (such as 600kb / s) compression can be used.

[0080] Exemplarily, the processing method is explained by taking the block processing of the data part of each data frame as an example. Specifically, the specific implementation method of the block processing is as follows: the audio is divided into a main signal part and a detail signal part. For the convenience of expression, the main signal part is called basic data, and the detail signal part is called extended data. Usually, the low-frequency signal is the basic data, and the high-frequency signal is the extended data. Therefore, in the actual application process, since the adjacent data frames of the low-frequency data are more similar, the characteristics are more obvious. Therefore, in order to make the calculation results of the subsequent correction data more accurate, it can be divided according to frequency, such as data less than a preset frequency threshold is the basic data, and data greater than or equal to the preset frequency threshold is the extended data. In addition, the block process can be to perform block processing on each data frame of the audio, and finally obtain the basic data and extended data of each data frame.

[0081] The data frames of the audio signal are recorded as HP1, HP2...HPN, N represents the number of data frames, the basic data after block is recorded as HPNB, and the extended data after block is recorded as HPNE. Exemplarily, the block process can be implemented in the following two ways:

[0082] In the first method, during the compression process, the low-frequency signal and the high-frequency signal divided according to the preset frequency threshold are compressed separately, so that the compressed basic data is called the block-based basic data, and the compressed extended data is called the block-based extended data. The frequency of the basic data obtained in this way is less than the preset frequency threshold, and the frequency of the extended data obtained is greater than or equal to the preset frequency threshold.

[0083] The second method is to downsample the audio signal according to a preset sampling rate, compress the downsampling result to obtain a signal as basic data, and compress the difference between the signal obtained by decoding the basic data and the audio signal to obtain an extended part.

[0084] Regardless of which of the above methods is used, the audio data after block division is compressed data, and the basic data includes the low-frequency information part, that is, the main part; the extended data includes the high-frequency information part, that is, the detail part.

[0085] In S202, adding an appropriate amount of auxiliary information is helpful for recovering the audio data. The auxiliary information may be correction data. In actual application, the correction data is determined for each data frame, so that when the data frame is damaged and recovered, the corresponding correction data frame can be used to recover it.

[0086] For each data frame, correction data corresponding to the basic data of the data frame is determined according to the basic data of the data frame and basic data of at least one other data frame.

[0087] Specifically, since audio signals are generally continuous and have a continuous feature when being transmitted, the signals of two or more frames before and after are usually similar or close, that is, the signal waveforms are close or the frequency domain coefficients are close, and the similarity between the two is high. This feature is particularly obvious in low-frequency information. Therefore, here, a coefficient that can restore or estimate the original signal is calculated based on certain operation rules, and the coefficient can become a correction coefficient or correction data.

[0088] Exemplarily, at least one other data frame may be an adjacent frame, adjacent multiple frames, or non-adjacent one or more frames of the current data frame. The number and position (adjacent or non-adjacent) of the selected other data frames are different, and the correction rules used may be different. For example, if the selected other data frame is one frame, the difference or XOR method can be used for calculation. If the selected other data frame is multiple frames, the average value of the energy value can be used for calculation.

[0089] The acquisition process of calibration data can be achieved in the following ways:

[0090] Specifically, according to the signal value of the basic data of the data frame and the signal value of the basic data of the adjacent frames, a correction data set consisting of the correction data corresponding to the basic data of each data frame is determined; wherein, there are one or more correction data sets; if there are multiple correction data sets, then for each correction data set, the amount of data in the correction data set is determined; wherein different correction rules are applied to different correction data sets; the correction data set with the smallest data amount is determined as the target correction data set; wherein, each correction data of the target data set is the correction data corresponding to the basic data of each corresponding data frame.

[0091] In a specific example, in order to improve the accuracy of the signal recovery process, the principle that adjacent frames are usually similar is used, and other data frames are selected to illustrate the process of determining the correction data by taking adjacent frames as examples.

[0092] The signal value refers to the time domain sampling value (waveform) or the frequency domain energy value (spectrum), and the frequency domain energy value contains the main characteristics of the signal. Usually, the frequency domain energy value can be stable. Therefore, the signal value in the embodiment of the present application can be taken as an example of the frequency domain energy value.

[0093] As an example, the signal value of the basic data of the current data frame is recorded as x, and the signal value of the basic data of the adjacent frame is recorded as y. Then, based on the difference operation rule, the correction coefficient z for correcting x or y can be calculated, where z=xy. In this way, when the basic data of the current data frame is damaged, it can be restored by y and z. The recovery principle is that x=y+(xy)=y+z. Similarly, when the adjacent frame is damaged, it can be restored by x and z, y=xz. Similarly, it can also be calculated by the correlation coefficient method or XOR operation rules. In the first case, the correction rule is one, and the obtained correction data set is one. At this time, in order to further reduce the amount of correction data, it can also be achieved by data compression: using a preset compression coding method to further reduce the amount of data, the preset compression coding method can be Huffman coding, and then selecting a suitable code book can further reduce the amount of data. In the actual application process, in order to make the correction data more suitable for compression, the correction data can be fine-tuned according to certain rules, and the correction coefficient after fine-tuning can be compressed to reduce the amount of data.

[0094] In the second case, if there are multiple correction rules, multiple correction data sets are obtained. In order to reduce the amount of data processing and improve the data processing speed, multiple correction data sets are screened according to the data size on the basis of compressing each correction data set. The data size of the correction data set finally applied should be as small as possible. Therefore, in each correction data set obtained in each correction rule applied, the data size of each correction data set is determined, and the correction data set with the smallest data size is the target data set, wherein each correction data in the target correction data set is the correction data corresponding to the basic data of each corresponding data frame.

[0095] In the second case above, among the multiple correction data sets obtained by multiple correction rules, the compression coding method and codebook corresponding to the correction rule used in the correction set with the least amount of data can be selected as the compression coding method and codebook for the correction data determined this time. For example, among the three correction rules of applying the difference operation rule, the correlation coefficient method, and the XOR algorithm, the difference operation rule with the least amount of data and its corresponding compression coding method and codebook are finally determined as the compression coding method and codebook for the correction data determined this time, which can reduce the amount of data and reduce the bandwidth burden.

[0096] Involving S203, the basic data, extended data and correction data corresponding to each data frame of the audio are sent to the data receiving end, so that the data receiving end can restore the damaged basic data according to the corresponding correction data, and then combine the restored basic data with the extended data to play the combined target audio.

[0097] refer to Figure 3The embodiment of the present application provides a method for processing audio data, which is applied to a data receiving end and at least includes the following steps:

[0098] S301. Receive basic data, extended data and correction data corresponding to each data frame of the audio; wherein the basic data and the extended data are obtained by processing the data part of each data frame of the audio, and the correction data corresponding to the basic data of the data frame is determined based on the basic data of the data frame and the basic data of at least one other data frame.

[0099] S302: If the received basic data is damaged, restore the damaged basic data according to the corresponding correction data.

[0100] S303, combining the restored basic data and the matched extended data to perform a play operation; wherein the matched extended data is the extended data obtained by processing the same data frame as the restored basic data.

[0101] In the embodiment of the present application, after receiving the basic data, extended data and correction data corresponding to each data frame, the data receiving end determines the correction data according to the basic data of the data frame and the basic data of at least one other data frame when the received basic data is damaged, so that when the damaged basic data is restored, the main signal is guaranteed to be restored so that it can be combined with the extended data after restoration and played. Therefore, the probability of audio data recovery and the quality of the restored audio data are improved.

[0102] In S301, the data receiving end receives basic data (HPNB1, HPNB2...HPNBN), extended data (HPNE1, HPNE2...HPNEN) and correction data corresponding to each data frame from the data sending end. The corresponding data is obtained by the data sending end in the following manner:

[0103] The basic data and extended data are obtained by processing the audio, the basic data is data with a frequency less than a preset frequency threshold, the extended data is data with a frequency greater than or equal to the preset frequency threshold, and the correction data corresponding to the basic data of the data frame is determined based on the basic data of the data frame and the basic data of at least one other data frame.

[0104] Involving S302, the receiving end determines whether the received data is abnormal. If so, it determines whether the received basic data is damaged in the abnormal situation. If damaged, the damaged basic data is restored based on the correction data corresponding to the damaged basic data and the adjacent data frames of the data frame to which the damaged basic data belongs; wherein the adjacent data frame is at least one other data frame applied when determining the correction data.

[0105] Exemplarily, the signal value of the basic data of the adjacent data frame applied when determining the correction data is y, and the correction data is z. The process of determining the correction data includes applying correction rules, such as difference operation rules, and applying compression coding rules and code books when compressing the correction data to perform reverse recovery to obtain the signal value x of the basic data of the damaged data frame.

[0106] Regarding S303, in the actual application process, the extended data may also be damaged. If the extended data is damaged, the damaged extended data is restored by applying smoothing processing or bandwidth expansion; and the restored extended data is used as the matching extended data.

[0107] Specifically, if the data transmitting end adopts the first method mentioned above when dividing the data into blocks, the damaged extended data can be restored by smoothing; if the data transmitting end adopts the second method mentioned above when dividing the data into blocks, the damaged extended data can be restored by bandwidth expansion. Among them, one type of smoothing is blurring, and the other is noise elimination, for example, the average value of the signal values ​​of adjacent frames can be used for calculation processing; the bandwidth expansion method can refer to the implementation in the relevant technology, which will not be described here.

[0108] In addition, during the data recovery process, there are several situations as follows:

[0109] (1) In the abnormal data, if the basic data is not damaged, the basic data is directly restored, and the restoration process is the regular decoding process.

[0110] (2) In the abnormal data, the basic data is damaged, and one of the basic data and the correction data of the adjacent data frame used to determine the correction data is also damaged, then the packet loss compensation method in the related art is used to restore the damaged basic data.

[0111] (3) If the extended data in the abnormal data is not damaged, the extended data is directly restored.

[0112] In order to make the technical solution of this application easier to understand, Figure 4 A signaling flow chart of a method for processing audio data is provided.

[0113] S401: The data transmitting end processes the data part of each data frame of the audio to obtain basic data and extended data.

[0114] S402: The data transmitting end determines a correction data set consisting of correction data corresponding to the basic data of each data frame according to the signal value of the basic data of the data frame and the signal value of the basic data of the adjacent frames.

[0115] Therein, the correction data set is one or more.

[0116] S403: If there are multiple correction data sets, the data sending end determines the amount of data in each correction data set.

[0117] Among them, different correction rules are applied to different correction data sets.

[0118] S404: The data sending end determines a correction data set with the smallest data amount as a target correction data set; wherein each correction data in the target data set is correction data corresponding to the basic data of each corresponding data frame.

[0119] S405: The data transmitting end sends the basic data, the extended data and the corrected data corresponding to each data frame of the audio to the data receiving end.

[0120] S406: The data receiving end receives basic data, extended data and correction data corresponding to each data frame of the audio.

[0121] The basic data and the extended data are obtained by processing the data part of the audio data frame, and the correction data corresponding to the basic data of the data frame is determined based on the basic data of the data frame and the basic data of at least one other data frame.

[0122] S407: If the basic data received by the data receiving end is damaged, the damaged basic data is restored based on the correction data corresponding to the damaged basic data and the adjacent data frames of the data frame to which the damaged basic data belongs.

[0123] S408: If the extended data at the data receiving end is damaged, the damaged extended data is restored by applying smoothing processing or bandwidth expansion; and the restored extended data is used as the matching extended data.

[0124] S409: The data receiving end combines the restored basic data and the matched extended data and performs a play operation.

[0125] The matched extended data is the extended data obtained by processing the same data frame as the restored basic data.

[0126] It should be noted that there is no obvious order between S407 and S408. Figure 4 This is just an example and does not constitute a specific limitation.

[0127] In summary, in the embodiment of the present application, when packet loss makes retransmission inconvenient, the main part of the signal is restored using the undamaged basic data and correction data, and the restoration effect is better than the conventional packet loss compensation method; because the interference has local characteristics in time, and the data frame and the smaller correction data are transmitted separately, as long as the correction data and any corresponding data frame are not abnormal, the main signal of the abnormal data frame can be restored, and the probability of recovery is high, and there is no need for retransmission. In addition, due to the low deployment cost, it is widely applicable to scenarios of embedded application platforms such as Bluetooth broadcasting. Compared with the prior art, it not only ensures a higher signal quality in the absence of interference, but also significantly improves the signal restoration quality in the case of interference.

[0128] like Figure 5 As shown, based on the same inventive concept as the above-mentioned audio data processing method, an embodiment of the present application also provides an audio data processing device, which is integrated at the data sending end, and the audio data processing device includes a data processing module 51, a correction data determination module 52 and a data sending module 53.

[0129] The data processing module 51 is used to process the data part of each data frame of the audio to obtain basic data and extended data;

[0130] A correction data determination module 52, configured to determine correction data corresponding to the basic data of the data frame according to the basic data of the data frame and the basic data of at least one other data frame;

[0131] The data sending module 53 is used to send the basic data, extended data and correction data corresponding to each data frame of the audio to the data receiving end, so that the data receiving end can restore the damaged basic data according to the corresponding correction data and then combine it with the extended data for playback operation.

[0132] In some exemplary embodiments, the correction data determination module 52 is specifically configured to:

[0133] Correction data corresponding to the basic data of the data frame is determined according to the signal value of the basic data of the data frame and the signal value of the basic data of the adjacent frame.

[0134] In some exemplary embodiments, the correction data determination module 52 is further configured to:

[0135] Determine, according to the signal value of the basic data of the data frame and the signal value of the basic data of the adjacent frames, a correction data set consisting of correction data corresponding to the basic data of each data frame; wherein the correction data set is one or more;

[0136] If there are multiple correction data sets, then for each correction data set, determine the amount of data in the correction data set; wherein different correction rules applied to different correction data sets are obtained;

[0137] The correction data set with the smallest data amount is determined as the target correction data set; wherein each correction data in the target data set is the correction data corresponding to the basic data of each corresponding data frame.

[0138] The audio data processing device provided in the embodiment of the present application adopts the same inventive concept as the above-mentioned audio data processing method and can achieve the same beneficial effects, which will not be described in detail here.

[0139] like Figure 6 As shown, based on the same inventive concept as the above-mentioned audio data processing method, an embodiment of the present application also provides an audio data processing device, which is integrated in a receiving terminal. The audio data processing device may include a data receiving module 61, a data recovery module 62 and a playback module 63.

[0140] The data receiving module 61 is used to receive basic data, extended data and correction data corresponding to each data frame of the audio; wherein the basic data and the extended data are obtained by processing the data part of each data frame of the audio, and the correction data corresponding to the basic data of the data frame is determined based on the basic data of the data frame and the basic data of at least one other data frame;

[0141] A data recovery module 62, configured to recover the damaged basic data according to the corresponding correction data if the received basic data is damaged;

[0142] The playing module 63 is used to combine the restored basic data and the matched extended data to perform a playing operation; wherein the matched extended data is the extended data obtained by processing the same data frame as the restored basic data.

[0143] In some exemplary embodiments, the data recovery module 62 is specifically used to:

[0144] Restoring the damaged basic data based on the correction data corresponding to the damaged basic data and the adjacent data frames of the data frame to which the damaged basic data belongs;

[0145] The adjacent data frame is at least one other data frame used when determining the correction data.

[0146] In some exemplary embodiments, an extended data recovery module is further included, which is used to recover the damaged extended data by applying smoothing processing or bandwidth expansion before the restored basic data and the matched extended data are combined and played, and use the restored extended data as the matched extended data.

[0147] The audio data processing device provided in the embodiment of the present application adopts the same inventive concept as the above-mentioned audio data processing method and can achieve the same beneficial effects, which will not be described in detail here.

[0148] Based on the same inventive concept as the above-mentioned audio data processing method, the embodiment of the present application also provides an electronic device, which can be a desktop computer, a portable computer, a smart phone, a tablet computer, a personal digital assistant (PDA), a server, etc. In the embodiment of the present application, the electronic device can be a sending terminal or a receiving terminal, wherein the sending terminal and the receiving terminal can respectively execute the corresponding audio data processing method in the embodiment of the present application. Figure 7 As shown, the electronic device 7 may include a processor 701 and a memory 702 .

[0149] Processor 701 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor (DigitalSignal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as a hardware processor to be executed, or a combination of hardware and software modules in the processor can be executed.

[0150] The memory 702 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 702 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0151] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; the above computer storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to: mobile storage devices, random access memory (RAM, Random Access Memory), magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO)), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)) and other media that can store program codes.

[0152] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods of each embodiment of the present application. The aforementioned storage medium includes: a mobile storage device, a random access memory (RAM, Random Access Memory), a magnetic storage device (such as a floppy disk, a hard disk, a magnetic tape, a magneto-optical disk (MO), etc.), an optical storage device (such as CD, DVD, BD, HVD, etc.), and a semiconductor memory (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), a solid-state drive (SSD)) and other various media that can store program codes.

[0153] The above embodiments are only used to introduce the technical solutions of the present application in detail, but the description of the above embodiments is only used to help understand the methods of the embodiments of the present application and should not be understood as limiting the embodiments of the present application. Any changes or substitutions that can be easily thought of by technicians in this technical field should be included in the protection scope of the embodiments of the present application.

Claims

1. A method for processing audio data, characterized in that: include: For each data frame of the audio, a data portion of the data frame is processed to obtain basic data and extended data; Determining correction data corresponding to the basic data of the data frame according to the basic data of the data frame and basic data of at least one other data frame; The basic data, the extended data and the correction data corresponding to each data frame of the audio are sent to a data receiving end, so that the data receiving end recovers the damaged basic data according to the corresponding correction data and then combines it with the extended data for playback operation.

2. The method according to claim 1, characterized in that Determining, according to the basic data of the data frame and basic data of at least one other data frame, correction data corresponding to the basic data of the data frame, comprises: Correction data corresponding to the basic data of the data frame is determined according to a signal value of the basic data of the data frame and a signal value of basic data of an adjacent frame.

3. The method according to claim 2, characterized in that The determining, according to the signal value of the basic data of the data frame and the signal value of the basic data of the adjacent frame, the correction data corresponding to the basic data of the data frame comprises: Determine, according to the signal value of the basic data of the data frame and the signal value of the basic data of the adjacent frames, a correction data set consisting of correction data corresponding to the basic data of each of the data frames; wherein the correction data set is one or more; If there are multiple correction data sets, then for each correction data set, determine the amount of data in the correction data set; wherein different correction rules applied to different correction data sets are obtained; A correction data set with the smallest data amount is determined as a target correction data set; wherein each correction data in the target correction data set is correction data corresponding to the basic data of each corresponding data frame.

4. A method for processing audio data, characterized in that: include: Receive basic data, extended data and correction data corresponding to each data frame of the audio; wherein the basic data and the extended data are obtained by processing the data part of each data frame of the audio, and the correction data corresponding to the basic data of the data frame is determined based on the basic data of the data frame and the basic data of at least one other data frame; If the received basic data is damaged, restoring the damaged basic data according to the corresponding correction data; The restored basic data and the matched extended data are combined to perform a play operation; wherein the matched extended data is extended data obtained by processing the same data frame as the restored basic data.

5. The method according to claim 4, characterized in that The restoring the damaged basic data according to the corresponding correction data includes: Restoring the damaged basic data based on the correction data corresponding to the damaged basic data and the adjacent data frames of the data frame to which the damaged basic data belongs; The adjacent data frame is at least one other data frame used when determining the correction data.

6. The method according to claim 4 or 5, characterized in that: Before combining the restored basic data and the matched extended data and playing them, the method further includes: If the extended data is damaged, the damaged extended data is restored by applying smoothing or bandwidth expansion; The restored extended data is used as the matched extended data.

7. An audio data processing device, characterized in that: include: A data processing module, for processing a data portion of each data frame of the audio to obtain basic data and extended data; A correction data determination module, configured to determine correction data corresponding to the basic data of the data frame based on the basic data of the data frame and basic data of at least one other data frame; The data sending module is used to send the basic data, the extended data and the correction data corresponding to each data frame of the audio to a data receiving end, so that the data receiving end can restore the damaged basic data according to the corresponding correction data and then combine it with the extended data for playback operation.

8. An audio data processing device, characterized in that: include: A data receiving module, configured to receive basic data, extended data and correction data corresponding to each data frame of the audio; wherein the basic data and the extended data are obtained by processing the data part of each data frame of the audio, and the correction data corresponding to the basic data of the data frame is determined based on the basic data of the data frame and the basic data of at least one other data frame; A data recovery module, used for recovering the damaged basic data according to the corresponding correction data if the received basic data is damaged; The playing module is used to combine the restored basic data and the matching extended data to perform a playing operation; wherein the matching extended data is the extended data obtained by processing the same data frame as the restored basic data.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images