Audio data resampling method, electronic device and storage medium

Abstract

The present invention relates to an audio data resampling method, an electronic device and a storage medium, comprising: S1, obtaining initial audio data; S2, obtaining an initial sampling rate and a target sampling rate as well as target audio data and a target number of sampling points; S3, when the target number of sampling points is not an integer multiple of an audio unit, obtaining the minimum number of remaining sampling points to obtain a cumulative offset; S4, executing S5 when the cumulative offset is less than the audio unit, otherwise executing S6; S5, obtaining the next data packet as the initial audio data, and executing S2; S6, obtaining compensation audio according to the last data of the previous target audio data and the first data of the current target audio data, and adding the compensation audio before the first data of the current target audio data and transmitting it; S7, subtracting the number of bytes corresponding to the compensation audio from the cumulative offset to obtain an updated cumulative offset, and executing S2 after obtaining new initial audio data. The implementation of the present invention can ensure the synchronization of the time series of audio before and after resampling.

Description

Technical Field

[0001] The present invention relates to the technical field of audio data processing, and more specifically, to an audio data resampling method, electronic equipment and storage medium. Background Art

[0002] Currently, a lot of audio uses a 44.1kHz sampling rate, but in many environments or electronic devices such as Android phones and Windows computers, only support 48kHz audio playback, so 44.1kHz->48kHz sampling rate conversion is very common. However, due to the limited computing power of some platforms, the sinc filtering method cannot be used for resampling, and the linear interpolation method cannot retain the details of the previous processing due to its discrete characteristics. When the input data sampling rate cannot be divided by the input length, the output result cannot be divided, resulting in the time series of the resampled audio and the original audio being out of sync. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide an audio data resampling method, an electronic device and a storage medium in view of the above-mentioned technical defects of the prior art.

[0004] The technical solution adopted by the present invention to solve the technical problem is: constructing an audio data resampling method, comprising the steps of:

[0005] S1, obtaining a data packet in the audio transmission process as initial audio data;

[0006] S2, obtaining an initial sampling rate corresponding to the initial audio data and a target sampling rate corresponding to the resampling process, so as to obtain target audio data corresponding to the initial audio data and a target number of sampling points corresponding to the resampling process;

[0007] S3. When the target number of sampling points is not an integer multiple of the audio unit, obtain the minimum remaining number of sampling points of the target number of sampling points relative to the audio unit, and obtain a cumulative offset according to the minimum remaining number of sampling points;

[0008] S4, determining whether the accumulated offset is less than the audio unit, if so, executing step S5, otherwise executing step S6;

[0009] S5, obtaining the next data packet in the audio transmission process as the initial audio data, and executing step S2;

[0010] S6, obtaining at least one audio unit of compensation audio according to the last data of the previous target audio data and the first data of the current target audio data, and adding the compensation audio before the first data of the current target audio data to transmit the current target audio data;

[0011] S7. Subtract the number of bytes corresponding to the compensated audio from the cumulative offset to obtain an updated cumulative offset, obtain the next data packet in the audio transmission process as the initial audio data, and execute step S2.

[0012] Preferably, the audio data resampling method of the present invention further includes:

[0013] S31. When the target number of sampling points is an integer multiple of the audio unit, directly output the target audio data, obtain the next data packet in the audio transmission process as the initial audio data, and execute the step S2.

[0014] Preferably, the audio data resampling method of the present invention further includes:

[0015] During the audio transmission process, when the audio is transmitted through a mono channel, the audio unit is 2, and when the audio is transmitted through a dual channel, the audio unit is 4.

[0016] Preferably, the audio data resampling method of the present invention further includes:

[0017] The compensation audio is obtained by interpolating between the last data of the previous target audio data and the first data of the current target audio data.

[0018] Preferably, in the audio data resampling method of the present invention, obtaining the compensated audio by interpolation includes obtaining the compensated audio by linear interpolation.

[0019] Preferably, the audio data resampling method of the present invention further includes:

[0020] The actual number of sampling points corresponding to the resampling process is obtained to resample the initial audio data according to the actual number of sampling points; wherein the actual number of sampling points is an integer part of the target number of sampling points.

[0021] Preferably, the audio data resampling method of the present invention further includes:

[0022] For each actual sampling point, obtaining a first audio signal and a second audio signal in the initial audio data that are closest to the actual sampling point;

[0023] Acquire a first distance between the first audio signal and the actual sampling point and a second distance between the second audio signal and the actual sampling point;

[0024] A resampled signal acquisition function corresponding to the actual sampling points is constructed with the first distance and the second distance as weights to acquire a resampled signal corresponding to the actual sampling points.

[0025] Preferably, in the audio data resampling method of the present invention, the resampled signal acquisition function comprises the following formula:

[0026] T(x i )=F(α)*w a +F(b)*w b

[0027] Where T(x i ) is the resampled signal corresponding to the actual sampling point, w a , w b are the first distance and the second distance respectively, and F(a) and F(b) are the first audio signal and the second audio signal respectively.

[0028] The present invention also constructs an electronic device, comprising a memory and a processor;

[0029] The memory is used to store computer programs;

[0030] The processor is configured to execute the computer program to implement any of the methods described above.

[0031] The present invention also constructs a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, any of the methods described above is implemented.

[0032] An audio data resampling method, electronic device and storage medium implementing the present invention have the following beneficial effects: by judging overflow and filling in missing data segments, the time series synchronization of the audio before and after resampling is ensured, and noise caused by missing data can be prevented. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0034] Figure 1 is a program flow chart of an embodiment of an audio data resampling method of the present invention;

[0035] Figure 2 is a program flow chart of another embodiment of an audio data resampling method of the present invention;

[0036] Figure 3 It is a program flow chart of another embodiment of an audio data resampling method of the present invention. DETAILED DESCRIPTION

[0037] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0038] like Figure 1 As shown, in a first embodiment of an audio data resampling method of the present invention, the steps include: S1, obtaining a data packet in the audio transmission process as initial audio data. Specifically, in the audio data transmission process, the data packet for each transmission can be obtained according to the audio data transmission protocol, wherein the data packet is the data obtained based on the original sampling process in the audio data transmission. Each data packet is used as the initial audio data in the entire data process. For example, in an audio data transmission process, the audio data can be packaged based on the data transmission protocol, wherein each data packet has 512 bytes obtained according to the original sampling process.

[0039] S2. Obtain the initial sampling rate corresponding to the initial audio data and the target sampling rate corresponding to the resampling process, so as to obtain the target audio data corresponding to the initial audio data and the target number of sampling points corresponding to the resampling process. Specifically, the size of the original audio data can be obtained according to the initial sampling rate corresponding to the initial audio data. The resampled audio data, i.e., the target audio data, can be obtained according to the resampling sampling rate, i.e., the target sampling rate, during the resampling process. At the same time, the target number of sampling points corresponding to the resampling process can be obtained according to theoretical calculations, and the target number of sampling points can be the number of sampling points required after resampling in theory. However, in the actual process of generating the target audio data, sampling is not necessarily performed completely according to the target number of sampling points. The target sampling points can be calculated according to the formula Obtain, where K is the target number of sampling points, M is the target sampling rate, N is the data size of the initial audio data, and L is the initial sampling rate.

[0040] S3. When the target number of sampling points is not an integer multiple of the audio unit, the minimum remaining number of sampling points of the target number of sampling points relative to the audio unit is obtained, and the cumulative offset is obtained according to the minimum remaining number of sampling points. Specifically, during the audio sampling process, the number of audio bytes can only be an integer. When the target number of sampling points is not an integer, it is usually necessary to retain the integer part for sampling to obtain an integer number of audio bytes. At the same time, during the audio transmission process, the audio data must also be transmitted in audio units, that is, one audio unit is transmitted at a time. When the number of audio bytes, that is, the remaining part of the corresponding sampling signal number, is not enough for one audio unit and cannot be transmitted, the bytes will be discarded. The transmission of the few bytes will cause the data transmission process to be out of sync in the time series. At this time, the remaining minimum number of remaining sampling points, that is, the remaining number of sampling points that is less than one audio unit, is obtained. The minimum remaining number of sampling points is accumulated to obtain the cumulative offset. It can be understood here that the accumulation process is to accumulate the current minimum remaining number of sampling points and the historical cumulative offset to obtain a new cumulative offset. When the initial audio data is the first data packet, the historical cumulative offset can be understood as zero, and the cumulative offset at this time can be directly the minimum number of remaining sampling points at that time. In one embodiment, the number of bytes of the initial audio data is 512, the initial sampling rate is 44.1kHz, and the target sampling rate is 48kHz. According to the following formula, the target number of sampling points can be 557.279, which also corresponds to the target number of bytes.

[0041]

[0042] However, based on different audio transmission modes, for example, when it is single-channel transmission, the audio unit is 2 bytes, so only 556 bytes can be output. According to the formula, the current minimum number of remaining sampling points is 557.279-556=1.279. The cumulative offset is updated according to the remaining number of sampling points. The specific formula is: Where E is the cumulative offset and [K] is the bytes that can be transmitted.

[0043] In one embodiment, during audio transmission, when the audio is transmitted through a mono channel, the audio unit is 2, and when the audio is transmitted through a dual channel, the audio unit is 4. That is, the number of bytes contained in the audio unit can be obtained according to the audio transmission mode. It can be understood that the number of bytes corresponds to the number of sampling points of the resampled. When the mono channel is transmitted, 2 bytes are transmitted each time, so the corresponding audio unit is 2. When the dual channel is transmitted, 4 bytes are transmitted each time, so the corresponding audio unit is 4.

[0044] S4, determine whether the cumulative offset is less than the audio unit, if so, execute step S5, otherwise execute step S6; S5, obtain the next data packet in the audio transmission process as the initial audio data, and execute step S2; S6, obtain at least one audio unit of compensation audio according to the last data of the previous target audio data and the first data of the current target audio data, and add the compensation audio before the first data of the current target audio data to transmit the current target audio data; S7, subtract the number of bytes corresponding to the compensation audio from the cumulative offset to obtain the updated cumulative offset, obtain the next data packet in the audio transmission process as the initial audio data, and execute step S2. Specifically, when the cumulative offset E is less than one audio unit, it means that the current time offset is within an acceptable range, no operation is required, and the initial audio data processing process is performed again. When the cumulative offset E is an audio unit that can be transmitted independently, that is, when the value of E is greater than or equal to one audio unit, the number of audio units contained in E can be obtained, and the compensation audio of the corresponding number of audio units is generated according to the number to compensate for the transmission process. The compensation audio can be obtained based on the last data of the previous target audio data and the first data of the current target audio data, and transmitted between the last data of the previous target audio data and the first data of the current target audio data to achieve a smooth transition of the data. After generating the compensation audio according to the cumulative offset E, the cumulative offset is subtracted from the number of bytes corresponding to the compensation audio to obtain the updated cumulative offset. It can be understood that after compensating one audio unit, the cumulative offset E can be directly subtracted from one audio unit, and the remainder can be used as the historical cumulative offset. Then a new data packet is obtained to perform the next resampling process for the initial audio data.

[0045] Optional, such as Figure 2 As shown, in the audio data resampling method of the present invention, it also includes: S31, when the target sampling point number is an integer multiple of the audio unit, directly output the target audio data, obtain the next data packet in the audio transmission process as the initial audio data, and execute step S2. That is, when the obtained target sampling point number is an integer multiple of the audio unit, it can be understood that there is no minimum remaining sampling point number at this time, and it can also be understood that the minimum remaining sampling point number is zero. At this time, the current target audio data can be directly transmitted without judging the cumulative offset. Then a new data packet is obtained as the initial audio data to perform the next resampling process.

[0046] Optionally, in the audio data resampling method of the present invention, compensating audio is obtained by interpolation between the last data of the previous target audio data and the first data of the current target audio data. Specifically, compensating audio can be obtained between the last data of the previous target audio data and the first data of the current target audio data by interpolation. The interpolation process can be implemented using some known difference functions. For example, compensating audio can be obtained by linear interpolation. For example, the median data of two data can be obtained. The data here can be understood as data of an audio unit. For example, in mono, the last 2 bytes in the previous data packet and the first 2 bytes in the current data packet are median.

[0047] According to Shannon's sampling theorem, as long as the sampling frequency is greater than or equal to twice the highest frequency of the valid signal, the sampled signal can contain all the sampled information, but in actual processing, the signal is usually damaged. For example, non-integer resampling will cause loss of details and output length mismatch. The above process can solve the synchronization problem on the time series.

[0048] Optionally, the audio data resampling method of the present invention further includes: obtaining the actual number of sampling points corresponding to the resampling process, so as to resample the initial audio data according to the actual number of sampling points; wherein the actual number of sampling points is the integer part of the target number of sampling points. Specifically, the actual number of sampling points corresponding to the resampling process can be obtained by taking the integer part according to the target number of sampling points corresponding to the resampling process, and the initial audio data can be resampled according to the actual number of sampling points to obtain the target audio data.

[0049] Optional, such as Figure 3 As shown, in the audio data resampling method of the present invention, it also includes: A1, for each actual sampling point, obtaining the first audio signal and the second audio signal in the initial audio data that are closest to the actual sampling point; A2, obtaining the first distance between the first audio signal and the actual sampling point and the second distance between the second audio signal and the actual sampling point; A3, constructing a resampled signal acquisition function corresponding to the actual sampling point with the first distance and the second distance as weights to obtain the resampled signal corresponding to the actual sampling point number. Specifically, according to the obtained actual sampling point i (1<=i<=K), for each sampling point, its position x is obtained. i for:

[0050]

[0051] For each resampling point x i , take two points a and b of the original audio signal, usually the position closest to the resampling point, satisfying the following relationship:

[0052] a<=x i <=b

[0053] Resample point x according to distance i The distance, weight w a , w b ,

[0054] w a =x i -a

[0055] w b =bx i

[0056] According to the weight w a , w b Construct the resampled signal acquisition function to get the resampled point x i The corresponding signal.

[0057] In one embodiment, the resampling point x can be obtained specifically according to the following formula: i The corresponding signal T(x i ),

[0058] T(x i )=F(α)*w a +F(b)*w b w a , w b are the first distance and the second distance respectively, and F(a) and F(b) are the first audio signal and the second audio signal respectively.

[0059] Compared with upsampling and downsampling, the number of iterations is K, and the time complexity of resampling is O(K). The execution efficiency is greatly improved compared with the sinc filtering method. On the same platform, for example, the processing test of two-channel audio data (1920 bytes) with a sampling rate of 10ms and 48k is as follows:

[0060] Methods / Results Time Consumption Spectrum changes Listening Sinc Filtering 4ms Less loss Noise-free Process of the invention 0.1ms High frequency noise appears Noise-free

[0061] The above process has a great advantage in processing speed, because the noise it generates is basically high-frequency noise, which exceeds the frequency that the human ear can perceive in most cases (20khz), so it basically has no effect on the hearing sense and can be effectively used in platforms with limited spare computing power.

[0062] The above process can solve the problem that linear interpolation cannot retain non-integer overflow data when resampling the signal, ensure the synchronization of the time series before and after the resampled signal, and solve the problem of time-consuming filtering resampling from a mathematical method. It has great practical significance for audio signal processing on low-computing power MCU platforms.

[0063] In the above resampling process, the CPU occupancy rate does not exceed 1%, which provides a lot of room for the flexibility of software development. The high-frequency noise generated can be filtered by a low-pass filter on the hardware, with basically no distortion, ensuring high sound quality while not occupying computing power. This process can be widely used in audio software, reducing hardware costs, increasing available resources for software development, and solving the common problem of insufficient computing power under streaming audio transmission.

[0064] In addition, an electronic device of the present invention may be an audio device, which has the function of implementing the corresponding steps performed in the above method. Each function may be implemented by hardware, or by hardware executing corresponding software. The corresponding hardware or software includes one or more modules corresponding to the above functions. That is, the steps in the above method are respectively performed by one or more modules. The specific coordination operations between the modules may refer to the specific process of the above method, which will not be repeated here.

[0065] In addition, an electronic device of the present invention may be an audio device, and may also include a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program to implement any of the above methods. Specifically, according to an embodiment of the present invention, the process described with reference to the flowchart above may be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed by an electronic device and when executed, it executes the above functions defined in the method of the embodiment of the present invention. The electronic device in the present invention may be a terminal such as a notebook, a desktop, a tablet computer, a smart phone, or a server.

[0066] In addition, a computer storage medium of the present invention stores a computer program thereon, and when the computer program is executed by a processor, any of the above methods is implemented. Specifically, it should be noted that the above-mentioned computer-readable medium of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0067] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0068] It can be understood that the above embodiments only express the preferred implementation modes of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the patent scope of the present invention. It should be pointed out that, for ordinary technicians in this field, the above technical features can be freely combined without departing from the concept of the present invention, and several deformations and improvements can be made, which all belong to the protection scope of the present invention. Therefore, all equivalent changes and modifications made to the scope of the claims of the present invention should belong to the coverage of the claims of the present invention.

Claims

1. An audio data resampling method, characterized in that: Includes steps: S1, obtaining a data packet in the audio transmission process as initial audio data; S2, obtaining an initial sampling rate corresponding to the initial audio data and a target sampling rate corresponding to the resampling process, obtaining original audio data according to the initial sampling rate corresponding to the initial audio data, and resampling the original audio data corresponding to the initial audio data according to the target sampling rate to obtain target audio data corresponding to the initial audio data and a target number of sampling points corresponding to the resampling process; S3. When the target number of sampling points is not an integer multiple of the audio unit, obtain the minimum remaining number of sampling points of the target number of sampling points relative to the audio unit, and obtain a new cumulative offset according to the minimum remaining number of sampling points and the historical cumulative offset; the audio unit is the number of bytes of audio data transmitted in a single transmission during audio transmission; S4, determining whether the accumulated offset is less than the audio unit, if so, executing step S5, otherwise executing step S6; S5, obtaining the next data packet in the audio transmission process as the initial audio data, and executing step S2; S6, obtaining at least one audio unit of compensation audio according to the last data of the previous target audio data and the first data of the current target audio data, and adding the compensation audio before the first data of the current target audio data to transmit the current target audio data; S7. Subtract the number of bytes corresponding to the compensated audio from the cumulative offset to obtain an updated cumulative offset, obtain the next data packet in the audio transmission process as the initial audio data, and execute step S2.

2. The audio data resampling method according to claim 1, characterized in that: The method further comprises: S31. When the target number of sampling points is an integer multiple of the audio unit, directly output the target audio data, obtain the next data packet in the audio transmission process as the initial audio data, and execute the step S2.

3. The audio data resampling method according to claim 1, characterized in that: The method further comprises: During the audio transmission process, when the audio is transmitted through a mono channel, the audio unit is 2, and when the audio is transmitted through a dual channel, the audio unit is 4.

4. The audio data resampling method according to claim 1, characterized in that: The method further comprises: The compensation audio is obtained by interpolating between the last data of the previous target audio data and the first data of the current target audio data.

5. The audio data resampling method according to claim 4, characterized in that: Obtaining the compensated audio by interpolation includes obtaining the compensated audio by linear interpolation.

6. The audio data resampling method according to claim 1, characterized in that: The method further comprises: The actual number of sampling points corresponding to the resampling process is obtained to resample the initial audio data according to the actual number of sampling points; wherein the actual number of sampling points is an integer part of the target number of sampling points.

7. An electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is configured to execute the computer program to implement the method according to any one of claims 1 to 6.

8. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images