Audio sampling rate conversion method based on multi-rate signal processing and related device

Abstract

The invention discloses an audio sampling rate conversion method based on multi-rate signal processing and a related device, and the method comprises the steps: carrying out the self-adaptive multi-resolution time-frequency representation, perception weighting factor generation and weighted spectrum reconstruction of an original audio signal to be converted, and obtaining the weighted time-frequency representation; frequency sub-band division and sub-band independent resampling are carried out on the weighted time-frequency representation and the target sampling rate according to the Bark scale, and a resampling sub-band frequency spectrum is obtained; and performing sub-band synthesis on the re-sampling sub-band frequency spectrum to obtain a target frequency spectrum of a target sampling rate, performing inverse short-time Fourier transform on the target frequency spectrum to reconstruct a time domain audio signal, and obtaining a converted audio signal with the target sampling rate. According to the method, through weighted time-frequency representation and frequency sub-band division and independent resampling according with human ear hearing characteristics, the tone quality performance of audio sampling rate conversion is remarkably improved, and the perception quality of human ears on converted audio is optimized on the premise of ensuring signal integrity.

Description

Technical Field

[0001] Specifically, this relates to an audio sampling rate conversion method and apparatus, and a computing device based on multi-rate signal processing. Background Technology

[0002] In digital audio processing, the sampling rate parameter describes the quality and bandwidth of a digital audio signal, representing the number of times the analog signal is sampled per second. Different audio devices, applications, and storage media often have different requirements for audio sampling rates. For example, CD quality typically uses a 44.1kHz sampling rate, high-quality recordings usually use 48kHz, 96kHz, or even higher sampling rates, while telephone voice communication typically uses lower sampling rates of 8kHz or 16kHz. Due to mismatches between audio sources, processing platforms, and output devices, it is often necessary to convert audio signals from one sampling rate to another, a process known as sampling rate conversion. For example, when playing audio files with different sampling rates, it is necessary to convert them to a sampling rate supported by the sound card.

[0003] In audio mixing or effects processing, multiple audio tracks with different sampling rates need to be converted to the same sampling rate to ensure consistent processing standards. To reduce file size or adapt to bandwidth limitations (such as streaming media and mobile communications), high-sampling-rate audio is typically downsampled to a lower sampling rate.

[0004] Existing audio sampling rate conversion methods mainly include interpolation and decimation-based methods and Fourier transform-based methods. Interpolation and decimation-based methods (FIR / IIR filters) typically involve upsampling (inserting zeros and performing low-pass filtering) and downsampling (low-pass filtering and decimation). The main challenge is designing suitable digital filters (such as finite impulse response-FIR filters or infinite impulse response-IIR filters) to remove aliasing components (during downsampling) or suppress image frequencies (during upsampling). To achieve good anti-aliasing / anti-image effects, filters with steep transition bands and high stopband attenuation are required, necessitating higher-order filters and increasing computational complexity. Furthermore, the transition band characteristics of the filter cause attenuation or phase distortion of edge frequencies (such as high-frequency components) in the original signal, especially during significant upsampling or downsampling, making it difficult to completely avoid these problems. Fourier transform-based methods (frequency domain interpolation / decimation) convert the signal to the frequency domain (e.g., via FFT), perform spectral expansion or truncation in the frequency domain, and then convert it back to the time domain using an inverse Fourier transform. For example, upsampling fills the frequency domain with zeros outside the Nyquist frequency before performing an IFFT; downsampling can truncate the high-frequency portion. Since the Fourier transform typically processes the entire signal or fixed-length frames, the use of window functions can introduce spectral leakage and artifacts, especially at frame boundaries, if the signal is processed in frames. Furthermore, because it processes the entire spectrum uniformly, it lacks fine-grained control over local spectral characteristics and does not consider the perceptual characteristics of the human ear, potentially leading to ringing effects in the time domain when the spectrum is truncated.

[0005] Therefore, there is an urgent need for a new sampling rate conversion method that can not only perform the basic function of sampling rate conversion, but also significantly improve the subjective auditory quality of the converted audio and adaptively process audio signals with different characteristics. To address the above problems, this invention proposes an audio sampling rate conversion method based on multi-rate signal processing. Through weighted time-frequency representation and frequency sub-band division and independent resampling that conform to human auditory characteristics, it improves the sound quality performance of the converted audio, optimizing the perceived quality of the converted audio while ensuring signal integrity. Summary of the Invention

[0006] In view of the above problems, the present invention provides an audio sampling rate conversion method and apparatus, and a computing device based on multi-rate signal processing.

[0007] According to one aspect of the present invention, an audio sampling rate conversion method based on multi-rate signal processing is provided, comprising: The original audio signal to be converted is subjected to adaptive multi-resolution time-frequency representation, perceptual weighting factor generation, and weighted spectrum reconstruction to obtain a weighted time-frequency representation; Based on the Bark scale, the weighted time-frequency representation and the target sampling rate are divided into frequency sub-bands and resampled independently in each sub-band to obtain the resampled sub-band spectrum. The resampled subband spectrum is synthesized into a target spectrum with the target sampling rate. The time-domain audio signal is reconstructed by performing an inverse short-time Fourier transform on the target spectrum, resulting in a converted audio signal with the target sampling rate.

[0008] In an alternative embodiment, the adaptive multi-resolution time-frequency representation further includes: Calculate the short-time energy envelope and zero-crossing rate time-domain characteristics of the original audio signal, and select a time-frequency analysis window function based on the short-time energy envelope and zero-crossing rate time-domain characteristics; wherein, in periods of rapid signal transients and fast instantaneous frequency changes, a window function with shorter time support is selected to sacrifice some frequency resolution to obtain more accurate time positioning; in periods of stable signal and stable spectral structure, a window function with longer time support is selected to obtain higher frequency resolution. The original audio signal is subjected to a short-time Fourier transform using the time-frequency analysis window function to obtain an adaptive time-frequency complex matrix; wherein, the row index of the adaptive time-frequency complex matrix corresponds to the time frame, the column index corresponds to the frequency point, and the element value corresponds to the complex spectrum coefficient on the time-frequency unit.

[0009] In an alternative approach, the generation of the perceptual weighting factor further includes: The linear frequency axis is nonlinearly divided according to the Bark scale to form a series of subbands corresponding to the critical frequency band of the cochlea; The power density of the signal is calculated in each Bark subband based on the spectral amplitude information contained in the adaptive time-frequency complex matrix. The masking threshold, including the mute masking threshold and the global masking threshold, is calculated for each sub-band based on the power density of the signal calculated within each Bark sub-band. Based on the masking threshold of each Bark subband and the preset mapping function, a sensing weighting factor is generated for each discrete frequency point; wherein, the length of the sensing weighting factor is exactly the same as the number of frequency points of the adaptive time-frequency complex matrix, and each element value is the relative importance weight of the spectral component of the corresponding frequency point in sensing.

[0010] In an alternative approach, the weighted spectrum reconstruction further includes: The complex spectral coefficients at each time frame and frequency point in the adaptive time-frequency complex matrix are subjected to complex number operations with the weight values ​​at the corresponding frequency points in the perceptual weighting factor vector to obtain the weighted time-frequency representation, which is then used to perform perceptual domain filtering on the original time-frequency representation of the original audio signal.

[0011] In one alternative approach, dividing the weighted time-frequency representation and the target sampling rate into frequency subbands and performing independent subband resampling based on the Bark scale to obtain the resampled subband spectrum further includes: The nominal Nyquist frequency is determined according to the Nyquist sampling theorem and the target sampling rate. The linear frequency range from zero Hz to the nominal Nyquist frequency is non-uniformly divided into several continuous Bark subbands according to the Bark scale lookup table or analytical formula, wherein each subband corresponds to the critical bandwidth of the cochlea. Based on the mapping position of the Bark subband boundary on the linear frequency axis, the weighted time-frequency representation is divided in the frequency dimension. For each Bark subband, all frequency columns corresponding to it in the weighted time-frequency matrix are extracted to form the subband time-frequency matrix. The frequency axis of the spectrum of each time frame of the sub-band time-frequency matrix is ​​resampled using the sinc function to obtain a set of resampled sub-band spectra.

[0012] In one alternative approach, reconstructing the time-domain audio signal by performing an inverse short-time Fourier transform on the target spectrum to obtain a converted audio signal with the target sampling rate further includes: For each time frame in the target spectrum, the complex values ​​at all its frequency points are extracted sequentially to form an independent frequency vector; The time-domain signal segment of the corresponding time frame is calculated based on the complex values ​​in the frequency vector; wherein each time-domain signal segment contains a series of continuous sample points; The time-domain signal segments obtained by the inverse discrete Fourier transform are overlapped, added together, and then spliced ​​in chronological order to obtain the converted audio signal at the target sampling rate.

[0013] In an alternative approach, resampling the frequency axis of the spectrum of each time frame of the sub-band time-frequency matrix according to the sinc function to obtain the resampled sub-band spectrum set further includes: Process the frequency column vector corresponding to each time frame within the sub-band time-frequency matrix one by one; The frequency column vector is resampled along the frequency axis using the sinc function; the sinc function ensures that the original signal information is preserved to the maximum extent during the interpolation process while avoiding the introduction of spurious frequency components. The new frequency vectors obtained by resampling each time frame through the frequency axis are reorganized back into their respective sub-band structures to form a series of new sub-band time-frequency matrices. The new sub-band time-frequency matrices are aggregated to form a resampled sub-band spectrum set; wherein each element in the sampled sub-band spectrum set is a sub-band spectrum that adapts to the frequency point distribution under the target sampling rate.

[0014] In one alternative approach, the process of overlapping, adding, and synthesizing the time-domain signal segments obtained through inverse discrete Fourier transform, and then splicing them together in chronological order to obtain the converted audio signal at the target sampling rate, further includes: The time-domain signal segment is multiplied sample by sample with the selected synthesis window function, and the time-domain frame obtained after the multiplication is positioned on the time axis according to the preset frame shift, so as to obtain the time-domain frame sequence on the time axis. The sample values ​​of the overlapping regions in the time-domain frame sequence on the time axis are added together to obtain a preliminary time-domain waveform; wherein, for each time index position in the overlapping region, the corresponding sample values ​​of all time-domain frames covering the index position are algebraically added together; for non-overlapping regions, the sample values ​​of the current frame are retained; The initial time-domain waveform is subjected to DC offset correction, the arithmetic mean of the entire signal sequence is calculated, and the arithmetic mean is subtracted from each sample value; the numerical information of the target sampling rate is bound to the initial time-domain waveform, and the converted audio signal of the target sampling rate is output.

[0015] According to another aspect of the present invention, an audio sampling rate conversion device based on multi-rate signal processing is provided, comprising: The adaptive perceptual weighted time-frequency representation module is used to perform adaptive multi-resolution time-frequency representation, perceptual weighting factor generation, and weighted spectrum reconstruction on the original audio signal to be converted, so as to obtain a weighted time-frequency representation. The frequency sub-band resampling module is used to divide the weighted time-frequency representation and the target sampling rate into frequency sub-bands and perform independent sub-band resampling according to the Bark scale to obtain the resampled sub-band spectrum. The target spectrum reconstruction and time-domain synthesis module is used to perform sub-band synthesis on the resampled sub-band spectrum to obtain the target spectrum of the target sampling rate, and to perform inverse short-time Fourier transform on the target spectrum to reconstruct the time-domain audio signal, thereby obtaining the converted audio signal with the target sampling rate.

[0016] According to another aspect of the present invention, a computing device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the audio sampling rate conversion method based on multi-rate signal processing described above.

[0017] According to the scheme provided by the present invention, the original audio signal to be converted is subjected to adaptive multi-resolution time-frequency representation, perceptual weighting factor generation, and weighted spectrum reconstruction to obtain a weighted time-frequency representation; the weighted time-frequency representation and the target sampling rate are divided into frequency sub-bands and independently resampled according to the Bark scale to obtain the resampled sub-band spectrum; the resampled sub-band spectrum is synthesized to obtain the target spectrum of the target sampling rate; the target spectrum is subjected to inverse short-time Fourier transform to reconstruct the time-domain audio signal, resulting in a converted audio signal with the target sampling rate. The present invention significantly improves the sound quality performance of audio sampling rate conversion through weighted time-frequency representation and frequency sub-band division and independent resampling that conform to human hearing characteristics, optimizing the perceptual quality of the converted audio while ensuring signal integrity.

[0018] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating the audio sampling rate conversion method based on multi-rate signal processing according to an embodiment of the present invention is shown. Figure 2 The diagram shows a schematic representation of the signal waveforms before and after resampling at the receiver in an embodiment of the present invention. Figure 3 This diagram illustrates the comparison of bit error rate before and after resampling in an embodiment of the present invention. Figure 4 A schematic diagram of the framework of an audio sampling rate conversion device based on multi-rate signal processing according to an embodiment of the present invention is shown; Figure 5 A schematic diagram of the structure of a computing device according to an embodiment of the present invention is shown. Detailed Implementation

[0020] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0021] Figure 1 A flowchart illustrating an audio sampling rate conversion method based on multi-rate signal processing according to an embodiment of the present invention is shown. Specifically, as... Figure 1 As shown, it includes the following steps: Step S101: The original audio signal to be converted is subjected to adaptive multi-resolution time-frequency representation, perceptual weighting factor generation and weighted spectrum reconstruction to obtain a weighted time-frequency representation.

[0022] In this embodiment, by using adaptive multi-resolution time-frequency representation and perceptual weighting factors, the perceptual characteristics of the human ear can be more accurately simulated. This allows for dynamic adjustment of the processing priority for these perceptual characteristics, avoiding over-processing of signals to which the human ear is insensitive. This saves computational resources while concentrating more resources on frequency bands and time periods to which the human ear is sensitive. Adaptive multi-resolution time-frequency representation provides better time localization during periods of rapid transients and better frequency resolution during periods of stability, helping to more accurately capture and reconstruct the transient and steady-state characteristics of signals, thereby reducing such distortions. Traditional sampling rate conversion methods may introduce auditoryly unacceptable artifacts (such as aliasing, pre-ringing / post-ringing), especially in the transient parts of music.

[0023] For example, consider an audio signal of a guitar solo, whose sampling rate is converted from 44.1kHz to 48kHz. When a sharp, transient note (string strike) occurs, a drastic change in the energy envelope and zero-crossing rate is detected. A short analysis window (e.g., 256 sample points) is selected to clearly capture the start time of the note, preventing the sound from sounding "muddled." When the guitar note vibrates steadily (long note), a relatively stable energy envelope and zero-crossing rate are detected. A longer analysis window (e.g., 1024 sample points) is selected to accurately distinguish the fundamental frequency and overtones, making the timbre sound fuller and clearer. In this process, the perceptual weighting factor is calculated by dividing the linear frequency axis into 24 Bark sub-bands. Within a Bark sub-band (e.g., between 1 kHz and 1.2 kHz), playing a loud A4 note (approximately 440 Hz) and its overtones (e.g., 880 Hz) might strongly mask other faint noise within that sub-band. The calculation determines that noise below a certain threshold is barely audible to the human ear within this sub-band. Based on this threshold, a perceptual weighting factor is generated for each frequency point within that sub-band. For overtone frequency points, the weight might be close to 1; while for faint background noise frequencies heavily masked by overtones, the weight might be close to 0.1 or lower. For the weighted spectrum, for example, the spectral coefficient of the guitar A4 overtone (approximately 880 Hz) is multiplied by a higher weight value (e.g., 0.95) to maintain its importance in the weighted spectrum. The resulting weighted time-frequency representation is more audibly optimized, suppressing noise and unimportant details while preserving and emphasizing important musical elements.

[0024] In an alternative embodiment, the adaptive multi-resolution time-frequency representation further includes: Calculate the short-time energy envelope and zero-crossing rate time-domain characteristics of the original audio signal, and select a time-frequency analysis window function based on the short-time energy envelope and zero-crossing rate time-domain characteristics; wherein, in periods of rapid signal transients and fast instantaneous frequency changes, a window function with shorter time support is selected to sacrifice some frequency resolution to obtain more accurate time positioning; in periods of stable signal and stable spectral structure, a window function with longer time support is selected to obtain higher frequency resolution. The original audio signal is subjected to a short-time Fourier transform using the time-frequency analysis window function to obtain an adaptive time-frequency complex matrix; wherein, the row index of the adaptive time-frequency complex matrix corresponds to the time frame, the column index corresponds to the frequency point, and the element value corresponds to the complex spectrum coefficient on the time-frequency unit.

[0025] In this embodiment, a dynamic balance between time resolution and frequency resolution is achieved by adaptively selecting the window function length, ensuring optimal time-frequency representation in different signal characteristic regions. Assuming the original audio signal sampling rate is 44.1kHz, different window function lengths are selected based on its time-domain characteristics, as shown in Table 1. Traditional STFTs use a fixed window function length throughout the signal analysis process. Long window functions provide high frequency resolution but low time resolution (unable to accurately capture transient events), while short window functions provide high time resolution but low frequency resolution (unable to accurately analyze harmonic structures). For transient events in audio signals (such as the start of an instrument, percussion sounds, and consonants in speech), their frequency content changes rapidly within a short time. Using a window function with a shorter time support allows for more precise location of these transient events on the time axis, avoiding ambiguity caused by excessively long window functions. For steady-state components in audio signals (such as sustained notes and vowels), their spectral structure is relatively stable. Using a window function with a longer time support provides higher frequency resolution, thereby more finely distinguishing harmonic components and overtone structures in the signal.

[0026] Specifically, the energy changes of an audio signal over a short period are calculated. Drastic energy changes typically indicate transient events (such as pitch changes or the start / end of a musical performance). The number of times the signal crosses zero points per unit time is also calculated. A high zero-crossing rate usually indicates high-frequency or noise components, and transient signals (especially some speech consonants) are often accompanied by a high zero-crossing rate. When the short-time energy envelope changes rapidly and significantly, or the zero-crossing rate increases sharply within a short period, the current time period is considered a period of intense signal transients and rapid instantaneous frequency changes. When the short-time energy envelope and zero-crossing rate change gradually over a certain period, the current time period is considered a period of signal stability and stable spectral structure. A window function with a short time support is selected (such as a Hanning window or Hamming window with a length of 256 or 51 sample points). A short window means fewer samples participate in the Fourier transform, resulting in a narrower time axis and more precise time positioning, but lower frequency resolution. A long window means more samples participate in the Fourier transform, resulting in a narrower frequency axis and higher frequency resolution, but relatively less precise time positioning.

[0027] Optionally, when switching between different window functions, to avoid discontinuities in the spectrum, frame overlap or a gradually changing window function is used when the window function length changes. For Short Time Fourier Transform (STFT), the original audio signal is divided into frames, and the previously adaptively selected window function is applied to each frame. A Fast Fourier Transform (FFT) is then performed on each windowed frame to obtain the spectral information of that frame. The spectral results (complex form, containing amplitude and phase information) of all frames are arranged in chronological order to form a two-dimensional complex matrix, where the row index represents different time frames (i.e., time), the column index represents different frequency points (i.e., frequency), and the element values ​​correspond to the complex spectral coefficients at each time-frequency unit, representing the complex amplitude of that time frame at that frequency point.

[0028] Table 1 Time period characteristics judgment Judgment basis Selected window function type / length advantage Adaptive time-frequency complex matrix The signal undergoes rapid transient changes and its instantaneous frequency changes quickly. The short-time energy envelope increases by more than 30 dB within 10 ms; the zero-crossing rate increases from a low value (e.g., 100 Hz) to a high value (e.g., 2 kHz) within 5 ms. A shorter Hanning window (e.g., 256 sample points) More precise time positioning (high time resolution) Row indexes (time frames) have closer intervals; column indexes (frequency points) have relatively lower resolution. The signal is stable and the spectral structure is stable. The short-time energy envelope changes by less than 5 dB within 50 ms; the zero-crossing rate fluctuates by less than 200 Hz within 50 ms. A longer Hanning window (e.g., 1024 sample points) Higher frequency resolution (able to distinguish closer frequency components) Row indexes (time frames) have relatively loose intervals; column indexes (frequency points) have relatively high resolution. Mixed or transitional regions The changes in energy and zero-crossing rate are moderate, or they transition from transient to stable. A window function of medium length (e.g., 512 sample points) Balanced time and frequency resolution The time-frequency matrix exhibits different resolution characteristics in different regions.

[0029] In an alternative approach, the generation of the perceptual weighting factor further includes: The linear frequency axis is nonlinearly divided according to the Bark scale to form a series of subbands corresponding to the critical frequency band of the cochlea; The power density of the signal is calculated in each Bark subband based on the spectral amplitude information contained in the adaptive time-frequency complex matrix. The masking threshold, including the mute masking threshold and the global masking threshold, is calculated for each sub-band based on the power density of the signal calculated within each Bark sub-band. Based on the masking threshold of each Bark subband and the preset mapping function, a sensing weighting factor is generated for each discrete frequency point; wherein, the length of the sensing weighting factor is exactly the same as the number of frequency points of the adaptive time-frequency complex matrix, and each element value is the relative importance weight of the spectral component of the corresponding frequency point in sensing.

[0030] In this embodiment, by non-linearly dividing the linear frequency axis into Bark sub-bands, the differences in human ear sensitivity to different frequencies are simulated (the human ear has higher resolution in the low-frequency region and lower resolution in the high-frequency region). This allows for better focus on the frequency range that the human ear is sensitive to, thereby improving overall auditory quality. The calculation of silence masking thresholds and global masking thresholds directly applies the auditory masking effect. During resampling, spectral components that are not easily perceived by the human ear (i.e., masked components) can be handled more effectively. For these spectral components, appropriate reduction or lower weighting can be applied to reduce unnecessary computational overhead and potential artifacts, while retaining components important to auditory quality. A perceptual weighting factor is generated, assigning a weight based on the relative importance of perception to each discrete frequency point. This allows subsequent spectrum reconstruction and resampling processes to allocate resources "intelligently," giving more attention and higher precision to frequency components that are sensitive and important to the human ear, while coarser processing is applied to insensitive or masked components.

[0031] Specifically, a frequency range is defined, for example, from 0Hz to the Nyquist frequency of the original signal. Then, as shown in Table 2 (the original audio signal sampling rate is 44.1kHz, and after STFT, there are 1024 frequency points per time frame, which are resampled to 48kHz), the linear frequency Hz is converted to a Bark value using the Bark-scale conversion formula (e.g., Bark = 13 × arctan(0.00076f) + 3.5 × arctan((f / 7500)²)). Based on the Bark values, a series of sub-band boundaries with equal Bark widths are determined, for example, dividing the signal into 24 critical bands (Bark Bands). The width of each band is approximately equal on the Bark scale, but on the linear frequency axis, the width of the low-frequency region is narrower and the width of the high-frequency region is wider. For each time frame in the adaptive time-frequency complex matrix, all frequency points are traversed, and all frequency points are assigned to the corresponding Bark sub-band according to the determined Bark sub-band boundaries. Within each Bark subband, the sum of squared amplitudes (or the sum of squared moduli of complex spectral coefficients) of all frequency points belonging to that subband is accumulated. The accumulated result is divided by the number of frequency points (or bandwidth) within the subband to obtain the average power density of that subband, resulting in the power density value for each time frame and each Bark subband. The silence masking threshold is calculated based on a fixed threshold table pre-set according to frequency changes, based on the hearing curve of the human ear in a quiet environment, or calculated using a psychoacoustic model. The global masking threshold is calculated based on the signal power density of the current subband and adjacent subbands. For example, for each Bark subband, its masking threshold is determined by the strongest sound (mask) within that subband, and simultaneous masking (within the same time frame) and asynchronous masking (masking of the current frame by the signal from the previous or subsequent time frame) also need to be considered. The higher of the silence masking threshold and the global masking threshold is taken as the final masking threshold for that subband; that is, a signal must be strong enough to overcome silence masking and strong enough not to be masked by other sounds in order to be perceived by the human ear. The masking threshold (typically energy or sound pressure level) of the Bark subband is mapped to weight values, where higher weights (close to 1) are assigned to frequencies much above the masking threshold, and lower weights (close to 0) are assigned to frequencies close to or below the masking threshold. At frequencies above the masking threshold but not very strong, the weights can transition linearly or non-linearly from low to high values. The output is a perceptual weighting factor vector with the same length as the number of frequency points in the adaptive time-frequency complex matrix, where each element represents the perceptual relative importance weight of the spectral component at the corresponding frequency point.

[0032] Table 2 step describe Example Bark scale division The linear frequency axis is nonlinearly divided into Bark subbands to simulate human ear perception. The original Nyquist frequency is 22.05 kHz. According to the Bark scaling formula, the 0-22.05 kHz range is divided into 24 Bark sub-bands (a list containing 24 frequency range intervals), for example: Sub-band 1: 0-100 Hz (Bark 0-1); Sub-band 2: 100-200 Hz (Bark 1-2); Sub-band 15: 3400-4400 Hz (Bark 14-15, where the bandwidth has significantly increased); Sub-band 24: 15500-22050 Hz (Bark 23-24, with the largest bandwidth). Calculate power density Calculate the power density of the signal within each Bark subband. At a given time frame, the adaptive time-frequency complex matrix contains a complex spectrum of 1024 frequency points. For subband 15 (3400-4400 Hz), it contains 30 discrete frequency points. The sum of the squared moduli (energy) of the 30 frequency points is accumulated and then divided by 30 to obtain the average power density of the subband at that time frame, for example, 0.05 units / Hz. This operation is repeated for all 24 subbands. For each time frame, a vector containing 24 power density values ​​is generated. Calculate the masking threshold The mute threshold and global masking threshold for each subband are calculated based on the power density. The power density of subband 15 is 0.05. The silence masking threshold: Calculated from a table or by looking up a table, the silence masking threshold near 4 kHz (center of subband 15) is approximately 3 dB SPL (approximately 0.002 units). The global masking threshold: Calculated using a psychoacoustic model based on the power density of subband 15 itself (0.05) and the power densities of adjacent subbands (e.g., subbands 14, 16), for example, 0.01 units. The final masking threshold: The maximum of the two, 0.01 units (approximately 10 dB SPL), is taken. This process is repeated for all 24 subbands, generating a vector containing 24 masking thresholds (energy or dB SPL) for each time frame. Generate perceptual weighting factors Weights are generated for each discrete frequency point based on the masking threshold and a preset mapping function. The final masking threshold for subband 15 is 0.01. For the 30 frequency points within this subband, weights are calculated based on the relationship between the specific spectral amplitude and 0.01. If a frequency point is below the threshold, its weight is mapped to 0.1. If a frequency point is significantly above the threshold, its weight is mapped to 0.95. If a frequency point is slightly above the threshold, its weight is mapped to 0.4. For each time frame, a perceptual weighting factor vector of length 1024 is generated, where each element is a floating-point number between 0 and 1.

[0033] In an alternative approach, the weighted spectrum reconstruction further includes: The complex spectral coefficients at each time frame and frequency point in the adaptive time-frequency complex matrix are subjected to complex number operations with the weight values ​​at the corresponding frequency points in the perceptual weighting factor vector to obtain the weighted time-frequency representation, which is then used to perform perceptual domain filtering on the original time-frequency representation of the original audio signal.

[0034] In this embodiment, for example, when converting an audio segment containing human voice and some background noise, each human voice frequency component (e.g., the fundamental frequency of 200Hz) in the original time-frequency complex matrix is ​​multiplied by a perceptual weighting factor close to 1, while the amplitude and phase of the human voice component remain essentially unchanged. The background noise frequency component (e.g., a slight hiss at 3kHz, whose energy is below the masking threshold of human voice in that frequency band) in the original time-frequency complex matrix is ​​multiplied by a small perceptual weighting factor (0.2), significantly reducing the amplitude of the hiss while maintaining its phase. For very quiet frequencies where weak, inaudible noise still exists, the weighting factor may be very close to 0, causing these noise components to almost disappear after weighting. Therefore, the resulting weighted time-frequency representation retains clear human voice information in the ear-sensitive region while effectively suppressing or removing background noise that is insensitive to the ear or masked. When the weighted time-frequency representation is resampled and inversely transformed, the final reconstructed audio signal will have a superior subjective listening experience, with more prominent human voice and less background noise, thus achieving high-quality sampling rate conversion.

[0035] Step S102: Based on the Bark scale, the weighted time-frequency representation and the target sampling rate are divided into frequency sub-bands and sub-bands are independently resampled to obtain the resampled sub-band spectrum.

[0036] In this embodiment, the frequency resolution of the human ear differs across different Bark subbands. Independent subband resampling allows for customized resampling strategies for each subband (e.g., for low-frequency subbands with high human ear resolution, finer interpolation or denser sampling points can be used; for high-frequency subbands with low human ear resolution, the number of sampling points can be appropriately reduced while ensuring the preservation of key perceptual information). Each subband can use different resampling coefficients, and even different filters can be designed for different subbands to cope with various extreme sampling rate conversion scenarios (from extremely high sampling rate to extremely low sampling rate).

[0037] In one alternative approach, dividing the weighted time-frequency representation and the target sampling rate into frequency subbands and performing independent subband resampling based on the Bark scale to obtain the resampled subband spectrum further includes: The nominal Nyquist frequency is determined according to the Nyquist sampling theorem and the target sampling rate. The linear frequency range from zero Hz to the nominal Nyquist frequency is non-uniformly divided into several continuous Bark subbands according to the Bark scale lookup table or analytical formula, wherein each subband corresponds to the critical bandwidth of the cochlea. Based on the mapping position of the Bark subband boundary on the linear frequency axis, the weighted time-frequency representation is divided in the frequency dimension. For each Bark subband, all frequency columns corresponding to it in the weighted time-frequency matrix are extracted to form the subband time-frequency matrix. The frequency axis of the spectrum of each time frame of the sub-band time-frequency matrix is ​​resampled using the sinc function to obtain a set of resampled sub-band spectra.

[0038] In this embodiment, a new Nyquist frequency is determined based on the target sampling rate to ensure that the resampled spectrum does not exceed the effective range, avoiding aliasing that may occur when downsampling high-frequency signals. Using the Sinc function as an interpolation function enables theoretically lossless signal resampling (provided the Nyquist sampling theorem is satisfied). Resampling the frequency axis using the Sinc function in the spectral domain maximizes the preservation of original spectral information, avoids introducing unnecessary harmonics or spurious frequency components, and ensures the smoothness of the resampled spectrum.

[0039] In an alternative approach, resampling the frequency axis of the spectrum of each time frame of the sub-band time-frequency matrix according to the sinc function to obtain the resampled sub-band spectrum set further includes: Process the frequency column vector corresponding to each time frame within the sub-band time-frequency matrix one by one; The frequency column vector is resampled along the frequency axis using the sinc function; the sinc function ensures that the original signal information is preserved to the maximum extent during the interpolation process while avoiding the introduction of spurious frequency components. The new frequency vectors obtained by resampling each time frame through the frequency axis are reorganized back into their respective sub-band structures to form a series of new sub-band time-frequency matrices. The new sub-band time-frequency matrices are aggregated to form a resampled sub-band spectrum set; wherein each element in the sampled sub-band spectrum set is a sub-band spectrum that adapts to the frequency point distribution under the target sampling rate.

[0040] In this embodiment, if the entire spectrum were resampled instead of dividing the subbands, a complex filter covering the entire frequency band would be required, which would be very difficult to optimize and implement. Before entering the resampling stage, the original spectrum has already undergone perceptual weighting processing to remove unimportant or masked spectral components. Based on this, sinc function resampling is performed, resampling only the perceptually optimized information with high fidelity, avoiding the waste of computational resources on auditoryly unimportant parts, and further improving conversion efficiency and sound quality.

[0041] Specifically, for each sub-band time-frequency matrix, all rows (i.e., all time frames) are traversed, and all frequency columns of the current time frame in the current sub-band time-frequency matrix are extracted to form a complex frequency vector (containing the spectral information of that time frame within that specific Bark sub-band). Based on the target sampling rate and the frequency range of the current Bark sub-band, the positions of the new frequency points to be generated within that sub-band are determined. The new frequency points are either denser or sparser than the original frequency points and are usually not perfectly aligned with the original frequency points. For each original frequency point in the frequency vector, its corresponding spectral value is calculated at the new target frequency point. The formula for sinc interpolation is: , in, Indicates the new target frequency point The spectral values ​​obtained after sinc interpolation; It is the number of original frequency points; It is the interval between the original frequency points; Indicates the original frequency point The original spectral value at that location.

[0042] For each resampled time frame, its spectral vector at the new target frequency point replaces the original spectral vector. The updated frequency vectors are then recombined according to their respective Bark subbands and time frames to form a series of new subband time-frequency matrices. The number of columns (frequency points) of the new subband time-frequency matrices may differ from the original subband time-frequency matrices, but the number of rows (time frames) remains unchanged. All the resampled and reorganized subband time-frequency matrices are collected to form a set. Each element in the set is a subband spectrum adapted to the frequency point distribution at the target sampling rate, collectively constituting the resampled time-frequency representation of the entire signal in the perceptual domain and at the target sampling rate. For example, for a target sampling rate of 48kHz, this subband (0-100 Hz) may require new frequency points [25Hz, 50Hz, 75Hz]. The sinc function is used to interpolate the vectors [S(20Hz), S(40Hz), S(60Hz), S(80Hz)] to obtain the spectral values ​​at the three new frequency points [25Hz, 50Hz, 75Hz]. Calculate S(25Hz)S(25Hz) (a weighted sum of sinc values ​​of S(20Hz), S(40Hz), S(60Hz), and S(80Hz)S(80Hz)). Similarly, calculate S(50Hz)S(50Hz) and S(75Hz)S(75Hz). After resampling, the frequency vector of the first time frame becomes [S'(25Hz), S'(50Hz), S'(75Hz)]. Repeat the above process for all time frames to obtain the new time-frequency matrix of the first Bark sub-band adjusted for the 48 kHz sampling rate. Perform the same resampling and reassembly operation on all 24 Bark sub-bands. Finally, all the new sub-band time-frequency matrices will be aggregated to form a resampled sub-band spectrum set, in which the frequency distribution of all sub-band spectra has been adapted to the 48 kHz sampling rate requirement while preserving the perceptually important information of the original signal.

[0043] Step S103: Subband synthesis is performed on the resampled subband spectrum to obtain the target spectrum of the target sampling rate. Inverse short-time Fourier transform is performed on the target spectrum to reconstruct the time-domain audio signal, thereby obtaining the converted audio signal with the target sampling rate.

[0044] In this embodiment, the Inverse Short-Time Fourier Transform (ISTFT) reconstructs the complex time-frequency representation (target spectrum) into a continuous time-domain signal. By overlapping and adding ISTFT, the time-domain signal segments of each frame can be smoothly connected, avoiding artifacts and discontinuities in the reconstructed signal and ensuring the natural and smooth audio. For example, STFT uses 50% overlap, and the same frame shift is used during ISTFT synthesis (e.g., for a frame length of 1024 points, the frame shift is 512 points). The first windowed time-domain signal segment is placed at the origin of the time axis, and the second windowed time-domain signal segment is shifted by 512 sample points to overlap with the first signal segment. In the overlapping region, the corresponding sample values ​​of the two signal segments are added. This overlapping and adding process is repeated for all subsequent signal segments to finally obtain a continuous preliminary time-domain waveform. The average value of all samples of this preliminary time-domain waveform is calculated, and this average value is subtracted from each sample to eliminate the DC component. The final waveform is labeled as an audio signal with a 24kHz sampling rate. This results in a converted audio signal with the same duration as the original audio but a sampling rate of 24kHz. This converted audio signal retains important spectral characteristics perceptible to the human ear and avoids the distortion that may result from simple downsampling.

[0045] In one alternative approach, reconstructing the time-domain audio signal by performing an inverse short-time Fourier transform on the target spectrum to obtain a converted audio signal with the target sampling rate further includes: For each time frame in the target spectrum, the complex values ​​at all its frequency points are extracted sequentially to form an independent frequency vector; The time-domain signal segment of the corresponding time frame is calculated based on the complex values ​​in the frequency vector; wherein each time-domain signal segment contains a series of continuous sample points; The time-domain signal segments obtained by the inverse discrete Fourier transform are overlapped, added together, and then spliced ​​in chronological order to obtain the converted audio signal at the target sampling rate.

[0046] In this embodiment, by performing an inverse discrete Fourier transform on the frequency vector of each time frame, the frequency domain information can be accurately converted back to the time domain, ensuring that the converted audio signal can faithfully reflect its spectral characteristics in the time dimension.

[0047] Specifically, each time-domain signal segment processed by the window function is positioned on the time axis according to the frame shift used in the inverse discrete Fourier transform analysis. For example, if the frame shift is H, the starting position of the nth time-domain signal segment is n×H. The sample values ​​of the overlapping regions of adjacent time-domain signal segments are added together. For example, the end of the nth signal segment overlaps with the beginning of the (n+1)th signal segment, and the sample values ​​of this overlapping region are added point by point. For non-overlapping regions, the sample values ​​of the current signal segment are directly retained. All time-domain signal segments that have undergone overlapping and addition processing are spliced ​​together in chronological order to form a complete and continuous time-domain audio signal, outputting a converted audio signal with the target sampling rate.

[0048] In one alternative approach, the process of overlapping, adding, and synthesizing the time-domain signal segments obtained through inverse discrete Fourier transform, and then splicing them together in chronological order to obtain the converted audio signal at the target sampling rate, further includes: The time-domain signal segment is multiplied sample by sample with the selected synthesis window function, and the time-domain frame obtained after the multiplication is positioned on the time axis according to the preset frame shift, so as to obtain the time-domain frame sequence on the time axis. The sample values ​​of the overlapping regions in the time-domain frame sequence on the time axis are added together to obtain a preliminary time-domain waveform; wherein, for each time index position in the overlapping region, the corresponding sample values ​​of all time-domain frames covering the index position are algebraically added together; for non-overlapping regions, the sample values ​​of the current frame are retained; The initial time-domain waveform is subjected to DC offset correction, the arithmetic mean of the entire signal sequence is calculated, and the arithmetic mean is subtracted from each sample value; the numerical information of the target sampling rate is bound to the initial time-domain waveform, and the converted audio signal of the target sampling rate is output.

[0049] In this embodiment, the overlapping and summing process of the window function and the overlap factor ensures that the amplitude distortion of the reconstructed signal is minimized, thereby guaranteeing that the energy of the original signal in the overlapping region is correctly recovered, avoiding signal energy loss or excessive amplification, which is directly related to the fidelity of the converted audio signal. After a series of spectral processing steps, the signal may introduce a small DC offset. Although this offset may be small, in some cases, especially in the low-frequency range, it may produce unwanted vibrations or "pops" in the power amplifier or speaker. Calculating the arithmetic mean of the entire signal sequence and subtracting this offset ensures that the reconstructed audio signal is symmetrical around the zero mean, further improving signal purity and playback compatibility. Finally, the numerical information of the target sampling rate is bound to the initial time-domain waveform and output, ensuring that the converted audio file not only contains the correct waveform data but also the correct metadata (sampling rate), avoiding usage problems caused by missing information. Figure 2The image shows the waveforms of the received signal before and after resampling. By demodulating the resampled data and comparing it with the demodulated result without resampling during transmission and reception, we can obtain... Figure 3 The bit error rate curve shown is derived from... Figure 3 It can be seen that the bit error rate (BER) after resampling is very close to the theoretical curve. Under the same signal-to-noise ratio (SNR), the BER after resampling is lower than that without resampling. This is because the signal spectrum is compressed at the transmitting end after resampling, reducing noise introduced into the channel; at the receiving end, the spectrum of the useful signal is expanded after the opposite resampling. Therefore, the resampling process can improve the SNR to some extent. Thus, the resampling process in this application can meet the system rate requirements while also helping to reduce the BER.

[0050] According to the scheme provided by the present invention, the original audio signal to be converted is subjected to adaptive multi-resolution time-frequency representation, perceptual weighting factor generation, and weighted spectrum reconstruction to obtain a weighted time-frequency representation; the weighted time-frequency representation and the target sampling rate are divided into frequency sub-bands and independently resampled according to the Bark scale to obtain the resampled sub-band spectrum; the resampled sub-band spectrum is synthesized to obtain the target spectrum of the target sampling rate; the target spectrum is subjected to inverse short-time Fourier transform to reconstruct the time-domain audio signal, resulting in a converted audio signal with the target sampling rate. The present invention significantly improves the sound quality performance of audio sampling rate conversion through weighted time-frequency representation and frequency sub-band division and independent resampling that conform to human hearing characteristics, optimizing the perceptual quality of the converted audio while ensuring signal integrity.

[0051] Figure 4 A schematic diagram of the framework of an audio sampling rate conversion device based on multi-rate signal processing according to an embodiment of the present invention is shown. The audio sampling rate conversion device based on multi-rate signal processing includes: The adaptive perceptual weighted time-frequency representation module 410 is used to perform adaptive multi-resolution time-frequency representation, perceptual weighting factor generation, and weighted spectrum reconstruction on the original audio signal to be converted to obtain a weighted time-frequency representation. The frequency sub-band resampling module 420 is used to perform frequency sub-band division and independent sub-band resampling on the weighted time-frequency representation and the target sampling rate according to the Bark scale, so as to obtain the resampled sub-band spectrum. The target spectrum reconstruction and time-domain synthesis module 430 is used to perform sub-band synthesis on the resampled sub-band spectrum to obtain the target spectrum of the target sampling rate, and to perform inverse short-time Fourier transform on the target spectrum to reconstruct the time-domain audio signal to obtain the converted audio signal with the target sampling rate.

[0052] Figure 5The diagram shows a structural schematic of an embodiment of the computing device of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the computing device.

[0053] like Figure 5 As shown, the computing device may include: a processor 502, a communications interface 504, a memory 506, and a communications bus 508.

[0054] The processor 502, communication interface 504, and memory 506 communicate with each other via communication bus 508. Communication interface 504 is used to communicate with other network elements such as clients or other servers. Processor 502 executes program 510, specifically performing the relevant steps in the above-described embodiment of the audio sampling rate conversion method based on multi-rate signal processing.

[0055] Specifically, program 510 may include program code that includes computer operation instructions.

[0056] Processor 502 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The computing device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0057] Memory 506 is used to store program 510. Memory 506 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0058] According to the scheme provided by the present invention, the original audio signal to be converted is subjected to adaptive multi-resolution time-frequency representation, perceptual weighting factor generation, and weighted spectrum reconstruction to obtain a weighted time-frequency representation; the weighted time-frequency representation and the target sampling rate are divided into frequency sub-bands and independently resampled according to the Bark scale to obtain the resampled sub-band spectrum; the resampled sub-band spectrum is synthesized to obtain the target spectrum of the target sampling rate; the target spectrum is subjected to inverse short-time Fourier transform to reconstruct the time-domain audio signal, resulting in a converted audio signal with the target sampling rate. The present invention significantly improves the sound quality performance of audio sampling rate conversion through weighted time-frequency representation and frequency sub-band division and independent resampling that conform to human hearing characteristics, optimizing the perceptual quality of the converted audio while ensuring signal integrity.

[0059] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination of all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed can be employed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose. Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims listing several devices, several of these devices may be embodied by the same hardware item. Unless otherwise specified, the steps in the above embodiments should not be construed as limiting the order of execution.

Claims

1. An audio sampling rate conversion method based on multi-rate signal processing, characterized in that, include: The original audio signal to be converted is subjected to adaptive multi-resolution time-frequency representation, perceptual weighting factor generation, and weighted spectrum reconstruction to obtain a weighted time-frequency representation; Based on the Bark scale, the weighted time-frequency representation and the target sampling rate are divided into frequency sub-bands and resampled independently in each sub-band to obtain the resampled sub-band spectrum. The resampled subband spectrum is synthesized into a target spectrum with the target sampling rate. The time-domain audio signal is reconstructed by performing an inverse short-time Fourier transform on the target spectrum, resulting in a converted audio signal with the target sampling rate.

2. The audio sampling rate conversion method based on multi-rate signal processing according to claim 1, characterized in that, The adaptive multi-resolution time-frequency representation further includes: Calculate the short-time energy envelope and zero-crossing rate time-domain characteristics of the original audio signal, and select a time-frequency analysis window function based on the short-time energy envelope and zero-crossing rate time-domain characteristics; wherein, in periods of rapid signal transients and fast instantaneous frequency changes, a window function with shorter time support is selected to sacrifice some frequency resolution to obtain more accurate time positioning; in periods of stable signal and stable spectral structure, a window function with longer time support is selected to obtain higher frequency resolution. The original audio signal is subjected to a short-time Fourier transform using the time-frequency analysis window function to obtain an adaptive time-frequency complex matrix; wherein, the row index of the adaptive time-frequency complex matrix corresponds to the time frame, the column index corresponds to the frequency point, and the element value corresponds to the complex spectrum coefficient on the time-frequency unit.

3. The audio sampling rate conversion method based on multi-rate signal processing according to claim 2, characterized in that, The generation of the perceptual weighting factor further includes: The linear frequency axis is nonlinearly divided according to the Bark scale to form a series of subbands corresponding to the critical frequency band of the cochlea; The power density of the signal is calculated in each Bark subband based on the spectral amplitude information contained in the adaptive time-frequency complex matrix. The masking threshold, including the mute masking threshold and the global masking threshold, is calculated for each sub-band based on the power density of the signal calculated within each Bark sub-band. Based on the masking threshold of each Bark subband and the preset mapping function, a sensing weighting factor is generated for each discrete frequency point; wherein, the length of the sensing weighting factor is exactly the same as the number of frequency points of the adaptive time-frequency complex matrix, and each element value is the relative importance weight of the spectral component of the corresponding frequency point in sensing.

4. The audio sampling rate conversion method based on multi-rate signal processing according to claim 3, characterized in that, The weighted spectrum reconstruction further includes: The complex spectral coefficients at each time frame and frequency point in the adaptive time-frequency complex matrix are subjected to complex number operations with the weight values ​​at the corresponding frequency points in the perceptual weighting factor vector to obtain the weighted time-frequency representation, which is then used to perform perceptual domain filtering on the original time-frequency representation of the original audio signal.

5. The audio sampling rate conversion method based on multi-rate signal processing according to claim 4, characterized in that, Based on the Bark scale, the weighted time-frequency representation and the target sampling rate are divided into frequency subbands and independently resampled within each subband to obtain the resampled subband spectrum, which further includes: The nominal Nyquist frequency is determined according to the Nyquist sampling theorem and the target sampling rate. The linear frequency range from zero Hz to the nominal Nyquist frequency is non-uniformly divided into several continuous Bark subbands according to the Bark scale lookup table or analytical formula, wherein each subband corresponds to the critical bandwidth of the cochlea. Based on the mapping position of the Bark subband boundary on the linear frequency axis, the weighted time-frequency representation is divided in the frequency dimension. For each Bark subband, all frequency columns corresponding to it in the weighted time-frequency matrix are extracted to form the subband time-frequency matrix. The frequency axis of the spectrum of each time frame of the sub-band time-frequency matrix is ​​resampled using the sinc function to obtain a set of resampled sub-band spectra.

6. The audio sampling rate conversion method based on multi-rate signal processing according to claim 1, characterized in that, Reconstructing the time-domain audio signal by performing an inverse short-time Fourier transform on the target spectrum to obtain a converted audio signal with the target sampling rate further includes: For each time frame in the target spectrum, the complex values ​​at all its frequency points are extracted sequentially to form an independent frequency vector; The time-domain signal segment of the corresponding time frame is calculated based on the complex values ​​in the frequency vector; wherein each time-domain signal segment contains a series of continuous sample points; The time-domain signal segments obtained by the inverse discrete Fourier transform are overlapped, added together, and then spliced ​​in chronological order to obtain the converted audio signal at the target sampling rate.

7. The audio sampling rate conversion method based on multi-rate signal processing according to claim 5, characterized in that, The frequency axis of the spectrum of each time frame of the sub-band time-frequency matrix is ​​resampled according to the sinc function to obtain the resampled sub-band spectrum set, which further includes: Process the frequency column vector corresponding to each time frame within the sub-band time-frequency matrix one by one; The frequency column vector is resampled along the frequency axis using the sinc function; the sinc function ensures that the original signal information is preserved to the maximum extent during the interpolation process while avoiding the introduction of spurious frequency components. The new frequency vectors obtained by resampling each time frame through the frequency axis are reorganized back into their respective sub-band structures to form a series of new sub-band time-frequency matrices. The new sub-band time-frequency matrices are aggregated to form a resampled sub-band spectrum set; wherein each element in the sampled sub-band spectrum set is a sub-band spectrum that adapts to the frequency point distribution under the target sampling rate.

8. The audio sampling rate conversion method based on multi-rate signal processing according to claim 6, characterized in that, The converted audio signal at the target sampling rate is obtained by overlapping, adding, and synthesizing the time-domain signal segments obtained through inverse discrete Fourier transform, and then splicing them together in chronological order. This process further includes: The time-domain signal segment is multiplied sample by sample with the selected synthesis window function, and the time-domain frame obtained after the multiplication is positioned on the time axis according to the preset frame shift, so as to obtain the time-domain frame sequence on the time axis. The sample values ​​of the overlapping regions in the time-domain frame sequence on the time axis are added together to obtain a preliminary time-domain waveform; wherein, for each time index position in the overlapping region, the corresponding sample values ​​of all time-domain frames covering the index position are algebraically added together; for non-overlapping regions, the sample values ​​of the current frame are retained; The initial time-domain waveform is subjected to DC offset correction, the arithmetic mean of the entire signal sequence is calculated, and the arithmetic mean is subtracted from each sample value; the numerical information of the target sampling rate is bound to the initial time-domain waveform, and the converted audio signal of the target sampling rate is output.

9. An audio sampling rate conversion device based on multi-rate signal processing, characterized in that, The audio sampling rate conversion method based on multi-rate signal processing according to any one of claims 1-8 includes: The adaptive perceptual weighted time-frequency representation module is used to perform adaptive multi-resolution time-frequency representation, perceptual weighting factor generation, and weighted spectrum reconstruction on the original audio signal to be converted, so as to obtain a weighted time-frequency representation. The frequency sub-band resampling module is used to divide the weighted time-frequency representation and the target sampling rate into frequency sub-bands and perform independent sub-band resampling according to the Bark scale to obtain the resampled sub-band spectrum. The target spectrum reconstruction and time-domain synthesis module is used to perform sub-band synthesis on the resampled sub-band spectrum to obtain the target spectrum of the target sampling rate, and to perform inverse short-time Fourier transform on the target spectrum to reconstruct the time-domain audio signal, thereby obtaining the converted audio signal with the target sampling rate.

10. A computing device, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the audio sampling rate conversion method based on multi-rate signal processing according to any one of claims 1-8.

Images