Audio processing method and system, electronic equipment and storage medium

Abstract

The invention provides an audio processing method and system, electronic equipment and a storage medium, and the method comprises the steps: carrying out the blocking processing of a time-frequency feature of a to-be-processed audio, and obtaining a plurality of data blocks; respectively coding the original sampling rate and the original position information of each data block to obtain a sampling rate embedding feature and a position coding feature of each data block; obtaining a target data block sequence according to the position coding feature, the sampling rate embedding feature and each data block; inputting the target data block sequence into a target feature extraction model to obtain audio features of the to-be-processed audio in a target feature space; the target feature extraction model is used for mapping multiple audios with different native sampling rates to a target feature space based on the shared network parameters. According to the method, lossless processing is carried out on the audio signals with the heterogeneous native sampling rate, unified feature representation of physical scale perception is achieved, and the fidelity of audio feature modeling and the capturing capacity for complex acoustic details are improved.

Description

Technical Field

[0001] This invention relates to the field of electrical data processing technology, and in particular to an audio processing method, system, electronic device, and storage medium. Background Technology

[0002] With the development of artificial intelligence technology, there is an increasing demand for unified feature representation of audio signals with variable physical scales in scenarios such as speech recognition and audio analysis, in order to achieve acoustic modeling across scenarios and devices.

[0003] In related technologies, static adaptation schemes based on predefined specifications are typically used to process audio with heterogeneous sampling rates. The core of this approach lies in forcibly unifying the physical scale of the input data during the preprocessing stage by resampling the signal or using fixed-dimensional feature mapping, in order to input audio with different native sampling rates into a neural network with a fixed structure. However, this approach causes irreversible loss of high-frequency information or damage to frequency domain resolution, resulting in a significant decrease in the fidelity of the original audio signal and limiting the model's ability to capture complex acoustic details. Summary of the Invention

[0004] This invention provides an audio processing method, system, electronic device, and storage medium to address the shortcomings of existing technologies that use static adaptation schemes to forcibly unify the physical scale of audio, resulting in the loss of high-frequency information, damage to frequency domain resolution, and a decrease in the fidelity of the original audio signal. It achieves lossless transformation of audio signals with heterogeneous native sampling rates, unified feature representation with physical scale awareness, and improves the fidelity of audio feature modeling and the ability to capture complex acoustic details.

[0005] This invention provides an audio processing method, comprising: The time-frequency features of the audio to be processed are divided into blocks to obtain multiple data blocks; the frequency domain resolution of the time-frequency features is matched with the original sampling rate of the audio to be processed. The original sampling rate and the original position information of each data block in the time-frequency space corresponding to the time-frequency feature are encoded respectively to obtain the sampling rate embedding feature corresponding to the original sampling rate and the position encoding feature of each data block; Based on the location encoding features, the sampling rate embedding features, and each of the data blocks, obtain the target data block sequence; The target data block sequence is input into the target feature extraction model to obtain the audio features of the audio to be processed in the target feature space; the target feature extraction model is used to map multiple audio samples with different native sampling rates to the target feature space based on shared network parameters.

[0006] According to an audio processing method provided by the present invention, for each data block, the step of obtaining the position encoding feature of the data block includes: The embedding space of the target feature extraction model is decoupled into a time-encoding subspace and a frequency-encoding subspace; Based on the time axis coordinates in the original location information of the data block, the time position vector of the data block is calculated in the time encoding subspace; Based on the frequency axis coordinates in the original position information of the data block, the frequency position vector of the data block is calculated in the frequency coding subspace; The time position vector and the frequency position vector are fused to obtain the positional coding features of the data block.

[0007] According to an audio processing method provided by the present invention, obtaining a target data block sequence based on the position encoding feature, the sampling rate embedding feature, and each of the data blocks includes: Based on the location encoding features and the sampling rate embedding features, feature enhancement is performed on each data block to obtain the enhanced data block corresponding to each data block; According to the arrangement order of each data block in the time-frequency space corresponding to the time-frequency feature, the enhanced data blocks corresponding to each data block are serialized and combined to obtain the target data block sequence.

[0008] According to an audio processing method provided by the present invention, the step of performing feature enhancement on each data block based on the position coding feature and the sampling rate embedding feature to obtain an enhanced data block corresponding to each data block includes: Based on the sampling rate embedding features, determine the global features corresponding to the absolute scale of the audio to be processed; Based on the positional coding features of each data block, determine the local features of the relative position of each data block in the time-frequency space; The global features and the local features corresponding to each data block are fused with the content information of each data block to obtain the enhanced data block corresponding to each data block.

[0009] According to an audio processing method provided by the present invention, the step of obtaining the sampling rate embedding feature includes: Based on the original sampling rate of the audio to be processed, the sampling rate embedding feature is retrieved in the sampling rate embedding unit; The sampling rate embedding unit pre-stores mapping information between the native sampling rate of the audio to be processed and the sampling rate embedding feature.

[0010] According to an audio processing method provided by the present invention, the step of obtaining the time-frequency features of the audio to be processed includes: Obtain the time-domain signal of the audio to be processed; Based on the original sampling rate of the audio to be processed, determine the frequency axis parameters corresponding to the time-frequency features; Based on the audio duration of the audio to be processed, determine the time axis parameters corresponding to the time-frequency features; Based on the frequency axis parameters and the time axis parameters, a short-time Fourier transform is performed on the time-domain signal to obtain the time-frequency characteristics.

[0011] According to an audio processing method provided by the present invention, the step of dividing the time-frequency features of the audio to be processed into multiple data blocks includes: Based on the original sampling rate of the audio to be processed and the audio duration of the audio to be processed, calculate the feature area of ​​the time-frequency feature in the time-frequency space; The total number of data blocks to be divided is determined based on the feature area. Based on the total amount of data blocks, the time-frequency features are segmented and flattened to obtain multiple data blocks.

[0012] The present invention also provides an audio processing system, comprising: The flexible front-end module is used to divide the time-frequency features of the audio to be processed into multiple data blocks; the frequency domain resolution of the time-frequency features is matched with the original sampling rate of the audio to be processed. A dual-channel context encoding module is used to encode the original sampling rate and the original position information of each data block in the time-frequency space corresponding to the time-frequency feature, respectively, to obtain the sampling rate embedding feature corresponding to the original sampling rate and the position encoding feature of each data block; The information fusion module is used to obtain a target data block sequence based on the location encoding features, the sampling rate embedding features, and each of the data blocks; The feature representation module is used to input the target data block sequence into the target feature extraction model to obtain the audio features of the audio to be processed in the target feature space; the target feature extraction model is used to map multiple audio samples with different native sampling rates to the target feature space based on shared network parameters.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the audio processing methods described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the audio processing method as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the audio processing method described above.

[0016] The audio processing method, system, electronic device, and storage medium provided by this invention achieve lossless and physically scale-aware unified feature representation of audio signals with heterogeneous native sampling rates through dynamic dimensional block processing combined with dual-channel context coding. This not only solves the problem of information loss caused by resampling in existing technologies and improves the fidelity of audio feature modeling and the ability to capture complex acoustic details, but also enables audio signals with heterogeneous native sampling rates to share the same set of network parameters for flexible and unified representation in the feature space through a unified modeling architecture. While greatly simplifying the model structure, it maintains the stability and controllability of the computational load and improves the processing efficiency and model versatility for heterogeneous audio data. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating an existing audio processing method that utilizes a multi-branch model architecture.

[0019] Figure 2 This is one of the flowcharts of the audio processing method provided by the present invention.

[0020] Figure 3 This is one of the structural schematic diagrams of the audio processing system provided by the present invention.

[0021] Figure 4 This is the second flowchart of the audio processing method provided by the present invention.

[0022] Figure 5 This is the second schematic diagram of the audio processing system provided by the present invention.

[0023] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] In recent years, large-scale pre-trained models have achieved significant success in the field of audio processing. However, existing technologies generally face a significant bottleneck when processing heterogeneous audio data from real-world scenarios: the diverse sources of real-world audio data, with sampling rates ranging from 8kHz (e.g., telephone voice) to 48kHz or even higher (e.g., professional recordings). To input this heterogeneous data into a neural network with a fixed input size, existing technologies typically employ static adaptation schemes based on predefined specifications to process audio with heterogeneous sampling rates. The core of this scheme lies in forcibly unifying the physical scale of the input data during the preprocessing stage by mandating uniform resampling or fixed-dimensional acoustic feature extraction.

[0026] In forced uniform resampling schemes, regardless of the original audio signal's sampling rate (e.g., 8kHz, 32kHz, or 48kHz), digital signal processing algorithms, such as polyphase filter interpolation or decimation algorithms, uniformly resample all original audio signals to a lower, fixed sampling rate, such as 16kHz. This ensures that all resampled audio signals have the same sampling rate, allowing for unified feature extraction and model input. For example, a 48kHz audio signal might be downsampled to 16kHz, and an 8kHz audio signal might be upsampled to 16kHz. This operation directly discards all high-frequency information above the Nyquist frequency, causing irreversible signal loss. Specifically, when a 48kHz high-fidelity audio signal is downsampled to 16kHz, all signal details in the 8kHz to 24kHz frequency band are permanently discarded. For tasks that rely on high-frequency details, such as music, bioacoustics, and industrial fault detection, this loss of high-frequency information is fatal. For example, distinguishing the calls of different birds, identifying subtle overtones in music, or detecting high-frequency abnormal noises caused by early failures in industrial equipment—this loss of high-frequency information limits the performance ceiling of the model from the source, making it impossible to perform true high-fidelity acoustic analysis.

[0027] In fixed-dimensional acoustic feature extraction schemes, the core is to convert time-domain audio signals with different sampling rates into two-dimensional time-domain spectrograms using a Short-Time Fourier Transform (STFT). However, since high-sampling-rate audio generates very high-dimensional frequency domain vectors, to reduce computational complexity and unify the input dimension, this scheme further employs a fixed Mel Filterbank (MFB) to map the two-dimensional time-domain spectrogram generated by the STFT to a low-dimensional space, such as 128 Mel frequency bands, thereby generating a Log-Mel Spectrogram (LMS). For example, regardless of whether the input is a 16kHz (Nyquist frequency 8kHz) or 48kHz (Nyquist frequency 24kHz) two-dimensional time-domain spectrogram, it will ultimately be mapped to the same 128 Mel frequency bands. That is, the final input to the model is this dimensionally compressed, fixed log-Mel spectrogram. Because multiple linear frequency points are crudely merged into a single Mel band in the high-frequency region, the fine structure of the spectrum is severely blurred, and the frequency domain resolution is impaired. This causes the model to be unable to distinguish sounds with subtle differences in similar high-frequency regions, reducing its recognition accuracy.

[0028] In summary, forcibly unifying the physical scale of audio using a static adaptation scheme will lead to the loss of high-frequency information, damage to frequency domain resolution, and a decrease in the fidelity of the original audio signal, thereby limiting the model's ability to capture complex acoustic details.

[0029] Figure 1 This is a flowchart illustrating existing audio processing methods utilizing a multi-branch model architecture. Furthermore, to mitigate the information loss problem in the aforementioned approaches, some studies have attempted to design independent model branches for different sampling rates or sampling rate ranges.

[0030] like Figure 1As shown, in the multi-model branching scheme, a model is assumed to contain one branch specifically for processing 8-16kHz audio, one branch specifically for processing 16-32kHz audio, and one branch specifically for processing 32-48kHz audio. Each branch can have different network parameters (such as convolutional kernel size) to adapt to the corresponding data characteristics. In this scheme, after receiving the input audio, the sampling rate is first determined by the sampling rate determination module; then, based on the determination result, the input audio is routed to an independent network branch with a specific sampling rate range pre-set for processing. Although this scheme can retain more information, it has significant drawbacks: First, structural redundancy, requiring the maintenance of multiple sets of independent or semi-independent network parameters for different sampling rates, increasing the cost of training and inference; second, knowledge fragmentation, as knowledge cannot be shared between different branches, limiting generalization ability; third, poor scalability, as new branches must be added whenever a new sampling rate range needs to be supported, resulting in a rigid architecture and high maintenance costs.

[0031] In summary, the core flaw of existing technologies lies in the fact that they are all static and rigid solutions, failing to find an effective balance between preserving the fidelity of the original information, maintaining computational efficiency, and achieving model uniformity.

[0032] To address the shortcomings of existing technologies, this application provides an audio processing method. This method designs a dynamic, scalable, and flexible front-end processing mechanism that enables lossless representation of arbitrary audio signals with heterogeneous native sampling rates using a unified and concise model, while maintaining stable and controllable computational load. This significantly improves the fidelity of audio feature modeling, the ability to capture complex acoustic details, and the processing efficiency and model universality for heterogeneous audio data, achieving an effective balance between original information fidelity, computational efficiency, and model uniformity.

[0033] Figure 2 This is one of the flowcharts illustrating the audio processing method provided by the present invention; for example... Figure 2 As shown, the execution entity of this method can be an audio processing system, used to generate a high-level, information-rich feature representation in the target feature space for audio signals with arbitrary native sampling rates. This method can be implemented using the Elastic Audio Representation Network (EAR-Net) of the audio processing system, which includes a dual-channel coding network, a fusion network, and a target feature extraction model.

[0034] The system can be an electronic device, a component within an electronic device, an integrated circuit, or a chip. The electronic device can be a mobile electronic device or a non-mobile electronic device. For example, a mobile electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, wearable device, super mobile personal computer, netbook, or personal digital assistant, etc., while a non-mobile electronic device can be a server, network-attached storage, personal computer, etc. This invention does not impose specific limitations.

[0035] like Figure 2 As shown, the method includes steps 210, 220, 230 and 240.

[0036] Step 210: The time-frequency features of the audio to be processed are divided into blocks to obtain multiple data blocks; the frequency domain resolution of the time-frequency features is matched with the original sampling rate of the audio to be processed.

[0037] The audio to be processed here refers to the input digital audio signal with its native sampling rate. The native sampling rate of the audio to be processed refers to the original physical sampling rate of the audio during acquisition or storage. It can cover any other discrete value such as 2kHz, 8kHz, 16kHz, 44.1kHz, 48kHz, etc., without any resampling processing, effectively ensuring that all the acoustic information it contains can enter the subsequent processing stage.

[0038] Figure 3 This is one of the structural schematic diagrams of the audio processing system provided by the present invention.

[0039] like Figure 3 As shown, after obtaining the audio to be processed through the input unit, the lossless spectrum generation unit in the flexible front-end module can be used to extract the time-frequency features of the audio to be processed. The time-frequency features here are the representational features of the audio to be processed in both time and frequency dimensions, such as a two-dimensional time-frequency spectrum, etc., which are not specifically limited in this embodiment. It is worth noting that the frequency domain resolution of the time-frequency features here matches the original sampling rate of the audio to be processed. This means that, compared to the fixed-band compression method using Mel filter banks in the prior art, the time-frequency features in this embodiment are time-frequency domain features without any resampling or fixed band compression. They completely preserve the original frequency domain resolution of the audio to be processed, and their specifications in the frequency dimension are directly determined by their original sampling rate, exhibiting dynamically variable characteristics.

[0040] like Figure 3As shown, after obtaining the time-frequency features of the audio to be processed, the serialization processing unit in the flexible front-end module can divide the time-frequency features of the audio to be processed into blocks to obtain the original data block sequence. This original data block sequence includes multiple data blocks, and also records the original position information of each data block in the time-frequency space corresponding to the time-frequency features in real time. The original position information here refers to the horizontal and vertical coordinates of each data block in the original two-dimensional time-frequency feature map before segmentation.

[0041] The block processing here refers to the operation of dividing the variable-shape time-frequency features into a series of data blocks. These data blocks are the basic information units after segmentation. In this embodiment, they can be data blocks formed by directly dividing according to a fixed size, or they can be data blocks formed by dynamically dividing based on the sampling rate and / or timing length, as well as the total number of data blocks to be divided, determined by the fixed size. This embodiment does not specifically limit this. Since the two-dimensional shape of the time-frequency features is jointly determined by the audio duration and the original sampling rate, for audio with a high sampling rate or long duration, more data blocks will naturally be generated after segmentation, and vice versa. Thus, through block processing, this embodiment achieves an elastic mapping of heterogeneous two-dimensional physical dimensions into one-dimensional sequence lengths, thereby adapting to the subsequent unified modeling architecture.

[0042] Step 220: Encode the original sampling rate and the original position information of each data block in the time-frequency space corresponding to the time-frequency feature to obtain the sampling rate embedding feature corresponding to the original sampling rate and the position encoding feature of each data block.

[0043] like Figure 3 As shown, because block processing flattens two-dimensional spatial data into a one-dimensional sequence, it loses the original two-dimensional spatial location information and absolute physical scale information. Therefore, this embodiment introduces a dual-channel context encoding module. This dual-channel context encoding module is responsible for injecting two crucial pieces of information—relative position and absolute scale—into the contextless data blocks through a dual-channel encoder. The dual-channel context encoding module consists of two parallel units: a dynamic time-frequency dual position encoding unit (also called a position encoding unit) and a sampling rate embedding unit. The dynamic time-frequency dual position encoding unit is used to recover the relative spatial position of each data block in the time-frequency space corresponding to its respective time-frequency feature through the position encoder in the dual-channel encoder. The sampling rate embedding unit aims to inject global features of absolute scale into each data block through the scale encoder in the dual-channel encoder, so that the target feature extraction model can perceive the physical scale of the audio being processed.

[0044] like Figure 3As shown, during the encoding process, a sampling rate embedding unit can be used to perform physical scale-aware context encoding on the native sampling rate to obtain the sampling rate embedding feature corresponding to the native sampling rate. A dynamic time-frequency dual position encoding unit can then perform position-aware context encoding on the original position information of each data block in the time-frequency space corresponding to the time-frequency feature to obtain the position encoding feature of each data block. The position encoding feature is used to recover the relative positional relationship of each data block in the time-frequency space; the sampling rate embedding feature refers to the global offset feature characterizing the absolute physical scale of the current audio, which can inform the subsequent model at what sampling resolution the data being processed was generated, thus giving the model physical scale awareness.

[0045] The encoding of the original sampling rate and the encoding of the original location information can be implemented in the same way or in different ways. For example, both can be dynamically encoded by a neural network model, or the encoding of the original sampling rate can be implemented by mapping through a pre-built mapping relationship, while the encoding of the original location information can be dynamically encoded by a neural network model. This embodiment does not make specific limitations on this.

[0046] Step 230: Obtain the target data block sequence based on the location encoding feature, the sampling rate embedding feature, and each data block.

[0047] like Figure 3 As shown, after obtaining the sampling rate embedding features and the positional encoding features of each data block, the information fusion module can use a fusion model to jointly generate the target data block sequence by combining the sampling rate embedding features, the positional encoding features of each data block, and the serialization of each data block. During this process, the sampling rate embedding features and the positional encoding features of each data block can be used to enhance the features of each data block, thereby serializing the enhanced data blocks corresponding to each data block to form the target data block sequence. Feature enhancement here includes, but is not limited to, fusion operations such as element addition or concatenation. Each element (enhanced data block) in the target data block sequence is a context-enhanced data block, fully restoring and enhancing the contextual information of the original serialized data block, avoiding information loss problems caused by serialization. Specifically, the enhanced data block not only contains the original acoustic content information of the audio to be processed, but also integrates rich contextual features, that is, it integrates physical resolution constraints (i.e., sampling rate embedding features) and time-frequency structure information (i.e., positional encoding features).

[0048] Step 240: Input the target data block sequence into the target feature extraction model to obtain the audio features of the audio to be processed in the target feature space; the target feature extraction model is used to map multiple audio samples with different native sampling rates to the target feature space based on shared network parameters.

[0049] like Figure 3 As shown, after obtaining the target data block sequence, it can be input into the target feature extraction model in the feature representation module. The target feature extraction model extracts features from the target data block sequence in the target feature space and outputs the audio features (i.e., audio elastic representation) of the audio to be processed in the target feature space through the output unit. Here, the target feature extraction model refers to a deep learning model that natively supports variable-length sequence modeling capabilities, such as an encoder based on the Transformer architecture (also known as a Transformer encoder).

[0050] For example, when the target feature extraction model is a Transformer encoder, the target data block sequence can be input into the Transformer encoder. The Transformer encoder can then use its self-attention mechanism to perform deep modeling on this variable-length target data block sequence to capture the complex dependencies between the modules in the target data block sequence, and thus output the audio features of the audio to be processed in the target feature space. This audio feature is a high-level feature representation that not only retains all the information of the audio to be processed, but also incorporates the target feature extraction model's deep understanding of the time-frequency structure and physical scale of the audio to be processed, that is, a deep understanding of different acoustic scenes and physical scales.

[0051] It should be noted that, due to its inherent self-attention (SA) mechanism, this target feature extraction model possesses the ability to model variable-length sequence data. Therefore, in practical applications, this target feature extraction model can adaptively receive and process target data block sequences of different lengths generated from audio with different native sampling rates. Specifically, the target feature extraction model, through its internal Transformer architecture, performs global dependency modeling on each element in the target data block sequence and can ignore differences in absolute sequence length, seamlessly mapping heterogeneous input data to the target feature space. Within the target feature space, regardless of the original audio sampling rate, the generated audio features are aligned to a unified feature dimension and semantic representation dimension, thereby achieving a normalized representation of acoustic features.

[0052] Based on this, since audio features possess high specification consistency and semantic compatibility within the target feature space, they can serve as a universal acoustic representation and be directly input into subsequent neural networks with fixed input parameter specifications to perform various downstream audio processing tasks. These downstream tasks include, but are not limited to: speech recognition tasks that convert acoustic signals into text sequences, speech emotion recognition tasks that identify emotional states contained in audio, audio event detection tasks that monitor and classify environmental sound signals, and industrial equipment fault diagnosis tasks that determine equipment operating status by analyzing mechanical vibration acoustic signals, etc. This embodiment does not specifically limit these tasks.

[0053] As can be seen from the above, through this unified feature space mapping mechanism, this embodiment not only eliminates the modeling difficulties caused by the heterogeneity of audio physical scales, but also enables the same set of model parameters to learn common acoustic features across sampling rates through large-scale heterogeneous data training, which significantly enhances the system's generalization performance and robustness in complex and ever-changing application scenarios.

[0054] The method provided in this invention, through dynamic dimensional block processing combined with dual-channel context coding, achieves lossless and physically scale-aware unified feature representation of audio signals with heterogeneous native sampling rates. This not only solves the problem of information loss caused by resampling in existing technologies and improves the fidelity of audio feature modeling and the ability to capture complex acoustic details, but also enables audio signals with heterogeneous native sampling rates to share the same set of network parameters for flexible and unified representation in the feature space through a unified modeling architecture. While greatly simplifying the model structure, it maintains the stability and controllability of the computational load and improves the processing efficiency and model versatility for heterogeneous audio data.

[0055] In some embodiments, for each data block, the step of obtaining the location encoding features of the data block includes: First, the embedding space of the target feature extraction model is decoupled into a time-encoding subspace and a frequency-encoding subspace.

[0056] The embedding space here refers to the high-dimensional vector space used by the target feature extraction model to represent feature vectors. The decoupling refers to the operation of splitting the total dimension of the embedding space into two independent sets of dimensions, used to encode temporal and frequency information respectively. Through this subspace decoupling mechanism, the target feature extraction model can independently handle the temporal topology and frequency scales during feature learning, thereby improving the accuracy of spatial representation.

[0057] Secondly, based on the time axis coordinates in the original location information of the data block, the time position vector of the data block is calculated in the time encoding subspace; based on the frequency axis coordinates in the original location information of the data block, the frequency position vector of the data block is calculated in the frequency encoding subspace.

[0058] Optionally, for each data block, the time position vector of the data block can be dynamically calculated in the time coding subspace based on the time axis coordinates in the original position information of the data block, and the frequency position vector of the data block can be dynamically calculated in the frequency coding subspace based on the frequency axis coordinates in the original position information of the data block. Here, calculation refers to the process of dynamically generating vectors using a preset coordinate function; during the calculation process, it can be implemented through a position encoder, which can be constructed using a non-learned periodic function or a neural network, such as a multi-layer perceptron (MLP), to construct a non-linear mapping model. This embodiment does not specifically limit this approach.

[0059] Finally, the time position vector and the frequency position vector are fused to obtain the positional coding features of the data block.

[0060] The fusion here includes, but is not limited to, vector addition or vector concatenation operations. The positional encoding features produced by the fusion contain the precise relative spatial location information of the data block in the time-frequency space corresponding to the time-frequency features.

[0061] It should be noted that this dynamic calculation method does not rely on a fixed lookup table, and therefore can adaptively generate matching position representations (i.e., positional encoding features) for audio data with arbitrary native sampling rates and arbitrary durations.

[0062] The method provided in this embodiment solves the technical problem of losing two-dimensional topological information after audio signal serialization by decoupling the embedding space and dynamically calculating the time-frequency dual position encoding. It achieves accurate recovery of the positional relationship of time-frequency diagrams of different shapes under a unified architecture, and significantly enhances the model's ability to perceive the intrinsic spatial structure of audio signals.

[0063] like Figure 3 As shown, data streams from different processing paths converge at the information fusion module. Therefore, step 230 specifically includes: Step 231: Based on the location encoding features and the sampling rate embedding features, perform feature enhancement on each data block to obtain the enhanced data block corresponding to each data block.

[0064] Feature enhancement here refers to the process of injecting physical dimension features into semantic features. In this embodiment, feature enhancement can be achieved through operations such as element-wise addition or feature concatenation. Specifically, the sampling rate embedding feature representing the absolute physical scale and the position encoding feature representing the relative topological position can be fused with the feature vector of each data block to generate the enhanced data block corresponding to each data block. The enhanced data block here is a representation unit enhanced by context, which simultaneously contains the original acoustic information, relative spatial position information, and absolute sampling rate scale information.

[0065] Step 232: According to the arrangement order of each data block in the time-frequency space corresponding to the time-frequency feature, the enhanced data blocks corresponding to each data block are serialized and combined to obtain the target data block sequence.

[0066] The arrangement order here refers to the logical scanning order of each enhanced data block in the time-frequency space corresponding to the time-frequency characteristics of the audio to be processed, either from left to right or from top to bottom, or according to a time-axis-first sorting rule. The serialization combination here refers to the operation of arranging the scattered enhanced data blocks into a one-dimensional vector sequence according to the corresponding arrangement order.

[0067] The method provided in this embodiment transforms the sampling rate heterogeneity problem into a variable-length sequence modeling problem by performing deep fusion and flexible serialization of multi-dimensional information on data blocks. This enables the generated input sequence to possess complete physical scale information and time-frequency topological information, laying a data foundation for a unified model to produce high-fidelity, cross-scale audio representations.

[0068] In some embodiments, step 231 specifically includes: Based on the sampling rate embedding features, determine the global features corresponding to the absolute scale of the audio to be processed; Based on the positional coding features of each data block, determine the local features of the relative position of each data block in the time-frequency space; The global features and the local features corresponding to each data block are fused with the content information of each data block to obtain the enhanced data block corresponding to each data block.

[0069] Optionally, after obtaining the sampling rate embedding features and the positional encoding features of each data block, in order to enable the target feature extraction model to globally perceive the current physical resolution, the global features corresponding to the absolute scale of the audio to be processed, determined by the sampling rate embedding features, can be broadcast to the embedding features added to each data block. Broadcasting refers to copying the global feature vector and applying it to all data blocks, thereby giving each data block the same physical scale background information. In this process, the sampling rate embedding features can be directly used as the global features; or the global features can be obtained by aligning the sampling rate embedding features with the content information, specifically determined based on the spatial dimension matching relationship between the sampling rate embedding features and the content information of the data blocks.

[0070] Furthermore, to recover the spatial topology lost after data block serialization, the local features of the relative positions of each data block in the time-frequency space, determined by the positional coding features of each data block, can be added to the embedding features of each data block. In this process, the positional coding features of each data block can be directly used as local features; or the positional coding features of each data block can be aligned with the content information of each data block to obtain local features, depending on the spatial dimension matching relationship between the positional coding features and the corresponding content information.

[0071] Subsequently, the global and local features embedded in each data block are fused with the content information of each data block to obtain the augmented data block corresponding to each data block. Specific fusion processing can be achieved through element-wise addition or feature concatenation.

[0072] The method provided in this embodiment injects global features representing absolute scale and local features representing relative position into the content features of data blocks in a coordinated manner, so that each enhanced data block has complete acoustic semantics, spatial structure and physical scale information. This solves the modeling ambiguity problem caused by the lack of physical scale when the neural network processes variable-length sequences, and significantly improves the fidelity of audio feature representation and the model's perception accuracy of heterogeneous data.

[0073] In some embodiments, the step of obtaining the sampling rate embedding feature includes: Based on the original sampling rate of the audio to be processed, the sampling rate embedding feature is retrieved in the sampling rate embedding unit; like Figure 3As shown, the sampling rate embedding unit is responsible for injecting absolute physical scale information into the target feature extraction model. It pre-stores the mapping information between the native sampling rate of the audio to be processed and the sampling rate embedded features. Specifically, the sampling rate embedding unit pre-constructs a learnable lookup table, which stores multiple sets of preset discrete sampling rate values, such as 8kHz, 16kHz, 32kHz, 44.1kHz, and 48kHz, as well as a global physical scale vector corresponding to each sampling rate value.

[0074] During the retrieval process, the native sampling rate of the audio to be processed is used to search within the sampling rate embedding unit by index matching. This extracts the global physical scale vector that matches the native sampling rate of the audio to be processed and uses it as the sampling rate embedding feature of the audio to be processed.

[0075] The method provided in this embodiment realizes the continuous feature representation of discrete physical scale information by constructing a sampling rate embedding unit based on a learnable mapping relationship. This enables the target feature extraction model to obtain the absolute physical scale information of the audio signal with extremely low online computational overhead, significantly enhancing the system's adaptive perception capability for heterogeneous audio data with multiple sampling rates.

[0076] In some embodiments, the step of obtaining the time-frequency characteristics of the audio to be processed can be specifically implemented by the lossless spectrum generation unit by performing the following steps: First, the time-domain signal of the audio to be processed is obtained.

[0077] The time-domain signal here refers to the original digital sequence of audio in terms of amplitude variation over time. Specifically, it can be acquired in real time through audio pickup devices such as microphones, or by reading existing audio files from storage media such as flash memory or hard drives.

[0078] Secondly, based on the native sampling rate of the audio to be processed, the frequency axis parameter corresponding to the time-frequency feature is determined; based on the audio duration of the audio to be processed, the time axis parameter corresponding to the time-frequency feature is determined.

[0079] The frequency axis parameter here determines the number of discrete points or resolution of the spectrum along the frequency axis. In this embodiment, the frequency axis parameter is positively correlated with the native sampling rate. This dynamically determined mechanism ensures that high-sampling-rate audio can be mapped to higher frequency dimensions, thereby fully preserving all frequency components below the Nyquist frequency.

[0080] The time axis parameter here determines the number of frames of the spectrum in the time axis direction, which is determined by the audio duration combined with the preset frame shift parameter.

[0081] Finally, based on the frequency axis parameters and the time axis parameters, a short-time Fourier transform is performed on the time-domain signal to obtain the time-frequency characteristics.

[0082] The time-frequency characteristics here refer to the two-dimensional time-frequency spectrum that characterizes the distribution of audio energy on the time and frequency axes.

[0083] As can be seen from the above, by dynamically configuring the calculation specifications of the short-time Fourier transform, such as the number of points in the fast Fourier transform, through frequency axis parameters and time axis parameters, an original two-dimensional spectrum diagram whose shape dynamically changes with the sampling rate and duration can be produced.

[0084] The method provided in this embodiment achieves precise matching between frequency domain resolution and physical sampling rate from the source by dynamically determining the axis parameters of time-frequency transformation based on the original sampling rate. This completely solves the problem of high-frequency detail loss and spectrum blurring caused by resampling or fixed-dimensional mapping in existing technical solutions, and achieves fidelity in audio feature modeling.

[0085] In some embodiments, step 210 may be implemented by the serialization processing unit performing the following steps: First, based on the original sampling rate of the audio to be processed and the audio duration of the audio to be processed, the feature area of ​​the time-frequency feature in the time-frequency space is calculated.

[0086] The feature area here refers to the total number of scalar points covered by the time-frequency features of the audio to be processed. Since the frequency axis parameter is positively correlated with the native sampling rate and the time axis parameter is positively correlated with the audio duration, the feature area obtained from this can comprehensively reflect the total amount of physical information of the audio to be processed.

[0087] Secondly, the total number of data blocks to be divided is determined based on the feature area.

[0088] Optionally, the feature area can be divided by the size of a preset sampling window to obtain the total number of data blocks to be divided. This allows for the automatic determination of a larger total number of data blocks for high sampling rates or long durations of audio. Here, the total number of data blocks refers to the total number of independent basic units after the entire time-frequency feature is divided.

[0089] Finally, based on the total amount of data blocks, the time-frequency features are segmented and flattened to obtain multiple data blocks.

[0090] In this process, the data can be slidably segmented on the time-frequency features according to the total amount of data to be divided, thereby extracting multiple local blocks, and flattening each local block from a two-dimensional tensor into a one-dimensional feature vector. Through this process, the heterogeneous two-dimensional physical scale (variable height and width) can be successfully transformed into a one-dimensional elastic sequence that can be uniformly received by the target feature extraction model.

[0091] The method provided in this embodiment realizes a technical closed loop for transforming heterogeneous audio challenges into variable-length sequence processing by dynamically calculating and performing block and flattening operations based on feature area. This ensures a reasonable fit between computational load and audio physical scale, and provides a standardized input format for elastic modeling under a unified architecture, thereby improving the flexibility and processing efficiency of audio elastic representation.

[0092] To further illustrate the effectiveness of the method provided in this application, the method provided in this application will be compared with the method provided in the prior art.

[0093] like Figure 1 As shown, in the multi-branch model architecture scheme of related technologies, after receiving the input audio, the system must perform conditional distribution through a sampling rate judgment module. The core logic of this scheme is to design and maintain multiple independent or semi-independent network branches for different sampling rate ranges. For example, branch 1 uses independent network A to specifically handle audio with a native sampling rate of 8kHz to 16kHz, branch 2 uses independent network B to specifically handle audio with a native sampling rate of 16kHz to 32kHz, and branch 3 uses independent network C to specifically handle audio with a native sampling rate of 32kHz to 48kHz. Each branch has a corresponding independent network with different network parameters (such as convolutional kernel size or linear layer weights) to adapt to the corresponding data characteristics.

[0094] However, this multi-branch architecture has obvious inherent flaws: First, structural redundancy: the device needs to maintain an independent set of parameters for each preset sampling rate range, resulting in a bloated overall model size and a significant increase in storage and computational burden.

[0095] Second, knowledge fragmentation: Because the parameters of different branches are isolated, the model cannot learn within a unified feature space. For example, high-frequency detail knowledge learned from 48kHz audio cannot be effectively shared with the branch processing 8kHz audio, limiting the model's global generalization ability.

[0096] Third, poor scalability: The architecture is significantly rigid. Whenever a new sampling rate range needs to be supported, a new model branch must be redesigned and trained, resulting in an inflexible architecture and high maintenance costs.

[0097] Figure 4 This is the second flowchart of the audio processing method provided by the present invention.

[0098] In stark contrast, such as Figure 4As shown, the method provided in this application adopts a unified elastic architecture. Specifically, the method provided in this application abandons all complex conditional judgments and redundant branches, and adopts a concise and efficient linear processing pipeline. Its core lies in the lossless serialization of audio input with arbitrary sampling rates through an elastic front-end module, and the enhancement of each data block generated by the serialization through a dual-channel context encoding module to obtain the enhanced data block sequence (i.e., the target data block sequence). After the elastic adaptive processing of the front end, the target data block sequences corresponding to all audio signals, regardless of their original sampling rates, are fed into a unified target feature extraction model for deep modeling, and finally output the normalized representation of audio signals with different original sampling rates in the target feature space. This target feature space is a semantic space shared by multiple audio signals with different original sampling rates. This means that no matter what the original physical scale of the input signal is, the final audio features are uniform in feature dimension and have the ability to perceive physical scale, which can be directly used for various downstream tasks such as audio recognition, classification or generation.

[0099] Therefore, compared with multi-branch architecture, this application has the following technical advantages: First, existing technologies require the design and maintenance of multiple independent or semi-independent model branches for different sampling rate ranges, resulting in bloated model structures, a significant increase in the number of parameters, and high training and maintenance costs. The method provided in this application, however, eliminates the need for any conditional distribution routes or dedicated sampling rate branches. Instead, it employs a unified, linear processing pipeline. Regardless of whether the native sampling rate of the input audio is 8kHz, 16kHz, or 48kHz, all data streams are modeled using the same target feature extraction model. This not only makes the model more concise and elegant but also greatly improves parameter efficiency and reduces deployment and maintenance complexity.

[0100] Second, in existing technologies, model parameters are isolated between different branches, making it difficult to effectively transfer and share knowledge learned from low-sampling-rate data (such as basic sound patterns) to branches processing high-sampling-rate data, or vice versa. The method provided in this application, however, extracts features from all heterogeneous sampling-rate data within the same model, enabling deep semantic alignment and complementarity of acoustic knowledge at different physical scales within the target feature space. For example, the model can simultaneously utilize long-term semantic information from low-sampling-rate data and fine frequency details from high-sampling-rate data. For instance, high-frequency detail knowledge learned from 48kHz audio can complement and enhance long-term semantic knowledge learned from 8kHz audio, thereby improving the model's overall generalization ability across all sampling rates.

[0101] Third, the architecture of existing technologies is rigid. Whenever a new sampling rate range needs to be supported, a new model branch must be redesigned, added, and trained, resulting in huge engineering costs. In contrast, the method provided in this application, thanks to dynamic temporal adjustment and dynamic positional encoding, has an architecture that is flexible and dynamically scalable. When a new sampling rate is introduced, the system can automatically calculate the corresponding input duration and generate matching positional codes without any modification to the model structure, achieving seamless, zero-cost expansion and exhibiting strong adaptability.

[0102] In summary, the audio processing method provided in this application transforms the heterogeneous physical scale challenge into a variable-length sequence processing problem that transformer architecture is natively adept at. It is supplemented by sampling rate embedding features and spatial encoding features that can characterize the absolute physical scale. This enables a single model to represent audio with infinite sampling rates in a lossless and unified manner. It completely eliminates model redundancy and knowledge fragmentation while improving the fidelity of audio feature modeling. As a result, it significantly improves the efficiency, generalization and scalability of audio feature extraction, while enhancing the ability to capture complex acoustic details.

[0103] The audio processing system provided by the present invention is described below. The audio processing system described below can be referred to in correspondence with the audio processing method described above.

[0104] Figure 5 This is a second schematic diagram of the audio processing system provided by the present invention; as shown below. Figure 5 As shown, the system includes: The flexible front-end module 510 is used to divide the time-frequency features of the audio to be processed into blocks to obtain multiple data blocks; the frequency domain resolution of the time-frequency features is matched with the original sampling rate of the audio to be processed. The dual-channel context coding module 520 is used to encode the original sampling rate and the original position information of each data block in the time-frequency space corresponding to the time-frequency feature, respectively, to obtain the sampling rate embedding feature corresponding to the original sampling rate and the position coding feature of each data block; The information fusion module 530 is used to obtain a target data block sequence based on the location encoding features, the sampling rate embedding features, and each of the data blocks; The feature representation module 540 is used to input the target data block sequence into the target feature extraction model to obtain the audio features of the audio to be processed in the target feature space; the target feature extraction model is used to map multiple audios with different native sampling rates to the target feature space based on shared network parameters.

[0105] The system provided in this embodiment achieves lossless and physically scale-aware unified feature representation of audio signals with heterogeneous native sampling rates by combining dynamic dimension block processing with dual-channel context coding. This not only solves the problem of information loss caused by resampling in the prior art and improves the fidelity of audio feature modeling and the ability to capture complex acoustic details, but also enables audio signals with heterogeneous native sampling rates to share the same set of network parameters for flexible and unified representation in the feature space through a unified modeling architecture. While greatly simplifying the model structure, it maintains the stability and controllability of the computational load and improves the processing efficiency and model versatility for heterogeneous audio data.

[0106] In some embodiments, for each data block, the step of obtaining the location encoding features of the data block includes: The embedding space of the target feature extraction model is decoupled into a time-encoding subspace and a frequency-encoding subspace; Based on the time axis coordinates in the original location information of the data block, the time position vector of the data block is calculated in the time encoding subspace; Based on the frequency axis coordinates in the original position information of the data block, the frequency position vector of the data block is calculated in the frequency coding subspace; The time position vector and the frequency position vector are fused to obtain the positional coding features of the data block.

[0107] In some embodiments, obtaining the target data block sequence based on the location encoding feature, the sampling rate embedding feature, and each of the data blocks includes: Based on the location encoding features and the sampling rate embedding features, feature enhancement is performed on each data block to obtain the enhanced data block corresponding to each data block; According to the arrangement order of each data block in the time-frequency space corresponding to the time-frequency feature, the enhanced data blocks corresponding to each data block are serialized and combined to obtain the target data block sequence.

[0108] In some embodiments, the step of performing feature enhancement on each data block based on the location encoding feature and the sampling rate embedding feature to obtain an enhanced data block corresponding to each data block includes: Based on the sampling rate embedding features, determine the global features corresponding to the absolute scale of the audio to be processed; Based on the positional coding features of each data block, determine the local features of the relative position of each data block in the time-frequency space; The global features and the local features corresponding to each data block are fused with the content information of each data block to obtain the enhanced data block corresponding to each data block.

[0109] In some embodiments, the step of obtaining the sampling rate embedding feature includes: Based on the original sampling rate of the audio to be processed, the sampling rate embedding feature is retrieved in the sampling rate embedding unit; The sampling rate embedding unit pre-stores mapping information between the native sampling rate of the audio to be processed and the sampling rate embedding feature.

[0110] In some embodiments, the step of obtaining the time-frequency features of the audio to be processed includes: Obtain the time-domain signal of the audio to be processed; Based on the original sampling rate of the audio to be processed, determine the frequency axis parameters corresponding to the time-frequency features; Based on the audio duration of the audio to be processed, determine the time axis parameters corresponding to the time-frequency features; Based on the frequency axis parameters and the time axis parameters, a short-time Fourier transform is performed on the time-domain signal to obtain the time-frequency characteristics.

[0111] In some embodiments, the step of dividing the time-frequency features of the audio to be processed into multiple data blocks includes: Based on the original sampling rate of the audio to be processed and the audio duration of the audio to be processed, calculate the feature area of ​​the time-frequency feature in the time-frequency space; The total number of data blocks to be divided is determined based on the feature area. Based on the total amount of data blocks, the time-frequency features are segmented and flattened to obtain multiple data blocks.

[0112] The system provided by this invention is used to execute the above-described method embodiments. For specific processes and details, please refer to the above embodiments, which will not be repeated here.

[0113] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communications bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other through the communications bus 640. The processor 610 can call logical instructions in the memory 630 to execute an audio processing method, which includes: dividing the time-frequency features of the audio to be processed into blocks to obtain multiple data blocks; the frequency domain resolution of the time-frequency features is matched with the original sampling rate of the audio to be processed; encoding the original sampling rate and the original position information of each data block in the time-frequency space corresponding to the time-frequency features to obtain the sampling rate embedding feature corresponding to the original sampling rate and the position encoding feature of each data block; obtaining a target data block sequence based on the position encoding feature, the sampling rate embedding feature and each data block; inputting the target data block sequence into a target feature extraction model to obtain the audio features of the audio to be processed in the target feature space; the target feature extraction model is used to map multiple audios with different original sampling rates to the target feature space based on shared network parameters.

[0114] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0115] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the audio processing methods provided by the above methods. The method includes: dividing the time-frequency features of the audio to be processed into blocks to obtain multiple data blocks; matching the frequency domain resolution of the time-frequency features with the original sampling rate of the audio to be processed; encoding the original sampling rate and the original position information of each data block in the time-frequency space corresponding to the time-frequency features to obtain the sampling rate embedding feature corresponding to the original sampling rate and the position encoding feature of each data block; obtaining a target data block sequence based on the position encoding feature, the sampling rate embedding feature and each data block; inputting the target data block sequence into a target feature extraction model to obtain the audio features of the audio to be processed in the target feature space; the target feature extraction model is used to map multiple audios with different original sampling rates to the target feature space based on shared network parameters.

[0116] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the audio processing method provided by the above methods. The method includes: dividing the time-frequency features of the audio to be processed into blocks to obtain multiple data blocks; the frequency domain resolution of the time-frequency features is matched with the original sampling rate of the audio to be processed; encoding the original sampling rate and the original position information of each data block in the time-frequency space corresponding to the time-frequency features, respectively, to obtain a sampling rate embedding feature corresponding to the original sampling rate and a position encoding feature of each data block; obtaining a target data block sequence based on the position encoding feature, the sampling rate embedding feature, and each data block; inputting the target data block sequence into a target feature extraction model to obtain the audio features of the audio to be processed in a target feature space; the target feature extraction model is used to map multiple audio samples with different original sampling rates to the target feature space based on shared network parameters.

[0117] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0118] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An audio processing method, characterized in that, include: The time-frequency features of the audio to be processed are divided into blocks to obtain multiple data blocks; the frequency domain resolution of the time-frequency features is matched with the original sampling rate of the audio to be processed. The original sampling rate and the original position information of each data block in the time-frequency space corresponding to the time-frequency feature are encoded respectively to obtain the sampling rate embedding feature corresponding to the original sampling rate and the position encoding feature of each data block; Based on the location encoding features, the sampling rate embedding features, and each of the data blocks, obtain the target data block sequence; The target data block sequence is input into the target feature extraction model to obtain the audio features of the audio to be processed in the target feature space; The target feature extraction model is used to map multiple audio samples with different native sampling rates to the target feature space based on shared network parameters.

2. The audio processing method according to claim 1, characterized in that, For each data block, the steps for obtaining the location encoding features of the data block include: The embedding space of the target feature extraction model is decoupled into a time-encoding subspace and a frequency-encoding subspace; Based on the time axis coordinates in the original location information of the data block, the time position vector of the data block is calculated in the time encoding subspace; Based on the frequency axis coordinates in the original position information of the data block, the frequency position vector of the data block is calculated in the frequency coding subspace; The time position vector and the frequency position vector are fused to obtain the positional coding features of the data block.

3. The audio processing method according to claim 1, characterized in that, The step of obtaining the target data block sequence based on the location encoding feature, the sampling rate embedding feature, and each of the data blocks includes: Based on the location encoding features and the sampling rate embedding features, feature enhancement is performed on each data block to obtain the enhanced data block corresponding to each data block; According to the arrangement order of each data block in the time-frequency space corresponding to the time-frequency feature, the enhanced data blocks corresponding to each data block are serialized and combined to obtain the target data block sequence.

4. The audio processing method according to claim 3, characterized in that, The step of performing feature enhancement on each data block based on the location encoding feature and the sampling rate embedding feature to obtain the enhanced data block corresponding to each data block includes: Based on the sampling rate embedding features, determine the global features corresponding to the absolute scale of the audio to be processed; Based on the positional coding features of each data block, determine the local features of the relative position of each data block in the time-frequency space; The global features and the local features corresponding to each data block are fused with the content information of each data block to obtain the enhanced data block corresponding to each data block.

5. The audio processing method according to any one of claims 1-4, characterized in that, The steps for obtaining the sampling rate embedding features include: Based on the original sampling rate of the audio to be processed, the sampling rate embedding feature is retrieved in the sampling rate embedding unit; The sampling rate embedding unit pre-stores mapping information between the native sampling rate of the audio to be processed and the sampling rate embedding feature.

6. The audio processing method according to any one of claims 1-4, characterized in that, The steps for obtaining the time-frequency features of the audio to be processed include: Obtain the time-domain signal of the audio to be processed; Based on the original sampling rate of the audio to be processed, determine the frequency axis parameters corresponding to the time-frequency features; Based on the audio duration of the audio to be processed, determine the time axis parameters corresponding to the time-frequency features; Based on the frequency axis parameters and the time axis parameters, a short-time Fourier transform is performed on the time-domain signal to obtain the time-frequency characteristics.

7. The audio processing method according to any one of claims 1-4, characterized in that, The step of dividing the time-frequency features of the audio to be processed into multiple data blocks includes: Based on the original sampling rate of the audio to be processed and the audio duration of the audio to be processed, calculate the feature area of ​​the time-frequency feature in the time-frequency space; The total number of data blocks to be divided is determined based on the feature area. Based on the total amount of data blocks, the time-frequency features are segmented and flattened to obtain multiple data blocks.

8. An audio processing system, characterized in that, include: The flexible front-end module is used to divide the time-frequency features of the audio to be processed into multiple data blocks; the frequency domain resolution of the time-frequency features is matched with the original sampling rate of the audio to be processed. A dual-channel context encoding module is used to encode the original sampling rate and the original position information of each data block in the time-frequency space corresponding to the time-frequency feature, respectively, to obtain the sampling rate embedding feature corresponding to the original sampling rate and the position encoding feature of each data block; The information fusion module is used to obtain a target data block sequence based on the location encoding features, the sampling rate embedding features, and each of the data blocks; The feature representation module is used to input the target data block sequence into the target feature extraction model to obtain the audio features of the audio to be processed in the target feature space; The target feature extraction model is used to map multiple audio samples with different native sampling rates to the target feature space based on shared network parameters.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the audio processing method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the audio processing method as described in any one of claims 1 to 7.

Images