Audio digital signal processing method and device based on SOC chip

Abstract

The invention relates to an audio digital signal processing method and device based on an SOC chip, and the method comprises the following steps: carrying out the audio digital signal collection of a target site through a preset multi-channel sensor, and obtaining a multi-dimensional audio data stream; performing multi-resolution spectrum decomposition on the multi-dimensional audio data stream to obtain hierarchical spectrum features; performing complex field transformation on the hierarchical spectrum features to obtain complex spectrum characterization data; performing frequency domain reconstruction on the complex spectrum representation data through a sub-band energy analysis technology to obtain an acoustic parameter sequence; performing group delay compensation on the acoustic parameter sequence to obtain a calibrated spectrum sequence; and performing inverse Fourier reconstruction on the calibrated spectrum sequence to obtain an enhanced audio digital signal, thereby solving the technical problem of how to efficiently process the data without losing key information along with the increase of the dimension of the audio data.

Description

Technical Field

[0001] The present invention relates to the technical field of SOC chips, and in particular to an audio digital signal processing method and device based on a SOC chip. Background Art

[0002] In the development of modern audio processing technology, the demand for high-quality audio signals is growing, especially in the fields of professional audio recording, speech recognition and communication. Audio digital signal processing methods based on SOC (System on Chip) chips have emerged, aiming to improve processing efficiency and reduce power consumption by integrating multiple processing functions on a single chip. However, traditional audio processing methods often face problems such as insufficient resolution, high computational complexity and inaccurate spectral feature extraction, which limit the further improvement of audio signal processing quality.

[0003] Specifically, in the audio collection stage, how to effectively separate useful sound information from complex environmental noise is a major challenge. Traditional methods rely on a single or a few fixed parameters for spectral analysis, which makes it difficult to capture the subtle differences in audio signals when facing a changing acoustic environment. In addition, as the dimensionality of audio data increases, how to efficiently process this data without losing key information has become an urgent problem to be solved. The existence of these problems not only affects the clarity and accuracy of audio signals, but also restricts the innovation and development of related application fields to a certain extent.

[0004] In order to solve the above problems, researchers are constantly exploring new audio signal processing technologies and algorithms. For example, the combination of hierarchical spectral feature decomposition and complex domain transformation can more accurately represent the characteristics of audio signals and effectively improve the accuracy and efficiency of processing. Despite this, existing technologies still face challenges such as high consumption of computing resources and limited real-time processing capabilities. Therefore, it is particularly important to develop an audio digital signal processing method based on a SOC chip that can meet the needs of high-quality audio processing while achieving low energy consumption and high efficiency. This research direction is of great significance to promoting the development of audio processing technology. Summary of the invention

[0005] The main purpose of the present invention is to provide an audio digital signal processing method and device based on a SOC chip, which solves the technical problem of how to efficiently process these data without losing key information as the dimension of audio data increases.

[0006] To achieve the above object, the present invention provides an audio digital signal processing method based on a SOC chip, comprising the following steps: The audio digital signal is collected at the target scene through a preset multi-channel sensor to obtain a multi-dimensional audio data stream; Performing multi-resolution spectrum decomposition on the multi-dimensional audio data stream to obtain hierarchical spectrum features; Performing complex domain transformation on the hierarchical spectrum features to obtain complex spectrum representation data; Reconstructing the complex spectrum characterization data in the frequency domain by using a sub-band energy analysis technique to obtain an acoustic parameter sequence; Performing group delay compensation on the acoustic parameter sequence to obtain a calibrated frequency spectrum sequence; Based on the preset segmented fast Fourier algorithm in the SOC chip, the calibrated spectrum sequence is subjected to inverse Fourier reconstruction to obtain an enhanced audio digital signal.

[0007] Furthermore, the audio digital signal is collected from the target scene by a preset multi-channel sensor to obtain a multi-dimensional audio data stream, including: The target scene is spatially sampled by a multi-channel sensor in a microphone array to obtain multi-channel sound field data, and sound source localization calculation is performed based on the multi-channel sound field data to obtain sound source spatial distribution data; wherein the sound source spatial distribution data includes a sound source azimuth angle, a sound source distance parameter, and a sound source intensity distribution; Extracting acoustic features from the sound source spatial distribution data to obtain an acoustic feature vector, and reconstructing the sound field based on the acoustic feature vector to obtain a three-dimensional sound field data stream; The three-dimensional sound field data stream is subjected to modal analysis by sound field decomposition to obtain sound field modal coefficients, and sound field synthesis is performed based on the sound field modal coefficients to obtain a multi-dimensional audio data stream; wherein the multi-dimensional audio data stream includes time-varying characteristics of the sound field, spatial characteristics of the sound field, and spectral characteristics of the sound field.

[0008] Furthermore, performing multi-resolution spectrum decomposition on the multi-dimensional audio data stream to obtain hierarchical spectrum features includes: The multi-dimensional audio data stream is divided into frequency bands by wavelet packet decomposition technology to obtain multi-level sub-band coefficients, and energy density is calculated based on the multi-level sub-band coefficients to obtain frequency band energy distribution data; wherein the frequency band energy distribution data includes critical frequency band coefficients and frequency band energy matrices; Performing spectrum envelope extraction on the frequency band energy distribution data to obtain an envelope feature sequence, and performing frequency domain modulation analysis based on the envelope feature sequence to obtain a modulation spectrum matrix; Based on the modulation spectrum matrix, the harmonic structure of the multi-dimensional audio data stream is analyzed to obtain a harmonic component matrix, and the harmonic component matrix is ​​grouped and synthesized to obtain the hierarchical spectrum feature; wherein the hierarchical spectrum feature includes a fundamental frequency trajectory and a harmonic energy ratio.

[0009] Furthermore, the step of performing complex domain transformation on the hierarchical spectrum features to obtain complex spectrum representation data includes: Performing time-frequency domain mapping on the hierarchical spectrum features to obtain a time-frequency characterization vector, and performing phase demodulation on the time-frequency characterization vector to obtain phase modulation data; wherein the phase modulation data includes amplitude envelope data and instantaneous phase data; The phase modulation data is orthogonally decomposed by a preset Hilbert filtering technique to obtain an orthogonal component sequence, and the orthogonal component sequence is polar mapped to obtain a polar coordinate parameter set; wherein the polar coordinate parameter set includes an amplitude radius and a phase angle; Performing phase expansion on the polar coordinate parameter set to obtain a phase expansion coefficient matrix, and performing harmonic reconstruction based on the phase expansion coefficient matrix to obtain a harmonic decomposition matrix; The harmonic decomposition matrix is ​​mapped to the frequency domain by a preset complex exponential transformation technology to obtain a complex exponential sequence, and complex domain synthesis is performed based on the complex exponential sequence to obtain complex frequency domain coefficients; wherein the complex frequency domain coefficients include an amplitude modulation component and a phase modulation component; Performing frequency response analysis on the complex frequency domain coefficients to obtain a frequency response characteristic matrix, and performing phase compensation on the frequency response characteristic matrix to obtain a compensation parameter set; The complex frequency domain coefficients are reconstructed in a complex domain based on the compensation parameter set to obtain the complex frequency spectrum characterization data; wherein the complex frequency spectrum characterization data includes a frequency domain amplitude spectrum and a frequency domain phase spectrum.

[0010] Furthermore, the frequency domain reconstruction of the complex spectrum characterization data by sub-band energy analysis technology to obtain an acoustic parameter sequence includes: Performing energy integration on the complex spectrum characterization data by a preset critical subband analysis technique to obtain a subband energy spectrum, and performing spectrum peak detection based on the subband energy spectrum to obtain a frequency band energy distribution matrix; Calculating the spectral centroid of the frequency band energy distribution matrix to obtain a frequency band centroid sequence, and merging sub-bands based on the frequency band centroid sequence to obtain a reconstructed frequency band matrix; Performing sub-band separation on the reconstructed frequency band matrix by a preset cepstrum analysis method to obtain an independent sub-band sequence, and performing harmonic enhancement on the independent sub-band sequence to obtain enhanced spectrum coefficients; Performing phase reconstruction on the enhanced spectrum coefficients to obtain a complex frequency domain parameter set, and performing frequency band reorganization based on the complex frequency domain parameter set to obtain a spectrum reconstruction matrix; wherein the complex frequency domain parameter set includes a harmonic gain coefficient and a phase offset coefficient; The spectrum reconstruction matrix is ​​feature mapped by a preset acoustic parameter extraction model to obtain an acoustic feature set, and the acoustic feature set is parameter sorted to obtain the acoustic parameter sequence; wherein the acoustic parameter sequence includes an acoustic feature vector and an acoustic parameter matrix.

[0011] Furthermore, performing group delay compensation on the acoustic parameter sequence to obtain a calibrated spectrum sequence includes: Expanding the acoustic parameter sequence in the time domain to obtain a parameter time series matrix, and performing group delay estimation based on the parameter time series matrix to obtain a delay feature vector; wherein the delay feature vector includes a phase delay coefficient and a group delay gradient; Decomposing the delay characteristic vector by a preset full-pole analysis technique to obtain a delay component set, and performing phase unwrapping on the delay component set to obtain a phase compensation matrix; Performing polynomial fitting on the phase compensation matrix to obtain a compensation coefficient sequence, and performing group delay correction on the compensation coefficient sequence to obtain a correction parameter set; Performing frequency band mapping on the correction parameter set by sub-band decomposition to obtain a frequency band correction matrix, and performing delay compensation on the frequency band correction matrix to obtain a compensated spectrum set; Performing harmonic reconstruction on the compensation spectrum set to obtain reconstructed spectrum coefficients, and performing spectrum synthesis based on the reconstructed spectrum coefficients to obtain a synthesized spectrum matrix; The synthetic spectrum matrix is ​​calibrated in the spectrum domain by spectrum shaping technology to obtain a calibration spectrum sequence, and phase alignment is performed based on the calibration spectrum sequence to obtain the calibrated spectrum sequence; wherein the calibrated spectrum sequence includes a calibration amplitude spectrum and a calibration phase spectrum.

[0012] Furthermore, the segmented fast Fourier algorithm in the preset SOC chip is used to perform inverse Fourier reconstruction on the calibrated spectrum sequence to obtain an enhanced audio digital signal, including: Performing complex spectrum separation on the calibrated spectrum sequence to obtain a complex spectrum feature matrix, and performing time domain mapping on the complex spectrum feature matrix to obtain a preliminarily reconstructed time domain signal; Filtering the preliminarily reconstructed time domain signal through a preset time domain filter to obtain a denoised signal matrix, and performing amplitude equalization on the denoised signal matrix to obtain an equalized time domain signal; Performing dynamic range correction on the equalized time domain signal to obtain a dynamic correction signal, and performing peak constraint processing on the dynamic correction signal to obtain a constrained time domain signal; wherein the dynamic correction signal includes a dynamic range coefficient and a signal peak distribution; Performing timing alignment on the constrained time domain signal by using a timing adjustment technology to obtain an aligned signal sequence, and performing short-time energy modulation on the aligned signal sequence to obtain a modulated signal matrix; Performing harmonic enhancement on the modulated signal matrix to obtain an enhanced signal set, and performing spectrum interpolation on the enhanced signal set to obtain an interpolated signal matrix; Through the preset segmented fast Fourier algorithm in the SOC chip, the calibrated spectrum sequence is subjected to inverse Fourier transform based on the interpolation signal matrix to obtain an enhanced audio digital signal.

[0013] The present invention also provides an audio digital signal processing device based on a SOC chip, comprising: The acquisition module is used to acquire audio digital signals from the target scene through a preset multi-channel sensor to obtain a multi-dimensional audio data stream; A decomposition module, used for performing multi-resolution spectrum decomposition on the multi-dimensional audio data stream to obtain hierarchical spectrum features; A transformation module, used for performing complex domain transformation on the hierarchical spectrum features to obtain complex spectrum representation data; A reconstruction module, used for reconstructing the complex spectrum characterization data in the frequency domain by using a sub-band energy analysis technique to obtain an acoustic parameter sequence; A compensation module, used for performing group delay compensation on the acoustic parameter sequence to obtain a calibrated spectrum sequence; The reconstruction module is used to perform inverse Fourier reconstruction on the calibrated spectrum sequence based on a segmented fast Fourier algorithm preset in the SOC chip to obtain an enhanced audio digital signal.

[0014] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0015] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods are implemented.

[0016] The present invention provides an audio digital signal processing method based on a SOC chip, comprising the following steps: collecting audio digital signals at a target site through a preset multi-channel sensor to obtain a multi-dimensional audio data stream; performing multi-resolution spectrum decomposition on the multi-dimensional audio data stream to obtain hierarchical spectrum features; performing complex domain transformation on the hierarchical spectrum features to obtain complex spectrum representation data; reconstructing the complex spectrum representation data in the frequency domain through a sub-band energy analysis technique to obtain an acoustic parameter sequence; performing group delay compensation on the acoustic parameter sequence to obtain a calibrated spectrum sequence; performing reverse Fourier reconstruction on the calibrated spectrum sequence based on a preset segmented fast Fourier algorithm in a SOC chip to obtain an enhanced audio digital signal, solving the technical problem of how to efficiently process these data without losing key information as the dimension of audio data increases, realizing the processing of complex spectrum representation data using sub-band energy analysis technology, and obtaining an acoustic parameter sequence through frequency domain reconstruction. This process can effectively reduce noise interference, while enhancing useful audio signals, and ensure the technical effect of the quality and clarity of the output signal. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the steps of an audio digital signal processing method based on a SOC chip in one embodiment of the present invention; Figure 2 is a structural block diagram of an audio digital signal processing device based on a SOC chip in one embodiment of the present invention; Figure 3 It is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0018] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0020] like Figure 1 As shown, Figure 1 It is a schematic diagram of the steps of an audio digital signal processing method based on a SOC chip in one embodiment of the present invention; An embodiment of the present invention provides an audio digital signal processing method based on a SOC chip, comprising the following steps: Step S1, collecting audio digital signals from a target scene through a preset multi-channel sensor to obtain a multi-dimensional audio data stream.

[0021] Specifically, the process of collecting audio digital signals at the target site through a preset multi-channel sensor to obtain a multi-dimensional audio data stream is the first step of the audio digital signal processing method based on the SOC chip. The core is to use multiple sensors to simultaneously capture sound information from different directions and frequencies. Specifically, the multi-channel sensors are arranged at key positions at the target site, and each sensor is responsible for monitoring the sound environment of a specific area. These sensors can work synchronously to ensure that the data obtained from all angles has a high degree of temporal consistency and spatial resolution. For example, in a professional music recording scene, in order to capture the sound details of each instrument when the band is playing, multiple microphones (i.e., multi-channel sensors) can be placed in different corners of the venue, so that not only the voice of the lead singer can be recorded, but also the sounds of various instruments in the background music, including drums, guitars, bass, etc. When these sensors start working, they will convert the captured sound waves into electrical signals in real time, and further digitize them into audio digital signals. Since each sensor is located in a different position and orientation, the collected audio data also has its own characteristics, forming the so-called "multi-dimensional audio data stream". This multi-dimensional data not only contains information about the intensity of the sound, but also covers rich features such as its propagation direction, time delay, and frequency components. For example, in the above example of music recording, by analyzing the time difference of sound arrival recorded by different microphones, the relative position of each instrument in space can be accurately determined, which is crucial for mixing processing in post-production. In addition, considering the complexity of actual application scenarios, such as the presence of background noise, the design of multi-channel sensors also needs to consider how to effectively filter out unnecessary noise. This usually involves using advanced filtering techniques and algorithms to optimize signal quality. For example, during the recording of an outdoor concert, surrounding traffic noise or other interference sources may affect the recording. Through a carefully designed sensor layout and built-in noise reduction algorithm, the interference of these external factors can be minimized to ensure that the final audio data stream is both rich and pure. In this way, the subsequent multi-resolution spectrum decomposition and other processing steps can more accurately extract useful information, laying a solid foundation for high-quality audio signal processing. In short, this preliminary audio acquisition step is crucial to the entire audio processing process, which directly determines the effectiveness of subsequent processing steps and the quality of the final audio output.

[0022] Step S2, performing multi-resolution spectrum decomposition on the multi-dimensional audio data stream to obtain hierarchical spectrum features.

[0023] Specifically, the multi-resolution spectrum decomposition of the multi-dimensional audio data stream is performed to obtain hierarchical spectrum features. This process is one of the key steps in the audio digital signal processing method based on the SOC chip, which aims to extract spectrum features with different frequency resolutions from the collected multi-dimensional audio data stream. First, the multi-resolution spectrum decomposition technology allows the system to analyze the audio signal at different resolutions in different frequency ranges, thereby capturing richer acoustic information. Specifically, this process is achieved by dividing the audio data stream into multiple sub-bands and applying a spectrum analysis algorithm suitable for its frequency range to each sub-band. For example, in a professional music recording scenario, in order to accurately capture the unique timbre and frequency components of each instrument when the band plays, the system needs to apply high-resolution spectrum analysis methods in the low frequency, medium frequency and high frequency ranges respectively. In this process, the system will first pre-process the multi-dimensional audio data stream, such as noise reduction and smoothing, to ensure the quality of the input data. Then, the audio data stream is decomposed using advanced spectrum analysis tools and techniques, such as short-time Fourier transform (STFT) or wavelet transform. These tools can provide a detailed view on the time-frequency plane, so that different frequency components can be clearly separated. For example, in the example of the music recording above, by carefully analyzing the low-frequency band of the drum sound, its unique low-frequency resonance can be identified; while analyzing the mid- and high-frequency bands of the guitar and bass sounds helps to capture their harmonious overtone structure. This hierarchical spectrum analysis not only improves the frequency resolution, but also provides a more accurate representation of the spectrum features for subsequent processing. In addition, the extraction of hierarchical spectrum features is not just a simple spectrum decomposition, it also includes further analysis and integration of signal characteristics in each frequency band. For example, by comparing the energy distribution and phase relationship between different frequency bands, the system can identify which frequency components are most important for a specific sound feature. This step is crucial to improving the accuracy and efficiency of audio signal processing. In practical applications, considering that there may be complex background noise at a concert, such as cheers from the audience or other environmental noise, multi-resolution spectrum decomposition technology can effectively enhance useful audio signals while suppressing unnecessary noise by distinguishing the frequency bands where the main sound source and the interfering noise are located. Therefore, by performing multi-resolution spectral decomposition on the multidimensional audio data stream, the system can generate spectral features with rich layering. These features lay a solid foundation for subsequent complex domain transformation and other advanced processing steps, and ultimately achieve the reconstruction and optimization of high-quality audio signals.

[0024] Step S3, performing complex domain transformation on the hierarchical spectrum features to obtain complex spectrum representation data.

[0025] Specifically, the process of performing complex domain transformation on the hierarchical spectral features to obtain complex spectral representation data is one of the key steps in the audio digital signal processing method based on the SOC chip, which aims to obtain richer audio information representation by further mathematical transformation of the extracted hierarchical spectral features. Specifically, complex domain transformation is a method of converting data in the real domain to the complex domain, through which the phase and amplitude information of the audio signal can be captured and represented more accurately. In actual operation, the hierarchical spectral features need to be decomposed into their real and imaginary parts first for subsequent complex number operations. For example, in a professional music recording scenario, assuming that we have obtained the spectral features of different musical instrument sounds through multi-resolution spectral decomposition, we need to use complex domain transformation to enhance the expression of these features. In this process, the system will apply algorithms such as discrete Fourier transform (DFT) or its improved version such as fast Fourier transform (FFT) to map the spectral features in each frequency band from the real domain to the complex domain. This transformation not only retains the original amplitude information, but also increases the dimension of phase information, making the representation of the audio signal more comprehensive and accurate. For example, in the above example of music recording, by performing complex domain transformation on the sounds of guitar and bass, not only can their respective frequency components be identified, but also the relative phase relationship between the various frequency components can be accurately captured, which is crucial for subsequent sound quality optimization. Phase information is particularly important when reconstructing high-quality audio signals because it directly affects the naturalness and clarity of the sound. In addition, complex domain transformation also allows more complex mathematical analysis and processing of audio signals. For example, when performing frequency domain reconstruction, complex spectrum characterization data can provide more flexibility and accuracy. In actual application scenarios, considering that there may be complex background noise interference at the concert site, complex domain transformation can effectively separate pure audio signals by analyzing the phase difference between noise and useful audio signals. This step is crucial to improving the accuracy and efficiency of audio signal processing. For example, when processing the sound of drums, due to its large and complex low-frequency components, complex domain transformation can more accurately extract its unique resonance mode while suppressing other irrelevant frequency components. Therefore, by transforming the hierarchical spectral features in the complex domain, the system can generate complex spectrum representation data containing rich phase and amplitude information. These data provide a solid foundation for subsequent advanced processing steps such as sub-band energy analysis, group delay compensation, and inverse Fourier reconstruction, ultimately achieving the enhancement and optimization of high-quality audio signals. In summary, complex domain transformation is not only an important link in the audio signal processing process, but the complex spectrum representation data obtained through this process greatly improves the accuracy and effect of subsequent processing steps, thereby ensuring the quality and clarity of the final output audio signal. Whether in professional music recording or other high-fidelity audio processing application scenarios, this technology has demonstrated its unique advantages and value.

[0026] Step S4: reconstruct the complex spectrum characterization data in the frequency domain by using a sub-band energy analysis technique to obtain an acoustic parameter sequence.

[0027] Specifically, the sub-band energy analysis technology is used to reconstruct the complex spectrum representation data in the frequency domain to obtain the acoustic parameter sequence. This process is one of the important links in the audio digital signal processing method based on the SOC chip, and aims to extract more accurate acoustic features by further analyzing and optimizing the data after complex domain transformation. Specifically, the sub-band energy analysis technology first divides the complex spectrum representation data into different frequency sub-bands, and each sub-band represents a signal component within a specific frequency range. In actual operation, the system will perform detailed energy calculation and analysis on the complex spectrum data in each sub-band to identify the energy distribution of each sub-band. For example, in a professional music recording scene, assuming that we have obtained the complex spectrum representation data of the sounds of instruments such as guitar, bass and drums through complex domain transformation, the sub-band energy analysis technology is then needed to further optimize these data. In this process, the system calculates the energy in each sub-band and reconstructs the frequency domain according to the energy distribution. This reconstruction not only takes into account the energy level of each sub-band, but also combines the phase information, so that the final generated acoustic parameter sequence can more accurately reflect the characteristics of the original audio signal. For example, in the example of the music recording above, by performing sub-band energy analysis on the sounds of guitar and bass, the energy distribution of each of them in different frequency sub-bands can be identified, and the energy ratio of each sub-band can be adjusted accordingly to optimize the sound quality. This step is crucial to improving the clarity and naturalness of the audio signal, because the sounds of different instruments often have unique energy characteristics within a specific frequency range, and these characteristics directly affect the quality and audibility of the sound. In addition, sub-band energy analysis technology can also help remove unnecessary noise interference. In actual application scenarios, there may be complex background noise at the concert site, such as the cheers of the audience or other environmental noise. By analyzing the energy distribution of each sub-band, the system can identify which sub-bands mainly contain useful audio signals and which sub-bands are mainly noise. For example, when processing the sound of drums, due to its large and complex low-frequency components, sub-band energy analysis can more accurately extract its unique resonance mode while suppressing other irrelevant frequency components. In this way, by adjusting the energy ratio of each sub-band and removing noise, the system can generate a high-quality acoustic parameter sequence, which provides a solid foundation for subsequent group delay compensation and inverse Fourier reconstruction. In the specific implementation process, the system will apply advanced algorithms and technologies, such as adaptive filtering and dynamic range compression, to optimize the energy distribution of each sub-band. For example, when processing mixed recordings of multiple instruments, the system can dynamically adjust the energy ratio of each sub-band according to the sound characteristics of different instruments to ensure that the sound of each instrument can be clearly captured and reproduced.Therefore, by performing sub-band energy analysis on the complex spectrum representation data and reconstructing it in the frequency domain, the system can generate acoustic parameter sequences with rich details and high precision. These sequences not only retain the main features of the original audio signal, but also significantly improve the clarity and quality of the audio signal. Whether in professional music recording or other high-fidelity audio processing application scenarios, this technology has demonstrated its unique advantages and value, and ultimately achieved the enhancement and optimization of high-quality audio signals.

[0028] Step S5, performing group delay compensation on the acoustic parameter sequence to obtain a calibrated frequency spectrum sequence.

[0029] Specifically, the process of performing group delay compensation on the acoustic parameter sequence to obtain a calibrated spectrum sequence is one of the key steps in the audio digital signal processing method based on the SOC chip, which aims to ensure the time consistency of the final output audio signal by correcting the time delay difference of different frequency components. Specifically, the group delay compensation technology first identifies the relative time delay between each frequency component in the acoustic parameter sequence, which is usually caused by filters or other nonlinear elements in the signal processing process. In actual operation, the system analyzes the phase information in each frequency subband and calculates the corresponding group delay value based on this information. For example, in a professional music recording scene, assuming that we have obtained the acoustic parameter sequence of instruments such as guitar, bass and drums through subband energy analysis, the next step is to use group delay compensation technology to further optimize these data. In this process, the system performs a detailed phase analysis of the acoustic parameter sequence in each frequency subband to determine the actual arrival time of each frequency component. Since different frequency components may experience different delays during transmission and processing, this will cause distortion of the time structure of the audio signal. In order to restore the time consistency of the original audio signal, the system applies a specific algorithm to calculate and compensate for these group delays. For example, in the above example of music recording, by performing group delay compensation on the sounds of guitar and bass, the time delay of each frequency component can be accurately adjusted so that the sounds of different instruments can reach the listener's ears synchronously, thereby improving the clarity and naturalness of the overall sound quality. This step is crucial to ensure high-fidelity reproduction of audio signals, because any deviation in time will affect the harmony and audibility of the sound. In addition, group delay compensation also involves adjusting the phase relationship across the entire spectrum. In actual application scenarios, there may be complex background noise or reverberation effects at concerts, which may increase the time delay differences between different frequency components. Through precise group delay compensation, the system can effectively eliminate these delay differences and ensure that each frequency component accurately reflects its time position in the original audio signal. For example, when processing the sound of drums, due to the large number and complexity of its low-frequency components, group delay compensation can more accurately adjust the time delay of its various frequency components to keep them synchronized with the sounds of other instruments. In this way, by adjusting the time delay of each frequency component and eliminating phase errors, the system can generate a high-quality calibrated spectrum sequence. In the specific implementation process, the system will apply advanced algorithms and technologies, such as minimum phase filtering and adaptive delay compensation, to optimize the time delay of each frequency sub-band. For example, when processing mixed recordings of multiple instruments, the system can dynamically adjust the time delay of each frequency component according to the sound characteristics of different instruments to ensure that the sound of each instrument can be clearly captured and reproduced.Therefore, by performing group delay compensation on the acoustic parameter sequence and generating a calibrated spectrum sequence based on this, the system can significantly improve the time consistency and overall quality of the audio signal. Whether in professional music recording or other high-fidelity audio processing application scenarios, this technology has demonstrated its unique advantages and value, ultimately achieving the enhancement and optimization of high-quality audio signals. In this way, not only the clarity and naturalness of the audio signal are improved, but also a more accurate data basis is provided for subsequent inverse Fourier reconstruction.

[0030] Step S6, performing inverse Fourier reconstruction on the calibrated spectrum sequence based on a segmented fast Fourier algorithm preset in the SOC chip to obtain an enhanced audio digital signal.

[0031] Specifically, the calibrated spectrum sequence is inversely reconstructed based on the segmented fast Fourier algorithm in the preset SOC chip to obtain an enhanced audio digital signal. This process is the last step in the entire audio processing flow, which aims to reconvert the frequency domain data that has been optimized and adjusted many times into a high-quality time domain signal. Specifically, the system will first perform complex spectrum separation on the calibrated spectrum sequence, generate a complex spectrum feature matrix, and perform time domain mapping on these matrices to obtain a preliminary reconstructed time domain signal. For example, in a professional music recording scenario, assuming that we have obtained the calibrated spectrum sequence of instruments such as guitar, bass, and drums through group delay compensation, then we need to use inverse Fourier reconstruction technology to further optimize these data. In this process, the system will analyze the calibrated spectrum sequence in detail and identify each frequency component and its corresponding phase information. For example, in the example of the music recording above, suppose we divide the audio signal into multiple frequency subbands. By performing complex spectrum separation on the calibrated spectrum sequence of each subband, we can find that the low-frequency band (such as the 1st to 4th subband) mainly contains the sound of drums and bass, while the high-frequency band (such as the 13th to 16th subband) mainly captures the high-pitched part of the guitar. These complex spectrum feature matrices provide an important basis for subsequent time domain mapping. Suppose that in a 5-second clip, the amplitude of a certain frequency component of the guitar in the low frequency band is 0.8, and the amplitude in the high frequency band is 0.9, which indicates that its high-frequency component is more significant. Then, the system performs time domain mapping based on the complex spectrum feature matrix to generate a preliminary reconstructed time domain signal. Time domain mapping can convert frequency domain data back to time domain signals and restore the time waveform of the original audio signal. For example, when processing the sound of the guitar, assuming the fundamental frequency is 440 Hz, its time waveform can be reconstructed through time domain mapping. Assume that in a clip lasting 10 seconds, the fundamental frequency time waveform of the guitar shows an upward trend from 0.5 seconds to 2 seconds, then remains stable from 2 seconds to 8 seconds, and finally gradually decreases from 8 seconds to 10 seconds. These time waveforms provide a reference for subsequent filtering processing. Finally, based on the preset segmented fast Fourier algorithm in the SOC chip, the system performs an inverse Fourier transform on the preliminary reconstructed time domain signal to generate an enhanced audio digital signal. The segmented fast Fourier algorithm can efficiently convert frequency domain data back to time domain signals, ensuring the high quality of the final output audio signal. For example, in the example of the music recording above, assume that the system's segmented fast Fourier algorithm can identify the fundamental frequency trajectory and harmonic energy ratio of the guitar and use it as an enhanced audio digital signal. Suppose in a 15-second clip, the fundamental frequency trajectory of the guitar shows a gradual increase from 440 Hz to 494 Hz. At the same time, its harmonic energy ratio shows that the energy of the second harmonic is about 70% of the fundamental frequency, and the energy of the third harmonic is about 50% of the fundamental frequency. These detailed enhanced audio digital signals not only improve the resolution of the audio signal, but also provide more accurate data support for subsequent complex processing.In this way, the system can more accurately capture the sound characteristics of each instrument, giving listeners a more realistic and immersive listening experience.

[0032] In a specific embodiment, the method of collecting audio digital signals from a target scene through a preset multi-channel sensor to obtain a multi-dimensional audio data stream includes: The target scene is spatially sampled by a multi-channel sensor in a microphone array to obtain multi-channel sound field data, and sound source localization calculation is performed based on the multi-channel sound field data to obtain sound source spatial distribution data; wherein the sound source spatial distribution data includes a sound source azimuth angle, a sound source distance parameter, and a sound source intensity distribution; Extracting acoustic features from the sound source spatial distribution data to obtain acoustic feature vectors, and reconstructing the sound field based on the acoustic feature vectors to obtain a three-dimensional sound field data stream; The three-dimensional sound field data stream is subjected to modal analysis by sound field decomposition to obtain sound field modal coefficients, and sound field synthesis is performed based on the sound field modal coefficients to obtain a multi-dimensional audio data stream; wherein the multi-dimensional audio data stream includes time-varying characteristics of the sound field, spatial characteristics of the sound field, and spectral characteristics of the sound field.

[0033] Specifically, the process of collecting audio digital signals from the target scene through a preset multi-channel sensor to obtain a multi-dimensional audio data stream includes multiple complex but closely connected steps, aiming to capture as much acoustic information as possible from the physical space and convert it into a high-fidelity digital signal. First, the system spatially samples the target scene through the multi-channel sensor in the microphone array. This process not only collects the intensity information of the sound, but also records details such as its propagation direction and time delay. For example, in a professional music recording scene, suppose we use an array composed of multiple microphones to capture the sound of the band playing. These microphones are placed at different locations in the venue, so that the sound of each instrument can be recorded from different angles and distances. Next, based on the collected multi-channel sound field data, the system performs sound source localization calculations to determine the spatial distribution of each sound source. This step involves complex mathematical models and algorithms, such as beamforming technology and time difference of arrival (TDOA) method, which are used to estimate the azimuth angle, distance parameters and sound source intensity distribution of the sound source. For example, in the above example of music recording, by analyzing the sound signals received by different microphones, the specific positions and relative strengths of instruments such as guitar, bass and drums can be accurately identified, which is crucial for post-mixing processing. These sound source spatial distribution data not only provide basic location information of the sound source, but also lay the foundation for subsequent acoustic feature extraction. Subsequently, the system extracts acoustic features from the sound source spatial distribution data to generate acoustic feature vectors. In this process, the system applies a variety of advanced signal processing techniques, such as spectrum analysis and pattern recognition, to extract the most representative features from the original data. For example, when processing the sound of guitar and bass, the system not only pays attention to their frequency components, but also analyzes their phase relationship and energy distribution to generate detailed acoustic feature vectors. Based on these feature vectors, the system further reconstructs the sound field and generates a three-dimensional sound field data stream. This process simulates the actual sound field environment so that the position and characteristics of each sound source can be accurately reproduced. For example, at a concert, the specific position of each instrument in space can be simulated through a three-dimensional sound field data stream, allowing the audience to experience a more realistic and immersive listening experience. Next, the system performs modal analysis on the three-dimensional sound field data stream through sound field decomposition to obtain the sound field modal coefficients. This process uses modal analysis technology to decompose the complex three-dimensional sound field into multiple independent modes, each of which represents a specific vibration mode or frequency component. For example, when processing the sound of a drum, due to its large and complex low-frequency components, modal analysis can more accurately separate each resonance mode and assign corresponding modal coefficients to it. Based on these modal coefficients, the system further performs sound field synthesis to generate a multi-dimensional audio data stream. This data stream not only contains the time-varying characteristics of the sound field, but also covers its spatial characteristics and spectral characteristics, thus providing comprehensive and detailed information for the final audio processing.In the specific implementation process, the system will apply a series of advanced algorithms and technologies, such as adaptive filtering, dynamic range compression and minimum phase filtering, to optimize the effects of each processing step. For example, when processing mixed recordings of multiple instruments, the system can dynamically adjust the proportion of each modal coefficient according to the sound characteristics of different instruments to ensure that the sound of each instrument can be clearly captured and reproduced. Therefore, through a series of complex operations such as spatial sampling of multi-channel sound field data, sound source localization calculation, acoustic feature extraction, sound field reconstruction, modal analysis and sound field synthesis, the system can generate high-quality multi-dimensional audio data streams. Whether in professional music recording or other high-fidelity audio processing application scenarios, this technology has demonstrated its unique advantages and value, and ultimately achieved the enhancement and optimization of high-quality audio signals. In this way, not only the clarity and naturalness of the audio signal are improved, but also a solid data foundation is provided for subsequent multi-resolution spectrum decomposition and other advanced processing steps. The entire process is closely linked, and each step contributes an indispensable force to the final high-quality audio output.

[0034] In a specific embodiment, performing multi-resolution spectrum decomposition on the multi-dimensional audio data stream to obtain hierarchical spectrum features includes: The multi-dimensional audio data stream is divided into frequency bands by wavelet packet decomposition technology to obtain multi-level sub-band coefficients, and energy density is calculated based on the multi-level sub-band coefficients to obtain frequency band energy distribution data; wherein the frequency band energy distribution data includes critical frequency band coefficients and frequency band energy matrices; Performing spectrum envelope extraction on the frequency band energy distribution data to obtain an envelope feature sequence, and performing frequency domain modulation analysis based on the envelope feature sequence to obtain a modulation spectrum matrix; Based on the modulation spectrum matrix, the harmonic structure of the multi-dimensional audio data stream is analyzed to obtain a harmonic component matrix, and the harmonic component matrix is ​​grouped and synthesized to obtain the hierarchical spectrum feature; wherein the hierarchical spectrum feature includes a fundamental frequency trajectory and a harmonic energy ratio.

[0035] Specifically, the multi-resolution spectrum decomposition of the multi-dimensional audio data stream is performed to obtain the hierarchical spectrum features. This process is one of the key steps in the audio digital signal processing method based on the SOC chip, which aims to extract spectrum features with different frequency resolutions from the multi-dimensional audio data stream through a series of complex mathematical transformations and analysis techniques. First, the system divides the multi-dimensional audio data stream into frequency bands through wavelet packet decomposition technology, divides the original audio signal into multiple sub-bands, and calculates the energy density of each sub-band. For example, in a professional music recording scene, suppose that we have collected sound data of instruments such as guitar, bass and drums through a microphone array, and these data contain rich frequency information. During the wavelet packet decomposition process, the system generates multi-level sub-band coefficients, which represent signal components in different frequency bands. In order to further analyze these coefficients, the system calculates their energy density to obtain band energy distribution data. Specifically, the band energy distribution data includes critical band coefficients and band energy matrices. Taking a typical music recording as an example, suppose we divide the audio signal into 16 sub-bands. By calculating the energy density of each sub-band, we can find that the low-frequency band (such as the 1st to 4th sub-bands) mainly contains the sound of drums and bass, while the high-frequency band (such as the 13th to 16th sub-bands) mainly captures the high-pitched part of the guitar. This band energy distribution data provides an important basis for the subsequent spectrum envelope extraction. Next, the system performs spectrum envelope extraction on the band energy distribution data to obtain the envelope feature sequence. This step not only captures the energy change trend of each band, but also lays the foundation for further frequency domain modulation analysis. For example, in the example of the above music recording, by analyzing the envelope feature sequence, we can identify the change pattern of the volume of the guitar during the performance and the intensity fluctuation of the drum at different rhythms. Based on these envelope feature sequences, the system performs frequency domain modulation analysis to obtain the modulation spectrum matrix. The modulation spectrum matrix can reveal the hidden modulation characteristics in the audio signal. For example, in a 5-second music clip, the modulation spectrum matrix may show 3 obvious modulation phenomena per second, which is crucial to understanding the overall dynamic changes of the music. Subsequently, the system performs harmonic structure analysis on the multidimensional audio data stream based on the modulation spectrum matrix to obtain a harmonic component matrix. This process uses harmonic analysis technology to separate the fundamental frequency and its harmonic components from complex audio signals. For example, when processing guitar sounds, assuming that the fundamental frequency is 440 Hz, the harmonic structure analysis can identify its second harmonic (880 Hz), third harmonic (1320 Hz) and other components, and calculate the energy ratio of each harmonic. These harmonic component matrices not only reveal the harmonic structure of the audio signal, but also provide key information for the final hierarchical spectrum feature extraction. Finally, the system groups and synthesizes the harmonic component matrices to obtain hierarchical spectrum features. Hierarchical spectrum features include fundamental frequency trajectory and harmonic energy ratio.For example, in the example of the music recording above, suppose that in a 10-second clip, the fundamental frequency trajectory of the guitar shows a gradual increase from 440 Hz to 494 Hz, and its harmonic energy ratio shows that the energy of the second harmonic is about 70% of the fundamental frequency, and the energy of the third harmonic is about 50% of the fundamental frequency. This detailed spectral feature not only improves the resolution of the audio signal, but also provides a solid foundation for subsequent complex domain transformation and other advanced processing steps. The whole process uses multi-resolution spectral decomposition technology to accurately capture and reproduce every detail of the audio signal. Whether in professional music recording or other high-fidelity audio processing application scenarios, this technology has demonstrated its unique advantages and value, and ultimately achieved the enhancement and optimization of high-quality audio signals. In this way, not only the clarity and naturalness of the audio signal are improved, but also more accurate data support is provided for subsequent complex processing.

[0036] In a specific embodiment, the step of performing complex domain transformation on the hierarchical spectrum features to obtain complex spectrum representation data includes: Performing time-frequency domain mapping on the hierarchical spectrum features to obtain a time-frequency characterization vector, and performing phase demodulation on the time-frequency characterization vector to obtain phase modulation data; wherein the phase modulation data includes amplitude envelope data and instantaneous phase data; The phase modulation data is orthogonally decomposed by a preset Hilbert filtering technique to obtain an orthogonal component sequence, and the orthogonal component sequence is polar mapped to obtain a polar coordinate parameter set; wherein the polar coordinate parameter set includes an amplitude radius and a phase angle; Performing phase expansion on the polar coordinate parameter set to obtain a phase expansion coefficient matrix, and performing harmonic reconstruction based on the phase expansion coefficient matrix to obtain a harmonic decomposition matrix; The harmonic decomposition matrix is ​​mapped to the frequency domain by a preset complex exponential transformation technology to obtain a complex exponential sequence, and complex domain synthesis is performed based on the complex exponential sequence to obtain complex frequency domain coefficients; wherein the complex frequency domain coefficients include an amplitude modulation component and a phase modulation component; Performing frequency response analysis on the complex frequency domain coefficients to obtain a frequency response characteristic matrix, and performing phase compensation on the frequency response characteristic matrix to obtain a compensation parameter set; The complex frequency domain coefficients are reconstructed in a complex domain based on the compensation parameter set to obtain the complex frequency spectrum characterization data; wherein the complex frequency spectrum characterization data includes a frequency domain amplitude spectrum and a frequency domain phase spectrum.

[0037] Specifically, the process of performing complex domain transformation on the hierarchical spectrum features to obtain complex spectrum representation data is one of the key steps in the audio digital signal processing method based on the SOC chip, which aims to extract richer audio information representation from the hierarchical spectrum features through a series of complex mathematical transformations and analysis techniques. First, the system maps the hierarchical spectrum features in the time-frequency domain, generates time-frequency representation vectors, and phase demodulates these vectors to obtain phase modulation data. For example, in a professional music recording scenario, assuming that we have obtained the sound hierarchical spectrum features of instruments such as guitar, bass and drums through multi-resolution spectrum decomposition, then we need to use time-frequency domain mapping technology to convert these features into time-frequency representation vectors. In this process, the system generates phase modulation data containing amplitude envelope data and instantaneous phase data. Specifically, assuming that in a 5-second music clip, the amplitude change of a certain frequency component of the guitar on the time axis is recorded to form amplitude envelope data; at the same time, the instantaneous phase of the frequency component is also accurately captured to form instantaneous phase data. These phase modulation data provide an important basis for subsequent orthogonal decomposition. Subsequently, the system performs orthogonal decomposition of the phase modulation data through the preset Hilbert filtering technology to generate an orthogonal component sequence. The Hilbert filtering technology can effectively separate the real and imaginary parts of the signal, thus preparing for polar coordinate mapping. For example, in the above-mentioned music recording example, by performing Hilbert filtering on a certain frequency component of the guitar, its corresponding orthogonal component sequence can be obtained. Next, the system performs polar coordinate mapping on these orthogonal component sequences to generate a polar coordinate parameter set, including the amplitude radius and phase angle. Assuming that in a 10-second clip, the average amplitude radius of a certain frequency component of the guitar is 0.8, and the average phase angle is 45 degrees, these parameters provide a detailed reference for subsequent phase unwrapping. Next, the system performs phase unwrapping on the polar coordinate parameter set to obtain a phase unwrapping coefficient matrix, and performs harmonic reconstruction based on these coefficients to generate a harmonic decomposition matrix. For example, when processing the sound of a bass, assuming that the fundamental frequency is 55 Hz, its second harmonic (110 Hz), third harmonic (165 Hz) and other components can be identified through phase unwrapping, and the energy ratio of each harmonic can be calculated. These harmonic decomposition matrices not only reveal the harmonic structure of the audio signal, but also provide key information for the final complex exponential transformation. Then, the system maps the harmonic decomposition matrix to the frequency domain through the preset complex exponential transformation technology, generates a complex exponential sequence, and performs complex domain synthesis based on these sequences to obtain complex frequency domain coefficients. The complex frequency domain coefficients include amplitude modulation components and phase modulation components, which lay the foundation for subsequent frequency response analysis. For example, when processing the sound of a drum, assuming that the amplitude modulation component of a certain frequency component is 0.7 and the phase modulation component is 30 degrees, these detailed coefficients can better reflect the true characteristics of the audio signal.Furthermore, the system performs frequency response analysis on the complex frequency domain coefficients, generates frequency response characteristic matrices, and performs phase compensation on these matrices to obtain compensation parameter sets. For example, in a music clip lasting 20 seconds, suppose that after frequency response analysis, a certain frequency component of the guitar is found to need 3 degrees of phase compensation to ensure that its phase relationship with other frequency components is consistent. These compensation parameter sets provide the necessary adjustment basis for the final complex domain reconstruction. Finally, the system performs complex domain reconstruction on the complex frequency domain coefficients based on the compensation parameter set to generate complex spectrum representation data. The complex spectrum representation data includes frequency domain amplitude spectrum and frequency domain phase spectrum, which not only retain the main features of the original audio signal, but also significantly improve the clarity and quality of the audio signal. For example, in the above music recording example, suppose that in a 30-second clip, the frequency domain amplitude spectrum of the guitar shows that there is a significant energy enhancement in the frequency band from 440 Hz to 880 Hz, while the frequency domain phase spectrum shows that the phase relationship between each frequency component is accurately calibrated. The entire process uses complex domain transformation technology to accurately capture and reproduce every detail of the audio signal. Whether in professional music recording or other high-fidelity audio processing application scenarios, this technology has demonstrated its unique advantages and value, and ultimately achieved the enhancement and optimization of high-quality audio signals. In this way, not only the clarity and naturalness of the audio signal are improved, but also more accurate data support is provided for subsequent complex processing.

[0038] In a specific embodiment, the frequency domain reconstruction of the complex spectrum characterization data by using a sub-band energy analysis technique to obtain an acoustic parameter sequence includes: Performing energy integration on the complex spectrum characterization data by a preset critical subband analysis technique to obtain a subband energy spectrum, and performing spectrum peak detection based on the subband energy spectrum to obtain a frequency band energy distribution matrix; Calculating the spectral centroid of the frequency band energy distribution matrix to obtain a frequency band centroid sequence, and merging sub-bands based on the frequency band centroid sequence to obtain a reconstructed frequency band matrix; Performing sub-band separation on the reconstructed frequency band matrix by a preset cepstrum analysis method to obtain an independent sub-band sequence, and performing harmonic enhancement on the independent sub-band sequence to obtain enhanced spectrum coefficients; Performing phase reconstruction on the enhanced spectrum coefficients to obtain a complex frequency domain parameter set, and performing frequency band reorganization based on the complex frequency domain parameter set to obtain a spectrum reconstruction matrix; wherein the complex frequency domain parameter set includes a harmonic gain coefficient and a phase offset coefficient; The spectrum reconstruction matrix is ​​feature mapped by a preset acoustic parameter extraction model to obtain an acoustic feature set, and the acoustic feature set is parameter sorted to obtain the acoustic parameter sequence; wherein the acoustic parameter sequence includes an acoustic feature vector and an acoustic parameter matrix.

[0039] Specifically, the complex spectrum characterization data is reconstructed in the frequency domain by sub-band energy analysis technology to obtain an acoustic parameter sequence. This process is one of the important links in the audio digital signal processing method based on the SOC chip, and aims to generate a high-quality acoustic parameter sequence by performing detailed energy analysis and frequency domain reconstruction on the complex spectrum characterization data. First, the system performs energy integration on the complex spectrum characterization data through a preset critical sub-band analysis technology to obtain a sub-band energy spectrum, and performs spectrum peak detection based on these energy spectra to generate a frequency band energy distribution matrix. For example, in a professional music recording scenario, assuming that we have obtained the complex spectrum characterization data of the sounds of instruments such as guitar, bass and drums through complex domain transformation, then we need to use sub-band energy analysis technology to further optimize these data. In this process, the system will perform detailed energy integration calculations on the complex spectrum characterization data to identify the energy distribution of each frequency sub-band. For example, in the example of the music recording above, suppose we divide the audio signal into 16 sub-bands. By integrating the energy of each sub-band, we can find that the low-frequency band (such as the 1st to 4th sub-bands) mainly contains the sounds of drums and bass, while the high-frequency band (such as the 13th to 16th sub-bands) mainly captures the high-pitched part of the guitar. These sub-band energy spectra provide an important basis for subsequent spectral peak detection. Assume that in a 5-second clip, a certain frequency component of the guitar accounts for 20% of the energy in the low-frequency band and 80% of the energy in the high-frequency band, which indicates that its high-frequency component is more significant. Then, the system performs spectral peak detection based on the sub-band energy spectrum to generate a frequency band energy distribution matrix. Spectral peak detection can help the system identify the main energy peaks in each sub-band, so as to better understand the frequency characteristics of the audio signal. For example, when processing the sound of the guitar, assuming the fundamental frequency is 440 Hz, the energy peaks of its second harmonic (880 Hz), third harmonic (1320 Hz) and other components can be identified through spectral peak detection. Assume that in a 10-second clip, the guitar's fundamental frequency energy peak is 0.7, the second harmonic energy peak is 0.5, and the third harmonic energy peak is 0.3. These energy peaks reflect the relative strength of different frequency components. Subsequently, the system calculates the spectral centroid of the band energy distribution matrix to obtain the band centroid sequence, and merges sub-bands based on these sequences to generate a reconstructed band matrix. The spectral centroid calculation can provide the center frequency position of each sub-band, which is crucial for understanding the overall frequency distribution of the audio signal. For example, in the above music recording example, suppose that the spectral centroid of a certain frequency component of the guitar in the low frequency band is 50 Hz, and the spectral centroid in the high frequency band is 1000 Hz. These spectral centroid values ​​provide a reference for subsequent sub-band merging. Assume that in a 15-second clip, multiple frequency components of the guitar are merged through sub-bands to form a new reconstructed band matrix, which contains the main frequency components from 50 Hz to 1000 Hz.Then, the system uses the preset cepstrum analysis method to perform sub-band separation on the reconstructed frequency band matrix, generate independent sub-band sequences, and perform harmonic enhancement on these sequences to generate enhanced spectrum coefficients. Cepstrum analysis can effectively separate independent frequency components in each sub-band, thus providing a basis for harmonic enhancement. For example, when processing the sound of bass, assuming that the fundamental frequency is 55 Hz, cepstrum analysis can separate its second harmonic (110 Hz), third harmonic (165 Hz) and other components, and perform harmonic enhancement on them. Assume that in a 20-second clip, the fundamental frequency energy of the bass is enhanced by 20%, the second harmonic energy is enhanced by 15%, and the third harmonic energy is enhanced by 10%. These enhanced spectrum coefficients not only improve the quality of the audio signal, but also lay the foundation for subsequent phase reconstruction. Next, the system performs phase reconstruction on the enhanced spectrum coefficients, generates a complex frequency domain parameter set, and performs frequency band reorganization based on these parameter sets to generate a spectrum reconstruction matrix. Phase reconstruction can restore the temporal structure of the audio signal and ensure its continuity and naturalness in the time domain. For example, when processing the sound of drums, assuming that the amplitude modulation component of a certain frequency component is 0.7 and the phase modulation component is 30 degrees, its phase relationship can be accurately adjusted through phase reconstruction to keep it synchronized with other frequency components. Assume that in a 25-second clip, multiple frequency components of the drum are reconstructed to form a new spectrum reconstruction matrix, which contains the main frequency components from 20 Hz to 500 Hz and their corresponding phase information. Further, the system performs feature mapping on the spectrum reconstruction matrix through a preset acoustic parameter extraction model, generates an acoustic feature set, and sorts these sets by parameters to finally generate an acoustic parameter sequence. The acoustic parameter extraction model can extract the most representative acoustic features from the spectrum reconstruction matrix, which are crucial for subsequent audio processing. For example, in the above-mentioned music recording example, it is assumed that the system's acoustic parameter extraction model can identify the fundamental frequency trajectory and harmonic energy ratio of the guitar and use them as acoustic feature vectors. Assume that in a 30-second clip, the fundamental frequency trajectory of the guitar shows a gradual increase from 440 Hz to 494 Hz, and its harmonic energy ratio shows that the energy of the second harmonic is about 70% of the fundamental frequency, and the energy of the third harmonic is about 50% of the fundamental frequency. These detailed acoustic features not only improve the resolution of the audio signal, but also provide more accurate data support for subsequent complex processing. Throughout the process, the system extracts a rich sequence of acoustic parameters from the complex spectrum representation data through a series of complex mathematical transformations and analysis techniques. Whether in professional music recording or other high-fidelity audio processing application scenarios, this technology has demonstrated its unique advantages and value, and ultimately achieved the enhancement and optimization of high-quality audio signals. In this way, not only the clarity and naturalness of the audio signal are improved, but also more accurate data support is provided for subsequent complex processing.For example, at a concert, through these detailed acoustic parameter sequences, the system can more accurately capture the sound characteristics of each instrument, allowing the audience to experience a more realistic and immersive listening experience. In addition, these acoustic parameters can also be used for post-mixing processing to ensure that the sound of each instrument can be clearly captured and reproduced. The entire process uses sub-band energy analysis technology to accurately capture and reproduce every detail of the audio signal, ultimately achieving the goal of enhancing and optimizing high-quality audio signals.

[0040] In a specific embodiment, performing group delay compensation on the acoustic parameter sequence to obtain a calibrated spectrum sequence includes: Expanding the acoustic parameter sequence in the time domain to obtain a parameter time series matrix, and performing group delay estimation based on the parameter time series matrix to obtain a delay feature vector; wherein the delay feature vector includes a phase delay coefficient and a group delay gradient; Decomposing the delay characteristic vector by a preset full-pole analysis technique to obtain a delay component set, and performing phase unwrapping on the delay component set to obtain a phase compensation matrix; Performing polynomial fitting on the phase compensation matrix to obtain a compensation coefficient sequence, and performing group delay correction on the compensation coefficient sequence to obtain a correction parameter set; Performing frequency band mapping on the correction parameter set by sub-band decomposition to obtain a frequency band correction matrix, and performing delay compensation on the frequency band correction matrix to obtain a compensated spectrum set; Performing harmonic reconstruction on the compensation spectrum set to obtain reconstructed spectrum coefficients, and performing spectrum synthesis based on the reconstructed spectrum coefficients to obtain a synthesized spectrum matrix; The synthetic spectrum matrix is ​​calibrated in the spectrum domain by spectrum shaping technology to obtain a calibration spectrum sequence, and phase alignment is performed based on the calibration spectrum sequence to obtain the calibrated spectrum sequence; wherein the calibrated spectrum sequence includes a calibration amplitude spectrum and a calibration phase spectrum.

[0041] Specifically, the process of performing group delay compensation on the acoustic parameter sequence to obtain a calibrated spectrum sequence is one of the key steps in the audio digital signal processing method based on the SOC chip. It aims to eliminate the time delay differences between different frequency components by performing a series of complex mathematical transformations and analysis techniques on the acoustic parameter sequence, thereby ensuring the time consistency of the final output audio signal. First, the system performs time domain expansion on the acoustic parameter sequence to generate a parameter timing matrix, and performs group delay estimation based on these matrices to obtain a delay feature vector. For example, in a professional music recording scenario, assuming that we have obtained the acoustic parameter sequence of instruments such as guitar, bass, and drums through sub-band energy analysis, we need to use group delay compensation technology to further optimize these data. In this process, the system performs detailed time domain expansion calculations on the acoustic parameter sequence to identify the relative time delays between the various frequency components. For example, in the example of the music recording above, suppose we divide the audio signal into multiple frequency subbands. By expanding the acoustic parameter sequence of each subband in the time domain, we can find that the low-frequency band (such as the 1st to 4th subband) mainly contains the sound of drums and bass, while the high-frequency band (such as the 13th to 16th subband) mainly captures the high-pitched part of the guitar. These parameter timing matrices provide an important basis for the subsequent group delay estimation. Assume that in a 5-second clip, the phase delay coefficient of a certain frequency component of the guitar in the low frequency band is 0.2, and the phase delay coefficient in the high frequency band is 0.8, which indicates that there is a significant time delay in its high-frequency component. Then, the system performs group delay estimation based on the parameter timing matrix and generates a delay feature vector. Group delay estimation can help the system identify the actual arrival time of each frequency component, so as to better understand the temporal structure of the audio signal. For example, when processing the sound of the guitar, assuming the fundamental frequency is 440 Hz, the actual arrival time of its second harmonic (880Hz), third harmonic (1320 Hz) and other components can be identified through group delay estimation. Assume that in a 10-second clip, the fundamental frequency time delay of the guitar is 2 milliseconds, the second harmonic time delay is 4 milliseconds, and the third harmonic time delay is 6 milliseconds. These time delays reflect the relative delay relationship between different frequency components. Subsequently, the system uses the preset full-pole analysis technology to perform delay decomposition on the delay characteristic vector, generate a set of delay components, and perform phase unwrapping on these sets to generate a phase compensation matrix. The full-pole analysis technology can effectively separate the delay characteristics of each frequency component, thereby providing a basis for phase unwrapping. For example, when processing the sound of the bass, assuming the fundamental frequency is 55 Hz, the full-pole analysis can separate the delay components of its second harmonic (110 Hz), third harmonic (165 Hz) and other components, and perform phase unwrapping on them.Assume that in a 15-second segment, the fundamental frequency of the bass has a time delay of 1 millisecond, the second harmonic has a time delay of 2 milliseconds, and the third harmonic has a time delay of 3 milliseconds. These delay components provide a reference for the subsequent polynomial fitting. Then, the system performs polynomial fitting on the phase compensation matrix to generate a compensation coefficient sequence, and performs group delay correction on these sequences to generate a correction parameter set. Polynomial fitting can smoothly adjust the time delay of each frequency component to ensure its consistency across the entire spectrum. For example, when processing the sound of a drum, assuming that the amplitude modulation component of a certain frequency component is 0.7 and the phase modulation component is 30 degrees, the polynomial fitting can accurately adjust its phase relationship to keep it synchronized with other frequency components. Assume that in a 20-second segment, multiple frequency components of the drum are fitted by polynomials to form a new correction parameter set, which contains the main frequency components from 20 Hz to 500 Hz and their corresponding phase adjustment values. Next, the system performs band mapping on the correction parameter set through sub-band decomposition to generate a band correction matrix, and performs delay compensation on these matrices to generate a compensation spectrum set. Subband decomposition can divide the frequency components in the entire spectrum into multiple independent subbands, thus providing a basis for delay compensation. For example, in the above music recording example, assuming that the band correction matrix of a certain frequency component of the guitar in the low frequency band is [[0.8, 0.9], [0.7, 0.8]], and the band correction matrix in the high frequency band is [[0.6, 0.7], [0.5, 0.6]], these band correction matrices provide a reference for subsequent harmonic reconstruction. Assume that in a 25-second clip, multiple frequency components of the guitar are delayed compensated to form a new compensation spectrum set, which contains the main frequency components from 50 Hz to 1000 Hz and their corresponding delay compensation values. Furthermore, the system reconstructs the compensation spectrum set harmonically, generates reconstructed spectrum coefficients, and performs spectrum synthesis based on these coefficients to generate a synthetic spectrum matrix. Harmonic reconstruction can restore the original frequency structure of the audio signal and ensure its continuity and naturalness in the frequency domain. For example, when processing the sound of a bass, assuming the fundamental frequency is 55 Hz, its second harmonic (110 Hz), third harmonic (165 Hz) and other components can be separated and recombined through harmonic reconstruction. Assuming that in a 30-second clip, the fundamental frequency energy of the bass is enhanced by 20%, the second harmonic energy is enhanced by 15%, and the third harmonic energy is enhanced by 10%. These enhanced reconstructed spectral coefficients not only improve the quality of the audio signal, but also lay the foundation for subsequent spectrum shaping. Finally, the system uses spectrum shaping technology to calibrate the synthetic spectrum matrix in the spectral domain, generate a calibration spectrum sequence, and perform phase alignment based on these sequences to generate a calibrated spectrum sequence. Spectral shaping technology can smoothly adjust the amplitude and phase of each frequency component to ensure its consistency across the entire spectrum.For example, in the example of the music recording above, assume that the system's spectrum shaping technology can identify the fundamental frequency trajectory and harmonic energy ratio of the guitar and use it as a calibration spectrum sequence. Assume that in a 35-second clip, the fundamental frequency trajectory of the guitar shows a gradual increase from 440 Hz to 494 Hz, and its harmonic energy ratio shows that the energy of the second harmonic is about 70% of the fundamental frequency, and the energy of the third harmonic is about 50% of the fundamental frequency. These detailed calibration spectrum sequences not only improve the resolution of the audio signal, but also provide more accurate data support for subsequent complex processing. Throughout the process, the system extracts high-quality calibrated spectrum sequences from the acoustic parameter sequence through a series of complex mathematical transformations and analysis techniques. Whether in professional music recording or other high-fidelity audio processing application scenarios, this technology has demonstrated its unique advantages and value, and ultimately achieved the enhancement and optimization of high-quality audio signals. In this way, not only the clarity and naturalness of the audio signal are improved, but also more accurate data support is provided for subsequent complex processing. For example, at a concert, through these detailed calibrated spectrum sequences, the system can more accurately capture the sound characteristics of each instrument, allowing the audience to experience a more realistic and immersive listening experience. In addition, these calibrated spectra can also be used for post-mixing processing to ensure that the sound of each instrument can be clearly captured and reproduced. The entire process uses group delay compensation technology to accurately capture and reproduce every detail of the audio signal, ultimately achieving the goal of enhancing and optimizing high-quality audio signals.

[0042] In a specific embodiment, the segmented fast Fourier algorithm in the preset SOC chip is used to perform inverse Fourier reconstruction on the calibrated spectrum sequence to obtain an enhanced audio digital signal, including: Performing complex spectrum separation on the calibrated spectrum sequence to obtain a complex spectrum feature matrix, and performing time domain mapping on the complex spectrum feature matrix to obtain a preliminarily reconstructed time domain signal; Filtering the preliminarily reconstructed time domain signal through a preset time domain filter to obtain a denoised signal matrix, and performing amplitude equalization on the denoised signal matrix to obtain an equalized time domain signal; Performing dynamic range correction on the equalized time domain signal to obtain a dynamic correction signal, and performing peak constraint processing on the dynamic correction signal to obtain a constrained time domain signal; wherein the dynamic correction signal includes a dynamic range coefficient and a signal peak distribution; Performing timing alignment on the constrained time domain signal by using a timing adjustment technology to obtain an aligned signal sequence, and performing short-time energy modulation on the aligned signal sequence to obtain a modulated signal matrix; Performing harmonic enhancement on the modulated signal matrix to obtain an enhanced signal set, and performing spectrum interpolation on the enhanced signal set to obtain an interpolated signal matrix; Through the preset segmented fast Fourier algorithm in the SOC chip, the calibrated spectrum sequence is subjected to inverse Fourier transform based on the interpolation signal matrix to obtain an enhanced audio digital signal.

[0043] Specifically, the calibrated spectrum sequence is inversely reconstructed based on the preset segmented fast Fourier algorithm in the SOC chip to obtain an enhanced audio digital signal. This process is the last step in the audio digital signal processing method based on the SOC chip, which aims to convert the frequency domain data that has been optimized and adjusted many times into high-quality time domain signals through a series of complex mathematical transformations and analysis techniques. First, the system performs complex spectrum separation on the calibrated spectrum sequence, generates a complex spectrum feature matrix, and performs time domain mapping on these matrices to obtain a preliminary reconstructed time domain signal. For example, in a professional music recording scenario, assuming that we have obtained the calibrated spectrum sequence of instruments such as guitar, bass and drums through group delay compensation, we need to use the inverse Fourier reconstruction technology to further optimize these data. In this process, the system will perform detailed complex spectrum separation calculations on the calibrated spectrum sequence to identify each frequency component and its corresponding phase information. For example, in the example of the music recording above, suppose we divide the audio signal into multiple frequency subbands. By performing complex spectrum separation on the calibrated spectrum sequence of each subband, we can find that the low-frequency band (such as the 1st to 4th subband) mainly contains the sound of drums and bass, while the high-frequency band (such as the 13th to 16th subband) mainly captures the high-pitched part of the guitar. These complex spectrum feature matrices provide an important basis for subsequent time domain mapping. Suppose that in a 5-second clip, the amplitude of a certain frequency component of the guitar in the low frequency band is 0.8, and the amplitude in the high frequency band is 0.9, which indicates that its high-frequency component is more significant. Then, the system performs time domain mapping based on the complex spectrum feature matrix to generate a preliminary reconstructed time domain signal. Time domain mapping can convert frequency domain data back to time domain signals and restore the time waveform of the original audio signal. For example, when processing the sound of the guitar, assuming the fundamental frequency is 440 Hz, its time waveform can be reconstructed through time domain mapping. Assume that in a 10-second segment, the fundamental frequency time waveform of the guitar shows an upward trend from 0.5 seconds to 2 seconds, then remains stable from 2 seconds to 8 seconds, and finally gradually decreases from 8 seconds to 10 seconds. These time waveforms provide a reference for subsequent filtering processing. Subsequently, the system filters the preliminarily reconstructed time domain signal through a preset time domain filter to generate a denoised signal matrix, and performs amplitude equalization on these matrices to generate an equalized time domain signal. The time domain filter can effectively remove background noise and other interfering signals to ensure the clarity of the audio signal. For example, when processing the sound of the bass, assuming that the fundamental frequency is 55Hz, the environmental noise can be removed through time domain filtering and the main frequency components can be retained. Assume that in a 15-second segment, the fundamental frequency energy of the bass is enhanced by 20%, the second harmonic energy is enhanced by 15%, and the third harmonic energy is enhanced by 10%. These enhanced denoised signals not only improve the quality of the audio signal, but also lay the foundation for subsequent amplitude equalization.Assume that in a 20-second clip, multiple frequency components of the bass are equalized by amplitude to form a new equalized time domain signal, which contains the main frequency components from 50 Hz to 1000 Hz and their corresponding amplitude adjustment values. Then, the system performs dynamic range correction on the equalized time domain signal to generate a dynamic correction signal, and performs peak constraint processing on these signals to generate a constrained time domain signal. Dynamic range correction can smoothly adjust the dynamic range of the audio signal to ensure its consistency across the entire time axis. For example, when processing the sound of a drum, assuming that the amplitude modulation component of a certain frequency component is 0.7 and the phase modulation component is 30 degrees, the dynamic range correction can accurately adjust its dynamic range to keep it synchronized with other frequency components. Assume that in a 25-second clip, multiple frequency components of the drum are corrected by dynamic range to form a new dynamic correction signal, which contains the main frequency components from 20 Hz to 500 Hz and their corresponding dynamic range coefficients. Assume that in this clip, the average value of the dynamic range coefficient is 0.8, and the signal peak distribution ranges from 0.6 to 0.9. These adjusted signals provide a reference for subsequent peak constraint processing. Next, the system uses the timing adjustment technology to perform time alignment on the constrained time domain signal to generate an alignment signal sequence, and performs short-time energy modulation on these sequences to generate a modulation signal matrix. The timing adjustment technology can ensure that the relative time relationship between each frequency component remains consistent, thereby improving the time consistency of the audio signal. For example, in the above-mentioned music recording example, suppose that the alignment signal sequence of a certain frequency component of the guitar in the low frequency band is [[0.8, 0.9], [0.7, 0.8]], and the alignment signal sequence in the high frequency band is [[0.6, 0.7], [0.5,0.6]], these alignment signal sequences provide a reference for subsequent short-time energy modulation. Assume that in a 30-second clip, multiple frequency components of the guitar are short-time energy modulated to form a new modulation signal matrix, which contains the main frequency components from 50 Hz to 1000 Hz and their corresponding energy adjustment values. Further, the system performs harmonic enhancement on the modulation signal matrix to generate an enhanced signal set, and performs spectral interpolation on these sets to generate an interpolated signal matrix. Harmonic enhancement can restore the original frequency structure of the audio signal and ensure its continuity and naturalness in the frequency domain. For example, when processing the sound of a bass, assuming the fundamental frequency is 55 Hz, harmonic enhancement can separate its second harmonic (110 Hz), third harmonic (165 Hz) and other components and reassemble them. Assuming that in a 35-second clip, the fundamental frequency energy of the bass is enhanced by 20%, the second harmonic energy is enhanced by 15%, and the third harmonic energy is enhanced by 10%. These enhanced signal sets not only improve the quality of the audio signal, but also lay the foundation for subsequent spectrum interpolation.Assume that in a 40-second clip, multiple frequency components of the bass are spectrally interpolated to form a new interpolation signal matrix, which contains the main frequency components from 50 Hz to 1000 Hz and their corresponding interpolation adjustment values. Finally, based on the preset segmented fast Fourier algorithm in the SOC chip, the system performs an inverse Fourier transform on the calibrated spectrum sequence based on the interpolation signal matrix to generate an enhanced audio digital signal. The segmented fast Fourier algorithm can efficiently convert frequency domain data back to time domain signals to ensure the high quality of the final output audio signal. For example, in the example of the music recording above, assume that the system's segmented fast Fourier algorithm can identify the fundamental frequency trajectory and harmonic energy ratio of the guitar and use it as an enhanced audio digital signal. Assume that in a 45-second clip, the fundamental frequency trajectory of the guitar shows a gradual increase from 440 Hz to 494 Hz, and its harmonic energy ratio shows that the energy of the second harmonic is about 70% of the fundamental frequency, and the energy of the third harmonic is about 50% of the fundamental frequency. These detailed enhanced audio digital signals not only improve the resolution of the audio signal, but also provide more accurate data support for subsequent complex processing. Throughout the process, the system extracts high-quality enhanced audio digital signals from the calibrated spectrum sequence through a series of complex mathematical transformations and analysis techniques. Whether in professional music recording or other high-fidelity audio processing application scenarios, this technology has demonstrated its unique advantages and value, and ultimately achieved the enhancement and optimization of high-quality audio signals. In this way, not only the clarity and naturalness of the audio signal are improved, but also more accurate data support is provided for subsequent complex processing. For example, at a concert, through these detailed enhanced audio digital signals, the system can more accurately capture the sound characteristics of each instrument, allowing the audience to experience a more realistic and immersive listening experience. In addition, these enhanced audio digital signals can also be used for post-mixing processing to ensure that the sound of each instrument can be clearly captured and reproduced. The entire process uses inverse Fourier reconstruction technology, so that every detail of the audio signal can be accurately captured and reproduced, ultimately achieving the goal of enhancing and optimizing high-quality audio signals.

[0044] The above describes the audio digital signal processing method based on the SOC chip in the embodiment of the present invention. The following describes the audio digital signal processing device based on the SOC chip in the embodiment of the present invention. Figure 2 In one embodiment of the present invention, an audio digital signal processing device based on a SOC chip includes: The acquisition module 21 is used to acquire audio digital signals from the target scene through a preset multi-channel sensor to obtain a multi-dimensional audio data stream; A decomposition module 22, configured to perform multi-resolution spectrum decomposition on the multi-dimensional audio data stream to obtain hierarchical spectrum features; A transformation module 23, used for performing complex domain transformation on the hierarchical spectrum features to obtain complex spectrum representation data; A reconstruction module 24 is used to reconstruct the complex spectrum characterization data in the frequency domain by using a sub-band energy analysis technique to obtain an acoustic parameter sequence; A compensation module 25, configured to perform group delay compensation on the acoustic parameter sequence to obtain a calibrated spectrum sequence; The reconstruction module 26 is used to perform inverse Fourier reconstruction on the calibrated spectrum sequence based on a segmented fast Fourier algorithm preset in the SOC chip to obtain an enhanced audio digital signal.

[0045] In this embodiment, for the specific implementation of each unit in the above device embodiment, please refer to the above method embodiment, which will not be repeated here.

[0046] Reference Figure 3 The present invention also provides a computer device in an embodiment, wherein the internal structure of the computer device can be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0047] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0048] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0049] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.

[0050] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0051] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. An audio digital signal processing method based on a SOC chip, characterized in that: The following steps are involved: The audio digital signal is collected at the target scene through a preset multi-channel sensor to obtain a multi-dimensional audio data stream; Performing multi-resolution spectrum decomposition on the multi-dimensional audio data stream to obtain hierarchical spectrum features; Performing complex domain transformation on the hierarchical spectrum features to obtain complex spectrum representation data; Reconstructing the complex spectrum characterization data in the frequency domain by using a sub-band energy analysis technique to obtain an acoustic parameter sequence; Performing group delay compensation on the acoustic parameter sequence to obtain a calibrated frequency spectrum sequence; Based on the preset segmented fast Fourier algorithm in the SOC chip, the calibrated spectrum sequence is subjected to inverse Fourier reconstruction to obtain an enhanced audio digital signal.

2. The audio digital signal processing method based on SOC chip according to claim 1, characterized in that: The method collects audio digital signals from the target scene through a preset multi-channel sensor to obtain a multi-dimensional audio data stream, including: The target scene is spatially sampled by a multi-channel sensor in a microphone array to obtain multi-channel sound field data, and sound source localization calculation is performed based on the multi-channel sound field data to obtain sound source spatial distribution data; wherein the sound source spatial distribution data includes a sound source azimuth angle, a sound source distance parameter, and a sound source intensity distribution; Extracting acoustic features from the sound source spatial distribution data to obtain an acoustic feature vector, and reconstructing the sound field based on the acoustic feature vector to obtain a three-dimensional sound field data stream; The three-dimensional sound field data stream is subjected to modal analysis by sound field decomposition to obtain sound field modal coefficients, and sound field synthesis is performed based on the sound field modal coefficients to obtain a multi-dimensional audio data stream; wherein the multi-dimensional audio data stream includes time-varying characteristics of the sound field, spatial characteristics of the sound field, and spectral characteristics of the sound field.

3. The audio digital signal processing method based on SOC chip according to claim 1, characterized in that: The performing multi-resolution spectrum decomposition on the multi-dimensional audio data stream to obtain hierarchical spectrum features includes: The multi-dimensional audio data stream is divided into frequency bands by wavelet packet decomposition technology to obtain multi-level sub-band coefficients, and energy density is calculated based on the multi-level sub-band coefficients to obtain frequency band energy distribution data; wherein the frequency band energy distribution data includes critical frequency band coefficients and frequency band energy matrices; Performing spectrum envelope extraction on the frequency band energy distribution data to obtain an envelope feature sequence, and performing frequency domain modulation analysis based on the envelope feature sequence to obtain a modulation spectrum matrix; Based on the modulation spectrum matrix, the harmonic structure of the multi-dimensional audio data stream is analyzed to obtain a harmonic component matrix, and the harmonic component matrix is ​​grouped and synthesized to obtain the hierarchical spectrum feature; wherein the hierarchical spectrum feature includes a fundamental frequency trajectory and a harmonic energy ratio.

4. The audio digital signal processing method based on SOC chip according to claim 1, characterized in that: The performing complex domain transformation on the hierarchical spectrum features to obtain complex spectrum representation data includes: Performing time-frequency domain mapping on the hierarchical spectrum features to obtain a time-frequency characterization vector, and performing phase demodulation on the time-frequency characterization vector to obtain phase modulation data; wherein the phase modulation data includes amplitude envelope data and instantaneous phase data; The phase modulation data is orthogonally decomposed by a preset Hilbert filtering technique to obtain an orthogonal component sequence, and the orthogonal component sequence is polar mapped to obtain a polar coordinate parameter set; wherein the polar coordinate parameter set includes an amplitude radius and a phase angle; Performing phase expansion on the polar coordinate parameter set to obtain a phase expansion coefficient matrix, and performing harmonic reconstruction based on the phase expansion coefficient matrix to obtain a harmonic decomposition matrix; The harmonic decomposition matrix is ​​mapped to the frequency domain by a preset complex exponential transformation technology to obtain a complex exponential sequence, and complex domain synthesis is performed based on the complex exponential sequence to obtain complex frequency domain coefficients; wherein the complex frequency domain coefficients include an amplitude modulation component and a phase modulation component; Performing frequency response analysis on the complex frequency domain coefficients to obtain a frequency response characteristic matrix, and performing phase compensation on the frequency response characteristic matrix to obtain a compensation parameter set; The complex frequency domain coefficients are reconstructed in a complex domain based on the compensation parameter set to obtain the complex frequency spectrum characterization data; wherein the complex frequency spectrum characterization data includes a frequency domain amplitude spectrum and a frequency domain phase spectrum.

5. The audio digital signal processing method based on SOC chip according to claim 1, characterized in that: The method of reconstructing the complex spectrum characterization data in the frequency domain by using the sub-band energy analysis technology to obtain an acoustic parameter sequence includes: Performing energy integration on the complex spectrum characterization data by a preset critical subband analysis technique to obtain a subband energy spectrum, and performing spectrum peak detection based on the subband energy spectrum to obtain a frequency band energy distribution matrix; Calculating the spectral centroid of the frequency band energy distribution matrix to obtain a frequency band centroid sequence, and merging sub-bands based on the frequency band centroid sequence to obtain a reconstructed frequency band matrix; Performing sub-band separation on the reconstructed frequency band matrix by a preset cepstrum analysis method to obtain an independent sub-band sequence, and performing harmonic enhancement on the independent sub-band sequence to obtain enhanced spectrum coefficients; Performing phase reconstruction on the enhanced spectrum coefficients to obtain a complex frequency domain parameter set, and performing frequency band reorganization based on the complex frequency domain parameter set to obtain a spectrum reconstruction matrix; wherein the complex frequency domain parameter set includes a harmonic gain coefficient and a phase offset coefficient; The spectrum reconstruction matrix is ​​feature mapped by a preset acoustic parameter extraction model to obtain an acoustic feature set, and the acoustic feature set is parameter sorted to obtain the acoustic parameter sequence; wherein the acoustic parameter sequence includes an acoustic feature vector and an acoustic parameter matrix.

6. The audio digital signal processing method based on SOC chip according to claim 1, characterized in that: The performing group delay compensation on the acoustic parameter sequence to obtain a calibrated spectrum sequence includes: Expanding the acoustic parameter sequence in the time domain to obtain a parameter time series matrix, and performing group delay estimation based on the parameter time series matrix to obtain a delay feature vector; wherein the delay feature vector includes a phase delay coefficient and a group delay gradient; Decomposing the delay characteristic vector by a preset full-pole analysis technique to obtain a delay component set, and performing phase unwrapping on the delay component set to obtain a phase compensation matrix; Performing polynomial fitting on the phase compensation matrix to obtain a compensation coefficient sequence, and performing group delay correction on the compensation coefficient sequence to obtain a correction parameter set; Performing frequency band mapping on the correction parameter set by sub-band decomposition to obtain a frequency band correction matrix, and performing delay compensation on the frequency band correction matrix to obtain a compensated spectrum set; Performing harmonic reconstruction on the compensation spectrum set to obtain reconstructed spectrum coefficients, and performing spectrum synthesis based on the reconstructed spectrum coefficients to obtain a synthesized spectrum matrix; The synthetic spectrum matrix is ​​calibrated in the spectrum domain by spectrum shaping technology to obtain a calibration spectrum sequence, and phase alignment is performed based on the calibration spectrum sequence to obtain the calibrated spectrum sequence; wherein the calibrated spectrum sequence includes a calibration amplitude spectrum and a calibration phase spectrum.

7. The audio digital signal processing method based on SOC chip according to claim 1, characterized in that: The step of performing reverse Fourier reconstruction on the calibrated spectrum sequence based on the segmented fast Fourier algorithm in the preset SOC chip to obtain an enhanced audio digital signal includes: Performing complex spectrum separation on the calibrated spectrum sequence to obtain a complex spectrum feature matrix, and performing time domain mapping on the complex spectrum feature matrix to obtain a preliminarily reconstructed time domain signal; Filtering the preliminarily reconstructed time domain signal through a preset time domain filter to obtain a denoised signal matrix, and performing amplitude equalization on the denoised signal matrix to obtain an equalized time domain signal; Performing dynamic range correction on the equalized time domain signal to obtain a dynamic correction signal, and performing peak constraint processing on the dynamic correction signal to obtain a constrained time domain signal; wherein the dynamic correction signal includes a dynamic range coefficient and a signal peak distribution; Performing timing alignment on the constrained time domain signal by using a timing adjustment technology to obtain an aligned signal sequence, and performing short-time energy modulation on the aligned signal sequence to obtain a modulated signal matrix; Performing harmonic enhancement on the modulated signal matrix to obtain an enhanced signal set, and performing spectrum interpolation on the enhanced signal set to obtain an interpolated signal matrix; Through the preset segmented fast Fourier algorithm in the SOC chip, the calibrated spectrum sequence is subjected to inverse Fourier transform based on the interpolation signal matrix to obtain an enhanced audio digital signal.

8. An audio digital signal processing device based on a SOC chip, characterized in that: include: The acquisition module is used to acquire audio digital signals from the target scene through a preset multi-channel sensor to obtain a multi-dimensional audio data stream; A decomposition module, used for performing multi-resolution spectrum decomposition on the multi-dimensional audio data stream to obtain hierarchical spectrum features; A transformation module, used for performing complex domain transformation on the hierarchical spectrum features to obtain complex spectrum representation data; A reconstruction module, used for reconstructing the complex spectrum characterization data in the frequency domain by using a sub-band energy analysis technique to obtain an acoustic parameter sequence; A compensation module, used for performing group delay compensation on the acoustic parameter sequence to obtain a calibrated spectrum sequence; The reconstruction module is used to perform inverse Fourier reconstruction on the calibrated spectrum sequence based on a segmented fast Fourier algorithm preset in the SOC chip to obtain an enhanced audio digital signal.

9. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images